universal-dev-standards 5.3.2 → 5.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/ai/standards/adversarial-test.ai.yaml +277 -0
- package/bundled/ai/standards/agent-communication-protocol.ai.yaml +32 -166
- package/bundled/ai/standards/agent-dispatch.ai.yaml +32 -58
- package/bundled/ai/standards/audit-trail.ai.yaml +113 -0
- package/bundled/ai/standards/branch-completion.ai.yaml +34 -70
- package/bundled/ai/standards/change-batching-standards.ai.yaml +31 -180
- package/bundled/ai/standards/chaos-injection-tests.ai.yaml +91 -0
- package/bundled/ai/standards/container-image-standards.ai.yaml +88 -0
- package/bundled/ai/standards/container-security.ai.yaml +331 -0
- package/bundled/ai/standards/cost-budget-test.ai.yaml +96 -0
- package/bundled/ai/standards/data-contract.ai.yaml +110 -0
- package/bundled/ai/standards/data-migration-testing.ai.yaml +96 -0
- package/bundled/ai/standards/data-pipeline.ai.yaml +113 -0
- package/bundled/ai/standards/disaster-recovery-drill.ai.yaml +89 -0
- package/bundled/ai/standards/execution-history.ai.yaml +30 -288
- package/bundled/ai/standards/flaky-test-management.ai.yaml +89 -0
- package/bundled/ai/standards/flow-based-testing.ai.yaml +240 -0
- package/bundled/ai/standards/iac-design-principles.ai.yaml +83 -0
- package/bundled/ai/standards/incident-response.ai.yaml +107 -0
- package/bundled/ai/standards/license-compliance.ai.yaml +106 -0
- package/bundled/ai/standards/llm-output-validation.ai.yaml +269 -0
- package/bundled/ai/standards/mock-boundary.ai.yaml +250 -0
- package/bundled/ai/standards/mutation-testing.ai.yaml +192 -0
- package/bundled/ai/standards/pii-classification.ai.yaml +109 -0
- package/bundled/ai/standards/pipeline-integration-standards.ai.yaml +28 -169
- package/bundled/ai/standards/policy-as-code-testing.ai.yaml +227 -0
- package/bundled/ai/standards/prd-standards.ai.yaml +88 -0
- package/bundled/ai/standards/product-metrics-standards.ai.yaml +111 -0
- package/bundled/ai/standards/prompt-regression.ai.yaml +94 -0
- package/bundled/ai/standards/property-based-testing.ai.yaml +105 -0
- package/bundled/ai/standards/release-quality-manifest.ai.yaml +135 -0
- package/bundled/ai/standards/replay-test.ai.yaml +111 -0
- package/bundled/ai/standards/runbook.ai.yaml +104 -0
- package/bundled/ai/standards/sast-advanced.ai.yaml +135 -0
- package/bundled/ai/standards/schema-evolution.ai.yaml +111 -0
- package/bundled/ai/standards/secret-management-standards.ai.yaml +105 -0
- package/bundled/ai/standards/secure-op.ai.yaml +365 -0
- package/bundled/ai/standards/security-testing.ai.yaml +171 -0
- package/bundled/ai/standards/server-ops-security.ai.yaml +274 -0
- package/bundled/ai/standards/slo-sli.ai.yaml +97 -0
- package/bundled/ai/standards/smoke-test.ai.yaml +87 -0
- package/bundled/ai/standards/supply-chain-attestation.ai.yaml +109 -0
- package/bundled/ai/standards/test-completeness-dimensions.ai.yaml +52 -5
- package/bundled/ai/standards/user-story-mapping.ai.yaml +108 -0
- package/bundled/ai/standards/workflow-enforcement.ai.yaml +34 -240
- package/bundled/ai/standards/workflow-state-protocol.ai.yaml +31 -107
- package/bundled/core/adversarial-test.md +212 -0
- package/bundled/core/chaos-injection-tests.md +116 -0
- package/bundled/core/container-security.md +521 -0
- package/bundled/core/cost-budget-test.md +69 -0
- package/bundled/core/data-migration-testing.md +110 -0
- package/bundled/core/disaster-recovery-drill.md +73 -0
- package/bundled/core/flaky-test-management.md +73 -0
- package/bundled/core/flow-based-testing.md +142 -0
- package/bundled/core/llm-output-validation.md +178 -0
- package/bundled/core/mock-boundary.md +100 -0
- package/bundled/core/mutation-testing.md +97 -0
- package/bundled/core/policy-as-code-testing.md +188 -0
- package/bundled/core/prompt-regression.md +72 -0
- package/bundled/core/property-based-testing.md +73 -0
- package/bundled/core/release-quality-manifest.md +147 -0
- package/bundled/core/replay-test.md +86 -0
- package/bundled/core/sast-advanced.md +300 -0
- package/bundled/core/secure-op.md +314 -0
- package/bundled/core/security-testing.md +87 -0
- package/bundled/core/server-ops-security.md +493 -0
- package/bundled/core/smoke-test.md +65 -0
- package/bundled/core/supply-chain-attestation.md +117 -0
- package/bundled/locales/zh-CN/CHANGELOG.md +3 -3
- package/bundled/locales/zh-CN/README.md +1 -1
- package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +5 -5
- package/bundled/locales/zh-TW/CHANGELOG.md +3 -3
- package/bundled/locales/zh-TW/README.md +1 -1
- package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +183 -79
- package/bundled/skills/README.md +4 -3
- package/bundled/skills/SKILL_NAMING.md +94 -0
- package/bundled/skills/ai-instruction-standards/SKILL.md +181 -88
- package/bundled/skills/atdd-assistant/SKILL.md +8 -0
- package/bundled/skills/bdd-assistant/SKILL.md +7 -0
- package/bundled/skills/checkin-assistant/SKILL.md +8 -0
- package/bundled/skills/code-review-assistant/SKILL.md +7 -0
- package/bundled/skills/journey-test-assistant/SKILL.md +203 -0
- package/bundled/skills/orchestrate/SKILL.md +167 -0
- package/bundled/skills/plan/SKILL.md +234 -0
- package/bundled/skills/pr-automation-assistant/SKILL.md +8 -0
- package/bundled/skills/push/SKILL.md +49 -2
- package/bundled/skills/{process-automation → skill-builder}/SKILL.md +1 -1
- package/bundled/skills/{forward-derivation → spec-derivation}/SKILL.md +1 -1
- package/bundled/skills/spec-driven-dev/SKILL.md +7 -0
- package/bundled/skills/sweep/SKILL.md +145 -0
- package/bundled/skills/tdd-assistant/SKILL.md +7 -0
- package/package.json +1 -1
- package/src/commands/flow.js +8 -0
- package/src/commands/start.js +14 -0
- package/src/commands/sweep.js +8 -0
- package/src/commands/workflow.js +8 -0
- package/standards-registry.json +474 -12
- package/bundled/locales/zh-CN/skills/ac-coverage-assistant/SKILL.md +0 -190
- package/bundled/locales/zh-CN/skills/forward-derivation/SKILL.md +0 -71
- package/bundled/locales/zh-CN/skills/forward-derivation/guide.md +0 -130
- package/bundled/locales/zh-CN/skills/methodology-system/SKILL.md +0 -88
- package/bundled/locales/zh-CN/skills/methodology-system/create-methodology.md +0 -350
- package/bundled/locales/zh-CN/skills/methodology-system/guide.md +0 -131
- package/bundled/locales/zh-CN/skills/methodology-system/runtime.md +0 -279
- package/bundled/locales/zh-CN/skills/process-automation/SKILL.md +0 -143
- package/bundled/locales/zh-TW/skills/ac-coverage-assistant/SKILL.md +0 -195
- package/bundled/locales/zh-TW/skills/deploy-assistant/SKILL.md +0 -178
- package/bundled/locales/zh-TW/skills/forward-derivation/SKILL.md +0 -69
- package/bundled/locales/zh-TW/skills/forward-derivation/guide.md +0 -415
- package/bundled/locales/zh-TW/skills/methodology-system/SKILL.md +0 -86
- package/bundled/locales/zh-TW/skills/methodology-system/create-methodology.md +0 -350
- package/bundled/locales/zh-TW/skills/methodology-system/guide.md +0 -131
- package/bundled/locales/zh-TW/skills/methodology-system/runtime.md +0 -279
- package/bundled/locales/zh-TW/skills/process-automation/SKILL.md +0 -144
- /package/bundled/skills/{ac-coverage-assistant → ac-coverage}/SKILL.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/SKILL.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/create-methodology.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/guide.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/integrated-flow.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/prerequisite-check.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/runtime.md +0 -0
- /package/bundled/skills/{forward-derivation → spec-derivation}/guide.md +0 -0
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# Release Quality Manifest Standards - AI Optimized
|
|
2
|
+
# Source: core/release-quality-manifest.md
|
|
3
|
+
|
|
4
|
+
id: release-quality-manifest
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-05"
|
|
8
|
+
source: core/release-quality-manifest.md
|
|
9
|
+
description: Automated per-release Quality Manifest that aggregates all quality gate results into a single machine-readable artifact
|
|
10
|
+
|
|
11
|
+
requirements:
|
|
12
|
+
REQ-1:
|
|
13
|
+
id: REQ-RQM-001
|
|
14
|
+
title: Machine-Readable Format
|
|
15
|
+
rule: >
|
|
16
|
+
Every release MUST produce a Quality Manifest in YAML or JSON format that
|
|
17
|
+
aggregates the results of all defined quality gates. The manifest MUST be
|
|
18
|
+
committed to source control or attached to the release artifact.
|
|
19
|
+
rationale: >
|
|
20
|
+
Machine-readable manifests enable automated release gates and customer audits;
|
|
21
|
+
prose-only release notes cannot be parsed by downstream tooling.
|
|
22
|
+
|
|
23
|
+
REQ-2:
|
|
24
|
+
id: REQ-RQM-002
|
|
25
|
+
title: Gate Coverage
|
|
26
|
+
rule: >
|
|
27
|
+
The Quality Manifest MUST include at minimum: unit test coverage %, mutation
|
|
28
|
+
score %, SCA CVE counts (critical/high), SAST finding counts (high/medium),
|
|
29
|
+
E2E pass rate %, container CVE scan status, image signature status, SBOM
|
|
30
|
+
presence, and (if applicable) LLM hallucination rate and prompt injection
|
|
31
|
+
resistance score.
|
|
32
|
+
rationale: >
|
|
33
|
+
Partial manifests create false confidence; a complete manifest proves end-to-end
|
|
34
|
+
quality rather than cherry-picked metrics.
|
|
35
|
+
|
|
36
|
+
REQ-3:
|
|
37
|
+
id: REQ-RQM-003
|
|
38
|
+
title: Pass/Warn/Fail Status per Gate
|
|
39
|
+
rule: >
|
|
40
|
+
Each gate entry MUST carry a status field: "pass" (meets target), "warn"
|
|
41
|
+
(within acceptable deviation from target), or "fail" (blocks release).
|
|
42
|
+
The manifest MUST have an overall status field derived from the worst gate.
|
|
43
|
+
rationale: >
|
|
44
|
+
Binary pass/fail per gate plus an aggregate status enables release go/no-go
|
|
45
|
+
automation without human judgment on individual metrics.
|
|
46
|
+
|
|
47
|
+
REQ-4:
|
|
48
|
+
id: REQ-RQM-004
|
|
49
|
+
title: Automated Generation in CI
|
|
50
|
+
rule: >
|
|
51
|
+
The Quality Manifest MUST be generated automatically by CI (not manually
|
|
52
|
+
authored). Each gate's value MUST be extracted from the corresponding tool
|
|
53
|
+
output (vitest coverage JSON, stryker JSON, trivy SARIF, etc.).
|
|
54
|
+
rationale: >
|
|
55
|
+
Manually authored manifests are unreliable; CI-generated manifests are the
|
|
56
|
+
only form of evidence that meets audit requirements.
|
|
57
|
+
|
|
58
|
+
REQ-5:
|
|
59
|
+
id: REQ-RQM-005
|
|
60
|
+
title: Customer-Facing Summary
|
|
61
|
+
rule: >
|
|
62
|
+
A human-readable summary of the Quality Manifest (e.g., Markdown table)
|
|
63
|
+
MUST be generated alongside the machine-readable format and included in
|
|
64
|
+
the release notes or documentation.
|
|
65
|
+
rationale: >
|
|
66
|
+
Customers and auditors need a scannable summary; the machine-readable format
|
|
67
|
+
alone does not satisfy human review requirements.
|
|
68
|
+
|
|
69
|
+
manifest_schema:
|
|
70
|
+
release: "string — semver tag e.g. v1.2.0"
|
|
71
|
+
generated_at: "ISO 8601 timestamp"
|
|
72
|
+
commit: "git SHA"
|
|
73
|
+
gates:
|
|
74
|
+
unit_coverage:
|
|
75
|
+
actual: "percentage string e.g. '73%'"
|
|
76
|
+
target: "threshold e.g. '80%'"
|
|
77
|
+
status: "pass | warn | fail"
|
|
78
|
+
mutation_score:
|
|
79
|
+
actual: "percentage string"
|
|
80
|
+
target: "threshold"
|
|
81
|
+
status: "pass | warn | fail"
|
|
82
|
+
sca_critical_cve:
|
|
83
|
+
actual: "integer"
|
|
84
|
+
target: "0"
|
|
85
|
+
status: "pass | fail"
|
|
86
|
+
sca_high_cve:
|
|
87
|
+
actual: "integer"
|
|
88
|
+
target: "0"
|
|
89
|
+
status: "pass | warn | fail"
|
|
90
|
+
sast_high:
|
|
91
|
+
actual: "integer"
|
|
92
|
+
target: "0"
|
|
93
|
+
status: "pass | warn | fail"
|
|
94
|
+
e2e_pass_rate:
|
|
95
|
+
actual: "percentage string"
|
|
96
|
+
target: "threshold"
|
|
97
|
+
status: "pass | warn | fail"
|
|
98
|
+
container_cve_critical:
|
|
99
|
+
actual: "integer"
|
|
100
|
+
target: "0"
|
|
101
|
+
status: "pass | fail"
|
|
102
|
+
image_signed:
|
|
103
|
+
actual: "boolean"
|
|
104
|
+
target: "true"
|
|
105
|
+
status: "pass | fail"
|
|
106
|
+
sbom_present:
|
|
107
|
+
actual: "boolean"
|
|
108
|
+
target: "true"
|
|
109
|
+
status: "pass | fail"
|
|
110
|
+
overall: "PASS | WARN | FAIL"
|
|
111
|
+
|
|
112
|
+
generation_guidance: >
|
|
113
|
+
Extract coverage from vitest --coverage --reporter=json (summary.total.lines.pct).
|
|
114
|
+
Extract mutation score from stryker's mutation-testing-report.json (metrics.mutationScore).
|
|
115
|
+
Extract CVE counts from trivy JSON output (Results[].Vulnerabilities filtered by Severity).
|
|
116
|
+
Extract SAST from CodeQL SARIF (runs[].results filtered by level=error).
|
|
117
|
+
Combine into manifest YAML via a CI shell script or Node.js release script.
|
|
118
|
+
|
|
119
|
+
anti_patterns:
|
|
120
|
+
- description: >
|
|
121
|
+
Generating the manifest after all gates have passed — gates should use
|
|
122
|
+
the manifest values, not precede them.
|
|
123
|
+
- description: >
|
|
124
|
+
Hardcoding metric values in the manifest generation script — all values
|
|
125
|
+
MUST be extracted from tool outputs to remain accurate.
|
|
126
|
+
- description: >
|
|
127
|
+
Using 'warn' status for critical security gates (sca_critical_cve,
|
|
128
|
+
container_cve_critical) — critical security gates are binary pass/fail.
|
|
129
|
+
|
|
130
|
+
related_standards:
|
|
131
|
+
- testing
|
|
132
|
+
- security-testing
|
|
133
|
+
- supply-chain-attestation
|
|
134
|
+
- verification-evidence
|
|
135
|
+
- deployment-standards
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# SPDX-License-Identifier: MIT
|
|
2
|
+
name: Replay Test Standards
|
|
3
|
+
nameZh: 回放測試標準
|
|
4
|
+
id: replay-test
|
|
5
|
+
version: "1.0.0"
|
|
6
|
+
category: testing
|
|
7
|
+
scope: ai-agent-systems
|
|
8
|
+
summary: >
|
|
9
|
+
Golden fixture recording and deterministic replay for AI agent pipelines.
|
|
10
|
+
Enables customer bug reproduction, verdict regression detection, and
|
|
11
|
+
on-site incident investigation without requiring a live LLM.
|
|
12
|
+
|
|
13
|
+
requirements:
|
|
14
|
+
- id: REQ-01
|
|
15
|
+
title: Golden Fixture Format
|
|
16
|
+
titleZh: 黃金 fixture 格式
|
|
17
|
+
level: MUST
|
|
18
|
+
description: >
|
|
19
|
+
Each replay fixture MUST be a JSON file containing: (1) the exact input
|
|
20
|
+
that triggered the behaviour, (2) the expected output (decision/verdict),
|
|
21
|
+
(3) metadata (date recorded, source — customer report / CI regression /
|
|
22
|
+
incident, description). Fixtures MUST be deterministic (same input always
|
|
23
|
+
produces same output for pure-function components).
|
|
24
|
+
|
|
25
|
+
- id: REQ-02
|
|
26
|
+
title: Replay Test Suite
|
|
27
|
+
titleZh: 回放測試套件
|
|
28
|
+
level: MUST
|
|
29
|
+
description: >
|
|
30
|
+
A dedicated replay test file MUST load each fixture and assert that
|
|
31
|
+
re-running the component under test produces the recorded expected output.
|
|
32
|
+
For AI components with LLM dependencies, replay MUST mock the LLM layer
|
|
33
|
+
and test only the deterministic logic (scoring, routing, policy evaluation).
|
|
34
|
+
|
|
35
|
+
- id: REQ-03
|
|
36
|
+
title: Bug Regression Capture
|
|
37
|
+
titleZh: Bug 回歸捕捉
|
|
38
|
+
level: MUST
|
|
39
|
+
description: >
|
|
40
|
+
When a production bug is reported, a fixture MUST be created from the
|
|
41
|
+
failing input within the same PR that fixes the bug. The fixture prevents
|
|
42
|
+
the bug from being reintroduced silently.
|
|
43
|
+
|
|
44
|
+
- id: REQ-04
|
|
45
|
+
title: Fixture Coverage
|
|
46
|
+
titleZh: Fixture 覆蓋
|
|
47
|
+
level: SHOULD
|
|
48
|
+
description: >
|
|
49
|
+
The fixture set SHOULD include at least one representative for each
|
|
50
|
+
decision outcome (e.g. ALLOW / REQUIRE_HITL / DENY for Guardian).
|
|
51
|
+
Edge cases reported by customers or from red-team exercises SHOULD be
|
|
52
|
+
added as separate fixtures.
|
|
53
|
+
|
|
54
|
+
- id: REQ-05
|
|
55
|
+
title: Fixture Naming Convention
|
|
56
|
+
titleZh: Fixture 命名規範
|
|
57
|
+
level: MUST
|
|
58
|
+
description: >
|
|
59
|
+
Fixture files MUST follow the pattern:
|
|
60
|
+
`<component>-<outcome>-<short-description>.json`
|
|
61
|
+
e.g. `guardian-deny-prod-drop-table.json`,
|
|
62
|
+
`guardian-allow-dev-npm-install.json`
|
|
63
|
+
|
|
64
|
+
examples:
|
|
65
|
+
- name: "Guardian replay fixture file"
|
|
66
|
+
code: |
|
|
67
|
+
{
|
|
68
|
+
"meta": {
|
|
69
|
+
"recorded": "2026-05-05",
|
|
70
|
+
"source": "red-team-exercise",
|
|
71
|
+
"description": "DROP TABLE in prod should DENY"
|
|
72
|
+
},
|
|
73
|
+
"input": {
|
|
74
|
+
"session_id": "replay-001",
|
|
75
|
+
"source_agent": "operator",
|
|
76
|
+
"intent": "Clean up test data",
|
|
77
|
+
"plan": [{"command": "DROP TABLE users;", "command_type": "mutate", "target_resource": "db_schema", "reversible": false}],
|
|
78
|
+
"target_env": "prod",
|
|
79
|
+
"reversible": false
|
|
80
|
+
},
|
|
81
|
+
"expected": {
|
|
82
|
+
"decision": "DENY"
|
|
83
|
+
}
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
- name: "Replay test loading fixtures"
|
|
87
|
+
code: |
|
|
88
|
+
const fixtures = readdirSync('src/guardian/__fixtures__')
|
|
89
|
+
.filter(f => f.endsWith('.json'))
|
|
90
|
+
.map(f => JSON.parse(readFileSync(join('src/guardian/__fixtures__', f), 'utf-8')))
|
|
91
|
+
|
|
92
|
+
for (const { meta, input, expected } of fixtures) {
|
|
93
|
+
it(meta.description, () => {
|
|
94
|
+
const result = scoreReviewable(input)
|
|
95
|
+
const decision = deriveDecision(result.score)
|
|
96
|
+
expect(decision).toBe(expected.decision)
|
|
97
|
+
})
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
anti_patterns:
|
|
101
|
+
- description: >
|
|
102
|
+
Fixtures without metadata fields — without source and date, it's
|
|
103
|
+
impossible to know why a fixture exists or when it was added.
|
|
104
|
+
- description: >
|
|
105
|
+
Creating fixtures only for the happy path — the most valuable fixtures
|
|
106
|
+
are customer-reported failures and red-team findings.
|
|
107
|
+
|
|
108
|
+
related_standards:
|
|
109
|
+
- adversarial-test
|
|
110
|
+
- testing
|
|
111
|
+
- verification-evidence
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
# Runbook Standards - AI Optimized
|
|
2
|
+
# Source: XSPEC-063 Wave 3 SRE Pack
|
|
3
|
+
|
|
4
|
+
id: runbook
|
|
5
|
+
title: Runbook Writing Standards
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
status: Active
|
|
8
|
+
tags: [sre, operations, runbook, incident, oncall]
|
|
9
|
+
summary: |
|
|
10
|
+
Defines how operational runbooks are written, organized, maintained, and
|
|
11
|
+
tested. Covers required sections, writing principles (reproducible,
|
|
12
|
+
unambiguous steps), directory structure, review cadence, and drill
|
|
13
|
+
frequency. A well-written runbook reduces Mean Time To Repair (MTTR) by
|
|
14
|
+
ensuring any on-call engineer can execute recovery steps without requiring
|
|
15
|
+
tribal knowledge.
|
|
16
|
+
|
|
17
|
+
requirements:
|
|
18
|
+
- id: REQ-001
|
|
19
|
+
title: Required Runbook Sections
|
|
20
|
+
description: |
|
|
21
|
+
Every runbook MUST include the following sections in order:
|
|
22
|
+
(1) Overview — alert name, severity, affected services, owner, last
|
|
23
|
+
updated, last drilled date; (2) Symptoms — observable indicators;
|
|
24
|
+
(3) Impact Assessment — user-facing effect and blast radius;
|
|
25
|
+
(4) Diagnostic Steps — ordered steps with copy-pasteable commands;
|
|
26
|
+
(5) Fix Steps — ordered remediation with verification for each step;
|
|
27
|
+
(6) Escalation — specific contacts with role and availability;
|
|
28
|
+
(7) Post-Actions — follow-up tasks, tickets, postmortem triggers.
|
|
29
|
+
level: MUST
|
|
30
|
+
examples:
|
|
31
|
+
- "Overview: Alert=HighErrorRate, Severity=P2, Service=payment-api, Owner=@alice"
|
|
32
|
+
- "Diagnostic: `kubectl get pods -n payments | grep -v Running`"
|
|
33
|
+
- "Escalation: If not resolved in 30min, page @payment-lead (PD: pd-payments)"
|
|
34
|
+
|
|
35
|
+
- id: REQ-002
|
|
36
|
+
title: Reproducible and Unambiguous Steps
|
|
37
|
+
description: |
|
|
38
|
+
Each step in a runbook MUST be reproducible and unambiguous. Steps
|
|
39
|
+
MUST use copy-pasteable commands with no placeholders left undefined.
|
|
40
|
+
Decision points MUST include explicit branch conditions (if X then Y,
|
|
41
|
+
else Z). Every fix step MUST include a verification command confirming
|
|
42
|
+
the fix worked before proceeding. Expected output MUST be shown.
|
|
43
|
+
level: MUST
|
|
44
|
+
examples:
|
|
45
|
+
- "Bad: 'Restart the service' — Good: `systemctl restart payment-api && systemctl status payment-api`"
|
|
46
|
+
- "Branch: If error count > 100/min go to step 5a, else go to step 5b"
|
|
47
|
+
- "Verify: `curl -s https://api/health | jq .status` should return 'ok'"
|
|
48
|
+
|
|
49
|
+
- id: REQ-003
|
|
50
|
+
title: Runbook Naming and Directory Organization
|
|
51
|
+
description: |
|
|
52
|
+
Runbooks MUST use kebab-case names that describe the problem, not the
|
|
53
|
+
solution. Files MUST be organized into typed directories:
|
|
54
|
+
alerts/ for alert-response runbooks, operations/ for standard ops,
|
|
55
|
+
emergency/ for major incident procedures, troubleshooting/ for
|
|
56
|
+
general investigation guides. Each runbook file MUST declare its type
|
|
57
|
+
in the front matter.
|
|
58
|
+
level: MUST
|
|
59
|
+
examples:
|
|
60
|
+
- "Good: alerts/payment-api-high-error-rate.md"
|
|
61
|
+
- "Bad: runbooks/restart-payment-service.md"
|
|
62
|
+
- "Front matter: type: alert_response, alert: HighErrorRateAlert"
|
|
63
|
+
|
|
64
|
+
- id: REQ-004
|
|
65
|
+
title: Review and Drill Cadence
|
|
66
|
+
description: |
|
|
67
|
+
Runbooks MUST be reviewed on schedule based on type: alert-response
|
|
68
|
+
runbooks quarterly, emergency procedures monthly, standard operation
|
|
69
|
+
and troubleshooting guides bi-annually, change procedures after each
|
|
70
|
+
use. Runbooks MUST be drilled: P1 runbooks monthly, P2 quarterly,
|
|
71
|
+
emergency procedures quarterly. Drill records must be appended to the
|
|
72
|
+
runbook or linked from it.
|
|
73
|
+
level: MUST
|
|
74
|
+
examples:
|
|
75
|
+
- "Drill record: 2026-03-15, participants: @alice @bob, result: pass, MTTR: 12min"
|
|
76
|
+
- "Quarterly review: updated diagnostic commands after k8s upgrade"
|
|
77
|
+
- "Post-change update: change-procedure/db-failover.md updated after March 14 drill"
|
|
78
|
+
|
|
79
|
+
- id: REQ-005
|
|
80
|
+
title: Rollback and Fallback Steps
|
|
81
|
+
description: |
|
|
82
|
+
Any runbook describing a change or fix MUST include a clearly labeled
|
|
83
|
+
rollback section describing how to undo the change if the fix fails
|
|
84
|
+
or causes additional issues. The rollback section MUST appear before
|
|
85
|
+
the escalation section and include its own verification steps.
|
|
86
|
+
level: MUST
|
|
87
|
+
examples:
|
|
88
|
+
- "Rollback: `helm rollback payment-api 1 && kubectl rollout status deployment/payment-api`"
|
|
89
|
+
- "Rollback verify: error rate returns below 1% within 3 minutes"
|
|
90
|
+
- "If rollback fails: escalate immediately, do not attempt further fixes"
|
|
91
|
+
|
|
92
|
+
- id: REQ-006
|
|
93
|
+
title: Alert Integration Metadata
|
|
94
|
+
description: |
|
|
95
|
+
Alert-response runbooks SHOULD include a metadata block linking the
|
|
96
|
+
runbook to specific alert rules. This enables automatic runbook
|
|
97
|
+
linking in alerting tools (PagerDuty, Alertmanager). Metadata MUST
|
|
98
|
+
include the alert name, dashboard URL, and Prometheus/logging query
|
|
99
|
+
used to investigate.
|
|
100
|
+
level: SHOULD
|
|
101
|
+
examples:
|
|
102
|
+
- "alert_name: PaymentAPIHighErrorRate5xx"
|
|
103
|
+
- "dashboard: https://grafana/d/payment-api-overview"
|
|
104
|
+
- "query: sum(rate(http_requests_total{status=~'5..'}[5m])) by (service)"
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# Advanced SAST Standards - AI Optimized
|
|
2
|
+
# Source: XSPEC-161 SAST Advanced
|
|
3
|
+
|
|
4
|
+
id: sast-advanced
|
|
5
|
+
title: Advanced SAST Standards
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
status: Active
|
|
8
|
+
tags: [sast, codeql, gitleaks, trufflehog, secret-scanning, biome, static-analysis, security, typescript, ci]
|
|
9
|
+
summary: |
|
|
10
|
+
Defines advanced Static Application Security Testing (SAST) practices
|
|
11
|
+
beyond basic dependency auditing. Covers three complementary layers:
|
|
12
|
+
(1) CodeQL semantic analysis for TypeScript/JavaScript — detects injection
|
|
13
|
+
vulnerabilities, path traversal, prototype pollution, and XSS that npm audit
|
|
14
|
+
misses; (2) Secret scanning with gitleaks or TruffleHog — prevents committing
|
|
15
|
+
API keys, tokens, and credentials; (3) Biome-based code quality rules used as
|
|
16
|
+
a security lint layer for projects that adopt Biome instead of ESLint.
|
|
17
|
+
Specifies CI enforcement thresholds: block merge on any CRITICAL or HIGH
|
|
18
|
+
severity finding; 0 High findings is the passing gate.
|
|
19
|
+
|
|
20
|
+
requirements:
|
|
21
|
+
- id: REQ-001
|
|
22
|
+
title: CodeQL Semantic Analysis
|
|
23
|
+
description: |
|
|
24
|
+
Projects using TypeScript or JavaScript MUST run CodeQL semantic analysis
|
|
25
|
+
on every push to the default branch and on every pull request targeting
|
|
26
|
+
the default branch. Additionally, a weekly scheduled scan MUST be
|
|
27
|
+
configured to catch newly published query packs. Use the
|
|
28
|
+
`security-extended` query suite rather than the default suite to include
|
|
29
|
+
injection, path-traversal, prototype-pollution, and XSS queries.
|
|
30
|
+
CodeQL MUST be configured with `github/codeql-action/init@v3` with
|
|
31
|
+
`languages: javascript-typescript` and query filters narrowed to
|
|
32
|
+
`security-extended`. The `autobuild` action MUST be used unless a
|
|
33
|
+
custom build command is required. Results MUST be uploaded to GitHub
|
|
34
|
+
Code Scanning so that SARIF findings appear in the Security tab and
|
|
35
|
+
block PRs when severity >= HIGH.
|
|
36
|
+
level: MUST
|
|
37
|
+
examples:
|
|
38
|
+
- "Workflow trigger: push to main + pull_request targeting main + schedule weekly"
|
|
39
|
+
- "init step: uses: github/codeql-action/init@v3 with languages: javascript-typescript"
|
|
40
|
+
- "query-filters: include tags contain security-extended"
|
|
41
|
+
- "analyze step uploads SARIF; GitHub blocks PR merge when CRITICAL/HIGH found"
|
|
42
|
+
- "Weekly cron: '0 2 * * 1' catches newly published CodeQL query packs"
|
|
43
|
+
|
|
44
|
+
- id: REQ-002
|
|
45
|
+
title: Secret Scanning on Every Push and PR
|
|
46
|
+
description: |
|
|
47
|
+
A secret scanning step MUST run on every push and pull request to detect
|
|
48
|
+
accidentally committed API keys, tokens, private keys, and credentials
|
|
49
|
+
before they reach the repository history. The recommended tool is
|
|
50
|
+
gitleaks (via `gitleaks/gitleaks-action@v2`). Projects MUST maintain a
|
|
51
|
+
`.gitleaks.toml` configuration file in the repository root to declare
|
|
52
|
+
custom patterns for project-specific secret formats and to whitelist
|
|
53
|
+
documented false positives via allowlist rules. The CI step MUST fail
|
|
54
|
+
with a non-zero exit code when any secret is detected, blocking merge.
|
|
55
|
+
Baseline false positives MUST be documented and reviewed quarterly.
|
|
56
|
+
level: MUST
|
|
57
|
+
examples:
|
|
58
|
+
- "CI job: uses gitleaks/gitleaks-action@v2 on push and pull_request events"
|
|
59
|
+
- ".gitleaks.toml defines custom patterns for project-specific tokens"
|
|
60
|
+
- "Allowlist rule added with documented justification for known false positive"
|
|
61
|
+
- "Secret detected → CI fails → developer rotates credential before re-pushing"
|
|
62
|
+
|
|
63
|
+
- id: REQ-003
|
|
64
|
+
title: Biome Security Rules as Lint Gate
|
|
65
|
+
description: |
|
|
66
|
+
Projects using Biome as their linter MUST enable the security-relevant
|
|
67
|
+
linting rules as part of the standard lint pass. Biome's `nursery` and
|
|
68
|
+
`suspicious` rule groups include patterns such as `noConsoleLog` (prevents
|
|
69
|
+
accidental secret logging), `noGlobalEval` (prevents dynamic code execution),
|
|
70
|
+
`noWith` (prevents scope pollution), and `noDebugger` (prevents debug
|
|
71
|
+
breakpoints in production). The `biome check .` command MUST be run in CI
|
|
72
|
+
as a blocking step. Projects MUST NOT use `--allow-errors` to bypass
|
|
73
|
+
security-relevant rule failures. Configuration in `biome.json` MUST set
|
|
74
|
+
`linter.enabled: true` and include the `recommended: true` ruleset plus
|
|
75
|
+
any additional security rules appropriate to the project.
|
|
76
|
+
level: MUST
|
|
77
|
+
examples:
|
|
78
|
+
- "biome.json: linter.enabled true, rules.suspicious.noGlobalEval error"
|
|
79
|
+
- "CI step: `npm run lint` maps to `biome check .`; fails on security rule violations"
|
|
80
|
+
- "noConsoleLog prevents accidental logging of JWT tokens or API keys"
|
|
81
|
+
- "noGlobalEval prevents eval() usage that could enable code injection"
|
|
82
|
+
|
|
83
|
+
- id: REQ-004
|
|
84
|
+
title: CI Quality Gate Thresholds
|
|
85
|
+
description: |
|
|
86
|
+
SAST findings MUST be classified by severity and the following merge-blocking
|
|
87
|
+
thresholds MUST be enforced: CRITICAL findings — block merge immediately,
|
|
88
|
+
no exceptions; HIGH findings — block merge; target is 0 HIGH findings in
|
|
89
|
+
all open PRs; MEDIUM findings — create tracking issue, do not block merge,
|
|
90
|
+
resolve within 30 days; LOW/INFORMATIONAL findings — optional resolution,
|
|
91
|
+
logged for visibility. GitHub Code Scanning branch protection rules MUST
|
|
92
|
+
be configured to require the CodeQL check to pass before merging.
|
|
93
|
+
The `sast` CI job MUST be listed as a required status check in branch
|
|
94
|
+
protection settings.
|
|
95
|
+
level: MUST
|
|
96
|
+
examples:
|
|
97
|
+
- "Branch protection: require CodeQL check + sast job to pass before merge"
|
|
98
|
+
- "CRITICAL finding in CodeQL → PR blocked; developer must fix or accept risk with CISO sign-off"
|
|
99
|
+
- "HIGH finding threshold: 0 HIGH open on main branch at all times"
|
|
100
|
+
- "MEDIUM finding: GitHub issue opened automatically via SARIF annotation; 30-day SLA"
|
|
101
|
+
|
|
102
|
+
- id: REQ-005
|
|
103
|
+
title: What CodeQL Finds That npm audit Misses
|
|
104
|
+
description: |
|
|
105
|
+
Teams MUST understand that npm audit only checks published CVEs in
|
|
106
|
+
dependency manifests (package.json lock file). CodeQL performs semantic
|
|
107
|
+
data-flow analysis on the actual source code and detects:
|
|
108
|
+
(1) Command injection — user-controlled data flowing into child_process.exec
|
|
109
|
+
or child_process.spawn with shell:true; (2) Path traversal — user input
|
|
110
|
+
used in fs.readFile / fs.writeFile without path.resolve + allowlist check;
|
|
111
|
+
(3) Prototype pollution — assignment to obj[userKey] = value where userKey
|
|
112
|
+
is unsanitized; (4) XSS via DOM sinks — user data assigned to innerHTML,
|
|
113
|
+
document.write, or passed to eval; (5) SQL injection via string concatenation
|
|
114
|
+
in ORM raw queries. These vulnerabilities exist in first-party code and are
|
|
115
|
+
invisible to dependency scanners.
|
|
116
|
+
level: SHOULD
|
|
117
|
+
examples:
|
|
118
|
+
- "CodeQL flags: exec(`git log ${userInput}`) as command injection"
|
|
119
|
+
- "CodeQL flags: fs.readFile(path.join(base, req.params.file)) as path traversal"
|
|
120
|
+
- "CodeQL flags: target[req.body.key] = req.body.value as prototype pollution"
|
|
121
|
+
- "npm audit shows 0 vulnerabilities; CodeQL finds path traversal in custom route handler"
|
|
122
|
+
|
|
123
|
+
anti_patterns:
|
|
124
|
+
- "Running only npm audit and considering security 'done'"
|
|
125
|
+
- "Disabling CodeQL on PRs to speed up CI — defeats the purpose of shift-left security"
|
|
126
|
+
- "Adding entire file paths to gitleaks allowlist without reviewing the actual content"
|
|
127
|
+
- "Ignoring MEDIUM CodeQL findings indefinitely without tracking issues"
|
|
128
|
+
- "Using biome check --allow-errors to bypass security lint failures"
|
|
129
|
+
- "Not uploading SARIF results to GitHub Code Scanning (findings invisible in UI)"
|
|
130
|
+
|
|
131
|
+
related_standards:
|
|
132
|
+
- security-standards
|
|
133
|
+
- secret-management-standards
|
|
134
|
+
- checkin-standards
|
|
135
|
+
- container-security
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Schema Evolution Standards - AI Optimized
|
|
2
|
+
# Source: XSPEC-068 Wave 3 Data Engineering Pack
|
|
3
|
+
|
|
4
|
+
id: schema-evolution
|
|
5
|
+
title: Schema Evolution Standards
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
status: Active
|
|
8
|
+
tags: [data-engineering, schema, migration, backward-compatibility, database]
|
|
9
|
+
summary: |
|
|
10
|
+
Defines how database and data store schemas are evolved safely without
|
|
11
|
+
breaking existing consumers. Covers backward-compatible change patterns,
|
|
12
|
+
prohibited breaking changes, expand-contract migration strategy, schema
|
|
13
|
+
versioning, automated compatibility checking in CI/CD, and rollback
|
|
14
|
+
procedures for schema changes. Applicable to relational databases,
|
|
15
|
+
document stores, event schemas (Avro/Protobuf), and API request/response
|
|
16
|
+
schemas.
|
|
17
|
+
|
|
18
|
+
requirements:
|
|
19
|
+
- id: REQ-001
|
|
20
|
+
title: Backward-Compatible Change Patterns
|
|
21
|
+
description: |
|
|
22
|
+
All schema changes MUST be backward-compatible unless a formal
|
|
23
|
+
breaking-change process is followed. Backward-compatible changes
|
|
24
|
+
include: adding new nullable columns/fields with defaults, adding
|
|
25
|
+
new tables/collections, adding new optional message fields, adding
|
|
26
|
+
new enum values (with unknown handling), widening a data type
|
|
27
|
+
(INT → BIGINT), and adding new indexes. These changes MUST be
|
|
28
|
+
deployable without coordinating consumer updates.
|
|
29
|
+
level: MUST
|
|
30
|
+
examples:
|
|
31
|
+
- "Add column: ALTER TABLE users ADD COLUMN middle_name VARCHAR(100) DEFAULT NULL"
|
|
32
|
+
- "Add optional Protobuf field: optional string nickname = 15; — safe, zero default"
|
|
33
|
+
- "Add index: CREATE INDEX CONCURRENTLY idx_orders_user_id ON orders(user_id)"
|
|
34
|
+
|
|
35
|
+
- id: REQ-002
|
|
36
|
+
title: Prohibited Breaking Changes Without Migration Plan
|
|
37
|
+
description: |
|
|
38
|
+
The following schema changes are classified as BREAKING and MUST NOT
|
|
39
|
+
be deployed without a formal expand-contract migration plan and
|
|
40
|
+
consumer coordination: renaming or dropping columns/fields, changing
|
|
41
|
+
data types incompatibly (VARCHAR → INT, widening to narrowing),
|
|
42
|
+
adding NOT NULL constraints to existing columns without defaults,
|
|
43
|
+
changing primary or foreign key definitions, removing enum values,
|
|
44
|
+
and changing field semantics (repurposing a column for different data).
|
|
45
|
+
level: MUST
|
|
46
|
+
examples:
|
|
47
|
+
- "PROHIBITED: ALTER TABLE users DROP COLUMN legacy_id → requires expand-contract"
|
|
48
|
+
- "PROHIBITED: ALTER TABLE orders ALTER COLUMN amount TYPE BIGINT — evaluate consumer impact first"
|
|
49
|
+
- "PROHIBITED: adding NOT NULL to existing column with NULL values in production data"
|
|
50
|
+
|
|
51
|
+
- id: REQ-003
|
|
52
|
+
title: Expand-Contract Migration Strategy
|
|
53
|
+
description: |
|
|
54
|
+
Breaking schema changes MUST use the expand-contract (parallel change)
|
|
55
|
+
pattern: Phase 1 (Expand) — add new structure alongside old; Phase 2
|
|
56
|
+
(Migrate) — backfill data from old to new, update all writers;
|
|
57
|
+
Phase 3 (Contract) — update all readers to use new structure;
|
|
58
|
+
Phase 4 (Cleanup) — remove old structure after all consumers updated.
|
|
59
|
+
Each phase MUST be a separate deployment, verified before proceeding.
|
|
60
|
+
Minimum wait between phases: 1 full deployment cycle.
|
|
61
|
+
level: MUST
|
|
62
|
+
examples:
|
|
63
|
+
- "Rename column email → contact_email: add contact_email, dual-write for 1 sprint, migrate readers, drop email"
|
|
64
|
+
- "Phase gate: 'Phase 2 complete when 0 reads of old column seen in query logs over 7 days'"
|
|
65
|
+
- "Cleanup verified: `SELECT COUNT(*) FROM users WHERE email IS NOT NULL` returns 0 before dropping"
|
|
66
|
+
|
|
67
|
+
- id: REQ-004
|
|
68
|
+
title: Schema Versioning and Registry
|
|
69
|
+
description: |
|
|
70
|
+
Event-driven and API schemas (Avro, Protobuf, JSON Schema) MUST be
|
|
71
|
+
registered in a schema registry with explicit version numbers.
|
|
72
|
+
Schema versions MUST follow semantic versioning: PATCH for backward-
|
|
73
|
+
compatible additions, MINOR for new optional fields, MAJOR for
|
|
74
|
+
breaking changes. Every schema change MUST be reviewed and approved
|
|
75
|
+
before registration. Consumers MUST specify the schema version they
|
|
76
|
+
consume.
|
|
77
|
+
level: MUST
|
|
78
|
+
examples:
|
|
79
|
+
- "Confluent Schema Registry: subject 'order-events-value', version 3 registered"
|
|
80
|
+
- "Schema compatibility: BACKWARD_TRANSITIVE enforced — new schema must be readable by all prior consumers"
|
|
81
|
+
- "Consumer declaration: {schema_id: 'order-events', version: '>=2.0.0 <3.0.0'}"
|
|
82
|
+
|
|
83
|
+
- id: REQ-005
|
|
84
|
+
title: Automated Schema Compatibility Checking in CI
|
|
85
|
+
description: |
|
|
86
|
+
Every PR modifying schema definitions MUST trigger automated
|
|
87
|
+
compatibility checks in CI. For relational schemas, migration scripts
|
|
88
|
+
MUST be run against a production-snapshot database in CI to detect
|
|
89
|
+
errors before merge. For event schemas, compatibility MUST be checked
|
|
90
|
+
against all registered consumer versions. Compatibility failures MUST
|
|
91
|
+
block the PR merge.
|
|
92
|
+
level: MUST
|
|
93
|
+
examples:
|
|
94
|
+
- "CI step: `flyway validate -url=jdbc:postgresql://test-db/snapshots` on every migration PR"
|
|
95
|
+
- "Avro compatibility: `schema-registry-validate --mode BACKWARD_TRANSITIVE new-schema.avsc`"
|
|
96
|
+
- "Failed check message: 'Schema change removes required field order_id — breaks consumer v2.3'"
|
|
97
|
+
|
|
98
|
+
- id: REQ-006
|
|
99
|
+
title: Schema Change Rollback Procedures
|
|
100
|
+
description: |
|
|
101
|
+
Every schema migration script MUST have a corresponding rollback
|
|
102
|
+
(down) migration script. Rollback scripts MUST be tested in CI
|
|
103
|
+
alongside the forward migration. For destructive changes (drops,
|
|
104
|
+
type changes), a data backup MUST be taken and verified before
|
|
105
|
+
execution. The rollback plan MUST be documented in the migration
|
|
106
|
+
PR and referenced in the deployment runbook.
|
|
107
|
+
level: MUST
|
|
108
|
+
examples:
|
|
109
|
+
- "V12__add_user_tier.sql paired with V12__add_user_tier.undo.sql tested in CI"
|
|
110
|
+
- "Pre-migration backup: `pg_dump orders_table > backup-2026-04-30-pre-v12.dump`"
|
|
111
|
+
- "Rollback PR comment: 'Rollback: run V12.undo.sql, takes ~2min, no data loss'"
|