system-design-primer-skill 1.2.0 → 2.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +106 -60
- package/SKILL.md +417 -93
- package/VERSION +1 -1
- package/package.json +44 -3
- package/references/10-workflows-and-checklists.md +1 -1
- package/references/13-architecture-intelligence.md +91 -0
- package/references/14-api-and-data-contracts.md +80 -0
- package/references/15-failure-analysis.md +84 -0
- package/references/16-multi-agent-architecture.md +76 -0
- package/references/17-agent-harness-and-runtime.md +87 -0
- package/references/18-mcp-architecture.md +67 -0
- package/references/19-agent-security-and-permissions.md +74 -0
- package/references/20-agent-state-and-memory.md +72 -0
- package/references/21-agent-evaluation-and-observability.md +90 -0
- package/references/22-kubernetes-and-container-platforms.md +29 -0
- package/references/23-service-mesh-and-traffic-management.md +21 -0
- package/references/24-multi-region-and-disaster-recovery.md +24 -0
- package/references/25-platform-engineering-and-developer-experience.md +26 -0
- package/references/26-finops-and-cloud-cost.md +29 -0
- package/references/27-production-observability-and-slo.md +29 -0
- package/references/28-event-streaming-and-kafka-pulsar.md +27 -0
- package/references/29-event-sourcing-and-cqrs.md +23 -0
- package/references/30-change-data-capture-and-debezium.md +27 -0
- package/references/31-data-lakehouse-and-stream-batch.md +24 -0
- package/references/32-workflow-orchestration-and-temporal.md +24 -0
- package/references/33-schema-registry-and-event-governance.md +22 -0
- package/references/34-postgresql-internals-and-performance.md +43 -0
- package/references/35-mysql-innodb-and-performance.md +27 -0
- package/references/36-sharding-and-distributed-sql.md +22 -0
- package/references/37-consensus-and-distributed-transactions.md +25 -0
- package/references/38-search-architecture-elasticsearch-opensearch.md +24 -0
- package/references/39-vector-databases-and-hybrid-retrieval.md +31 -0
- package/references/40-zero-downtime-database-migrations.md +27 -0
- package/references/41-api-gateway-and-edge-architecture.md +23 -0
- package/references/42-rest-api-design-and-lifecycle.md +23 -0
- package/references/43-graphql-architecture-and-governance.md +16 -0
- package/references/44-grpc-and-service-contracts.md +16 -0
- package/references/45-websocket-sse-and-realtime-systems.md +16 -0
- package/references/46-realtime-collaboration-ot-crdt.md +16 -0
- package/references/47-api-security-rate-limiting-and-abuse.md +16 -0
- package/references/48-oauth-oidc-and-passkeys.md +20 -0
- package/references/49-iam-rbac-abac-rebac.md +16 -0
- package/references/50-zero-trust-and-workload-identity.md +16 -0
- package/references/51-secrets-kms-pki-and-key-management.md +16 -0
- package/references/52-supply-chain-security-sbom-slsa.md +16 -0
- package/references/53-threat-modeling-stride-and-attack-trees.md +16 -0
- package/references/54-privacy-engineering-and-data-governance.md +16 -0
- package/references/55-opentelemetry-and-instrumentation.md +50 -0
- package/references/56-distributed-tracing-and-context-propagation.md +43 -0
- package/references/57-continuous-profiling-and-performance-diagnostics.md +45 -0
- package/references/58-load-stress-soak-testing-and-workload-modeling.md +44 -0
- package/references/59-capacity-planning-and-performance-modeling.md +34 -0
- package/references/60-chaos-engineering-and-resilience-experiments.md +41 -0
- package/references/61-incident-management-and-postmortems.md +34 -0
- package/references/62-dns-and-service-discovery.md +35 -0
- package/references/63-load-balancing-and-global-traffic-management.md +31 -0
- package/references/64-cdn-and-edge-caching.md +34 -0
- package/references/65-cloud-networking-and-connectivity.md +34 -0
- package/references/66-edge-computing-architecture.md +33 -0
- package/references/67-storage-infrastructure-and-storage-systems.md +30 -0
- package/references/68-tcp-quic-and-network-performance.md +27 -0
- package/references/69-task-routing-and-evidence.md +58 -0
- package/scripts/install.ps1 +10 -1
- package/scripts/install.sh +13 -6
- package/scripts/validate.py +133 -0
- package/templates/agent-evaluation-plan-template.md +57 -0
- package/templates/agent-security-review-template.md +46 -0
- package/templates/agent-system-design-template.md +74 -0
- package/templates/api-architecture-design-template.md +33 -0
- package/templates/api-security-and-rate-limit-review-template.md +29 -0
- package/templates/architecture-decision-matrix-template.md +48 -0
- package/templates/architecture-review-scorecard-template.md +36 -0
- package/templates/cdc-data-pipeline-template.md +41 -0
- package/templates/cdn-and-edge-strategy-template.md +36 -0
- package/templates/chaos-experiment-template.md +39 -0
- package/templates/cloud-cost-model-template.md +37 -0
- package/templates/database-architecture-review-template.md +33 -0
- package/templates/edge-computing-readiness-template.md +36 -0
- package/templates/event-sourcing-cqrs-template.md +41 -0
- package/templates/event-streaming-design-template.md +47 -0
- package/templates/failure-analysis-template.md +36 -0
- package/templates/identity-and-access-architecture-template.md +29 -0
- package/templates/incident-postmortem-template.md +39 -0
- package/templates/multi-region-disaster-recovery-template.md +43 -0
- package/templates/network-architecture-design-template.md +38 -0
- package/templates/observability-architecture-template.md +52 -0
- package/templates/performance-test-plan-template.md +52 -0
- package/templates/privacy-data-governance-template.md +24 -0
- package/templates/production-readiness-checklist-template.md +52 -0
- package/templates/realtime-system-design-template.md +28 -0
- package/templates/security-threat-model-template.md +26 -0
- package/templates/sharding-distributed-sql-design-template.md +34 -0
- package/templates/storage-architecture-design-template.md +35 -0
- package/templates/supply-chain-security-review-template.md +20 -0
- package/templates/zero-downtime-database-migration-template.md +32 -0
package/scripts/validate.py
CHANGED
|
@@ -22,11 +22,98 @@ REQUIRED_REFERENCES = {
|
|
|
22
22
|
"10-workflows-and-checklists.md",
|
|
23
23
|
"11-case-studies.md",
|
|
24
24
|
"12-ai-systems-architecture.md",
|
|
25
|
+
"13-architecture-intelligence.md",
|
|
26
|
+
"14-api-and-data-contracts.md",
|
|
27
|
+
"15-failure-analysis.md",
|
|
28
|
+
"16-multi-agent-architecture.md",
|
|
29
|
+
"17-agent-harness-and-runtime.md",
|
|
30
|
+
"18-mcp-architecture.md",
|
|
31
|
+
"19-agent-security-and-permissions.md",
|
|
32
|
+
"20-agent-state-and-memory.md",
|
|
33
|
+
"21-agent-evaluation-and-observability.md",
|
|
34
|
+
"22-kubernetes-and-container-platforms.md",
|
|
35
|
+
"23-service-mesh-and-traffic-management.md",
|
|
36
|
+
"24-multi-region-and-disaster-recovery.md",
|
|
37
|
+
"25-platform-engineering-and-developer-experience.md",
|
|
38
|
+
"26-finops-and-cloud-cost.md",
|
|
39
|
+
"27-production-observability-and-slo.md",
|
|
40
|
+
"28-event-streaming-and-kafka-pulsar.md",
|
|
41
|
+
"29-event-sourcing-and-cqrs.md",
|
|
42
|
+
"30-change-data-capture-and-debezium.md",
|
|
43
|
+
"31-data-lakehouse-and-stream-batch.md",
|
|
44
|
+
"32-workflow-orchestration-and-temporal.md",
|
|
45
|
+
"33-schema-registry-and-event-governance.md",
|
|
46
|
+
"34-postgresql-internals-and-performance.md",
|
|
47
|
+
"35-mysql-innodb-and-performance.md",
|
|
48
|
+
"36-sharding-and-distributed-sql.md",
|
|
49
|
+
"37-consensus-and-distributed-transactions.md",
|
|
50
|
+
"38-search-architecture-elasticsearch-opensearch.md",
|
|
51
|
+
"39-vector-databases-and-hybrid-retrieval.md",
|
|
52
|
+
"40-zero-downtime-database-migrations.md",
|
|
53
|
+
"41-api-gateway-and-edge-architecture.md",
|
|
54
|
+
"42-rest-api-design-and-lifecycle.md",
|
|
55
|
+
"43-graphql-architecture-and-governance.md",
|
|
56
|
+
"44-grpc-and-service-contracts.md",
|
|
57
|
+
"45-websocket-sse-and-realtime-systems.md",
|
|
58
|
+
"46-realtime-collaboration-ot-crdt.md",
|
|
59
|
+
"47-api-security-rate-limiting-and-abuse.md",
|
|
60
|
+
"48-oauth-oidc-and-passkeys.md",
|
|
61
|
+
"49-iam-rbac-abac-rebac.md",
|
|
62
|
+
"50-zero-trust-and-workload-identity.md",
|
|
63
|
+
"51-secrets-kms-pki-and-key-management.md",
|
|
64
|
+
"52-supply-chain-security-sbom-slsa.md",
|
|
65
|
+
"53-threat-modeling-stride-and-attack-trees.md",
|
|
66
|
+
"54-privacy-engineering-and-data-governance.md",
|
|
67
|
+
"55-opentelemetry-and-instrumentation.md",
|
|
68
|
+
"56-distributed-tracing-and-context-propagation.md",
|
|
69
|
+
"57-continuous-profiling-and-performance-diagnostics.md",
|
|
70
|
+
"58-load-stress-soak-testing-and-workload-modeling.md",
|
|
71
|
+
"59-capacity-planning-and-performance-modeling.md",
|
|
72
|
+
"60-chaos-engineering-and-resilience-experiments.md",
|
|
73
|
+
"61-incident-management-and-postmortems.md",
|
|
74
|
+
"62-dns-and-service-discovery.md",
|
|
75
|
+
"63-load-balancing-and-global-traffic-management.md",
|
|
76
|
+
"64-cdn-and-edge-caching.md",
|
|
77
|
+
"65-cloud-networking-and-connectivity.md",
|
|
78
|
+
"66-edge-computing-architecture.md",
|
|
79
|
+
"67-storage-infrastructure-and-storage-systems.md",
|
|
80
|
+
"68-tcp-quic-and-network-performance.md",
|
|
81
|
+
"69-task-routing-and-evidence.md",
|
|
25
82
|
}
|
|
26
83
|
REQUIRED_TEMPLATES = {
|
|
27
84
|
"adr-template.md",
|
|
28
85
|
"architecture-design-template.md",
|
|
29
86
|
"architecture-review-template.md",
|
|
87
|
+
"architecture-decision-matrix-template.md",
|
|
88
|
+
"architecture-review-scorecard-template.md",
|
|
89
|
+
"failure-analysis-template.md",
|
|
90
|
+
"agent-system-design-template.md",
|
|
91
|
+
"agent-security-review-template.md",
|
|
92
|
+
"agent-evaluation-plan-template.md",
|
|
93
|
+
"production-readiness-checklist-template.md",
|
|
94
|
+
"multi-region-disaster-recovery-template.md",
|
|
95
|
+
"cloud-cost-model-template.md",
|
|
96
|
+
"event-streaming-design-template.md",
|
|
97
|
+
"event-sourcing-cqrs-template.md",
|
|
98
|
+
"cdc-data-pipeline-template.md",
|
|
99
|
+
"database-architecture-review-template.md",
|
|
100
|
+
"sharding-distributed-sql-design-template.md",
|
|
101
|
+
"zero-downtime-database-migration-template.md",
|
|
102
|
+
"api-architecture-design-template.md",
|
|
103
|
+
"realtime-system-design-template.md",
|
|
104
|
+
"api-security-and-rate-limit-review-template.md",
|
|
105
|
+
"identity-and-access-architecture-template.md",
|
|
106
|
+
"security-threat-model-template.md",
|
|
107
|
+
"supply-chain-security-review-template.md",
|
|
108
|
+
"privacy-data-governance-template.md",
|
|
109
|
+
"observability-architecture-template.md",
|
|
110
|
+
"performance-test-plan-template.md",
|
|
111
|
+
"chaos-experiment-template.md",
|
|
112
|
+
"incident-postmortem-template.md",
|
|
113
|
+
"network-architecture-design-template.md",
|
|
114
|
+
"cdn-and-edge-strategy-template.md",
|
|
115
|
+
"storage-architecture-design-template.md",
|
|
116
|
+
"edge-computing-readiness-template.md",
|
|
30
117
|
}
|
|
31
118
|
PACKAGE_ENTRIES = (
|
|
32
119
|
"SKILL.md", "references", "templates", "scripts",
|
|
@@ -176,6 +263,43 @@ def validate(root: Path) -> list[str]:
|
|
|
176
263
|
f"does not match VERSION {version!r}"
|
|
177
264
|
)
|
|
178
265
|
|
|
266
|
+
eval_file = root / "evals" / "system-design-cases.json"
|
|
267
|
+
if not eval_file.is_file():
|
|
268
|
+
errors.append("evaluation case file is missing: evals/system-design-cases.json")
|
|
269
|
+
else:
|
|
270
|
+
try:
|
|
271
|
+
evaluation = json.loads(eval_file.read_text(encoding="utf-8"))
|
|
272
|
+
except (OSError, json.JSONDecodeError) as exc:
|
|
273
|
+
errors.append(f"evaluation case file is invalid JSON: {exc}")
|
|
274
|
+
evaluation = {}
|
|
275
|
+
cases = evaluation.get("cases", []) if isinstance(evaluation, dict) else []
|
|
276
|
+
if not isinstance(cases, list) or not cases:
|
|
277
|
+
errors.append("evaluation case file must contain a non-empty 'cases' array")
|
|
278
|
+
else:
|
|
279
|
+
seen_case_ids: set[str] = set()
|
|
280
|
+
for index, case in enumerate(cases):
|
|
281
|
+
prefix = f"evaluation case {index + 1}"
|
|
282
|
+
if not isinstance(case, dict):
|
|
283
|
+
errors.append(f"{prefix} must be an object")
|
|
284
|
+
continue
|
|
285
|
+
case_id = case.get("id")
|
|
286
|
+
if not isinstance(case_id, str) or not case_id.strip():
|
|
287
|
+
errors.append(f"{prefix} must have a non-empty string id")
|
|
288
|
+
elif case_id in seen_case_ids:
|
|
289
|
+
errors.append(f"duplicate evaluation case id: {case_id}")
|
|
290
|
+
else:
|
|
291
|
+
seen_case_ids.add(case_id)
|
|
292
|
+
for field in ("task", "expected_mode", "required_outputs", "guardrails"):
|
|
293
|
+
if not case.get(field):
|
|
294
|
+
errors.append(f"{prefix} is missing required field: {field}")
|
|
295
|
+
references = case.get("references", [])
|
|
296
|
+
if not isinstance(references, list):
|
|
297
|
+
errors.append(f"{prefix} references must be an array")
|
|
298
|
+
else:
|
|
299
|
+
for reference in references:
|
|
300
|
+
if not isinstance(reference, str) or not (refs_dir / reference).is_file():
|
|
301
|
+
errors.append(f"{prefix} references a missing reference: {reference!r}")
|
|
302
|
+
|
|
179
303
|
package_file = root / "package.json"
|
|
180
304
|
if not package_file.is_file():
|
|
181
305
|
errors.append("package.json is missing")
|
|
@@ -196,6 +320,15 @@ def validate(root: Path) -> list[str]:
|
|
|
196
320
|
errors.append(
|
|
197
321
|
f"package.json version {package_version!r} does not match VERSION {version!r}"
|
|
198
322
|
)
|
|
323
|
+
if version_file.is_file() and (root / "README.md").is_file():
|
|
324
|
+
readme_text = (root / "README.md").read_text(encoding="utf-8")
|
|
325
|
+
if not re.search(r"(?m)^- \*\*Version:\*\* " + re.escape(version) + r"$", readme_text):
|
|
326
|
+
errors.append(f"README version must match VERSION {version!r}")
|
|
327
|
+
changelog_file = root / "CHANGELOG.md"
|
|
328
|
+
if changelog_file.is_file():
|
|
329
|
+
first_version = re.search(r"(?m)^##\s*([^ —]+)", changelog_file.read_text(encoding="utf-8"))
|
|
330
|
+
if not first_version or first_version.group(1) != version:
|
|
331
|
+
errors.append(f"latest CHANGELOG version must match VERSION {version!r}")
|
|
199
332
|
repository = package.get("repository", {})
|
|
200
333
|
if not isinstance(repository, dict) or repository.get("url") != (
|
|
201
334
|
"https://github.com/jack-001-a/system-design-primer-skill.git"
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Agent Evaluation Plan: <system / workflow>
|
|
2
|
+
|
|
3
|
+
## Evaluation objective
|
|
4
|
+
- Task classes and intended users:
|
|
5
|
+
- Business success criteria:
|
|
6
|
+
- Prohibited actions / safety invariants:
|
|
7
|
+
- Risk tiers and release scope:
|
|
8
|
+
|
|
9
|
+
## Versioned test assets
|
|
10
|
+
| Asset | Version / source | Owner | Change trigger |
|
|
11
|
+
|---|---|---|---|
|
|
12
|
+
| Model / provider | | | |
|
|
13
|
+
| Prompt / context policy | | | |
|
|
14
|
+
| Tool schemas / MCP servers | | | |
|
|
15
|
+
| Workflow / policy | | | |
|
|
16
|
+
| Dataset / graders | | | |
|
|
17
|
+
|
|
18
|
+
## Dataset coverage
|
|
19
|
+
| Scenario | Expected behavior | Risk class | Deterministic assertion / rubric |
|
|
20
|
+
|---|---|---|---|
|
|
21
|
+
| Normal task | | | |
|
|
22
|
+
| Ambiguous request | | | |
|
|
23
|
+
| Prompt injection | | | |
|
|
24
|
+
| Authorization denial | | | |
|
|
25
|
+
| Invalid / unavailable tool | | | |
|
|
26
|
+
| Duplicate / timeout / partial side effect | | | |
|
|
27
|
+
| Resume / cancellation | | | |
|
|
28
|
+
| Multi-agent disagreement | | | |
|
|
29
|
+
|
|
30
|
+
## Metrics and release gates
|
|
31
|
+
| Dimension | Metric / definition | Baseline | Threshold | Sample size / confidence |
|
|
32
|
+
|---|---|---|---|---|
|
|
33
|
+
| Task completion / correctness | | | | |
|
|
34
|
+
| Groundedness / provenance | | | | |
|
|
35
|
+
| Tool selection / argument validity | | | | |
|
|
36
|
+
| Policy compliance / unauthorized action | | | | |
|
|
37
|
+
| Recovery / duplicate side effects | | | | |
|
|
38
|
+
| Latency / tokens / cost | | | | |
|
|
39
|
+
| Human escalation / override | | | | |
|
|
40
|
+
|
|
41
|
+
Safety-critical failures are independent blockers; do not average them into a composite score.
|
|
42
|
+
|
|
43
|
+
## Test methods
|
|
44
|
+
- Unit / policy / schema tests:
|
|
45
|
+
- Tool and provider contract tests:
|
|
46
|
+
- End-to-end scenario suite:
|
|
47
|
+
- Adversarial / red-team suite:
|
|
48
|
+
- Fault injection / recovery tests:
|
|
49
|
+
- Human review and grader calibration:
|
|
50
|
+
- Staged online / canary evaluation:
|
|
51
|
+
|
|
52
|
+
## Observability and response
|
|
53
|
+
- Correlation IDs and trace fields:
|
|
54
|
+
- Redaction, access, and retention:
|
|
55
|
+
- Alerts and SLOs:
|
|
56
|
+
- Rollback thresholds:
|
|
57
|
+
- Incident runbooks and owners:
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Agent Security Review: <system / agent / tool>
|
|
2
|
+
|
|
3
|
+
## Scope and trust boundaries
|
|
4
|
+
- Host / harness / agent:
|
|
5
|
+
- Tools, MCP servers, providers:
|
|
6
|
+
- Tenants, data classes, and external destinations:
|
|
7
|
+
- Reviewers and date:
|
|
8
|
+
|
|
9
|
+
## Threat register
|
|
10
|
+
| Threat / abuse case | Entry point / precondition | Impact | Existing control | Evidence / test | Gap / remediation |
|
|
11
|
+
|---|---|---|---|---|---|
|
|
12
|
+
| Prompt injection / indirect instruction | | | | | |
|
|
13
|
+
| Confused deputy / privilege escalation | | | | | |
|
|
14
|
+
| Cross-tenant access / data exfiltration | | | | | |
|
|
15
|
+
| SSRF / path traversal / unsafe execution | | | | | |
|
|
16
|
+
| Secret leakage / unsafe telemetry | | | | | |
|
|
17
|
+
| Tool or dependency compromise | | | | | |
|
|
18
|
+
| Resource exhaustion / runaway loops | | | | | |
|
|
19
|
+
|
|
20
|
+
## Identity and permissions
|
|
21
|
+
- User / tenant / agent / tool identities:
|
|
22
|
+
- Per-action authorization enforcement:
|
|
23
|
+
- Credential scope, audience, lifetime, and revocation:
|
|
24
|
+
- Read / draft / commit separation:
|
|
25
|
+
- Approval-bound action parameters and expiry:
|
|
26
|
+
|
|
27
|
+
## Isolation and data protection
|
|
28
|
+
- Worker/sandbox boundary:
|
|
29
|
+
- Filesystem, process, CPU, memory, and network egress:
|
|
30
|
+
- Context minimization and retrieval authorization:
|
|
31
|
+
- Encryption, retention, deletion, and redaction:
|
|
32
|
+
- Audit integrity and access:
|
|
33
|
+
|
|
34
|
+
## Findings
|
|
35
|
+
### [Critical / High / Medium / Low] <finding>
|
|
36
|
+
- Status: Confirmed / Likely / Unknown
|
|
37
|
+
- Evidence:
|
|
38
|
+
- Impact and trigger:
|
|
39
|
+
- Recommendation:
|
|
40
|
+
- Validation / owner:
|
|
41
|
+
|
|
42
|
+
## Release decision
|
|
43
|
+
- Blocking findings:
|
|
44
|
+
- Residual risks accepted by:
|
|
45
|
+
- Required tests:
|
|
46
|
+
- Kill switch / rollback / credential revocation:
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# AI Agent System Design: <system / workflow>
|
|
2
|
+
|
|
3
|
+
## 1. Goals and constraints
|
|
4
|
+
- Business outcome / user journey:
|
|
5
|
+
- Task classes and risk tiers:
|
|
6
|
+
- Functional requirements:
|
|
7
|
+
- Quality, latency, availability, and cost objectives:
|
|
8
|
+
- Data classification / tenant boundaries:
|
|
9
|
+
- Non-goals and hard constraints:
|
|
10
|
+
|
|
11
|
+
## 2. Baseline and architecture choice
|
|
12
|
+
- Deterministic or single-agent baseline:
|
|
13
|
+
- Why multiple agents are/are not justified:
|
|
14
|
+
- Chosen topology: Sequential / Router / Supervisor-worker / Parallel fan-out / Peer / Hierarchical
|
|
15
|
+
- Alternatives and trade-offs:
|
|
16
|
+
|
|
17
|
+
## 3. Agent inventory and contracts
|
|
18
|
+
| Agent | Responsibility / non-goals | Inputs / outputs | Tools / data scopes | Budgets / stop conditions | Owner |
|
|
19
|
+
|---|---|---|---|---|---|
|
|
20
|
+
| | | | | | |
|
|
21
|
+
|
|
22
|
+
## 4. Architecture and execution flow
|
|
23
|
+
```mermaid
|
|
24
|
+
flowchart TD
|
|
25
|
+
User --> Host
|
|
26
|
+
Host --> Orchestrator
|
|
27
|
+
Orchestrator --> AgentA
|
|
28
|
+
AgentA --> ToolGateway
|
|
29
|
+
ToolGateway --> Service
|
|
30
|
+
Orchestrator --> StateStore
|
|
31
|
+
```
|
|
32
|
+
Replace with the actual design. Mark trust boundaries, synchronous/asynchronous edges, and human approval paths.
|
|
33
|
+
|
|
34
|
+
## 5. Harness and tools
|
|
35
|
+
- Model/provider routing and fallback:
|
|
36
|
+
- Context assembly and provenance:
|
|
37
|
+
- Tool registry and schema versions:
|
|
38
|
+
- Authorization and policy decision point:
|
|
39
|
+
- Sandbox, network, filesystem, and resource limits:
|
|
40
|
+
- Deadlines, cancellation, retries, and budgets:
|
|
41
|
+
|
|
42
|
+
## 6. MCP and external integrations
|
|
43
|
+
- Host/client/server inventory:
|
|
44
|
+
- Capabilities (Tools / Resources / Prompts):
|
|
45
|
+
- Transport and identity:
|
|
46
|
+
- Per-operation authorization:
|
|
47
|
+
- Version compatibility and lifecycle:
|
|
48
|
+
|
|
49
|
+
## 7. State, memory, and recovery
|
|
50
|
+
- Ephemeral context vs history vs memory vs workflow state:
|
|
51
|
+
- Source of truth and ownership:
|
|
52
|
+
- Checkpoint and resume protocol:
|
|
53
|
+
- Idempotency / reconciliation:
|
|
54
|
+
- Retention, deletion, and tenant isolation:
|
|
55
|
+
|
|
56
|
+
## 8. Security and human oversight
|
|
57
|
+
- Threat model:
|
|
58
|
+
- Least-privilege scopes:
|
|
59
|
+
- High-impact approval gates:
|
|
60
|
+
- Audit and redaction:
|
|
61
|
+
- Incident revocation / kill switch:
|
|
62
|
+
|
|
63
|
+
## 9. Evaluation and observability
|
|
64
|
+
- Versioned evaluation dataset:
|
|
65
|
+
- Task success and correctness:
|
|
66
|
+
- Safety/policy release gates:
|
|
67
|
+
- Latency, tokens, cost, and human intervention:
|
|
68
|
+
- Tracing and alerting:
|
|
69
|
+
|
|
70
|
+
## 10. Failure modes and rollout
|
|
71
|
+
- Provider/tool/worker failures:
|
|
72
|
+
- Partial completion and ambiguous side effects:
|
|
73
|
+
- Canary and rollback:
|
|
74
|
+
- Operational owner and runbooks:
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# API Architecture Design
|
|
2
|
+
|
|
3
|
+
## Context and goals
|
|
4
|
+
- Product journeys:
|
|
5
|
+
- Consumers and trust boundaries:
|
|
6
|
+
- Functional and non-functional requirements:
|
|
7
|
+
- Explicit non-goals:
|
|
8
|
+
|
|
9
|
+
## Contract and topology
|
|
10
|
+
- Protocols (REST / GraphQL / gRPC / async):
|
|
11
|
+
- Gateway, ingress, BFF, and service ownership:
|
|
12
|
+
- OpenAPI / GraphQL schema / Protobuf links:
|
|
13
|
+
- Authentication and authorization model:
|
|
14
|
+
- Versioning and deprecation policy:
|
|
15
|
+
|
|
16
|
+
## Capacity and resilience
|
|
17
|
+
- Average/peak RPS, concurrency, payload sizes:
|
|
18
|
+
- P50/P95/P99 targets and SLO:
|
|
19
|
+
- Deadlines, retry budget, idempotency:
|
|
20
|
+
- Rate limits, quotas, admission control:
|
|
21
|
+
- Failure modes and degraded behavior:
|
|
22
|
+
|
|
23
|
+
## Security and observability
|
|
24
|
+
- Threat model and tenant isolation:
|
|
25
|
+
- Input, output, and data classification:
|
|
26
|
+
- Metrics, logs, traces, audit events:
|
|
27
|
+
- Abuse detection and incident response:
|
|
28
|
+
|
|
29
|
+
## Rollout and validation
|
|
30
|
+
- Compatibility and contract tests:
|
|
31
|
+
- Canary, rollback, and migration plan:
|
|
32
|
+
- Load, fault-injection, and security tests:
|
|
33
|
+
- Owners, open risks, and decision record:
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# API Security and Rate-Limit Review
|
|
2
|
+
|
|
3
|
+
## Scope and assets
|
|
4
|
+
- APIs, routes, methods, and data classes:
|
|
5
|
+
- Principals, tenants, and trust boundaries:
|
|
6
|
+
- Critical business actions and abuse impact:
|
|
7
|
+
|
|
8
|
+
## Authorization
|
|
9
|
+
- Authentication and token validation:
|
|
10
|
+
- Object-level and function-level authorization:
|
|
11
|
+
- Tenant scoping and cache-key isolation:
|
|
12
|
+
- Service identity and secret lifecycle:
|
|
13
|
+
|
|
14
|
+
## Abuse controls
|
|
15
|
+
- Rate-limit keys and dimensions:
|
|
16
|
+
- Algorithm, burst, concurrency, and cost units:
|
|
17
|
+
- Distributed enforcement consistency:
|
|
18
|
+
- 429/Retry-After and client retry policy:
|
|
19
|
+
- Limiter outage and emergency fallback:
|
|
20
|
+
|
|
21
|
+
## Detection and response
|
|
22
|
+
- Metrics, alerts, audit, and investigation:
|
|
23
|
+
- False-positive handling and overrides:
|
|
24
|
+
- Load shedding and noisy-neighbor controls:
|
|
25
|
+
|
|
26
|
+
## Verification
|
|
27
|
+
- BOLA/BFLA, replay, enumeration, and bypass tests:
|
|
28
|
+
- Burst, hot-key, clock-skew, and dependency-failure tests:
|
|
29
|
+
- Findings, severity, owners, due dates:
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Architecture Decision Matrix: <decision>
|
|
2
|
+
|
|
3
|
+
## Decision context
|
|
4
|
+
- Business outcome:
|
|
5
|
+
- Scope / non-goals:
|
|
6
|
+
- Decision owner:
|
|
7
|
+
- Date:
|
|
8
|
+
- Hard constraints (gates):
|
|
9
|
+
- Workload evidence and assumptions:
|
|
10
|
+
|
|
11
|
+
## Candidate options
|
|
12
|
+
| Option | Summary | Meets hard constraints? | Key dependencies |
|
|
13
|
+
|---|---|---|---|
|
|
14
|
+
| Simplest viable baseline | | | |
|
|
15
|
+
| Alternative A | | | |
|
|
16
|
+
| Alternative B | | | |
|
|
17
|
+
|
|
18
|
+
## Comparative assessment
|
|
19
|
+
Use Strong / Adequate / Weak / Unknown, or a defined numeric scale. Attach evidence and confidence. Do not treat unknown as a favorable score.
|
|
20
|
+
|
|
21
|
+
| Criterion | Priority / weight (source) | Baseline | Option A | Option B | Evidence / confidence |
|
|
22
|
+
|---|---|---|---|---|---|
|
|
23
|
+
| Performance and scalability | | | | | |
|
|
24
|
+
| Consistency and data integrity | | | | | |
|
|
25
|
+
| Reliability and availability | | | | | |
|
|
26
|
+
| Security and compliance | | | | | |
|
|
27
|
+
| Total cost of ownership | | | | | |
|
|
28
|
+
| Operational complexity | | | | | |
|
|
29
|
+
| Team capability | | | | | |
|
|
30
|
+
| Migration and exit cost | | | | | |
|
|
31
|
+
| Ecosystem and delivery risk | | | | | |
|
|
32
|
+
|
|
33
|
+
## Sensitivity and uncertainty
|
|
34
|
+
- Assumptions that could reverse the decision:
|
|
35
|
+
- Evidence gaps and cheapest validation experiment:
|
|
36
|
+
- Cost / capacity ranges:
|
|
37
|
+
|
|
38
|
+
## Recommendation
|
|
39
|
+
- Selected option:
|
|
40
|
+
- Why it is proportionate:
|
|
41
|
+
- Rejected alternatives and reasons:
|
|
42
|
+
- Risks and mitigations:
|
|
43
|
+
- Migration / rollback:
|
|
44
|
+
- Measurable revisit triggers:
|
|
45
|
+
|
|
46
|
+
## Approval
|
|
47
|
+
- Stakeholders:
|
|
48
|
+
- Decision status:
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Architecture Review Scorecard: <system>
|
|
2
|
+
|
|
3
|
+
## Review scope
|
|
4
|
+
- Components / paths inspected:
|
|
5
|
+
- Evidence sources:
|
|
6
|
+
- Exclusions and unknowns:
|
|
7
|
+
- Review date / reviewers:
|
|
8
|
+
|
|
9
|
+
## Dimension assessment
|
|
10
|
+
Use Pass / Needs work / Fail / Not assessed. Each assessment must cite evidence. Do not calculate a single average health percentage.
|
|
11
|
+
|
|
12
|
+
| Dimension | Assessment | Evidence | Business impact / risk | Required action |
|
|
13
|
+
|---|---|---|---|---|
|
|
14
|
+
| Architecture and boundaries | | | | |
|
|
15
|
+
| Data integrity | | | | |
|
|
16
|
+
| Scalability and performance | | | | |
|
|
17
|
+
| Reliability and recovery | | | | |
|
|
18
|
+
| Security and privacy | | | | |
|
|
19
|
+
| Observability and operations | | | | |
|
|
20
|
+
| Maintainability and delivery | | | | |
|
|
21
|
+
| Cost efficiency | | | | |
|
|
22
|
+
|
|
23
|
+
## Findings
|
|
24
|
+
### [Critical / High / Medium / Low] <finding>
|
|
25
|
+
- Status: Confirmed / Likely / Unknown
|
|
26
|
+
- Evidence: path, symbol, metric, trace, test, or reproduction
|
|
27
|
+
- Trigger / likelihood:
|
|
28
|
+
- Impact:
|
|
29
|
+
- Recommendation:
|
|
30
|
+
- Validation:
|
|
31
|
+
|
|
32
|
+
## Executive assessment
|
|
33
|
+
- Material risks:
|
|
34
|
+
- Strengths:
|
|
35
|
+
- Unverified assumptions:
|
|
36
|
+
- Priority sequence:
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# CDC and Data Pipeline Design
|
|
2
|
+
|
|
3
|
+
## Source and capture
|
|
4
|
+
- Source database / version:
|
|
5
|
+
- Connector and version:
|
|
6
|
+
- Log mechanism and retention window:
|
|
7
|
+
- Snapshot mode and consistency boundary:
|
|
8
|
+
- Captured tables / columns:
|
|
9
|
+
- Sensitive data excluded or masked:
|
|
10
|
+
|
|
11
|
+
## Delivery and semantics
|
|
12
|
+
- Broker / destination:
|
|
13
|
+
- Ordering guarantees and scope:
|
|
14
|
+
- Transaction metadata requirements:
|
|
15
|
+
- Duplicate handling and idempotency:
|
|
16
|
+
- Delete / tombstone semantics:
|
|
17
|
+
- Schema evolution and type mapping:
|
|
18
|
+
|
|
19
|
+
## Consumers and quality
|
|
20
|
+
| Consumer | Purpose | Expected freshness | Idempotency key | Data quality checks |
|
|
21
|
+
|---|---|---|---|---|
|
|
22
|
+
| | | | | |
|
|
23
|
+
|
|
24
|
+
- Lineage and data ownership:
|
|
25
|
+
- Quarantine / DLQ and replay process:
|
|
26
|
+
- Row counts, checksums, invariant validation:
|
|
27
|
+
|
|
28
|
+
## Capacity and operations
|
|
29
|
+
- Snapshot volume and throttling:
|
|
30
|
+
- Steady-state change rate:
|
|
31
|
+
- Source DB impact and log lag:
|
|
32
|
+
- Connector offset / schema history storage:
|
|
33
|
+
- Alerts and SLOs:
|
|
34
|
+
- Outage, log truncation, and recovery procedure:
|
|
35
|
+
|
|
36
|
+
## Migration / cutover
|
|
37
|
+
- Backfill and CDC catch-up:
|
|
38
|
+
- Write ownership and dual-write avoidance:
|
|
39
|
+
- Consistency evidence:
|
|
40
|
+
- Cutover gates and rollback:
|
|
41
|
+
- Old-source retirement criteria:
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# CDN and Edge Strategy
|
|
2
|
+
|
|
3
|
+
## 1. Content and traffic profile
|
|
4
|
+
- Content classes and owners:
|
|
5
|
+
- Geographic distribution and request/byte volume:
|
|
6
|
+
- Dynamic/static ratio and latency objectives:
|
|
7
|
+
- Sensitive or personalized response classes:
|
|
8
|
+
|
|
9
|
+
## 2. Cache policy
|
|
10
|
+
| Content class | Cache key dimensions | TTL/freshness | Stale policy | Shared cache allowed? |
|
|
11
|
+
|---|---|---|---|---|
|
|
12
|
+
| | | | | |
|
|
13
|
+
|
|
14
|
+
## 3. Origin architecture
|
|
15
|
+
- Origin locations and shielding:
|
|
16
|
+
- Origin authentication and bypass prevention:
|
|
17
|
+
- Failover and overload behavior:
|
|
18
|
+
- Cache miss/stampede controls:
|
|
19
|
+
|
|
20
|
+
## 4. Invalidation and consistency
|
|
21
|
+
- Versioned URL/key strategy:
|
|
22
|
+
- Purge/tag mechanism and propagation expectations:
|
|
23
|
+
- Retry, ordering, and verification:
|
|
24
|
+
- Deletion and privacy requirements:
|
|
25
|
+
|
|
26
|
+
## 5. Edge execution and security
|
|
27
|
+
- Edge compute use cases and runtime constraints:
|
|
28
|
+
- Authentication/authorization boundaries:
|
|
29
|
+
- WAF, bot, DDoS, TLS, and rate-limit ownership:
|
|
30
|
+
- Cache poisoning/deception and tenant isolation controls:
|
|
31
|
+
|
|
32
|
+
## 6. Economics and SLOs
|
|
33
|
+
- Hit ratio by requests and bytes:
|
|
34
|
+
- Origin offload and egress:
|
|
35
|
+
- Request, purge, compute, and storage costs:
|
|
36
|
+
- Latency, availability, stale-serving, and origin SLOs:
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Chaos Engineering Experiment
|
|
2
|
+
|
|
3
|
+
## 1. Hypothesis
|
|
4
|
+
- Steady state (user-visible):
|
|
5
|
+
- Resilience hypothesis:
|
|
6
|
+
- Expected behavior under fault:
|
|
7
|
+
|
|
8
|
+
## 2. Scope and authorization
|
|
9
|
+
- Environment/cohort/tenant:
|
|
10
|
+
- Target and dependencies:
|
|
11
|
+
- Explicit authorization/change window:
|
|
12
|
+
- Exclusions and blast-radius limit:
|
|
13
|
+
|
|
14
|
+
## 3. Fault injection
|
|
15
|
+
- Fault class and mechanism:
|
|
16
|
+
- Intensity, duration, and ramp:
|
|
17
|
+
- Prerequisites and data-safety constraints:
|
|
18
|
+
|
|
19
|
+
## 4. Observability and abort
|
|
20
|
+
- SLO/SLI and integrity signals:
|
|
21
|
+
- Observer and incident commander:
|
|
22
|
+
- Abort thresholds:
|
|
23
|
+
- Kill switch and recovery procedure:
|
|
24
|
+
|
|
25
|
+
## 5. Execution record
|
|
26
|
+
| Timestamp | Action/observation | Signal/evidence | Decision |
|
|
27
|
+
|---|---|---|---|
|
|
28
|
+
| | | | |
|
|
29
|
+
|
|
30
|
+
## 6. Outcome
|
|
31
|
+
- Detection and containment:
|
|
32
|
+
- User impact/degradation:
|
|
33
|
+
- Recovery and reconciliation:
|
|
34
|
+
- Hypothesis: validated / invalidated / inconclusive
|
|
35
|
+
|
|
36
|
+
## 7. Remediation
|
|
37
|
+
| Finding | Risk reduction | Owner | Due date | Retest |
|
|
38
|
+
|---|---|---|---|---|
|
|
39
|
+
| | | | | |
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# Cloud Cost and FinOps Model
|
|
2
|
+
|
|
3
|
+
## Scope and assumptions
|
|
4
|
+
- Workload / service / environment:
|
|
5
|
+
- Forecast period and currency:
|
|
6
|
+
- Expected, peak, and degraded demand:
|
|
7
|
+
- Data growth and retention:
|
|
8
|
+
- Regions and availability topology:
|
|
9
|
+
|
|
10
|
+
## Cost drivers
|
|
11
|
+
| Category | Unit / rate source | Expected quantity | Peak quantity | Monthly estimate | Owner / allocation |
|
|
12
|
+
|---|---|---:|---:|---:|---|
|
|
13
|
+
| Compute | | | | | |
|
|
14
|
+
| Storage / IOPS / backups | | | | | |
|
|
15
|
+
| Database / cache | | | | | |
|
|
16
|
+
| Network / egress / inter-region | | | | | |
|
|
17
|
+
| Load balancing / gateways | | | | | |
|
|
18
|
+
| Observability / retention | | | | | |
|
|
19
|
+
| Licenses / support | | | | | |
|
|
20
|
+
| Platform operations | | | | | |
|
|
21
|
+
|
|
22
|
+
## Unit economics
|
|
23
|
+
- Cost per request / transaction / active user:
|
|
24
|
+
- Cost per unit of business value:
|
|
25
|
+
- Shared-cost allocation method:
|
|
26
|
+
- Unallocated spend and confidence:
|
|
27
|
+
|
|
28
|
+
## Optimization hypotheses
|
|
29
|
+
| Change | Expected savings range | SLO / security risk | Validation method | Owner |
|
|
30
|
+
|---|---|---|---|---|
|
|
31
|
+
| | | | | |
|
|
32
|
+
|
|
33
|
+
## Commitments and governance
|
|
34
|
+
- Reserved / committed capacity assumptions:
|
|
35
|
+
- Break-even utilization and lock-in risk:
|
|
36
|
+
- Budget owner, alerts, and approval thresholds:
|
|
37
|
+
- Forecast confidence and review cadence:
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Database Architecture Review
|
|
2
|
+
|
|
3
|
+
## Context
|
|
4
|
+
- System / service:
|
|
5
|
+
- Database engine and exact version:
|
|
6
|
+
- Hosting / topology / regions:
|
|
7
|
+
- Workload (read/write, peak QPS, concurrency):
|
|
8
|
+
- Data volume, growth, retention:
|
|
9
|
+
- Critical SLOs and business invariants:
|
|
10
|
+
|
|
11
|
+
## Data model and access
|
|
12
|
+
- Source of truth and ownership:
|
|
13
|
+
- Primary/unique keys:
|
|
14
|
+
- Indexes and query patterns:
|
|
15
|
+
- Transaction boundaries and isolation:
|
|
16
|
+
- Hot keys, skew, and contention:
|
|
17
|
+
|
|
18
|
+
## Reliability and recovery
|
|
19
|
+
- Replication mode and lag:
|
|
20
|
+
- Backup/PITR and last restore exercise:
|
|
21
|
+
- RPO/RTO:
|
|
22
|
+
- Failover/fencing and failback:
|
|
23
|
+
- Upgrade and extension compatibility:
|
|
24
|
+
|
|
25
|
+
## Evidence
|
|
26
|
+
| Finding | Evidence / query plan / metric | Impact | Severity | Validation |
|
|
27
|
+
|---|---|---|---|---|
|
|
28
|
+
|
|
29
|
+
## Recommendations
|
|
30
|
+
| Change | Expected outcome | Risk / cost | Rollback | Success metric |
|
|
31
|
+
|---|---|---|---|---|
|
|
32
|
+
|
|
33
|
+
## Open questions and assumptions
|