system-design-primer-skill 1.2.0 → 2.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +114 -60
- package/SKILL.md +437 -93
- package/VERSION +1 -1
- package/evals/README.md +1022 -0
- package/evals/system-design-cases.json +414 -0
- package/package.json +46 -4
- package/references/03-data-and-storage.md +8 -2
- package/references/04-caching.md +3 -3
- package/references/05-async-and-messaging.md +4 -0
- package/references/06-distributed-systems.md +2 -2
- package/references/07-reliability-and-resilience.md +7 -3
- package/references/08-api-security-observability.md +4 -0
- package/references/10-workflows-and-checklists.md +1 -1
- package/references/13-architecture-intelligence.md +91 -0
- package/references/14-api-and-data-contracts.md +80 -0
- package/references/15-failure-analysis.md +88 -0
- package/references/16-multi-agent-architecture.md +76 -0
- package/references/17-agent-harness-and-runtime.md +87 -0
- package/references/18-mcp-architecture.md +67 -0
- package/references/19-agent-security-and-permissions.md +90 -0
- package/references/20-agent-state-and-memory.md +76 -0
- package/references/21-agent-evaluation-and-observability.md +90 -0
- package/references/22-kubernetes-and-container-platforms.md +29 -0
- package/references/23-service-mesh-and-traffic-management.md +21 -0
- package/references/24-multi-region-and-disaster-recovery.md +28 -0
- package/references/25-platform-engineering-and-developer-experience.md +26 -0
- package/references/26-finops-and-cloud-cost.md +29 -0
- package/references/27-production-observability-and-slo.md +33 -0
- package/references/28-event-streaming-and-kafka-pulsar.md +33 -0
- package/references/29-event-sourcing-and-cqrs.md +23 -0
- package/references/30-change-data-capture-and-debezium.md +27 -0
- package/references/31-data-lakehouse-and-stream-batch.md +24 -0
- package/references/32-workflow-orchestration-and-temporal.md +28 -0
- package/references/33-schema-registry-and-event-governance.md +22 -0
- package/references/34-postgresql-internals-and-performance.md +45 -0
- package/references/35-mysql-innodb-and-performance.md +29 -0
- package/references/36-sharding-and-distributed-sql.md +22 -0
- package/references/37-consensus-and-distributed-transactions.md +29 -0
- package/references/38-search-architecture-elasticsearch-opensearch.md +24 -0
- package/references/39-vector-databases-and-hybrid-retrieval.md +31 -0
- package/references/40-zero-downtime-database-migrations.md +27 -0
- package/references/41-api-gateway-and-edge-architecture.md +23 -0
- package/references/42-rest-api-design-and-lifecycle.md +23 -0
- package/references/43-graphql-architecture-and-governance.md +16 -0
- package/references/44-grpc-and-service-contracts.md +16 -0
- package/references/45-websocket-sse-and-realtime-systems.md +16 -0
- package/references/46-realtime-collaboration-ot-crdt.md +16 -0
- package/references/47-api-security-rate-limiting-and-abuse.md +16 -0
- package/references/48-oauth-oidc-and-passkeys.md +20 -0
- package/references/49-iam-rbac-abac-rebac.md +20 -0
- package/references/50-zero-trust-and-workload-identity.md +16 -0
- package/references/51-secrets-kms-pki-and-key-management.md +20 -0
- package/references/52-supply-chain-security-sbom-slsa.md +78 -0
- package/references/53-threat-modeling-stride-and-attack-trees.md +16 -0
- package/references/54-privacy-engineering-and-data-governance.md +20 -0
- package/references/55-opentelemetry-and-instrumentation.md +54 -0
- package/references/56-distributed-tracing-and-context-propagation.md +47 -0
- package/references/57-continuous-profiling-and-performance-diagnostics.md +45 -0
- package/references/58-load-stress-soak-testing-and-workload-modeling.md +44 -0
- package/references/59-capacity-planning-and-performance-modeling.md +38 -0
- package/references/60-chaos-engineering-and-resilience-experiments.md +45 -0
- package/references/61-incident-management-and-postmortems.md +38 -0
- package/references/62-dns-and-service-discovery.md +35 -0
- package/references/63-load-balancing-and-global-traffic-management.md +31 -0
- package/references/64-cdn-and-edge-caching.md +34 -0
- package/references/65-cloud-networking-and-connectivity.md +34 -0
- package/references/66-edge-computing-architecture.md +33 -0
- package/references/67-storage-infrastructure-and-storage-systems.md +34 -0
- package/references/68-tcp-quic-and-network-performance.md +27 -0
- package/references/69-task-routing-and-evidence.md +115 -0
- package/scripts/audit_chain.py +36 -0
- package/scripts/compare_reports.py +43 -0
- package/scripts/evaluate.py +475 -0
- package/scripts/evaluation_catalog.py +129 -0
- package/scripts/evaluation_fingerprints.py +20 -0
- package/scripts/evidence_archive.py +79 -0
- package/scripts/evidence_signing.py +98 -0
- package/scripts/evidence_timestamp.py +441 -0
- package/scripts/install.ps1 +12 -3
- package/scripts/install.sh +15 -8
- package/scripts/report_integrity.py +184 -0
- package/scripts/review_consensus.py +77 -0
- package/scripts/review_evidence.py +105 -0
- package/scripts/review_provenance.py +44 -0
- package/scripts/reviewer_calibration.py +215 -0
- package/scripts/rubric_compatibility.py +134 -0
- package/scripts/run_evals.py +130 -0
- package/scripts/run_manifest.py +179 -0
- package/scripts/statistical_analysis.py +308 -0
- package/scripts/tsa_attestation.py +709 -0
- package/scripts/tsa_validation_report.py +627 -0
- package/scripts/validate.py +356 -11
- package/scripts/verify_build_provenance.py +625 -0
- package/scripts/verify_tsa_ocsp.py +962 -0
- package/scripts/verify_tsa_revocation.py +782 -0
- package/scripts/windows_broker_protocol.py +274 -0
- package/scripts/windows_broker_transport.py +141 -0
- package/scripts/windows_broker_win32.py +713 -0
- package/templates/agent-evaluation-plan-template.md +57 -0
- package/templates/agent-security-review-template.md +46 -0
- package/templates/agent-system-design-template.md +74 -0
- package/templates/api-architecture-design-template.md +33 -0
- package/templates/api-security-and-rate-limit-review-template.md +29 -0
- package/templates/architecture-decision-matrix-template.md +48 -0
- package/templates/architecture-review-scorecard-template.md +36 -0
- package/templates/cdc-data-pipeline-template.md +41 -0
- package/templates/cdn-and-edge-strategy-template.md +36 -0
- package/templates/chaos-experiment-template.md +39 -0
- package/templates/cloud-cost-model-template.md +37 -0
- package/templates/database-architecture-review-template.md +33 -0
- package/templates/edge-computing-readiness-template.md +36 -0
- package/templates/event-sourcing-cqrs-template.md +41 -0
- package/templates/event-streaming-design-template.md +47 -0
- package/templates/failure-analysis-template.md +36 -0
- package/templates/identity-and-access-architecture-template.md +29 -0
- package/templates/incident-postmortem-template.md +39 -0
- package/templates/multi-region-disaster-recovery-template.md +43 -0
- package/templates/network-architecture-design-template.md +38 -0
- package/templates/observability-architecture-template.md +52 -0
- package/templates/performance-test-plan-template.md +52 -0
- package/templates/privacy-data-governance-template.md +24 -0
- package/templates/production-readiness-checklist-template.md +52 -0
- package/templates/realtime-system-design-template.md +28 -0
- package/templates/security-threat-model-template.md +26 -0
- package/templates/sharding-distributed-sql-design-template.md +34 -0
- package/templates/storage-architecture-design-template.md +35 -0
- package/templates/supply-chain-security-review-template.md +20 -0
- package/templates/zero-downtime-database-migration-template.md +32 -0
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Agent Evaluation Plan: <system / workflow>
|
|
2
|
+
|
|
3
|
+
## Evaluation objective
|
|
4
|
+
- Task classes and intended users:
|
|
5
|
+
- Business success criteria:
|
|
6
|
+
- Prohibited actions / safety invariants:
|
|
7
|
+
- Risk tiers and release scope:
|
|
8
|
+
|
|
9
|
+
## Versioned test assets
|
|
10
|
+
| Asset | Version / source | Owner | Change trigger |
|
|
11
|
+
|---|---|---|---|
|
|
12
|
+
| Model / provider | | | |
|
|
13
|
+
| Prompt / context policy | | | |
|
|
14
|
+
| Tool schemas / MCP servers | | | |
|
|
15
|
+
| Workflow / policy | | | |
|
|
16
|
+
| Dataset / graders | | | |
|
|
17
|
+
|
|
18
|
+
## Dataset coverage
|
|
19
|
+
| Scenario | Expected behavior | Risk class | Deterministic assertion / rubric |
|
|
20
|
+
|---|---|---|---|
|
|
21
|
+
| Normal task | | | |
|
|
22
|
+
| Ambiguous request | | | |
|
|
23
|
+
| Prompt injection | | | |
|
|
24
|
+
| Authorization denial | | | |
|
|
25
|
+
| Invalid / unavailable tool | | | |
|
|
26
|
+
| Duplicate / timeout / partial side effect | | | |
|
|
27
|
+
| Resume / cancellation | | | |
|
|
28
|
+
| Multi-agent disagreement | | | |
|
|
29
|
+
|
|
30
|
+
## Metrics and release gates
|
|
31
|
+
| Dimension | Metric / definition | Baseline | Threshold | Sample size / confidence |
|
|
32
|
+
|---|---|---|---|---|
|
|
33
|
+
| Task completion / correctness | | | | |
|
|
34
|
+
| Groundedness / provenance | | | | |
|
|
35
|
+
| Tool selection / argument validity | | | | |
|
|
36
|
+
| Policy compliance / unauthorized action | | | | |
|
|
37
|
+
| Recovery / duplicate side effects | | | | |
|
|
38
|
+
| Latency / tokens / cost | | | | |
|
|
39
|
+
| Human escalation / override | | | | |
|
|
40
|
+
|
|
41
|
+
Safety-critical failures are independent blockers; do not average them into a composite score.
|
|
42
|
+
|
|
43
|
+
## Test methods
|
|
44
|
+
- Unit / policy / schema tests:
|
|
45
|
+
- Tool and provider contract tests:
|
|
46
|
+
- End-to-end scenario suite:
|
|
47
|
+
- Adversarial / red-team suite:
|
|
48
|
+
- Fault injection / recovery tests:
|
|
49
|
+
- Human review and grader calibration:
|
|
50
|
+
- Staged online / canary evaluation:
|
|
51
|
+
|
|
52
|
+
## Observability and response
|
|
53
|
+
- Correlation IDs and trace fields:
|
|
54
|
+
- Redaction, access, and retention:
|
|
55
|
+
- Alerts and SLOs:
|
|
56
|
+
- Rollback thresholds:
|
|
57
|
+
- Incident runbooks and owners:
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Agent Security Review: <system / agent / tool>
|
|
2
|
+
|
|
3
|
+
## Scope and trust boundaries
|
|
4
|
+
- Host / harness / agent:
|
|
5
|
+
- Tools, MCP servers, providers:
|
|
6
|
+
- Tenants, data classes, and external destinations:
|
|
7
|
+
- Reviewers and date:
|
|
8
|
+
|
|
9
|
+
## Threat register
|
|
10
|
+
| Threat / abuse case | Entry point / precondition | Impact | Existing control | Evidence / test | Gap / remediation |
|
|
11
|
+
|---|---|---|---|---|---|
|
|
12
|
+
| Prompt injection / indirect instruction | | | | | |
|
|
13
|
+
| Confused deputy / privilege escalation | | | | | |
|
|
14
|
+
| Cross-tenant access / data exfiltration | | | | | |
|
|
15
|
+
| SSRF / path traversal / unsafe execution | | | | | |
|
|
16
|
+
| Secret leakage / unsafe telemetry | | | | | |
|
|
17
|
+
| Tool or dependency compromise | | | | | |
|
|
18
|
+
| Resource exhaustion / runaway loops | | | | | |
|
|
19
|
+
|
|
20
|
+
## Identity and permissions
|
|
21
|
+
- User / tenant / agent / tool identities:
|
|
22
|
+
- Per-action authorization enforcement:
|
|
23
|
+
- Credential scope, audience, lifetime, and revocation:
|
|
24
|
+
- Read / draft / commit separation:
|
|
25
|
+
- Approval-bound action parameters and expiry:
|
|
26
|
+
|
|
27
|
+
## Isolation and data protection
|
|
28
|
+
- Worker/sandbox boundary:
|
|
29
|
+
- Filesystem, process, CPU, memory, and network egress:
|
|
30
|
+
- Context minimization and retrieval authorization:
|
|
31
|
+
- Encryption, retention, deletion, and redaction:
|
|
32
|
+
- Audit integrity and access:
|
|
33
|
+
|
|
34
|
+
## Findings
|
|
35
|
+
### [Critical / High / Medium / Low] <finding>
|
|
36
|
+
- Status: Confirmed / Likely / Unknown
|
|
37
|
+
- Evidence:
|
|
38
|
+
- Impact and trigger:
|
|
39
|
+
- Recommendation:
|
|
40
|
+
- Validation / owner:
|
|
41
|
+
|
|
42
|
+
## Release decision
|
|
43
|
+
- Blocking findings:
|
|
44
|
+
- Residual risks accepted by:
|
|
45
|
+
- Required tests:
|
|
46
|
+
- Kill switch / rollback / credential revocation:
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# AI Agent System Design: <system / workflow>
|
|
2
|
+
|
|
3
|
+
## 1. Goals and constraints
|
|
4
|
+
- Business outcome / user journey:
|
|
5
|
+
- Task classes and risk tiers:
|
|
6
|
+
- Functional requirements:
|
|
7
|
+
- Quality, latency, availability, and cost objectives:
|
|
8
|
+
- Data classification / tenant boundaries:
|
|
9
|
+
- Non-goals and hard constraints:
|
|
10
|
+
|
|
11
|
+
## 2. Baseline and architecture choice
|
|
12
|
+
- Deterministic or single-agent baseline:
|
|
13
|
+
- Why multiple agents are/are not justified:
|
|
14
|
+
- Chosen topology: Sequential / Router / Supervisor-worker / Parallel fan-out / Peer / Hierarchical
|
|
15
|
+
- Alternatives and trade-offs:
|
|
16
|
+
|
|
17
|
+
## 3. Agent inventory and contracts
|
|
18
|
+
| Agent | Responsibility / non-goals | Inputs / outputs | Tools / data scopes | Budgets / stop conditions | Owner |
|
|
19
|
+
|---|---|---|---|---|---|
|
|
20
|
+
| | | | | | |
|
|
21
|
+
|
|
22
|
+
## 4. Architecture and execution flow
|
|
23
|
+
```mermaid
|
|
24
|
+
flowchart TD
|
|
25
|
+
User --> Host
|
|
26
|
+
Host --> Orchestrator
|
|
27
|
+
Orchestrator --> AgentA
|
|
28
|
+
AgentA --> ToolGateway
|
|
29
|
+
ToolGateway --> Service
|
|
30
|
+
Orchestrator --> StateStore
|
|
31
|
+
```
|
|
32
|
+
Replace with the actual design. Mark trust boundaries, synchronous/asynchronous edges, and human approval paths.
|
|
33
|
+
|
|
34
|
+
## 5. Harness and tools
|
|
35
|
+
- Model/provider routing and fallback:
|
|
36
|
+
- Context assembly and provenance:
|
|
37
|
+
- Tool registry and schema versions:
|
|
38
|
+
- Authorization and policy decision point:
|
|
39
|
+
- Sandbox, network, filesystem, and resource limits:
|
|
40
|
+
- Deadlines, cancellation, retries, and budgets:
|
|
41
|
+
|
|
42
|
+
## 6. MCP and external integrations
|
|
43
|
+
- Host/client/server inventory:
|
|
44
|
+
- Capabilities (Tools / Resources / Prompts):
|
|
45
|
+
- Transport and identity:
|
|
46
|
+
- Per-operation authorization:
|
|
47
|
+
- Version compatibility and lifecycle:
|
|
48
|
+
|
|
49
|
+
## 7. State, memory, and recovery
|
|
50
|
+
- Ephemeral context vs history vs memory vs workflow state:
|
|
51
|
+
- Source of truth and ownership:
|
|
52
|
+
- Checkpoint and resume protocol:
|
|
53
|
+
- Idempotency / reconciliation:
|
|
54
|
+
- Retention, deletion, and tenant isolation:
|
|
55
|
+
|
|
56
|
+
## 8. Security and human oversight
|
|
57
|
+
- Threat model:
|
|
58
|
+
- Least-privilege scopes:
|
|
59
|
+
- High-impact approval gates:
|
|
60
|
+
- Audit and redaction:
|
|
61
|
+
- Incident revocation / kill switch:
|
|
62
|
+
|
|
63
|
+
## 9. Evaluation and observability
|
|
64
|
+
- Versioned evaluation dataset:
|
|
65
|
+
- Task success and correctness:
|
|
66
|
+
- Safety/policy release gates:
|
|
67
|
+
- Latency, tokens, cost, and human intervention:
|
|
68
|
+
- Tracing and alerting:
|
|
69
|
+
|
|
70
|
+
## 10. Failure modes and rollout
|
|
71
|
+
- Provider/tool/worker failures:
|
|
72
|
+
- Partial completion and ambiguous side effects:
|
|
73
|
+
- Canary and rollback:
|
|
74
|
+
- Operational owner and runbooks:
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# API Architecture Design
|
|
2
|
+
|
|
3
|
+
## Context and goals
|
|
4
|
+
- Product journeys:
|
|
5
|
+
- Consumers and trust boundaries:
|
|
6
|
+
- Functional and non-functional requirements:
|
|
7
|
+
- Explicit non-goals:
|
|
8
|
+
|
|
9
|
+
## Contract and topology
|
|
10
|
+
- Protocols (REST / GraphQL / gRPC / async):
|
|
11
|
+
- Gateway, ingress, BFF, and service ownership:
|
|
12
|
+
- OpenAPI / GraphQL schema / Protobuf links:
|
|
13
|
+
- Authentication and authorization model:
|
|
14
|
+
- Versioning and deprecation policy:
|
|
15
|
+
|
|
16
|
+
## Capacity and resilience
|
|
17
|
+
- Average/peak RPS, concurrency, payload sizes:
|
|
18
|
+
- P50/P95/P99 targets and SLO:
|
|
19
|
+
- Deadlines, retry budget, idempotency:
|
|
20
|
+
- Rate limits, quotas, admission control:
|
|
21
|
+
- Failure modes and degraded behavior:
|
|
22
|
+
|
|
23
|
+
## Security and observability
|
|
24
|
+
- Threat model and tenant isolation:
|
|
25
|
+
- Input, output, and data classification:
|
|
26
|
+
- Metrics, logs, traces, audit events:
|
|
27
|
+
- Abuse detection and incident response:
|
|
28
|
+
|
|
29
|
+
## Rollout and validation
|
|
30
|
+
- Compatibility and contract tests:
|
|
31
|
+
- Canary, rollback, and migration plan:
|
|
32
|
+
- Load, fault-injection, and security tests:
|
|
33
|
+
- Owners, open risks, and decision record:
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# API Security and Rate-Limit Review
|
|
2
|
+
|
|
3
|
+
## Scope and assets
|
|
4
|
+
- APIs, routes, methods, and data classes:
|
|
5
|
+
- Principals, tenants, and trust boundaries:
|
|
6
|
+
- Critical business actions and abuse impact:
|
|
7
|
+
|
|
8
|
+
## Authorization
|
|
9
|
+
- Authentication and token validation:
|
|
10
|
+
- Object-level and function-level authorization:
|
|
11
|
+
- Tenant scoping and cache-key isolation:
|
|
12
|
+
- Service identity and secret lifecycle:
|
|
13
|
+
|
|
14
|
+
## Abuse controls
|
|
15
|
+
- Rate-limit keys and dimensions:
|
|
16
|
+
- Algorithm, burst, concurrency, and cost units:
|
|
17
|
+
- Distributed enforcement consistency:
|
|
18
|
+
- 429/Retry-After and client retry policy:
|
|
19
|
+
- Limiter outage and emergency fallback:
|
|
20
|
+
|
|
21
|
+
## Detection and response
|
|
22
|
+
- Metrics, alerts, audit, and investigation:
|
|
23
|
+
- False-positive handling and overrides:
|
|
24
|
+
- Load shedding and noisy-neighbor controls:
|
|
25
|
+
|
|
26
|
+
## Verification
|
|
27
|
+
- BOLA/BFLA, replay, enumeration, and bypass tests:
|
|
28
|
+
- Burst, hot-key, clock-skew, and dependency-failure tests:
|
|
29
|
+
- Findings, severity, owners, due dates:
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Architecture Decision Matrix: <decision>
|
|
2
|
+
|
|
3
|
+
## Decision context
|
|
4
|
+
- Business outcome:
|
|
5
|
+
- Scope / non-goals:
|
|
6
|
+
- Decision owner:
|
|
7
|
+
- Date:
|
|
8
|
+
- Hard constraints (gates):
|
|
9
|
+
- Workload evidence and assumptions:
|
|
10
|
+
|
|
11
|
+
## Candidate options
|
|
12
|
+
| Option | Summary | Meets hard constraints? | Key dependencies |
|
|
13
|
+
|---|---|---|---|
|
|
14
|
+
| Simplest viable baseline | | | |
|
|
15
|
+
| Alternative A | | | |
|
|
16
|
+
| Alternative B | | | |
|
|
17
|
+
|
|
18
|
+
## Comparative assessment
|
|
19
|
+
Use Strong / Adequate / Weak / Unknown, or a defined numeric scale. Attach evidence and confidence. Do not treat unknown as a favorable score.
|
|
20
|
+
|
|
21
|
+
| Criterion | Priority / weight (source) | Baseline | Option A | Option B | Evidence / confidence |
|
|
22
|
+
|---|---|---|---|---|---|
|
|
23
|
+
| Performance and scalability | | | | | |
|
|
24
|
+
| Consistency and data integrity | | | | | |
|
|
25
|
+
| Reliability and availability | | | | | |
|
|
26
|
+
| Security and compliance | | | | | |
|
|
27
|
+
| Total cost of ownership | | | | | |
|
|
28
|
+
| Operational complexity | | | | | |
|
|
29
|
+
| Team capability | | | | | |
|
|
30
|
+
| Migration and exit cost | | | | | |
|
|
31
|
+
| Ecosystem and delivery risk | | | | | |
|
|
32
|
+
|
|
33
|
+
## Sensitivity and uncertainty
|
|
34
|
+
- Assumptions that could reverse the decision:
|
|
35
|
+
- Evidence gaps and cheapest validation experiment:
|
|
36
|
+
- Cost / capacity ranges:
|
|
37
|
+
|
|
38
|
+
## Recommendation
|
|
39
|
+
- Selected option:
|
|
40
|
+
- Why it is proportionate:
|
|
41
|
+
- Rejected alternatives and reasons:
|
|
42
|
+
- Risks and mitigations:
|
|
43
|
+
- Migration / rollback:
|
|
44
|
+
- Measurable revisit triggers:
|
|
45
|
+
|
|
46
|
+
## Approval
|
|
47
|
+
- Stakeholders:
|
|
48
|
+
- Decision status:
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Architecture Review Scorecard: <system>
|
|
2
|
+
|
|
3
|
+
## Review scope
|
|
4
|
+
- Components / paths inspected:
|
|
5
|
+
- Evidence sources:
|
|
6
|
+
- Exclusions and unknowns:
|
|
7
|
+
- Review date / reviewers:
|
|
8
|
+
|
|
9
|
+
## Dimension assessment
|
|
10
|
+
Use Pass / Needs work / Fail / Not assessed. Each assessment must cite evidence. Do not calculate a single average health percentage.
|
|
11
|
+
|
|
12
|
+
| Dimension | Assessment | Evidence | Business impact / risk | Required action |
|
|
13
|
+
|---|---|---|---|---|
|
|
14
|
+
| Architecture and boundaries | | | | |
|
|
15
|
+
| Data integrity | | | | |
|
|
16
|
+
| Scalability and performance | | | | |
|
|
17
|
+
| Reliability and recovery | | | | |
|
|
18
|
+
| Security and privacy | | | | |
|
|
19
|
+
| Observability and operations | | | | |
|
|
20
|
+
| Maintainability and delivery | | | | |
|
|
21
|
+
| Cost efficiency | | | | |
|
|
22
|
+
|
|
23
|
+
## Findings
|
|
24
|
+
### [Critical / High / Medium / Low] <finding>
|
|
25
|
+
- Status: Confirmed / Likely / Unknown
|
|
26
|
+
- Evidence: path, symbol, metric, trace, test, or reproduction
|
|
27
|
+
- Trigger / likelihood:
|
|
28
|
+
- Impact:
|
|
29
|
+
- Recommendation:
|
|
30
|
+
- Validation:
|
|
31
|
+
|
|
32
|
+
## Executive assessment
|
|
33
|
+
- Material risks:
|
|
34
|
+
- Strengths:
|
|
35
|
+
- Unverified assumptions:
|
|
36
|
+
- Priority sequence:
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# CDC and Data Pipeline Design
|
|
2
|
+
|
|
3
|
+
## Source and capture
|
|
4
|
+
- Source database / version:
|
|
5
|
+
- Connector and version:
|
|
6
|
+
- Log mechanism and retention window:
|
|
7
|
+
- Snapshot mode and consistency boundary:
|
|
8
|
+
- Captured tables / columns:
|
|
9
|
+
- Sensitive data excluded or masked:
|
|
10
|
+
|
|
11
|
+
## Delivery and semantics
|
|
12
|
+
- Broker / destination:
|
|
13
|
+
- Ordering guarantees and scope:
|
|
14
|
+
- Transaction metadata requirements:
|
|
15
|
+
- Duplicate handling and idempotency:
|
|
16
|
+
- Delete / tombstone semantics:
|
|
17
|
+
- Schema evolution and type mapping:
|
|
18
|
+
|
|
19
|
+
## Consumers and quality
|
|
20
|
+
| Consumer | Purpose | Expected freshness | Idempotency key | Data quality checks |
|
|
21
|
+
|---|---|---|---|---|
|
|
22
|
+
| | | | | |
|
|
23
|
+
|
|
24
|
+
- Lineage and data ownership:
|
|
25
|
+
- Quarantine / DLQ and replay process:
|
|
26
|
+
- Row counts, checksums, invariant validation:
|
|
27
|
+
|
|
28
|
+
## Capacity and operations
|
|
29
|
+
- Snapshot volume and throttling:
|
|
30
|
+
- Steady-state change rate:
|
|
31
|
+
- Source DB impact and log lag:
|
|
32
|
+
- Connector offset / schema history storage:
|
|
33
|
+
- Alerts and SLOs:
|
|
34
|
+
- Outage, log truncation, and recovery procedure:
|
|
35
|
+
|
|
36
|
+
## Migration / cutover
|
|
37
|
+
- Backfill and CDC catch-up:
|
|
38
|
+
- Write ownership and dual-write avoidance:
|
|
39
|
+
- Consistency evidence:
|
|
40
|
+
- Cutover gates and rollback:
|
|
41
|
+
- Old-source retirement criteria:
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# CDN and Edge Strategy
|
|
2
|
+
|
|
3
|
+
## 1. Content and traffic profile
|
|
4
|
+
- Content classes and owners:
|
|
5
|
+
- Geographic distribution and request/byte volume:
|
|
6
|
+
- Dynamic/static ratio and latency objectives:
|
|
7
|
+
- Sensitive or personalized response classes:
|
|
8
|
+
|
|
9
|
+
## 2. Cache policy
|
|
10
|
+
| Content class | Cache key dimensions | TTL/freshness | Stale policy | Shared cache allowed? |
|
|
11
|
+
|---|---|---|---|---|
|
|
12
|
+
| | | | | |
|
|
13
|
+
|
|
14
|
+
## 3. Origin architecture
|
|
15
|
+
- Origin locations and shielding:
|
|
16
|
+
- Origin authentication and bypass prevention:
|
|
17
|
+
- Failover and overload behavior:
|
|
18
|
+
- Cache miss/stampede controls:
|
|
19
|
+
|
|
20
|
+
## 4. Invalidation and consistency
|
|
21
|
+
- Versioned URL/key strategy:
|
|
22
|
+
- Purge/tag mechanism and propagation expectations:
|
|
23
|
+
- Retry, ordering, and verification:
|
|
24
|
+
- Deletion and privacy requirements:
|
|
25
|
+
|
|
26
|
+
## 5. Edge execution and security
|
|
27
|
+
- Edge compute use cases and runtime constraints:
|
|
28
|
+
- Authentication/authorization boundaries:
|
|
29
|
+
- WAF, bot, DDoS, TLS, and rate-limit ownership:
|
|
30
|
+
- Cache poisoning/deception and tenant isolation controls:
|
|
31
|
+
|
|
32
|
+
## 6. Economics and SLOs
|
|
33
|
+
- Hit ratio by requests and bytes:
|
|
34
|
+
- Origin offload and egress:
|
|
35
|
+
- Request, purge, compute, and storage costs:
|
|
36
|
+
- Latency, availability, stale-serving, and origin SLOs:
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Chaos Engineering Experiment
|
|
2
|
+
|
|
3
|
+
## 1. Hypothesis
|
|
4
|
+
- Steady state (user-visible):
|
|
5
|
+
- Resilience hypothesis:
|
|
6
|
+
- Expected behavior under fault:
|
|
7
|
+
|
|
8
|
+
## 2. Scope and authorization
|
|
9
|
+
- Environment/cohort/tenant:
|
|
10
|
+
- Target and dependencies:
|
|
11
|
+
- Explicit authorization/change window:
|
|
12
|
+
- Exclusions and blast-radius limit:
|
|
13
|
+
|
|
14
|
+
## 3. Fault injection
|
|
15
|
+
- Fault class and mechanism:
|
|
16
|
+
- Intensity, duration, and ramp:
|
|
17
|
+
- Prerequisites and data-safety constraints:
|
|
18
|
+
|
|
19
|
+
## 4. Observability and abort
|
|
20
|
+
- SLO/SLI and integrity signals:
|
|
21
|
+
- Observer and incident commander:
|
|
22
|
+
- Abort thresholds:
|
|
23
|
+
- Kill switch and recovery procedure:
|
|
24
|
+
|
|
25
|
+
## 5. Execution record
|
|
26
|
+
| Timestamp | Action/observation | Signal/evidence | Decision |
|
|
27
|
+
|---|---|---|---|
|
|
28
|
+
| | | | |
|
|
29
|
+
|
|
30
|
+
## 6. Outcome
|
|
31
|
+
- Detection and containment:
|
|
32
|
+
- User impact/degradation:
|
|
33
|
+
- Recovery and reconciliation:
|
|
34
|
+
- Hypothesis: validated / invalidated / inconclusive
|
|
35
|
+
|
|
36
|
+
## 7. Remediation
|
|
37
|
+
| Finding | Risk reduction | Owner | Due date | Retest |
|
|
38
|
+
|---|---|---|---|---|
|
|
39
|
+
| | | | | |
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# Cloud Cost and FinOps Model
|
|
2
|
+
|
|
3
|
+
## Scope and assumptions
|
|
4
|
+
- Workload / service / environment:
|
|
5
|
+
- Forecast period and currency:
|
|
6
|
+
- Expected, peak, and degraded demand:
|
|
7
|
+
- Data growth and retention:
|
|
8
|
+
- Regions and availability topology:
|
|
9
|
+
|
|
10
|
+
## Cost drivers
|
|
11
|
+
| Category | Unit / rate source | Expected quantity | Peak quantity | Monthly estimate | Owner / allocation |
|
|
12
|
+
|---|---|---:|---:|---:|---|
|
|
13
|
+
| Compute | | | | | |
|
|
14
|
+
| Storage / IOPS / backups | | | | | |
|
|
15
|
+
| Database / cache | | | | | |
|
|
16
|
+
| Network / egress / inter-region | | | | | |
|
|
17
|
+
| Load balancing / gateways | | | | | |
|
|
18
|
+
| Observability / retention | | | | | |
|
|
19
|
+
| Licenses / support | | | | | |
|
|
20
|
+
| Platform operations | | | | | |
|
|
21
|
+
|
|
22
|
+
## Unit economics
|
|
23
|
+
- Cost per request / transaction / active user:
|
|
24
|
+
- Cost per unit of business value:
|
|
25
|
+
- Shared-cost allocation method:
|
|
26
|
+
- Unallocated spend and confidence:
|
|
27
|
+
|
|
28
|
+
## Optimization hypotheses
|
|
29
|
+
| Change | Expected savings range | SLO / security risk | Validation method | Owner |
|
|
30
|
+
|---|---|---|---|---|
|
|
31
|
+
| | | | | |
|
|
32
|
+
|
|
33
|
+
## Commitments and governance
|
|
34
|
+
- Reserved / committed capacity assumptions:
|
|
35
|
+
- Break-even utilization and lock-in risk:
|
|
36
|
+
- Budget owner, alerts, and approval thresholds:
|
|
37
|
+
- Forecast confidence and review cadence:
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Database Architecture Review
|
|
2
|
+
|
|
3
|
+
## Context
|
|
4
|
+
- System / service:
|
|
5
|
+
- Database engine and exact version:
|
|
6
|
+
- Hosting / topology / regions:
|
|
7
|
+
- Workload (read/write, peak QPS, concurrency):
|
|
8
|
+
- Data volume, growth, retention:
|
|
9
|
+
- Critical SLOs and business invariants:
|
|
10
|
+
|
|
11
|
+
## Data model and access
|
|
12
|
+
- Source of truth and ownership:
|
|
13
|
+
- Primary/unique keys:
|
|
14
|
+
- Indexes and query patterns:
|
|
15
|
+
- Transaction boundaries and isolation:
|
|
16
|
+
- Hot keys, skew, and contention:
|
|
17
|
+
|
|
18
|
+
## Reliability and recovery
|
|
19
|
+
- Replication mode and lag:
|
|
20
|
+
- Backup/PITR and last restore exercise:
|
|
21
|
+
- RPO/RTO:
|
|
22
|
+
- Failover/fencing and failback:
|
|
23
|
+
- Upgrade and extension compatibility:
|
|
24
|
+
|
|
25
|
+
## Evidence
|
|
26
|
+
| Finding | Evidence / query plan / metric | Impact | Severity | Validation |
|
|
27
|
+
|---|---|---|---|---|
|
|
28
|
+
|
|
29
|
+
## Recommendations
|
|
30
|
+
| Change | Expected outcome | Risk / cost | Rollback | Success metric |
|
|
31
|
+
|---|---|---|---|---|
|
|
32
|
+
|
|
33
|
+
## Open questions and assumptions
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Edge Computing Readiness Review
|
|
2
|
+
|
|
3
|
+
## 1. Business justification
|
|
4
|
+
- Latency, locality, bandwidth, or offline requirement:
|
|
5
|
+
- Why regional cloud or client-side execution is insufficient:
|
|
6
|
+
- Measurable success criteria:
|
|
7
|
+
|
|
8
|
+
## 2. Placement and authority
|
|
9
|
+
| Capability/data | Device/site/metro/region/cloud | Authoritative owner | Offline behavior |
|
|
10
|
+
|---|---|---|---|
|
|
11
|
+
| | | | |
|
|
12
|
+
|
|
13
|
+
## 3. Fleet lifecycle
|
|
14
|
+
- Enrollment and device identity:
|
|
15
|
+
- Secure boot/attestation:
|
|
16
|
+
- Artifact signing and staged rollout:
|
|
17
|
+
- Configuration, rollback, decommissioning:
|
|
18
|
+
- Hardware replacement and field support:
|
|
19
|
+
|
|
20
|
+
## 4. Data synchronization
|
|
21
|
+
- Local durability and queue bounds:
|
|
22
|
+
- Replication direction and conflict policy:
|
|
23
|
+
- Retry, deduplication, replay, and reconciliation:
|
|
24
|
+
- Deletion and retention propagation:
|
|
25
|
+
|
|
26
|
+
## 5. Security and resilience
|
|
27
|
+
- Secrets and credential rotation during disconnection:
|
|
28
|
+
- Local privilege and physical compromise:
|
|
29
|
+
- Encrypted storage and tamper-aware audit:
|
|
30
|
+
- Degraded mode and recovery:
|
|
31
|
+
|
|
32
|
+
## 6. Operations and economics
|
|
33
|
+
- Fleet health, version skew, sync lag, last-seen:
|
|
34
|
+
- Telemetry buffering and replay controls:
|
|
35
|
+
- Hardware, connectivity, energy, support, and lifecycle cost:
|
|
36
|
+
- Pilot scope, rollback, and exit criteria:
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# Event Sourcing and CQRS Design
|
|
2
|
+
|
|
3
|
+
## Domain boundaries
|
|
4
|
+
- Aggregate and invariant boundaries:
|
|
5
|
+
- Commands and authorization:
|
|
6
|
+
- Domain events and ownership:
|
|
7
|
+
- Source of truth:
|
|
8
|
+
- Why event sourcing is justified over conventional transactional storage:
|
|
9
|
+
|
|
10
|
+
## Event model
|
|
11
|
+
| Event type | Aggregate | Version | Required fields | PII / retention |
|
|
12
|
+
|---|---|---|---|---|
|
|
13
|
+
| | | | | |
|
|
14
|
+
|
|
15
|
+
- Event identity and sequence:
|
|
16
|
+
- Expected-version concurrency strategy:
|
|
17
|
+
- Command idempotency and deduplication:
|
|
18
|
+
- Correction / compensation semantics:
|
|
19
|
+
|
|
20
|
+
## Projections and read models
|
|
21
|
+
| Projection | Consumers | Ordering | Checkpoint | Freshness SLO | Rebuild procedure |
|
|
22
|
+
|---|---|---|---|---|---|
|
|
23
|
+
| | | | | | |
|
|
24
|
+
|
|
25
|
+
- Read-your-writes behavior:
|
|
26
|
+
- Projection generation and atomic cutover:
|
|
27
|
+
- Replay and backfill limits:
|
|
28
|
+
|
|
29
|
+
## Evolution and operations
|
|
30
|
+
- Schema compatibility and upcasters:
|
|
31
|
+
- Snapshot policy and replay verification:
|
|
32
|
+
- PII minimization, deletion, legal hold:
|
|
33
|
+
- Event-store backup and restore:
|
|
34
|
+
- Projection lag, replay storms, alerting:
|
|
35
|
+
- Recovery and reconciliation tests:
|
|
36
|
+
|
|
37
|
+
## Decision and risks
|
|
38
|
+
- Alternatives and TCO:
|
|
39
|
+
- Accepted consistency trade-offs:
|
|
40
|
+
- Migration / exit strategy:
|
|
41
|
+
- Risks, owners, and review date:
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Event Streaming Design
|
|
2
|
+
|
|
3
|
+
## Use case and requirements
|
|
4
|
+
- Business capability and event consumers:
|
|
5
|
+
- Throughput (average / peak) and payload size:
|
|
6
|
+
- Latency and freshness objectives:
|
|
7
|
+
- Retention and replay requirements:
|
|
8
|
+
- Ordering and consistency requirements:
|
|
9
|
+
- Availability and recovery objectives:
|
|
10
|
+
|
|
11
|
+
## Platform decision
|
|
12
|
+
- Candidates considered:
|
|
13
|
+
- Selected broker / managed service:
|
|
14
|
+
- Why a queue, database outbox, or simpler alternative is insufficient:
|
|
15
|
+
- Operational owner and exit strategy:
|
|
16
|
+
|
|
17
|
+
## Topic and event contract
|
|
18
|
+
| Topic | Owner | Key / partition strategy | Schema | Retention | Consumers |
|
|
19
|
+
|---|---|---|---|---|---|
|
|
20
|
+
| | | | | | |
|
|
21
|
+
|
|
22
|
+
- Event ID, type, version, timestamps, correlation/causation:
|
|
23
|
+
- PII classification and access policy:
|
|
24
|
+
- Schema compatibility mode:
|
|
25
|
+
|
|
26
|
+
## Delivery semantics
|
|
27
|
+
- Producer acknowledgement and idempotency:
|
|
28
|
+
- Replication / durability settings:
|
|
29
|
+
- Ordering scope:
|
|
30
|
+
- Consumer offset / checkpoint strategy:
|
|
31
|
+
- Duplicate and idempotency handling:
|
|
32
|
+
- Retry, DLQ, replay and poison-event workflow:
|
|
33
|
+
|
|
34
|
+
## Capacity and operations
|
|
35
|
+
- Partitions, replication, retention storage:
|
|
36
|
+
- Network and consumer fan-out:
|
|
37
|
+
- Lag-age and broker SLOs:
|
|
38
|
+
- Quotas, hot-key and skew mitigation:
|
|
39
|
+
- Upgrade, backup, restore and disaster recovery:
|
|
40
|
+
|
|
41
|
+
## Failure tests and acceptance
|
|
42
|
+
- Broker / zone loss:
|
|
43
|
+
- Consumer crash and rebalance:
|
|
44
|
+
- Source or destination outage:
|
|
45
|
+
- Schema incompatibility:
|
|
46
|
+
- Replay and catch-up:
|
|
47
|
+
- Acceptance criteria and owners:
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Failure Analysis: <system / critical journey>
|
|
2
|
+
|
|
3
|
+
## Scope and objectives
|
|
4
|
+
- Critical journey / invariant:
|
|
5
|
+
- Availability target:
|
|
6
|
+
- RTO:
|
|
7
|
+
- RPO:
|
|
8
|
+
- Owners / escalation:
|
|
9
|
+
|
|
10
|
+
## Dependency and failure-mode register
|
|
11
|
+
| Component | Failure mode / trigger | Detection | Blast radius | User impact | Containment / degradation | Recovery / reconciliation | Validation |
|
|
12
|
+
|---|---|---|---|---|---|---|---|
|
|
13
|
+
| | | | | | | | |
|
|
14
|
+
|
|
15
|
+
## Propagation analysis
|
|
16
|
+
- Shared dependencies / common-mode risks:
|
|
17
|
+
- Retry and fan-out amplification:
|
|
18
|
+
- Pool, queue, and resource exhaustion:
|
|
19
|
+
- Tenant / region isolation:
|
|
20
|
+
|
|
21
|
+
## Degraded behavior
|
|
22
|
+
- Operations that may continue:
|
|
23
|
+
- Operations that must fail closed:
|
|
24
|
+
- Staleness / data-loss tolerance:
|
|
25
|
+
- User-visible status and communication:
|
|
26
|
+
|
|
27
|
+
## Recovery plan
|
|
28
|
+
- Failover authority / split-brain prevention:
|
|
29
|
+
- Restore / rollback / forward fix:
|
|
30
|
+
- Backlog drain and reconciliation:
|
|
31
|
+
- Recovery verification and owner:
|
|
32
|
+
|
|
33
|
+
## Failure-test plan
|
|
34
|
+
| Scenario | Expected behavior | Safety boundary / stop condition | Evidence of success |
|
|
35
|
+
|---|---|---|---|
|
|
36
|
+
| | | | |
|