system-design-primer-skill 0.0.0-stage → 2.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (112) hide show
  1. package/ATTRIBUTION.md +10 -0
  2. package/LICENSE +7 -0
  3. package/README.md +180 -2
  4. package/SKILL.md +447 -0
  5. package/VERSION +1 -0
  6. package/package.json +78 -4
  7. package/references/00-primer-map-and-attribution.md +29 -0
  8. package/references/01-requirements-and-capacity.md +107 -0
  9. package/references/02-architecture-and-networking.md +94 -0
  10. package/references/03-data-and-storage.md +88 -0
  11. package/references/04-caching.md +78 -0
  12. package/references/05-async-and-messaging.md +77 -0
  13. package/references/06-distributed-systems.md +59 -0
  14. package/references/07-reliability-and-resilience.md +71 -0
  15. package/references/08-api-security-observability.md +62 -0
  16. package/references/09-modern-engineering.md +58 -0
  17. package/references/10-workflows-and-checklists.md +97 -0
  18. package/references/11-case-studies.md +74 -0
  19. package/references/12-ai-systems-architecture.md +139 -0
  20. package/references/13-architecture-intelligence.md +91 -0
  21. package/references/14-api-and-data-contracts.md +80 -0
  22. package/references/15-failure-analysis.md +84 -0
  23. package/references/16-multi-agent-architecture.md +76 -0
  24. package/references/17-agent-harness-and-runtime.md +87 -0
  25. package/references/18-mcp-architecture.md +67 -0
  26. package/references/19-agent-security-and-permissions.md +74 -0
  27. package/references/20-agent-state-and-memory.md +72 -0
  28. package/references/21-agent-evaluation-and-observability.md +90 -0
  29. package/references/22-kubernetes-and-container-platforms.md +29 -0
  30. package/references/23-service-mesh-and-traffic-management.md +21 -0
  31. package/references/24-multi-region-and-disaster-recovery.md +24 -0
  32. package/references/25-platform-engineering-and-developer-experience.md +26 -0
  33. package/references/26-finops-and-cloud-cost.md +29 -0
  34. package/references/27-production-observability-and-slo.md +29 -0
  35. package/references/28-event-streaming-and-kafka-pulsar.md +27 -0
  36. package/references/29-event-sourcing-and-cqrs.md +23 -0
  37. package/references/30-change-data-capture-and-debezium.md +27 -0
  38. package/references/31-data-lakehouse-and-stream-batch.md +24 -0
  39. package/references/32-workflow-orchestration-and-temporal.md +24 -0
  40. package/references/33-schema-registry-and-event-governance.md +22 -0
  41. package/references/34-postgresql-internals-and-performance.md +43 -0
  42. package/references/35-mysql-innodb-and-performance.md +27 -0
  43. package/references/36-sharding-and-distributed-sql.md +22 -0
  44. package/references/37-consensus-and-distributed-transactions.md +25 -0
  45. package/references/38-search-architecture-elasticsearch-opensearch.md +24 -0
  46. package/references/39-vector-databases-and-hybrid-retrieval.md +31 -0
  47. package/references/40-zero-downtime-database-migrations.md +27 -0
  48. package/references/41-api-gateway-and-edge-architecture.md +23 -0
  49. package/references/42-rest-api-design-and-lifecycle.md +23 -0
  50. package/references/43-graphql-architecture-and-governance.md +16 -0
  51. package/references/44-grpc-and-service-contracts.md +16 -0
  52. package/references/45-websocket-sse-and-realtime-systems.md +16 -0
  53. package/references/46-realtime-collaboration-ot-crdt.md +16 -0
  54. package/references/47-api-security-rate-limiting-and-abuse.md +16 -0
  55. package/references/48-oauth-oidc-and-passkeys.md +20 -0
  56. package/references/49-iam-rbac-abac-rebac.md +16 -0
  57. package/references/50-zero-trust-and-workload-identity.md +16 -0
  58. package/references/51-secrets-kms-pki-and-key-management.md +16 -0
  59. package/references/52-supply-chain-security-sbom-slsa.md +16 -0
  60. package/references/53-threat-modeling-stride-and-attack-trees.md +16 -0
  61. package/references/54-privacy-engineering-and-data-governance.md +16 -0
  62. package/references/55-opentelemetry-and-instrumentation.md +50 -0
  63. package/references/56-distributed-tracing-and-context-propagation.md +43 -0
  64. package/references/57-continuous-profiling-and-performance-diagnostics.md +45 -0
  65. package/references/58-load-stress-soak-testing-and-workload-modeling.md +44 -0
  66. package/references/59-capacity-planning-and-performance-modeling.md +34 -0
  67. package/references/60-chaos-engineering-and-resilience-experiments.md +41 -0
  68. package/references/61-incident-management-and-postmortems.md +34 -0
  69. package/references/62-dns-and-service-discovery.md +35 -0
  70. package/references/63-load-balancing-and-global-traffic-management.md +31 -0
  71. package/references/64-cdn-and-edge-caching.md +34 -0
  72. package/references/65-cloud-networking-and-connectivity.md +34 -0
  73. package/references/66-edge-computing-architecture.md +33 -0
  74. package/references/67-storage-infrastructure-and-storage-systems.md +30 -0
  75. package/references/68-tcp-quic-and-network-performance.md +27 -0
  76. package/references/69-task-routing-and-evidence.md +58 -0
  77. package/scripts/install.ps1 +124 -0
  78. package/scripts/install.sh +105 -0
  79. package/scripts/validate.py +376 -0
  80. package/templates/adr-template.md +32 -0
  81. package/templates/agent-evaluation-plan-template.md +57 -0
  82. package/templates/agent-security-review-template.md +46 -0
  83. package/templates/agent-system-design-template.md +74 -0
  84. package/templates/api-architecture-design-template.md +33 -0
  85. package/templates/api-security-and-rate-limit-review-template.md +29 -0
  86. package/templates/architecture-decision-matrix-template.md +48 -0
  87. package/templates/architecture-design-template.md +95 -0
  88. package/templates/architecture-review-scorecard-template.md +36 -0
  89. package/templates/architecture-review-template.md +52 -0
  90. package/templates/cdc-data-pipeline-template.md +41 -0
  91. package/templates/cdn-and-edge-strategy-template.md +36 -0
  92. package/templates/chaos-experiment-template.md +39 -0
  93. package/templates/cloud-cost-model-template.md +37 -0
  94. package/templates/database-architecture-review-template.md +33 -0
  95. package/templates/edge-computing-readiness-template.md +36 -0
  96. package/templates/event-sourcing-cqrs-template.md +41 -0
  97. package/templates/event-streaming-design-template.md +47 -0
  98. package/templates/failure-analysis-template.md +36 -0
  99. package/templates/identity-and-access-architecture-template.md +29 -0
  100. package/templates/incident-postmortem-template.md +39 -0
  101. package/templates/multi-region-disaster-recovery-template.md +43 -0
  102. package/templates/network-architecture-design-template.md +38 -0
  103. package/templates/observability-architecture-template.md +52 -0
  104. package/templates/performance-test-plan-template.md +52 -0
  105. package/templates/privacy-data-governance-template.md +24 -0
  106. package/templates/production-readiness-checklist-template.md +52 -0
  107. package/templates/realtime-system-design-template.md +28 -0
  108. package/templates/security-threat-model-template.md +26 -0
  109. package/templates/sharding-distributed-sql-design-template.md +34 -0
  110. package/templates/storage-architecture-design-template.md +35 -0
  111. package/templates/supply-chain-security-review-template.md +20 -0
  112. package/templates/zero-downtime-database-migration-template.md +32 -0
@@ -0,0 +1,16 @@
1
+ # Zero Trust and Workload Identity
2
+
3
+ ## Trust model
4
+ Zero Trust removes implicit trust based on network location. Authenticate and authorize each workload-to-workload interaction using explicit identity, resource, action, and context. Network segmentation remains useful defense in depth, not proof of identity.
5
+
6
+ ## Workload identity
7
+ Prefer short-lived, automatically rotated identities bound to workload properties and environment. Use SPIFFE/SPIRE or cloud-native workload identity where appropriate; compare federation, operational maturity, trust-domain boundaries, and portability. Avoid long-lived static service credentials and shared identities that erase attribution.
8
+
9
+ ## Service-to-service controls
10
+ Use mTLS for peer authentication and transport protection where suitable, but still authorize methods and resources. Define certificate issuance, trust bundles, rotation overlap, revocation, identity mapping, and failure behavior. Prevent confused-deputy behavior by constraining audience and delegated authority.
11
+
12
+ ## Policy and segmentation
13
+ Apply least privilege at service, method, data, and administrative boundaries. Combine identity-aware policy with network policy, egress restrictions, and workload isolation. Define policy propagation and emergency revocation latency.
14
+
15
+ ## Operations
16
+ Measure authentication failures, certificate expiry, identity issuance latency, policy denials, and trust-bundle drift. Test clock skew, CA outage, certificate rotation, compromised workload containment, failover, and recovery. Document bootstrap trust and break-glass procedures.
@@ -0,0 +1,16 @@
1
+ # Secrets Management, KMS, PKI, and Key Management
2
+
3
+ ## Secret lifecycle
4
+ Inventory secrets, owners, consumers, environments, purpose, classification, and expiry. Prefer workload identity and dynamic credentials over static secrets. Centralize issuance, access policy, rotation, revocation, and audit; avoid distributing a universal root secret to every service.
5
+
6
+ ## Cryptographic key hierarchy
7
+ Separate data-encryption keys from key-encryption keys. Use envelope encryption and managed KMS/HSM capabilities where appropriate. Define key purpose, algorithm, allowed operations, region, rotation, backup, recovery, deletion, and separation of duties. Key rotation does not automatically re-encrypt all historical data.
8
+
9
+ ## PKI operations
10
+ Define root and intermediate CA custody, issuance policy, certificate identity, validity, renewal, revocation, trust distribution, and emergency rollover. Protect CA signing keys with stronger controls than ordinary application secrets. Plan overlapping trust during rotations and verify client behavior under expired or unknown certificates.
11
+
12
+ ## Application integration
13
+ Avoid secrets in source, images, build logs, crash dumps, environment diagnostics, and telemetry. Use narrowly scoped runtime retrieval with bounded caching and fail-safe behavior. Do not log secret values when reporting access failures.
14
+
15
+ ## Resilience and audit
16
+ KMS and secret stores are critical dependencies: model quotas, regional availability, latency, caching, outage behavior, and recovery. Test restore and key-loss scenarios before relying on encrypted backups. Audit key use, policy changes, export, deletion, and break-glass operations.
@@ -0,0 +1,16 @@
1
+ # Software Supply Chain Security: SBOM, Provenance, and SLSA
2
+
3
+ ## Threat model
4
+ Cover dependency confusion, typosquatting, compromised maintainers, malicious build steps, poisoned artifacts, leaked signing keys, untrusted registries, and release pipeline compromise. Map trust boundaries from source and dependencies through build, signing, registry, deployment, and runtime.
5
+
6
+ ## Dependency governance
7
+ Pin and review dependencies using lockfiles, trusted registries, provenance where available, vulnerability and license scanning, and controlled update automation. Pinning alone does not establish trust. Define exceptions, remediation SLAs, ownership, and reachability-based prioritization.
8
+
9
+ ## SBOM and provenance
10
+ Generate machine-readable SBOMs for shipped artifacts and preserve component identity and version. Attest build provenance with authenticated builder identity, source revision, build parameters, and artifact digest. Verify attestations at promotion/deployment; do not trust unsigned metadata supplied beside an artifact.
11
+
12
+ ## Build integrity
13
+ Use isolated ephemeral builders, least-privilege tokens, protected branches, reviewed workflows, reproducible or independently verifiable builds where feasible, and separation between build and release authority. Protect signing keys and enforce artifact immutability.
14
+
15
+ ## Release and response
16
+ Define artifact signing, verification, promotion, rollback, and incident procedures for compromised dependencies or builders. Test provenance verification and quarantine paths. Treat SLSA levels as a framework for incremental assurance, not a blanket security certification.
@@ -0,0 +1,16 @@
1
+ # Threat Modeling: STRIDE, Attack Trees, and Risk Analysis
2
+
3
+ ## Scope and assets
4
+ Record system boundaries, data flows, stores, actors, trust boundaries, entry points, sensitive assets, and security objectives. Include control planes, CI/CD, support tooling, third parties, and operational access—not only runtime APIs.
5
+
6
+ ## Analysis methods
7
+ Use STRIDE to systematically identify spoofing, tampering, repudiation, information disclosure, denial of service, and elevation of privilege. Use attack trees to decompose attacker goals into feasible paths. Supplement with abuse cases, misuse cases, and privacy harms. No single taxonomy replaces domain expertise.
8
+
9
+ ## Risk prioritization
10
+ Describe each scenario as actor, preconditions, attack path, affected asset, impact, likelihood rationale, existing controls, and residual risk. Use a consistent scoring model and distinguish technical severity from business risk. Avoid false precision in numeric scores.
11
+
12
+ ## Mitigation and verification
13
+ Prefer eliminating or reducing attack surface, then enforce preventive controls, detection, and recovery. Assign each mitigation an owner, due date, and verification test. Track accepted risks with an approver and expiry/review date.
14
+
15
+ ## Continuous lifecycle
16
+ Revisit threat models when architecture, trust boundaries, data classes, dependencies, deployment, or attacker capabilities change. Validate assumptions with security tests, tabletop exercises, and incident learnings.
@@ -0,0 +1,16 @@
1
+ # Privacy Engineering and Data Governance
2
+
3
+ ## Data inventory and purpose
4
+ Map personal and sensitive data across collection, inference, processing, sharing, storage, logs, analytics, backups, and deletion. Document purpose, lawful basis where applicable, data owner, recipients, retention, residency, and user expectations. Minimize collection and avoid retaining data merely because storage is cheap.
5
+
6
+ ## Privacy by design
7
+ Use purpose limitation, minimization, transparency, access controls, pseudonymization, encryption, and privacy-preserving defaults. Assess re-identification risk in supposedly de-identified datasets and telemetry. Treat derived attributes and embeddings as potentially sensitive when they can reveal or link information.
8
+
9
+ ## Lifecycle controls
10
+ Define retention schedules, legal holds, correction, export, deletion, backup expiry, replica propagation, and downstream processor obligations. A deletion request is incomplete if data remains searchable in caches, indexes, analytics, or derived stores without a documented policy.
11
+
12
+ ## Governance and accountability
13
+ Assign data stewards, classification, quality rules, lineage, access approval, sharing agreements, and incident ownership. Apply residency and cross-border transfer controls based on applicable law and contractual commitments; obtain qualified legal review rather than inferring legal compliance from technical controls.
14
+
15
+ ## Verification
16
+ Test data discovery, access logs, retention enforcement, deletion propagation, backup expiration, subject-access workflows, and least-privilege analytics. Measure policy exceptions and stale data inventories.
@@ -0,0 +1,50 @@
1
+ # OpenTelemetry Architecture and Instrumentation
2
+
3
+ ## Purpose and scope
4
+ OpenTelemetry (OTel) provides vendor-neutral APIs, SDKs, semantic conventions, and telemetry pipelines for traces, metrics, and logs. Treat it as an instrumentation and data-delivery architecture, not as an observability product or a guarantee that telemetry is correct.
5
+
6
+ ## Architecture decisions
7
+ - Define signal ownership and collection paths: application SDK/auto-instrumentation → OTLP → Collector gateway/agent → processors/exporters → backend.
8
+ - Choose agent, sidecar, gateway, or direct export based on deployment model, network boundaries, buffering needs, tenancy, and operational ownership.
9
+ - Prefer OTLP and stable semantic conventions where supported. Pin SDK, Collector, and instrumentation versions; verify stability status before adopting conventions.
10
+ - Separate application instrumentation from vendor-specific exporters to preserve portability.
11
+
12
+ ## Instrumentation design
13
+ - Instrument user journeys and service boundaries; add spans around meaningful operations, not every function.
14
+ - Record HTTP/RPC/database/messaging operations with consistent service.name, service.version, deployment.environment, and resource attributes.
15
+ - Propagate W3C Trace Context across synchronous and asynchronous boundaries. Define behavior for queues, batch jobs, scheduled work, and context loss.
16
+ - Use span events and links for meaningful milestones and causal relationships; do not model every event as a child span.
17
+ - Use histograms for latency distributions; counters for monotonic event totals; gauges/observable instruments for sampled state. Avoid high-cardinality labels.
18
+ - Correlate logs with trace_id and span_id where available. Keep logs useful when no trace exists.
19
+
20
+ ## Collector pipeline
21
+ - Define receiver protocols, authentication, TLS/mTLS, tenant routing, memory limits, batching, queueing, retry, and exporter timeouts.
22
+ - Apply filtering, attribute normalization, redaction, sampling, and routing deliberately. Processors must not silently remove fields needed for incident response.
23
+ - Size queues and memory limiters for bursts; monitor dropped data, export failures, queue occupancy, and retry exhaustion.
24
+ - Protect the Collector as a production service: capacity, HA, upgrades, configuration rollout, and failure isolation matter.
25
+
26
+ ## Sampling and data governance
27
+ - Use head sampling for inexpensive early decisions; use tail sampling when outcomes or errors determine retention. Tail sampling requires trace affinity and bounded state.
28
+ - Define separate policies for errors, slow requests, critical business journeys, and routine traffic. Validate that parent/child spans remain coherent.
29
+ - Treat telemetry as potentially sensitive data. Minimize attributes, redact secrets and PII, restrict access, define retention, and document residency.
30
+ - Never put user IDs, request IDs, raw URLs, SQL literals, or arbitrary error text in metric labels.
31
+
32
+ ## Validation and operations
33
+ - Test instrumentation with representative traffic and verify propagation across services and queues.
34
+ - Monitor telemetry pipeline health independently of the application: ingest rate, export success, dropped spans/metrics/logs, queue age, memory, and backend rejection.
35
+ - Use synthetic spans and known test requests to detect broken propagation and missing instrumentation.
36
+ - Establish telemetry cost budgets and retention tiers; telemetry volume is a workload and a cost center.
37
+
38
+ ## Anti-patterns
39
+ - Assuming auto-instrumentation captures business semantics or all async context.
40
+ - Exporting directly from every workload without a failure and backpressure strategy.
41
+ - Treating trace sampling as statistically neutral for every question.
42
+ - Using unbounded attributes or cardinality as a substitute for useful dimensions.
43
+
44
+ ## Design checklist
45
+ - [ ] Signal coverage and ownership are defined for critical journeys.
46
+ - [ ] Resource attributes and semantic conventions are consistent.
47
+ - [ ] Context propagation is tested across process and messaging boundaries.
48
+ - [ ] Collector topology, security, buffering, and failure behavior are explicit.
49
+ - [ ] Sampling, redaction, retention, access, and cost policies are documented.
50
+ - [ ] Pipeline loss and instrumentation regressions are observable.
@@ -0,0 +1,43 @@
1
+ # Distributed Tracing and Context Propagation
2
+
3
+ ## Trace model
4
+ A trace is a causal graph of spans, not necessarily a single linear tree. Define trace boundaries, parent-child relationships, span links, baggage policy, and the behavior of asynchronous work before interpreting latency or causality.
5
+
6
+ ## Propagation
7
+ - Standardize W3C Trace Context (traceparent and tracestate) where interoperable; explicitly document legacy formats and translation boundaries.
8
+ - Propagate context through HTTP, RPC, message headers, job metadata, and workflow state. Verify framework and broker instrumentation behavior for deployed versions.
9
+ - For fan-out/fan-in, preserve causal relationships. Use span links when work is initiated by multiple messages or has multiple causal parents.
10
+ - Baggage is propagated metadata, not a trusted identity or authorization claim. Allowlist keys, constrain size, and exclude secrets and PII.
11
+ - At trust boundaries, validate and optionally restart trace context to prevent untrusted callers from forging internal trace relationships or injecting excessive baggage.
12
+
13
+ ## Latency interpretation
14
+ - Distinguish wall-clock end-to-end latency from summed span durations. Concurrent spans overlap; summing them overstates request duration.
15
+ - Identify queue wait, network time, server processing, lock waits, retries, and downstream dependency time separately where instrumentation permits.
16
+ - Check clock skew before interpreting cross-host timestamps. Prefer duration measurements from a single process clock for local spans.
17
+ - A missing span can mean instrumentation gaps, sampling, propagation loss, or exporter/backend loss; do not infer that the operation did not happen.
18
+
19
+ ## Sampling
20
+ - Define head/tail sampling by service criticality, error status, latency, tenant, and cost.
21
+ - Tail sampling needs trace affinity, bounded decision windows, memory limits, and explicit behavior when late spans arrive.
22
+ - Ensure sampling decisions are consistent across services or understand why partial traces are expected.
23
+ - Preserve representative baseline traffic; error-only retention cannot measure normal latency distributions.
24
+
25
+ ## Security and privacy
26
+ - Do not expose credentials, request bodies, raw query strings, or sensitive identifiers in span attributes.
27
+ - Sanitize untrusted header values and cap attribute/event sizes.
28
+ - Restrict trace search and export by tenant and role. Consider cross-tenant leakage through shared trace IDs and query interfaces.
29
+
30
+ ## Troubleshooting workflow
31
+ 1. Start from a user-visible symptom and identify the affected journey, time window, and cohort.
32
+ 2. Compare golden signals and SLO burn against baseline.
33
+ 3. Inspect representative traces, including successful and failed requests.
34
+ 4. Separate application time, queue wait, retries, and dependency time.
35
+ 5. Correlate with deploys, configuration changes, saturation, and dependency incidents.
36
+ 6. Validate the hypothesis with logs, metrics, profiles, or a controlled experiment.
37
+
38
+ ## Checklist
39
+ - [ ] Propagation covers sync and async boundaries.
40
+ - [ ] Baggage is bounded, allowlisted, and non-sensitive.
41
+ - [ ] Sampling preserves useful normal and exceptional traffic.
42
+ - [ ] Trace gaps and partial traces are understood.
43
+ - [ ] Latency interpretation accounts for concurrency and clock behavior.
@@ -0,0 +1,45 @@
1
+ # Continuous Profiling and Performance Diagnostics
2
+
3
+ ## Purpose
4
+ Profiling explains where CPU, memory, allocation, lock, and execution time are spent. Use it to test a specific performance hypothesis; do not treat a flame graph as a root cause without workload and runtime context.
5
+
6
+ ## Profile types
7
+ - CPU profiles: sampled on-CPU execution; distinguish hot code from time blocked on I/O.
8
+ - Heap/allocation profiles: retained objects, allocation rate, and allocation hot spots.
9
+ - Wall-clock profiles: on-CPU plus off-CPU waiting, subject to profiler/runtime support.
10
+ - Lock and contention profiles: mutex waits, blocked goroutines/threads, and synchronization bottlenecks.
11
+ - Runtime profiles: GC pauses, scheduling, thread pools, JIT behavior, and event-loop stalls.
12
+
13
+ ## Method
14
+ 1. Define the affected endpoint/job, service version, environment, workload, and target metric.
15
+ 2. Capture representative profiles under controlled and production-safe conditions.
16
+ 3. Compare baseline and candidate profiles using equivalent traffic, data shape, warm-up, and runtime settings.
17
+ 4. Attribute costs to code paths; correlate with traces, request mix, allocation rates, GC, and saturation.
18
+ 5. Change one material factor at a time and repeat the measurement.
19
+ 6. Validate user-visible latency/throughput and resource cost after the change.
20
+
21
+ ## Continuous profiling architecture
22
+ - Select agent, in-process, or eBPF-based collection according to language/runtime support, privilege boundaries, overhead, and operational model.
23
+ - Define sampling frequency, profile duration, upload cadence, symbolization, retention, and tenant isolation.
24
+ - Monitor profiler overhead, missing samples, upload failures, and storage cost.
25
+ - Restrict access to profiles: symbols, stack traces, and labels can reveal proprietary code paths or sensitive metadata.
26
+
27
+ ## Performance diagnosis
28
+ - CPU saturation: inspect hot functions, serialization, compression, cryptography, regex, and inefficient algorithms.
29
+ - Memory pressure: inspect allocation churn, retained references, caches, fragmentation, and GC behavior.
30
+ - Lock contention: identify shared mutable state, coarse locks, hot keys, and queueing around critical sections.
31
+ - I/O waits: correlate profiles with traces, storage latency, connection pools, and network retransmissions.
32
+ - Tail latency: inspect rare paths, GC pauses, lock convoys, retries, cold caches, noisy neighbors, and dependency variance.
33
+
34
+ ## Pitfalls
35
+ - Comparing profiles from different workloads or runtime configurations.
36
+ - Assuming sampled CPU time explains blocked time.
37
+ - Optimizing a hot function that contributes little to end-to-end latency.
38
+ - Ignoring profiler overhead, symbolization gaps, and production data governance.
39
+
40
+ ## Checklist
41
+ - [ ] Hypothesis, workload, baseline, and target are recorded.
42
+ - [ ] Profile type matches the suspected bottleneck.
43
+ - [ ] Samples are representative and overhead is measured.
44
+ - [ ] Findings are correlated with traces and service metrics.
45
+ - [ ] Before/after results include user impact and resource cost.
@@ -0,0 +1,44 @@
1
+ # Load, Stress, Soak Testing, and Workload Modeling
2
+
3
+ ## Workload model
4
+ Model production traffic rather than a single peak RPS number. Include arrival patterns, concurrency, request mix, payload sizes, user think time, read/write ratio, tenant skew, cache state, data volume, geographic distribution, and background jobs.
5
+
6
+ Distinguish:
7
+ - Open workload: arrivals continue independently of response time; useful for detecting queue growth and overload.
8
+ - Closed workload: a fixed population waits for responses before issuing more work; useful for concurrency-limited clients but can hide overload through coordinated omission.
9
+ - Burst workload: short high-rate arrivals and recovery periods.
10
+ - Stateful workload: realistic sessions, hot keys, cache behavior, and data growth.
11
+
12
+ ## Test types
13
+ - Baseline/load: expected steady-state and forecast traffic.
14
+ - Stress: increase load to discover saturation, queue growth, and failure boundaries.
15
+ - Spike: abrupt arrival-rate changes and autoscaling response.
16
+ - Soak: sustained load to expose leaks, resource drift, compaction, GC, and backlog accumulation.
17
+ - Capacity breakpoint: find the highest load that still meets latency/error/SLO constraints with headroom.
18
+ - Resilience: combine dependency degradation and partial failures with representative load.
19
+
20
+ ## Test validity
21
+ - Define arrival-rate model and concurrency separately.
22
+ - Prevent coordinated omission; use tools and metrics that account for delayed request scheduling.
23
+ - Validate generator capacity, network path, TLS, DNS, and client-side limits.
24
+ - Use production-like data distributions and data sizes; avoid testing only warm caches or tiny datasets.
25
+ - Include retries and background work, or explicitly isolate them.
26
+ - Warm up JITs, caches, and connection pools consistently; report warm-up and steady-state windows separately.
27
+
28
+ ## Safety and controls
29
+ - Obtain explicit authorization for production tests. Prefer isolated environments and synthetic or anonymized data.
30
+ - Set hard limits for RPS, concurrency, duration, spend, and generated data.
31
+ - Define abort thresholds for error rate, latency, saturation, queue depth, and downstream impact.
32
+ - Coordinate with on-call owners and external providers; use allowlisted test identities and clear traffic tagging.
33
+ - Ensure cleanup and data reset are safe and repeatable.
34
+
35
+ ## Reporting
36
+ Record build/configuration, infrastructure shape, dataset, workload profile, generator placement, test duration, warm-up, run IDs, and environmental differences. Report throughput, P50/P95/P99, errors/timeouts, saturation, queue depth/age, resource use, and SLO attainment. Include confidence, variance, and limitations.
37
+
38
+ ## Checklist
39
+ - [ ] Workload represents realistic request mix and arrival behavior.
40
+ - [ ] Open/closed model and coordinated omission treatment are explicit.
41
+ - [ ] Data size, skew, cache state, and background work are representative.
42
+ - [ ] Generator and network are not the bottleneck.
43
+ - [ ] Abort controls and production authorization are defined.
44
+ - [ ] Results include tail latency, errors, saturation, and queue behavior.
@@ -0,0 +1,34 @@
1
+ # Capacity Planning and Performance Modeling
2
+
3
+ ## Planning inputs
4
+ Use measured traffic and growth where available. Record average/peak arrival rates, concurrency, request mix, payload sizes, CPU time/request, memory/request, database queries/request, connection occupancy, queue arrival/service rates, storage growth, replication, retention, and failure scenarios. Mark forecasts and assumptions explicitly.
5
+
6
+ ## Capacity model
7
+ - Little's Law: L = λW, where L is average in-system work, λ is throughput, and W is average time in the system. Apply only to a stable observation window with consistent boundaries.
8
+ - CPU demand: approximate cores required as throughput × CPU-seconds/request; add measured headroom and account for throttling and burst behavior.
9
+ - Connection demand: estimate concurrency from arrival rate × connection occupancy time; validate pool limits and downstream capacity.
10
+ - Queue stability: sustained arrival rate must remain below sustainable service rate. Model burst recovery and backlog drain time.
11
+ - Storage: model logical growth, indexes, replicas, WAL/binlogs, backups, retention, compaction, and migration headroom separately.
12
+ - Network: include payload, protocol/TLS overhead, retries, replication, telemetry, and cross-zone/region traffic.
13
+
14
+ ## Bottleneck and headroom
15
+ - Find the first constrained resource across the full dependency chain; scaling stateless application replicas may simply move the bottleneck to the database or a shared service.
16
+ - Define safe operating envelopes using SLOs, not theoretical maximum throughput.
17
+ - Include N+1 failure capacity, rolling deployments, zone loss, and maintenance events where required.
18
+ - Model autoscaling detection and provisioning delay; reactive scaling cannot prevent every burst.
19
+ - Evaluate vertical, horizontal, queue-based, and workload-shaping options before introducing partitioning or sharding.
20
+
21
+ ## Sensitivity and uncertainty
22
+ Build low/base/high scenarios for growth, peak factors, request mix, retry amplification, payload size, and dependency latency.
23
+ Show which assumptions dominate the result and which measurements would reduce uncertainty.
24
+ Do not convert benchmark throughput into production capacity without accounting for workload differences, failure headroom, and SLO limits.
25
+
26
+ ## Operational planning
27
+ Define thresholds and owners for scaling, load shedding, admission control, queue limits, and capacity procurement. Review forecasts after major launches, architectural changes, and observed growth deviations.
28
+
29
+ ## Checklist
30
+ - [ ] Inputs are measured or explicitly labeled assumptions.
31
+ - [ ] End-to-end dependency constraints and failure headroom are modeled.
32
+ - [ ] Queue stability and burst recovery are included.
33
+ - [ ] Low/base/high sensitivity analysis is available.
34
+ - [ ] Scaling and load-shedding actions have owners and triggers.
@@ -0,0 +1,41 @@
1
+ # Chaos Engineering and Resilience Experiments
2
+
3
+ ## Principles
4
+ Chaos engineering is a controlled method for testing a falsifiable resilience hypothesis. It is not random disruption in production. Begin with steady-state signals that represent user-visible behavior, then introduce the smallest fault that can test the hypothesis.
5
+
6
+ ## Experiment design
7
+ 1. State the hypothesis and expected steady state.
8
+ 2. Identify target service, dependency, cohort, environment, and blast-radius boundary.
9
+ 3. Enumerate hazards, prerequisites, and possible data-integrity consequences.
10
+ 4. Define measurable abort conditions, owner, observer, and communication channel.
11
+ 5. Start in a safe environment with a small cohort; increase scope only after evidence supports it.
12
+ 6. Inject one fault class at a time and observe detection, containment, degradation, recovery, and reconciliation.
13
+ 7. Remove the fault, verify recovery, and document findings and follow-up actions.
14
+
15
+ ## Fault classes
16
+ - Network: latency, packet loss, partitions, DNS failures, connection resets.
17
+ - Dependency: slow/unavailable database, cache, queue, identity provider, or third-party API.
18
+ - Compute: process termination, CPU pressure, memory pressure, node/zone loss.
19
+ - Control plane: configuration errors, certificate expiry, deployment failure, autoscaler or scheduler disruption.
20
+ - Data path: delayed/out-of-order/duplicate messages, partial writes, replay, and stale replicas. Use non-destructive synthetic data and strict invariants.
21
+
22
+ ## Safety controls
23
+ - Explicit authorization and change window for production experiments.
24
+ - Hard scope allowlists, tenant/test cohort boundaries, concurrency and duration caps.
25
+ - Automated kill switch and independent abort monitoring.
26
+ - Exclude critical systems and irreversible operations unless separately approved and safely simulated.
27
+ - Verify backups, restore paths, and data reconciliation before experiments that can affect persistence.
28
+ - Coordinate on-call, security, platform, and vendor stakeholders.
29
+
30
+ ## What to measure
31
+ User journey success, SLO burn, latency percentiles, error rate, queue age/depth, saturation, failover time, data correctness, alert detection time, mitigation time, and recovery time. A system that remains available but corrupts state has not passed.
32
+
33
+ ## Learning loop
34
+ Convert findings into engineering work: missing timeout, unbounded retry, weak isolation, inadequate capacity, poor alerting, unsafe failover, or incomplete runbooks. Assign owners and deadlines; repeat the experiment after remediation. Track experiment coverage and validated hypotheses, not a vanity count of injected faults.
35
+
36
+ ## Checklist
37
+ - [ ] Hypothesis and steady-state metrics are defined.
38
+ - [ ] Scope, authorization, blast radius, and abort criteria are explicit.
39
+ - [ ] Data integrity and recovery are included.
40
+ - [ ] On-call owners can observe and stop the experiment.
41
+ - [ ] Findings become tracked remediation and retested controls.
@@ -0,0 +1,34 @@
1
+ # Incident Management, On-call, and Postmortems
2
+
3
+ ## Operational model
4
+ Define severity levels by user/business impact, not by component count or internal effort. Establish incident command, technical lead, communications lead, and scribe roles appropriate to incident scale. Publish escalation paths, service ownership, dependency contacts, and decision authority.
5
+
6
+ ## Detection and response
7
+ - Alert on user-visible symptoms and SLO error-budget burn; use metrics, logs, traces, and synthetic checks as complementary evidence.
8
+ - Make alerts actionable: condition, impact, owner, runbook, and escalation. Avoid paging on every transient threshold breach.
9
+ - Triage scope, affected cohorts, start time, recent changes, dependencies, and data-integrity risk.
10
+ - Stabilize first: rollback, traffic shift, disable a feature, shed load, isolate a dependency, or invoke a tested degraded mode.
11
+ - Preserve evidence and timeline. Communicate known facts, uncertainty, impact, and next update time.
12
+ - Verify recovery using user-facing signals and reconciliation, not merely healthy processes.
13
+
14
+ ## On-call sustainability
15
+ - Define primary/secondary coverage, escalation timeout, handoff, compensation/rest, and backup ownership.
16
+ - Track page volume, actionable-page ratio, toil, after-hours load, and recurring incidents.
17
+ - Runbooks should include symptoms, diagnostic queries, safe mitigations, rollback, verification, and escalation. Keep commands environment-specific and reviewed.
18
+ - Automate repetitive safe actions with guardrails and audit; do not automate irreversible actions without explicit controls.
19
+
20
+ ## Postmortems
21
+ Use blameless, evidence-based analysis. Capture impact, timeline, detection, contributing conditions, response effectiveness, what went well, where response struggled, and prioritized corrective actions with owners and due dates. Distinguish trigger from systemic contributors; avoid assigning root cause to the last person or deploy.
22
+ - Corrective actions should address prevention, detection, containment, and recovery.
23
+ - Rank actions by risk reduction and recurrence likelihood; verify completion and effectiveness.
24
+ - Share learnings across teams without exposing secrets or sensitive customer data.
25
+
26
+ ## Operational metrics
27
+ Track MTTD, time to acknowledge, time to mitigate, time to recover, recurrence, SLO impact, customer impact, and action completion. These metrics diagnose process friction; do not use them as simplistic individual performance scores.
28
+
29
+ ## Checklist
30
+ - [ ] Severity, roles, escalation, and communications are defined.
31
+ - [ ] Alerts are actionable and tied to user impact.
32
+ - [ ] Runbooks contain safe, tested mitigation and verification steps.
33
+ - [ ] Recovery includes data correctness and backlog reconciliation.
34
+ - [ ] Postmortem actions have owners, due dates, and effectiveness checks.
@@ -0,0 +1,35 @@
1
+ # DNS and Service Discovery Architecture
2
+
3
+ ## Responsibilities and boundaries
4
+ DNS resolves names to records; service discovery identifies healthy, eligible service instances and their metadata. Decide whether discovery is client-side, server-side, DNS-based, registry-backed, or provided by a platform control plane. Do not assume DNS alone provides immediate health-aware routing.
5
+
6
+ ## Naming and record design
7
+ - Define stable service names independently of ephemeral instance addresses.
8
+ - Choose A/AAAA, CNAME, SRV, TXT, and HTTPS/SVCB records according to client and protocol support.
9
+ - Document authoritative zones, delegation, split-horizon behavior, private/public namespaces, and ownership.
10
+ - Set TTLs based on change frequency, resolver behavior, failover objectives, and cache layers. TTL is not a hard upper bound on propagation because clients and resolvers may cache differently.
11
+ - Plan negative caching, NXDOMAIN behavior, stale answers, and resolver failure modes.
12
+
13
+ ## Service discovery
14
+ - Model registration, health state, metadata, endpoint eligibility, deregistration, and stale-instance cleanup.
15
+ - Prefer platform-native discovery when it meets requirements; add a registry only for a demonstrated cross-platform or control-plane need.
16
+ - Define client-side versus server-side load balancing, endpoint refresh cadence, connection reuse, and behavior when no healthy endpoint exists.
17
+ - Avoid synchronized refresh storms; use jitter, bounded caching, and backoff.
18
+ - Treat discovery data as routing input, not authorization. Enforce identity and policy at the connection and request layers.
19
+
20
+ ## Failure and security
21
+ - Consider authoritative DNS outage, resolver saturation, stale cache, split-brain registration, control-plane partition, and partial network reachability.
22
+ - Use redundant resolvers and authoritative services; test recursive resolver dependencies and local caching behavior.
23
+ - Protect zone administration with least privilege, MFA, audit, DNSSEC where appropriate, and controlled change processes.
24
+ - Defend against cache poisoning, dangling records/subdomain takeover, unauthorized zone transfer, and SSRF through attacker-controlled DNS answers. Revalidate resolved destinations where threat models require it.
25
+ - Do not use DNS as a substitute for service identity, mTLS, or application authorization.
26
+
27
+ ## Operations
28
+ Track query volume, latency, SERVFAIL/NXDOMAIN, cache hit ratio, registration churn, stale endpoint rate, and resolution failures by resolver and zone. Test failover and record changes from real client resolver paths.
29
+
30
+ ## Checklist
31
+ - [ ] Namespaces, zones, ownership, and record lifecycle are documented.
32
+ - [ ] TTL and negative-cache behavior match failover objectives.
33
+ - [ ] Discovery freshness, health, and empty-endpoint behavior are defined.
34
+ - [ ] Resolver and control-plane failure modes are tested.
35
+ - [ ] DNS administration and destination validation are secured.
@@ -0,0 +1,31 @@
1
+ # Load Balancing and Global Traffic Management
2
+
3
+ ## Traffic layers
4
+ Separate global traffic steering (DNS, anycast, or global load balancer), regional ingress, application load balancing, and service-to-service balancing. Document which layer owns health decisions, retries, TLS termination, affinity, and routing policy.
5
+
6
+ ## Algorithms and selection
7
+ - Compare round robin, weighted round robin, least connections, least response time, consistent hashing, and locality-aware routing against workload characteristics.
8
+ - Account for long-lived connections, uneven request cost, HTTP/2 and HTTP/3 multiplexing, connection pools, and slow-start behavior.
9
+ - Use weights for controlled rollout and capacity-aware distribution; avoid assuming equal request counts imply equal resource cost.
10
+ - Define session affinity only when required; explain failover and rebalancing behavior when affinity breaks.
11
+ - Prevent retry amplification by assigning retry ownership, deadlines, retry budgets, and idempotency requirements.
12
+
13
+ ## Health and failover
14
+ - Distinguish liveness, readiness, synthetic user-journey checks, and dependency health. A process responding to a health endpoint may still be unable to serve traffic correctly.
15
+ - Use passive health signals with active probes carefully; define thresholds, hysteresis, and recovery slow start to avoid flapping.
16
+ - Specify zone and region failure policy, drain behavior, connection termination, and failback criteria.
17
+ - Avoid removing every instance simultaneously due to a shared dependency failure; use dependency-aware and failure-domain-aware policies.
18
+ - Define behavior when health state is unavailable or inconsistent.
19
+
20
+ ## Global traffic management
21
+ Compare DNS-based steering, anycast, and proxy-based global load balancing. Consider resolver caching, client location accuracy, BGP convergence, TLS, data residency, active-active data consistency, and regional control-plane dependencies. Route users to a region that can actually serve their data and authorization context.
22
+
23
+ ## Security and operations
24
+ Define trusted proxy boundaries, forwarded-header normalization, source IP preservation, DDoS controls, TLS policy, certificate lifecycle, and administrative access. Monitor distribution skew, queueing, connection saturation, health transitions, failover time, rejected traffic, and per-region SLOs.
25
+
26
+ ## Checklist
27
+ - [ ] Each routing layer has explicit ownership and policy.
28
+ - [ ] Algorithms match connection and request-cost patterns.
29
+ - [ ] Health checks reflect serving ability without causing correlated ejection.
30
+ - [ ] Retry, drain, failover, and failback behavior are bounded and tested.
31
+ - [ ] Global routing accounts for data locality, consistency, and resolver behavior.
@@ -0,0 +1,34 @@
1
+ # CDN and Edge Caching Architecture
2
+
3
+ ## Cache placement
4
+ Choose browser, application, reverse-proxy, CDN point-of-presence, and regional cache layers based on content ownership, latency objectives, freshness, privacy, and invalidation needs. A cache is a derived copy, not automatically a source of truth.
5
+
6
+ ## Cache key and policy
7
+ - Define cache keys from normalized scheme, host, path, query parameters, selected headers, content encoding, and tenant context.
8
+ - Avoid accidental variation explosion and cache poisoning caused by inconsistent normalization.
9
+ - Set Cache-Control, s-maxage, stale-while-revalidate, stale-if-error, and Vary policies deliberately; verify CDN-specific semantics.
10
+ - Separate public immutable assets from personalized, authenticated, and sensitive responses. Never share-cache user-specific content without a proven safe key and authorization model.
11
+ - Use versioned asset URLs for immutable content and explicit TTL/invalidation for mutable content.
12
+
13
+ ## Invalidation and consistency
14
+ - Prefer versioned keys or bounded TTLs where possible; use purge APIs for urgent invalidation with documented propagation guarantees.
15
+ - Define invalidation fan-out, retry, ordering, and observability. Purge success at the control plane does not prove every edge has evicted an object.
16
+ - Use surrogate keys/tags where supported to invalidate related content groups.
17
+ - Decide whether stale content is acceptable during origin failure; encode this per content class.
18
+ - Prevent stampedes with request coalescing, stale-while-revalidate, jittered refresh, and origin shielding.
19
+
20
+ ## Dynamic and API traffic
21
+ Evaluate edge compute for authentication prechecks, redirects, routing, lightweight personalization, and request normalization only when data locality, runtime limits, and debugging support are acceptable. Keep authorization and sensitive business invariants at trusted services unless the edge has equivalent policy and identity guarantees.
22
+
23
+ ## Security
24
+ Protect origins from direct access and bypass; authenticate origin pulls; validate host and forwarding headers; defend against cache deception, poisoning, request smuggling, and cross-tenant cache leakage. Define WAF, bot management, rate limits, TLS, and DDoS responsibilities across provider and origin.
25
+
26
+ ## Economics and observability
27
+ Model cache hit ratio by bytes and requests, origin offload, egress, purge volume, request fees, shield traffic, and regional distribution. Monitor hit/miss, stale responses, origin latency, revalidation, purge lag, cache-key cardinality, and origin errors.
28
+
29
+ ## Checklist
30
+ - [ ] Cacheability and privacy classes are explicit.
31
+ - [ ] Cache keys include every representation/tenant dimension and exclude unsafe noise.
32
+ - [ ] Freshness, invalidation, and stale-serving policies are documented.
33
+ - [ ] Origin protection and cache-bypass paths are secured.
34
+ - [ ] Hit ratio, origin offload, purge lag, and cost are measured.
@@ -0,0 +1,34 @@
1
+ # Cloud Networking and Connectivity Architecture
2
+
3
+ ## Network model
4
+ Document accounts/projects, regions, availability zones, VPC/VNet boundaries, subnets, route tables, gateways, firewalls, DNS, private endpoints, and shared network services. Distinguish control-plane configuration from packet data paths.
5
+
6
+ ## Addressing and segmentation
7
+ - Establish non-overlapping CIDR plans across environments, regions, acquisitions, and partner networks.
8
+ - Segment by trust, workload, environment, and operational ownership; avoid relying on subnet boundaries as the only authorization control.
9
+ - Define ingress, egress, east-west, and management planes with explicit default-deny policies where practical.
10
+ - Plan IPv4 exhaustion, IPv6 dual-stack support, NAT port capacity, ephemeral ports, and overlapping-address translation.
11
+ - Document route propagation, longest-prefix matching, asymmetric paths, and blackhole behavior.
12
+
13
+ ## Connectivity patterns
14
+ Compare public endpoints, NAT, VPN, dedicated private connectivity, peering, transit hubs, private service endpoints, and service-mesh connectivity. Evaluate bandwidth, latency, resiliency, routing limits, encryption, DNS integration, and cost. Peering is not transitive unless the platform explicitly provides a transit design.
15
+
16
+ ## Availability and scale
17
+ - Design redundant gateways and paths across failure domains; define failover convergence and route ownership.
18
+ - Model throughput, packets per second, connection tracking, NAT port exhaustion, MTU, fragmentation, and cross-zone/region egress.
19
+ - Avoid central inspection bottlenecks and single-region transit dependencies.
20
+ - Test asymmetric routing, failover, DNS resolution, and effective policy from workload subnets.
21
+ - Treat network policy and firewall changes as versioned infrastructure with review, staged rollout, and rollback.
22
+
23
+ ## Security and governance
24
+ Apply least privilege to network administration, log flow decisions, restrict management access, and use workload identity and encryption in addition to network location. Review broad CIDRs, unrestricted egress, exposed control planes, stale rules, and shadow connectivity. Keep segmentation aligned with data classification and threat models.
25
+
26
+ ## Operations
27
+ Monitor flow logs, route changes, drops, retransmits, latency, connection tracking, NAT utilization, gateway saturation, and cross-zone/region transfer. Maintain diagrams generated from deployed state and reconcile drift.
28
+
29
+ ## Checklist
30
+ - [ ] Address plan avoids overlap and includes IPv6/NAT capacity.
31
+ - [ ] Trust zones and ingress/egress policies are explicit.
32
+ - [ ] Routing, DNS, MTU, and asymmetric paths are validated.
33
+ - [ ] Redundancy and failover are tested across independent failure domains.
34
+ - [ ] Flow visibility, change control, and cost allocation are available.
@@ -0,0 +1,33 @@
1
+ # Edge Computing Architecture
2
+
3
+ ## When edge is justified
4
+ Use edge computing when latency, intermittent connectivity, data locality, bandwidth reduction, physical proximity, or local autonomy materially changes the solution. Compare with regional cloud deployment and client-side execution before adding distributed edge operations.
5
+
6
+ ## Placement and topology
7
+ Define device, site, metro, regional, and cloud tiers. Assign workloads to tiers based on latency budgets, data gravity, compute limits, connectivity, and failure domains. Document which tier is authoritative for identity, configuration, state, and policy.
8
+
9
+ ## Workload lifecycle
10
+ - Package immutable, signed workloads with reproducible builds and versioned configuration.
11
+ - Define enrollment, attestation, provisioning, staged rollout, health, rollback, decommissioning, and secure recovery.
12
+ - Support offline operation with bounded local queues, durable state, clock uncertainty, and explicit conflict/reconciliation rules.
13
+ - Design for intermittent links, bandwidth caps, delayed updates, duplicate delivery, and long periods without control-plane contact.
14
+ - Keep secrets short-lived where possible; support rotation and revocation during disconnection with a documented risk window.
15
+
16
+ ## Data and consistency
17
+ Classify data as local ephemeral, locally durable, replicated, or cloud-authoritative. Define write ownership, synchronization direction, conflict resolution, retention, deletion propagation, and behavior after device replacement. Do not assume eventual connectivity guarantees automatic convergence.
18
+
19
+ ## Security and fleet management
20
+ Use hardware-backed identity/secure boot where available, signed artifacts, least-privilege workload identities, encrypted storage, tamper-aware logs, secure update channels, and remote revocation. Treat physical access and device compromise as realistic threats. Bound local privileges and lateral movement.
21
+
22
+ ## Reliability and observability
23
+ Define local degraded modes and safe shutdown behavior. Buffer telemetry during disconnection with bounded storage and priority; avoid unbounded replay storms after reconnect. Monitor fleet version skew, last-seen time, update success, resource pressure, synchronization lag, and policy compliance.
24
+
25
+ ## Economics
26
+ Include edge hardware, replacement, connectivity, field support, remote hands, energy, local storage, fleet management, and lifecycle costs—not only cloud compute savings.
27
+
28
+ ## Checklist
29
+ - [ ] Edge placement solves a measured latency, locality, or connectivity constraint.
30
+ - [ ] Offline behavior, local authority, and reconciliation are explicit.
31
+ - [ ] Artifact signing, device identity, updates, and revocation are designed.
32
+ - [ ] Fleet observability and bounded telemetry replay are available.
33
+ - [ ] Full lifecycle and field-support costs are modeled.
@@ -0,0 +1,30 @@
1
+ # Storage Infrastructure and Storage Systems
2
+
3
+ ## Workload characterization
4
+ Classify data by access pattern, durability, latency, throughput, IOPS, block size, concurrency, growth, retention, sharing, locality, and recovery objectives. Separate application database semantics from underlying block, file, and object storage guarantees.
5
+
6
+ ## Storage types
7
+ - Block storage: low-level volumes for databases and filesystems; evaluate IOPS, throughput, latency, queue depth, burst credits, multi-attach semantics, snapshots, and resize behavior.
8
+ - File storage: shared hierarchical namespace; evaluate metadata operations, locking, consistency, throughput, client caching, and concurrent writers.
9
+ - Object storage: API-addressed objects; evaluate durability, consistency model, multipart uploads, lifecycle tiers, versioning, retention lock, event notifications, and request cost.
10
+ - Ephemeral/local NVMe: high performance with host or instance lifecycle coupling; define replication and rebuild behavior.
11
+ - Distributed storage: understand replication/erasure coding, quorum, failure domains, repair bandwidth, rebalancing, and correlated failure risks.
12
+
13
+ ## Durability and recovery
14
+ Distinguish durability from availability and backup. Define replication scope, versioning, immutable backups, retention, encryption keys, restore testing, point-in-time recovery, and deletion protection. Validate recovery time and data integrity, not only snapshot creation. Understand consistency and atomicity boundaries across objects, files, and volumes.
15
+
16
+ ## Performance and capacity
17
+ Model IOPS, throughput, queue depth, request size, latency percentiles, metadata hot spots, cache hit ratio, compaction, garbage collection, and rebuild overhead. Include burst limits, throttling, noisy neighbors, and performance degradation during repair or snapshots. Benchmark with production-like data and concurrency.
18
+
19
+ ## Cost and lifecycle
20
+ Include provisioned capacity, provisioned performance, operations/requests, snapshots, replication, retrieval, egress, minimum retention, and early deletion fees. Define tiering, lifecycle transitions, orphan detection, capacity alerts, and ownership.
21
+
22
+ ## Security
23
+ Use encryption at rest and in transit, scoped identities, private endpoints, least-privilege access, access logging, key lifecycle, tenant isolation, and secure deletion. For object storage, explicitly review public access, bucket policies, presigned URL expiry, and cross-account access.
24
+
25
+ ## Checklist
26
+ - [ ] Storage class matches access pattern and latency/throughput needs.
27
+ - [ ] Durability, availability, replication, backup, and restore are distinguished.
28
+ - [ ] IOPS, throughput, queue depth, burst, and rebuild behavior are modeled.
29
+ - [ ] Lifecycle, retention, encryption, and tenant access policies are explicit.
30
+ - [ ] Restore and failure-domain recovery are tested.