system-design-primer-skill 1.2.0 → 2.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (95) hide show
  1. package/README.md +106 -60
  2. package/SKILL.md +417 -93
  3. package/VERSION +1 -1
  4. package/package.json +44 -3
  5. package/references/10-workflows-and-checklists.md +1 -1
  6. package/references/13-architecture-intelligence.md +91 -0
  7. package/references/14-api-and-data-contracts.md +80 -0
  8. package/references/15-failure-analysis.md +84 -0
  9. package/references/16-multi-agent-architecture.md +76 -0
  10. package/references/17-agent-harness-and-runtime.md +87 -0
  11. package/references/18-mcp-architecture.md +67 -0
  12. package/references/19-agent-security-and-permissions.md +74 -0
  13. package/references/20-agent-state-and-memory.md +72 -0
  14. package/references/21-agent-evaluation-and-observability.md +90 -0
  15. package/references/22-kubernetes-and-container-platforms.md +29 -0
  16. package/references/23-service-mesh-and-traffic-management.md +21 -0
  17. package/references/24-multi-region-and-disaster-recovery.md +24 -0
  18. package/references/25-platform-engineering-and-developer-experience.md +26 -0
  19. package/references/26-finops-and-cloud-cost.md +29 -0
  20. package/references/27-production-observability-and-slo.md +29 -0
  21. package/references/28-event-streaming-and-kafka-pulsar.md +27 -0
  22. package/references/29-event-sourcing-and-cqrs.md +23 -0
  23. package/references/30-change-data-capture-and-debezium.md +27 -0
  24. package/references/31-data-lakehouse-and-stream-batch.md +24 -0
  25. package/references/32-workflow-orchestration-and-temporal.md +24 -0
  26. package/references/33-schema-registry-and-event-governance.md +22 -0
  27. package/references/34-postgresql-internals-and-performance.md +43 -0
  28. package/references/35-mysql-innodb-and-performance.md +27 -0
  29. package/references/36-sharding-and-distributed-sql.md +22 -0
  30. package/references/37-consensus-and-distributed-transactions.md +25 -0
  31. package/references/38-search-architecture-elasticsearch-opensearch.md +24 -0
  32. package/references/39-vector-databases-and-hybrid-retrieval.md +31 -0
  33. package/references/40-zero-downtime-database-migrations.md +27 -0
  34. package/references/41-api-gateway-and-edge-architecture.md +23 -0
  35. package/references/42-rest-api-design-and-lifecycle.md +23 -0
  36. package/references/43-graphql-architecture-and-governance.md +16 -0
  37. package/references/44-grpc-and-service-contracts.md +16 -0
  38. package/references/45-websocket-sse-and-realtime-systems.md +16 -0
  39. package/references/46-realtime-collaboration-ot-crdt.md +16 -0
  40. package/references/47-api-security-rate-limiting-and-abuse.md +16 -0
  41. package/references/48-oauth-oidc-and-passkeys.md +20 -0
  42. package/references/49-iam-rbac-abac-rebac.md +16 -0
  43. package/references/50-zero-trust-and-workload-identity.md +16 -0
  44. package/references/51-secrets-kms-pki-and-key-management.md +16 -0
  45. package/references/52-supply-chain-security-sbom-slsa.md +16 -0
  46. package/references/53-threat-modeling-stride-and-attack-trees.md +16 -0
  47. package/references/54-privacy-engineering-and-data-governance.md +16 -0
  48. package/references/55-opentelemetry-and-instrumentation.md +50 -0
  49. package/references/56-distributed-tracing-and-context-propagation.md +43 -0
  50. package/references/57-continuous-profiling-and-performance-diagnostics.md +45 -0
  51. package/references/58-load-stress-soak-testing-and-workload-modeling.md +44 -0
  52. package/references/59-capacity-planning-and-performance-modeling.md +34 -0
  53. package/references/60-chaos-engineering-and-resilience-experiments.md +41 -0
  54. package/references/61-incident-management-and-postmortems.md +34 -0
  55. package/references/62-dns-and-service-discovery.md +35 -0
  56. package/references/63-load-balancing-and-global-traffic-management.md +31 -0
  57. package/references/64-cdn-and-edge-caching.md +34 -0
  58. package/references/65-cloud-networking-and-connectivity.md +34 -0
  59. package/references/66-edge-computing-architecture.md +33 -0
  60. package/references/67-storage-infrastructure-and-storage-systems.md +30 -0
  61. package/references/68-tcp-quic-and-network-performance.md +27 -0
  62. package/references/69-task-routing-and-evidence.md +58 -0
  63. package/scripts/install.ps1 +10 -1
  64. package/scripts/install.sh +13 -6
  65. package/scripts/validate.py +133 -0
  66. package/templates/agent-evaluation-plan-template.md +57 -0
  67. package/templates/agent-security-review-template.md +46 -0
  68. package/templates/agent-system-design-template.md +74 -0
  69. package/templates/api-architecture-design-template.md +33 -0
  70. package/templates/api-security-and-rate-limit-review-template.md +29 -0
  71. package/templates/architecture-decision-matrix-template.md +48 -0
  72. package/templates/architecture-review-scorecard-template.md +36 -0
  73. package/templates/cdc-data-pipeline-template.md +41 -0
  74. package/templates/cdn-and-edge-strategy-template.md +36 -0
  75. package/templates/chaos-experiment-template.md +39 -0
  76. package/templates/cloud-cost-model-template.md +37 -0
  77. package/templates/database-architecture-review-template.md +33 -0
  78. package/templates/edge-computing-readiness-template.md +36 -0
  79. package/templates/event-sourcing-cqrs-template.md +41 -0
  80. package/templates/event-streaming-design-template.md +47 -0
  81. package/templates/failure-analysis-template.md +36 -0
  82. package/templates/identity-and-access-architecture-template.md +29 -0
  83. package/templates/incident-postmortem-template.md +39 -0
  84. package/templates/multi-region-disaster-recovery-template.md +43 -0
  85. package/templates/network-architecture-design-template.md +38 -0
  86. package/templates/observability-architecture-template.md +52 -0
  87. package/templates/performance-test-plan-template.md +52 -0
  88. package/templates/privacy-data-governance-template.md +24 -0
  89. package/templates/production-readiness-checklist-template.md +52 -0
  90. package/templates/realtime-system-design-template.md +28 -0
  91. package/templates/security-threat-model-template.md +26 -0
  92. package/templates/sharding-distributed-sql-design-template.md +34 -0
  93. package/templates/storage-architecture-design-template.md +35 -0
  94. package/templates/supply-chain-security-review-template.md +20 -0
  95. package/templates/zero-downtime-database-migration-template.md +32 -0
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "system-design-primer-skill",
3
- "version": "1.2.0",
4
- "description": "A portable System Design Agent Skill for architecture design, reviews, scalability, reliability, and AI systems.",
3
+ "version": "2.3.0",
4
+ "description": "A portable System Design Agent Skill with evidence-first task routing, architecture design and review, distributed systems, security, data platforms, AI systems, observability, performance, networking, and resilience.",
5
5
  "license": "CC-BY-4.0",
6
6
  "repository": {
7
7
  "type": "git",
@@ -20,7 +20,48 @@
20
20
  "codex",
21
21
  "claude-code",
22
22
  "cursor",
23
- "github-copilot"
23
+ "github-copilot",
24
+ "deepseek-harness",
25
+ "api-gateway",
26
+ "rest-api",
27
+ "graphql",
28
+ "grpc",
29
+ "websocket",
30
+ "real-time-systems",
31
+ "api-security",
32
+ "oauth2",
33
+ "openid-connect",
34
+ "passkeys",
35
+ "iam",
36
+ "zero-trust",
37
+ "workload-identity",
38
+ "kms",
39
+ "pki",
40
+ "sbom",
41
+ "slsa",
42
+ "threat-modeling",
43
+ "privacy-engineering",
44
+ "opentelemetry",
45
+ "distributed-tracing",
46
+ "continuous-profiling",
47
+ "performance-testing",
48
+ "capacity-planning",
49
+ "chaos-engineering",
50
+ "incident-management",
51
+ "sre",
52
+ "dns",
53
+ "service-discovery",
54
+ "load-balancing",
55
+ "cdn",
56
+ "edge-computing",
57
+ "cloud-networking",
58
+ "storage-architecture",
59
+ "tcp",
60
+ "quic",
61
+ "architecture-evaluation",
62
+ "evidence-based-review",
63
+ "task-routing",
64
+ "system-design-evals"
24
65
  ],
25
66
  "files": [
26
67
  "SKILL.md",
@@ -64,7 +64,7 @@ Severity must be evidence-based. Include likelihood and impact separately when u
64
64
 
65
65
  ## 4. Decision matrix
66
66
 
67
- Compare candidate options using criteria relevant to the workload: correctness, latency, throughput, availability, consistency, complexity, operational burden, cost, migration risk, ecosystem, team familiarity, and vendor lock-in. Weight criteria only when the user supplies priorities or explain the weights as assumptions.
67
+ Use the structured architecture decision process in `13-architecture-intelligence.md`. Separate hard constraints from weighted preferences; compare the simplest viable baseline with credible alternatives; assess performance, consistency, reliability, security, TCO, operational complexity, team capability, migration/exit cost, and ecosystem risk. Attach evidence and confidence to each assessment, test sensitivity to uncertain assumptions, and record revisit triggers in an ADR. Never let an aggregate score compensate for a violated hard constraint.
68
68
 
69
69
  ## 5. Architecture debt and evolution triggers
70
70
 
@@ -0,0 +1,91 @@
1
+ # Architecture intelligence: decisions and evidence-based reviews
2
+
3
+ Use this reference when selecting an architecture, comparing technology options, or conducting a structured architecture review. The goal is a defensible decision under stated constraints—not a universal ranking of patterns or vendors.
4
+
5
+ ## 1. Decision process
6
+
7
+ 1. Frame the decision and the business outcome it enables.
8
+ 2. Record hard constraints separately from preferences: compliance, data residency, latency ceiling, budget ceiling, team skills, existing contracts, and delivery deadline.
9
+ 3. Define workload shape and critical paths using measured or explicitly assumed inputs.
10
+ 4. Identify viable options, including the simplest baseline and the option to defer the decision.
11
+ 5. Eliminate options that violate hard constraints.
12
+ 6. Compare remaining options against criteria that matter to this workload.
13
+ 7. Run sensitivity analysis: identify which assumptions could reverse the ranking.
14
+ 8. Record the decision, rejected alternatives, risks, validation plan, and revisit triggers in an ADR.
15
+
16
+ Do not start with a preferred technology and retrofit requirements to justify it.
17
+
18
+ ## 2. Architecture decision matrix
19
+
20
+ Use a qualitative rating (Strong / Adequate / Weak / Unknown) or a numeric scale only when the team defines what each score means. Never imply false precision. Suggested dimensions:
21
+
22
+ | Dimension | Questions and evidence |
23
+ |---|---|
24
+ | Performance and scalability | Does it meet measured peak throughput, tail latency, concurrency, and growth needs? What is the bottleneck and scaling unit? |
25
+ | Consistency and data integrity | Can it enforce business invariants, transaction boundaries, ordering, idempotency, and recovery semantics? |
26
+ | Reliability and availability | What are failure domains, dependency assumptions, degraded modes, RPO/RTO, and operational SLOs? |
27
+ | Security and compliance | Does it support required identity, authorization, isolation, encryption, audit, residency, and retention? |
28
+ | Total cost of ownership | Include infrastructure, licenses, egress, observability, support, on-call, migration, and engineering time. |
29
+ | Operational complexity | What must be deployed, upgraded, monitored, backed up, restored, and debugged? Who owns it? |
30
+ | Team capability | Does the team have the skills and capacity to operate it safely? What training or hiring is required? |
31
+ | Migration and exit cost | How difficult is adoption, data movement, rollback, portability, and eventual replacement? |
32
+ | Ecosystem and delivery risk | Are integrations, support lifecycle, maturity, and vendor dependencies acceptable? |
33
+
34
+ ### Weighting rules
35
+
36
+ - Use weights only when stakeholders provide priorities or the architect clearly labels them as provisional assumptions.
37
+ - Record the reason for each weight and test whether small changes alter the preferred option.
38
+ - A hard constraint is a gate, not a weighted preference. A compliance failure cannot be compensated for by a high performance score.
39
+ - Unknown is not zero and must not silently become a favorable score. Identify the experiment or evidence needed.
40
+ - Include confidence and evidence references beside each rating.
41
+ - Prefer a short written rationale over an aggregate score when options are close or evidence is weak.
42
+
43
+ ### Decision output
44
+
45
+ State: recommended option; viable alternatives; constraints satisfied; material trade-offs; assumptions; evidence gaps; implementation and migration plan; rollback; and measurable revisit triggers.
46
+
47
+ ## 3. Architecture review scorecard
48
+
49
+ Assess each dimension as **Pass**, **Needs work**, **Fail**, or **Not assessed**. Attach evidence and business impact. A score without evidence is not a finding.
50
+
51
+ | Dimension | Review questions |
52
+ |---|---|
53
+ | Architecture and boundaries | Are responsibilities, dependencies, ownership, and sync/async boundaries explicit? Is complexity justified? |
54
+ | Data integrity | Are source of truth, invariants, transactions, concurrency, idempotency, and schema evolution correct? |
55
+ | Scalability and performance | Are workload assumptions credible? Are hotspots, tail latency, capacity limits, and resource bounds measured? |
56
+ | Reliability and recovery | Are failure domains, timeouts, retries, overload, degradation, backups, restore, RPO, and RTO addressed? |
57
+ | Security and privacy | Are identity, authorization, tenant isolation, secrets, data minimization, and threat boundaries enforced? |
58
+ | Observability and operations | Can operators detect, diagnose, mitigate, and learn from failures? Are ownership and runbooks clear? |
59
+ | Maintainability and delivery | Are contracts, tests, deployment, rollback, dependency lifecycle, and team ownership sustainable? |
60
+ | Cost efficiency | Are cost drivers visible and bounded? Are cost/performance trade-offs supported by workload evidence? |
61
+
62
+ ### Finding quality
63
+
64
+ Every material finding includes:
65
+ - Status: Confirmed, Likely, or Unknown.
66
+ - Evidence: file path and symbol/configuration, runtime metric, test, trace, or reproducible observation.
67
+ - Impact: user/business consequence and affected boundary.
68
+ - Trigger or conditions: when the issue manifests.
69
+ - Severity: Critical, High, Medium, or Low, based on impact and likelihood.
70
+ - Recommendation: a proportionate fix and alternatives.
71
+ - Validation: test, measurement, or operational signal that proves the fix.
72
+
73
+ Do not average scorecard dimensions into a single health percentage. A critical security or data-integrity failure must remain visible and cannot be diluted by strong scores elsewhere.
74
+
75
+ ## 4. Trade-off and sensitivity analysis
76
+
77
+ For each leading option, ask:
78
+ - Which workload assumption has the greatest effect on the decision?
79
+ - What happens at 2x or 10x traffic, during a burst, or after one failure domain is lost?
80
+ - What happens if the team cannot hire or train the required operator?
81
+ - What are the recurring and one-time costs, including migration and exit?
82
+ - Which requirement could make the option unacceptable?
83
+ - What is the cheapest experiment that would reduce uncertainty?
84
+
85
+ Use ranges for uncertain costs and capacity. Do not invent vendor benchmarks. Compare alternatives under the same workload and operational assumptions.
86
+
87
+ ## 5. Evolution triggers
88
+
89
+ Define observable thresholds rather than arbitrary user-count milestones. Examples include sustained p99 budget exhaustion, database saturation after query tuning, replica lag violating freshness objectives, queue age exceeding business deadlines, recovery tests missing RTO, or a stable team/domain boundary requiring independent deployment.
90
+
91
+ Revisit the decision when a trigger is met, not merely because a newer technology exists.
@@ -0,0 +1,80 @@
1
+ # API and data contract design
2
+
3
+ Treat public APIs, internal service APIs, events, database schemas, and cached representations as contracts with consumers and compatibility windows. Design them around business capabilities and data ownership rather than exposing persistence internals.
4
+
5
+ ## 1. Contract definition
6
+
7
+ For each API operation or event, document:
8
+ - Purpose, owner, consumers, and trust boundary.
9
+ - Method/topic, resource or command semantics, and stable identifiers.
10
+ - Request/response or event schema, required/optional fields, validation, and size limits.
11
+ - Authentication, authorization, tenant scope, and data classification.
12
+ - Error taxonomy, retryability, timeout/deadline, and rate limits.
13
+ - Idempotency, ordering, concurrency, and duplicate handling.
14
+ - Pagination/filtering, consistency and freshness expectations.
15
+ - Versioning, deprecation policy, compatibility window, and telemetry.
16
+
17
+ Prefer explicit business commands for state transitions. Do not expose internal database table layouts as the external API contract.
18
+
19
+ ## 2. HTTP and RPC semantics
20
+
21
+ Use HTTP method semantics consistently, but do not confuse protocol semantics with business idempotency. A payment-creation POST, for example, needs a persisted idempotency key and operation result. Reusing a key with a different normalized payload should be rejected or follow a documented conflict rule.
22
+
23
+ Use stable error codes and structured error bodies. Distinguish validation failures, authorization failures, conflicts, rate limits, transient dependency failures, and unknown outcomes. Avoid leaking stack traces or sensitive internals.
24
+
25
+ For pagination, prefer opaque cursor pagination for large, changing datasets where offset drift is problematic. Bind cursors to the tenant, authorization scope, filter, sort order, and snapshot semantics; integrity-protect them. Set maximum page sizes.
26
+
27
+ ## 3. Versioning and compatibility
28
+
29
+ Version only when needed by a clear compatibility policy. Common approaches include URI, header, or media-type versioning; choose one deliberately and document how long versions coexist.
30
+
31
+ Prefer additive, backward-compatible changes:
32
+ - Add optional fields with safe defaults.
33
+ - Do not change field meaning, units, nullability, ordering, or enum behavior silently.
34
+ - Consumers should tolerate unknown fields where the format permits.
35
+ - Producers should not emit new enum values until consumers can safely handle them.
36
+ - Removing or changing a field requires usage evidence, a deprecation window, migration guidance, and a removal date.
37
+ - Breaking changes require a new contract version or an explicitly coordinated migration.
38
+
39
+ Compatibility must be tested across producer and consumer versions, not only against the latest schema.
40
+
41
+ ## 4. Data ownership and source of truth
42
+
43
+ Assign one authoritative owner for each business entity or invariant. Other services should access owned data through contracts, events, or explicitly governed read models—not by writing directly into another service's private tables.
44
+
45
+ For every entity, define:
46
+ - Stable identity and uniqueness constraints.
47
+ - Authoritative writer and permitted mutation paths.
48
+ - Transaction boundary and invariants.
49
+ - Lifecycle, retention, deletion, and audit requirements.
50
+ - Read models, replication lag, and acceptable staleness.
51
+ - Event publication and reconciliation strategy.
52
+
53
+ A shared database can be appropriate for a modular monolith, but table ownership and module boundaries should still be explicit.
54
+
55
+ ## 5. Schema and event evolution
56
+
57
+ Use expand-migrate-contract for online schema changes:
58
+ 1. Expand: add compatible schema and code paths.
59
+ 2. Migrate: backfill or dual-read with progress and reconciliation checks.
60
+ 3. Switch: move traffic or writers after compatibility is proven.
61
+ 4. Contract: remove obsolete fields/paths only after all consumers have migrated.
62
+
63
+ Avoid uncoordinated dual writes. Prefer transactional outbox/inbox or CDC where appropriate; define deduplication, ordering, replay, and poison-message behavior. Event schemas need stable event IDs, event type/version, occurred-at time, aggregate identity, and trace/correlation context where useful.
64
+
65
+ ## 6. Contract validation
66
+
67
+ Use schema validation, consumer-driven contract tests, compatibility checks in CI, migration tests against representative data, and end-to-end tests for critical business invariants. Include negative cases: malformed payloads, unknown fields/enum values, duplicate requests, replay, stale versions, authorization mismatch, and partial rollout.
68
+
69
+ ## 7. Review checklist
70
+
71
+ - [ ] Every contract has an owner and identified consumers.
72
+ - [ ] Authentication, authorization, and tenant scope are enforced server-side.
73
+ - [ ] Idempotency and concurrency semantics are explicit for writes.
74
+ - [ ] Errors distinguish retryable, terminal, conflict, and ambiguous outcomes.
75
+ - [ ] Pagination, filtering, payload size, and rate limits are bounded.
76
+ - [ ] Versioning and deprecation windows are documented.
77
+ - [ ] Data ownership and source of truth are unambiguous.
78
+ - [ ] Schema evolution supports old and new versions during rollout.
79
+ - [ ] Contract, migration, and compatibility tests run in CI.
80
+ - [ ] Sensitive fields are minimized and redacted from telemetry.
@@ -0,0 +1,84 @@
1
+ # Failure analysis and resilience design
2
+
3
+ Analyze failures across the full request, event, and operational lifecycle. A component diagram is not a failure model. Identify how faults propagate, what users observe, how the system degrades, and how it recovers.
4
+
5
+ ## 1. Failure-analysis sequence
6
+
7
+ 1. Identify critical business journeys, assets, and invariants.
8
+ 2. Enumerate dependencies and failure modes: application, database, network, identity, queue, cache, provider, operator, region, and deployment.
9
+ 3. Record trigger, detection signal, affected scope, and blast radius.
10
+ 4. Trace propagation: retries, fan-out, shared pools, queues, locks, and common-mode dependencies.
11
+ 5. Define prevention, containment, graceful degradation, and recovery.
12
+ 6. Specify reconciliation for ambiguous or partial outcomes.
13
+ 7. Define validation through fault injection, game days, restore drills, and incident exercises.
14
+
15
+ ## 2. Failure mode register
16
+
17
+ For each failure mode capture:
18
+
19
+ | Field | Description |
20
+ |---|---|
21
+ | Component / dependency | The affected system and owner |
22
+ | Failure mode | Timeout, crash, stale data, overload, corruption, credential expiry, bad deploy, etc. |
23
+ | Trigger / preconditions | Conditions under which it occurs |
24
+ | Detection | Metric, log, trace, alert, synthetic check, or user report |
25
+ | Blast radius | Users, tenants, regions, data, and downstream systems affected |
26
+ | User/business impact | Failed, delayed, duplicated, stale, or incorrect operation |
27
+ | Containment | Bulkhead, rate limit, circuit breaker, isolation, load shedding, feature disablement |
28
+ | Degraded behavior | Safe fallback, read-only mode, queued work, cached response, or explicit unavailability |
29
+ | Recovery | Restart, failover, replay, restore, rollback, reconciliation |
30
+ | Validation | Test or exercise and expected result |
31
+
32
+ ## 3. Blast-radius analysis
33
+
34
+ Check:
35
+ - Shared database, cache, queue, credentials, network, DNS, identity, and control-plane dependencies.
36
+ - Retry storms and fan-out amplification.
37
+ - Connection/thread/worker pool exhaustion.
38
+ - Noisy tenants and hot partitions.
39
+ - Region/AZ failure and common-mode dependencies.
40
+ - Deployment and schema compatibility across mixed versions.
41
+ - Correlated failures from shared providers or software supply chain.
42
+
43
+ Bound queues, concurrency, payload sizes, retries, cache memory, and per-tenant resource use. A retry is additional load; use deadlines, exponential backoff with jitter, retry budgets, and idempotency. Never retry an ambiguous side effect blindly.
44
+
45
+ ## 4. Degradation and recovery
46
+
47
+ For each critical journey, define:
48
+ - What can continue safely and what must fail closed?
49
+ - Which data may be stale, for how long, and how is freshness communicated?
50
+ - Can writes be queued safely? What is the durability and expiry guarantee?
51
+ - How are partial/unknown outcomes reconciled?
52
+ - What is the failover authority and how is split brain prevented?
53
+ - What is the rollback or forward-fix path?
54
+ - Who declares recovery and verifies backlog drain and data correctness?
55
+
56
+ Do not equate process restart with recovery. Recovery includes restoring business invariants, reconciling side effects, draining queues safely, and validating user-visible outcomes.
57
+
58
+ ## 5. RTO and RPO
59
+
60
+ - **RTO (Recovery Time Objective):** target maximum time to restore an acceptable service level after disruption.
61
+ - **RPO (Recovery Point Objective):** target maximum tolerable data loss measured in time.
62
+ - Define the service level that counts as recovered; partial read-only operation may differ from full write capability.
63
+ - Derive backup frequency, replication, restore procedures, and staffing from business RTO/RPO.
64
+ - Test restore and failover. A successful backup job does not prove recoverability.
65
+ - Include dependencies, credentials, encryption keys, DNS, and application compatibility in recovery exercises.
66
+
67
+ ## 6. Failure testing
68
+
69
+ Test realistic combinations: dependency timeout plus retry, zone loss under peak load, stale replica reads, queue backlog and poison messages, expired credentials, partial deployment, schema skew, cache outage, restore from backup, and ambiguous provider responses.
70
+
71
+ Set expected outcomes and stop conditions before fault injection. Protect production and obtain explicit authorization for disruptive exercises.
72
+
73
+ ## 7. Operational review checklist
74
+
75
+ - [ ] Critical journeys and invariants are identified.
76
+ - [ ] Failure modes and owners are recorded.
77
+ - [ ] Detection signals and alert thresholds map to user impact.
78
+ - [ ] Blast radius and shared dependencies are understood.
79
+ - [ ] Retries, deadlines, queues, and concurrency are bounded.
80
+ - [ ] Degraded modes preserve correctness and security.
81
+ - [ ] Ambiguous writes and partial side effects are reconciled.
82
+ - [ ] RTO/RPO are business-defined and restore-tested.
83
+ - [ ] Runbooks, escalation, and recovery ownership are clear.
84
+ - [ ] Failure tests cover the highest-risk scenarios.
@@ -0,0 +1,76 @@
1
+ # Multi-agent architecture and orchestration
2
+
3
+ Use multiple agents only when the workload benefits from bounded specialization, independent context, parallel execution, or distinct trust boundaries. A multi-agent design is not inherently more capable or reliable than a single agent or deterministic workflow.
4
+
5
+ ## 1. Start with the baseline
6
+
7
+ Compare:
8
+ - Deterministic workflow or state machine.
9
+ - Single agent with a bounded toolset.
10
+ - Single agent with specialist tools or structured subroutines.
11
+ - Multi-agent orchestration.
12
+
13
+ Estimate coordination cost: additional model calls, context duplication, message volume, retries, synchronization, evaluation complexity, and failure surface. Prefer the least complex design that meets quality, latency, cost, and safety objectives.
14
+
15
+ ## 2. Common topologies
16
+
17
+ | Topology | Suitable when | Main risks / controls |
18
+ |---|---|---|
19
+ | Sequential pipeline | Work has clear ordered stages and stable handoffs | Error propagation; validate each stage output |
20
+ | Router / specialist | Requests fall into separable domains | Misrouting; confidence threshold and safe fallback |
21
+ | Supervisor / worker | A coordinator can decompose tasks and integrate bounded results | Supervisor bottleneck, runaway delegation; cap depth/fan-out and validate outputs |
22
+ | Parallel fan-out / aggregation | Independent subtasks can run concurrently | Cost amplification, inconsistent evidence; bound concurrency and define merge rules |
23
+ | Peer collaboration | Agents need iterative critique or negotiation | Non-termination and consensus theater; cap rounds and require evidence |
24
+ | Hierarchical delegation | Large tasks have stable subdomains and ownership | Deep call trees and opaque accountability; enforce depth and trace parent-child runs |
25
+
26
+ Choose topology based on dependency structure and task semantics, not fashion. A graph of agents must have explicit termination and cycle controls.
27
+
28
+ ## 3. Agent contracts
29
+
30
+ For every agent define:
31
+ - Stable role and responsibility; explicit non-goals.
32
+ - Input schema, output schema, and validation rules.
33
+ - Allowed tools, data scopes, and permissions.
34
+ - Context sources and trust classification.
35
+ - Token, time, step, concurrency, and cost budgets.
36
+ - Success criteria, stop conditions, and escalation behavior.
37
+ - Owner, version, and compatibility policy.
38
+
39
+ Treat peer outputs as untrusted proposals. Validate facts, schemas, citations, and permissions at the receiving boundary. Do not let one agent grant another agent authority by assertion.
40
+
41
+ ## 4. Orchestration and state
42
+
43
+ Represent long-running work as an explicit workflow with run ID, parent/child task IDs, state transitions, deadlines, cancellation, and durable checkpoints. Separate orchestration state from model conversation context. Persist task ownership and result provenance.
44
+
45
+ Define:
46
+ - Dispatch, queueing, backpressure, concurrency and tenant quotas.
47
+ - Correlation IDs and causality across parent/child traces.
48
+ - Duplicate delivery and idempotent task creation.
49
+ - Partial success, retries, timeout, cancellation, and compensation.
50
+ - Result aggregation, conflict resolution, evidence precedence, and unresolved disagreement.
51
+ - Human approval and handoff points.
52
+
53
+ Use deterministic code for critical transitions, budget enforcement, authorization, and side effects. Model-generated plans may propose transitions but must not bypass the workflow state machine.
54
+
55
+ ## 5. Consistency and coordination
56
+
57
+ Avoid shared mutable agent memory as an implicit coordination mechanism. Prefer explicit messages, versioned artifacts, or a workflow-owned state store. Define the source of truth and concurrency strategy for shared artifacts. Use optimistic concurrency/version checks when multiple workers can update the same task. Record provenance and distinguish observed facts, inferences, and hypotheses.
58
+
59
+ For parallel work, define merge semantics before fan-out. Examples include set union with provenance, deterministic conflict resolution, human review, or rejecting incompatible outputs. Do not silently select the most confident-sounding response.
60
+
61
+ ## 6. Reliability and cost controls
62
+
63
+ Bound fan-out, nesting depth, rounds, tokens, wall-clock duration, queue age, and total spend. Apply admission control, deadlines, cancellation propagation, retry budgets, and per-tenant quotas. A failed child task should have a defined retry, fallback, partial-result, or escalation path. Prevent retries from creating duplicate external effects.
64
+
65
+ Measure end-to-end task success and total orchestration cost—not only each agent's local completion rate.
66
+
67
+ ## 7. Review checklist
68
+
69
+ - [ ] A single-agent/deterministic baseline was considered.
70
+ - [ ] Each agent has a bounded role, contract, owner, and permission set.
71
+ - [ ] Topology and dependency graph have explicit termination rules.
72
+ - [ ] Fan-out, depth, concurrency, time, tokens, and cost are bounded.
73
+ - [ ] State ownership, message correlation, and conflict resolution are explicit.
74
+ - [ ] Partial success, cancellation, retry, and human escalation are designed.
75
+ - [ ] Child outputs are validated and retain provenance.
76
+ - [ ] Evaluation covers end-to-end tasks and coordination failures.
@@ -0,0 +1,87 @@
1
+ # Agent harness and runtime architecture
2
+
3
+ An agent harness is the deterministic application/runtime layer that hosts a model-driven loop, exposes capabilities, enforces policy, manages state, and controls execution. The model is a probabilistic component inside this system—not the runtime, policy engine, or source of truth.
4
+
5
+ ## 1. Reference execution loop
6
+
7
+ A robust loop typically performs:
8
+
9
+ 1. Receive a request and authenticate the principal.
10
+ 2. Load authorized context and durable workflow state.
11
+ 3. Construct bounded model context from versioned instructions, relevant data, and tool schemas.
12
+ 4. Request a model response under explicit token, latency, and cost limits.
13
+ 5. Parse and validate the response against an expected schema.
14
+ 6. If a tool action is proposed, resolve the tool and authorize the specific action.
15
+ 7. Execute within a bounded runtime and validate the result.
16
+ 8. Persist state/checkpoint and append an auditable event.
17
+ 9. Continue, return a result, request approval, or stop according to deterministic policy.
18
+
19
+ The model may suggest an action; only the application decides whether it is permitted and whether the workflow can advance.
20
+
21
+ ## 2. Runtime components
22
+
23
+ | Component | Responsibility |
24
+ |---|---|
25
+ | Request/session gateway | Authentication, tenant scope, request limits, cancellation |
26
+ | Orchestrator / state machine | Run lifecycle, transitions, deadlines, retries, human handoff |
27
+ | Model adapter | Provider abstraction, structured output, timeout, quota and routing |
28
+ | Context builder | Authorized retrieval, token budgeting, provenance and redaction |
29
+ | Tool registry | Typed schemas, versions, ownership, permission metadata |
30
+ | Policy decision point | Principal/resource/action authorization and approvals |
31
+ | Tool executor | Sandboxing, network/filesystem constraints, secrets broker |
32
+ | State store | Durable run state, checkpoints, idempotency and event history |
33
+ | Telemetry pipeline | Traces, metrics, audit events, redaction and retention |
34
+
35
+ Keep policy evaluation and execution separate. A tool registry entry is not itself permission to call the tool.
36
+
37
+ ## 3. Control plane and execution plane
38
+
39
+ The control plane owns identity, configuration, tool registration, policy, budgets, scheduling, and audit. The execution plane runs model calls and tools in isolated workers. Separate them when the threat model, scale, or operational requirements justify it.
40
+
41
+ Use workload identity and short-lived scoped credentials. Do not pass broad platform credentials into model context or untrusted tool processes. Apply network egress allowlists, filesystem boundaries, CPU/memory/process limits, and execution timeouts appropriate to the tool.
42
+
43
+ ## 4. Context assembly
44
+
45
+ Treat context as a derived, bounded view—not durable storage. Define precedence and provenance for system policy, developer configuration, user request, retrieved evidence, conversation summary, and tool results. Keep untrusted content clearly delimited and never elevate retrieved text into trusted instructions.
46
+
47
+ Budget context explicitly:
48
+ - Fixed instruction and tool-schema overhead.
49
+ - Current user request and relevant conversation.
50
+ - Authorized retrieved evidence and state summary.
51
+ - Reserved output and tool-call capacity.
52
+ - Safety margin for tokenizer/provider variation.
53
+
54
+ Summarize with provenance and retain links to source records. Re-check authorization when data is retrieved and before side effects.
55
+
56
+ ## 5. Tool execution contract
57
+
58
+ Every tool needs a stable ID/version, owner, typed input/output schema, authentication model, permission scope, timeout, resource budget, side-effect classification, idempotency strategy, and audit policy.
59
+
60
+ Validate arguments server-side. Reject unknown or excessive fields, unsafe paths/URLs, oversized payloads, invalid identifiers, and unbounded queries. Validate outputs before returning them to the model. Use structured errors that distinguish validation, authorization, rate limit, timeout, transient failure, and ambiguous outcome.
61
+
62
+ ## 6. Stop conditions and budgets
63
+
64
+ Enforce deterministic maximums for:
65
+ - Model turns and tool calls per run.
66
+ - Wall-clock deadline and per-call timeout.
67
+ - Input/output tokens and total cost.
68
+ - Tool concurrency, recursion/delegation depth, and queue wait.
69
+ - Retry count and retry budget.
70
+ - Output size and retained state.
71
+
72
+ Stop on completion, denial, budget exhaustion, cancellation, deadline, repeated invalid output, or policy-defined escalation. Do not allow the model to override these controls through a tool argument or prompt.
73
+
74
+ ## 7. Runtime resilience
75
+
76
+ Use bounded queues, backpressure, worker isolation, circuit breakers for failing providers, cancellation propagation, and explicit unknown/pending states for ambiguous external effects. Persist checkpoints before and after meaningful side effects. Use idempotency keys and reconciliation rather than blindly repeating uncertain actions.
77
+
78
+ ## 8. Review checklist
79
+
80
+ - [ ] Application code owns the execution loop and state transitions.
81
+ - [ ] Authentication and per-action authorization precede tool execution.
82
+ - [ ] Context is bounded, authorized, versioned, and provenance-aware.
83
+ - [ ] Tool schemas and outputs are validated.
84
+ - [ ] Execution is isolated and resource/network limits are enforced.
85
+ - [ ] Budgets, deadlines, cancellation, and stop conditions are deterministic.
86
+ - [ ] Checkpoints, idempotency, and ambiguous outcomes are handled.
87
+ - [ ] Telemetry and audit are redacted and access-controlled.
@@ -0,0 +1,67 @@
1
+ # Model Context Protocol (MCP) architecture
2
+
3
+ Use MCP as a protocol boundary for connecting AI applications to external capabilities and context. Treat protocol support, server implementation, and authorization policy as separate concerns. MCP does not automatically make a tool safe or authorize a caller.
4
+
5
+ ## 1. Roles and boundaries
6
+
7
+ Model the main roles explicitly:
8
+ - **Host application:** owns the user experience, policy, model orchestration, and trust decisions.
9
+ - **MCP client:** manages a protocol connection to a server and negotiates supported capabilities.
10
+ - **MCP server:** exposes declared capabilities and mediates access to underlying resources or services.
11
+
12
+ A host may manage multiple client/server connections. Each connection is a separate trust and failure boundary. Do not assume that a server can authenticate the end user merely because a host connected to it.
13
+
14
+ ## 2. Capability categories
15
+
16
+ | Capability | Architectural responsibility | Security considerations |
17
+ |---|---|---|
18
+ | Tools | Expose callable operations | Treat as potential side effects; authorize each invocation and validate arguments |
19
+ | Resources | Expose addressable context/data | Enforce access control, tenant scope, freshness, and data minimization |
20
+ | Prompts | Provide reusable prompt templates | Treat content as untrusted configuration unless reviewed and versioned |
21
+ | Sampling / elicitation (where supported) | Request model interaction or user input through the host | Keep user consent, policy, and host authority explicit |
22
+
23
+ Negotiate and inspect capabilities rather than assuming every server implements every feature. Verify the exact protocol revision and supported capability behavior against current official MCP documentation.
24
+
25
+ ## 3. Transport and session architecture
26
+
27
+ Choose transport based on deployment and trust boundaries. Local process transports introduce executable, filesystem, environment, and process-lifecycle risks. Network transports introduce endpoint authentication, TLS, network egress, session lifecycle, and proxy risks.
28
+
29
+ Define connection establishment, initialization, capability negotiation, request correlation, timeouts, cancellation, reconnection, and shutdown. Treat session identifiers as protocol state, not as proof of user identity. Avoid sharing a privileged server connection across tenants unless isolation and authorization are explicitly enforced.
30
+
31
+ ## 4. Authorization model
32
+
33
+ For each operation evaluate:
34
+ - Authenticated principal and tenant.
35
+ - Requested tool/resource and specific action.
36
+ - Object-level authorization and data classification.
37
+ - Scope, rate limit, budget, and approval requirement.
38
+ - Credential audience, expiry, and revocation.
39
+
40
+ Use delegated or workload identity with narrow scopes. Avoid ambient credentials, static broad tokens, and passing secrets through prompts or tool arguments. The host should mediate user consent where required; the server must still enforce its own access policy.
41
+
42
+ ## 5. Reliability and operations
43
+
44
+ Set per-server concurrency, request size, timeouts, retry budgets, and circuit breakers. Distinguish protocol errors from business errors and ambiguous side-effect outcomes. Make mutating operations idempotent where possible and provide operation IDs/status lookup for long-running work.
45
+
46
+ Observe server identity/version, negotiated capabilities, transport health, request latency, error classes, authorization decisions, and tool outcomes. Redact sensitive arguments and resource content. Define server upgrade, compatibility, rollback, and decommissioning procedures.
47
+
48
+ ## 6. Threat model
49
+
50
+ Consider malicious or compromised servers, tool-description prompt injection, confused-deputy attacks, SSRF, local process compromise, path traversal, credential theft, cross-tenant resource exposure, oversized responses, and denial of service. Treat tool descriptions and returned content as untrusted data. Keep protocol discovery separate from permission grants.
51
+
52
+ ## 7. Implementation guidance
53
+
54
+ Pin and review server packages and launch configuration. Restrict local executable paths and environment variables. For remote servers, enforce endpoint allowlists, TLS, authentication, DNS/IP validation where relevant, and outbound network policy. Validate every argument and result at the host/server boundary. Do not assume a protocol handshake is a security review.
55
+
56
+ Protocol details evolve. Before implementing a client or server, consult the official MCP specification and SDK documentation for the exact protocol version, transport, authorization, and capability semantics in use.
57
+
58
+ ## 8. Review checklist
59
+
60
+ - [ ] Host, client, and server responsibilities are explicit.
61
+ - [ ] Capabilities are negotiated and reviewed rather than blindly trusted.
62
+ - [ ] Every tool call and resource read is authorized in the correct tenant/object scope.
63
+ - [ ] Transport, process, endpoint, and credential risks are controlled.
64
+ - [ ] Tool schemas, payload sizes, timeouts, concurrency, and retries are bounded.
65
+ - [ ] Side effects are idempotent or have reconciliation/status mechanisms.
66
+ - [ ] Tool descriptions and results are treated as untrusted content.
67
+ - [ ] Version compatibility, audit, telemetry, and decommissioning are planned.
@@ -0,0 +1,74 @@
1
+ # Agent security, permissions, and isolation
2
+
3
+ An agent is not a security principal with inherent authority. The application must bind each action to an authenticated principal, tenant, resource, and policy decision. Prompt instructions and model refusals are not authorization controls.
4
+
5
+ ## 1. Threat model
6
+
7
+ Include:
8
+ - Direct and indirect prompt injection in user input, files, retrieved documents, web pages, and tool results.
9
+ - Confused-deputy behavior where an agent uses its host's authority for an untrusted requester.
10
+ - Excessive tool permissions, privilege escalation, and cross-tenant access.
11
+ - SSRF, DNS rebinding, path traversal, unsafe deserialization, command/code execution, and malicious file content.
12
+ - Secret leakage through prompts, traces, errors, generated artifacts, or tool output.
13
+ - Data exfiltration, unsafe outbound communication, and covert channels.
14
+ - Compromised MCP servers, dependencies, plugins, model providers, and tool supply chain.
15
+ - Denial of service through recursion, fan-out, large context, expensive calls, or unbounded tool loops.
16
+
17
+ ## 2. Principal and authorization model
18
+
19
+ Represent the effective authority for an operation as the intersection of:
20
+ - Authenticated end-user or initiating service identity.
21
+ - Tenant and organization scope.
22
+ - Agent/workflow identity and delegated scopes.
23
+ - Tool/service identity and resource permissions.
24
+ - Current policy, consent, approval, and environmental constraints.
25
+
26
+ Authorize each action at execution time, not only when the session begins. Enforce object-level checks at the service that owns the resource. Re-check authorization after long waits or privilege changes. Never trust model-supplied user IDs, tenant IDs, role claims, file paths, or URLs.
27
+
28
+ ## 3. Least privilege and capability design
29
+
30
+ Prefer narrow tools that perform one business operation over generic shell, SQL, browser, or filesystem access. Use explicit allowlists for operations, resources, domains, and paths. Scope credentials by audience, tenant, action, and lifetime. Use short-lived tokens and a secrets broker; do not expose raw credentials to model context.
31
+
32
+ Separate read, draft, and commit capabilities. A workflow that can inspect a payment should not automatically be able to issue a refund. Keep high-impact actions behind independent policy checks and approval gates.
33
+
34
+ ## 4. Human approval
35
+
36
+ Require explicit approval for irreversible, financial, legal, privacy-sensitive, externally visible, or privilege-changing actions according to risk. Bind approval to a canonical action summary, target resource, parameters, and expiry. Reject stale or modified approvals. Record approver identity, decision, timestamp, and the exact action authorized.
37
+
38
+ Human approval does not replace server-side validation or transaction controls.
39
+
40
+ ## 5. Isolation and execution safety
41
+
42
+ Run untrusted tools and generated code in isolated workers or sandboxes. Apply CPU, memory, process, filesystem, network egress, and wall-clock limits. Use read-only mounts and ephemeral workspaces where possible. Separate tenant workloads and prevent cross-run state leakage. Restrict environment variables and inherited file descriptors. Do not mount host credentials or container sockets into untrusted execution.
43
+
44
+ ## 6. Data protection and audit
45
+
46
+ Classify context and outputs. Minimize data before model/provider transmission; apply tenant filters before retrieval results enter context. Define retention, deletion, residency, encryption, and provider policy. Redact secrets and sensitive content from logs by default.
47
+
48
+ Audit principal, tenant, run ID, tool/version, action, target, authorization decision, approval, result status, and correlation IDs. Protect audit integrity and restrict who can inspect it. Avoid storing full sensitive payloads when metadata and hashes suffice.
49
+
50
+ ## 7. Defense in depth
51
+
52
+ Use layered controls:
53
+ 1. Input validation and content classification.
54
+ 2. Context isolation and untrusted-content delimiting.
55
+ 3. Model/tool selection policy.
56
+ 4. Deterministic authorization and approval.
57
+ 5. Sandboxed execution and egress restrictions.
58
+ 6. Output validation and data-loss prevention.
59
+ 7. Audit, alerting, and incident response.
60
+
61
+ No single prompt filter or model classifier is a sufficient defense.
62
+
63
+ ## 8. Security review checklist
64
+
65
+ - [ ] Threat model includes direct/indirect prompt injection and confused deputy.
66
+ - [ ] Principal, tenant, agent, and tool identities are distinct.
67
+ - [ ] Every action has server-side object-level authorization.
68
+ - [ ] Tool scopes and credentials follow least privilege and short lifetimes.
69
+ - [ ] High-impact actions use parameter-bound approvals.
70
+ - [ ] Untrusted execution is isolated with resource and egress limits.
71
+ - [ ] Retrieved data is tenant-filtered before model context.
72
+ - [ ] Secrets and sensitive content are redacted from telemetry.
73
+ - [ ] Audit records capture decisions and outcomes without leaking payloads.
74
+ - [ ] Revocation, incident response, and permission review are operationalized.