@teckedd-code2save/b2dp 1.0.1 → 1.1.0

package/skills/cloud-solution-architect/references/architecture-styles.md

@@ -0,0 +1,365 @@
+# Azure Architecture Styles Reference
+
+## Comparison Table
+
+| Style | Dependency Management | Domain Type |
+|---|---|---|
+| N-tier | Horizontal tiers divided by subnet | Traditional business, low update frequency |
+| Web-Queue-Worker | Front/back-end decoupled by async messaging | Simple domain, resource-intensive tasks |
+| Microservices | Vertically decomposed services via APIs | Complex domain, frequent updates |
+| Event-driven | Producer/consumer, independent views | IoT, real-time systems |
+| Big data | Divide into small chunks, parallel processing | Batch/real-time data analysis, ML |
+| Big compute | Data allocation to thousands of cores | Compute-intensive (simulation) |
+
+---
+
+## 1. N-tier
+
+Traditional architecture that divides an application into logical layers and physical tiers. Each layer has a specific responsibility and communicates only with the layer directly below it.
+
+### Logical Diagram
+
+```
+┌──────────────────────────────────┐
+│        Presentation Tier         │  ← Web / UI
+│          (Subnet A)              │
+├──────────────────────────────────┤
+│       Business Logic Tier        │  ← Rules / Workflows
+│          (Subnet B)              │
+├──────────────────────────────────┤
+│        Data Access Tier          │  ← Database / Storage
+│          (Subnet C)              │
+└──────────────────────────────────┘
+```
+
+### Benefits
+
+- Familiar pattern for most development teams
+- Natural mapping for migrating existing layered applications to Azure
+- Clear separation of concerns between tiers
+
+### Challenges
+
+- Horizontal layering makes cross-cutting changes difficult — a single feature may touch every tier
+- Limits agility and release velocity as tiers are tightly coupled vertically
+
+### Best Practices
+
+- Use VNet subnets to isolate tiers and control traffic flow with NSGs
+- Keep each tier stateless where possible to enable horizontal scaling
+- Use managed services (App Service, Azure SQL) to reduce operational overhead
+
+### Dependency Management
+
+Horizontal tiers divided by subnet. Each tier depends only on the tier directly below it, enforced through network segmentation.
+
+### Recommended Azure Services
+
+- Azure App Service
+- Azure SQL Database
+- Azure Virtual Machines
+- Azure Virtual Network (subnets)
+
+---
+
+## 2. Web-Queue-Worker
+
+A web front end handles HTTP requests while a worker process performs resource-intensive or long-running tasks. The two components communicate through an asynchronous message queue.
+
+### Logical Diagram
+
+```
+                ┌───────────┐
+ HTTP ─────────►│    Web    │
+ Requests       │ Front End │
+                └─────┬─────┘
+                      │
+                      ▼
+               ┌──────────────┐
+               │  Message     │
+               │  Queue       │
+               └──────┬───────┘
+                      │
+                      ▼
+                ┌───────────┐
+                │  Worker   │
+                │  Process  │
+                └─────┬─────┘
+                      │
+                      ▼
+                ┌───────────┐
+                │  Database │
+                └───────────┘
+```
+
+### Benefits
+
+- Easy to understand and deploy, especially with managed compute services
+- Clean separation between interactive and background workloads
+- Each component can scale independently
+
+### Challenges
+
+- Without careful design, the front end and worker can become monolithic components that are hard to maintain and update
+- Hidden dependencies may emerge if front end and worker share data schemas or storage
+
+### Best Practices
+
+- Keep the web front end thin — delegate heavy processing to the worker
+- Use durable message queues to ensure work is not lost on failure
+- Design idempotent worker operations to handle message retries safely
+
+### Dependency Management
+
+Front-end and back-end jobs are decoupled by asynchronous messaging. The web tier never calls the worker directly; all communication flows through the queue.
+
+### Recommended Azure Services
+
+- Azure App Service
+- Azure Functions
+- Azure Queue Storage
+- Azure Service Bus
+
+---
+
+## 3. Microservices
+
+A collection of small, autonomous services where each service implements a single business capability. Each service owns its bounded context and data, and communicates with other services via well-defined APIs.
+
+### Logical Diagram
+
+```
+┌──────────┐   ┌──────────┐   ┌──────────┐
+│ Service  │   │ Service  │   │ Service  │
+│    A     │   │    B     │   │    C     │
+│ ┌──────┐ │   │ ┌──────┐ │   │ ┌──────┐ │
+│ │ Data │ │   │ │ Data │ │   │ │ Data │ │
+│ └──────┘ │   │ └──────┘ │   │ └──────┘ │
+└────┬─────┘   └────┬─────┘   └────┬─────┘
+     │              │              │
+     └──────┬───────┘──────────────┘
+            ▼
+     ┌──────────────┐
+     │ API Gateway  │
+     └──────┬───────┘
+            │
+         Clients
+```
+
+### Benefits
+
+- Autonomous teams can develop, deploy, and scale services independently
+- Enables frequent updates and higher release velocity
+- Technology diversity — each service can use the stack best suited to its task
+
+### Challenges
+
+- Service discovery and inter-service communication add complexity
+- Data consistency across services requires patterns like Saga or eventual consistency
+- Distributed system management (monitoring, debugging, tracing) is inherently harder
+
+### Best Practices
+
+- Define clear bounded contexts — avoid sharing databases between services
+- Use an API gateway for cross-cutting concerns (auth, rate limiting, routing)
+- Implement health checks, circuit breakers, and distributed tracing from day one
+
+### Dependency Management
+
+Vertically decomposed services calling each other via APIs. Each service is independently deployable with its own data store, minimizing coupling.
+
+### Recommended Azure Services
+
+- Azure Kubernetes Service (AKS)
+- Azure Container Apps
+- Azure API Management
+- Azure Service Bus
+- Azure Cosmos DB
+
+---
+
+## 4. Event-driven
+
+A publish-subscribe architecture where event producers emit events and event consumers react to them. Producers and consumers are fully decoupled, communicating only through event channels or brokers.
+
+### Logical Diagram
+
+```
+┌──────────┐     ┌──────────────────┐     ┌──────────┐
+│ Producer │────►│                  │────►│ Consumer │
+│    A     │     │   Event Broker   │     │    A     │
+└──────────┘     │   / Channel     │     └──────────┘
+                 │                  │
+┌──────────┐     │  ┌────────────┐ │     ┌──────────┐
+│ Producer │────►│  │  Pub/Sub   │ │────►│ Consumer │
+│    B     │     │  │  or Stream │ │     │    B     │
+└──────────┘     │  └────────────┘ │     └──────────┘
+                 └──────────────────┘
+```
+
+**Two models:** Pub/Sub (events delivered to subscribers) and Event Streaming (events written to an ordered log for consumers to read).
+
+**Consumer variations:** Simple event processing, basic correlation, complex event processing, event stream processing.
+
+### Benefits
+
+- Producers and consumers are fully decoupled — they can evolve independently
+- Highly scalable — add consumers without affecting producers
+- Responsive and well-suited to real-time processing pipelines
+
+### Challenges
+
+- Guaranteed delivery requires careful broker configuration and dead-letter handling
+- Event ordering can be difficult to maintain across partitions
+- Eventual consistency — consumers may see stale data temporarily
+- Error handling and poison message management add operational complexity
+
+### Best Practices
+
+- Design events as immutable facts with clear schemas
+- Use dead-letter queues for events that fail processing
+- Implement idempotent consumers to handle duplicate delivery safely
+
+### Dependency Management
+
+Producer/consumer model with independent views per subsystem. Producers have no knowledge of consumers; each subsystem maintains its own projection of the event stream.
+
+### Recommended Azure Services
+
+- Azure Event Grid
+- Azure Event Hubs
+- Azure Functions
+- Azure Service Bus
+- Azure Stream Analytics
+
+---
+
+## 5. Big Data
+
+Architecture designed to handle ingestion, processing, and analysis of data that is too large or complex for traditional database systems.
+
+### Logical Diagram
+
+```
+┌─────────────┐    ┌──────────────────────────────────┐
+│ Data Sources│───►│          Data Storage             │
+│ (logs, IoT, │    │         (Data Lake)               │
+│  files)     │    └──┬──────────────┬─────────────────┘
+└─────────────┘       │              │
+                      ▼              ▼
+              ┌──────────────┐ ┌──────────────┐
+              │    Batch     │ │  Real-time   │
+              │  Processing  │ │  Processing  │
+              └──────┬───────┘ └──────┬───────┘
+                     │                │
+                     ▼                ▼
+              ┌───────────────────────────────┐
+              │   Analytical Data Store       │
+              └──────────────┬────────────────┘
+                             │
+              ┌──────────────▼────────────────┐
+              │   Analysis & Reporting        │
+              │   (Dashboards, ML Models)     │
+              └───────────────────────────────┘
+
+         Orchestration manages the full pipeline
+```
+
+**Components:** Data sources → Data storage (data lake) → Batch processing → Real-time processing → Analytical data store → Analysis and reporting → Orchestration.
+
+### Benefits
+
+- Process massive datasets that exceed traditional database capacity
+- Support both batch and real-time analytics in a single architecture
+- Enable predictive analytics and machine learning at scale
+
+### Challenges
+
+- Complexity of coordinating batch and real-time processing paths
+- Data quality and governance across a data lake require disciplined schema management
+- Cost management — large-scale storage and compute can grow unpredictably
+
+### Best Practices
+
+- Use parallelism for both batch and real-time processing
+- Partition data to enable parallel reads and writes
+- Apply schema-on-read semantics to keep ingestion flexible
+- Process data in batches on arrival rather than waiting for scheduled windows
+- Balance usage costs against time-to-insight requirements
+
+### Dependency Management
+
+Divide huge datasets into small chunks for parallel processing. Each chunk can be processed independently, with an orchestration layer coordinating the overall pipeline.
+
+### Recommended Azure Services
+
+- Microsoft Fabric
+- Azure Data Lake Storage
+- Azure Event Hubs
+- Azure SQL Database
+- Azure Cosmos DB
+- Power BI
+
+---
+
+## 6. Big Compute
+
+Architecture for large-scale workloads that require hundreds or thousands of cores running in parallel. Tasks can be independent (embarrassingly parallel) or tightly coupled requiring inter-node communication.
+
+### Logical Diagram
+
+```
+┌─────────────────────────────────────────────┐
+│              Job Scheduler                  │
+│         (submit, monitor, manage)           │
+└─────────────────┬───────────────────────────┘
+                  │
+    ┌─────────────┼─────────────┐
+    ▼             ▼             ▼
+┌────────┐  ┌────────┐  ┌────────────────┐
+│ Core   │  │ Core   │  │ Core           │
+│ Pool 1 │  │ Pool 2 │  │ Pool N         │
+│(100s)  │  │(100s)  │  │(1000s of cores)│
+└───┬────┘  └───┬────┘  └───┬────────────┘
+    │           │            │
+    └─────────┬─┘────────────┘
+              ▼
+     ┌──────────────┐
+     │   Results    │
+     │   Storage    │
+     └──────────────┘
+```
+
+**Use cases:** Simulations, financial risk modeling, oil exploration, drug design, image rendering.
+
+### Benefits
+
+- High performance through massive parallel processing
+- Access to specialized hardware (GPU, FPGA, InfiniBand) for compute-intensive workloads
+- Scales to thousands of cores for embarrassingly parallel problems
+
+### Challenges
+
+- Managing VM infrastructure at scale (provisioning, patching, decommissioning)
+- Provisioning thousands of cores in a timely manner to meet job deadlines
+- Cost control — idle compute resources are expensive
+
+### Best Practices
+
+- Use low-priority or spot VMs to reduce cost for fault-tolerant workloads
+- Auto-scale compute pools based on job queue depth
+- Partition work into independent tasks when possible to maximize parallelism
+
+### Dependency Management
+
+Data allocation to thousands of cores. The job scheduler distributes work units across the compute pool, with each core processing its assigned data partition independently.
+
+### Recommended Azure Services
+
+- Azure Batch
+- Microsoft HPC Pack
+- H-series Virtual Machines (HPC-optimized)
+
+---
+
+> Source: [Azure Architecture Center](https://learn.microsoft.com/en-us/azure/architecture/)

package/skills/cloud-solution-architect/references/best-practices.md

@@ -0,0 +1,311 @@
+# Cloud Application Best Practices
+
+Twelve best practices from the Azure Architecture Center for designing, building, and operating cloud applications.
+
+---
+
+## 1. API Design
+
+Design RESTful web APIs that promote platform independence and loose coupling between clients and services.
+
+### Key Recommendations
+
+- Organize APIs around resources using nouns, not verbs, in URIs
+- Use standard HTTP methods (GET, POST, PUT, PATCH, DELETE) with correct semantics
+- Use plural nouns for collection endpoints (e.g., `/orders`, `/customers`)
+- Support HATEOAS to enable client navigation of the API without prior knowledge
+- Design coarse-grained operations to avoid chatty request patterns
+- Do not expose internal database structure through the API surface
+- Version APIs to manage breaking changes without disrupting existing clients
+- Return appropriate HTTP status codes and consistent error response bodies
+
+### WAF Pillar Alignment
+
+Performance Efficiency · Operational Excellence
+
+### Common Mistakes
+
+- Using verbs in URIs (e.g., `/getOrders`) instead of resource-based paths
+- Exposing database schema directly through API contracts
+- Creating chatty APIs that require multiple round-trips for a single logical operation
+
+---
+
+## 2. API Implementation
+
+Implement web APIs to be efficient, responsive, scalable, and available for consuming clients.
+
+### Key Recommendations
+
+- Make actions idempotent so retries are safe (especially PUT and DELETE)
+- Support content negotiation via `Accept` and `Content-Type` headers
+- Follow the HTTP specification for status codes, methods, and headers
+- Handle exceptions gracefully and return meaningful error responses
+- Support resource discovery through links and metadata
+- Limit and paginate large result sets to minimize network traffic
+- Handle large requests asynchronously using `202 Accepted` with status polling
+- Compress responses where appropriate to reduce payload size
+
+### WAF Pillar Alignment
+
+Operational Excellence
+
+### Common Mistakes
+
+- Not handling large requests asynchronously, causing timeouts
+- Not minimizing network traffic through pagination, filtering, or compression
+
+---
+
+## 3. Autoscaling
+
+Dynamically allocate and deallocate resources to match performance requirements while optimizing cost.
+
+### Key Recommendations
+
+- Use Azure Monitor autoscale and built-in platform autoscaling features
+- Scale based on metrics that directly correlate with load (CPU, queue length, request rate)
+- Combine schedule-based and metric-based scaling for predictable traffic patterns
+- Set appropriate minimum, maximum, and default instance counts
+- Configure scale-in rules as carefully as scale-out rules
+- Use cooldown periods to prevent oscillation (flapping)
+- Plan for the delay between triggering a scale event and resources becoming available
+
+### WAF Pillar Alignment
+
+Performance Efficiency · Cost Optimization
+
+### Common Mistakes
+
+- Not setting appropriate minimum and maximum limits for scaling
+- Not considering scale-in behavior, leading to premature resource removal
+- Using metrics that do not accurately reflect application load
+
+---
+
+## 4. Background Jobs
+
+Implement batch processing, long-running tasks, and workflows as background jobs decoupled from the user interface.
+
+### Key Recommendations
+
+- Use Azure platform services such as Functions, WebJobs, and Batch for hosting
+- Trigger background jobs with events, schedules, or message queues
+- Return results to calling tasks through queues, events, or shared storage
+- Design jobs to be independently deployable, scalable, and versioned
+- Handle partial failures and support safe restarts with checkpointing
+- Monitor job health with logging, metrics, and alerting
+- Implement graceful shutdown to allow in-progress work to complete
+
+### WAF Pillar Alignment
+
+Operational Excellence
+
+### Common Mistakes
+
+- Not handling failures or partial completion within long-running jobs
+- Not monitoring background job health, missing silent failures
+
+---
+
+## 5. Caching
+
+Copy frequently read, rarely modified data to fast storage close to the application to improve performance.
+
+### Key Recommendations
+
+- Cache data that is read often but changes infrequently
+- Use Azure Cache for Redis for distributed, high-throughput caching
+- Set appropriate expiration policies (TTL) to balance freshness and hit rates
+- Handle cache misses gracefully with a cache-aside pattern
+- Address concurrency issues when multiple processes update the same cached data
+- Implement cache invalidation strategies aligned with data change patterns
+- Pre-populate caches for known hot data during application startup
+
+### WAF Pillar Alignment
+
+Performance Efficiency
+
+### Common Mistakes
+
+- Caching highly volatile data that expires before it can be served
+- Not handling cache invalidation, serving stale data to users
+- Cache stampede — many concurrent requests regenerating the same expired entry
+
+---
+
+## 6. CDN (Content Delivery Network)
+
+Use CDNs to deliver static and dynamic web content efficiently to users from edge locations worldwide.
+
+### Key Recommendations
+
+- Offload static assets (images, scripts, stylesheets) to the CDN to reduce origin load
+- Configure appropriate cache-control headers for each content type
+- Version static content via file names or query strings for reliable cache busting
+- Use HTTPS and enforce TLS for secure delivery
+- Plan for CDN fallback so the application degrades gracefully if the CDN is unavailable
+- Handle deployment and versioning so users receive updated content promptly
+
+### WAF Pillar Alignment
+
+Performance Efficiency
+
+### Common Mistakes
+
+- Not versioning content, causing users to receive stale cached assets after deployments
+- Setting improper cache headers, resulting in under-caching or over-caching
+
+---
+
+## 7. Data Partitioning
+
+Divide data stores into partitions to improve scalability, availability, and performance while reducing contention and storage costs.
+
+### Key Recommendations
+
+- Choose partition keys that distribute data and load evenly across partitions
+- Use horizontal (sharding), vertical, or functional partitioning based on access patterns
+- Minimize cross-partition queries to avoid performance degradation
+- Design partitions to match the most common query patterns
+- Plan for rebalancing as data volume and access patterns evolve
+- Consider partition limits and throughput caps of the target data store
+- Reduce contention and storage costs by separating hot and cold data
+
+### WAF Pillar Alignment
+
+Performance Efficiency · Cost Optimization
+
+### Common Mistakes
+
+- Creating hotspots by selecting a partition key with skewed distribution
+- Not considering the cost and latency of cross-partition queries
+
+---
+
+## 8. Data Partitioning Strategies (by Service)
+
+Apply service-specific partitioning strategies across Azure SQL Database, Azure Table Storage, Azure Blob Storage, and other data services.
+
+### Key Recommendations
+
+- Shard Azure SQL Database to distribute data for horizontal scaling
+- Design Azure Table Storage partition keys around query access patterns
+- Organize Azure Blob Storage using virtual directories and naming conventions
+- Align partition boundaries with the most frequent query predicates
+- Reduce latency by co-locating related data within the same partition
+- Monitor partition metrics and rebalance when skew is detected
+
+### WAF Pillar Alignment
+
+Performance Efficiency · Cost Optimization
+
+### Common Mistakes
+
+- Not aligning the partition strategy with actual query patterns, causing full scans
+- Ignoring service-specific partition limits and throttling thresholds
+
+---
+
+## 9. Host Name Preservation
+
+Preserve the original HTTP host name between reverse proxies and backend web applications to avoid issues with cookies, redirects, and CORS.
+
+### Key Recommendations
+
+- Forward the original `Host` header from the reverse proxy to the backend
+- Configure Azure Front Door, Application Gateway, and API Management for host preservation
+- Ensure cookies are set with the correct domain matching the original host name
+- Verify redirect URLs reference the external host name, not the internal backend address
+- Test CORS configurations end-to-end with the preserved host name
+- Document host name flow across all network hops in the architecture
+
+### WAF Pillar Alignment
+
+Reliability
+
+### Common Mistakes
+
+- Not preserving host headers, causing redirect loops or incorrect absolute URLs
+- Breaking session cookies because the cookie domain does not match the forwarded host
+
+---
+
+## 10. Message Encoding
+
+Choose the right payload structure, encoding format, and serialization library for asynchronous messages exchanged between distributed components.
+
+### Key Recommendations
+
+- Evaluate JSON, Avro, Protobuf, and MessagePack based on performance and interoperability needs
+- Use schema registries to enforce and version message contracts
+- Validate incoming messages against their schemas before processing
+- Prefer compact binary formats (Protobuf, Avro) for high-throughput, latency-sensitive paths
+- Use JSON for human-readable messages and broad ecosystem compatibility
+- Consider backward and forward compatibility when evolving message schemas
+
+### WAF Pillar Alignment
+
+Security
+
+### Common Mistakes
+
+- Using an inefficient encoding format for high-volume message streams
+- Not validating message schemas, allowing malformed data into the processing pipeline
+
+---
+
+## 11. Monitoring and Diagnostics
+
+Track system health, usage, and performance through a comprehensive monitoring pipeline that turns raw data into alerts, reports, and automated triggers.
+
+### Key Recommendations
+
+- Instrument applications with structured logging, metrics, and distributed tracing
+- Use Azure Monitor, Application Insights, and Log Analytics as the monitoring backbone
+- Define actionable alerts with clear thresholds, severity levels, and response procedures
+- Detect and correct issues before they affect users by monitoring leading indicators
+- Correlate telemetry across services using distributed trace context (e.g., correlation IDs)
+- Establish performance baselines and track deviations over time
+- Build dashboards for operational visibility across all tiers of the architecture
+- Review and tune alert rules regularly to reduce noise
+
+### WAF Pillar Alignment
+
+Operational Excellence
+
+### Common Mistakes
+
+- Insufficient logging, making incident root-cause analysis slow or impossible
+- Not using distributed tracing in microservice or multi-tier architectures
+- Alert fatigue from poorly tuned thresholds that generate excessive false positives
+
+---
+
+## 12. Transient Fault Handling
+
+Detect and handle transient faults caused by momentary loss of network connectivity, temporary service unavailability, or resource throttling.
+
+### Key Recommendations
+
+- Implement retry logic with exponential backoff and jitter for transient failures
+- Use a circuit breaker pattern to stop retrying when failures are persistent
+- Distinguish transient faults (e.g., HTTP 429, 503) from permanent errors (e.g., 400, 404)
+- Leverage built-in retry policies in Azure SDKs before adding custom retry logic
+- Avoid duplicating retry layers across middleware and application code
+- Log every retry attempt for post-incident analysis
+- Set a maximum retry count and total timeout to bound retry duration
+
+### WAF Pillar Alignment
+
+Reliability
+
+### Common Mistakes
+
+- Retrying non-transient faults (e.g., authentication failures, bad requests)
+- Not using exponential backoff, overwhelming a recovering service with constant retries
+- Retry storms caused by multiple layers retrying simultaneously without coordination
+
+---
+
+> Source: [Azure Architecture Center](https://learn.microsoft.com/en-us/azure/architecture/)