@booklib/core 2.0.0

package/skills/microservices-patterns/evals/evals.json

@@ -0,0 +1,45 @@
+{
+  "evals": [
+    {
+      "id": "eval-01-distributed-transaction-no-saga",
+      "prompt": "Review this microservices code for an e-commerce checkout flow:\n\n```java\n// OrderService — orchestrates the entire checkout in one HTTP transaction\n@RestController\npublic class CheckoutController {\n    private final OrderRepository orderRepository;\n    private final InventoryServiceClient inventoryClient;\n    private final PaymentServiceClient paymentClient;\n    private final ShippingServiceClient shippingClient;\n    private final NotificationServiceClient notificationClient;\n    \n    @PostMapping(\"/checkout\")\n    @Transactional  // <-- spans the entire method\n    public OrderConfirmation checkout(@RequestBody CheckoutRequest request) {\n        // Step 1: Reserve inventory\n        inventoryClient.reserveItems(request.getItems());\n        \n        // Step 2: Create order in our DB\n        Order order = orderRepository.save(new Order(request));\n        \n        // Step 3: Charge payment\n        PaymentResult payment = paymentClient.charge(request.getPaymentInfo(), order.getTotal());\n        \n        // Step 4: Create shipping label\n        ShippingLabel label = shippingClient.createLabel(order.getId(), request.getAddress());\n        \n        // Step 5: Send confirmation email\n        notificationClient.sendConfirmation(request.getEmail(), order.getId());\n        \n        order.setShippingLabel(label.getTrackingNumber());\n        order.setStatus(\"CONFIRMED\");\n        return new OrderConfirmation(order.getId(), label.getTrackingNumber());\n    }\n}\n```",
+      "expectations": [
+        "Identifies the core anti-pattern: a distributed transaction where @Transactional spans HTTP calls to 3 external services — this does not work as intended",
+        "Explains why @Transactional cannot span HTTP calls: the ACID transaction boundary is the local database only; remote service calls are not rolled back if a later step fails",
+        "Identifies the failure scenario: if shippingClient.createLabel() throws after paymentClient.charge() succeeds, the customer is charged but never gets a shipping label — and inventory is reserved but no rollback occurs",
+        "Names the pattern that is needed: the Saga pattern (orchestration-based) to handle this cross-service workflow",
+        "Explains compensating transactions: each step needs a corresponding undo — if shipping fails, a ReversePaymentSaga step should issue a refund and a ReleaseInventory step should free reserved stock",
+        "Notes the synchronous chain anti-pattern: OrderService → Inventory → Payment → Shipping → Notification is a fragile chain — if Notification is slow or down, the entire checkout hangs",
+        "Recommends making notifications asynchronous via a message queue (fire-and-forget)",
+        "Suggests using Saga orchestration with explicit saga states (PENDING_INVENTORY, PENDING_PAYMENT, PENDING_SHIPPING, CONFIRMED, FAILED) stored durably"
+      ]
+    },
+    {
+      "id": "eval-02-shared-database",
+      "prompt": "Review this microservices architecture:\n\n```java\n// ProductService — manages product catalog\n@Service\npublic class ProductService {\n    @Autowired\n    private JdbcTemplate jdbc;  // Connects to: jdbc:postgresql://shared-db:5432/platform\n    \n    public Product getProduct(Long id) {\n        return jdbc.queryForObject(\n            \"SELECT * FROM products WHERE id = ?\",\n            new ProductRowMapper(), id);\n    }\n    \n    public void updatePrice(Long productId, BigDecimal newPrice) {\n        jdbc.update(\"UPDATE products SET price = ?, updated_at = NOW() WHERE id = ?\",\n            newPrice, productId);\n    }\n}\n\n// OrderService — in a SEPARATE deployable service/process\n@Service\npublic class OrderService {\n    @Autowired\n    private JdbcTemplate jdbc;  // ALSO connects to: jdbc:postgresql://shared-db:5432/platform\n    \n    public Order createOrder(CreateOrderRequest req) {\n        // Directly reads from products table owned by ProductService\n        BigDecimal price = jdbc.queryForObject(\n            \"SELECT price FROM products WHERE id = ?\",\n            BigDecimal.class, req.getProductId());\n        \n        // Directly joins across service boundaries\n        List<OrderLine> lines = jdbc.query(\n            \"SELECT o.*, p.name, p.sku FROM orders o JOIN products p ON o.product_id = p.id WHERE o.customer_id = ?\",\n            new OrderLineRowMapper(), req.getCustomerId());\n        \n        jdbc.update(\"INSERT INTO orders (product_id, customer_id, price) VALUES (?, ?, ?)\",\n            req.getProductId(), req.getCustomerId(), price);\n        return buildOrder(lines);\n    }\n}\n```",
+      "expectations": [
+        "Identifies the Shared Database anti-pattern: both ProductService and OrderService connect to the same database and directly access each other's tables",
+        "Explains why this is problematic: ProductService cannot change its products table schema without coordinating with OrderService — the services are coupled at the data layer despite being separate deployables",
+        "Flags that OrderService reads from products — meaning ProductService's internal data model is now OrderService's public API; any rename or restructure breaks OrderService",
+        "Flags the cross-service JOIN in SQL: joins across service boundaries are a strong indicator of incorrect service decomposition or a shared database violation",
+        "Explains the correct pattern: each service owns its own database/schema; OrderService should get product information by calling ProductService's API (synchronous) or by maintaining a local read model via events (CQRS)",
+        "Recommends that OrderService should store the price at order-creation time (denormalized) rather than joining live to the products table — price at time of purchase is correct business behavior",
+        "Notes that this architecture makes independent deployment impossible: upgrading ProductService's database schema requires a coordinated deployment with OrderService"
+      ]
+    },
+    {
+      "id": "eval-03-event-driven-saga",
+      "prompt": "Review this saga implementation for a ride-sharing trip booking:\n\n```java\n// TripSaga — orchestrator (Spring Saga / Axon-style pseudocode)\n@Saga\npublic class BookTripSaga {\n    private String tripId;\n    private String driverId;\n    private String paymentAuthId;\n    \n    @StartSaga\n    @SagaEventHandler(associationProperty = \"tripId\")\n    public void on(TripRequestedEvent event) {\n        this.tripId = event.getTripId();\n        commandGateway.send(new FindAvailableDriverCommand(event.getTripId(), event.getLocation()));\n    }\n    \n    @SagaEventHandler(associationProperty = \"tripId\")\n    public void on(DriverAssignedEvent event) {\n        this.driverId = event.getDriverId();\n        commandGateway.send(new AuthorizePaymentCommand(event.getTripId(), event.getEstimatedFare()));\n    }\n    \n    @SagaEventHandler(associationProperty = \"tripId\")\n    public void on(PaymentAuthorizedEvent event) {\n        this.paymentAuthId = event.getAuthorizationId();\n        commandGateway.send(new ConfirmTripCommand(tripId, driverId));\n    }\n    \n    @SagaEventHandler(associationProperty = \"tripId\")\n    public void on(NoDriverAvailableEvent event) {\n        commandGateway.send(new CancelTripCommand(tripId, \"No drivers available\"));\n        SagaLifecycle.end();\n    }\n    \n    @SagaEventHandler(associationProperty = \"tripId\")\n    public void on(PaymentDeclinedEvent event) {\n        // Compensate: release the reserved driver\n        commandGateway.send(new ReleaseDriverCommand(driverId, tripId));\n        commandGateway.send(new CancelTripCommand(tripId, \"Payment declined\"));\n        SagaLifecycle.end();\n    }\n    \n    @EndSaga\n    @SagaEventHandler(associationProperty = \"tripId\")\n    public void on(TripConfirmedEvent event) {\n        // Saga complete — trip is live\n    }\n}\n```",
+      "expectations": [
+        "Recognizes this as a well-designed orchestration-based saga and says so explicitly",
+        "Praises the compensating transactions: PaymentDeclinedEvent triggers both ReleaseDriverCommand and CancelTripCommand — each forward step has a corresponding undo",
+        "Praises that the saga stores intermediate state (driverId, paymentAuthId) to enable compensation — the saga has the information it needs to undo each step",
+        "Praises that the saga is event-driven: each step reacts to an event rather than making synchronous calls — this decouples the booking flow from service availability",
+        "Praises explicit saga lifecycle management: SagaLifecycle.end() on failure paths and @EndSaga on success ensure the saga doesn't leak memory",
+        "Praises modeling failure paths as first-class events (NoDriverAvailableEvent, PaymentDeclinedEvent) — not exceptions, but domain events the saga handles explicitly",
+        "Does NOT manufacture fake issues just to have something to say",
+        "May offer optional suggestions (timeout handling if DriverAssignedEvent never arrives, idempotent command handlers) but frames them as additional robustness, not defects in the current design"
+      ]
+    }
+  ]
+}

package/skills/microservices-patterns/evals/results.json

@@ -0,0 +1,13 @@
+{
+  "pass_rate": 1,
+  "passed": 23,
+  "total": 23,
+  "baseline_pass_rate": 0.696,
+  "baseline_passed": 16,
+  "baseline_total": 23,
+  "delta": 0.304,
+  "model": "default",
+  "evals_run": 3,
+  "date": "2026-03-28",
+  "non_standard_provider": true
+}

package/skills/microservices-patterns/examples/after.md

@@ -0,0 +1,69 @@
+# After
+
+`InventoryService` owns its own read model, populated by consuming `OrderLineItemAdded` domain events published by the Order Service — no shared database access.
+
+```java
+// --- Order Service publishes domain events (its own codebase) ---
+
+@DomainEvent
+public record OrderLineItemAdded(
+    String orderId,
+    String productId,
+    int quantity,
+    Instant occurredAt
+) {}
+
+// Order Service publishes this event after saving the order aggregate
+orderEventPublisher.publish(new OrderLineItemAdded(
+    order.getId(), line.getProductId(), line.getQuantity(), Instant.now()
+));
+
+
+// --- Inventory Service: owns its own read model, no shared DB ---
+
+// Private denormalized table — owned exclusively by Inventory Service
+@Entity @Table(name = "product_weekly_sales")
+public class ProductWeeklySales {
+    @Id private String productId;
+    private int unitsSoldLast7Days;
+    private Instant lastUpdated;
+}
+
+@Component
+public class OrderLineItemAddedConsumer {
+
+    @KafkaListener(topics = "order.events", groupId = "inventory-service")
+    public void handle(OrderLineItemAdded event) {
+        // Idempotent: uses event's occurredAt to filter stale events
+        if (event.occurredAt().isBefore(Instant.now().minus(7, DAYS))) return;
+
+        weeklySalesRepository.incrementUnitsSold(event.productId(), event.quantity());
+    }
+}
+
+@RestController @RequestMapping("/inventory")
+public class InventoryController {
+
+    @GetMapping("/reorder-candidates")
+    public List<ReorderItem> getReorderCandidates() {
+        // Queries Inventory Service's OWN database — no cross-service DB access
+        return weeklySalesRepository.findAll().stream()
+            .filter(sales -> {
+                int stockLevel = stockRepository.getLevel(sales.getProductId());
+                return stockLevel < sales.getUnitsSoldLast7Days() * 2;
+            })
+            .map(sales -> new ReorderItem(
+                sales.getProductId(),
+                sales.getUnitsSoldLast7Days() * 3 - stockRepository.getLevel(sales.getProductId())
+            ))
+            .toList();
+    }
+}
+```
+
+Key improvements:
+- Each service owns its database — `InventoryService` never touches the `orders` schema (Database per Service pattern)
+- `OrderLineItemAdded` domain event decouples the services; Order Service does not know Inventory Service exists
+- `ProductWeeklySales` is a denormalized read model maintained by consuming events — a lightweight CQRS view
+- The Kafka consumer is idempotent: events outside the 7-day window are skipped, making re-delivery safe
+- Deleting the coupling to `sharedDataSource` eliminates the risk that a schema change in Order Service breaks Inventory Service at runtime

package/skills/microservices-patterns/examples/before.md

@@ -0,0 +1,40 @@
+# Before
+
+An `InventoryService` that directly queries the `orders` database table owned by another service, creating tight coupling and a shared-database anti-pattern.
+
+```java
+@RestController
+@RequestMapping("/inventory")
+public class InventoryController {
+
+    @Autowired
+    private DataSource sharedDataSource; // connected to the orders DB
+
+    @GetMapping("/reorder-candidates")
+    public List<ReorderItem> getReorderCandidates() {
+        List<ReorderItem> candidates = new ArrayList<>();
+
+        // Directly querying the Order Service's database table
+        try (Connection conn = sharedDataSource.getConnection();
+             PreparedStatement ps = conn.prepareStatement(
+                 "SELECT product_id, SUM(quantity) as sold_qty " +
+                 "FROM orders.order_lines " +
+                 "WHERE created_at > NOW() - INTERVAL 7 DAY " +
+                 "GROUP BY product_id")) {
+
+            ResultSet rs = ps.executeQuery();
+            while (rs.next()) {
+                String productId = rs.getString("product_id");
+                int soldQty = rs.getInt("sold_qty");
+                int stockLevel = getStockLevel(productId);
+                if (stockLevel < soldQty * 2) {
+                    candidates.add(new ReorderItem(productId, soldQty * 3 - stockLevel));
+                }
+            }
+        } catch (SQLException e) {
+            throw new RuntimeException(e);
+        }
+        return candidates;
+    }
+}
+```

package/skills/microservices-patterns/references/patterns-catalog.md

@@ -0,0 +1,391 @@
+# Microservices Patterns Catalog
+
+Comprehensive reference of patterns from Chris Richardson's *Microservices Patterns*.
+Organized by problem category. Read the section relevant to the code you're generating.
+
+---
+
+## Table of Contents
+
+1. [Decomposition Patterns](#decomposition-patterns)
+2. [Communication Patterns](#communication-patterns)
+3. [API Gateway Patterns](#api-gateway-patterns)
+4. [Transaction Management — Sagas](#transaction-management--sagas)
+5. [Business Logic Patterns](#business-logic-patterns)
+6. [Event Sourcing](#event-sourcing)
+7. [Query Patterns](#query-patterns)
+8. [Testing Patterns](#testing-patterns)
+9. [Deployment Patterns](#deployment-patterns)
+10. [Observability Patterns](#observability-patterns)
+
+---
+
+## Decomposition Patterns
+
+### Decompose by Business Capability
+
+Map services to what the organization *does*. Business capabilities are stable
+over time even as org structure changes.
+
+- Identify top-level capabilities (Order Management, Delivery, Billing, etc.)
+- Each capability becomes a candidate service
+- Services own the data for their capability
+
+**When to use**: Starting a new microservices project or breaking up a monolith.
+
+### Decompose by Subdomain (DDD)
+
+Use Domain-Driven Design to identify bounded contexts. Each bounded context
+becomes a service.
+
+- Core subdomains: the competitive advantage (invest the most here)
+- Supporting subdomains: necessary but not differentiating
+- Generic subdomains: solved problems (use off-the-shelf solutions)
+
+**When to use**: Complex domain where business capability mapping isn't granular enough.
+
+### Strangler Fig Pattern
+
+Incrementally migrate from monolith to microservices by building new functionality
+as services and gradually routing traffic away from the monolith.
+
+- Stand up new service alongside monolith
+- Route specific requests to new service
+- Gradually migrate functionality
+- Eventually decommission monolith component
+
+**When to use**: Migrating an existing monolith without a big-bang rewrite.
+
+---
+
+## Communication Patterns
+
+### Synchronous — REST
+
+- Use for simple request/response interactions
+- Design APIs using the Richardson Maturity Model (ideally Level 2+)
+- Define IDL: OpenAPI/Swagger for REST
+- Handle partial failure: timeouts, retries, circuit breakers
+
+### Synchronous — gRPC
+
+- Binary protocol, strongly typed via Protocol Buffers
+- More efficient than REST for inter-service calls
+- Supports streaming (server, client, bidirectional)
+- Good for polyglot environments (code generation for many languages)
+
+### Asynchronous — Messaging
+
+- Services communicate via message broker (Kafka, RabbitMQ, etc.)
+- Message types: **Document** (carries data), **Command** (request action), **Event** (notification of change)
+- Channels: **Point-to-point** (one consumer) or **Publish-subscribe** (many consumers)
+
+Key implementation concerns:
+
+- **Message ordering**: Use partitioned channels (e.g., Kafka partitions keyed by aggregate ID)
+- **Duplicate handling**: Make consumers idempotent (track processed message IDs, or make operations naturally idempotent)
+- **Transactional outbox**: Write events to an OUTBOX table in the same transaction as business data, then relay to broker — ensures atomicity without distributed transactions
+- **Polling publisher or Transaction log tailing**: Two strategies for relaying outbox messages to the broker
+
+### Circuit Breaker
+
+Wrap remote calls in a circuit breaker to handle downstream failures gracefully:
+
+- **Closed**: Requests pass through normally
+- **Open**: Requests fail immediately (after failure threshold exceeded)
+- **Half-open**: Periodically try a request; if it succeeds, close the circuit
+
+Use libraries like Resilience4j (Java) or Polly (.NET).
+
+### Service Discovery
+
+Services need to locate each other. Two approaches:
+
+- **Client-side discovery**: Client queries a registry (e.g., Eureka) and load-balances
+- **Server-side discovery**: Client calls a router/load balancer that queries the registry (e.g., Kubernetes services, AWS ELB)
+
+---
+
+## API Gateway Patterns
+
+### API Gateway
+
+Single entry point for external clients. Responsibilities:
+
+- Request routing to appropriate microservice
+- API composition (aggregate responses from multiple services)
+- Protocol translation (external REST to internal gRPC/messaging)
+- Authentication and rate limiting
+- Edge functions (caching, monitoring, etc.)
+
+### Backend for Frontend (BFF)
+
+Separate API gateway per client type (web, mobile, third-party). Each BFF:
+
+- Tailors the API to its client's needs
+- Handles client-specific data aggregation
+- Is owned by the client team
+- Reduces coupling between client and backend service evolution
+
+**When to use BFF over single gateway**: When different clients have significantly different API needs.
+
+---
+
+## Transaction Management — Sagas
+
+### The Problem
+
+Microservices use Database per Service, so you cannot use ACID transactions
+across services. Sagas maintain data consistency using a sequence of local
+transactions with compensating transactions for rollback.
+
+### Choreography-based Saga
+
+Services publish events and subscribe to each other's events:
+
+```
+OrderService -> OrderCreated event
+  -> PaymentService listens, processes payment, publishes PaymentAuthorized
+    -> KitchenService listens, creates ticket, publishes TicketCreated
+      -> OrderService listens, approves order
+```
+
+- Pros: Simple, no central coordinator, loose coupling
+- Cons: Hard to understand the flow, cyclic dependencies possible
+
+### Orchestration-based Saga
+
+A central saga orchestrator tells participants what to do:
+
+```
+CreateOrderSaga:
+  1. OrderService.createOrder(PENDING)
+  2. PaymentService.authorize() -> fail? -> OrderService.rejectOrder()
+  3. KitchenService.createTicket() -> fail? -> PaymentService.reverseAuth(), OrderService.rejectOrder()
+  4. OrderService.approveOrder()
+```
+
+- Pros: Clear flow, easy to understand, avoids cyclic dependencies
+- Cons: Risk of centralizing too much logic in orchestrator
+
+### Saga Design Rules
+
+- Each saga step modifies one aggregate in one service
+- Every forward step needs a compensating transaction (undo/rollback action)
+- Compensating transactions must be idempotent (safe to retry)
+- Use semantic locks — mark records as "pending" during saga execution
+- Countermeasures for lack of isolation: semantic locks, commutative updates, pessimistic/optimistic views, re-reading values
+
+---
+
+## Business Logic Patterns
+
+### Aggregate Pattern (DDD)
+
+An aggregate is a cluster of domain objects treated as a unit for data changes.
+
+- **Aggregate root**: Top-level entity through which all external access occurs
+- **Invariants**: Business rules enforced within the aggregate
+- **Transaction boundary**: One transaction = one aggregate update
+- **References between aggregates**: Use IDs, not object references
+
+Design rules:
+- Keep aggregates small — reference other aggregates by identity
+- Business logic lives in the aggregate, not in service classes
+- Aggregates publish domain events when their state changes
+
+### Domain Events
+
+Events represent something meaningful that happened in the domain:
+
+- Named in past tense: OrderCreated, PaymentAuthorized, TicketAccepted
+- Contain relevant data (IDs, state at time of event)
+- Published by aggregates after state changes
+- Consumed by other services for integration
+
+### Domain Event Publishing
+
+Two approaches to reliably publish events:
+
+1. **Transactional outbox**: Store events in an outbox table in the same DB transaction as business data, then asynchronously publish to message broker
+2. **Event sourcing**: Events are the primary store (see below)
+
+---
+
+## Event Sourcing
+
+Instead of storing current state, store a sequence of state-changing events.
+Reconstruct current state by replaying events.
+
+### How It Works
+
+- Each aggregate stored as a sequence of events in an event store
+- To load: fetch all events, replay to reconstruct state
+- To save: append new events to the store
+- Event store is append-only (never update, never delete)
+
+### Benefits
+
+- Reliable event publishing (events ARE the data — no outbox needed)
+- Complete audit trail
+- Temporal queries (reconstruct state at any point in time)
+- Enables CQRS naturally
+
+### Challenges
+
+- Learning curve; different way of thinking
+- Querying the event store is hard (need CQRS for queries)
+- Event schema evolution — events are immutable, so versioning matters
+- Deleting data (e.g., GDPR) requires special handling like encryption with deletable keys
+
+### Snapshots
+
+For aggregates with many events, periodically save a snapshot of current state
+to avoid replaying the entire history. Load: last snapshot + events after snapshot.
+
+---
+
+## Query Patterns
+
+### API Composition
+
+For queries spanning multiple services, an API composer calls each service
+and combines results in memory.
+
+```
+API Composer (or API Gateway):
+  1. Call OrderService.getOrders(customerId)
+  2. Call DeliveryService.getDeliveries(orderIds)
+  3. Call RestaurantService.getRestaurants(restaurantIds)
+  4. Join results in memory, return to client
+```
+
+- Pros: Simple to implement
+- Cons: Increased latency, no efficient join/filter across services, reduced availability
+
+**When to use**: Simple queries where data from 2-3 services needs combining.
+
+### CQRS (Command Query Responsibility Segregation)
+
+Separate the write model (commands) from the read model (queries):
+
+- **Command side**: Services handle commands, publish domain events
+- **Query side**: A separate view service subscribes to events and maintains denormalized read-optimized database
+
+```
+OrderService publishes OrderCreated ->
+DeliveryService publishes DeliveryStatusUpdated ->
+  OrderHistoryViewService subscribes to both,
+  maintains denormalized OrderHistoryView table,
+  serves GET /order-history queries
+```
+
+- Pros: Efficient queries, scales reads independently, supports complex cross-service queries
+- Cons: Extra complexity, eventual consistency, additional infrastructure
+
+**When to use**: Complex queries spanning many services, or when read performance is critical.
+
+---
+
+## Testing Patterns
+
+### Consumer-Driven Contract Testing
+
+Each consumer defines a contract: "I expect your API to behave like X."
+Provider runs these contracts as tests to ensure compatibility.
+
+- Tools: Spring Cloud Contract, Pact
+- Catches breaking changes before deployment
+
+### Service Component Testing
+
+Test a service in isolation by stubbing its dependencies:
+
+- In-memory stubs for downstream services
+- In-memory or containerized database
+- Verify behavior end-to-end through the service's API
+
+### Integration Testing
+
+Test interaction between a service and its infrastructure:
+
+- Database integration tests (verify SQL, schema migrations)
+- Messaging integration tests (verify event publishing/consuming)
+- REST client integration tests (verify HTTP calls)
+- Use Docker (Testcontainers) for realistic infrastructure
+
+### Testing Pyramid for Microservices
+
+From bottom (fast, many) to top (slow, few):
+
+1. Unit tests — domain logic, aggregates, sagas
+2. Integration tests — persistence, messaging, REST clients
+3. Component tests — single service end-to-end with stubs
+4. Contract tests — API compatibility between consumer and provider
+5. End-to-end tests — full system (keep these minimal)
+
+---
+
+## Deployment Patterns
+
+### Service per Container
+
+Package each service as a Docker container image:
+
+- Encapsulates service and dependencies
+- Consistent across environments
+- Use multi-stage builds to keep images small
+
+### Kubernetes Orchestration
+
+- Deployment: Manages replicas and rolling updates
+- Service: Stable network endpoint for service discovery
+- ConfigMap/Secret: Externalized configuration
+- Probes: Liveness and readiness health checks
+
+### Service Mesh
+
+Infrastructure layer handling service-to-service communication:
+
+- Automatic mTLS between services
+- Traffic management (retries, timeouts, circuit breaking)
+- Observability (distributed tracing, metrics)
+- Tools: Istio, Linkerd
+
+### Externalized Configuration
+
+- Environment variables, config servers (Spring Cloud Config), Kubernetes ConfigMaps
+- Secrets managed separately (Vault, Kubernetes Secrets)
+- Same artifact runs in different environments
+
+---
+
+## Observability Patterns
+
+### Health Check API
+
+Every service exposes a health endpoint (GET /health) reporting:
+- Service status (UP/DOWN)
+- Dependency status (database, message broker)
+- Used by orchestrators and load balancers
+
+### Distributed Tracing
+
+Assign unique trace ID to each external request, propagate through all service calls.
+- Tools: Zipkin, Jaeger, AWS X-Ray
+- Instrument with OpenTelemetry
+
+### Log Aggregation
+
+Centralize logs from all services:
+- Include trace IDs for correlation
+- Structured logging (JSON format)
+- Tools: ELK Stack, Fluentd, Datadog
+
+### Application Metrics
+
+Collect and expose metrics from each service:
+- Request rate, latency, error rate (RED method)
+- Resource utilization
+- Business metrics
+- Tools: Prometheus + Grafana, Datadog, CloudWatch