@cubis/foundry 0.3.82 → 0.3.84

package/workflows/skills/microservices-design/references/communication-patterns.md

@@ -1,236 +0,0 @@
-# Communication Patterns
-
-## Pattern Selection Matrix
-
-| Pattern | Coupling | Latency | Reliability | Use when |
-|---------|---------|---------|-------------|----------|
-| **Synchronous HTTP/REST** | High | Low | Caller fails if callee fails | Simple CRUD, real-time queries |
-| **Synchronous gRPC** | High | Very low | Caller fails if callee fails | Internal high-perf communication |
-| **Async messaging (queue)** | Low | Higher | Messages buffered during outage | Commands that tolerate delay |
-| **Async events (pub/sub)** | Very low | Higher | Events buffered during outage | Domain events, notifications |
-| **Request-reply (async)** | Medium | Higher | Correlation-based response | Long-running operations |
-
-## Synchronous Communication
-
-### REST over HTTP
-
-```
-Service A ──HTTP GET /products/123──► Service B
-Service A ◄──200 OK { product }────── Service B
-```
-
-When to use:
-- Public-facing APIs consumed by browsers or mobile apps
-- Simple CRUD operations with immediate response needs
-- Stateless, cacheable reads
-
-When to avoid:
-- Long chains of synchronous calls (A → B → C → D)
-- Commands where the caller does not need an immediate result
-- High-throughput internal communication (HTTP overhead is significant)
-
-### gRPC
-
-```
-Service A ──gRPC GetProduct(id=123)──► Service B
-Service A ◄──Product { ... }────────── Service B
-```
-
-Advantages over REST:
-- Binary serialization (Protobuf) — 3-10x smaller payloads
-- HTTP/2 multiplexing — multiple requests over one connection
-- Streaming — server-side, client-side, and bidirectional streams
-- Code generation — typed clients and servers from .proto files
-
-When to use:
-- Internal service-to-service communication
-- High-throughput, low-latency requirements
-- When strong typing between services matters
-
-### Service Mesh
-
-A service mesh (Istio, Linkerd) handles cross-cutting communication concerns:
-
-| Concern | Without mesh | With mesh |
-|---------|-------------|-----------|
-| mTLS encryption | Each service implements TLS | Mesh handles automatically |
-| Retries | Each service implements retry logic | Mesh retries with configurable policy |
-| Circuit breaking | Each service implements circuit breaker | Mesh circuit breaker per route |
-| Load balancing | Client-side or external LB | Mesh sidecar proxy handles |
-| Observability | Each service instruments traces | Mesh emits traces automatically |
-| Traffic splitting | Application-level routing | Mesh traffic rules (canary, A/B) |
-
-Architecture:
-```
-┌───────────┐     ┌───────────┐
-│ Service A │     │ Service B │
-│           │     │           │
-│ ┌───────┐ │     │ ┌───────┐ │
-│ │ App   │ │     │ │ App   │ │
-│ └───┬───┘ │     │ └───┬───┘ │
-│     │     │     │     │     │
-│ ┌───▼───┐ │     │ ┌───▼───┐ │
-│ │Sidecar│ │◄───►│ │Sidecar│ │
-│ │ Proxy │ │     │ │ Proxy │ │
-│ └───────┘ │     │ └───────┘ │
-└───────────┘     └───────────┘
-```
-
-The sidecar proxy (Envoy) intercepts all inbound and outbound traffic. The application code communicates with localhost; the proxy handles everything else.
-
-## API Gateway
-
-The API gateway is the single entry point for external traffic:
-
-```
-External clients
-      │
-      ▼
-┌──────────────────────────────┐
-│         API Gateway          │
-│                              │
-│  ┌─────────────────────────┐ │
-│  │ Authentication (JWT)    │ │
-│  │ Rate limiting           │ │
-│  │ Request routing         │ │
-│  │ Protocol translation    │ │
-│  │ Response aggregation    │ │
-│  │ TLS termination         │ │
-│  │ Request/response logging│ │
-│  └─────────────────────────┘ │
-└──────────┬───────────────────┘
-           │
-     ┌─────┼─────────┐
-     │     │         │
-     ▼     ▼         ▼
-  Svc A  Svc B    Svc C
-```
-
-### Gateway Anti-Patterns
-
-- **Business logic in the gateway:** The gateway routes; it does not process. Move business logic to services.
-- **Gateway as orchestrator:** Do not have the gateway call multiple services and merge results. Use a dedicated aggregation service or BFF (Backend-for-Frontend).
-- **Single gateway for all consumers:** Mobile, web, and internal consumers have different needs. Consider separate gateways or BFF per client type.
-
-## Asynchronous Communication
-
-### Command via Queue
-
-One producer, one consumer group. The message is a command to perform an action:
-
-```
-Order Service ──"ProcessPayment"──► Payment Queue ──► Payment Consumer
-```
-
-The command is consumed once. The consumer performs the action and may publish a result event.
-
-### Event via Topic
-
-One producer, multiple consumer groups. The message is a notification that something happened:
-
-```
-Order Service ──"OrderPlaced"──► Orders Topic ──► Payment Consumer
-                                              ──► Inventory Consumer
-                                              ──► Notification Consumer
-                                              ──► Analytics Consumer
-```
-
-Each consumer group processes the event independently. The producer does not know or care who consumes.
-
-### Request-Reply (Async)
-
-For long-running operations where the caller needs a result:
-
-```
-1. Client sends request with correlation_id and reply_to topic
-2. Server processes (may take seconds to minutes)
-3. Server publishes result to reply_to topic with same correlation_id
-4. Client matches response by correlation_id
-```
-
-Use for: Report generation, data exports, batch processing, AI inference.
-
-## Communication Pattern Decisions
-
-### Choreography vs. Orchestration
-
-| Aspect | Choreography | Orchestration |
-|--------|-------------|---------------|
-| Control flow | Decentralized (each service reacts to events) | Centralized (orchestrator manages sequence) |
-| Coupling | Very loose (services know events, not each other) | Medium (orchestrator knows all services) |
-| Visibility | Hard to see end-to-end flow | Easy to see flow in orchestrator |
-| Debugging | Trace events across services | Debug the orchestrator |
-| Best for | Simple flows (< 4 steps), loosely related services | Complex flows (4+ steps), strict sequencing |
-
-### Choosing Sync vs. Async
-
-```
-Decision tree:
-  1. Does the caller NEED the result RIGHT NOW?
-     Yes → Synchronous (HTTP/gRPC)
-     No  → Continue
-  2. Can the operation fail and be retried later?
-     Yes → Async via message queue
-     No  → Synchronous with retry
-  3. Do multiple services need to react?
-     Yes → Async event (pub/sub)
-     No  → Async command (queue)
-```
-
-## Resilience Patterns
-
-### Retry with Backoff
-
-```
-attempt 1: immediate
-attempt 2: wait 1s
-attempt 3: wait 2s
-attempt 4: wait 4s
-give up → fallback or error
-```
-
-### Circuit Breaker
-
-```
-CLOSED → requests pass through, track failure rate
-  failure rate > 50% for 10 requests → OPEN
-OPEN → requests fail immediately, return fallback
-  after 30s → HALF-OPEN
-HALF-OPEN → allow 1 probe request
-  success → CLOSED
-  failure → OPEN
-```
-
-### Bulkhead
-
-Isolate resources per downstream service:
-
-```
-Payment calls:     10 concurrent max (semaphore)
-Inventory calls:   10 concurrent max (semaphore)
-Notification calls: 5 concurrent max (semaphore)
-```
-
-If Payment calls exhaust their semaphore, Inventory and Notification calls are unaffected.
-
-### Timeout Cascade
-
-Each layer has a shorter timeout than its caller:
-
-```
-API Gateway:  10s timeout
-  └── Order Service: 8s timeout
-       └── Payment Service: 5s timeout
-            └── External Payment Gateway: 3s timeout
-```
-
-### Fallback
-
-When a downstream service fails, return a degraded but useful response:
-
-| Service down | Fallback behavior |
-|-------------|-------------------|
-| Recommendation engine | Show popular items instead of personalized |
-| Search service | Show cached results, disable autocomplete |
-| Payment service | Queue the charge, confirm order tentatively |
-| Notification service | Log the notification, retry later (no user impact) |

package/workflows/skills/microservices-design/references/decomposition.md

@@ -1,177 +0,0 @@
-# Service Decomposition
-
-## When to Decompose
-
-Microservices add operational complexity. Decompose only when the benefits outweigh the costs:
-
-| Signal that decomposition helps | Signal that a monolith is fine |
-|-------------------------------|-------------------------------|
-| Different parts scale independently (catalog reads vs. payment writes) | Uniform scaling needs across the system |
-| Multiple teams need to deploy independently | Single team owns the entire codebase |
-| Different technology stacks are optimal for different parts | One stack serves all needs well |
-| Compliance requires isolation (PCI scope for payments) | No regulatory isolation requirements |
-| Parts have fundamentally different availability needs | Uniform availability target |
-
-## Domain-Driven Design Process
-
-### Step 1: Event Storming
-
-Gather domain experts and map out the business events:
-
-```
-Timeline of events for an e-commerce order:
-  Customer searches catalog
-  Customer adds item to cart
-  Customer places order         ← domain event: OrderPlaced
-  System validates payment      ← domain event: PaymentAuthorized
-  System reserves inventory     ← domain event: InventoryReserved
-  Warehouse picks items
-  Carrier picks up package      ← domain event: ShipmentDispatched
-  Customer receives package     ← domain event: ShipmentDelivered
-  Customer submits review       ← domain event: ReviewSubmitted
-```
-
-### Step 2: Identify Aggregates
-
-Group related events around the entities they affect:
-
-```
-Order aggregate:     OrderPlaced, OrderConfirmed, OrderCancelled
-Payment aggregate:   PaymentAuthorized, PaymentCaptured, PaymentRefunded
-Inventory aggregate: InventoryReserved, InventoryReleased, StockDepleted
-Shipment aggregate:  ShipmentCreated, ShipmentDispatched, ShipmentDelivered
-```
-
-### Step 3: Draw Bounded Context Boundaries
-
-Group aggregates that share a ubiquitous language into bounded contexts:
-
-```
-┌─ Order Context ──────────────┐  ┌─ Payment Context ──────────┐
-│ Order, OrderLine, OrderStatus │  │ Payment, Refund, Method    │
-│ "Order" means a purchase      │  │ "Order" means a payment ref│
-└───────────────────────────────┘  └────────────────────────────┘
-
-┌─ Catalog Context ────────────┐  ┌─ Inventory Context ────────┐
-│ Product, Category, Price      │  │ StockLevel, Reservation    │
-│ "Product" means catalog entry │  │ "Product" means stock unit │
-└───────────────────────────────┘  └────────────────────────────┘
-```
-
-Key insight: the same word ("Product," "Order") means different things in different contexts. This is expected and healthy.
-
-### Step 4: Define Context Maps
-
-Document how bounded contexts relate:
-
-| Relationship | Description | Example |
-|-------------|-------------|---------|
-| **Customer-Supplier** | Upstream provides data, downstream consumes | Catalog (supplier) → Order (customer) |
-| **Shared Kernel** | Two contexts share a subset of the model | Identity shared between all contexts (user ID) |
-| **Anti-Corruption Layer** | Downstream translates upstream's model | Legacy billing system → Payment context adapter |
-| **Conformist** | Downstream accepts upstream's model as-is | Analytics conforms to event schemas from all contexts |
-| **Published Language** | Shared schema for inter-context communication | Event schemas in Avro/Protobuf |
-
-## Service Boundary Validation
-
-After identifying boundaries, validate each one:
-
-### The Team Test
-Can a single team (5-8 people) own this service end-to-end? If a service requires coordination across multiple teams, the boundary is wrong.
-
-### The Independent Deployment Test
-Can this service be deployed without deploying any other service? If a change requires coordinated deployment, the coupling is too tight.
-
-### The Data Ownership Test
-Does this service own all the data it needs? If it shares a database table with another service, the boundary cuts through data ownership.
-
-### The Failure Isolation Test
-If this service goes down, does it take other services with it? If yes, the coupling is too tight or the failure handling is missing.
-
-### The Business Capability Test
-Does this service map to a business capability that a non-technical person could describe? "Payment processing" makes sense. "Database access layer" does not.
-
-## Decomposition Anti-Patterns
-
-### Distributed Monolith
-
-Services that must be deployed together, share a database, or cannot function without calling each other synchronously. You get the complexity of distribution with none of the benefits.
-
-Signs:
-- Every feature change touches 3+ services
-- Deployment order matters
-- Shared database tables between services
-- High fan-out: one request triggers 10+ inter-service calls
-
-Fix: Merge tightly coupled services. Decompose along different boundaries.
-
-### Entity Service
-
-Services defined around database entities rather than business capabilities: UserService, ProductService, OrderService each as thin CRUD wrappers.
-
-Signs:
-- Services are anemic (no business logic, just pass-through to database)
-- Business logic lives in the caller, not the service
-- Changes to a business process touch every entity service
-
-Fix: Define services around capabilities (Checkout, Fulfillment, Search) not entities.
-
-### Nano-Services
-
-Services that are too small to justify their operational overhead.
-
-Signs:
-- Service has < 500 lines of code
-- Service has a single API endpoint
-- Service team is < 2 people
-- Service deployment pipeline is more complex than the service itself
-
-Fix: Merge with the related service. Not everything needs to be a separate deployable.
-
-## Shared Data Strategies
-
-When two services need access to the same data:
-
-| Strategy | How | When |
-|----------|-----|------|
-| **API call** | Service A calls Service B's API | Infrequent access, real-time needed |
-| **Event propagation** | Service B publishes events, Service A stores a local copy | Frequent access, eventual consistency OK |
-| **Data replication** | Service B publishes change events, Service A maintains a read model | High-frequency reads, low-frequency writes |
-| **Shared reference data** | Common data (countries, currencies) in a library or config | Rarely changes, needed everywhere |
-
-Never: Share a database between services. This creates hidden coupling that defeats the purpose of microservices.
-
-## Migration Strategy: Strangler Fig
-
-### Phases
-
-```
-Phase 1: Proxy
-  External traffic → API Gateway → Monolith
-  (Gateway routes everything to monolith)
-
-Phase 2: Extract first service
-  External traffic → API Gateway ──→ New Service (one capability)
-                              └──→ Monolith (everything else)
-
-Phase 3: Extract more services
-  External traffic → API Gateway ──→ Service A
-                              ├──→ Service B
-                              ├──→ Service C
-                              └──→ Monolith (shrinking)
-
-Phase 4: Decommission monolith
-  External traffic → API Gateway ──→ Service A
-                              ├──→ Service B
-                              ├──→ Service C
-                              └──→ Service D
-```
-
-### Migration Rules
-
-1. **Extract the easiest context first** — lowest coupling, clearest boundary (often notifications or search).
-2. **Dual-write during migration** — write to both old and new data stores, verify consistency.
-3. **Feature flags on every extraction** — instant rollback to monolith behavior.
-4. **Shadow mode before cut-over** — run both paths, compare results, alert on divergence.
-5. **One service per migration phase** — never extract multiple services simultaneously.
-6. **Verify before proceeding** — confirm the extracted service is stable for 2+ weeks before starting the next extraction.

package/workflows/skills/microservices-design/references/deployment.md

@@ -1,250 +0,0 @@
-# Deployment Strategies
-
-## Deployment Patterns
-
-### Rolling Deployment
-
-```
-Time 0:  [v1] [v1] [v1] [v1]    ← All instances on v1
-Time 1:  [v2] [v1] [v1] [v1]    ← First instance updated
-Time 2:  [v2] [v2] [v1] [v1]    ← Second instance updated
-Time 3:  [v2] [v2] [v2] [v1]    ← Third instance updated
-Time 4:  [v2] [v2] [v2] [v2]    ← All instances on v2
-```
-
-| Pro | Con |
-|-----|-----|
-| Zero downtime | Both versions run simultaneously during rollout |
-| Simple to implement | Slow rollback (must roll forward or re-deploy v1) |
-| Resource efficient (no extra instances) | Database schema must be compatible with both versions |
-
-Configuration:
-```yaml
-strategy:
-  type: RollingUpdate
-  rollingUpdate:
-    maxSurge: 25%         # Allow 25% extra instances during update
-    maxUnavailable: 25%   # Allow 25% instances to be down during update
-```
-
-### Blue-Green Deployment
-
-```
-Blue (v1):  [v1] [v1] [v1] [v1]  ← Currently serving traffic
-Green (v2): [v2] [v2] [v2] [v2]  ← Deployed, tested, not serving
-
-Switch: Load balancer routes traffic from Blue to Green
-Rollback: Load balancer routes traffic back to Blue
-```
-
-| Pro | Con |
-|-----|-----|
-| Instant rollback (switch LB back) | Requires 2x infrastructure |
-| Full testing of v2 before exposure | Database migrations must be backward compatible |
-| No mixed-version traffic | Expensive for large deployments |
-
-### Canary Deployment
-
-```
-Time 0: 100% → [v1] [v1] [v1] [v1]
-
-Time 1:   5% → [v2]
-          95% → [v1] [v1] [v1]
-
-Time 2:  25% → [v2] [v2]
-          75% → [v1] [v1] [v1]
-
-Time 3:  50% → [v2] [v2] [v2]
-          50% → [v1] [v1]
-
-Time 4: 100% → [v2] [v2] [v2] [v2]
-```
-
-| Pro | Con |
-|-----|-----|
-| Limited blast radius | More complex routing configuration |
-| Real production traffic testing | Metrics must be split by version |
-| Automated promotion/rollback based on metrics | Both versions must coexist |
-
-### Canary Configuration
-
-```yaml
-analysis:
-  metrics:
-    - name: error-rate
-      threshold: 1%        # Rollback if error rate > 1%
-      interval: 60s
-    - name: p99-latency
-      threshold: 500ms     # Rollback if p99 > 500ms
-      interval: 60s
-  steps:
-    - setWeight: 5
-      pause: { duration: 5m }
-    - setWeight: 25
-      pause: { duration: 10m }
-    - setWeight: 50
-      pause: { duration: 15m }
-    - setWeight: 100
-```
-
-## CI/CD Pipeline
-
-### Pipeline Stages
-
-```
-┌────────┐   ┌──────┐   ┌──────────┐   ┌─────────┐   ┌────────┐   ┌──────────┐
-│ Build  │──►│ Test │──►│ Security │──►│ Package │──►│ Deploy │──►│ Verify  │
-│        │   │      │   │  Scan    │   │         │   │        │   │         │
-└────────┘   └──────┘   └──────────┘   └─────────┘   └────────┘   └──────────┘
-```
-
-| Stage | Actions | Gate |
-|-------|---------|------|
-| Build | Compile, resolve dependencies | Compilation succeeds |
-| Test | Unit tests, integration tests, contract tests | All tests pass, coverage > threshold |
-| Security | SAST, dependency vulnerability scan, secret detection | No critical/high vulnerabilities |
-| Package | Build container image, push to registry | Image builds and passes health check |
-| Deploy | Deploy to staging, then production (canary) | Staging smoke tests pass |
-| Verify | Post-deployment smoke tests, metric validation | Error rate and latency within SLO |
-
-### Independent Deployability
-
-Each microservice must have its own CI/CD pipeline:
-
-```
-Repo: order-service/
-  .github/workflows/deploy.yml    ← Triggered on push to main
-  Dockerfile
-  src/
-  tests/
-
-Repo: payment-service/
-  .github/workflows/deploy.yml    ← Independent pipeline
-  Dockerfile
-  src/
-  tests/
-```
-
-Rule: A change to order-service must NEVER require deploying payment-service. If it does, the service boundary is wrong.
-
-### Contract Testing in CI
-
-```
-┌─────────────────┐     contract      ┌──────────────────┐
-│ Order Service   │ ←────────────────► │ Payment Service  │
-│ (consumer)      │                    │ (provider)       │
-│                 │     Pact broker    │                  │
-│ Generates pact  │ ──────────────► ──►│ Verifies pact   │
-│ in CI           │                    │ in CI            │
-└─────────────────┘                    └──────────────────┘
-```
-
-1. Consumer (Order) generates a contract (Pact file) during its CI.
-2. Contract is published to a Pact broker.
-3. Provider (Payment) CI fetches the contract and runs verification.
-4. If verification fails, Payment CI fails. The provider team must not break the consumer's expectations.
-
-## Database Migration Strategy
-
-### Expand-Contract Pattern
-
-When a schema change breaks backward compatibility:
-
-```
-Phase 1: Expand (v1 and v2 compatible)
-  - Add new column (nullable or with default)
-  - Both v1 and v2 code work with the schema
-  - Deploy v2 code
-
-Phase 2: Migrate
-  - Backfill new column with data from old column
-  - Verify data integrity
-
-Phase 3: Contract (v2 only)
-  - Remove old column
-  - This is safe because v1 code is no longer deployed
-```
-
-### Migration Rules
-
-1. **Never rename a column** in a single step. Add new, migrate data, remove old.
-2. **Never drop a column** until all services using it are updated.
-3. **Always add columns as nullable** or with a default value.
-4. **Run migrations before deploying** the code that uses them.
-5. **Test rollback** — every migration must have a reverse migration.
-6. **Separate deployment from migration** — schema changes in one PR, code changes in the next.
-
-## API Versioning During Deployment
-
-When deploying a breaking API change:
-
-```
-Phase 1: Deploy v2 endpoint alongside v1
-  GET /v1/orders → existing behavior
-  GET /v2/orders → new behavior
-
-Phase 2: Migrate consumers to v2
-  - Update each consumer to use /v2/orders
-  - Monitor v1 traffic dropping to zero
-
-Phase 3: Deprecate v1
-  - Return Sunset header: Sunset: Sat, 01 Jun 2025 00:00:00 GMT
-  - Return Deprecation header: Deprecation: true
-  - Log v1 usage for tracking
-
-Phase 4: Remove v1
-  - After sunset date, return 410 Gone
-  - Eventually remove v1 code
-```
-
-## Rollback Procedures
-
-### Automated Rollback Triggers
-
-| Trigger | Action | Threshold |
-|---------|--------|-----------|
-| Error rate spike | Rollback canary, route all traffic to previous version | > 1% error rate for 2 minutes |
-| Latency increase | Rollback canary | p99 > 2x baseline for 5 minutes |
-| Health check failure | Kill unhealthy pod, stop rollout | 3 consecutive failures |
-| Crash loop | Stop rollout, alert on-call | > 3 restarts in 5 minutes |
-
-### Manual Rollback
-
-```bash
-# Kubernetes: rollback to previous revision
-kubectl rollout undo deployment/order-service
-
-# Verify rollback
-kubectl rollout status deployment/order-service
-
-# Check pods are healthy
-kubectl get pods -l app=order-service
-```
-
-### Rollback Checklist
-
-1. Verify the previous version is still available in the container registry.
-2. Confirm the database schema is backward compatible (expand-contract).
-3. Check that no new message formats were introduced that old consumers cannot parse.
-4. Verify feature flags can disable new functionality without rollback.
-5. After rollback, investigate root cause before re-deploying.
-
-## Feature Flags
-
-Use feature flags to decouple deployment from release:
-
-```
-Deploy v2 code with flag: ORDER_NEW_CHECKOUT_FLOW=false
-  → v2 code is in production but new behavior is disabled
-  → Gradually enable for 1%, 5%, 25%, 50%, 100% of users
-  → If issues arise, disable flag (instant rollback, no deployment)
-```
-
-| Flag type | Lifetime | Example |
-|-----------|----------|---------|
-| Release flag | Days to weeks | New checkout flow |
-| Experiment flag | Weeks to months | A/B test pricing page |
-| Ops flag | Permanent | Disable search during maintenance |
-| Permission flag | Permanent | Beta features for premium users |
-
-Clean up release flags after rollout is complete. Stale flags are tech debt.