swe-workflow-skills 0.1.0

package/skills/deployment-repo/references/deployment-repo-patterns.md

@@ -0,0 +1,258 @@
+# Deployment Repo Patterns by Scale Tier
+
+## Lightweight Tier (1 team, 2-5 services)
+
+Best for small teams that need coordinated local dev and basic version tracking without Kubernetes complexity.
+
+### Structure
+
+```
+deployment-repo/
+├── docker-compose.yml           # Full system for local dev
+├── docker-compose.override.yml  # Developer-specific overrides (gitignored)
+├── .versions.env                # Pinned service versions
+├── .env.example                 # Environment template (copy to .env)
+├── Makefile                     # Developer commands
+├── tests/
+│   └── e2e/
+│       ├── package.json         # E2E test dependencies
+│       └── specs/               # E2E test files
+├── docs/
+│   ├── architecture.md          # System overview
+│   └── local-setup.md           # Getting started guide
+└── scripts/
+    ├── setup.sh                 # First-time setup (clone repos, build images)
+    ├── seed.sh                  # Seed dev database
+    └── health-check.sh          # Verify all services are running
+```
+
+### Example docker-compose.yml
+
+```yaml
+services:
+  api:
+    image: ${REGISTRY}/api:${API_VERSION}
+    build:
+      context: ../api-repo          # For local dev with source
+      dockerfile: Dockerfile
+    ports:
+      - "8000:8000"
+    environment:
+      DATABASE_URL: postgres://user:pass@db:5432/app
+      REDIS_URL: redis://cache:6379
+    depends_on:
+      db:
+        condition: service_healthy
+      cache:
+        condition: service_healthy
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 10s
+      retries: 3
+
+  frontend:
+    image: ${REGISTRY}/frontend:${FRONTEND_VERSION}
+    build:
+      context: ../frontend-repo
+      dockerfile: Dockerfile
+    ports:
+      - "3000:3000"
+    environment:
+      API_URL: http://api:8000
+    depends_on:
+      api:
+        condition: service_healthy
+
+  db:
+    image: postgres:16-alpine
+    volumes:
+      - pgdata:/var/lib/postgresql/data
+      - ./scripts/init.sql:/docker-entrypoint-initdb.d/init.sql
+    environment:
+      POSTGRES_USER: user
+      POSTGRES_PASSWORD: pass
+      POSTGRES_DB: app
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U user"]
+      interval: 5s
+      retries: 5
+
+  cache:
+    image: redis:7-alpine
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 5s
+      retries: 3
+
+volumes:
+  pgdata:
+```
+
+### Example Makefile
+
+```makefile
+.PHONY: up down test logs setup
+
+up:                        ## Start all services
+	docker compose up -d
+
+down:                      ## Stop all services
+	docker compose down
+
+test:                      ## Run E2E tests
+	docker compose up -d
+	cd tests/e2e && npm test
+
+logs:                      ## Tail all service logs
+	docker compose logs -f
+
+setup:                     ## First-time setup
+	cp .env.example .env
+	docker compose build
+	docker compose up -d
+	./scripts/seed.sh
+
+bump-%:                    ## Bump a service version: make bump-api VERSION=v1.2.3
+	sed -i 's/^$*_VERSION=.*/$*_VERSION=$(VERSION)/' .versions.env
+	@echo "Updated $* to $(VERSION). Run 'make test' to validate."
+```
+
+### Example .versions.env
+
+```env
+API_VERSION=v1.3.2
+FRONTEND_VERSION=v2.1.0
+REGISTRY=ghcr.io/myorg
+```
+
+## Standard Tier (2-3 teams, 5-15 services)
+
+Adds Helm/Kustomize for multi-environment K8s deployments, structured CI, and contract testing.
+
+### Structure
+
+```
+deployment-repo/
+├── versions.yaml                # System-wide version pins
+├── docker-compose.yml           # Local dev orchestration
+├── Makefile
+├── helm/
+│   ├── Chart.yaml               # Umbrella chart
+│   ├── values.yaml              # Shared defaults
+│   └── values/
+│       ├── dev.yaml             # Dev overrides
+│       ├── staging.yaml         # Staging overrides
+│       └── production.yaml      # Production overrides
+├── contracts/
+│   ├── apis/                    # OpenAPI specs per service
+│   │   ├── user-service.v1.yaml
+│   │   └── api-gateway.v2.yaml
+│   └── events/                  # Event schemas
+│       └── user-created.v1.avsc
+├── tests/
+│   ├── e2e/
+│   └── contract/                # Contract verification tests
+├── ci/
+│   ├── validate-versions.yml    # Contract + E2E on version bump PRs
+│   └── promote.yml              # Environment promotion pipeline
+├── environments/
+│   ├── dev/
+│   │   └── versions.yaml
+│   ├── staging/
+│   │   └── versions.yaml
+│   └── production/
+│       └── versions.yaml
+├── infrastructure/
+│   └── terraform/               # Shared infra (databases, queues, networking)
+├── docs/
+│   ├── architecture/
+│   │   └── adr/                 # Architecture Decision Records
+│   ├── runbooks/
+│   └── onboarding.md
+└── scripts/
+```
+
+### Kustomize alternative to Helm
+
+```
+deployment-repo/
+├── base/
+│   ├── api-gateway/
+│   │   ├── deployment.yaml
+│   │   ├── service.yaml
+│   │   └── kustomization.yaml
+│   ├── user-service/
+│   └── ...
+├── overlays/
+│   ├── dev/
+│   │   ├── kustomization.yaml   # Patches: replicas, resource limits, image tags
+│   │   └── patches/
+│   ├── staging/
+│   └── production/
+```
+
+## Enterprise Tier (many teams, 15+ services)
+
+Adds ArgoCD/Flux integration, ApplicationSets, automated promotion, and audit infrastructure.
+
+### Structure
+
+```
+deployment-repo/
+├── versions.yaml
+├── argocd/                      # ArgoCD Application definitions
+│   ├── app-of-apps.yaml         # Root Application
+│   ├── dev/
+│   │   └── applications.yaml    # Per-service ArgoCD Applications for dev
+│   ├── staging/
+│   └── production/
+├── helm/
+│   ├── charts/                  # Per-service Helm charts (or references to external charts)
+│   └── values/                  # Per-environment values
+├── contracts/
+├── tests/
+├── ci/
+│   ├── validate-versions.yml
+│   ├── promote.yml
+│   └── drift-report.yml         # Detect config drift
+├── environments/
+├── infrastructure/
+├── policies/                    # OPA/Gatekeeper policies
+├── docs/
+└── scripts/
+```
+
+### App of Apps pattern (ArgoCD)
+
+```yaml
+# argocd/app-of-apps.yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: system-apps
+  namespace: argocd
+spec:
+  project: default
+  source:
+    repoURL: https://github.com/org/deployment-repo
+    targetRevision: main
+    path: argocd/production
+  destination:
+    server: https://kubernetes.default.svc
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+```
+
+## Anti-Patterns
+
+**Mixing app source with deployment config**: Leads to infinite CI loops (deploy triggers build triggers deploy). Keep them separate.
+
+**Using `latest` tags instead of pinned versions**: Destroys reproducibility. You can't answer "what was running in production last Tuesday?"
+
+**Per-service deployment repos**: Defeats the purpose. The value is seeing the whole system in one place.
+
+**Manual version tracking** (Slack messages, wiki pages): Will drift from reality within days. The deployment repo IS the tracking system.
+
+**Promoting services independently without testing the set**: Version A of service X might work with version B of service Y in staging but fail with version C that was deployed separately. Always validate the full version set.

package/skills/deployment-repo/references/version-compatibility.md

@@ -0,0 +1,227 @@
+# Version Compatibility in Polyrepo Architectures
+
+## Semantic Versioning as the Compatibility Contract
+
+Each app repo must follow semver strictly — it's the language the deployment repo uses to reason about compatibility:
+
+- **MAJOR** (v2.0.0): Breaking changes to the service's external interface (API contracts, event schemas, shared database schemas). Consumers MUST update.
+- **MINOR** (v1.3.0): New functionality, backward-compatible. Consumers MAY update to use new features.
+- **PATCH** (v1.2.4): Bug fixes, no interface changes. Consumers SHOULD update.
+
+**What counts as the "external interface":**
+- REST/gRPC API request and response shapes
+- Published event schemas (message payloads)
+- Shared database tables that other services read
+- Configuration contracts (expected environment variables, feature flags)
+- CLI interfaces consumed by other tools
+
+**Versioning enforcement in app repos:**
+- Tag releases with `vMAJOR.MINOR.PATCH` in git
+- CI publishes Docker images tagged with the git tag
+- Breaking change detection should be automated where possible (OpenAPI diff, schema registry compatibility checks)
+
+## The Compatibility Lock
+
+The deployment repo's `versions.yaml` is a compatibility lock — it records version combinations that have been validated together.
+
+### Structure
+
+```yaml
+# versions.yaml
+services:
+  api-gateway:
+    image: registry.example.com/api-gateway
+    version: v2.3.1
+    min_compatible:
+      user-service: ">=v1.5.0"
+      billing-service: ">=v3.0.0"
+  user-service:
+    image: registry.example.com/user-service
+    version: v1.7.0
+  billing-service:
+    image: registry.example.com/billing-service
+    version: v3.1.2
+  notification-service:
+    image: registry.example.com/notification-service
+    version: v1.0.4
+  frontend:
+    image: registry.example.com/frontend
+    version: v4.2.0
+    min_compatible:
+      api-gateway: ">=v2.0.0"
+
+validated:
+  timestamp: "2025-01-15T14:30:00Z"
+  contract_tests: pass
+  e2e_tests: pass
+  validated_by: "ci-pipeline-run-1234"
+```
+
+### How the lock works
+
+1. Developer bumps `user-service` to `v1.8.0` in the deployment repo (via PR)
+2. CI reads `versions.yaml`, checks `min_compatible` constraints
+3. CI runs contract tests between all services at their pinned versions
+4. CI runs E2E tests against the composed system
+5. If all pass: PR is mergeable, `validated` block is updated
+6. If any fail: PR is blocked, developer must fix compatibility or coordinate a multi-service bump
+
+### Lightweight alternative (small teams)
+
+For the lightweight tier, a `.env` file serves the same purpose:
+
+```env
+# .versions.env — the compatibility lock (lightweight)
+API_GATEWAY_VERSION=v2.3.1
+USER_SERVICE_VERSION=v1.7.0
+BILLING_SERVICE_VERSION=v3.1.2
+FRONTEND_VERSION=v4.2.0
+```
+
+Referenced by `docker-compose.yml`:
+
+```yaml
+services:
+  api-gateway:
+    image: registry.example.com/api-gateway:${API_GATEWAY_VERSION}
+```
+
+## Consumer-Driven Contract Testing
+
+Contract tests are the mechanism that makes version pinning trustworthy. Without them, the compatibility lock records what was *assumed* compatible, not what was *verified*.
+
+### How Pact-style contracts work
+
+1. **Consumer defines expectations**: The frontend team writes a Pact test that says "when I call GET /users/123, I expect a response with `id`, `name`, and `email` fields"
+2. **Contract is published**: The contract is stored in a Pact Broker or the deployment repo's `contracts/` directory
+3. **Provider verifies**: The user-service CI runs the contract against its actual API — does it satisfy the frontend's expectations?
+4. **`can-i-deploy` check**: Before updating the deployment repo's version pins, query the Pact Broker: "is user-service v1.8.0 compatible with frontend v4.2.0?"
+
+### Contract testing in the deployment repo CI
+
+```yaml
+# .github/workflows/validate-versions.yml (simplified)
+name: Validate Version Compatibility
+on:
+  pull_request:
+    paths: ['versions.yaml']
+
+jobs:
+  contract-check:
+    steps:
+      - name: Read pinned versions
+        run: |
+          # Parse versions.yaml for all service versions
+      - name: Check Pact compatibility
+        run: |
+          pact-broker can-i-deploy \
+            --pacticipant api-gateway --version $API_GW_VERSION \
+            --pacticipant user-service --version $USER_SVC_VERSION \
+            --to-environment staging
+      - name: Run E2E smoke tests
+        run: |
+          docker compose up -d
+          npm run test:e2e
+```
+
+### Event schema compatibility
+
+For event-driven services, use a schema registry (Confluent, AWS Glue, or a simple Git-based registry in the deployment repo):
+
+- **Backward compatible**: New consumer can read old producer messages (add optional fields, don't remove fields)
+- **Forward compatible**: Old consumer can read new producer messages (ignore unknown fields)
+- **Full compatible**: Both directions work — this is the gold standard for event schemas
+
+Store schemas in the deployment repo:
+
+```
+contracts/
+├── events/
+│   ├── user-created.v1.avsc
+│   ├── user-created.v2.avsc    # Must be backward-compatible with v1
+│   └── order-placed.v1.json
+└── apis/
+    ├── api-gateway.openapi.yaml
+    └── user-service.openapi.yaml
+```
+
+CI validates schema evolution on every PR.
+
+## Breaking Change Coordination
+
+When a breaking change is unavoidable (major version bump), the deployment repo coordinates the migration:
+
+### Pattern: Coordinated multi-service bump
+
+```
+1. Provider deploys v2 alongside v1 (both endpoints active)
+2. Deployment repo PR: bump provider to v2, update consumers to use v2
+3. Contract tests verify all consumers work with provider v2
+4. E2E tests pass with the new version set
+5. Merge and promote through environments
+6. After all environments stable: provider removes v1 support
+7. Deployment repo PR: bump provider to v2.1 (v1 removed)
+```
+
+### Pattern: Expand-contract for shared data
+
+```
+1. Expand: Add new column/field alongside old (backward compatible)
+2. Deployment repo PR: bump service with migration, all other versions unchanged
+3. Migrate: Update consumers one by one to use new field
+4. Deployment repo PRs: bump each consumer as it's updated
+5. Contract: Remove old field once all consumers migrated
+6. Deployment repo PR: bump service with cleanup migration
+```
+
+### Version compatibility matrix
+
+For complex systems, maintain an explicit compatibility matrix:
+
+```yaml
+# compatibility-matrix.yaml
+api-gateway:
+  v2.x:
+    user-service: ">=v1.5.0, <v3.0.0"
+    billing-service: ">=v3.0.0"
+  v1.x:
+    user-service: ">=v1.0.0, <v2.0.0"
+    billing-service: ">=v2.0.0, <v3.0.0"
+```
+
+CI uses this matrix to validate that the pinned versions in `versions.yaml` are within the declared compatibility ranges.
+
+## Environment Promotion Workflows
+
+### Git-based promotion (recommended)
+
+Each environment has its own `versions.yaml` (or the deployment repo uses branches/directories per environment):
+
+```
+environments/
+├── dev/
+│   └── versions.yaml       # Latest validated versions
+├── staging/
+│   └── versions.yaml       # Promoted from dev after E2E pass
+└── production/
+    └── versions.yaml       # Promoted from staging after approval
+```
+
+**Promotion = copying versions.yaml from one environment to the next:**
+
+```bash
+# Promote dev to staging
+cp environments/dev/versions.yaml environments/staging/versions.yaml
+git commit -m "promote: dev version set -> staging"
+# PR triggers staging E2E tests
+```
+
+### Automated promotion pipeline
+
+```
+dev:      Auto-deploy on merge. E2E tests run post-deploy.
+staging:  Triggered by "promote to staging" PR. Full E2E + performance tests.
+prod:     Triggered by "promote to production" PR. Requires 2 approvals. Canary optional.
+```
+
+Rollback at any stage = revert the promotion commit. The previous version set is restored.

package/skills/deployment-repo/templates/deployment-repo-structure.md

@@ -0,0 +1,226 @@
+# Deployment Repo Structure Template
+
+Use this template to scaffold a deployment repo. Choose the tier that matches your scale and adjust to fit.
+
+## Lightweight Tier Scaffold
+
+```bash
+#!/bin/bash
+# scaffold-lightweight.sh — Create a lightweight deployment repo
+
+REPO_NAME="${1:-deployment-repo}"
+mkdir -p "$REPO_NAME"/{tests/e2e,docs,scripts}
+
+# Version pins
+cat > "$REPO_NAME/.versions.env" << 'EOF'
+# Service versions — the compatibility lock
+# Update via: make bump-<service> VERSION=vX.Y.Z
+API_VERSION=v0.1.0
+FRONTEND_VERSION=v0.1.0
+REGISTRY=ghcr.io/your-org
+EOF
+
+# Environment template
+cat > "$REPO_NAME/.env.example" << 'EOF'
+# Copy to .env and customize for local dev
+POSTGRES_USER=devuser
+POSTGRES_PASSWORD=devpass
+POSTGRES_DB=app_dev
+EOF
+
+# Docker Compose
+cat > "$REPO_NAME/docker-compose.yml" << 'EOF'
+# Load service versions from .versions.env
+# Usage: docker compose --env-file .versions.env up
+
+services:
+  api:
+    image: ${REGISTRY}/api:${API_VERSION}
+    ports:
+      - "8000:8000"
+    env_file: .env
+    depends_on:
+      db:
+        condition: service_healthy
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 10s
+      retries: 3
+
+  frontend:
+    image: ${REGISTRY}/frontend:${FRONTEND_VERSION}
+    ports:
+      - "3000:3000"
+    environment:
+      API_URL: http://api:8000
+    depends_on:
+      api:
+        condition: service_healthy
+
+  db:
+    image: postgres:16-alpine
+    env_file: .env
+    volumes:
+      - pgdata:/var/lib/postgresql/data
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
+      interval: 5s
+      retries: 5
+
+volumes:
+  pgdata:
+EOF
+
+# Makefile
+cat > "$REPO_NAME/Makefile" << 'MAKEOF'
+.PHONY: up down test logs setup health
+
+up:                          ## Start all services
+	docker compose --env-file .versions.env up -d
+
+down:                        ## Stop all services
+	docker compose down
+
+test:                        ## Run E2E tests against running services
+	cd tests/e2e && npm test
+
+logs:                        ## Tail all logs
+	docker compose logs -f
+
+setup:                       ## First-time setup
+	cp .env.example .env
+	docker compose --env-file .versions.env build
+	$(MAKE) up
+	@echo "System is starting. Run 'make health' to check status."
+
+health:                      ## Check all services are healthy
+	docker compose ps
+
+bump-%:                      ## Bump a service version: make bump-api VERSION=v1.2.3
+	@if [ -z "$(VERSION)" ]; then echo "Usage: make bump-$* VERSION=vX.Y.Z"; exit 1; fi
+	sed -i 's/^$(shell echo $* | tr a-z A-Z)_VERSION=.*/$(shell echo $* | tr a-z A-Z)_VERSION=$(VERSION)/' .versions.env
+	@echo "Updated $* to $(VERSION) in .versions.env"
+	@echo "Run 'make up && make test' to validate."
+MAKEOF
+
+# Gitignore
+cat > "$REPO_NAME/.gitignore" << 'EOF'
+.env
+docker-compose.override.yml
+node_modules/
+EOF
+
+# README
+cat > "$REPO_NAME/README.md" << 'EOF'
+# Deployment Repo
+
+Single source of truth for the system's deployment configuration.
+
+## Quick Start
+
+```bash
+make setup    # First-time: copies .env, builds, starts services
+make up       # Start all services
+make test     # Run E2E tests
+make down     # Stop all services
+```
+
+## Updating Service Versions
+
+```bash
+make bump-api VERSION=v1.2.3
+make up && make test
+git add .versions.env && git commit -m "bump api to v1.2.3"
+```
+
+## Adding a New Service
+
+1. Add the service to `docker-compose.yml`
+2. Add its version to `.versions.env`
+3. Update E2E tests in `tests/e2e/`
+4. Document in this README
+EOF
+
+echo "Scaffolded $REPO_NAME (lightweight tier)"
+```
+
+## Standard Tier Scaffold
+
+Extends lightweight with Helm, contract testing, and environment promotion:
+
+```bash
+#!/bin/bash
+# scaffold-standard.sh — Extend lightweight with Helm and contracts
+
+REPO_NAME="${1:-deployment-repo}"
+
+# Run lightweight scaffold first, then add:
+mkdir -p "$REPO_NAME"/{helm/values,contracts/{apis,events},tests/{e2e,contract}}
+mkdir -p "$REPO_NAME"/environments/{dev,staging,production}
+mkdir -p "$REPO_NAME"/{ci,infrastructure/terraform,docs/{architecture/adr,runbooks}}
+
+# versions.yaml (replaces .versions.env for standard tier)
+cat > "$REPO_NAME/versions.yaml" << 'EOF'
+# The compatibility lock — pinned service versions validated together
+services:
+  api-gateway:
+    image: registry.example.com/api-gateway
+    version: v1.0.0
+  user-service:
+    image: registry.example.com/user-service
+    version: v1.0.0
+  frontend:
+    image: registry.example.com/frontend
+    version: v1.0.0
+
+validated:
+  timestamp: null
+  contract_tests: pending
+  e2e_tests: pending
+EOF
+
+# Per-environment versions
+for env in dev staging production; do
+  cp "$REPO_NAME/versions.yaml" "$REPO_NAME/environments/$env/versions.yaml"
+done
+
+# Helm umbrella chart
+cat > "$REPO_NAME/helm/Chart.yaml" << 'EOF'
+apiVersion: v2
+name: system
+description: Umbrella chart for the full system
+version: 1.0.0
+dependencies:
+  - name: api-gateway
+    version: "1.0.0"
+    repository: "oci://registry.example.com/charts"
+  - name: user-service
+    version: "1.0.0"
+    repository: "oci://registry.example.com/charts"
+EOF
+
+echo "Scaffolded $REPO_NAME (standard tier)"
+```
+
+## Promotion Template
+
+PR template for environment promotion:
+
+```markdown
+## Promotion: [source] -> [target]
+
+### Version changes
+<!-- Diff of versions.yaml between environments -->
+
+### Validation
+- [ ] Contract tests pass in source environment
+- [ ] E2E tests pass in source environment
+- [ ] No open incidents related to these versions
+- [ ] Rollback plan reviewed (revert this commit)
+
+### Services affected
+| Service | Previous | New | Change type |
+|---------|----------|-----|-------------|
+| api-gateway | v2.3.0 | v2.3.1 | Patch (bug fix) |
+| user-service | v1.7.0 | v1.7.0 | No change |
+```