sea-dev 1.0.0

package/docs/aws-migration-plan.md

@@ -0,0 +1,267 @@
+# Render → Kubernetes Migration Plan for sea.dev
+
+## Design Philosophy: Two-Layer Architecture
+
+The core idea is to **cleanly separate what's cloud-specific from what's portable**.
+
+```
+┌──────────────────────────────────────────────────────┐
+│  Layer 2: Application (PORTABLE)                     │
+│  Helm charts, K8s manifests, GitOps (ArgoCD)         │
+│  → Same across AWS, GCP, Azure, on-prem              │
+├──────────────────────────────────────────────────────┤
+│  Layer 1: Infrastructure (CLOUD-SPECIFIC)            │
+│  Terraform per provider: VPC, managed K8s,           │
+│  managed DB, load balancer, DNS                      │
+│  → Rewritten per cloud, but small surface area       │
+└──────────────────────────────────────────────────────┘
+```
+
+**Layer 1 (Terraform)** provisions the cluster and cloud-managed services. This is
+the only part that changes between providers. It's intentionally thin — just
+networking, the K8s cluster, managed databases, and container registry.
+
+**Layer 2 (Helm/K8s)** defines how your services run. Deployments, services, ingress
+rules, scaling policies, jobs — all in portable K8s manifests. This travels with
+you to any provider.
+
+The migration from AWS to GCP or on-prem becomes: rewrite ~200 lines of Terraform
+(Layer 1), keep all your Helm charts (Layer 2) unchanged.
+
+---
+
+## 1. Current Render Setup & Estimated Costs
+
+| Service | Render Type | Plan | Est. Cost/mo |
+|---|---|---|---|
+| `seadotdev-app` | Web (Node SSR) | Pro Max (16GB/4vCPU) | ~$225 |
+| `seadotdev-api-v2` | Web (Node/Fastify) | Standard (2GB/1vCPU) | ~$25 |
+| `data-extraction-worker` | Worker (Node) | Pro (4GB/2vCPU) | ~$85 |
+| `data-extraction-worker-agentic` | Worker (Node) | Pro (4GB/2vCPU) | ~$85 |
+| `conversion-worker` | Worker (Docker) | Starter (512MB/0.5vCPU) | ~$7 |
+| `ph-proxy` | Static | Free | $0 |
+| `seadotdev-widget` | Static | Free | $0 |
+| `angular-test-app` | Static | Free | $0 |
+| **prod** DB | PostgreSQL 16 | Pro 8GB | ~$150 |
+| **staging** DB | PostgreSQL 16 | Pro 8GB | ~$150 |
+| Render Team Plan | — | — | ~$19–95 |
+
+**Estimated Render total: ~$750–830/mo**
+
+---
+
+## 2. Target Architecture
+
+```
+                         ┌──────────────────┐
+                         │   CloudFront /   │
+                         │   Cloud CDN      │
+                         └────────┬─────────┘
+                                  │
+              ┌───────────────────┼───────────────────┐
+              │           Kubernetes Cluster           │
+              │                                        │
+              │  ┌─────────────┐  ┌─────────────────┐ │
+              │  │ ingress-    │  │ cert-manager     │ │
+              │  │ nginx       │  │ (TLS)            │ │
+              │  └──────┬──────┘  └─────────────────┘ │
+              │         │                              │
+              │  ┌──────▼──────┐  ┌────────────────┐  │
+              │  │  main-app   │  │  api-v2        │  │
+              │  │  Deployment │  │  Deployment    │  │
+              │  │  (2 pods)   │  │  (1-4 pods)   │  │
+              │  └─────────────┘  └────────────────┘  │
+              │                                        │
+              │  ┌──────────────────────────────────┐  │
+              │  │  Workers (Deployments)            │  │
+              │  │  extraction  │ extraction-agentic │  │
+              │  │  conversion  │ (KEDA autoscaled)  │  │
+              │  └──────────────────────────────────┘  │
+              │                                        │
+              │  ┌──────────────────────────────────┐  │
+              │  │  Shared Services                  │  │
+              │  │  external-secrets │ KEDA          │  │
+              │  │  ArgoCD          │ metrics-server │  │
+              │  └──────────────────────────────────┘  │
+              └────────────────────────────────────────┘
+                                  │
+              ┌───────────────────┼───────────────────┐
+              │           Managed Services             │
+              │  (cloud-specific, Terraform-managed)   │
+              │                                        │
+              │  ┌─────────────┐  ┌────────────────┐  │
+              │  │ PostgreSQL  │  │ Container      │  │
+              │  │ (RDS/Cloud  │  │ Registry       │  │
+              │  │  SQL/bare)  │  │ (ECR/GCR/      │  │
+              │  │             │  │  Harbor)        │  │
+              │  └─────────────┘  └────────────────┘  │
+              └────────────────────────────────────────┘
+```
+
+### Key portable components
+
+**ingress-nginx** — Load balancer & routing. Works identically everywhere. Replaces
+cloud-specific ALBs.
+
+**cert-manager** — Automatic TLS certificates via Let's Encrypt. No cloud-specific
+certificate management needed.
+
+**external-secrets** — Syncs secrets from whatever backend you use (AWS Secrets
+Manager today, Vault on-prem tomorrow). Your pods just reference a K8s Secret —
+they don't know where it came from.
+
+**KEDA** — Event-driven autoscaling. Scale workers based on Hatchet queue depth,
+not just CPU/memory. Can scale to zero. Works the same on every cluster.
+
+**ArgoCD** — GitOps deployment. Push Helm chart changes to git, ArgoCD syncs them
+to the cluster. Same workflow regardless of where the cluster runs.
+
+---
+
+## 3. Cost Comparison
+
+### AWS (EKS) Estimated Monthly Cost
+
+| Component | Sizing | Est. Cost/mo |
+|---|---|---|
+| **EKS control plane** | 1 cluster | $75 |
+| **Node group** | 2x t4g.large (2vCPU/8GB each) spot+on-demand mix | ~$80–120 |
+| **RDS PostgreSQL — prod** | db.t4g.medium, Multi-AZ, 50GB | ~$140 |
+| **RDS PostgreSQL — staging** | db.t4g.small, Single-AZ, 20GB | ~$35 |
+| **NLB** (for ingress-nginx) | 1 Network Load Balancer | ~$20 |
+| **CloudFront** | Moderate traffic | ~$10–20 |
+| **S3 + ECR** | Storage | ~$6 |
+| **NAT Gateway** | 1 gateway | ~$35 |
+| **Data Transfer** | Moderate | ~$10–20 |
+
+**Estimated AWS+K8s total: ~$410–570/mo**
+
+Using EC2 nodes instead of Fargate pods means you can bin-pack services onto
+nodes more efficiently. Two `t4g.large` instances give you 4 vCPU and 16GB total,
+which comfortably fits all five services with headroom for scaling.
+
+For further savings: Spot Instances for worker nodes (~60% cheaper), Reserved
+Instances for the always-on node (~35% cheaper).
+
+### Cross-Cloud Portability Cost (what changes when you move)
+
+| Component | AWS | GCP | On-Prem |
+|---|---|---|---|
+| K8s cluster | EKS ($75/mo) | GKE ($0 standard / $75 private) | k3s ($0) |
+| Nodes | EC2 ($80-120) | GCE ($70-100) | Physical servers |
+| Database | RDS ($175) | Cloud SQL ($150) | Self-managed PG |
+| Load Balancer | NLB ($20) | Cloud LB ($20) | MetalLB ($0) |
+| Registry | ECR ($5) | GCR/Artifact ($5) | Harbor ($0) |
+| **Infra-only total** | **~$355–395** | **~$245–350** | **Hardware cost** |
+| **App layer** | **Same Helm charts** | **Same Helm charts** | **Same Helm charts** |
+
+The Terraform rewrite between clouds is ~200-300 lines. The Helm charts (the
+bulk of your deployment config) don't change at all.
+
+---
+
+## 4. Migration Phases
+
+### Phase 1: Foundation (Week 1)
+
+- Terraform: VPC, EKS cluster, node groups, ECR
+- Install cluster essentials: ingress-nginx, cert-manager, external-secrets, KEDA, ArgoCD
+- Set up AWS Secrets Manager with all env vars from Render's `app-secrets`
+
+### Phase 2: Helm Charts & CI (Week 1-2)
+
+- Create Helm chart for sea.dev services (see scaffolding below)
+- Create Dockerfiles for services that lack them (main-app, api-v2, workers)
+- Set up GitHub Actions → ECR → ArgoCD deploy pipeline
+
+### Phase 3: Database Migration (Week 2)
+
+- Provision RDS PostgreSQL 16
+- `pg_dump` / `pg_restore` with a short maintenance window
+- Verify data integrity
+- Update `external-secrets` to point connection strings at new RDS
+
+### Phase 4: Deploy & Validate (Week 2-3)
+
+- Deploy all services to K8s via ArgoCD
+- Run side-by-side with Render (both pointing at new RDS)
+- Validate Hatchet worker connectivity, extraction pipelines, SSR
+- Load test the ingress path
+
+### Phase 5: DNS Cutover (Week 3)
+
+- Lower DNS TTL to 60s a week before cutover
+- Switch DNS to new ingress
+- Monitor for 48 hours
+- Decommission Render services
+
+---
+
+## 5. On-Prem Readiness
+
+When the time comes for on-prem or private datacenter deployment, the path is:
+
+1. **k3s or RKE2** — lightweight Kubernetes distributions designed for edge/on-prem.
+   Single binary install, runs on any Linux box.
+2. **Harbor** — self-hosted container registry (replaces ECR)
+3. **MetalLB** — bare-metal load balancer (replaces cloud NLBs)
+4. **Vault** — replaces cloud secrets managers. external-secrets already supports it.
+5. **Self-managed PostgreSQL** or **CrunchyData PGO** (Postgres Operator for K8s)
+
+Your Helm charts deploy unchanged. The only work is writing a new Terraform module
+(or Ansible playbook) for the on-prem infrastructure layer.
+
+---
+
+## 6. Key Decisions
+
+### Node Strategy: Managed Node Groups vs. Karpenter
+
+**Managed Node Groups** (recommended to start): AWS manages the EC2 instances,
+you define instance types and sizes. Simple, predictable.
+
+**Karpenter** (for later): Dynamically provisions the right-sized nodes based on
+pod requirements. Better bin-packing, lower cost at scale. Worth adopting once
+you have 5+ nodes and variable workloads.
+
+### GitOps: ArgoCD vs. Flux
+
+Both work well. **ArgoCD** has a better UI for debugging deployments and is more
+widely adopted. **Flux** is lighter weight and more "git-native." Either is fine
+for portability. This plan assumes ArgoCD.
+
+### Secrets: external-secrets vs. sealed-secrets
+
+**external-secrets** (recommended): Syncs from cloud-native secret stores (AWS SM,
+GCP SM, Vault). When you switch providers, you update the SecretStore config — pods
+don't change.
+
+**sealed-secrets**: Encrypts secrets in git. Simpler, but less flexible for
+multi-cloud. Better for on-prem-only scenarios.
+
+---
+
+## 7. What You Gain vs. Render
+
+- **True portability**: Move between clouds by swapping ~200 lines of Terraform
+- **On-prem ready**: k3s + same Helm charts works in a datacenter
+- **Granular scaling**: KEDA scales workers on queue depth, HPA scales web services on CPU/RPS
+- **Cost control**: Bin-pack services on nodes, use spot instances, scale to zero
+- **GitOps**: Every deployment is a git commit, auditable, rollbackable
+- **Ecosystem**: Hundreds of Helm charts for monitoring, logging, tracing, etc.
+
+## 8. What You Lose vs. Render
+
+- **Simplicity**: K8s has a real learning curve; expect 2-3 weeks of setup
+- **Managed overhead**: Cluster upgrades, node patching, ingress config
+- **Preview environments**: Need to build this (ArgoCD ApplicationSets can help)
+- **Cost at small scale**: The $75/mo EKS tax hurts when you have 5 services
+
+## 9. Risk Mitigation
+
+1. **Run parallel** during migration — Render stays live until K8s is validated
+2. **Database migration** is the riskiest step — do a dry run first
+3. **Start with 2 nodes** — add more only when needed
+4. **Use node taints** to isolate workers from web services if resource contention appears
+5. **Set resource requests AND limits** on all pods to prevent noisy neighbors
+6. **PodDisruptionBudgets** on web services to prevent downtime during node drains