@sysnee/pgs 0.1.0

package/docs/EXECUTIVE_SUMMARY.md

@@ -0,0 +1,210 @@
+# Resumo Executivo: Análise para Produto Comercial
+
+## 🎯 Visão Geral
+
+**Objetivo**: Transformar a solução atual em um produto comercial de PostgreSQL gerenciado (PaaS)
+
+**Estado Atual**: ✅ Excelente para MVP/POC, ⚠️ Necessita melhorias críticas para produção comercial
+
+---
+
+## 🔴 GAPS CRÍTICOS (Bloqueadores)
+
+### 1. Docker Compose - Até onde é aceitável?
+
+**Limites Práticos:**
+- ✅ **ACEITÁVEL**: < 100 tenants, MVP, desenvolvimento
+- ❌ **INACEITÁVEL**: > 500 tenants, produção enterprise, multi-region
+
+**Problemas:**
+- Single host = single point of failure
+- Não escala além de ~50-100 containers/host
+- YAML único cresce indefinidamente
+- Sem orquestração automática
+
+**Recomendação:**
+- **Fase 1 (MVP)**: Docker Compose melhorado com sharding multi-host
+- **Fase 2-3**: Migrar para Kubernetes
+
+### 2. Alta Disponibilidade - CRÍTICO
+
+**Problema**: Zero redundância
+- 1 container PostgreSQL = 1 ponto de falha
+- HAProxy sem redundância
+- Sem failover automático
+
+**Solução**: 
+- Replicação PostgreSQL (Patroni/pg_auto_failover)
+- HAProxy redundante com Keepalived
+- Auto-failover configurado
+
+### 3. Backup e Recovery - ESSENCIAL
+
+**Problema**: Não existe sistema de backup
+
+**Solução**: 
+- Backup automático diário (pg_dump/pgBackRest)
+- Point-in-Time Recovery (PITR)
+- Retenção configurável (7/30/90 dias)
+- Storage offsite (S3/GCS)
+
+### 4. Monitoramento e Alertas
+
+**Problema**: Sem visibilidade de saúde do sistema
+
+**Solução**:
+- Prometheus + Grafana
+- Alertas críticos (container down, backup falhou)
+- SLA tracking
+- Métricas por tenant
+
+### 5. Segurança
+
+**Problemas**:
+- Senhas em texto plano
+- Sem SSL/TLS
+- Sem secret management
+
+**Solução**:
+- HashiCorp Vault para secrets
+- SSL/TLS obrigatório
+- Rotação automática de senhas
+- Network policies
+
+### 6. API e Automação
+
+**Problema**: Apenas CLI manual
+
+**Solução**:
+- REST API completa
+- Webhooks para eventos
+- SDK/CLI oficial
+- Integração com sistemas externos
+
+---
+
+## 📊 COMPARAÇÃO: Docker Compose vs Alternativas
+
+| Solução | Multi-Host | Auto-Scale | HA | Complexidade | Produção |
+|---------|-----------|------------|----|--------------| ---------|
+| **Docker Compose** | ❌ | ❌ | ❌ | ✅ Baixa | ❌ |
+| **Docker Swarm** | ✅ | ⚠️ | ✅ | ⚠️ Média | ⚠️ |
+| **Kubernetes** | ✅ | ✅ | ✅ | ❌ Alta | ✅ |
+| **Nomad** | ✅ | ✅ | ✅ | ⚠️ Média | ✅ |
+
+---
+
+## 🚀 ROADMAP RECOMENDADO
+
+### FASE 1: MVP Comercial (2-3 meses)
+```
+✅ Docker Compose melhorado (multi-host sharding)
+✅ Backup automático básico
+✅ API REST
+✅ Monitoring básico (Prometheus + Grafana)
+✅ Secrets management básico
+✅ SSL/TLS
+```
+
+**Custo estimado**: $5K-15K (infraestrutura + desenvolvimento)
+
+### FASE 2: Produção (3-4 meses)
+```
+✅ Alta disponibilidade (replicação)
+✅ Backup avançado (pgBackRest + PITR)
+✅ Dashboard web
+✅ Billing integrado
+✅ Alertas automáticos
+```
+
+**Custo estimado**: $10K-25K
+
+### FASE 3: Escala (6-12 meses)
+```
+✅ Migração para Kubernetes
+✅ Multi-region
+✅ Read replicas
+✅ Auto-scaling
+✅ Enterprise features
+```
+
+**Custo estimado**: $25K-50K
+
+---
+
+## 💰 MODELO DE NEGÓCIO
+
+### Planos Sugeridos
+
+**Starter** ($29/mês)
+- 1 CPU, 2GB RAM, 50GB storage
+- Backup diário (7 dias retenção)
+- 99.9% SLA
+
+**Professional** ($99/mês)
+- 2 CPU, 4GB RAM, 200GB storage
+- Backup horário (30 dias retenção)
+- 99.95% SLA
+- Read replica (opcional)
+
+**Enterprise** ($299/mês)
+- 4 CPU, 16GB RAM, 1TB storage
+- Backup contínuo (90 dias retenção)
+- 99.99% SLA
+- Multi-region
+- SLA garantido
+
+---
+
+## ⚠️ RISCOS PRINCIPAIS
+
+1. **Escalabilidade**: Docker Compose não escala além de ~100 tenants
+2. **Confiabilidade**: Sem HA, qualquer falha afeta múltiplos clientes
+3. **Compliance**: Sem backups/audit, difícil atender regulamentações
+4. **Operações**: Overhead manual aumenta com escala
+
+---
+
+## ✅ AÇÕES IMEDIATAS
+
+1. **Definir MVP Scope**
+   - Quantos tenants iniciais?
+   - SLA mínimo aceitável?
+   - Features essenciais?
+
+2. **Escolher Arquitetura Fase 1**
+   - Docker Compose melhorado (recomendado)
+   - Ou investir direto em Kubernetes?
+
+3. **Priorizar Features**
+   - Backup automático (ESSENCIAL)
+   - API REST (ESSENCIAL)
+   - Monitoring (ESSENCIAL)
+   - HA (Fase 2)
+
+4. **Estimativa de Custo**
+   - Infraestrutura cloud
+   - Desenvolvimento
+   - Operações
+
+---
+
+## 📚 DOCUMENTAÇÃO RELACIONADA
+
+- **[CRITICAL_REVIEW.md](./CRITICAL_REVIEW.md)** - Análise detalhada completa
+- **[ARCHITECTURE_RECOMMENDATIONS.md](./ARCHITECTURE_RECOMMENDATIONS.md)** - Recomendações técnicas práticas
+
+---
+
+## 🎯 CONCLUSÃO
+
+**Veredito**: Solução tem excelente base, mas precisa de investimento significativo em:
+
+1. ✅ **Alta Disponibilidade** (crítico)
+2. ✅ **Backup/Recovery** (crítico)
+3. ✅ **API REST** (essencial)
+4. ✅ **Monitoramento** (essencial)
+5. ✅ **Migração para orquestrador** (escalabilidade)
+
+**Recomendação Final**: Começar com Docker Compose melhorado para MVP rápido, mas planejar migração para Kubernetes quando atingir ~100 tenants ou necessidade de HA.
+

package/docs/PROJECT.md

@@ -0,0 +1,250 @@
+# PostgreSQL Multi-Tenant Instance Manager
+
+## Project Definition
+
+### What It Is
+
+A **dynamic PostgreSQL multi-tenant management system** that provides complete database isolation by creating dedicated PostgreSQL instances per tenant. The system uses HAProxy with custom PostgreSQL protocol parsing to route connections intelligently while maintaining complete tenant isolation at the database server level.
+
+### Core Concept
+
+Instead of sharing a single PostgreSQL instance with multiple databases (shared database architecture), this system creates **one PostgreSQL container per tenant**, ensuring:
+
+- **Complete Data Isolation**: Each tenant has its own PostgreSQL process and data directory
+- **Independent Scaling**: Resources can be allocated per tenant
+- **Enhanced Security**: No risk of cross-tenant data access
+- **Simplified Operations**: Each tenant can be managed independently
+
+### Key Features
+
+1. **Dynamic Tenant Provisioning**
+   - Create new PostgreSQL instances on-demand
+   - Automatic volume and network configuration
+   - Custom initialization scripts per tenant
+
+2. **Intelligent Routing**
+   - HAProxy parses PostgreSQL protocol to extract username
+   - Routes connections to correct tenant backend automatically
+   - Single external port (5432) for all tenants
+
+3. **Access Control**
+   - Per-tenant external access enable/disable
+   - Secure by default (access disabled on creation)
+   - Runtime access control without service restart
+
+4. **Complete Isolation**
+   - Separate Docker containers per tenant
+   - Isolated volumes for data persistence
+   - Network isolation via Docker bridge network
+   - No shared processes or memory
+
+5. **Zero-Downtime Operations**
+   - Graceful HAProxy reloads
+   - Independent tenant management
+   - No impact on other tenants during operations
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    External Access                       │
+│                  (localhost:5432)                        │
+└──────────────────────┬──────────────────────────────────┘
+                       │
+                       ▼
+┌─────────────────────────────────────────────────────────┐
+│                    HAProxy Proxy                         │
+│  ┌──────────────────────────────────────────────────┐   │
+│  │  Frontend: postgres_frontend                     │   │
+│  │  - Listens on port 5432                          │   │
+│  │  - Parses PostgreSQL protocol (Lua script)       │   │
+│  │  - Extracts username from startup packet         │   │
+│  │  - Checks tenant-access.json for permissions     │   │
+│  └──────────────────────────────────────────────────┘   │
+└──────────────────────┬──────────────────────────────────┘
+                       │
+        ┌──────────────┼──────────────┐
+        │              │              │
+        ▼              ▼              ▼
+┌──────────────┐ ┌──────────────┐ ┌──────────────┐
+│  Backend     │ │  Backend     │ │  Backend     │
+│ pgs_tenant1  │ │ pgs_tenant2  │ │ pgs_tenant3  │
+└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
+       │                │                │
+       ▼                ▼                ▼
+┌──────────────┐ ┌──────────────┐ ┌──────────────┐
+│ PostgreSQL   │ │ PostgreSQL   │ │ PostgreSQL   │
+│ Container 1  │ │ Container 2  │ │ Container 3  │
+│              │ │              │ │              │
+│ Port: 5432   │ │ Port: 5432   │ │ Port: 5432   │
+│ (internal)   │ │ (internal)   │ │ (internal)   │
+│              │ │              │ │              │
+│ Volume:      │ │ Volume:      │ │ Volume:      │
+│ pgdata_1     │ │ pgdata_2     │ │ pgdata_3     │
+└──────────────┘ └──────────────┘ └──────────────┘
+```
+
+## Technical Implementation
+
+### Components
+
+1. **Manager Script (manager.js)**
+   - Node.js CLI tool for tenant lifecycle management
+   - Dynamically generates docker-compose.yml entries
+   - Manages HAProxy configuration
+   - Controls tenant access permissions
+
+2. **HAProxy Reverse Proxy**
+   - TCP-level load balancer and router
+   - Custom Lua script for PostgreSQL protocol parsing
+   - Routes based on extracted username
+   - Per-tenant access control
+
+3. **PostgreSQL Protocol Parser (pg-route.lua)**
+   - Parses binary PostgreSQL startup packet
+   - Extracts username and connection parameters
+   - Handles SSL negotiation
+   - Enforces access control policies
+
+4. **Docker Infrastructure**
+   - Separate container per tenant
+   - Bridge network for internal communication
+   - Persistent volumes for data
+   - Isolated execution environments
+
+### Connection Flow
+
+1. Client connects to `localhost:5432` with username `tenant_id`
+2. HAProxy receives connection and invokes Lua script
+3. Script parses PostgreSQL startup packet and extracts username
+4. Script checks `tenant-access.json` for permission
+5. If allowed, routes to backend `pgs_{tenant_id}`
+6. Backend forwards to PostgreSQL container on internal network
+7. Connection established with complete isolation
+
+## Comparison with Similar Solutions
+
+### Shared Database Architecture
+
+**Traditional Multi-Tenant PostgreSQL:**
+- Single PostgreSQL instance
+- Multiple databases/schemas per instance
+- Shared processes and memory
+- Risk of cross-tenant data access
+- Less isolation
+
+**This Solution:**
+- Multiple PostgreSQL instances
+- One instance per tenant
+- Complete process isolation
+- Zero risk of cross-tenant access
+- Maximum isolation
+
+### Similar Open Source Solutions
+
+#### 1. **PgBouncer**
+- **Purpose**: Connection pooling, not tenant isolation
+- **Difference**: Pools connections to single instance; this creates separate instances
+- **Use Case**: Different - PgBouncer optimizes connections; this isolates tenants
+
+#### 2. **Citus**
+- **Purpose**: PostgreSQL extension for distributed PostgreSQL
+- **Difference**: Shards data across nodes; this creates separate instances per tenant
+- **Use Case**: Horizontal scaling vs. tenant isolation
+
+#### 3. **Patroni + HAProxy**
+- **Purpose**: High availability and load balancing
+- **Difference**: Replicates single database; this creates isolated instances
+- **Use Case**: HA for single database vs. multi-tenant isolation
+
+#### 4. **Schema-based Multi-tenancy**
+- **Purpose**: Share database, separate schemas
+- **Difference**: Shared instance; this uses separate instances
+- **Use Case**: Resource efficiency vs. complete isolation
+
+#### 5. **Row-level Security (RLS)**
+- **Purpose**: Application-level tenant isolation
+- **Difference**: Logic-based separation; this uses infrastructure isolation
+- **Use Case**: Application isolation vs. infrastructure isolation
+
+### Unique Aspects of This Solution
+
+1. **Instance-per-tenant at infrastructure level**
+   - Not just database or schema separation
+   - Complete process and memory isolation
+
+2. **Dynamic provisioning with single external port**
+   - No need for port management
+   - Automatic routing based on connection parameters
+
+3. **Protocol-aware routing**
+   - Parses PostgreSQL binary protocol
+   - Routes before connection completion
+   - Handles SSL negotiation
+
+4. **Runtime access control**
+   - Enable/disable tenant access without restart
+   - No downtime for access changes
+
+5. **Docker-native architecture**
+   - Leverages container isolation
+   - Simple deployment and scaling
+   - Resource limits per tenant
+
+## Use Cases
+
+### Ideal For
+
+- **SaaS Applications** requiring strict tenant data isolation
+- **Healthcare/Finance** applications with compliance requirements
+- **Multi-tenant platforms** needing independent scaling
+- **Development/Testing** environments with isolated databases
+- **Legacy application migration** requiring tenant separation
+
+### Not Ideal For
+
+- Thousands of tenants (resource overhead)
+- Shared resource requirements
+- Simple multi-tenant applications without strict isolation needs
+- Environments requiring minimal resource usage
+
+## Advantages
+
+✅ **Maximum Isolation**: Complete process and data separation
+✅ **Security**: Zero risk of cross-tenant data access
+✅ **Flexibility**: Independent scaling and management per tenant
+✅ **Simplicity**: Single external port, automatic routing
+✅ **Compliance**: Easier to meet regulatory requirements
+✅ **Debugging**: Isolated environments simplify troubleshooting
+
+## Trade-offs
+
+⚠️ **Resource Usage**: Higher memory/CPU per tenant
+⚠️ **Management Overhead**: More containers to manage
+⚠️ **Scaling Limits**: Practical limit on number of tenants per host
+⚠️ **Backup Complexity**: Need to backup multiple instances
+
+## Technology Stack
+
+- **Runtime**: Node.js (ES Modules)
+- **Container Orchestration**: Docker Compose
+- **Reverse Proxy**: HAProxy with Lua scripting
+- **Database**: PostgreSQL 18+
+- **Protocol Parsing**: Custom Lua script
+- **Configuration**: YAML (docker-compose.yml), JSON (tenant-access.json)
+
+## Future Enhancements
+
+- [ ] Health checks and automatic failover
+- [ ] Backup/restore automation per tenant
+- [ ] Resource limits (CPU/memory) per tenant
+- [ ] Monitoring and metrics collection
+- [ ] Tenant migration tools
+- [ ] Kubernetes support
+- [ ] Connection pooling per tenant
+- [ ] SSL/TLS termination
+
+## License & Status
+
+This is a custom solution built for specific multi-tenant requirements. It combines open-source tools (HAProxy, PostgreSQL, Docker) with custom routing logic to achieve instance-per-tenant isolation with intelligent connection routing.
+

package/haproxy-lua/pg-route.lua

@@ -0,0 +1,177 @@
+-- PostgreSQL Protocol Parser for HAProxy
+-- Routes connections based on username extracted from startup packet
+-- Checks tenant-access.json for access control
+
+-- Simple JSON parser for our limited use case (flat object with string keys and boolean values)
+local function parse_json(str)
+    local result = {}
+    -- Match patterns like "key": true or "key": false
+    for key, value in string.gmatch(str, '"([^"]+)":%s*(%w+)') do
+        if value == "true" then
+            result[key] = true
+        elseif value == "false" then
+            result[key] = false
+        end
+    end
+    return result
+end
+
+-- Cache for tenant access configuration
+local tenant_access_cache = {}
+local cache_timestamp = 0
+local CACHE_TTL = 5 -- seconds
+
+-- Load tenant access configuration
+local function load_tenant_access()
+    local now = core.now().sec
+    if now - cache_timestamp < CACHE_TTL then
+        return tenant_access_cache
+    end
+    
+    local file = io.open("/etc/haproxy/tenant-access.json", "r")
+    if not file then
+        core.Warning("tenant-access.json not found, denying all connections")
+        tenant_access_cache = {}
+        cache_timestamp = now
+        return tenant_access_cache
+    end
+    
+    local content = file:read("*all")
+    file:close()
+    
+    local data = parse_json(content)
+    tenant_access_cache = data or {}
+    cache_timestamp = now
+    
+    return tenant_access_cache
+end
+
+-- Parse PostgreSQL startup packet to extract username
+-- PostgreSQL startup packet format:
+-- [4 bytes: length] [4 bytes: protocol version] [key=value pairs\0]
+local function parse_startup_packet(data)
+    if #data < 8 then
+        return nil
+    end
+    
+    -- Read packet length (big-endian)
+    local len = (string.byte(data, 1) * 16777216) +
+                (string.byte(data, 2) * 65536) +
+                (string.byte(data, 3) * 256) +
+                string.byte(data, 4)
+    
+    if #data < len then
+        return nil
+    end
+    
+    -- Read protocol version
+    local major = (string.byte(data, 5) * 256) + string.byte(data, 6)
+    local minor = (string.byte(data, 7) * 256) + string.byte(data, 8)
+    
+    -- Check for SSL request (protocol 1234.5679)
+    if major == 1234 and minor == 5679 then
+        return { ssl_request = true }
+    end
+    
+    -- Check for cancel request (protocol 1234.5678)
+    if major == 1234 and minor == 5678 then
+        return { cancel_request = true }
+    end
+    
+    -- Normal startup message (protocol 3.0)
+    if major ~= 3 then
+        return nil
+    end
+    
+    -- Parse key-value pairs starting at byte 9
+    local params = {}
+    local pos = 9
+    
+    while pos < len do
+        -- Read key
+        local key_start = pos
+        while pos <= len and string.byte(data, pos) ~= 0 do
+            pos = pos + 1
+        end
+        
+        if pos > len then break end
+        
+        local key = string.sub(data, key_start, pos - 1)
+        pos = pos + 1 -- skip null
+        
+        if key == "" then break end -- end of parameters
+        
+        -- Read value
+        local val_start = pos
+        while pos <= len and string.byte(data, pos) ~= 0 do
+            pos = pos + 1
+        end
+        
+        local value = string.sub(data, val_start, pos - 1)
+        pos = pos + 1 -- skip null
+        
+        params[key] = value
+    end
+    
+    return params
+end
+
+-- Main routing function called by HAProxy
+function pg_route(txn)
+    -- Get data from the request buffer
+    local data = txn.req:data(0)
+    
+    if not data or #data == 0 then
+        -- No data yet, need to wait
+        return
+    end
+    
+    local params = parse_startup_packet(data)
+    
+    if not params then
+        core.Warning("Failed to parse PostgreSQL startup packet")
+        txn:set_var("txn.pg_blocked", true)
+        return
+    end
+    
+    -- Handle SSL request - route to default pool for SSL negotiation
+    -- PostgreSQL will respond 'N' (no SSL), then client retries without SSL
+    if params.ssl_request then
+        core.Info("SSL request received, routing to SSL negotiation pool")
+        -- Route to SSL pool backend which forwards to any available PostgreSQL
+        -- This allows SSL negotiation to complete
+        txn:set_var("txn.pg_backend", "pg_ssl_pool")
+        return
+    end
+    
+    -- Handle cancel request
+    if params.cancel_request then
+        txn:set_var("txn.pg_blocked", true)
+        return
+    end
+    
+    local username = params["user"]
+    if not username then
+        core.Warning("No username in PostgreSQL startup packet")
+        txn:set_var("txn.pg_blocked", true)
+        return
+    end
+    
+    -- Load tenant access configuration
+    local access = load_tenant_access()
+    
+    -- Check if tenant has external access enabled
+    if not access[username] then
+        core.Warning("Access denied: " .. username)
+        txn:set_var("txn.pg_blocked", true)
+        return
+    end
+    
+    -- Route to tenant's backend
+    local backend = "pgs_" .. username
+    core.Info("Routing user " .. username .. " to backend " .. backend)
+    txn:set_var("txn.pg_backend", backend)
+end
+
+-- Register the action with HAProxy
+core.register_action("pg_route", { "tcp-req" }, pg_route, 0)