@sysnee/pgs 0.1.7-rc.11 → 0.1.7-rc.13

package/docs/DEV_SETUP.md

@@ -0,0 +1,282 @@
+# Local Dev Environment — PGS
+
+Simulates the production setup (Node.js + `pgs` CLI + Traefik + PostgreSQL containers) locally, without installing the package globally or needing real DNS/SSL certificates.
+
+## Overview
+
+| Concern | Production | Local Dev |
+|---|---|---|
+| Domain | `pgs.br-sp-1.sysnee.com` | `pgs.local` |
+| DNS | Real DNS wildcard A record | dnsmasq (`*.pgs.local → 127.0.0.1`) |
+| TLS cert | Let's Encrypt wildcard | Self-signed wildcard CA |
+| CLI | `pgs` (global npm install) | `node ./manager.js` (source directly) |
+| Setup | `pgs setup` (certbot, installs Docker) | `dev-setup.sh` (manual bootstrap) |
+
+---
+
+## Prerequisites
+
+- Docker & Docker Compose installed
+- Node.js 24
+- `dnsmasq` (install below)
+- `openssl` (usually pre-installed)
+
+---
+
+## Step 1 — Make `DOMAIN` configurable in `manager.js`
+
+Change line 20 from:
+
+```js
+const DOMAIN = 'pgs.br-sp-1.sysnee.com';
+```
+
+To:
+
+```js
+const DOMAIN = process.env.PGS_DOMAIN || 'pgs.br-sp-1.sysnee.com';
+```
+
+This is the only code change needed. Production behavior is unaffected (env var not set → falls back to real domain).
+
+---
+
+## Step 2 — Install and configure dnsmasq
+
+dnsmasq resolves all `*.pgs.local` subdomains to `127.0.0.1` automatically, without needing to edit `/etc/hosts` per tenant.
+
+```bash
+sudo apt-get install -y dnsmasq
+```
+
+Add the wildcard rule:
+
+```bash
+echo "address=/.pgs.local/127.0.0.1" | sudo tee /etc/dnsmasq.d/pgs-local.conf
+sudo systemctl restart dnsmasq
+```
+
+On systems using `systemd-resolved` (Ubuntu 20.04+), you need to tell it to use dnsmasq for `.local` queries. Check if it's active:
+
+```bash
+systemctl is-active systemd-resolved
+```
+
+If active, add to `/etc/systemd/resolved.conf`:
+
+```ini
+[Resolve]
+DNS=127.0.0.1
+Domains=~local
+```
+
+Then restart:
+
+```bash
+sudo systemctl restart systemd-resolved
+```
+
+Verify resolution works:
+
+```bash
+dig tenant1.pgs.local @127.0.0.1
+# or
+nslookup tenant1.pgs.local 127.0.0.1
+```
+
+---
+
+## Step 3 — Bootstrap `~/.sysnee-config/` (replaces `pgs setup`)
+
+Create the file `pgs/dev-setup.sh`:
+
+```bash
+#!/usr/bin/env bash
+set -e
+
+CONFIG_DIR="$HOME/.sysnee-config"
+CERTS_DIR="$CONFIG_DIR/certs"
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+
+echo "── Bootstrapping ~/.sysnee-config/ ──"
+
+mkdir -p "$CERTS_DIR"
+
+# Copy template files from source
+cp "$SCRIPT_DIR/docker-compose.yml" "$CONFIG_DIR/docker-compose.yml"
+cp "$SCRIPT_DIR/traefik.yml"        "$CONFIG_DIR/traefik.yml"
+
+[ -f "$CONFIG_DIR/tenant-access.json" ] || echo '{}' > "$CONFIG_DIR/tenant-access.json"
+
+# Write setup status so checkSetupStatus() passes
+echo -n "container:ok" > "$CONFIG_DIR/status.txt"
+
+echo "── Generating self-signed wildcard cert for *.pgs.local ──"
+
+# CA
+openssl genrsa -out "$CERTS_DIR/ca.key" 4096 2>/dev/null
+openssl req -x509 -new -nodes \
+  -key "$CERTS_DIR/ca.key" \
+  -sha256 -days 825 \
+  -out "$CERTS_DIR/ca.pem" \
+  -subj "/CN=PGS Dev CA/O=Local Dev"
+
+# Wildcard cert
+openssl genrsa -out "$CERTS_DIR/privkey.pem" 4096 2>/dev/null
+openssl req -new \
+  -key "$CERTS_DIR/privkey.pem" \
+  -out /tmp/pgs-dev.csr \
+  -subj "/CN=*.pgs.local"
+
+# SAN required by modern TLS clients (Chrome, DBeaver, psql)
+cat > /tmp/pgs-dev-ext.cnf <<EOF
+subjectAltName=DNS:*.pgs.local,DNS:pgs.local
+basicConstraints=CA:FALSE
+keyUsage=digitalSignature,keyEncipherment
+extendedKeyUsage=serverAuth
+EOF
+
+openssl x509 -req \
+  -in /tmp/pgs-dev.csr \
+  -CA "$CERTS_DIR/ca.pem" \
+  -CAkey "$CERTS_DIR/ca.key" \
+  -CAcreateserial \
+  -out "$CERTS_DIR/fullchain.pem" \
+  -days 825 \
+  -sha256 \
+  -extfile /tmp/pgs-dev-ext.cnf
+
+echo "── Generating initial dynamic.yml ──"
+cat > "$CONFIG_DIR/dynamic.yml" <<'EOF'
+tcp:
+  routers: {}
+  services: {}
+tls:
+  certificates:
+    - certFile: /etc/traefik/certs/fullchain.pem
+      keyFile: /etc/traefik/certs/privkey.pem
+EOF
+
+echo ""
+echo "✓ Dev environment ready at $CONFIG_DIR"
+echo ""
+echo "Cert files:"
+echo "  CA:   $CERTS_DIR/ca.pem       ← import this into DBeaver / OS trust store"
+echo "  Cert: $CERTS_DIR/fullchain.pem"
+echo "  Key:  $CERTS_DIR/privkey.pem"
+echo ""
+echo "Next: PGS_DOMAIN=pgs.local node ./manager.js create <tenant-id>"
+```
+
+Make it executable:
+
+```bash
+chmod +x pgs/dev-setup.sh
+```
+
+Run once:
+
+```bash
+cd pgs && ./dev-setup.sh
+```
+
+---
+
+## Step 4 — Workflow: creating and starting a tenant
+
+All commands use `node ./manager.js` directly from source. Changes are reflected instantly.
+
+```bash
+# From the pgs/ directory
+export PGS_DOMAIN=pgs.local
+
+node ./manager.js create tenant1
+node ./manager.js start tenant1
+
+# Verify
+node ./manager.js list
+```
+
+No reinstall required between code changes.
+
+---
+
+## Step 5 — Connect from DBeaver
+
+### Trust the dev CA (once)
+
+DBeaver needs to trust the self-signed CA to accept the TLS connection.
+
+In DBeaver: **Window → Preferences → Connections → SSL** → add `~/.sysnee-config/certs/ca.pem` as a trusted CA.
+
+Alternatively, add it to the system trust store (so any tool trusts it):
+
+```bash
+sudo cp ~/.sysnee-config/certs/ca.pem /usr/local/share/ca-certificates/pgs-dev-ca.crt
+sudo update-ca-certificates
+```
+
+### Connection settings
+
+| Field | Value |
+|---|---|
+| Host | `tenant1.pgs.local` |
+| Port | `5432` |
+| Database | `tenant1` |
+| Username | `postgres` |
+| Password | (set at create time) |
+| SSL mode | `require` |
+| SSL CA cert | `~/.sysnee-config/certs/ca.pem` |
+
+### psql equivalent
+
+```bash
+psql "postgresql://postgres:PASSWORD@tenant1.pgs.local:5432/tenant1?sslmode=require&sslrootcert=$HOME/.sysnee-config/certs/ca.pem"
+```
+
+---
+
+## Teardown
+
+```bash
+export PGS_DOMAIN=pgs.local
+
+node ./manager.js stop
+node ./manager.js remove tenant1
+
+# Full reset (removes all config and containers)
+docker compose -f ~/.sysnee-config/docker-compose.yml down -v
+rm -rf ~/.sysnee-config
+```
+
+---
+
+## Troubleshooting
+
+**DNS not resolving**
+
+```bash
+# Check dnsmasq is running
+sudo systemctl status dnsmasq
+
+# Test directly
+dig +short tenant1.pgs.local @127.0.0.1
+```
+
+**TLS handshake error in DBeaver / psql**
+
+Make sure the CA cert is trusted and `sslrootcert` points to `ca.pem`, not `fullchain.pem`.
+
+**`checkSetupStatus()` fails (exits with "Please run pgs setup")**
+
+```bash
+echo -n "container:ok" > ~/.sysnee-config/status.txt
+```
+
+**Port 5432 already in use**
+
+A local PostgreSQL instance may be listening. Stop it temporarily:
+
+```bash
+sudo systemctl stop postgresql
+```

package/manager.js

@@ -8,6 +8,7 @@ import { Command } from 'commander';
 import { execSync } from 'child_process';
 import { fileURLToPath } from 'url';
 import { dirname } from 'path';
+import { createInterface } from 'readline';
 
 import packageJson from './package.json' with { type: 'json' };
 
@@ -16,6 +17,7 @@ const __filename = fileURLToPath(import.meta.url);
 const __dirname = dirname(__filename);
 
 const CONFIG_DIR = path.join(os.homedir(), '.sysnee-config');
+const DOMAIN = 'pgs.br-sp-1.sysnee.com';
 
 const COMPOSE_FILE_PATH = path.join(CONFIG_DIR, 'docker-compose.yml');
 const INIT_DIR_PATH = path.join(CONFIG_DIR, 'init');
@@ -27,7 +29,6 @@ const SETUP_STATUS_PATH = path.join(CONFIG_DIR, 'status.txt');
 
 function loadCompose() {
     const content = fs.readFileSync(COMPOSE_FILE_PATH, 'utf8');
-    console.debug(`docker-compose file readed from ${COMPOSE_FILE_PATH}`)
     return yaml.load(content);
 }
 
@@ -114,7 +115,10 @@ function generateTraefikDynamicConfig(tenantsManifests = {}) {
 
             if (sourceRanges.length > 0) {
                 router.middlewares = [middlewareName];
-                
+
+                if (!dynamicConfig.tcp.middlewares) {
+                    dynamicConfig.tcp.middlewares = {};
+                }
                 if (!dynamicConfig.tcp.middlewares[middlewareName]) {
                     dynamicConfig.tcp.middlewares[middlewareName] = {
                         ipAllowList: {
@@ -230,15 +234,15 @@ function ensureNetwork(doc) {
     }
 }
 
-function initialSetup() {
+async function initialSetup() {
     installDocker()
     createInitialFiles()
     ensureDockerPrivilegies()
+    await setupCertificates()
     console.log('All ready!')
 }
 
 function installDocker() {
-
     try {
         execSync('docker info', { stdio: 'pipe' })
         console.log('Docker already installed')
@@ -249,7 +253,7 @@ function installDocker() {
     execSync('curl -fsSL https://get.docker.com -o get-docker.sh && sudo sh get-docker.sh', { stdio: 'inherit' })
     execSync('rm get-docker.sh')
     console.log('Docker installed successfully')
-    
+
     console.log('Installing Docker Compose plugin...')
     execSync('sudo apt-get install -y docker-compose-plugin', { stdio: 'inherit' })
     console.log('Docker Compose plugin installed successfully')
@@ -260,6 +264,107 @@ function ensureDockerPrivilegies() {
     writeFileSync(SETUP_STATUS_PATH, 'container:ok')
 }
 
+function waitForEnter(message) {
+    const rl = createInterface({ input: process.stdin, output: process.stdout });
+    return new Promise(resolve => {
+        rl.question(message, () => {
+            rl.close();
+            resolve();
+        });
+    });
+}
+
+function installCertbot() {
+    try {
+        execSync('certbot --version', { stdio: 'pipe' });
+        console.log('Certbot already installed');
+        return;
+    } catch (_) {}
+
+    console.log('Installing certbot...');
+    execSync('sudo apt-get install -y certbot', { stdio: 'inherit' });
+    console.log('Certbot installed successfully');
+}
+
+function findCertbotLivePath() {
+    const basePath = '/etc/letsencrypt/live';
+    const candidates = [
+        path.join(basePath, DOMAIN),
+        path.join(basePath, `*.${DOMAIN}`),
+    ];
+
+    try {
+        const dirs = fs.readdirSync(basePath);
+        for (const dir of dirs) {
+            const p = path.join(basePath, dir);
+            if (!candidates.includes(p)) candidates.push(p);
+        }
+    } catch (_) {}
+
+    for (const candidate of candidates) {
+        const fullchain = path.join(candidate, 'fullchain.pem');
+        const privkey = path.join(candidate, 'privkey.pem');
+        if (existsSync(fullchain) && existsSync(privkey)) {
+            return candidate;
+        }
+    }
+    return null;
+}
+
+function copyCertificates(retries = 5, retryDelaySecs = 3) {
+    mkdirSync(CERTS_DIR_PATH, { recursive: true });
+
+    for (let attempt = 1; attempt <= retries; attempt++) {
+        const certPath = findCertbotLivePath();
+        if (certPath) {
+            const username = os.userInfo().username;
+            execSync(`sudo cp "${path.join(certPath, 'fullchain.pem')}" "${path.join(CERTS_DIR_PATH, 'fullchain.pem')}"`, { stdio: 'inherit' });
+            execSync(`sudo cp "${path.join(certPath, 'privkey.pem')}" "${path.join(CERTS_DIR_PATH, 'privkey.pem')}"`, { stdio: 'inherit' });
+            execSync(`sudo chown ${username}:${username} "${CERTS_DIR_PATH}/fullchain.pem" "${CERTS_DIR_PATH}/privkey.pem"`, { stdio: 'inherit' });
+            console.log(`✓ Certificates copied from ${certPath}`);
+            return;
+        }
+        if (attempt < retries) {
+            console.log(`Certificate path not found, retrying in ${retryDelaySecs}s... (${attempt}/${retries})`);
+            execSync(`sleep ${retryDelaySecs}`);
+        }
+    }
+    throw new Error('Could not find certbot certificates. Check /etc/letsencrypt/live/');
+}
+
+async function setupCertificates() {
+    installCertbot();
+
+    let serverIp = 'YOUR_SERVER_IP';
+    try {
+        serverIp = execSync('curl -s --max-time 5 ifconfig.me || hostname -I | awk \'{print $1}\'', { stdio: 'pipe' }).toString().trim();
+    } catch (_) {}
+
+    console.log('\n── DNS Setup Required ────────────────────────────');
+    console.log(`  Add the following A records in your DNS provider:`);
+    console.log(`    ${DOMAIN}    →  ${serverIp}`);
+    console.log(`    *.${DOMAIN}  →  ${serverIp}`);
+    console.log('──────────────────────────────────────────────────\n');
+    await waitForEnter('Press Enter once the A records are set (propagation may take a few minutes)...\n');
+
+    console.log('Running certbot DNS-01 challenge...');
+    console.log(`Certbot will ask you to create a TXT record at _acme-challenge.${DOMAIN}`);
+    execSync(
+        `sudo certbot certonly --manual --preferred-challenges dns -d "*.${DOMAIN}" --agree-tos`,
+        { stdio: 'inherit' }
+    );
+
+    console.log('Copying certificates...');
+    copyCertificates();
+
+    console.log('Restarting Traefik...');
+    try {
+        execSync('docker restart postgres_proxy', { stdio: 'inherit' });
+    } catch (_) {
+        console.log('Traefik not running yet, skipping restart');
+    }
+}
+
 function createInitialFiles() {
     // config dir
     if (!existsSync(CONFIG_DIR)) {
@@ -445,10 +550,21 @@ function removeTenant(tenantId, manifests = {}) {
         throw new Error(`Tenant ${tenantId} not found`);
     }
 
+    try {
+        execSync(`docker compose stop ${serviceName}`, { stdio: 'inherit', cwd: CONFIG_DIR });
+    } catch (err) {
+        console.warn(`Warning: failed to stop ${serviceName}: ${err.message}`);
+    }
+    try {
+        execSync(`docker compose rm -f ${serviceName}`, { stdio: 'inherit', cwd: CONFIG_DIR });
+    } catch (err) {
+        console.warn(`Warning: failed to remove container ${serviceName}: ${err.message}`);
+    }
+
     delete doc.services[serviceName];
 
     const volumeName = `pgdata_${tenantId}`;
-    if (doc.volumes && doc.volumes[volumeName]) {
+    if (doc.volumes && Object.prototype.hasOwnProperty.call(doc.volumes, volumeName)) {
         delete doc.volumes[volumeName];
     }
 
@@ -463,9 +579,101 @@ function removeTenant(tenantId, manifests = {}) {
     generateTraefikDynamicConfig(manifests);
     updateTraefikService();
 
+    try {
+        const matching = execSync(
+            `docker volume ls --format '{{.Name}}' | grep -E '_${volumeName}$' || true`,
+            { stdio: 'pipe', cwd: CONFIG_DIR }
+        ).toString().trim();
+        if (matching) {
+            for (const vol of matching.split('\n')) {
+                if (vol.trim()) {
+                    try {
+                        execSync(`docker volume rm ${vol.trim()}`, { stdio: 'inherit', cwd: CONFIG_DIR });
+                    } catch (err) {
+                        console.warn(`Warning: failed to remove volume ${vol}: ${err.message}`);
+                    }
+                }
+            }
+        }
+    } catch (err) {
+        console.warn(`Warning: could not enumerate volumes: ${err.message}`);
+    }
+
     console.log(`✓ Removed tenant: ${tenantId}`);
-    console.log(`  Run: docker compose down ${serviceName}`);
-    console.log(`  Run: docker volume rm pgs_${volumeName}`);
+}
+
+function getTenantStatus(tenantId) {
+    const doc = loadCompose();
+    const serviceName = `pgs_${tenantId}`;
+    const exists = !!(doc.services && doc.services[serviceName]);
+
+    if (!exists) {
+        return { tenantId, exists: false, running: false };
+    }
+
+    try {
+        const inspect = JSON.parse(
+            execSync(`docker inspect ${serviceName}`, { stdio: 'pipe' }).toString()
+        );
+        const state = inspect[0]?.State || {};
+        return {
+            tenantId,
+            exists: true,
+            running: state.Running === true,
+            status: state.Status,
+            startedAt: state.StartedAt,
+        };
+    } catch (_) {
+        return { tenantId, exists: true, running: false, status: 'not-created' };
+    }
+}
+
+function showStatus() {
+    console.log('\n── Traefik Proxy ─────────────────────────────────');
+    try {
+        const inspect = JSON.parse(execSync('docker inspect postgres_proxy', { stdio: 'pipe' }).toString());
+        const state = inspect[0]?.State;
+        const running = state?.Running;
+        const status = state?.Status;
+        const startedAt = state?.StartedAt;
+        const exitCode = state?.ExitCode;
+
+        console.log(`  Status:      ${running ? '✓ running' : '✗ stopped'} (${status})`);
+        if (startedAt) console.log(`  Started at:  ${new Date(startedAt).toLocaleString()}`);
+        if (!running && exitCode !== undefined) console.log(`  Exit code:   ${exitCode}`);
+    } catch (_) {
+        console.log('  Status: container not found');
+    }
+
+    console.log('\n── Traefik Errors (last 100 lines) ───────────────');
+    try {
+        const logs = execSync('docker logs postgres_proxy --tail 100 2>&1', { stdio: 'pipe' }).toString();
+        const errorLines = logs
+            .split('\n')
+            .filter(line => /\b(ERR|error|WARN|warn)\b/i.test(line) && line.trim().length > 0);
+
+        if (errorLines.length === 0) {
+            console.log('  No errors or warnings found');
+        } else {
+            errorLines.forEach(line => console.log(`  ${line}`));
+        }
+    } catch (_) {
+        console.log('  Could not read container logs');
+    }
+
+    console.log('\n── Certificates ──────────────────────────────────');
+    const fullchainPath = path.join(CERTS_DIR_PATH, 'fullchain.pem');
+    if (existsSync(fullchainPath)) {
+        try {
+            const certInfo = execSync(`openssl x509 -in "${fullchainPath}" -noout -enddate -subject 2>/dev/null`, { stdio: 'pipe' }).toString().trim();
+            certInfo.split('\n').forEach(line => console.log(`  ${line}`));
+        } catch (_) {
+            console.log('  Could not read certificate info');
+        }
+    } else {
+        console.log('  ✗ fullchain.pem not found');
+    }
+    console.log('──────────────────────────────────────────────────\n');
 }
 
 function executeCommand(command) {
@@ -490,9 +698,33 @@ program
 
 program
     .command('setup')
-    .description('Initial required setup')
-    .action(() => {
-        initialSetup()
+    .description('Initial required setup (installs Docker, certbot, configures SSL)')
+    .action(async () => {
+        await initialSetup()
+    })
+
+program
+    .command('status')
+    .description('Show status of postgres_proxy container (or a tenant container if tenant-id is given)')
+    .argument('[tenant-id]', 'Tenant identifier (optional)')
+    .option('--json', 'Output tenant status as JSON (only with tenant-id)')
+    .action((tenantId, opts) => {
+        if (tenantId) {
+            checkSetupStatus()
+            try {
+                const info = getTenantStatus(tenantId);
+                if (opts.json) {
+                    process.stdout.write(JSON.stringify(info));
+                    return;
+                }
+                console.log(info);
+            } catch (error) {
+                console.error(`Error: ${error.message}`);
+                process.exit(1);
+            }
+            return;
+        }
+        showStatus()
     })
 
 program
@@ -509,9 +741,6 @@ program
         try {
 
             const providedTenantId = args[0];
-            const randomSuffix = Math.random().toString(36).substring(2, 8);
-            const tenantId = `${providedTenantId}-${randomSuffix}`;
-
             const opts = args[1];
 
             const tenantOptions = {
@@ -521,6 +750,8 @@ program
                 manifest: null
             }
 
+            let tenantId;
+
             if (opts.file) {
                 const manifestFilePath = path.resolve(opts.file)
                 const optsFromManifest = JSON.parse(readFileSync(manifestFilePath))
@@ -533,15 +764,23 @@ program
                 tenantOptions.limits = optsFromManifest.shared_limits
                 tenantOptions.version = optsFromManifest.version
                 tenantOptions.manifest = optsFromManifest
+                tenantOptions.password = opts.password || optsFromManifest.postgres?.password || generateRandomPassword();
+
+                tenantId = optsFromManifest.slug || providedTenantId;
+                if (!tenantId) {
+                    throw new Error('Manifest must contain "slug" or tenant-id must be provided');
+                }
             } else {
                 tenantOptions.version = opts.version;
                 tenantOptions.limits = {
                     cpu: parseFloat(opts.cpu),
                     memory: parseInt(opts.memory, 10),
                 };
-            }
+                tenantOptions.password = opts.password || generateRandomPassword();
 
-            tenantOptions.password = opts.password || generateRandomPassword();
+                const randomSuffix = Math.random().toString(36).substring(2, 8);
+                tenantId = `${providedTenantId}-${randomSuffix}`;
+            }
 
             createTenant(tenantId, tenantOptions);
         } catch (error) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@sysnee/pgs",
-  "version": "0.1.7-rc.11",
+  "version": "0.1.7-rc.13",
   "description": "Dynamic PostgreSQL service instance manager",
   "type": "module",
   "bin": {