vibed-infra 0.9.0 → 0.11.0

package/README.md

@@ -19,6 +19,8 @@ git add dist && git commit && git push
 
 ## Operator flow (VPS)
 
+Non-root agent prep (SSH, Docker group, uid 1000 data dirs): [`skills/agent-vps-prep/SKILL.md`](skills/agent-vps-prep/SKILL.md).
+
 ```bash
 # 1) DNS — paste dist/DNS-SKILL.md into AU browser agent
 # 2) App roles
@@ -62,7 +64,7 @@ See [`skills/infra-update-agent/SKILL.md`](skills/infra-update-agent/SKILL.md).
 | `templates/update-agent/` | Queue agent + webhook |
 | `templates/persist-logs/` | Shipper install |
 | `lib/persistlog/` | Python append / seal / replay |
-| `skills/` | system-gateway, infra-update-agent, infra-cicd, infra-packager, persist-logs, dns-configure |
+| `skills/` | system-gateway, agent-vps-prep, infra-update-agent, infra-cicd, infra-packager, persist-logs, dns-configure |
 
 ## Environment
 

package/lib/product_config.py

@@ -220,7 +220,16 @@ def dump_yaml(data: Any, indent: int = 0) -> str:
                     else:
                         lines.append(f"{sp}{pref}{k}: {v}")
             else:
-                lines.append(f"{sp}- {item}")
+                s = str(item)
+                if isinstance(item, bool):
+                    s = "true" if item else "false"
+                elif item is None:
+                    s = ""
+                elif any(c in s for c in ":{}[]#&*!|>'\"%@`") or s == "" or s.lower() in (
+                    "null", "true", "false", "yes", "no", "on", "off",
+                ):
+                    s = json.dumps(s)
+                lines.append(f"{sp}- {s}")
         return "\n".join(lines) + ("\n" if lines else "")
     return f"{sp}{data}\n"
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vibed-infra",
-  "version": "0.9.0",
+  "version": "0.11.0",
   "description": "Product-agnostic VPS packager: wget install, Docker Compose, TLS/nginx, auto-update",
   "license": "MIT",
   "repository": {

package/skills/agent-vps-prep/SKILL.md

@@ -0,0 +1,119 @@
+---
+name: agent-vps-prep
+description: >-
+  Prepare a fresh VPS for vibed-infra installs run by an automation agent
+  (cloud_agent): SSH key, sudoers, Docker group, data-dir ownership, firewall.
+---
+
+# Prepare a VPS for vibed-infra (agent operator)
+
+## Who this is for
+
+Non-root user (e.g. `cloud_agent`) that will run `wget …/dist/install-*.sh | bash`,
+`setup-tls.sh` (docker certbot), and `start-*.sh` without interactive sudo.
+
+## 1. Create the operator user
+
+(as root)
+
+```bash
+adduser --disabled-password --gecos "" cloud_agent
+mkdir -p /home/cloud_agent/.ssh
+chmod 700 /home/cloud_agent/.ssh
+# paste agent public key:
+echo 'ssh-ed25519 AAAA… cloud_agent' >> /home/cloud_agent/.ssh/authorized_keys
+chmod 600 /home/cloud_agent/.ssh/authorized_keys
+chown -R cloud_agent:cloud_agent /home/cloud_agent/.ssh
+```
+
+## 2. SSH from the agent machine
+
+Generate a key if needed, then connect with `StrictHostKeyChecking=accept-new`:
+
+```bash
+ssh-keygen -t ed25519 -f ~/.ssh/cloud_agent_vps -N ""
+ssh -o StrictHostKeyChecking=accept-new -i ~/.ssh/cloud_agent_vps cloud_agent@VPS_IP
+```
+
+## 3. Install Docker + add user to docker group
+
+```bash
+# as root — use distro docs or https://get.docker.com
+usermod -aG docker cloud_agent
+# cloud_agent must re-login for group to apply
+```
+
+Verify as `cloud_agent`: `docker ps` (no sudo).
+
+## 4. Home ownership
+
+If home was created oddly (root-owned `$HOME`):
+
+```bash
+sudo chown -R cloud_agent:cloud_agent /home/cloud_agent
+```
+
+vibed-infra defaults: `~/services/gateway`, `~/services/vibed-infra`, product dirs under `~/services/<app>/`.
+
+## 5. Container data directory ownership (uid 1000)
+
+Product images often run as `node` (uid/gid **1000**). Bind-mounted `./data` and `./logs` created by `cloud_agent` (e.g. uid 1001) cause `SQLITE_CANTOPEN` / `EACCES`.
+
+**One-shot after first install (or before first start), as root/sudo:**
+
+```bash
+# Trustless Commerce / typical vibed apps — adjust paths to your INSTALL_DIRs
+sudo chown -R 1000:1000 \
+  /home/cloud_agent/services/*/api/data \
+  /home/cloud_agent/services/*/api/persist-logs \
+  /home/cloud_agent/services/*/nodes/logs
+```
+
+Or per product:
+
+```bash
+sudo chown -R 1000:1000 ~/services/tctest/api/data ~/services/tctest/api/persist-logs
+sudo chown -R 1000:1000 ~/services/tcmain/api/data ~/services/tcmain/api/persist-logs
+sudo chown -R 1000:1000 ~/services/tctest/nodes/logs ~/services/tcmain/nodes/logs
+```
+
+Optional: allow passwordless chown for the agent (tight sudoers):
+
+```bash
+# /etc/sudoers.d/cloud_agent-chown
+cloud_agent ALL=(root) NOPASSWD: /usr/bin/chown -R 1000\:1000 /home/cloud_agent/services/*
+```
+
+Prefer the explicit one-shot; sudoers is optional.
+
+Note: start scripts may try `sudo chown` or a docker alpine chown — host sudo still helps for empty dirs created as the operator user.
+
+## 6. Ports + DNS
+
+- Free **80/443** for the host gateway (only vps-gateway binds them)
+- Point DNS A records at `gateway.publicIp` before `setup-tls.sh` (Let’s Encrypt)
+
+## 7. TLS (no host certbot required)
+
+With Docker, `setup-tls.sh` uses `certbot/certbot` and writes PEMs under `~/services/gateway/certs/`. No sudo needed for LE when docker works. See [system-gateway](../system-gateway/SKILL.md).
+
+## 8. First product install
+
+```bash
+export INSTALL_DIR=$HOME/services/myapp/api
+mkdir -p "$INSTALL_DIR"
+wget -qO- https://raw.githubusercontent.com/ORG/REPO/main/deploy/.../dist/install-api.sh | bash
+# edit .env, then ./start-api.sh
+# repeat ui, nodes, gateway
+```
+
+**Important:** `export INSTALL_DIR=...` before the pipe — `INSTALL_DIR=... wget | bash` does **not** pass the var into bash.
+
+## 9. Checklist
+
+- [ ] `ssh cloud_agent@VPS` works with key only
+- [ ] `docker ps` works without sudo
+- [ ] `$HOME` owned by `cloud_agent`
+- [ ] uid 1000 chown on data/logs after first mkdir/install
+- [ ] DNS → VPS IP
+- [ ] install-gateway + setup-tls + start-gateway

package/skills/system-gateway/SKILL.md

@@ -7,6 +7,8 @@ description: >-
 
 # System-wide host gateway (multi-app)
 
+Fresh VPS as non-root agent: see [agent-vps-prep](../agent-vps-prep/SKILL.md).
+
 ## Model
 
 ```
@@ -55,7 +57,7 @@ cd ~/services/gateway
 ./setup-tls.sh --force  # re-issue after host or IP change
 ```
 
-- **Production:** `setup-tls.sh` issues Let’s Encrypt via docker `certbot/certbot` by default (config under `$GATEWAY_HOME/letsencrypt`, PEMs copied to `$GATEWAY_HOME/certs/`). No sudo required when Docker is available. Host certbot + sudo still preferred when present (same `--config-dir` under gateway home). Webroot if gateway is up, else standalone. On failure, fix DNS then re-run `setup-tls.sh`.
+- **Production:** `setup-tls.sh` issues Let’s Encrypt via docker `certbot/certbot` by default (config under `$GATEWAY_HOME/letsencrypt`, PEMs installed into `$GATEWAY_HOME/certs/` — direct `cp` when readable, else alpine container copy+chown for root-owned `live/`). No sudo required when Docker is available. Host certbot + sudo still preferred when present (same `--config-dir` under gateway home). Webroot if gateway is up, else standalone. On failure, fix DNS then re-run `setup-tls.sh`.
 - **Lab/CI:** `TLS_MODE=lab` → multi-SAN self-signed under `./certs/`.
 
 ## Webhook path (immediate docker pull)
@@ -64,6 +66,7 @@ Host `00-default.conf` (HTTP) and each app `sites.conf` (HTTPS) proxy `/_vibed/h
 
 ## Pitfalls
 
+- Operator user / Docker / uid 1000 data dirs: [agent-vps-prep](../agent-vps-prep/SKILL.md).
 - Do not run a second standalone nginx on 80/443.
 - Container names in `sites.conf` must match running API/UI names on `vps-edge`.
 - After cert renewal or `setup-tls.sh`, reload happens automatically when the gateway container is already running.

package/templates/host-gateway/setup-tls.sh

@@ -254,13 +254,33 @@ EOF
     return 0
   fi
 
-  local live="${LE_HOME}/live/${primary}"
-  if [[ ! -f "${live}/fullchain.pem" || ! -f "${live}/privkey.pem" ]]; then
-    echo "certbot succeeded but ${live} PEMs missing" >&2
-    return 1
-  fi
-  cp -L "${live}/fullchain.pem" "${CERT_DIR}/fullchain.pem"
-  cp -L "${live}/privkey.pem" "${CERT_DIR}/privkey.pem"
+  # Copy PEMs into CERT_DIR without sudo. Docker certbot leaves root-owned live/
+  # (mode 700); host path may be readable after sudo chown above.
+  install_pems_to_certs() {
+    local primary="$1"
+    local live="${LE_HOME}/live/${primary}"
+    mkdir -p "$CERT_DIR"
+    if [[ -r "${live}/fullchain.pem" && -r "${live}/privkey.pem" ]]; then
+      cp -L "${live}/fullchain.pem" "${CERT_DIR}/fullchain.pem"
+      cp -L "${live}/privkey.pem" "${CERT_DIR}/privkey.pem"
+    elif command -v docker >/dev/null 2>&1; then
+      # certbot container wrote root-owned live/; copy+chown via alpine
+      docker run --rm \
+        -v "${LE_HOME}:/etc/letsencrypt:ro" \
+        -v "${CERT_DIR}:/out" \
+        "${CERTBOT_HELPER_IMAGE:-alpine:3.20}" \
+        sh -c "cp -L /etc/letsencrypt/live/${primary}/fullchain.pem /out/fullchain.pem && cp -L /etc/letsencrypt/live/${primary}/privkey.pem /out/privkey.pem && chown $(id -u):$(id -g) /out/fullchain.pem /out/privkey.pem && chmod 644 /out/fullchain.pem && chmod 600 /out/privkey.pem"
+    else
+      echo "Cannot read ${live} PEMs (root-owned?) and docker unavailable to copy them" >&2
+      return 1
+    fi
+    if [[ ! -f "${CERT_DIR}/fullchain.pem" || ! -f "${CERT_DIR}/privkey.pem" ]]; then
+      echo "Failed to install PEMs into ${CERT_DIR}" >&2
+      return 1
+    fi
+  }
+
+  install_pems_to_certs "$primary"
   chmod 644 "${CERT_DIR}/fullchain.pem"
   chmod 600 "${CERT_DIR}/privkey.pem"
   patch_env_key TLS_FULLCHAIN "${CERT_DIR}/fullchain.pem"