vibed-infra 0.12.0 → 0.13.0

package/README.md

@@ -19,7 +19,7 @@ git add dist && git commit && git push
 
 ## Operator flow (VPS)
 
-Non-root agent prep (SSH, Docker group, one-time uid-1000 operator or ~/services ACL): [`skills/agent-vps-prep/SKILL.md`](skills/agent-vps-prep/SKILL.md).
+Non-root agent prep (SSH, Docker group): [`skills/agent-vps-prep/SKILL.md`](skills/agent-vps-prep/SKILL.md).
 
 ```bash
 # 1) DNS — paste dist/DNS-SKILL.md into AU browser agent
@@ -44,7 +44,10 @@ Multi-app: further products’ `install-gateway.sh` only add `apps/{other}/sites
 |------|---------|
 | `~/services/gateway` | Host nginx (`GATEWAY_HOME`) |
 | `~/services/vibed-infra/update-agent` | Serial pull queue; GHCR notify via GitHub Actions OIDC |
-| `~/services/vibed-infra/persist-logs` | Per-app WALs + optional R2/S3 ship |
+| `~/services/vibed-infra/persist-logs` | Ship credentials + host cron leftover; per-app WALs live in named volumes |
+| `~/services/vibed-infra/monitor-vibed.sh` | TUI: list deployments, summary, tail docker/volume logs |
+
+App durable data (SQLite, persist WALs, worker logs) uses **Docker named volumes** by default — not bind mounts under `~/services`. Digests recreate containers; volumes keep the DB.
 
 ## Image updates (docker pull)
 
@@ -62,7 +65,8 @@ See [`skills/infra-update-agent/SKILL.md`](skills/infra-update-agent/SKILL.md).
 | `package.sh` / `lib/package.py` | Build `dist/` |
 | `templates/host-gateway/` | Shared gateway skeleton |
 | `templates/update-agent/` | Queue agent + webhook |
-| `templates/persist-logs/` | Shipper install |
+| `templates/persist-logs/` | Shipper install + persist sidecar |
+| `templates/monitor/` | Machine-wide `monitor-vibed.sh` |
 | `lib/persistlog/` | Python append / seal / replay |
 | `skills/` | system-gateway, agent-vps-prep, infra-update-agent, infra-cicd, infra-packager, persist-logs, dns-configure |
 
@@ -75,7 +79,9 @@ See [`skills/infra-update-agent/SKILL.md`](skills/infra-update-agent/SKILL.md).
 | `VIBED_UPDATE_AGENT` | Override update-agent dir (default `$VIBED_HOME/update-agent`) |
 | `GATEWAY_PUBLIC_IP` | VPS IPv4 (from `gateway.publicIp`) |
 | `TLS_EMAIL` / `TLS_MODE` | Let’s Encrypt email; `lab` or `letsencrypt` (docker certbot → `./certs` PEMs; optional `CERTBOT_IMAGE` / `LETSENCRYPT_HOME`) |
-| `PERSIST_LOG_DIR` | Per-service event log dir |
+| `DATA_VOLUME` / `PERSIST_LOG_VOLUME` / `WORKER_LOG_VOLUME` | Named volume overrides (default `{container}-{data,persist,logs}`) |
+| `PERSIST_LOGS` | Set `0` to disable persist volume + sidecar |
+| `DATA_BIND` / `PERSIST_LOG_BIND` / `LOGS_BIND` | Force host bind mounts (debug) |
 | `DOCKER_AUTO_PRUNE` | Prune dangling images after update (default on) |
 
 ## Tests

package/install.sh

@@ -397,9 +397,58 @@ else:
     echo "webhook token registered for ${PRODUCT_NAME}"
   fi
 fi
-if [[ -f "${PACKAGER_RAW}/templates/persist-logs/install-persist-logs.sh" ]]; then
-  PACKAGER_RAW="$PACKAGER_RAW" VIBED_HOME="${VIBED_HOME:-}" bash "${PACKAGER_RAW}/templates/persist-logs/install-persist-logs.sh" || true
-fi
+# Machine-wide persist-logs + monitor (idempotent; works for path or http PACKAGER_RAW)
+_install_machine_helpers() {
+  local helper_packager="$PACKAGER_RAW"
+  if [[ "$LOCAL_PACKAGER" != "1" ]]; then
+    helper_packager="${DEST}/.infra-packager"
+    mkdir -p "$helper_packager/templates/persist-logs/lib" \
+      "$helper_packager/templates/monitor" \
+      "$helper_packager/lib"
+    _fetch "${PACKAGER_RAW}/templates/persist-logs/install-persist-logs.sh" \
+      "$helper_packager/templates/persist-logs/install-persist-logs.sh" || true
+    _fetch "${PACKAGER_RAW}/lib/persistlog/ship.py" \
+      "$helper_packager/lib/persistlog/ship.py" || true
+    # Minimal persistlog package for host cron / sidecar copy
+    mkdir -p "$helper_packager/lib/persistlog"
+    for f in __init__.py ship.py; do
+      _fetch "${PACKAGER_RAW}/lib/persistlog/${f}" \
+        "$helper_packager/lib/persistlog/${f}" || true
+    done
+    _fetch "${PACKAGER_RAW}/templates/monitor/install-monitor.sh" \
+      "$helper_packager/templates/monitor/install-monitor.sh" || true
+    _fetch "${PACKAGER_RAW}/templates/monitor/monitor-vibed.sh" \
+      "$helper_packager/templates/monitor/monitor-vibed.sh" || true
+    _fetch "${PACKAGER_RAW}/templates/persist-logs/start-persist-sidecar.sh" \
+      "$helper_packager/templates/persist-logs/start-persist-sidecar.sh" || true
+  fi
+  if [[ -f "${helper_packager}/templates/persist-logs/install-persist-logs.sh" ]]; then
+    chmod +x "${helper_packager}/templates/persist-logs/"*.sh 2>/dev/null || true
+    PACKAGER_RAW="$helper_packager" VIBED_HOME="${VIBED_HOME:-}" \
+      bash "${helper_packager}/templates/persist-logs/install-persist-logs.sh" || true
+  fi
+  if [[ -f "${helper_packager}/templates/monitor/install-monitor.sh" ]]; then
+    chmod +x "${helper_packager}/templates/monitor/"*.sh 2>/dev/null || true
+    PACKAGER_RAW="$helper_packager" VIBED_HOME="${VIBED_HOME:-}" \
+      bash "${helper_packager}/templates/monitor/install-monitor.sh" || true
+  fi
+  if [[ "$ROLE" == "backend" || "$ROLE" == "workers" ]]; then
+    # Prefer product dist copy when present; else packager template.
+    if [[ -f "${PRODUCT_RAW}/start-persist-sidecar.sh" ]]; then
+      cp -f "${PRODUCT_RAW}/start-persist-sidecar.sh" "$DEST/start-persist-sidecar.sh"
+    elif [[ "${PRODUCT_RAW}" =~ ^https?:// ]]; then
+      _fetch "${PRODUCT_RAW}/start-persist-sidecar.sh" "$DEST/start-persist-sidecar.sh" || true
+    elif [[ -f "${helper_packager}/templates/persist-logs/start-persist-sidecar.sh" ]]; then
+      cp -f "${helper_packager}/templates/persist-logs/start-persist-sidecar.sh" \
+        "$DEST/start-persist-sidecar.sh"
+    elif [[ -f "${PACKAGER_RAW}/templates/persist-logs/start-persist-sidecar.sh" ]]; then
+      cp -f "${PACKAGER_RAW}/templates/persist-logs/start-persist-sidecar.sh" \
+        "$DEST/start-persist-sidecar.sh"
+    fi
+    chmod +x "$DEST/start-persist-sidecar.sh" 2>/dev/null || true
+  fi
+}
+_install_machine_helpers
 
 INFRA_PROFILE="$PROFILE" ./install-auto-update.sh || true
 

package/lib/env.sh

@@ -209,6 +209,7 @@ memory_args() {
 }
 
 chown_data_dir() {
+  # Legacy bind-mount helper (DATA_BIND=1). Named volumes do not use this.
   local dir="$1"
   local uid="${2:-1000}"
   mkdir -p "$dir"
@@ -223,6 +224,146 @@ chown_data_dir() {
   chmod 755 "$dir" 2>/dev/null || true
 }
 
+# Create a durable named volume (idempotent). Never removes existing volumes.
+# Sets VIBED_VOLUME_CREATED=1 when a new volume was created, else 0.
+# Extra label key=value pairs after role/container are optional.
+vibed_ensure_volume() {
+  local name="$1"
+  local role="${2:-}"
+  local container="${3:-}"
+  shift 3 || true
+  VIBED_VOLUME_CREATED=0
+  local -a labels=(--label "vibed.managed=1")
+  [[ -n "$role" ]] && labels+=(--label "vibed.role=${role}")
+  [[ -n "$container" ]] && labels+=(--label "vibed.container=${container}")
+  local kv
+  for kv in "$@"; do
+    [[ -n "$kv" ]] && labels+=(--label "$kv")
+  done
+  if docker volume inspect "$name" >/dev/null 2>&1; then
+    return 0
+  fi
+  docker volume create "${labels[@]}" "$name" >/dev/null
+  VIBED_VOLUME_CREATED=1
+}
+
+# True if dir exists and has any entries (including hidden except . and ..).
+_vibed_host_dir_has_files() {
+  local dir="$1"
+  [[ -d "$dir" ]] || return 1
+  local f
+  for f in "$dir"/* "$dir"/.[!.]* "$dir"/..?*; do
+    [[ -e "$f" ]] && return 0
+  done
+  return 1
+}
+
+# One-time copy from a legacy host bind dir into a newly created empty named volume.
+# Do NOT mount the volume before the app unless migrating — any first mount can
+# initialize ownership from that helper image instead of the app image.
+vibed_migrate_host_dir() {
+  local vol="$1"
+  local host_dir="$2"
+  local newly_created="${3:-0}"
+  [[ -n "$vol" && -n "$host_dir" ]] || return 0
+  [[ "$newly_created" == "1" ]] || return 0
+  _vibed_host_dir_has_files "$host_dir" || return 0
+  local abs
+  abs="$(cd "$host_dir" && pwd)"
+  echo "Migrating $abs → volume $vol (keeping host copy) ..."
+  docker run --rm \
+    -v "${vol}:/dest" \
+    -v "${abs}:/src:ro" \
+    alpine sh -c 'cp -a /src/. /dest/' >/dev/null
+}
+
+# Resolve data storage: named volume (default) vs host bind (DATA_BIND=1 or non-default DATA_DIR).
+# Sets: VIBED_DATA_MODE=volume|bind, VIBED_DATA_VOLUME, VIBED_DATA_BIND_PATH
+vibed_resolve_data_storage() {
+  local container_name="$1"
+  local role="${2:-api}"
+  VIBED_DATA_MODE=volume
+  VIBED_DATA_VOLUME="${DATA_VOLUME:-${container_name}-data}"
+  VIBED_DATA_BIND_PATH=""
+  if env_flag_on DATA_BIND; then
+    VIBED_DATA_MODE=bind
+    VIBED_DATA_BIND_PATH="${DATA_DIR:-./data}"
+    return 0
+  fi
+  local dd="${DATA_DIR-}"
+  if [[ -n "$dd" && "$dd" != "./data" && "$dd" != "data" ]]; then
+    VIBED_DATA_MODE=bind
+    VIBED_DATA_BIND_PATH="$dd"
+    return 0
+  fi
+  vibed_ensure_volume "$VIBED_DATA_VOLUME" "$role" "$container_name" "vibed.kind=data"
+  if [[ -z "$dd" || "$dd" == "./data" || "$dd" == "data" ]]; then
+    vibed_migrate_host_dir "$VIBED_DATA_VOLUME" "./data" "${VIBED_VOLUME_CREATED:-0}"
+  fi
+}
+
+# Persist logs: default named volume + sidecar. PERSIST_LOGS=0 disables.
+# Sets: VIBED_PERSIST_MODE=volume|bind|off, VIBED_PERSIST_VOLUME, VIBED_PERSIST_BIND_PATH
+vibed_resolve_persist_storage() {
+  local container_name="$1"
+  local role="${2:-api}"
+  VIBED_PERSIST_MODE=volume
+  VIBED_PERSIST_VOLUME="${PERSIST_LOG_VOLUME:-${container_name}-persist}"
+  VIBED_PERSIST_BIND_PATH=""
+  if [[ -n "${PERSIST_LOGS+x}" ]] && ! env_flag_on PERSIST_LOGS; then
+    VIBED_PERSIST_MODE=off
+    return 0
+  fi
+  if env_flag_on PERSIST_LOG_BIND; then
+    VIBED_PERSIST_MODE=bind
+    VIBED_PERSIST_BIND_PATH="${PERSIST_LOG_DIR:-./persist-logs}"
+    return 0
+  fi
+  local pd="${PERSIST_LOG_DIR-}"
+  if [[ -n "$pd" && "$pd" != "./persist-logs" && "$pd" != "persist-logs" ]]; then
+    VIBED_PERSIST_MODE=bind
+    VIBED_PERSIST_BIND_PATH="$pd"
+    return 0
+  fi
+  vibed_ensure_volume "$VIBED_PERSIST_VOLUME" "$role" "$container_name" "vibed.kind=persist"
+  if [[ -z "$pd" || "$pd" == "./persist-logs" || "$pd" == "persist-logs" ]]; then
+    vibed_migrate_host_dir "$VIBED_PERSIST_VOLUME" "./persist-logs" "${VIBED_VOLUME_CREATED:-0}"
+  fi
+}
+
+# Worker logs volume (default) or bind.
+# Sets: VIBED_LOGS_MODE=volume|bind, VIBED_LOGS_VOLUME, VIBED_LOGS_BIND_PATH
+vibed_resolve_logs_storage() {
+  local container_name="$1"
+  local role="${2:-nodes}"
+  VIBED_LOGS_MODE=volume
+  VIBED_LOGS_VOLUME="${WORKER_LOG_VOLUME:-${container_name}-logs}"
+  VIBED_LOGS_BIND_PATH=""
+  if env_flag_on LOGS_BIND; then
+    VIBED_LOGS_MODE=bind
+    VIBED_LOGS_BIND_PATH="${LOGS_DIR:-./logs}"
+    return 0
+  fi
+  local ld="${LOGS_DIR-}"
+  if [[ -n "$ld" && "$ld" != "./logs" && "$ld" != "logs" ]]; then
+    VIBED_LOGS_MODE=bind
+    VIBED_LOGS_BIND_PATH="$ld"
+    return 0
+  fi
+  vibed_ensure_volume "$VIBED_LOGS_VOLUME" "$role" "$container_name" "vibed.kind=logs"
+  if [[ -z "$ld" || "$ld" == "./logs" || "$ld" == "logs" ]]; then
+    vibed_migrate_host_dir "$VIBED_LOGS_VOLUME" "./logs" "${VIBED_VOLUME_CREATED:-0}"
+  fi
+}
+
+vibed_home_dir() {
+  if [[ -n "${VIBED_HOME:-}" ]]; then
+    echo "${VIBED_HOME/#\~/$HOME}"
+    return 0
+  fi
+  echo "${HOME}/services/vibed-infra"
+}
+
 # Default on during auto-update. Set DOCKER_AUTO_PRUNE=0 to disable.
 docker_auto_prune_on() {
   if [[ -n "${DOCKER_AUTO_PRUNE+x}" ]]; then

package/lib/package.py

@@ -45,10 +45,17 @@ DOCKER_NETWORK={meta['network']}
 DOCKER_NAME={meta['apiContainer']}
 BACKEND_IMAGE={conf['images']['backend']}
 HOST_PORT={meta['apiPort']}
-DATA_DIR=./data
+# Durable data uses Docker named volume ${{DOCKER_NAME}}-data by default.
+# DATA_VOLUME=custom-data-vol
+# DATA_BIND=1
+# DATA_DIR=./data
 CONFIG_FILE=api-app.yaml
 API_MEMORY_LIMIT=256m
-PERSIST_LOG_DIR=./persist-logs
+# Persist logs: named volume ${{DOCKER_NAME}}-persist + ship sidecar (default on).
+# PERSIST_LOG_VOLUME=custom-persist-vol
+# PERSIST_LOGS=0
+# PERSIST_LOG_BIND=1
+# PERSIST_LOG_DIR=./persist-logs
 {au['flag']}=0
 {au['intervalEnv']}=30
 {au['stopTimeoutEnv']}=60
@@ -65,7 +72,6 @@ DOCKER_NETWORK={meta['network']}
 UI_NAME={meta['uiContainer']}
 UI_IMAGE={conf['images']['ui']}
 UI_MEMORY_LIMIT=32m
-PERSIST_LOG_DIR=./persist-logs
 {au['flag']}=0
 {au['intervalEnv']}=20
 {au['stopTimeoutEnv']}=30
@@ -87,7 +93,12 @@ WORKER_CONTAINER_NAME={meta['workerContainer']}
 WORKER_ROLE=heartbeat
 INTERVAL_SEC=60
 WORKER_MEMORY_LIMIT=64m
-PERSIST_LOG_DIR=./persist-logs
+# Logs + persist use named volumes by default (${{WORKER_CONTAINER_NAME}}-logs / -persist).
+# WORKER_LOG_VOLUME=
+# PERSIST_LOG_VOLUME=
+# PERSIST_LOGS=0
+# LOGS_BIND=1
+# LOGS_DIR=./logs
 {au['flag']}=0
 {au['intervalEnv']}=30
 {au['stopTimeoutEnv']}=30
@@ -186,6 +197,14 @@ networks:
     external: true
     name: ${{DOCKER_NETWORK:-{meta['network']}}}
 
+volumes:
+  worker_logs:
+    external: true
+    name: ${{WORKER_LOG_VOLUME:-{meta['workerContainer']}-logs}}
+  worker_persist:
+    external: true
+    name: ${{PERSIST_LOG_VOLUME:-{meta['workerContainer']}-persist}}
+
 services:
   worker:
     image: ${{WORKER_IMAGE:-{conf['images']['worker']}}}
@@ -193,13 +212,20 @@ services:
     restart: unless-stopped
     mem_limit: ${{WORKER_MEMORY_LIMIT:-64m}}
     env_file: .env
+    labels:
+      vibed.managed: "1"
+      vibed.role: nodes
+      vibed.logs-volume: ${{WORKER_LOG_VOLUME:-{meta['workerContainer']}-logs}}
+      vibed.persist-volume: ${{PERSIST_LOG_VOLUME:-{meta['workerContainer']}-persist}}
     environment:
       WORKER_ROLE: ${{WORKER_ROLE:-heartbeat}}
       WORKER_CONFIG: /config/workers.yaml
       API_URL: ${{API_URL:-http://{meta['apiContainer']}:{meta['apiPort']}}}
       INTERVAL_SEC: ${{INTERVAL_SEC:-60}}
+      PERSIST_LOG_DIR: /persist-logs
     volumes:
-      - ./logs:/data/logs
+      - worker_logs:/data/logs
+      - worker_persist:/persist-logs
       - ./nodes-workers.yaml:/config/workers.yaml:ro
     extra_hosts:
       - "host.docker.internal:host-gateway"
@@ -308,6 +334,11 @@ def build_dist(
     ):
         _copy_generic(packager_root, out_dir, script)
 
+    sidecar = packager_root / "templates" / "persist-logs" / "start-persist-sidecar.sh"
+    if sidecar.is_file():
+        shutil.copy2(sidecar, out_dir / "start-persist-sidecar.sh")
+        _chmod_x(out_dir / "start-persist-sidecar.sh")
+
     # Bake HTTP URLs into install wrappers when packaging for wget serve; else GitHub default.
     if packager_raw_val.startswith("http://") or packager_raw_val.startswith("https://"):
         wrapper_packager = packager_raw_val

package/lib/persistlog/ship.py

@@ -2,18 +2,105 @@
 """Ship sealed persist-log segments to R2/S3-compatible storage (batched PUTs)."""
 from __future__ import annotations
 
+import hashlib
+import hmac
 import os
+import shutil
+import subprocess
 import sys
 import urllib.error
 import urllib.request
+from datetime import datetime, timezone
 from pathlib import Path
+from urllib.parse import quote, urlsplit
 
-# Optional: use aws CLI if present (works with R2 endpoint)
-def ship_file(local: Path, key: str, endpoint: str, bucket: str, access: str, secret: str) -> None:
-    # Prefer aws CLI for SigV4
-    import shutil
-    import subprocess
 
+def _sigv4_put(
+    local: Path,
+    key: str,
+    endpoint: str,
+    bucket: str,
+    access: str,
+    secret: str,
+    region: str = "auto",
+) -> None:
+    """Minimal SigV4 PUT for S3/R2-compatible endpoints (stdlib only)."""
+    body = local.read_bytes()
+    payload_hash = hashlib.sha256(body).hexdigest()
+    now = datetime.now(timezone.utc)
+    amz_date = now.strftime("%Y%m%dT%H%M%SZ")
+    date_stamp = now.strftime("%Y%m%d")
+
+    parts = urlsplit(endpoint.rstrip("/"))
+    host = parts.netloc
+    # Virtual-hosted style preferred for R2: https://bucket.account.r2.cloudflarestorage.com/key
+    # Path style: https://endpoint/bucket/key
+    if host.startswith(f"{bucket}."):
+        url_path = "/" + quote(key, safe="/")
+        canonical_uri = url_path
+        host_header = host
+    else:
+        url_path = f"/{bucket}/{quote(key, safe='/')}"
+        canonical_uri = f"/{bucket}/{quote(key, safe='/')}"
+        host_header = host
+
+    service = "s3"
+    credential_scope = f"{date_stamp}/{region}/{service}/aws4_request"
+    canonical_headers = (
+        f"host:{host_header}\n"
+        f"x-amz-content-sha256:{payload_hash}\n"
+        f"x-amz-date:{amz_date}\n"
+    )
+    signed_headers = "host;x-amz-content-sha256;x-amz-date"
+    canonical_request = "\n".join(
+        [
+            "PUT",
+            canonical_uri,
+            "",
+            canonical_headers,
+            signed_headers,
+            payload_hash,
+        ]
+    )
+    string_to_sign = "\n".join(
+        [
+            "AWS4-HMAC-SHA256",
+            amz_date,
+            credential_scope,
+            hashlib.sha256(canonical_request.encode("utf-8")).hexdigest(),
+        ]
+    )
+
+    def _sign(key: bytes, msg: str) -> bytes:
+        return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
+
+    k_date = _sign(("AWS4" + secret).encode("utf-8"), date_stamp)
+    k_region = _sign(k_date, region)
+    k_service = _sign(k_region, service)
+    k_signing = _sign(k_service, "aws4_request")
+    signature = hmac.new(k_signing, string_to_sign.encode("utf-8"), hashlib.sha256).hexdigest()
+    authorization = (
+        f"AWS4-HMAC-SHA256 Credential={access}/{credential_scope}, "
+        f"SignedHeaders={signed_headers}, Signature={signature}"
+    )
+
+    url = f"{parts.scheme}://{host_header}{url_path}"
+    req = urllib.request.Request(url, data=body, method="PUT")
+    req.add_header("Host", host_header)
+    req.add_header("x-amz-content-sha256", payload_hash)
+    req.add_header("x-amz-date", amz_date)
+    req.add_header("Authorization", authorization)
+    req.add_header("Content-Length", str(len(body)))
+    try:
+        with urllib.request.urlopen(req, timeout=120) as resp:
+            if resp.status not in (200, 201, 204):
+                raise RuntimeError(f"PUT {url} returned {resp.status}")
+    except urllib.error.HTTPError as exc:
+        detail = exc.read().decode("utf-8", errors="replace")[:500]
+        raise RuntimeError(f"PUT {url} failed: {exc.code} {detail}") from exc
+
+
+def ship_file(local: Path, key: str, endpoint: str, bucket: str, access: str, secret: str) -> None:
     if shutil.which("aws"):
         env = os.environ.copy()
         env["AWS_ACCESS_KEY_ID"] = access
@@ -30,7 +117,8 @@ def ship_file(local: Path, key: str, endpoint: str, bucket: str, access: str, se
         ]
         subprocess.run(cmd, check=True, env=env)
         return
-    raise RuntimeError("aws CLI required for persist-log ship (or set PERSIST_SHIP=0)")
+    region = os.environ.get("R2_REGION") or os.environ.get("AWS_DEFAULT_REGION") or "auto"
+    _sigv4_put(local, key, endpoint, bucket, access, secret, region=region)
 
 
 def main() -> int:
@@ -43,6 +131,7 @@ def main() -> int:
     access = os.environ.get("R2_ACCESS_KEY_ID") or os.environ.get("AWS_ACCESS_KEY_ID") or ""
     secret = os.environ.get("R2_SECRET_ACCESS_KEY") or os.environ.get("AWS_SECRET_ACCESS_KEY") or ""
     machine = os.environ.get("PERSIST_MACHINE_ID") or os.uname().nodename
+    prefix = (os.environ.get("PERSIST_SHIP_PREFIX") or "").strip().strip("/")
     if not endpoint or not bucket or not access or not secret:
         print("missing R2/S3 credentials — skip ship", file=sys.stderr)
         return 0
@@ -53,7 +142,10 @@ def main() -> int:
         if marker.is_file():
             continue
         rel = seg.relative_to(base).as_posix()
-        key = f"{machine}/{rel}"
+        if prefix:
+            key = f"{machine}/{prefix}/{rel}"
+        else:
+            key = f"{machine}/{rel}"
         try:
             ship_file(seg, key, endpoint, bucket, access, secret)
             marker.write_text("ok\n", encoding="utf-8")

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vibed-infra",
-  "version": "0.12.0",
+  "version": "0.13.0",
   "description": "Product-agnostic VPS packager: wget install, Docker Compose, TLS/nginx, auto-update",
   "license": "MIT",
   "repository": {

package/schema/packageconfig.md

@@ -43,7 +43,7 @@ Compiled `packageconfig.yaml` also includes `webhook.url` / `fallbackUrl` (from
 | `DNS-SKILL.md` | Paste into AU browser agent; includes domains + `publicIp` when set |
 | `packageconfig.yaml` | compiled config |
 | `start-*.sh` / `update-*.sh` | lifecycle |
-| `.env.*.example` | includes `PERSIST_LOG_DIR`, prune flags |
+| `.env.*.example` | includes named-volume / persist / prune flags |
 
 Gateway install writes `$GATEWAY_HOME/apps/{name}/sites.conf` (host-extension mode).
 
@@ -57,9 +57,13 @@ Install registers each role with the machine **update-agent**. Pulls are serial
 
 `webhook.token` in compiled packageconfig is only for local curl; CI does not need `VIBED_WEBHOOK_SECRET`.
 
-## Persist logs
+## Persist logs + named volumes
 
-`PERSIST_LOG_DIR` per role. See [`skills/persist-logs/SKILL.md`](../skills/persist-logs/SKILL.md).
+By default API `/data`, persist WALs, and worker `/data/logs` use Docker **named volumes** (stable across digest-gated recreates). A vibed-prepared **persist sidecar** ships sealed segments using machine-wide R2/S3 config. See [`skills/persist-logs/SKILL.md`](../skills/persist-logs/SKILL.md).
+
+`.env.*.example` documents `DATA_VOLUME`, `PERSIST_LOG_VOLUME`, `PERSIST_LOGS=0`, and bind escape hatches.
+
+Inspect: `~/services/vibed-infra/monitor-vibed.sh`.
 
 ## Environment (install)
 

package/skills/agent-vps-prep/SKILL.md

@@ -2,7 +2,7 @@
 name: agent-vps-prep
 description: >-
   Prepare a fresh VPS for vibed-infra installs run by an automation agent
-  (cloud_agent): SSH key, sudoers, Docker group, one-time uid-1000/ACL ownership, firewall.
+  (cloud_agent): SSH key, sudoers, Docker group, firewall; app data uses named volumes.
 ---
 
 # Prepare a VPS for vibed-infra (agent operator)
@@ -67,42 +67,43 @@ sudo chown -R cloud_agent:cloud_agent /home/cloud_agent
 
 vibed-infra defaults: `~/services/gateway`, `~/services/vibed-infra`, product dirs under `~/services/<app>/`.
 
-## 5. Bind-mount ownership (uid 1000) — configure once
+## 5. App data ownership — named volumes (default)
 
-Product images often run as `node` (uid/gid **1000**). If the operator creates `./data` / `./logs` as a *different* uid, containers hit `SQLITE_CANTOPEN` / `EACCES`. Fix this **once on the host**, not after every product install.
+API data, persist logs, and worker logs use **Docker named volumes** by default (`{container}-data`, `{container}-persist`, `{container}-logs`). They are not under `~/services`, so the operator does **not** chown them to uid 1000 and does not need a `u:1000` ACL for SQLite/WALs.
 
-### Preferred: operator is already uid 1000
+In-volume ownership comes from the **app image** on first empty mount. Digest-gated recreates remount the same volume — the database is not wiped.
 
-If §1 created `cloud_agent` as uid/gid 1000, new trees under `~/services` are already writable by the container user. Nothing else to do.
+### Inspect data / logs over SSH
 
-### Fallback (configure once on `~/services`): default ACL
+```bash
+~/services/vibed-infra/monitor-vibed.sh
+# non-interactive:
+~/services/vibed-infra/monitor-vibed.sh --list
+~/services/vibed-infra/monitor-vibed.sh --summary myapp-api
+```
 
-When the operator is **not** uid 1000, grant uid 1000 rwx on every new file under services regardless of who creates it. Requires the `acl` package.
+Or:
 
 ```bash
-# as root, once after creating cloud_agent (any uid)
-apt-get install -y acl
-install -d -o cloud_agent -g cloud_agent /home/cloud_agent/services
-setfacl -m u:1000:rwx /home/cloud_agent/services
-setfacl -d -m u:1000:rwx /home/cloud_agent/services
-# also grant the operator full access via ACL defaults
-setfacl -m u:cloud_agent:rwx /home/cloud_agent/services
-setfacl -d -m u:cloud_agent:rwx /home/cloud_agent/services
+docker run --rm -v myapp-api-data:/data:ro alpine ls -la /data
+docker run --rm -v myapp-api-persist:/persist-logs:ro alpine ls -laR /persist-logs
 ```
 
-### Recovery only
+### Optional: bind-mount escape hatch
+
+`DATA_BIND=1` / `PERSIST_LOG_BIND=1` / `LOGS_BIND=1` with a host path. Only then does host uid matter for those dirs. Prefer named volumes.
 
-If someone already created wrong-owned dirs **before** uid-1000 or ACL setup:
+### Install-dir scripts under `~/services`
+
+Gateway certs, product `.env`, and yaml configs still live under `~/services`. Prefer operator uid 1000 **or** a one-time ACL on `~/services` so the agent can write install trees — not for app SQLite.
 
 ```bash
-sudo chown -R 1000:1000 \
-  /home/cloud_agent/services/*/api/data \
-  /home/cloud_agent/services/*/api/persist-logs \
-  /home/cloud_agent/services/*/nodes/logs
+# optional ACL for install scripts only (operator not uid 1000)
+apt-get install -y acl
+install -d -o cloud_agent -g cloud_agent /home/cloud_agent/services
+setfacl -m u:cloud_agent:rwx -d -m u:cloud_agent:rwx /home/cloud_agent/services
 ```
 
-Do not rely on per-product `chown` as ongoing ops. Prefer fixing the host model above.
-
 ## 6. Ports + DNS
 
 - Free **80/443** for the host gateway (only vps-gateway binds them)
@@ -129,6 +130,6 @@ wget -qO- https://raw.githubusercontent.com/ORG/REPO/main/deploy/.../dist/instal
 - [ ] `ssh cloud_agent@VPS` works with key only
 - [ ] `docker ps` works without sudo
 - [ ] `$HOME` owned by `cloud_agent`
-- [ ] operator uid is 1000 **OR** `~/services` has default ACL for `u:1000` (and `u:cloud_agent` if needed)
 - [ ] DNS → VPS IP
 - [ ] install-gateway + setup-tls + start-gateway
+- [ ] `~/services/vibed-infra/monitor-vibed.sh --list` shows deployments after start

package/skills/persist-logs/SKILL.md

@@ -14,18 +14,28 @@ Each service writes **domain events** to a durable local log. After DB loss (or
 
 ## Paths
 
-- Machine root: `~/services/vibed-infra/persist-logs/` (one-time `install-persist-logs.sh`)
-- Per app/role: `$PERSIST_LOG_DIR` (set in `.env`, mounted into API as `/persist-logs`)
+- Machine config (credentials + host cron leftover): `~/services/vibed-infra/persist-logs/` (one-time `install-persist-logs.sh`)
+- Per deployment: Docker **named volume** `${DOCKER_NAME}-persist` mounted at `/persist-logs` (default). Set `PERSIST_LOGS=0` to disable. Bind escape: `PERSIST_LOG_BIND=1` + `PERSIST_LOG_DIR=…`.
+- **Ship sidecar** `${DOCKER_NAME}-persist-ship` shares only that persist volume (no Docker socket, no app `/data`). Uses machine `.env` for R2/S3; app container does not get those secrets.
+- Object keys: `{machine}/{PERSIST_SHIP_PREFIX}/{rel}` (prefix defaults to the app container name).
 
 ## Algorithm
 
 1. **Append** NDJSON to `wal.ndjson` with fsync (fast, free).
 2. **Seal** → `seg-NNNN.ndjson.gz` at ~8 MiB or ~60s.
-3. **Ship** sealed segments with one PUT each (`PERSIST_SHIP=1` + R2/S3 env). Default `PERSIST_SHIP=0` (local only).
+3. **Ship** sealed segments with one PUT each (`PERSIST_SHIP=1` + R2/S3 env). Default `PERSIST_SHIP=0` (local only). Sidecar prefers stdlib SigV4; host cron may use `aws` CLI.
 4. **Replay** sealed + wal in order.
 
 Python: `lib/persistlog` — `append`, `seal`, `iter_events`, `replay`.
 
+## Inspect on the VPS
+
+```bash
+~/services/vibed-infra/monitor-vibed.sh
+# or:
+docker run --rm -v "${DOCKER_NAME}-persist:/persist-logs:ro" alpine ls -laR /persist-logs
+```
+
 ## Event schema
 
 ```json