vibed-infra 0.3.0

package/schema/packageconfig.md

@@ -0,0 +1,88 @@
+# packageconfig.yaml schema
+
+Product-owned file. Infra reads **only** the fields below.
+
+## Top level
+
+```yaml
+name: my-product          # slug for cron markers / logs
+version: 1
+
+rawBase: https://raw.githubusercontent.com/org/repo/main/deploy/templates
+packagerRaw: https://raw.githubusercontent.com/naiemk/vibed-infra/main
+
+network:
+  edge: my-product-edge   # shared Docker network for gateway + APIs
+
+images:
+  backend: ghcr.io/org/my-api:main
+  ui: ghcr.io/org/my-ui:main
+  worker: ghcr.io/org/my-worker:main
+  nginx: ghcr.io/org/my-nginx:main   # optional; default nginx:alpine + mounted conf
+
+autoUpdate:
+  lockFile: /var/lock/infra-auto-update.lock  # optional
+```
+
+## profiles
+
+Each install directory maps to one profile key (`api`, `nodes`, `gateway`, …).
+
+```yaml
+profiles:
+  api:
+    role: backend           # backend | workers | gateway
+    templates:
+      config: app.yaml      # fetched from rawBase; never overwritten if exists
+      envExample: .env.api.example
+    startScript: start-api.sh    # copied from rawBase or generated
+    updateScript: update-api.sh
+    autoUpdate:
+      flag: API_AUTO_UPDATE
+      intervalEnv: API_AUTO_UPDATE_INTERVAL_MIN
+      offset: 0               # cron minute offset (api :00, nodes :10, gateway :20)
+      stopTimeoutEnv: API_STOP_TIMEOUT
+    extras: []                # optional scripts from rawBase (e.g. register-node.sh)
+
+  nodes:
+    role: workers
+    templates:
+      config: workers.yaml
+      envExample: .env.nodes.example
+      compose: docker-compose.workers.yml
+    workers:
+      services:
+        - name: worker-a
+          containerName: my-worker-a
+          roleEnv: WORKER_ROLE=a
+          activityLog: /data/logs/a.jsonl
+        - name: worker-b
+          profile: optional   # compose profile name
+    autoUpdate: { flag: NODES_AUTO_UPDATE, offset: 10, ... }
+
+  gateway:
+    role: gateway
+    mode: standalone          # standalone | bundled
+    templates:
+      envExample: .env.gateway.example
+      nginxInclude: gateway/extra.conf   # optional product snippet
+    sites:
+      - host: app.example.com
+        aliases: [www.app.example.com]
+        backend: app-api
+        backendPort: 8080
+        ui: app-ui
+        uiPort: 80
+        healthPath: /api/health      # optional; default /api/health
+        createPath: /api/items       # optional rate-limited POST; omitted if unset
+        tlsCertDir: /etc/letsencrypt/live/app.example.com
+    autoUpdate:
+      flags: [UI_AUTO_UPDATE, GATEWAY_AUTO_UPDATE]
+      offset: 20
+```
+
+## Infra ignores
+
+- Keys inside `config` YAML templates
+- Secret names inside `.env` (opaque strings)
+- Application health-check response bodies

package/skills/infra-cicd/SKILL.md

@@ -0,0 +1,47 @@
+---
+name: infra-cicd
+description: >-
+  Set up GitHub Actions CI/CD for infra-packaged products: GHCR image build,
+  compose smoke, and install e2e against published images. Use when adding deploy
+  pipelines for Backend/UI/worker images.
+---
+
+# Infra packager CI/CD
+
+## Image build (GHCR)
+
+Use reusable workflow [`github/workflows/docker-build-reusable.yml`](../../github/workflows/docker-build-reusable.yml):
+
+```yaml
+jobs:
+  images:
+    uses: naiemk/vibed-infra/.github/workflows/docker-build-reusable.yml@main
+    with:
+      dockerfile: deploy/Dockerfile.api
+      image: ghcr.io/${{ github.repository_owner }}/my-api
+    secrets: inherit
+```
+
+Or copy the workflow into the product repo. Tags: `:main`, `main-<sha>`, semver on tag push.
+
+## Install e2e (optional job)
+
+1. Serve repo over HTTP (Python `http.server` at repo root or multi-path).
+2. Set `PACKAGER_RAW` to the packager (npm path, git checkout, or HTTP) and `PACKAGECONFIG_URL=.../deploy/packageconfig.yaml`.
+3. Optionally set `PRODUCT_RAW` if it should differ from `packageconfig.rawBase`.
+4. Run the product `install-api.sh` into a temp dir.
+5. Fill `.env` with test secrets; `./start-*.sh`; curl health.
+
+See [`examples/vps-hello/scripts/try-install.sh`](../../examples/vps-hello/scripts/try-install.sh) for a packager-only dry-run.
+
+## packageconfig in CI
+
+- `IMAGE_TAG=main` in system tests matches GHCR `:main` from default branch push.
+- Keep `rawBase` pointing at `main` branch raw URLs for operator wget; override with `PRODUCT_RAW` on a feature branch.
+
+## Checklist
+
+- [ ] Dockerfiles under `deploy/`
+- [ ] `deploy/packageconfig.yaml` images match GHCR names
+- [ ] Install dry-run or workflow job passes
+- [ ] Secrets not in templates — only `.env.example` placeholders

package/skills/infra-packager/SKILL.md

@@ -0,0 +1,54 @@
+---
+name: infra-packager
+description: >-
+  Wire a new product to the infra VPS packager: packageconfig.yaml, Docker images,
+  deploy/templates, thin install wrappers, and CI. Use when adding Backend/UI/worker
+  deploy to a repo or migrating from ad-hoc deploy/install scripts.
+---
+
+# Infra packager — onboard a product
+
+## When to use
+
+- New repo needs wget VPS install (backend + UI + workers + HTTPS gateway)
+- Splitting deploy scripts into reusable vibed-infra + product templates
+
+## Steps
+
+1. **Depend on** [`vibed-infra`](https://www.npmjs.com/package/vibed-infra) (`npm install vibed-infra`) or wget `install.sh` from this repo. Set `packagerRaw` in packageconfig.
+
+2. **Create** `deploy/packageconfig.yaml` — see [`schema/packageconfig.md`](../../schema/packageconfig.md) and the VPS example [`examples/vps-hello/packageconfig.yaml`](../../examples/vps-hello/packageconfig.yaml).
+
+3. **Add templates** under `deploy/templates/`:
+   - `.env.*.example` (secrets — opaque to infra)
+   - App config YAML (opaque)
+   - `start-*.sh` / `update-*.sh` (or use generic `start.sh`)
+   - Worker `docker-compose.*.yml` if multi-runner
+
+4. **Thin wrappers** (set profile + product URLs):
+
+```bash
+# deploy/install/install-api.sh
+export INFRA_PROFILE=api
+export PACKAGECONFIG_URL=https://raw.githubusercontent.com/ORG/REPO/main/deploy/packageconfig.yaml
+export PRODUCT_RAW=https://raw.githubusercontent.com/ORG/REPO/main/deploy/templates
+wget -qO- https://raw.githubusercontent.com/naiemk/vibed-infra/main/install.sh | bash
+```
+
+5. **Build images** — Dockerfiles in `deploy/`; push to GHCR; reference in `packageconfig.images`.
+
+6. **CI** — use [`github/workflows/docker-build-reusable.yml`](../../github/workflows/docker-build-reusable.yml); add install e2e serving the packager + `deploy/templates/` over HTTP.
+
+7. **VPS** — per component directory:
+
+```bash
+mkdir -p ~/app/api && cd ~/app/api
+wget -qO- .../deploy/install/install-api.sh | bash
+# edit .env, then ./start-api.sh (or ./start.sh)
+```
+
+## Rules
+
+- Infra never parses app config keys — only image names, ports, volume paths, site hostnames.
+- Never overwrite existing `.env` on re-install.
+- Gateway container names must match `sites[].backend` / `sites[].ui` on shared `network.edge`.

package/skills/system-gateway/SKILL.md

@@ -0,0 +1,52 @@
+---
+name: system-gateway
+description: >-
+  Configure a system-wide HTTPS gateway on one VPS for multiple products/domains.
+  Use when several infra-packaged projects share one host, edge network, and Let's Encrypt.
+---
+
+# System-wide gateway (multi-project)
+
+## Model
+
+- One **gateway** install dir per host (or per edge): nginx on ports 80/443.
+- Each product **backend** + **UI** containers join the same Docker network (`network.edge` from each product's packageconfig — use one shared name, e.g. `vps-edge`).
+- Gateway `sites[]` in one product's packageconfig **or** a dedicated `gateway-only` packageconfig lists all vhosts.
+
+## Steps
+
+1. Pick shared network: `DOCKER_NETWORK=vps-edge` in every product `.env`.
+
+2. Install APIs/UIs with distinct container names:
+   - `product-a-api`, `product-a-ui`
+   - `product-b-api`, `product-b-ui`
+
+3. Install gateway once:
+
+```bash
+mkdir -p ~/vps/gateway && cd ~/vps/gateway
+wget -qO- .../install-gateway.sh | bash
+```
+
+4. Edit gateway `packageconfig` `sites[]` (or merge generated `gateway/conf.d/domains.conf`) so each `host` maps to the correct `backend` + `ui` container names.
+
+5. **TLS** — single SAN cert or per-host certs:
+
+```bash
+sudo certbot certonly --standalone \
+  -d app-a.example.com -d app-b.example.com -d www.app-b.example.com
+```
+
+Set `TLS_FULLCHAIN` / `TLS_PRIVKEY` in gateway `.env`. Defaults come from the first `sites[].tlsCertDir` (or `/etc/letsencrypt/live/<host>`).
+
+6. **Auto-update** — stagger cron: APIs :00, workers :10, gateway :20 (infra default).
+
+## Dual-domain on one host
+
+Two APIs + two UIs on one edge network, one nginx `sites[]` list. See [`examples/vps-hello/packageconfig.yaml`](../../examples/vps-hello/packageconfig.yaml) for a single-site starting point.
+
+## Pitfalls
+
+- API must listen on a container name resolvable by nginx (`app-api:8080`, not `localhost`).
+- Do not bind host port 443 twice — only gateway publishes 443.
+- Pull UI/nginx **before** stop on gateway update (infra update scripts do this).

package/start.sh

@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+# Generic start — delegates to profile startScript from .infra-profile.
+set -euo pipefail
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+cd "$SCRIPT_DIR"
+if [[ -f .infra-profile ]]; then
+  # shellcheck source=/dev/null
+  source .infra-profile
+fi
+TARGET="${START_SCRIPT:-start.sh}"
+if [[ -x "$SCRIPT_DIR/$TARGET" && "$TARGET" != "start.sh" ]]; then
+  exec "$SCRIPT_DIR/$TARGET" "$@"
+fi
+echo "No profile start script — run product-specific start script listed in .infra-profile" >&2
+exit 1

package/templates/docker-compose.backend.yml

@@ -0,0 +1,32 @@
+# Generic backend service (optional compose path).
+# Most products use profile start script (docker create + config cp) for remote Docker compatibility.
+#
+# Usage: docker compose --env-file .env -f docker-compose.backend.yml up -d
+
+services:
+  backend:
+    image: ${BACKEND_IMAGE:-ghcr.io/example/backend:main}
+    container_name: ${DOCKER_NAME:-backend}
+    restart: ${RESTART:-unless-stopped}
+    mem_limit: ${BACKEND_MEMORY_LIMIT:-384m}
+    env_file: .env
+    environment:
+      CONFIG_PATH: /config/server.yaml
+      DB_PATH: /data/app.db
+    ports:
+      - "${HOST_PORT:-8080}:8080"
+    volumes:
+      - ${DATA_DIR:-./data}:/data
+      - ${CONFIG_FILE:-./app.yaml}:/config/server.yaml:ro
+    networks:
+      - edge
+    healthcheck:
+      test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:8080/api/health || exit 1"]
+      interval: 10s
+      timeout: 5s
+      retries: 6
+
+networks:
+  edge:
+    name: ${DOCKER_NETWORK:-app-edge}
+    external: false

package/templates/docker-compose.gateway.yml

@@ -0,0 +1,25 @@
+# Standalone HTTPS gateway: nginx + optional UI containers on shared edge network.
+# Prefer product-specific start script when nginx image bundles domains; this template uses mounted conf.
+
+services:
+  gateway:
+    image: ${NGINX_IMAGE:-nginx:alpine}
+    container_name: ${GATEWAY_NAME:-app-gateway}
+    restart: unless-stopped
+    mem_limit: ${GATEWAY_MEMORY_LIMIT:-64m}
+    ports:
+      - "${HTTP_PORT:-80}:80"
+      - "${HTTPS_PORT:-443}:443"
+    volumes:
+      - ./gateway/nginx.conf:/etc/nginx/nginx.conf:ro
+      - ./gateway/conf.d:/etc/nginx/conf.d:ro
+      - ${TLS_FULLCHAIN:-/etc/letsencrypt/live/example.com/fullchain.pem}:/etc/nginx/certs/fullchain.pem:ro
+      - ${TLS_PRIVKEY:-/etc/letsencrypt/live/example.com/privkey.pem}:/etc/nginx/certs/privkey.pem:ro
+      - ${CERTBOT_WWW:-/var/www/certbot}:/var/www/certbot:ro
+    networks:
+      - edge
+
+networks:
+  edge:
+    name: ${DOCKER_NETWORK:-app-edge}
+    external: true

package/templates/docker-compose.workers.yml

@@ -0,0 +1,17 @@
+# Generic worker runners — customize via packageconfig workers.services[].
+# Product install copies a concrete compose (e.g. docker-compose.sweepers.yml) from templates/.
+
+services:
+  worker:
+    image: ${WORKER_IMAGE:-ghcr.io/example/worker:main}
+    container_name: ${WORKER_CONTAINER_NAME:-app-worker}
+    restart: unless-stopped
+    mem_limit: ${WORKER_MEMORY_LIMIT:-192m}
+    env_file: .env
+    environment:
+      WORKER_CONFIG: /config/workers.yaml
+    volumes:
+      - ./logs:/data/logs
+      - ${CONFIG_FILE:-./workers.yaml}:/config/workers.yaml:ro
+    extra_hosts:
+      - "host.docker.internal:host-gateway"

package/templates/nginx/nginx.conf

@@ -0,0 +1,21 @@
+user nginx;
+worker_processes auto;
+error_log /var/log/nginx/error.log warn;
+pid /var/run/nginx.pid;
+
+events {
+  worker_connections 1024;
+}
+
+http {
+  include /etc/nginx/mime.types;
+  default_type application/octet-stream;
+  sendfile on;
+  keepalive_timeout 65;
+
+  limit_req_zone $binary_remote_addr zone=api_create:10m rate=1r/s;
+  limit_req_zone $binary_remote_addr zone=api_public:10m rate=20r/s;
+  limit_conn_zone $binary_remote_addr zone=addr:10m;
+
+  include /etc/nginx/conf.d/*.conf;
+}

package/update.sh

@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+# Generic update — delegates to profile updateScript from .infra-profile.
+set -euo pipefail
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+cd "$SCRIPT_DIR"
+if [[ -f .infra-profile ]]; then
+  # shellcheck source=/dev/null
+  source .infra-profile
+fi
+TARGET="${UPDATE_SCRIPT:-update.sh}"
+if [[ -x "$SCRIPT_DIR/$TARGET" && "$TARGET" != "update.sh" ]]; then
+  exec "$SCRIPT_DIR/$TARGET" "$@"
+fi
+echo "No profile update script — run product-specific update script listed in .infra-profile" >&2
+exit 1