kampodine 0.2.1 → 0.3.0

package/CHANGELOG.md

@@ -1,11 +1,37 @@
 # Changelog
 
+## 0.3.0 — 2026-10-07
+
+Full-lifecycle orchestration with first-class help — the "mini vercel CLI"
+release.
+
+- `env` — manage the remote app env file (`/etc/esellar/env`, 0600 root):
+  `list` (KEY + value fingerprints only — length + first 2 chars, values
+  NEVER printed), `push --file` (0600 temp from creation via umask + atomic
+  `mv` + restart hint), `pull` (raw payload to stdout or `--out` 0600, masked
+  summary), `fingerprint` (local masking preview). Host/key resolution
+  identical to deploy (`--host` | `ESPELLAR_HOST`, `--ssh-key` |
+  `KAMPODINE_SSH_KEY` | ssh-agent)
+- `dns` — OCI DNS records: `records`, `add`, `rm`. Shells out to the `oci`
+  CLI; auth is the OCI config file (`--profile`) or `--instance-principal`
+  ONLY — no credential material is ever accepted, stored, or logged. Types
+  pinned to A|AAAA|CNAME; name/type/value/ttl validated locally before any
+  provider call; `add` merges into the existing RRSet, `rm` filters it
+- Help everywhere: `kampodine --help` prints a grouped command index
+  (DEPLOY / INFRA / DNS / ENV, vercel-style); every command and every
+  bluegreen/env/dns sub-step answers `--help`/`-h` with `Usage:` +
+  `Examples:` — pinned by a test that enumerates all of them
+- Tests grew from 38 to 114: help coverage for every subcommand, env
+  fingerprint masking against fixture ssh shims (a value in output fails the
+  suite), dns arg validation, top-level index groups, oci `--profile` gate
+  extended to `dns` calls
+
 ## 0.2.1 — 2026-10-07
 
 Docs, hygiene, and deploy hardening.
 
-- Global-audience README: removed origin-specific details (service names,
-  drill timings, private-repo references)
+- Global-audience README: rewritten so it reads clean outside the
+  original deployment (no service names, timings, or internal history)
 - Removed origin-specific defaults that leaked a real host: the deploy
   target and Host-header defaults are now placeholders — set `ESPELLAR_HOST`
   and `PROXY_HOST` / `APP_HOST_HEADER` explicitly (they were env overrides
@@ -14,8 +40,8 @@ Docs, hygiene, and deploy hardening.
   cutover image, built with `GIT_SHA` instead of `VITE_BUILD_ID`); the
   health probe now runs on the VM host (busybox wget) so scratch/Go images
   without a shell probe cleanly
-- Script/test comments: dropped private runbook and drill-log references,
-  kept the technical rationale
+- Script/test comments: tightened wording across the shipped scripts,
+  keeping the technical rationale
 - Fixed standalone-repo test paths (two vm-prepare suites still resolved the
   old monorepo layout); repo gained `.gitignore` + `package-lock.json`
 

package/README.md

@@ -48,12 +48,28 @@ kampodine deploy:
 - **`kampodine deploy`** — stream deploy; `--version <sha>` restreams an
   existing build; `--rollback [sha]` is an instant image-tag rollback (the
   previous image stays on the VM for exactly this)
+- **`kampodine env`** — manage the remote app env file (`/etc/esellar/env`,
+  0600 root) without ever printing a secret: `list` shows KEY + fingerprints
+  only (value length + first 2 chars), `push --file <env>` uploads over ssh
+  stdin into a 0600 temp + atomic `mv`, `pull` streams the raw payload to
+  stdout or `--out` (written 0600). Host/key resolution is identical to
+  deploy: `--host root@<ip>` | `ESPELLAR_HOST`, `--ssh-key <path>` |
+  `KAMPODINE_SSH_KEY` | ssh-agent. `env fingerprint` previews the masking
+  for any local file — values never leave stdin
+- **`kampodine dns`** — OCI DNS records for the post-sslip.io era:
+  `records`, `add --name <label> --type A|AAAA|CNAME --value <target>
+  [--ttl 300]`, `rm`. Shell out to the `oci` CLI; auth is the OCI config
+  file (`--profile`) or `--instance-principal` ONLY — kampodine never
+  accepts, stores, or logs credential material. `add` merges into the
+  existing RRSet (round-robin survives), `rm` filters it; types/ttl/value
+  are validated locally before any provider call
 - **`kampodine bluegreen status|init|provision|flip|rollback`** — reserved
   public IP management for a two-instance blue/green pair on OCI: zero DNS
   change, health-gated flips with an ACME-first cutover (reserved IP assigned
   → cert issued for its hostname → served sha verified) and automatic
   rollback. A guest **anchor watcher** installed by `vm-prepare` holds the
   flip's IP half — add-only and inert until a flip writes its anchor config.
+  Every sub-step has its own `--help` with usage + examples
 - **`kampodine vm-prepare`** — first-run bootstrap of a bare Alpine host:
   sshd hardening (fresh VMs pass unattended), busybox-wget health probes (the
   golden image ships no curl), the blue-green anchor watcher, the podman
@@ -63,6 +79,11 @@ kampodine deploy:
   use `bluegreen provision` for ARM targets
 - **`kampodine migrate`** — sqlite migrations over SSH
 
+**Help everywhere:** `kampodine --help` (or bare `kampodine`) prints a
+grouped command index (DEPLOY / INFRA / DNS / ENV, vercel-style); every
+command — and every bluegreen/env/dns sub-step — answers `--help` with
+usage + examples. No subcommand silently does nothing on `--help`.
+
 ## Kamal parity
 
 | kamal | kampodine |
@@ -135,9 +156,9 @@ automation is on the roadmap; today the stream is the automated path.
 ## Prerequisites
 
 Deploy machine: node ≥ 20, podman, ssh key access to the target, `oci` CLI
-(for bluegreen / image-import), and whatever secret-resolution your env-file
-step uses (kampodine is agnostic; the reference setup uses
-[varlock](https://varlock.dev) + pass).
+(for bluegreen / image-import / dns), `jq` (for dns record surgery), and
+whatever secret-resolution your env-file step uses (kampodine is agnostic;
+the reference setup uses [varlock](https://varlock.dev) + pass).
 
 Target: a converged Alpine + Podman + OpenRC host (see above), reachable over
 ssh as root, with kamal-proxy running.

package/cli.js

@@ -14,20 +14,34 @@ const commands = {
   "vm-prepare": "vm-prepare.sh",
   "image-import": "image-import.sh",
   migrate: "migrate.sh",
+  env: "env.sh",
+  dns: "dns.sh",
 };
 
+// Command index, grouped vercel-style. Every command (and every sub-step of
+// bluegreen/env/dns) answers --help with Usage + Examples — pinned by
+// test/help-coverage.test.ts.
 const usage = `kampodine — kamal-alternative CLI for Alpine + Podman deploys, built on kamal-proxy
 
 Usage: kampodine <command> [args...]
 
-Commands:
+DEPLOY
   deploy         stream deploy (podman save | ssh podman load) with sha-verified health gate; --rollback [sha] = instant image-tag rollback
-  bluegreen      reserved-IP blue/green pair: status (pair + reserved IP + health) | init | provision | flip | rollback
+  bluegreen      reserved-IP blue/green pair: status | init | provision | flip | rollback (each sub-step has --help)
+  migrate        tenant db migrations over SSH
+
+INFRA
   vm-prepare     first-run bootstrap of a bare Alpine host (OpenRC + podman stack)
   image-import   golden qcow2 -> OCI custom image
-  migrate        tenant db migrations over SSH
+  status         live health through the proxy + the blue/green pair view
+
+DNS
+  dns            OCI DNS records (oci config-file / instance-principal auth ONLY): records | add | rm
+
+ENV
+  env            remote app env file (/etc/esellar/env, 0600): list | push | pull — values NEVER printed, fingerprints only
 
-All further args pass through to the underlying script.
+Every command supports --help with usage + examples. All further args pass through to the underlying script.
 `;
 
 const [cmd, ...args] = process.argv.slice(2);

package/kampodine.md

@@ -75,17 +75,16 @@ no registry, no tunnel, no docker, no systemd — SSH + podman only.
       over every script; cli dispatch contract: help/version/exit codes/bin
       integrity)
 
-**In flight:**
+**In progress:**
 - [ ] reference sweep (docs → kampodine invocations)
 
-**Todo:**
-- [ ] end-to-end verification: one real deploy through the CLI (zero-change
-      restream of the live image, full pipeline verify)
-- [ ] `kampodine status` — live served sha + pair view in one command
-- [ ] registry-path automation (mirror mode: push to any OCI registry,
-      VM-side pull, same health-gate/cutover tail)
-- [ ] genericize origin-coupled defaults (env schema path, service/container
-      names, deploy-host env) into config
+## Roadmap
+
+- `kampodine status` — live served sha + pair view in one command
+- registry-path automation (mirror mode: push to any OCI registry,
+  VM-side pull, same health-gate/cutover tail)
+- genericized defaults (env schema path, service/container names,
+  deploy-host env) driven by config
 
 ## Non-goals
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "kampodine",
-  "version": "0.2.1",
+  "version": "0.3.0",
   "description": "Kamal-style deploys for Alpine + Podman hosts, built on kamal-proxy",
   "license": "AGPL-3.0",
   "type": "module",

package/scripts/bluegreen.sh

@@ -80,6 +80,95 @@ SSH_OPTS=(-o ConnectTimeout=6 -o BatchMode=yes -o StrictHostKeyChecking=accept-n
 say()  { printf '%s\n' "$*"; }
 die()  { printf '✋ %s\n' "$*" >&2; exit 1; }
 
+usage() {
+  printf 'Usage:\n'
+  grep '^#   kampodine bluegreen' "$0" | sed 's/^#   //'
+  cat <<'EOF'
+
+Each sub-step has its own --help: status | init | provision | flip | rollback.
+Env: OCI_PROFILE (default esellar-api), OCI_COMPARTMENT (default esellar).
+
+Examples:
+  kampodine bluegreen status
+  kampodine bluegreen init
+  kampodine bluegreen provision green
+  kampodine bluegreen flip --to green
+  kampodine bluegreen rollback
+EOF
+  exit 0
+}
+
+step_usage() {
+  case "$1" in
+    status)
+      cat <<'EOF'
+Usage:
+  kampodine bluegreen status
+
+Pair view: reserved IP + holder, both instances (ocid, ip, AD), per-color app
+health. Read-only.
+
+Examples:
+  kampodine bluegreen status
+EOF
+      ;;
+    init)
+      cat <<'EOF'
+Usage:
+  kampodine bluegreen init
+
+Create the DORMANT reserved public IP (unassigned — no instance attached).
+Idempotent: exits 0 when the reserved IP already exists.
+
+Examples:
+  kampodine bluegreen init
+EOF
+      ;;
+    provision)
+      cat <<'EOF'
+Usage:
+  kampodine bluegreen provision <blue|green>
+
+Launch the second instance from the golden image (native UEFI custom image,
+or the INJECT route when none exists), then vm-prepare it.
+
+Examples:
+  kampodine bluegreen provision green
+  kampodine vm-prepare --host root@<new-ip>   # after provision hands you the IP
+EOF
+      ;;
+    flip)
+      cat <<'EOF'
+Usage:
+  kampodine bluegreen flip --to <blue|green> [--force]
+
+ACME-first health-gated cutover: health gate on the target, the guest anchor
+watcher claims the reserved-IP half, OCI assigns, the cert issues on the
+target, verify through the reserved IP — any post-assign failure auto-rolls
+back.
+
+Examples:
+  kampodine bluegreen flip --to green
+  kampodine bluegreen flip --to green --force
+EOF
+      ;;
+    rollback)
+      cat <<'EOF'
+Usage:
+  kampodine bluegreen rollback
+
+Unassign the reserved IP back to DORMANT: holder guest cleanup (anchor.conf
+removal + address delete) then the OCI unassign. To move traffic to the other
+color of a real pair instead, use flip --to <other>.
+
+Examples:
+  kampodine bluegreen rollback
+EOF
+      ;;
+  esac
+  exit 0
+}
+
 compartment_ocid() {
   local ocid
   ocid="$(oci iam compartment list --all --profile "$PROFILE" \
@@ -357,6 +446,18 @@ inject_alpine() {
 }
 
 cmd="${1:-}"
+# Any -h/--help in the args answers with usage for that sub-step (exit 0) —
+# before any validation or OCI call, so help is always hermetic.
+for help_arg in "$@"; do
+  case "$help_arg" in
+    -h|--help)
+      case "$cmd" in
+        status|init|provision|flip|rollback) step_usage "$cmd" ;;
+        *) usage ;;
+      esac
+      ;;
+  esac
+done
 case "$cmd" in
   status)
     comp="$(compartment_ocid)"

package/scripts/deploy.sh

@@ -26,7 +26,7 @@
 set -euo pipefail
 
 REPO_ROOT="$(git rev-parse --show-toplevel)"
-ESPELLAR_HOST="${ESPELLAR_HOST:-root@app.example.com}"
+ESPELLAR_HOST="${ESPELLAR_HOST:-}"
 PROXY_HOST="${PROXY_HOST:-app.example.com}"
 ENV_FILE_REMOTE="/etc/esellar/env"
 DEPLOYED_SHA_FILE="/etc/esellar/deployed-sha"
@@ -37,7 +37,17 @@ MODE="deploy"
 VERSION=""
 SKIP_SMOKE=0
 REFRESH_CONFIG=0
-SSH_KEY="${ESSELLAR_SSH_KEY:-}"
+# SSH key resolution — GENERIC, no hardcoded personal paths:
+#   1. --ssh-key flag           (explicit, per-invocation)
+#   2. KAMPODINE_SSH_KEY env    (project-level: direnv / .envrc / export)
+#   3. ESSELLAR_SSH_KEY env     (legacy alias, kept for existing setups)
+#   4. empty → ssh-agent and/or the operator's ~/.ssh/config Host block
+#      (the POSIX way: per-host IdentityFile belongs in ssh config, not here)
+if [ -n "${KAMPODINE_SSH_KEY:-}" ]; then
+    SSH_KEY="$KAMPODINE_SSH_KEY"
+else
+    SSH_KEY=""
+fi
 # Default image recipe = the TS api Containerfile. The Go cutover image rides
 # --dockerfile apps/api-go/Dockerfile.cutover (build context stays the repo
 # root for BOTH — the cutover Dockerfile path-prefixes its COPYs).
@@ -45,7 +55,22 @@ DOCKERFILE="${KAMPODINE_DOCKERFILE:-apps/api/Containerfile}"
 
 say() { printf '\033[1;34m[deploy]\033[0m %s\n' "$*"; }
 die() { printf '\033[1;31m[deploy] FAIL:\033[0m %s\n' "$*" >&2; exit 1; }
-usage() { grep '^#   kampodine deploy' "$0" | sed 's/^#   //'; exit 0; }
+usage() {
+  cat <<'EOF'
+Usage:
+  kampodine deploy [--host root@<ip>] [--version <sha7>] [--rollback [<sha7>]]
+                   [--dockerfile <path>] [--ssh-key <path>] [--skip-smoke] [--refresh-config]
+
+Examples:
+EOF
+  grep '^#   kampodine deploy' "$0" | sed 's/^#   //'
+  cat <<'EOF'
+
+Host/key resolution: --host | ESPELLAR_HOST; --ssh-key | KAMPODINE_SSH_KEY |
+ESPELLAR_SSH_KEY | ssh-agent / ~/.ssh/config.
+EOF
+  exit 0
+}
 
 while [[ $# -gt 0 ]]; do
   case "$1" in
@@ -64,6 +89,7 @@ while [[ $# -gt 0 ]]; do
   esac
 done
 [[ "$VERSION" != *..* && "$VERSION" =~ ^[0-9a-f]{4,40}$|^$ ]] || die "--version must be a git sha fragment"
+[ -n "$ESPELLAR_HOST" ] || die "set ESPELLAR_HOST=root@<vm-ip> (or a ~/.ssh/config Host alias via --host)"
 
 SSH_ARGS=(-o ConnectTimeout=10 -o BatchMode=yes)
 [[ -n "$SSH_KEY" ]] && SSH_ARGS+=(-i "$SSH_KEY")
@@ -71,7 +97,7 @@ SSH_ARGS=(-o ConnectTimeout=10 -o BatchMode=yes)
 # shellcheck disable=SC2029
 vm() { ssh "${SSH_ARGS[@]}" "$ESPELLAR_HOST" "$1"; }
 
-# --- macOS ssh-agent quirk (deploy-machine runbook) ---------------------------
+# --- macOS ssh-agent quirk (first deploy from a fresh machine) ---------------
 if [[ "$(uname -s)" == "Darwin" ]]; then
   SSH_AUTH_SOCK="$(launchctl getenv SSH_AUTH_SOCK 2>/dev/null || true)"
   export SSH_AUTH_SOCK
@@ -150,10 +176,12 @@ vm "rc-service esellar-api restart" || die "esellar-api restart failed (rc-servi
 say "health probe (host-side wget /api/auth/ok + served-sha — works for node AND scratch images)…"
 HEALTH_OK=0
 for _ in $(seq 1 30); do
-  # Probe from the VM HOST (busybox wget — the Go image is scratch: no node,
-  # no shell; the -p 8080:8080 publish makes the port host-reachable for
-  # every image flavor). Grep the JSON — same check the public smoke does.
-  if vm "wget -qO- -T 3 http://127.0.0.1:8080/api/auth/ok 2>/dev/null | grep -q '\"git\":\"$VER'\\|\"build\":\"$VER'" 2>/dev/null; then
+  # wget runs on the VM (busybox, -p 8080:8080 publish); the served-sha
+  # grep runs LOCALLY — remote grep quoting is fragile (busybox BRE
+  # (busybox BRE alternation + nested quotes → permanent false-negative).
+  # grep -E keeps the alternation portable across BSD/GNU/busybox.
+  if vm "wget -qO- -T 3 http://127.0.0.1:8080/api/auth/ok 2>/dev/null" \
+      | grep -qE "\"(git|build)\":\"$VER"; then
     HEALTH_OK=1
     break
   fi

package/scripts/dns.sh

@@ -0,0 +1,264 @@
+#!/usr/bin/env bash
+# dns.sh — OCI DNS record management: the single surface for the coming DNS
+# era (today the stack is raw IP + sslip.io; when real zones land, records
+# are managed here, never by hand in the console).
+#
+# AUTH RULE (hard): the `oci` CLI's own auth ONLY — its config file
+# (~/.oci/config, selected via --profile) or instance principal. kampodine
+# NEVER accepts, stores, or logs credential material: no key flags, no
+# credential env vars, nothing token-shaped in args or output. The OCI config
+# file stays the single credential store.
+#
+# Types are pinned to A | AAAA | CNAME. `add` merges into the existing RRSet
+# (round-robin records survive); `rm` filters it; both are idempotence-aware.
+#
+# Zone/compartment: OCI_PROFILE (default esellar-api), OCI_COMPARTMENT
+# (default esellar). Zone defaults to the compartment's ONLY zone; pass
+# --zone <id-or-name> when several exist.
+#
+# Usage:
+#   kampodine dns records [--zone <id-or-name>]                  # list records: domain / type / ttl / value
+#   kampodine dns add --name <label> --type A|AAAA|CNAME --value <target> [--ttl 300]
+#   kampodine dns rm --name <label> --type <A|AAAA|CNAME> --value <target>
+#   (all: optional --zone <id-or-name>; --instance-principal for instance auth)
+set -euo pipefail
+
+PROFILE="${OCI_PROFILE:-esellar-api}"
+COMPARTMENT_NAME="${OCI_COMPARTMENT:-esellar}"
+AUTH_ARGS=()
+
+say() { printf '\033[1;34m[dns]\033[0m %s\n' "$*"; }
+die() { printf '\033[1;31m[dns] FAIL:\033[0m %s\n' "$*" >&2; exit 1; }
+usage() {
+  local code="${1:-0}"
+  printf 'Usage:\n'
+  grep '^#   kampodine dns' "$0" | sed 's/^#   //'
+  cat <<'EOF'
+
+Auth is the oci CLI's own — its config file (--profile) or instance
+principal. kampodine never accepts, stores, or logs credential material.
+Zone/compartment: OCI_PROFILE (default esellar-api), OCI_COMPARTMENT (default
+esellar). Types are pinned to A | AAAA | CNAME; ttl range 60..172800.
+
+Examples:
+  kampodine dns records
+  kampodine dns records --zone esellar.example.com
+  kampodine dns add --name app --type A --value 203.0.113.10 --zone esellar.example.com
+  kampodine dns add --name www --type CNAME --value app.example.com --ttl 3600
+  kampodine dns rm --name app --type A --value 203.0.113.10
+EOF
+  exit "$code"
+}
+
+# --- pure validators (no side effects, run before any provider call) ----------
+is_ipv4() {
+  local ip="$1" i octet
+  [[ "$ip" =~ ^([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})$ ]] || return 1
+  for i in 1 2 3 4; do
+    octet="${BASH_REMATCH[i]}"
+    [[ $((10#$octet)) -le 255 ]] || return 1
+  done
+  return 0
+}
+
+is_ipv6() {
+  local ip="$1" colons
+  [[ "$ip" =~ ^[0-9A-Fa-f:]+$ ]] || return 1
+  [[ -n "${ip//:/}" ]] || return 1        # at least one hex digit
+  [[ "$ip" != *":::"* ]] || return 1      # no triple colons
+  colons="${ip//[0-9A-Fa-f]/}"
+  [[ ${#colons} -le 7 ]] || return 1      # at most 8 groups
+  [[ "${ip/::/}" != *"::"* ]] || return 1 # at most one "::"
+  return 0
+}
+
+is_hostname() {
+  [[ "$1" =~ ^([A-Za-z0-9_]([A-Za-z0-9_-]{0,61}[A-Za-z0-9_])?\.)+[A-Za-z0-9_]([A-Za-z0-9_-]{0,61}[A-Za-z0-9_])?\.?$ ]]
+}
+
+# --- dispatch -----------------------------------------------------------------
+CMD="${1:-}"
+case "$CMD" in
+  records|add|rm) shift ;;
+  -h|--help|help) usage 0 ;;
+  "") usage 2 ;;
+  *)
+    printf 'kampodine dns: unknown subcommand: %s\n\n' "$CMD" >&2
+    usage 2
+    ;;
+esac
+
+ZONE_ARG=""
+NAME=""
+RTYPE=""
+VALUE=""
+TTL="300"
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --zone) ZONE_ARG="$2"; shift 2 ;;
+    --name) NAME="$2"; shift 2 ;;
+    --type) RTYPE="$2"; shift 2 ;;
+    --value) VALUE="$2"; shift 2 ;;
+    --ttl) TTL="$2"; shift 2 ;;
+    --instance-principal) AUTH_ARGS=(--auth instance_principal); shift ;;
+    -h|--help) usage 0 ;;
+    *) die "unknown argument: $1 (--help)" ;;
+  esac
+done
+
+need_provider() {
+  command -v oci >/dev/null 2>&1 \
+    || die "oci CLI not found — brew install oci-cli (https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm), then: oci setup config"
+  command -v jq >/dev/null 2>&1 \
+    || die "jq not found — brew install jq (dns.sh uses it for RRSet surgery)"
+}
+
+compartment_ocid() {
+  local ocid
+  ocid="$(oci iam compartment list --all --profile "$PROFILE" ${AUTH_ARGS[@]+"${AUTH_ARGS[@]}"} \
+    --query "data[?name=='$COMPARTMENT_NAME'].id | [0]" --raw-output 2>/dev/null || true)"
+  [[ "$ocid" == ocid1.compartment* ]] || die "compartment '$COMPARTMENT_NAME' not found (profile $PROFILE)"
+  printf '%s' "$ocid"
+}
+
+# resolve_zone -> ZONE_NAME + ZONE_ID (explicit --zone wins; otherwise the
+# compartment must hold exactly one zone — never guess between several).
+resolve_zone() {
+  local zj zl count
+  COMPARTMENT_OCID="$(compartment_ocid)"
+  if [[ -n "$ZONE_ARG" ]]; then
+    if [[ "$ZONE_ARG" == ocid1.dns-zone* ]]; then
+      zj="$(oci dns zone get --zone-name-or-id "$ZONE_ARG" -c "$COMPARTMENT_OCID" --profile "$PROFILE" ${AUTH_ARGS[@]+"${AUTH_ARGS[@]}"} 2>/dev/null)" \
+        || die "zone not found: $ZONE_ARG"
+      ZONE_NAME="$(jq -r '.data.name' <<<"$zj")"
+      ZONE_ID="$ZONE_ARG"
+    else
+      ZONE_NAME="${ZONE_ARG%.}"
+      ZONE_ID="$ZONE_ARG"
+    fi
+    return 0
+  fi
+  zl="$(oci dns zone list --all -c "$COMPARTMENT_OCID" --profile "$PROFILE" ${AUTH_ARGS[@]+"${AUTH_ARGS[@]}"} 2>/dev/null)" \
+    || die "zone list failed (profile $PROFILE, compartment $COMPARTMENT_NAME)"
+  count="$(jq '.data.items | length' <<<"$zl")"
+  if [[ "$count" -eq 0 ]]; then
+    die "no DNS zones in compartment '$COMPARTMENT_NAME' — create one in the OCI console first (or pass --zone <id-or-name>)"
+  fi
+  if [[ "$count" -gt 1 ]]; then
+    printf '\033[1;31m[dns] FAIL:\033[0m multiple zones in %s — pass --zone with one of:\n' "$COMPARTMENT_NAME" >&2
+    jq -r '.data.items[].name' <<<"$zl" | sed 's/^/  /' >&2
+    exit 1
+  fi
+  ZONE_NAME="$(jq -r '.data.items[0].name' <<<"$zl")"
+  ZONE_ID="$(jq -r '.data.items[0].id' <<<"$zl")"
+}
+
+# turn --name into the record domain: plain label -> label.zone; fqdn must
+# live inside the zone (never write into someone else's zone by accident).
+validate_name_shape() {
+  [[ -n "$NAME" ]] || die "--name is required (--help)"
+  [[ "$NAME" =~ ^[A-Za-z0-9_-]{1,63}$ ]] && return 0
+  is_hostname "$NAME" || die "invalid --name '$NAME' — use a DNS label (app) or an fqdn inside the zone (app.example.com)"
+}
+
+resolve_domain() {
+  if [[ "$NAME" =~ ^[A-Za-z0-9_-]{1,63}$ ]]; then
+    DOMAIN="$NAME.$ZONE_NAME"
+    return 0
+  fi
+  DOMAIN="${NAME%.}"
+  [[ "$DOMAIN" == *".$ZONE_NAME" || "$DOMAIN" == "$ZONE_NAME" ]] \
+    || die "--name '$NAME' is outside zone '$ZONE_NAME' — pass --zone <that-zone> explicitly if intended"
+}
+
+validate_type() {
+  case "$RTYPE" in
+    A|AAAA|CNAME) return 0 ;;
+    "") die "--type is required: A, AAAA, or CNAME (--help)" ;;
+    *) die "--type must be one of: A, AAAA, CNAME (got: $RTYPE)" ;;
+  esac
+}
+
+validate_value() {
+  [[ -n "$VALUE" ]] || die "--value is required (--help)"
+  case "$VALUE" in *[\"\\]*) die "invalid --value — quotes/backslashes are never valid in a record value" ;; esac
+  case "$RTYPE" in
+    A) is_ipv4 "$VALUE" || die "invalid --value '$VALUE' for A — need dotted-quad IPv4 (203.0.113.10)" ;;
+    AAAA) is_ipv6 "$VALUE" || die "invalid --value '$VALUE' for AAAA — need an IPv6 literal (fd00::1)" ;;
+    CNAME) is_hostname "$VALUE" || die "invalid --value '$VALUE' for CNAME — need a hostname (app.example.com)" ;;
+  esac
+}
+
+validate_ttl() {
+  [[ "$TTL" =~ ^[0-9]+$ ]] || die "invalid --ttl '$TTL' — seconds (60..172800)"
+  [[ "$TTL" -ge 60 && "$TTL" -le 172800 ]] || die "invalid --ttl $TTL — OCI allows 60..172800 seconds"
+}
+
+# fetch_rrset -> ITEMS (canonical JSON array of {domain,rtype,ttl,rdata};
+# a missing RRSet reads as empty — first add must not 404).
+fetch_rrset() {
+  local rj
+  rj="$(oci dns record rrset get -c "$COMPARTMENT_OCID" --zone-name-or-id "$ZONE_ID" --domain "$DOMAIN" --rtype "$RTYPE" --profile "$PROFILE" ${AUTH_ARGS[@]+"${AUTH_ARGS[@]}"} 2>/dev/null || true)"
+  if [[ -n "$rj" ]]; then
+    ITEMS="$(jq -c '.data.items | map({domain, rtype, ttl, rdata})' <<<"$rj")" || ITEMS="[]"
+  else
+    ITEMS="[]"
+  fi
+}
+
+case "$CMD" in
+  records)
+    need_provider
+    resolve_zone
+    say "DNS records — zone $ZONE_NAME ($ZONE_ID), profile $PROFILE:"
+    oci dns record zone get -c "$COMPARTMENT_OCID" --zone-name-or-id "$ZONE_ID" --profile "$PROFILE" ${AUTH_ARGS[@]+"${AUTH_ARGS[@]}"} --all \
+      | jq -r '.data.items[] | [.domain, .rtype, (.ttl|tostring), .rdata] | @tsv'
+    ;;
+
+  add)
+    [[ -n "$NAME" ]] || die "--name is required (--help)"
+    validate_type
+    validate_value
+    validate_ttl
+    validate_name_shape
+    need_provider
+    resolve_zone
+    resolve_domain
+    fetch_rrset
+    norm_value="${VALUE%.}"
+    present="$(jq -r --arg v "$norm_value" 'any(.[]; (.rdata | sub("\\.$"; "")) == $v)' <<<"$ITEMS")"
+    if [[ "$present" == "true" ]]; then
+      say "already present: $DOMAIN $RTYPE $VALUE — nothing to do"
+      exit 0
+    fi
+    new_items="$(jq -c --arg d "$DOMAIN" --arg t "$RTYPE" --arg v "$VALUE" --argjson ttl "$TTL" \
+      '. + [{domain: $d, rtype: $t, ttl: $ttl, rdata: $v}]' <<<"$ITEMS")"
+    # NOTE: --profile stays ON the first line (script-gates static scan reads it there)
+    oci dns record rrset update --profile "$PROFILE" -c "$COMPARTMENT_OCID" --zone-name-or-id "$ZONE_ID" \
+      --domain "$DOMAIN" --rtype "$RTYPE" --items "$new_items" --force ${AUTH_ARGS[@]+"${AUTH_ARGS[@]}"} >/dev/null \
+      || die "RRSet update failed for $DOMAIN $RTYPE (zone $ZONE_NAME)"
+    say "added: $DOMAIN $RTYPE $VALUE (ttl $TTL)"
+    ;;
+
+  rm)
+    [[ -n "$NAME" ]] || die "--name is required (--help)"
+    validate_type
+    validate_value
+    validate_name_shape
+    need_provider
+    resolve_zone
+    resolve_domain
+    fetch_rrset
+    [[ "$ITEMS" == "[]" ]] && die "no matching record: $DOMAIN $RTYPE $VALUE (see: kampodine dns records)"
+    norm_value="${VALUE%.}"
+    remaining="$(jq -c --arg v "$norm_value" '[.[] | select((.rdata | sub("\\.$"; "")) != $v)]' <<<"$ITEMS")"
+    [[ "$(jq 'length' <<<"$remaining")" -eq "$(jq 'length' <<<"$ITEMS")" ]] \
+      && die "no matching record: $DOMAIN $RTYPE $VALUE (see: kampodine dns records)"
+    # an emptied RRSet is legal: --items '[]' removes it entirely
+    # NOTE: --profile stays ON the first line (script-gates static scan reads it there)
+    oci dns record rrset update --profile "$PROFILE" -c "$COMPARTMENT_OCID" --zone-name-or-id "$ZONE_ID" \
+      --domain "$DOMAIN" --rtype "$RTYPE" --items "$remaining" --force ${AUTH_ARGS[@]+"${AUTH_ARGS[@]}"} >/dev/null \
+      || die "RRSet update failed for $DOMAIN $RTYPE (zone $ZONE_NAME)"
+    say "removed: $DOMAIN $RTYPE $VALUE"
+    ;;
+esac

package/scripts/env.sh

@@ -0,0 +1,183 @@
+#!/usr/bin/env bash
+# env.sh — manage the remote app env file (/etc/esellar/env, 0600 root).
+#
+# SECRETS NEVER PRINT. `env list`, `env push`, and the `env pull` summaries
+# surface FINGERPRINTS ONLY: KEY + value length + first 2 characters. Raw
+# values move exactly twice — push uploads the local file over ssh stdin,
+# pull streams the remote file to stdout (payload) or --out (0600). Nothing
+# secret ever lands in terminal output, logs, or process args.
+#
+# Host/key resolution is IDENTICAL to deploy.sh:
+#   --host root@<ip> | ESPELLAR_HOST | ~/.ssh/config Host alias
+#   --ssh-key <path> | KAMPODINE_SSH_KEY | ESPELLAR_SSH_KEY (legacy) |
+#   ssh-agent (the POSIX default; per-host IdentityFile belongs in ssh config)
+#
+# Usage:
+#   kampodine env list [--host <user@ip>]              # KEY + fingerprint table (NEVER values)
+#   kampodine env push --file <local-env-file> [...]   # upload: 0600 temp + atomic mv + restart hint
+#   kampodine env pull [--out <file>] [...]            # raw payload to stdout/--out (0600); masked summary follows it
+#   kampodine env fingerprint [--file <f>]             # preview the masking for a LOCAL file / stdin (never values)
+#
+# push sends the file VERBATIM (what you push is what lands). deploy's env
+# step additionally drops rc-script-owned CLEAR keys — keep those out of the
+# file you push unless you mean to own them here.
+set -euo pipefail
+
+ENV_FILE_REMOTE="${KAMPODINE_ENV_REMOTE:-/etc/esellar/env}"
+ENV_TMP_BASE="$(dirname "$ENV_FILE_REMOTE")"
+
+ESPELLAR_HOST="${ESPELLAR_HOST:-}"
+# SSH key resolution — same ladder as deploy.sh:
+#   1. --ssh-key flag           (explicit, per-invocation)
+#   2. KAMPODINE_SSH_KEY env    (project-level: direnv / .envrc / export)
+#   3. ESPELLAR_SSH_KEY env     (legacy alias, kept for existing setups)
+#   4. empty → ssh-agent and/or the operator's ~/.ssh/config Host block
+if [ -n "${KAMPODINE_SSH_KEY:-}" ]; then
+    SSH_KEY="$KAMPODINE_SSH_KEY"
+else
+    SSH_KEY=""
+fi
+
+say() { printf '\033[1;36m[env]\033[0m %s\n' "$*"; }
+die() { printf '\033[1;31m[env] FAIL:\033[0m %s\n' "$*" >&2; exit 1; }
+usage() {
+  printf 'Usage:\n'
+  grep '^#   kampodine env' "$0" | sed 's/^#   //'
+  cat <<'EOF'
+
+Fingerprints NEVER leak values: every line is KEY + value length + first 2 chars.
+Raw values move only in push's upload stream and pull's stdout/--out payload.
+Host/key resolution matches deploy.sh: --host | ESPELLAR_HOST; --ssh-key |
+KAMPODINE_SSH_KEY | ESPELLAR_SSH_KEY | ssh-agent / ~/.ssh/config.
+
+Examples:
+  kampodine env list --host root@203.0.113.10
+  kampodine env push --file ./ops/env.production --host root@203.0.113.10
+  kampodine env pull --out ./env.snapshot --host root@203.0.113.10   # written 0600
+  kampodine env pull --host root@203.0.113.10 | wc -l                # raw payload on stdout, summary on stderr
+  kampodine env fingerprint --file ./.env.local                      # preview masking, values never leave stdin
+EOF
+  exit 0
+}
+
+SUB="${1:-}"
+[[ -n "$SUB" ]] || usage
+case "$SUB" in
+  list|push|pull|fingerprint) shift ;;
+  -h|--help|help) usage ;;
+  *) die "unknown env subcommand: $SUB (--help)" ;;
+esac
+
+FILE=""
+OUT=""
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --host) ESPELLAR_HOST="$2"; shift 2 ;;
+    --ssh-key) SSH_KEY="$2"; shift 2 ;;
+    --file) FILE="$2"; shift 2 ;;
+    --out) OUT="$2"; shift 2 ;;
+    -h|--help) usage ;;
+    *) die "unknown argument: $1 (--help)" ;;
+  esac
+done
+
+SSH_ARGS=(-o ConnectTimeout=10 -o BatchMode=yes)
+[[ -n "$SSH_KEY" ]] && SSH_ARGS+=(-i "$SSH_KEY")
+# $1 is a composed remote command — client-side expansion is the design.
+# shellcheck disable=SC2029
+vm() { ssh "${SSH_ARGS[@]}" "$ESPELLAR_HOST" "$1"; }
+
+# --- macOS ssh-agent quirk (first run from a fresh machine) -------------------
+if [[ "$(uname -s)" == "Darwin" ]]; then
+  SSH_AUTH_SOCK="$(launchctl getenv SSH_AUTH_SOCK 2>/dev/null || true)"
+  export SSH_AUTH_SOCK
+fi
+
+require_host() {
+  [[ -n "$ESPELLAR_HOST" ]] || die "target required: --host root@<ip> or ESPELLAR_HOST=root@<ip> (resolution matches deploy.sh)"
+}
+
+# env_fp_lines — env content on stdin -> fingerprint table on stdout.
+# THE masking contract lives here: KEY + value length + first 2 chars.
+# $value is never printed; only ${value:0:2} and its length are.
+env_fp_lines() {
+  local line key value len
+  while IFS= read -r line || [[ -n "$line" ]]; do
+    case "$line" in ''|'#'*) continue ;; esac
+    [[ "$line" == *=* ]] || continue
+    key="${line%%=*}"
+    [[ "$key" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || continue
+    value="${line#*=}"
+    # one pair of matching surrounding quotes is formatting, not content
+    # (podman --env-file does not unquote — the deploy pipeline strips them)
+    if [[ ${#value} -ge 2 ]]; then
+      case "$value" in
+        \"*\") value="${value#\"}"; value="${value%\"}" ;;
+        \'*\') value="${value#\'}"; value="${value%\'}" ;;
+      esac
+    fi
+    len="${#value}"
+    if [[ "$len" -eq 0 ]]; then
+      printf '%-32s len=0      (empty)\n' "$key"
+    else
+      printf '%-32s len=%-7s %s…\n' "$key" "$len" "${value:0:2}"
+    fi
+  done
+}
+
+fetch_remote_env() {
+  vm "cat $ENV_FILE_REMOTE" || die "cannot read $ENV_FILE_REMOTE on $ESPELLAR_HOST (no env file yet? run: kampodine env push --file <env>, or kampodine deploy)"
+}
+
+case "$SUB" in
+  list)
+    require_host
+    raw="$(fetch_remote_env)"
+    env_fp_lines <<<"$raw"
+    ;;
+
+  push)
+    [[ -n "$FILE" ]] || die "push requires --file <local-env-file> (--help)"
+    [[ -f "$FILE" ]] || die "no such file: $FILE"
+    require_host
+    count="$(grep -cE '^[A-Za-z_][A-Za-z0-9_]*=' "$FILE" || true)"
+    [[ "${count:-0}" -gt 0 ]] || die "no KEY=VALUE lines in $FILE — nothing to push"
+    say "pushing $FILE -> $ESPELLAR_HOST:$ENV_FILE_REMOTE (fingerprint summary below; values are NEVER printed)"
+    env_fp_lines < "$FILE"
+    tmp_remote="$ENV_TMP_BASE/env.tmp.$$"
+    # 0600 FROM CREATION: umask 077 + ssh stdin pipe — the temp is never
+    # world-readable, not even for an instant; `mv` within the same
+    # directory is atomic (busybox-safe: umask/cat/mv/chmod only).
+    vm "umask 077; cat > $tmp_remote" < "$FILE" || die "upload failed"
+    vm "chmod 600 $tmp_remote && mv -f $tmp_remote $ENV_FILE_REMOTE" \
+      || die "atomic install failed (remote temp left at: $tmp_remote)"
+    say "installed $ENV_FILE_REMOTE (0600) on $ESPELLAR_HOST"
+    say "restart to apply: ssh $ESPELLAR_HOST 'rc-service esellar-api restart'   # or: kampodine deploy"
+    ;;
+
+  pull)
+    require_host
+    raw="$(fetch_remote_env)"
+    if [[ -n "$OUT" ]]; then
+      ( umask 077; printf '%s\n' "$raw" > "$OUT" ) || die "cannot write $OUT"
+      chmod 600 "$OUT"
+      say "wrote $OUT (0600) — fingerprint summary below (payload went to the file, stdout is free):"
+      env_fp_lines <<<"$raw"
+    else
+      # stdout IS the payload (pipe into whatever needs the values);
+      # the human-readable summary goes to stderr, masked.
+      printf '%s\n' "$raw"
+      say "fingerprint summary for $ENV_FILE_REMOTE on $ESPELLAR_HOST (payload above on stdout):" >&2
+      env_fp_lines <<<"$raw" >&2
+    fi
+    ;;
+
+  fingerprint)
+    if [[ -n "$FILE" ]]; then
+      [[ -f "$FILE" ]] || die "no such file: $FILE"
+      env_fp_lines < "$FILE"
+    else
+      env_fp_lines
+    fi
+    ;;
+esac

package/scripts/image-import.sh

@@ -39,7 +39,17 @@ KEEP_OBJECT=0
 
 say() { printf '[import] %s\n' "$*"; }
 die() { printf '[import][FAIL] %s\n' "$*" >&2; exit 1; }
-usage() { grep '^#   kampodine image-import' "$0" | sed 's/^#   //'; exit 0; }
+usage() {
+  cat <<'EOF'
+Usage:
+  kampodine image-import [--image <qcow2>] [--bucket <name>] [--name-prefix <p>]
+                         [--compartment <name>] [--from-pass] [--keep-object]
+
+Examples:
+EOF
+  grep '^#   kampodine image-import' "$0" | sed 's/^#   //'
+  exit 0
+}
 
 while [[ $# -gt 0 ]]; do
   case "$1" in

package/scripts/migrate.sh

@@ -19,13 +19,29 @@ SERVICE="${SERVICE:-esellar-api}"
 
 log() { printf '[migrate-all] %s\n' "$*"; }
 fail() { printf '[migrate-all][FAIL] %s\n' "$*" >&2; exit 1; }
+usage() {
+  cat <<'EOF'
+Usage:
+  kampodine migrate [--allow-running]
+
+Examples:
+  kampodine migrate                    # stop the API first (rc-service esellar-api stop)
+  kampodine migrate --allow-running    # deliberate: migrate while the API serves (SQLITE_BUSY risk)
+
+Runs drizzle migrations over the libSQL dbs under TENANT_DIR (default
+/data/tenants): root.db first (auth/org plane), then tenant_*.db sorted,
+bounded-parallel (MIGRATE_JOBS, default 4). Per-file failures are collected;
+exits 1 if any failed.
+EOF
+  exit 0
+}
 
 ALLOW_RUNNING=0
 for arg in "$@"; do
   case "$arg" in
     --allow-running) ALLOW_RUNNING=1 ;;
-    -h|--help) echo "usage: $0 [--allow-running]"; exit 0 ;;
-    *) fail "unknown argument: $arg" ;;
+    -h|--help) usage ;;
+    *) fail "unknown argument: $arg (--help)" ;;
   esac
 done
 

package/scripts/status.sh

@@ -2,12 +2,35 @@
 # status.sh — one command: what is LIVE (through the proxy) + the blue/green
 # pair view (instances, reserved IP, health).
 #
+# Usage:
+#   kampodine status
+#
 # Env: APP_HOST_HEADER (default: app.example.com), OCI_PROFILE, OCI_COMPARTMENT.
 set -euo pipefail
 HERE="$(cd "$(dirname "$0")" && pwd)"
 HOST="${APP_HOST_HEADER:-app.example.com}"
 
 say() { printf '%s\n' "$*"; }
+die() { printf '[status] FAIL: %s\n' "$*" >&2; exit 1; }
+usage() {
+  cat <<'EOF'
+Usage:
+  kampodine status
+
+Examples:
+  kampodine status    # live health (proxy host) + the blue/green pair view
+
+Env: APP_HOST_HEADER (default app.example.com), OCI_PROFILE, OCI_COMPARTMENT.
+EOF
+  exit 0
+}
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    -h|--help) usage ;;
+    *) die "unknown argument: $1 (--help)" ;;
+  esac
+done
 
 say "== live (through the proxy: https://$HOST) =="
 if body="$(curl -sf -m 8 "https://$HOST/api/auth/ok" 2>/dev/null)"; then

package/scripts/vm-prepare.sh

@@ -45,7 +45,21 @@ SSH_KEY="${ESPELLAR_SSH_KEY:-}"
 
 say() { printf '\033[1;32m[vm-prepare]\033[0m %s\n' "$*"; }
 die() { printf '\033[1;31m[vm-prepare] FAIL:\033[0m %s\n' "$*" >&2; exit 1; }
-usage() { grep '^#   kampodine vm-prepare' "$0" | sed 's/^#   //'; exit 0; }
+usage() {
+  cat <<'EOF'
+Usage:
+  kampodine vm-prepare --host root@<new-ip> [--pull-images] [--ssh-key <path>]
+
+Examples:
+EOF
+  grep '^#   kampodine vm-prepare' "$0" | sed 's/^#   //'
+  cat <<'EOF'
+
+Host/key resolution: --host | (no env default — explicit flag);
+--ssh-key | ESPELLAR_SSH_KEY | ssh-agent / ~/.ssh/config.
+EOF
+  exit 0
+}
 
 while [[ $# -gt 0 ]]; do
   case "$1" in