@aifabrix/server-setup 1.3.0 → 1.5.2

package/README.md

@@ -17,7 +17,8 @@ This is the **one document** you need to get your builder-server **from zero to
 **The builder-server Docker image is not on a public registry.** You get it with the **AI Fabrix platform**.
 
 1. Install the **AI Fabrix Builder** CLI: `npm install -g @aifabrix/builder`
-2. Use the CLI to run or deploy the platform (including builder-server). See [AI Fabrix Builder](https://github.com/esystemsdev/aifabrix-builder) and [docs/README.md](https://github.com/esystemsdev/aifabrix-builder/blob/main/docs/README.md).
+2. Install the **AI Fabrix server-setup** CLI (af-server): `npm install -g @aifabrix/server-setup`
+3. Use the Builder CLI to run or deploy the platform (including builder-server). See [AI Fabrix Builder](https://github.com/esystemsdev/aifabrix-builder) and [docs/README.md](https://github.com/esystemsdev/aifabrix-builder/blob/main/docs/README.md).
 
 Once you have the platform (and the builder-server image), use **af-server** to install that server on your own host.
 
@@ -39,19 +40,95 @@ Do the steps in [What you must do before running af-server](#what-you-must-do-be
 
 ## Install: from zero to running
 
-**From your PC** (SSH to server):
+Complete the [manual prerequisites](#what-you-must-do-before-running-af-server) first (DNS, SSL directory, certificate and key on the server).
+
+**Flow summary:** Only **step 1** runs over SSH from your PC. Steps **5** and **7** run **on the server** after you log in, so errors and output are visible directly there.
+
+| Step | Where     | Action |
+| ---- | --------- | ------ |
+| 1    | From PC   | `af-server install-init $SSH` — only command over SSH; installs on server: SSH (if needed), Node 18+, npm, and `af-server` CLI. |
+| 2    | One-time  | Log in to the server once (e.g. with password) to approve passwordless SSH; then from PC: `af-server ssh-cert install $SSH`. |
+| 3    | From PC   | Copy SSL certificate and key to the server (see [SSL directory and certificates](#ssl-directory-and-certificates)); example commands below. |
+| 4    | From PC   | Log in to the server via SSH (passwordless). |
+| 5    | **On server** | `sudo af-server install` — install all services (Docker, nginx package, Mutagen, cron, data dir); no builder-server container yet. |
+| 6    | On server | Get the builder-server image (e.g. `az login` / `docker pull`). |
+| 7    | **On server** | `sudo af-server install-server --dev-domain $DOMAIN` — nginx vhost, builder-server container, Docker TLS. |
+| 8    | —         | Done. |
+
+### Step 1: Bootstrap the server (from PC)
+
+Set your target and run the only command that uses SSH from your PC:
+
+```bash
+export SSH=serveradmin@builder02.aifabrix.dev
+export DOMAIN=builder02.aifabrix.dev
+af-server install-init $SSH
+```
+
+This installs on the server: openssh-server (if needed), Node 18+, npm, and `@aifabrix/builder` + `@aifabrix/server-setup` so `af-server` is available there. No Docker, nginx, or builder-server yet.
+
+### Step 2: Passwordless SSH (from PC)
+
+Log in to the server once (e.g. with password) to accept the host key and/or approve auth. Then from your PC:
+
+```bash
+af-server ssh-cert install $SSH
+```
+
+### Step 3: Copy SSL (from PC)
+
+Put `wildcard.crt` and `wildcard.key` in `/opt/aifabrix/ssl` on the server. Example (replace `$HDD` with the folder on your PC that has the cert and key):
+
+```bash
+export HDD=/workspace/aifabrix-setup/certificates
+ssh $SSH "sudo mkdir -p /opt/aifabrix/ssl"
+scp $HDD/wildcard.crt $SSH:/tmp/wildcard.crt
+scp $HDD/wildcard.key $SSH:/tmp/wildcard.key
+ssh $SSH "sudo mv /tmp/wildcard.crt /tmp/wildcard.key /opt/aifabrix/ssl/ && sudo chmod 600 /opt/aifabrix/ssl/wildcard.key"
+```
+
+### Step 4: Log in to the server
 
 ```bash
-af-server install user@host
+ssh $SSH
 ```
 
-**On the server itself** (Ubuntu, no target):
+### Step 5: Install services (on server)
 
 ```bash
-af-server install
+sudo af-server install
 ```
 
-After install, your builder-server is up. Use the **AI Fabrix Builder CLI** for all usage (users, secrets, certs, etc.)—see [Builder documentation](https://github.com/esystemsdev/aifabrix-builder/blob/main/docs/developer-isolation.md).
+This installs Docker, nginx (package only), Mutagen, data dir, apply-dev-users script and cron. It does **not** write the builder nginx vhost or start the builder-server container.
+
+### Step 6: Get the builder-server image (on server)
+
+The image is not on a public registry. Use your platform’s method (e.g. Azure CLI or Docker login), then pull. Example:
+
+```bash
+az login
+az acr login --name aifabrixdevacr
+docker pull aifabrixdevacr.azurecr.io/aifabrix/builder-server:latest
+```
+
+Or with Docker login (username/password from your registry):
+
+```bash
+docker login <registry> -u <user> -p <password>
+docker pull <registry>/aifabrix/builder-server:latest
+```
+
+### Step 7: Install server (nginx vhost + container) (on server)
+
+```bash
+sudo af-server install-server --dev-domain $DOMAIN
+```
+
+Use the same domain as your DNS and SSL. Optional: `--ssl-dir /opt/aifabrix/ssl`, `--data-dir /opt/aifabrix/builder-server/data`, `--builder-port 3000`.
+
+### Step 8: Done
+
+Your builder-server is up. Use the **AI Fabrix Builder CLI** for users, secrets, certs, etc.—see [Builder documentation](https://github.com/esystemsdev/aifabrix-builder/blob/main/docs/developer-isolation.md).
 
 ---
 
@@ -59,11 +136,14 @@ After install, your builder-server is up. Use the **AI Fabrix Builder CLI** for
 
 | Command | Description |
 | -------- | ----------- |
-| `af-server install [ user@host ] [ -d DATA_DIR ] [ --dev-domain DOMAIN ] [ --ssl-dir PATH ] [ -i SSH_KEY ]` | Install or update: Docker, nginx (builder vhost always updated from template), SSL proxy, sync user, cron. Re-run to apply latest config. |
+| `af-server install-init <user@host> [ -i SSH_KEY ]` | **From PC only.** One-time bootstrap over SSH: install on server SSH (if needed), Node 18+, npm, and af-server CLI. |
+| `af-server install [ user@host ] [ -d DATA_DIR ] [ --dev-domain DOMAIN ] [ --ssl-dir PATH ] [ -i SSH_KEY ]` | **Run on server** (omit target): `sudo af-server install`. Infra only: Docker, nginx pkg, Mutagen, data dir, cron. No builder vhost or container. With target: same infra over SSH. |
+| `af-server install-server --dev-domain DOMAIN [ -d DATA_DIR ] [ --ssl-dir PATH ] [ --builder-port PORT ]` | **On server only.** Nginx vhost, builder-server container, Docker TLS. Run after `sudo af-server install`. |
 | `af-server backup [ user@host ] [ -d DATA_DIR ] [ -o output.zip ] [ -i SSH_KEY ]` | On-demand backup (config + DB + keys). |
 | `af-server backup [ user@host ] --schedule [ --backup-dir PATH ] [ --keep-days N ] [ -i SSH_KEY ]` | Cron backup (daily 02:00, keep last N, default 7). |
 | `af-server restore backup.zip [ user@host ] [ -d DATA_DIR ] [ --force ] [ -i SSH_KEY ]` | Restore backup to DATA_DIR. |
 | `af-server ssh-cert install [ user@host ] [ -i SSH_KEY ]` | Add your SSH public key to server (passwordless auth). |
+| `af-server install-ssh [ user@host ] [ -i SSH_KEY ]` | Activate SSH server (install openssh-server, enable and start ssh) without login. Omit target for local. |
 
 Backups contain secrets; store encrypted. Cron backup needs SQLite (`builder.db`) and `zip` on the server; default backup dir: `/opt/aifabrix/backups`.
 
@@ -114,18 +194,23 @@ If you SSH as a non-root user, that user must be able to sudo. The script will a
 
 ---
 
-## High level: what the install does on your Ubuntu server
+## High level: what install vs install-server does
+
+**`af-server install`** (step 5 — run on the server) does **infra only**:
+
+- **System** — `apt update` and `apt upgrade`; optional hostname if `SETUP_HOSTNAME` is set.
+- **Docker** — Installs Docker if missing; enables and starts it.
+- **Admin user** — Adds the admin user (default `serveradmin`) to the `docker` group and grants passwordless sudo.
+- **Nginx** — Installs the nginx **package** only; enables and starts it. Does **not** write the builder vhost yet.
+- **Data dir** — Creates the data directory (default `/opt/aifabrix/builder-server/data`), workspace and ssh-keys subdirs, ownership for the container.
+- **Apply-dev-users** — Installs the script and cron job (every 2 minutes) that sync per-developer OS users from builder-server state.
+- **Mutagen** — Downloads and installs Mutagen; systemd service and daemon.
+- **Optional** — If `INSTALL_PORTAINER=1`, installs the Portainer container.
 
-When you run `af-server install`, the script (as root) does the following on the server:
+**`af-server install-server`** (step 7 — run on the server) does the **server phase**:
 
-- **System** — `apt update` and `apt upgrade`; optional hostname change if `SETUP_HOSTNAME` is set.
-- **Docker** — Installs Docker if missing; enables and starts the Docker service.
-- **Admin user** — Adds the admin user (default `serveradmin`) to the `docker` group and grants passwordless sudo (`/etc/sudoers.d/<admin>`).
-- **Nginx** — Installs nginx if missing; enables and starts it. Generates a site config for your domain that proxies HTTPS to the builder-server container (using `wildcard.crt` and `wildcard.key` from your SSL dir).
-- **Builder-server data dir** — Creates the data directory (default `/opt/aifabrix/builder-server/data`), sets ownership for the container. Starts the `builder-server` container if the image already exists on the server; otherwise prints how to build and run it (you get the image from the AI Fabrix platform).
-- **Sync user** — Creates a system user (default `aifabrix-sync`) for Mutagen SSH sync; home under the data dir; creates `.ssh` and `authorized_keys`.
-- **Cron job** — Installs a cron job (every 2 minutes) that copies the managed `authorized_keys` file into the sync user’s `.ssh/authorized_keys`.
-- **Mutagen** — Downloads and installs the Mutagen binary; creates a systemd service and starts the daemon.
-- **Optional** — If `INSTALL_PORTAINER=1`, installs the Portainer container. If Docker TLS is not skipped, writes `/etc/docker/daemon.json`; you can use the **same certificate** from `/opt/aifabrix/ssl` (e.g. symlink or copy `wildcard.crt` and `wildcard.key` to the paths Docker expects: `/etc/docker/server-cert.pem`, `/etc/docker/server-key.pem`, and if needed `ca.pem` for the CA), or provide separate Docker TLS certs.
+- **Nginx vhost** — Writes the builder site config from template (domain, SSL dir, proxy to builder-server), reloads nginx.
+- **Builder-server container** — Creates data dir (if needed), starts the builder-server container (if the image is present).
+- **Docker TLS** — Copies certs and configures `/etc/docker/daemon.json` for TLS (using website cert and builder-server CA).
 
-Result: your Ubuntu server has Docker, nginx (HTTPS for your domain), the builder-server container (if image present), the sync user and key sync, and Mutagen—ready for the Builder CLI to use.
+Result: after both steps, the server has Docker, nginx (HTTPS for your domain), the builder-server container (if image present), and Mutagen—ready for the Builder CLI to use.

package/assets/aifabrix-apply-dev-users.sh

@@ -0,0 +1,123 @@
+#!/bin/sh
+# Apply per-developer OS users from builder-server state (DATA_DIR/ssh-keys, pending-removals).
+# Run via cron every 2 minutes. Creates/updates dev<id> users, .ssh/authorized_keys, workspace symlink,
+# and optional ~/.aifabrix/config.yaml when missing. Processes pending-removals then applies keys.
+# Requires root. DATA_DIR must match builder-server (e.g. /opt/aifabrix/builder-server/data).
+
+set -e
+
+DATA_DIR="${DATA_DIR:-/opt/aifabrix/builder-server/data}"
+SSH_KEYS_DIR="${DATA_DIR}/ssh-keys"
+PENDING_REMOVALS="${DATA_DIR}/pending-removals"
+# Defaults matching builder-server env.template (override via env or apply-dev-users-defaults)
+AIFABRIX_SECRETS="${AIFABRIX_SECRETS:-/aifabrix-miso/builder/secrets.local.yaml}"
+AIFABRIX_ENV_CONFIG="${AIFABRIX_ENV_CONFIG:-aifabrix-miso/builder/env-config.yaml}"
+
+if [ ! -d "$DATA_DIR" ]; then
+  echo "DATA_DIR not found: $DATA_DIR"
+  exit 0
+fi
+
+# Read secrets-encryption key from server data dir (same as builder-server ENCRYPTION_KEY_PATH); do not log.
+secrets_encryption_value=""
+if [ -f "${DATA_DIR}/secrets-encryption.key" ]; then
+  secrets_encryption_value=$(cat "${DATA_DIR}/secrets-encryption.key" | tr -d '\n\r' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
+fi
+
+# --- 1. Process removals ---
+if [ -f "$PENDING_REMOVALS" ]; then
+  while IFS= read -r dev_id || [ -n "$dev_id" ]; do
+    dev_id=$(echo "$dev_id" | tr -d '\r\n ')
+    [ -z "$dev_id" ] && continue
+    user_name="dev${dev_id}"
+    if getent passwd "$user_name" >/dev/null 2>&1; then
+      userdel -r "$user_name" 2>/dev/null || userdel "$user_name" 2>/dev/null || true
+    fi
+  done < "$PENDING_REMOVALS"
+  : > "$PENDING_REMOVALS"
+fi
+
+# --- 2. Process each developer that has keys ---
+for key_file in "${SSH_KEYS_DIR}"/*/authorized_keys; do
+  [ -f "$key_file" ] || continue
+  dev_id=$(dirname "$key_file" | xargs basename)
+  [ -z "$dev_id" ] && continue
+  user_name="dev${dev_id}"
+  home_dir="/home/${user_name}"
+  workspace_target="${DATA_DIR}/workspace/${user_name}"
+
+  if ! getent passwd "$user_name" >/dev/null 2>&1; then
+    useradd -m -s /bin/bash "$user_name"
+  else
+    # Ensure shell is bash (e.g. after manual changes)
+    current_shell=$(getent passwd "$user_name" | cut -d: -f7)
+    if [ "$current_shell" != "/bin/bash" ]; then
+      usermod -s /bin/bash "$user_name" 2>/dev/null || true
+    fi
+  fi
+
+  mkdir -p "${home_dir}/.ssh"
+  cp "$key_file" "${home_dir}/.ssh/authorized_keys"
+  chown -R "${user_name}:${user_name}" "${home_dir}/.ssh"
+  chmod 700 "${home_dir}/.ssh"
+  chmod 600 "${home_dir}/.ssh/authorized_keys"
+
+  # .aifabrix/config.yaml only if missing (do not overwrite)
+  config_dir="${home_dir}/.aifabrix"
+  config_file="${config_dir}/config.yaml"
+  if [ ! -f "$config_file" ]; then
+    mkdir -p "$config_dir"
+    hostname_val=$(hostname -f 2>/dev/null || hostname 2>/dev/null || echo "localhost")
+    # Values from builder-server: secrets-encryption from DATA_DIR, paths from env (apply-dev-users-defaults or env.template)
+    yaml_escape() { echo "$1" | sed 's/\\/\\\\/g;s/"/\\"/g'; }
+    {
+      echo "user-mutagen-folder: ${workspace_target}"
+      printf "secrets-encryption: \"%s\"\n" "$(yaml_escape "$secrets_encryption_value")"
+      printf "aifabrix-secrets: \"%s\"\n" "$(yaml_escape "$AIFABRIX_SECRETS")"
+      printf "aifabrix-env-config: \"%s\"\n" "$(yaml_escape "$AIFABRIX_ENV_CONFIG")"
+      echo "remote-server: \"http://localhost:3000\""
+      echo "docker-endpoint: \"tcp://${hostname_val}:2376\""
+      echo "sync-ssh-user: \"${user_name}\""
+      echo "sync-ssh-host: \"${hostname_val}\""
+    } > "$config_file"
+    chown -R "${user_name}:${user_name}" "$config_dir"
+    chmod 700 "$config_dir"
+    chmod 600 "$config_file"
+  fi
+
+  # Workspace: ensure dir exists, owned by user; symlink from home
+  mkdir -p "$workspace_target"
+  chown -R "${user_name}:${user_name}" "$workspace_target"
+  if [ -L "${home_dir}/workspace" ]; then
+    current_target=$(readlink -f "${home_dir}/workspace" 2>/dev/null || true)
+    want_target=$(readlink -f "$workspace_target" 2>/dev/null || echo "$workspace_target")
+    if [ -n "$current_target" ] && [ -n "$want_target" ] && [ "$current_target" != "$want_target" ]; then
+      rm -f "${home_dir}/workspace"
+      ln -s "$workspace_target" "${home_dir}/workspace"
+      chown -h "${user_name}:${user_name}" "${home_dir}/workspace"
+    fi
+  elif [ ! -e "${home_dir}/workspace" ]; then
+    ln -s "$workspace_target" "${home_dir}/workspace"
+    chown -h "${user_name}:${user_name}" "${home_dir}/workspace"
+  fi
+
+  # Default directory on SSH login: cd to workspace
+  profile="${home_dir}/.profile"
+  if [ ! -f "$profile" ]; then
+    touch "$profile"
+    chown "${user_name}:${user_name}" "$profile"
+  fi
+  if ! grep -q 'cd.*workspace' "$profile" 2>/dev/null; then
+    echo '[ -d "$HOME/workspace" ] && cd "$HOME/workspace"' >> "$profile"
+    chown "${user_name}:${user_name}" "$profile"
+  fi
+  bashrc="${home_dir}/.bashrc"
+  if [ ! -f "$bashrc" ]; then
+    touch "$bashrc"
+    chown "${user_name}:${user_name}" "$bashrc"
+  fi
+  if ! grep -q 'cd.*workspace' "$bashrc" 2>/dev/null; then
+    echo '[ -d "$HOME/workspace" ] && cd "$HOME/workspace"' >> "$bashrc"
+    chown "${user_name}:${user_name}" "$bashrc"
+  fi
+done