@wpelevator/tug 0.11.0 → 0.13.0

package/docs/compose.md

@@ -0,0 +1,245 @@
+# Compose extensions
+
+Tug deploys standard Docker Compose files with an `x-tug` extension on services and volumes for placement and health checks. Compose is used only on the operator machine, where `tug deploy` normalizes the file; nodes run no Compose. Each agent turns the services placed on its node into Docker Engine API container definitions, without `x-tug`, `build`, `deploy` and `container_name`, with `extra_hosts` added for services on other nodes and `image` set to the built or declared image. Published `ports` without a host IP are bound to the node's `address` or `127.0.0.1` instead of every interface; write the host IP (for example `"0.0.0.0:80:80"`) to publish on purpose. See [Published ports](networking.md#published-ports-private-by-default).
+
+See [Health checks](healthchecks.md) to choose a probe and [Service placement and scaling](service-placement.md) to choose where services run.
+
+## Services
+
+```yaml
+services:
+  web:
+    image: nginx:1.27-alpine
+    ports:
+      - "8088:80"
+    environment:
+      SAMPLE_SECRET: ${SAMPLE_SECRET:-not-set}
+    x-tug:
+      placement:
+        nodes: [vps-a]
+      health:
+        type: http
+        port: 80
+        path: /
+```
+
+- `placement.nodes`: node names that run the service. With one registered node, a service without placement goes to that node. With multiple nodes, choose explicit nodes, labels or `global: true` for every service.
+- `placement.labels`: node labels (tags) that select the nodes running the service. Nodes must carry all listed labels. Combined with `replicas: N`, the control plane spreads the service across N matching nodes; `global: true` runs one replica on every matching node. Nodes whose agent has not been seen within `TUG_STALE_SECONDS` are skipped when labels or `global` are resolved (at deploy and on `tug scale`), so a dead host cannot hold a rollout open. Explicit `placement.nodes` still pins a service to named nodes, and `tug scale` never moves a pinned service off them.
+- `global`: run one replica on every node matching `placement.labels` (or every node when no labels are given).
+- `replicas`: optional. With explicit nodes and no labels, it must equal the number of named nodes. With label placement, it selects up to that many matching nodes. Tug runs at most one replica of a service per node.
+- `ingress`: publishes the service on a hostname through the managed Caddy edge (see [Networking](networking.md#layer-b-ingress-and-tls)). `host` is a fully qualified hostname, unique across the file; `port` is the service's container port or published host port and must appear in `ports`; `tls` is `auto` (default, ACME), `custom` or `off`. `tls: custom` also needs `cert` and `key`, the names of project secrets holding the PEM certificate chain and private key, which must exist before you deploy.
+- `rollout`: `strategy: recreate` (default) or `rolling`, plus `drain`; see [Rolling rollouts](#rolling-rollouts). The top-level `x-tug: { rollback: auto }` is described under [Automatic rollback](#automatic-rollback).
+- `health`: checked by the agent after the container starts; the service is `serving` only once it passes.
+  - `type: docker` waits for the container to be running and, if it defines a Docker `HEALTHCHECK`, healthy.
+  - `type: tcp` connects to `port` on the container IP.
+  - `type: http` requests `http://<container-ip>:<port><path>` (defaults `80` and `/`) and expects a 2xx response after following redirects.
+  - `interval` (default `2s`), `timeout` (default `2s`) and `retries` (default `10`) accept numbers in milliseconds or `ms`/`s`/`m` suffixes.
+
+`depends_on` sets start order on a node. `condition: service_healthy` waits on the dependency's `x-tug.health` when it has one; otherwise it checks that the container is running and, if present, its Docker `HEALTHCHECK` is healthy. Without either health check it only verifies that the container is running. A failed apply is included in the deployment error `tug deploy` prints, and recorded once as `service.failed`.
+
+```yaml
+    x-tug:
+      placement:
+        nodes: [app1]
+      ingress:
+        host: app.example.com
+        port: 80
+        tls: auto            # auto | custom | off
+```
+
+Services that run on a different node than a given service are reachable from it by name (`db` or `db.<project>.tug`) by default, as long as the target publishes its port and the client's node can route to the target node's private `address`. Tug binds the port to that address; the operator's firewall controls access to it. Set `TUG_SERVICE_NAMES=false` on control to disable the injected names. The agent adds these names to the containers as hosts-file (`extra_hosts`) entries, not DNS records; see [Networking](networking.md#layer-a-cross-node-service-names).
+
+Services with `build:` are built by control from uploaded source, or on the operator machine with `deploy --build-local`. Each build receives an immutable tag distinct from the display version and is delivered as a saved image archive from control. Build services require a source context or local image upload. Published `image:` tags are resolved once on control to `image@sha256:…` before deployment publication; explicit SHA-256 references remain pinned. Multi-platform indexes remain intact. Agents pull the stored digest from the upstream registry, so rollback and nodes added later use that same manifest. Private registry credentials use the encrypted project secret described in [Control plane](control-plane.md#published-images-and-private-registries). Uploaded source contexts must contain regular files/directories within the build root; symlinks and escaping build paths are rejected.
+
+### Rolling rollouts
+
+By default a changed service is recreated in place, which drops its traffic for a moment. A service behind the managed ingress can instead opt in to a rolling rollout:
+
+```yaml
+services:
+  web:
+    image: ghcr.io/acme/web:2
+    ports:
+      - "8080:80"
+    x-tug:
+      placement:
+        nodes: [app1]
+      health:
+        type: http
+        port: 80
+      ingress:
+        host: app.example.com
+        port: 80
+      rollout:
+        strategy: rolling      # recreate (default) | rolling
+        drain: 30s             # how long the previous revision keeps running after takeover (default 30s)
+```
+
+With `strategy: rolling` the agent starts the new revision next to the old one, health-checks it, and only then lets the ingress switch to it. The old revision is removed once `drain` has passed. If the new revision never becomes healthy it is removed, the old one keeps serving and the node reports `failed` for that deployment (see [automatic rollback](#automatic-rollback)). Nothing else on the node restarts: a rolling service whose rendered definition and secrets did not change is left alone, and so are recreated services.
+
+`validateCompose` requires a rolling service to have `x-tug.ingress` (the edge is what switches traffic), `x-tug.health` (a revision only takes over once healthy) and exactly one entry in `ports` (the ingress port). `drain` takes the same durations as health checks (`500ms`, `30s`, `1m`, or a number of milliseconds) and should be longer than the poll intervals of the app node and the ingress node together (5s each by default), because the ingress only learns about the new revision through the app node's next status report and the edge node's next poll. `maxSurge` and per-service batch sizes are not supported: one new revision is started per changed service.
+
+How it works, and its limits:
+
+- Rolling services are meant to be stateless. Each revision is its own container (`tug-<project>-<service>-<hash8>`), and old and new revisions run side by side, so named volumes are shared between them (the project's `tug_v<project-length>_<project>_<volume>`) and concurrent access is the operator's responsibility. Use `recreate` for databases and anything that cannot have two writers.
+- Each revision publishes the ingress port on its own host port from a free range (`TUG_ROLLOUT_PORTS` on the agent, default `20000-20999`), not on the declared host port. The declared port, for example `8080` above, is only the contract for the ingress validation. Reach a rolling service through its ingress hostname; see [Networking](networking.md#rolling-services-and-service-names) for what that means for service names.
+- `depends_on` is dropped for rolling services and from recreated services that point at them, because rolling revisions are started and replaced on their own schedule rather than as part of the project's start order. All revisions share the project network, so names such as `db` still resolve; use health checks and client retries rather than start order.
+- Switching a service from `recreate` to `rolling` keeps the old container in the main project serving until its replacement takes over and then removes it after the drain; switching back keeps the rolling revision for one drain after the recreated container is up. A service that had no container yet starts on a range port straight away.
+- A rejected revision is not retried on every poll; deploy again (or roll back) to try again.
+
+### Automatic rollback
+
+Add a top-level `x-tug` block to roll back failed deployments on its own:
+
+```yaml
+x-tug:
+  rollback: auto      # manual (default) | auto
+
+services:
+  # ...
+```
+
+When a deployment turns `failed` because an assigned node reported failed health for it, and the project has an earlier `succeeded` deployment, control creates a rollback deployment through the same path as `tug rollback` and records a `deployment.auto_rollback` event (`tug events` shows it). Both the rollback deployment and the event carry the actor `system:auto-rollback` (see [Access control](access-control.md#audit-trail)). If it cannot, it records `deployment.auto_rollback_skipped` with the reason (no earlier succeeded deployment, the project is locked, another deployment replaced the failed one). A failing rollback deployment is never rolled back again, so a bad rollback target cannot loop; for the same reason `rollback-` is reserved at the start of deployment versions and `deploy` rejects it. With rolling services the previous revision is still serving when the new one fails, so the rollback is only a state change and users see no downtime.
+
+## Volumes
+
+```yaml
+volumes:
+  cache-data:
+    x-tug:
+      type: local
+      persistent: true
+      placement:
+        nodes: [vps-c]
+```
+
+Control records volumes per project. A volume is included in a node's Compose file only when a service on that node uses it and, if the volume has `placement.nodes`, the node is listed. `tug volume remove` refuses to remove a persistent volume without `--delete-data`, and refuses any volume still referenced by a service in the current deployment. `--delete-data` queues a `docker volume rm` on the volume's placement node.
+
+Before starting workloads, the agent verifies that an existing Tug project network belongs to the project and that each existing non-external volume has the expected project and volume labels, driver and options. A mismatch stops reconciliation with an error instead of silently reusing the resource. Volume deletion also checks those ownership labels. Tug reserves the `tug.project`, `tug.volume` and `tug.restore` labels on managed volumes. External volumes are intentionally selected by name and are not checked as Tug-owned resources.
+
+Tug does not manage network file systems. To share a directory between nodes, mount it with the plain Compose `driver: local` and `driver_opts` against a server you run, such as a NAS, a managed export or your own NFS host, and install the NFS client (`nfs-common` on Debian and Ubuntu) on the nodes that use it:
+
+```yaml
+volumes:
+  uploads:
+    driver: local
+    driver_opts:
+      type: nfs
+      o: addr=10.0.0.10,rw,hard
+      device: ":/srv/uploads"
+```
+
+Every node that uses the volume mounts it itself, so the file server is a single point of failure that you operate. Prefer S3-compatible object storage for uploads when the application supports it.
+
+## Backups
+
+```yaml
+x-tug:
+  backup:
+    schedule: "0 3 * * *"
+    retention: { daily: 7, weekly: 4, monthly: 6 }
+
+services:
+  db:
+    image: mariadb:11.4
+    x-tug:
+      placement: { nodes: [app1] }
+      backup:
+        command: mariadb-dump -u root -p"$$MARIADB_ROOT_PASSWORD" --single-transaction --databases wordpress
+        filename: wordpress.sql
+        verify: "-- Dump completed"
+
+volumes:
+  uploads:
+    x-tug:
+      placement: { nodes: [app1] }
+      backup: "30 3 * * *"
+```
+
+`x-tug.backup` marks something to back up. Tug does not know what your services are, so there are only two kinds of target:
+
+- **A service**: Tug runs your `command` inside the running service container and stores its standard output. The container already has the right client, credentials in its environment and network access, so a database dump, `pg_dump`, `redis-cli --rdb -`, `tar` of a directory or `wp db export -` are all just commands. A string runs through `sh -c`; a list runs as written.
+- **A top-level volume**: Tug mounts the volume read-only and archives its files.
+
+The value is a cron string, `true` (a volume, with everything from the project defaults), `false`, or an object. The top-level `x-tug.backup` only holds defaults and enables nothing by itself.
+
+| Key | Applies to | Meaning | Default |
+| --- | --- | --- | --- |
+| `schedule` | both | Five-field cron in UTC, or `@hourly`, `@daily`, `@weekly`, `@monthly`, `@yearly` | the project default; an error if neither is set |
+| `retention` | both | restic keep policy with `last`, `hourly`, `daily`, `weekly`, `monthly`, `yearly` | the project default, else `daily: 7, weekly: 4, monthly: 6` |
+| `jitter` | both | Seconds of delay after the scheduled minute, 0 to 3600 | a stable per-target value under 15 minutes |
+| `timeout` | both | Hard limit for a run, `1m` to `24h` | `2h` |
+| `memory` | both | Memory limit of the restic helper container, `128m` to `16g` | `512m` |
+| `writers` | both | Services to hold during a restore | discovered from `depends_on`, host names in environment variables and volume mounts |
+| `command` | service | What to run in the container; its output is the backup | required |
+| `restore` | service | What to run in the container to load a backup back; the stored file arrives on its standard input | none (the service cannot be restored by Tug) |
+| `filename` | service | Name of the file inside the snapshot | `<service>.out` |
+| `verify` | service | Text that must appear in the last 4 KiB of output | none |
+| `exclude` | volume | restic exclude patterns | none |
+
+A run fails, and no snapshot is kept, when the command exits non-zero, writes nothing, or its output lacks the `verify` text (use the line your tool prints when it finishes, such as `-- Dump completed`, so a dump cut short is never mistaken for a good one).
+
+Deploy rejects a service target that is global, replicated or placed on several nodes, and a volume used on several nodes without `x-tug.placement.nodes`. Tug does not check that a volume backup and a command backup overlap or that a volume holds a live database: copying the files of a running database is rarely a consistent backup, so back databases up with a command. Recipes for common databases are in [Backups](backups.md#recipes).
+
+## Secrets in services
+
+Secrets are referenced by variable name and resolved by control at deploy time; the `@@tug:secret:NAME@@` placeholder you may see in a normalized file is internal and never written by hand. Reference secrets with `${NAME}` anywhere Compose interpolates, usually `environment`. `tug deploy` sets every project secret as a variable during `docker compose config`, so a secret wins over a local variable of the same name and `environment: [DB_PASSWORD]` passes it through. Only secrets a service references are released to its node, and a changed secret recreates exactly the services that use it. Secrets are substituted into the container environment; keep them out of `command` unless you accept them showing in `docker inspect`. Deploying a Compose file that references a secret the project does not have is rejected.
+
+## A database on the same node
+
+An app reaches a database by Compose's own DNS only when they are on the same node. A name injected for a service on another node points at that node's address, and the client has to use the published host port, so an unpublished database is not reachable across nodes. Local volumes are not copied between nodes; label placement can spread replicas across matching nodes.
+
+```yaml
+services:
+  db:
+    image: mariadb:11.4
+    command: ["--innodb-buffer-pool-size=48M"]
+    environment:
+      MARIADB_DATABASE: app
+      MARIADB_USER: app
+      MARIADB_PASSWORD: ${DB_PASSWORD}
+      MARIADB_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
+    volumes:
+      - db-data:/var/lib/mysql
+    x-tug:
+      placement:
+        nodes: [app]
+      health:
+        type: tcp
+        port: 3306
+
+  web:
+    image: nginx:1.27-alpine
+    depends_on: [db]
+    ports:
+      - "8080:80"
+    x-tug:
+      placement:
+        nodes: [app]
+      health:
+        type: http
+        port: 80
+      ingress:
+        host: app.example.com
+        port: 80
+        tls: auto
+
+volumes:
+  db-data:
+    x-tug:
+      type: local
+      persistent: true
+      placement:
+        nodes: [app]
+```
+
+Set `DB_PASSWORD` and `DB_ROOT_PASSWORD` with `tug secret set` before deploying. The MariaDB flag keeps the buffer pool small enough for a 1 GB host that is also running the agent. A `service_healthy` dependency can use the database service’s `x-tug.health`; without a Tug or Docker health check it verifies only that the container is running.
+
+## Not supported
+
+- `container_name` (Tug names containers `tug-<project>-<service>-<hash8>` so projects stay isolated).
+- `deploy` (Swarm semantics).
+
+## Supported keys
+
+`tug deploy` normalizes the file with `docker compose config` (Docker Compose v2 must be installed on the operator machine), and control accepts only these service keys: `image`, `build`, `command`, `entrypoint`, `environment`, `ports`, `expose`, `volumes` (named volumes and absolute bind mounts), `tmpfs`, `healthcheck`, `restart`, `labels`, `depends_on` (`service_started`, `service_healthy`), `extra_hosts`, `user`, `working_dir`, `hostname`, `init`, `read_only`, `shm_size`, `mem_limit`, `mem_reservation`, `cpus`, `pids_limit`, `cap_add`, `cap_drop`, `ulimits`, `logging`, `stop_grace_period`, `stop_signal` and `x-tug`. `pull_policy` is ignored because Tug decides when images change. Anything else is rejected with its key named, including `network_mode`, `privileged`, `devices`, `pid`, `ipc`, `container_name`, `env_file` (already merged by Compose), custom networks, and top-level `configs` and `secrets`. Top-level volumes accept `driver`, `driver_opts`, `labels`, `external`, `name` and `x-tug`. Containers use Tug's own network, volume and container names (`tug_<project>`, `tug_v<project-length>_<project>_<volume>`), so existing Compose-created volumes are not reused. If a matching owned volume exists under an earlier Tug name and its new-name volume does not, Tug refuses to create an empty replacement or restart the workload. The older volume remains on the node; recover or copy its data before deploying.

package/docs/configuration.md

@@ -0,0 +1,52 @@
+# Configuration
+
+## CLI
+
+CLI options override environment variables, which override the config files and documented fallback. The config is a YAML or JSON file that is safe to commit: it holds the control URL, default project, defaults for a few command options and the servers you install. The operator credential is never read from it. See [Config file](cli.md#config-file) for the keys and how they map to flags, and [Nodes](nodes.md#declaring-nodes-in-the-config-file) for the `nodes` section.
+
+| Variable | Purpose |
+|---|---|
+| `TUG_CONFIG` | Path to the CLI config file (default: `tug.yml`, `tug.yaml`, `tug.json` or `.tug.json` in the current directory). |
+| `TUG_CONTROL_URL` | Connect target for the control API (an `https://` URL; for example a tunnel). The server is verified by its generated control name regardless; overrides the config file and the credential's default address. |
+| `TUG_PROJECT` | Default project; overrides the config file. |
+| `TUG_CREDENTIAL` | Encoded operator credential (for CI). Conflicts with `TUG_CREDENTIAL_FILE`; overrides the overlay. |
+| `TUG_CREDENTIAL_FILE` | Path of a credential file; overrides the `credential` path in the untracked overlay. |
+
+## Control plane
+
+| Variable | Default | Purpose |
+|---|---|---|
+| `PORT` | `8443` | The single mTLS port. |
+| `TUG_CONTROL_URL` | required | Credential-free `https://` origin clients connect to by default, including the port. It is written into credentials and its host is added to the server certificate, but clients verify the generated control name, not this address. |
+| `TUG_EXTRA_SANS` | unset | Comma-separated extra DNS names or IPs for the server certificate. |
+| `TUG_RENEW_MIN_INTERVAL_MS` | `3600000` | Minimum time between certificate renewals per node. |
+| `TUG_DATA_DIR` | `./data` | SQLite database, master key, `pki/` (CA and server certificate), image archives, build contexts and backups. |
+| `TUG_UPLOAD_DIR` | `<data dir>/uploads` | Temporary storage for uploaded source contexts and local image archives. |
+| `TUG_MAX_IMAGE_BYTES` | `5368709120` | Maximum bytes per uploaded file (source context or compressed image); expanded archives are limited to 20 GiB and 100,000 entries. |
+| `TUG_LOCK_TTL_MS` | `600000` | Lease lifetime of per-project operation locks; active builds renew their lease. |
+| `TUG_STALE_SECONDS` | `60` | Seconds without an authenticated work fetch, status report or wake wait before a node is marked `stale`. |
+| `TUG_EVENT_RETENTION_DAYS` | `30` | Events older than this many days are deleted; `0` keeps events regardless of age. Pruning runs at startup and then at most once an hour while agents report events. |
+| `TUG_EVENT_MAX_ROWS` | `100000` | Only this many of the newest events are kept; `0` removes the limit. A value that is not a whole number falls back to the default. |
+| `TUG_CONTROL_BACKUP_DIR` | `<data dir>/control-backups` | Control backup archives. |
+| `TUG_CONTROL_RESTORE_DIR` | `<data dir>/control-restore` | Scratch directory for restores. |
+| `TUG_SERVICE_NAMES` | `true` | Publish health-aware `serviceEndpoints` so agents inject cross-node `extra_hosts` using each node's `address`. Set `false` to opt out. See [Networking](networking.md#layer-a-cross-node-service-names). |
+| `TUG_ACME_EMAIL` | unset | Email for ACME registration in rendered Caddy configs. |
+
+## Agent
+
+| Variable | Default | Purpose |
+|---|---|---|
+| `TUG_CONTROL_URL` | unset | Connect-only override of the control address (for example a private endpoint); the server is verified by its generated control name. |
+| `TUG_JOIN_KEY` / `TUG_JOIN_KEY_FILE` | unset | Join key (or a `0600` file holding it) used only when the node has no identity yet; ignored once `agent.json` exists. |
+| `TUG_NODE_NAME` | hostname | Node name when not bound by the enrollment key. |
+| `TUG_NODE_ADDRESS` | unset | Private inter-node address reported at enrollment; also used for service endpoints, ingress upstreams and published-port binding. |
+| `TUG_STATE_DIR` | `/var/lib/tug` | Identity (`agent.json`, `agent-pending.json`), applied revisions, managed-image identities, temporary image downloads, ingress config and generated project files. |
+| `TUG_POLL_INTERVAL_MS` | `5000` | Maximum quiet interval between reconciliations. A WebSocket or long-poll wake can start one sooner. |
+| `TUG_APPLY` | `true` | Set `false` to render files and report without touching Docker. |
+| `TUG_ROLLOUT_PORTS` | `20000-20999` | Host port range (`LOW-HIGH`) for revisions of rolling services; each running or draining revision uses one port. |
+| `TUG_CADDY_IMAGE` | `caddy:2-alpine` | Image for the managed Caddy container on ingress nodes. |
+| `NODE_EXTRA_CA_CERTS` | unset | Extra CA for a control plane with a private certificate. |
+
+## Entrypoint
+
+`TUG_ROLE=control|agent` selects the role when `src/main.ts` (`dist/main.js` in the image, whose entrypoint is `node dist/main.js`) is started without an argument. An argument always wins over the variable. The runtime image has no default command, so `docker run -e TUG_ROLE=agent tug` starts the agent and a bare `docker run tug` prints the usage message; the installers and the compose files pass the role as an argument.

package/docs/control-plane.md

@@ -0,0 +1,112 @@
+# Control plane
+
+## Install over SSH
+
+```bash
+tug control setup root@control.example.com
+```
+
+The address clients use by default comes from `controlUrl` in the CLI config file (`--config <file>` selects one), including any overlay override. Without it, the address is the SSH target's host plus the published port (`https://control.example.com:8443`), which `control setup` prints. Pass `--control-url` (or set `TUG_CONTROL_URL`) to override either default, for example when you install through an SSH alias or a private address but clients reach control elsewhere. The credential URL is never used as an installation address. A configured URL keeps its public port even when `--port` publishes a different Docker host port. It is only an address: clients verify the generated control name, not this URL. `control setup` runs SSH preflight checks on the host (Docker, Docker Compose, writable Tug paths, at least 2048 MB free under `/var/lib`, the mTLS port free), then starts the `control` container from `src/templates/control-compose.yml`, publishing one port (default `8443`). Add `--install-docker` to install Docker Engine and Compose first on a supported Debian, Ubuntu, Fedora and RHEL-compatible host.
+
+On a fresh install it then issues the **root** credential on the host (`docker exec ... control issue-credential --name root --role admin` over SSH) and imports it straight into the local config as a `0600` file (`--credential-out <file>` optionally writes an extra copy; there is no default file); it never lands in container logs or a permanent file on control. Upgrading by running `control setup` again preserves credentials and never reissues root. If the root credential is lost, `tug control setup <ssh-target> --control-url https://control.example.com:8443 --recover-root` replaces only root's key without the API, closes its sessions within a second and audits the recovery; other admins are unaffected.
+
+The control container runs the published image `ghcr.io/wpelevator/tug:<CLI version>` (the installed CLI's own package version, so control and the CLI match). Pass `--image <tag>` to use another tag, or `--build-local-image` to build the image from the current checkout and copy it to the host over SSH with `docker save` / `scp` / `docker load` (tagged `tug:local` unless `--image` is given); the latter is for development and needs the repository's Dockerfile in the working directory. See [Releases](releases.md). Running `control setup` again over an existing control upgrades it in place: the port preflight accepts the port when the existing `tug-control-control-1` container holds it, and still fails when anything else does.
+
+Built images always travel as compressed archives over the authenticated control connection. Published `image:` services pull directly from their upstream registry by the digest resolved on control before deployment publication.
+
+`control setup` (also available as `tug setup <ssh-target>`) imports the root credential into the local config automatically, the same way `tug init` does, so no separate step is needed. In a terminal it shows a plan and asks for confirmation (`-y, --yes` skips it) and offers to install Docker when it is missing. `--no-import` skips the import. To import a credential from elsewhere (for example one issued on another machine), run:
+
+```bash
+tug init root.credential.json
+```
+
+`tug init` copies the credential (a file, or `-` for stdin) into a `0600` file under `~/.config/tug/credentials/` and points the git-ignored `tug.local.yml` at it. CI passes `TUG_CREDENTIAL` (the encoded credential) or `TUG_CREDENTIAL_FILE`; precedence is `--credential <file>`, then the environment (setting both is an error), then the overlay's `credential` path. Tracked configuration may not name a credential.
+
+A release publishes the npm package a few minutes before its container image. A freshly updated CLI therefore fails `tug setup` and `tug node add` before touching the host, with a message that the image is not published yet; retry shortly, or pass `--image ghcr.io/wpelevator/tug:latest`.
+
+## Authentication and PKI
+
+Control listens on **one** TLS 1.3 port that requires a client certificate from every peer. There is no plain HTTP listener, no unauthenticated route (not even a health check: monitor TCP reachability separately from authenticated readiness) and no bearer token. Operators, CI and nodes all use certificates signed by control's private CA; see [Access control](access-control.md) for identity classes, roles and key-based authorization.
+
+- **CA.** A self-signed P-256 CA (10-year lifetime) is created only for a fresh installation and kept at `<data dir>/pki/ca.key` (`0600`) and `ca.crt`. Missing, corrupt or mismatched CA material alongside existing state is a startup error. CA rotation is out of scope; replacing the CA means redistributing trust and credentials.
+- **Control name.** Control has a generated name, `control-<first 12 hex of the CA key hash>.tug`, derived from its CA. It is stable for the life of the CA, needs no configuration and survives backup and restore with `pki/`. Clients recompute it from the CA they pin and verify the server against it, so verification never depends on the address they dial.
+- **Server certificate.** SANs are the control name, the host of `TUG_CONTROL_URL` (required, a credential-free `https://` origin including the published port; the default address written into credentials) and optional `TUG_EXTRA_SANS` (comma-separated DNS names and IPs). The server certificate lasts 90 days, is renewed at two thirds of its lifetime while running, and the TLS context reloads without a restart. Control warns when the CA nears expiry.
+- **Profiles.** Control assigns every certificate: it verifies CSR signatures and P-256 keys and never copies requester-supplied subjects, SANs, validity or extensions. Leaves are `CA=false` with one assigned identity URI, back-dated five minutes and capped by the CA's validity.
+- **Connections.** Chain validation rejects missing or foreign certificates during the handshake, before any HTTP is parsed. TLS session resumption is disabled and any reused session is closed, so every connection presents its certificate. A post-handshake gate then checks the database, and every request and WebSocket upgrade checks it again. Disable, replace, remove, revoke, root recovery and certificate expiry close the affected sockets (API-originated changes immediately, host-side changes within a second).
+- **Clients.** The CLI and agent pin the credential's CA and verify the control name derived from the CA. `--control-url` / `--agent-control-url` change only the TCP connect target (for example `127.0.0.1:18082` through an SSH tunnel), and the credential's stored URL is just the default target. Moving control to a new IP or DNS name therefore invalidates no credential; see [Moving the control plane](#moving-the-control-plane).
+
+Deployment boundaries: TCP passthrough works. TLS termination ahead of control does not (it breaks the client certificate), and TLS interception fails certificate validation. Browsers need explicit CA trust and an imported client credential; browser setup is out of scope. Control's mutual TLS does not encrypt workload traffic between nodes; that traffic uses the operator's network ([Networking](networking.md#bring-your-own-network)).
+
+### Backups and authorization history
+
+`control backup` includes `pki/` (except the regenerable server certificate) and the key bindings. Restore validates the CA before changing live state, closes live sessions and requires a restart. Because authorization follows keys, a snapshot older than a certificate renewal does not lock a node out. An older snapshot still forgets nodes and operators created after it, restores keys that were replaced afterwards, and revives principals disabled or revoked afterwards. After an older restore: recover root on the host if needed (`--recover-root`), reapply revocations and disables, run `tug node replace` for nodes whose key changed and `tug node add` for nodes created after the snapshot (`tug node replace --all` after a CA loss).
+
+## Image delivery without a registry
+
+Remote builds save immutable images under `<data dir>/images/<build ID>.tar.gz`. Agents download only the images assigned to them through `GET /agent/v1/images/:buildId`, verify the compressed SHA-256 and byte count, validate the saved image metadata and load the archive. The agent loads those artifacts without a registry pull. No registry certificate, daemon trust change or inbound connection to a node is needed; downloads use the same mTLS connection as every other agent request.
+
+`tug deploy --build-local` builds application images on the operator machine and uploads archives instead of source. Control validates and imports them, gives them immutable tags and serves them to assigned agents. Builds are single-platform Linux images (amd64 or arm64); use a Docker builder matching target nodes. Agents reject an incompatible image platform before starting it. This flag is distinct from the installers' development-only `--build-local-image`, which builds the Tug runtime itself.
+
+All published desired-state history is retained, including rollback and lagging-node references. Its image archives remain available until you run `tug image prune`, which deletes the archives of everything except the newest `--keep N` (default 5) deployments of each project, the current deployment and any version a node still reports; `--dry-run` lists what would go. The desired states stay in history, but rolling back to one whose archive was pruned is refused and the service must be deployed again. Run it to reclaim disk, since each source build adds a full compressed image under `images/`. Cleanup removes failed unpublished builds and orphan archives. Agents prune only their recorded, unused Tug images absent from their authorized retained set and never run global Docker prune. Disk usage therefore grows with retained successful builds. Each transfer sends a complete compressed image, with no layer reuse.
+
+## Published images and private registries
+
+Control resolves each distinct published image tag once per deployment through the registry HTTPS manifest API and stores its SHA-256 digest in both desired service state and canonical Compose. Multi-platform images retain their complete index digest, allowing each node to select its architecture from that same immutable index. An explicit SHA-256 pin is validated and kept without a network lookup. If resolution fails, the deployment fails before replacing current desired state. Rollback and scaling reuse the stored pins without resolving tags again. Registries must retain pinned manifests and layers for rollback and fresh-node pulls; Tug does not mirror published images.
+
+Private registry credentials live in the encrypted project secret `TUG_REGISTRY_AUTH`, a JSON object keyed by registry host (including a port when applicable):
+
+```json
+{"ghcr.io":{"username":"ci-user","password":"pull-token"},"docker.io":{"username":"hub-user","password":"hub-token"}}
+```
+
+Store it using `tug secret set TUG_REGISTRY_AUTH --project site` with the value on stdin. Use `docker.io` for Docker Hub, including short image names. HTTPS registries with Basic or Bearer authentication are supported; credential helpers and plaintext registries are not. Use credentials with pull access only. Control decrypts the secret for resolution; assigned agents receive only credentials for their published image hosts through the authenticated secrets endpoint and pass them directly as Engine API `X-Registry-Auth`. Credentials stay out of desired state, container environments and Docker login files; referencing the reserved secret in a service definition is rejected. Secret updates apply to future pull requests, including retries and rollback; keep access to retained image digests when rotating credentials.
+
+Tug issues one node identity (key and certificate) per node. It does not push built images, manage registry users or log agents into registries. Teams building images in CI can push them themselves and deploy with `image:`.
+
+## Backup and restore
+
+`tug control backup` takes a consistent SQLite snapshot (including operator credential records) and writes a `0600` `.tar.gz` (in a `0700` directory) with `tug.db`, `master.key`, `pki/` (the private CA), retained `images/` archives (leave them out with `tug control backup --no-images`; deploy again to rebuild them after a restore) and metadata to `TUG_CONTROL_BACKUP_DIR` (default `<data dir>/control backups`). `tug control restore <artifactRef> --confirm` overwrites the database, master key and image store from such an archive; restart control afterwards. SQLite snapshot creation and restore integrity checking run synchronously and pause control API traffic, WebSocket handling and timers. Duration grows with data size and storage speed; the synthetic benchmark and reference measurements are in [Testing](testing.md#sqlite-contract-and-blocking-measurements). Take larger backups outside latency-sensitive rollout windows and keep project lease durations above expected blocking time. Restore validates through a read-only connection before replacing live data, then rejects database-dependent requests with HTTP 503 until restart. This is the control-plane backup: it covers disaster recovery of the control plane only and does not capture workload volume contents. Back up project data (volumes, databases) with `x-tug.backup` as described in [Backups](backups.md), which also shows a scheduled `control backup`.
+
+Running containers keep serving while control is down; agents just cannot pick up new desired state until it is back.
+
+### Recover control on another host
+
+For a planned move of the whole installation, including workloads, see [Migrating to new servers](migration.md).
+
+This procedure was exercised with the published 0.9.0 release on two VPSes; see [Real-host evaluation](real-host-evaluation.md#recovery-drill-with-the-published-090-release-2-november-2026) for the exact scope and limitations. Restore the workload data separately using its project backups. Keep the control backup encrypted off-host and keep the original operator credential and backup repository passwords separately; the control archive contains the master key and private CA.
+
+1. Stop or isolate the old control plane before directing agents to the replacement. Two active control planes with divergent state are not a supported failover setup. Running workload containers continue serving while control is unavailable.
+2. Install the matching Tug release on the replacement host with `tug control setup`, writing its temporary bootstrap credential to a private file. This initializes a new control and CA solely to authorize the restore.
+3. Copy the decrypted control archive into the replacement control data directory with mode `0600`. The standard installer mounts host `/var/lib/tug-control` as `/data`: for example, a host file `/var/lib/tug-control/restore-input.tar.gz` is addressed to the API as `/data/restore-input.tar.gz`. The archive path is read on the control host; the CLI does not upload a local file.
+4. Run `tug control restore /data/restore-input.tar.gz --confirm --credential <bootstrap-credential>`, then restart the control container. Restore replaces the bootstrap CA, credentials, state and image store; the temporary credential no longer authenticates after restart.
+5. Connect with the original operator credential and use `--control-url https://<replacement-host>:8443` if the address changed. Verify the recovered projects, deployments and backup catalog. Agents that still point to the old address must be repointed; the tested 0.9.0 path was `tug node replace <node> <ssh-target> --control-url https://<replacement-host>:8443 --agent-control-url https://<replacement-host>:8443 --credential <original-credential>`. Verify that each node returns online and that the site serves the expected data.
+
+A snapshot older than later node or operator changes does not contain those changes; follow the identity and revocation recovery notes in [Authentication and PKI](#authentication-and-pki). This sequence does not replace a separate, tested backup of workload volumes or databases.
+
+## Moving the control plane
+
+Because clients verify the generated control name, the address is only a default:
+
+1. Move or re-address the host and set the new `TUG_CONTROL_URL` on control (or rerun `control setup` with the new `--control-url`). Control adds the new address to its certificate on restart.
+2. Until credentials are refreshed, point clients at the new address: `tug --control-url https://NEW:8443 ...` or `TUG_CONTROL_URL`, and for an agent set `TUG_CONTROL_URL` in `/etc/tug/agent.env`.
+3. Optionally reissue credentials so they carry the new default: `control setup --recover-root` for root, `credential create` for others, `node replace` for nodes.
+
+No credential becomes invalid because the address changed. The control name only changes if the CA itself is replaced, which requires redistributing every credential.
+
+## Lost or compromised something
+
+| Situation | What to do |
+| --- | --- |
+| Lost the root credential | `tug control setup <ssh-target> --recover-root` |
+| A CI credential leaked | `tug credential revoke <name>`; live sessions close within a second |
+| A node is suspect | `tug node disable <node>` now, then `tug node replace` |
+| A node's identity is corrupt or expired | `tug node repair` or `tug node replace` |
+| Restored an older control backup | Recover root if needed, reapply revocations, then `node replace --all` ([Control plane](control-plane.md#backups-and-authorization-history)) |
+
+## Access control
+
+Besides the `root` admin credential, control accepts named operator credentials with `read`, `deploy` or `admin` roles, optionally limited to projects, for CI and teammates. Every deployment and control event records the acting credential name. See [Access control](access-control.md) for the endpoint table, the audit trail and a GitHub Actions deploy example. Control stores each credential's name, role, scope and public-key hash (never a private key); these records travel with control backups.
+
+## Configuration
+
+See [Configuration](configuration.md#control-plane) for control environment variables.

package/docs/development.md

@@ -0,0 +1,140 @@
+# Development
+
+The source is TypeScript. Development and tests run the `.ts` files directly through Node's built-in type stripping, so there is no build step, but it needs Node >= 22.18 (`engines.node`); development and CI use Node 24 (`.nvmrc`). SQLite uses built-in `node:sqlite`; installation requires no SQLite npm addon, Python or C++ compiler. The module is marked active development on Node 22.18.0 and can emit an experimental warning on supported Node releases; warnings are not globally suppressed. Compilation to `dist/` happens when building the Docker image and when packing the npm package (`prepack`).
+
+```bash
+npm install
+npm test
+npm run typecheck
+npm run lint
+npm run format
+npm run build
+npm run test:smoke:dist
+```
+
+Code style is enforced by Prettier (`.prettierrc.json`: 100 columns, single quotes, trailing commas; indentation comes from `.editorconfig`). `npm run lint` runs ESLint and then `prettier --check .`, and CI runs it, so an unformatted file fails the build. `npm run format` rewrites files with Prettier and then applies ESLint's autofixes. `eslint-config-prettier` switches off the ESLint rules that would fight the formatter, so ESLint only reports correctness problems. The one style rule it adds is `max-depth` (5) with `no-else-return`: early `return` guards are the house style, but a loop or state machine whose body needs several `continue`/`break` exits, or nests deeper than that, should be split into a function per item or per state that returns early instead (see `serviceTarget` in `src/backup-config.ts` and the `advance*` handlers in `src/backup-restore.ts`). `no-continue` is deliberately off: a one-line `continue` guard at the top of a short loop is fine. Prettier covers TypeScript, JavaScript and JSON; Markdown, YAML, the install templates and generated files are listed in `.prettierignore`. Prettier moves a trailing `// eslint-disable-line` comment when it rewraps a statement, so put suppressions on their own line above the code they cover (`// eslint-disable-next-line`).
+
+`npm test` runs `node --test 'src/**/*.test.ts'`. `npm run typecheck` runs `tsc --noEmit` over `src/` and `tests/` with `strict` on. `npm run build` compiles `src/` to `dist/` (JavaScript plus `.d.ts`) and copies `src/templates` to `dist/templates`. `npm run test:smoke:dist` builds and checks that `control setup` and `node add` render their templates from `dist/`.
+
+Import specifiers in the source use the `.ts` extension (`import { x } from './util.ts'`) because Node's type stripping does not remap `.js` to `.ts`; `tsc` rewrites them to `.js` in `dist/` (`rewriteRelativeImportExtensions`). Only erasable TypeScript syntax is allowed (`erasableSyntaxOnly`: no enums or constructor parameter properties).
+
+Unit tests sit next to the source files as `src/*.test.ts`. The agent integration test lives in `tests/integration/`. [Testing](testing.md) explains how the test layers fit together and what each one is expected to catch.
+
+## Docker image
+
+The runtime Dockerfile is multi-stage: a build stage runs `npm ci && npm run build`, and the final image ships `dist/` plus production dependencies only, with entrypoint `node dist/main.js`. `tsc` does not copy `src/templates/*`, so `npm run build` copies them next to the emitted `dist/cli.js`; the CLI resolves them relative to the running file, so `node src/cli.ts` and `node dist/cli.js` both work.
+
+The runtime image ships the Docker client (control uses it to load, tag and remove image archives through the mounted socket) and the compiled `tug-runtime` entry (`dist/main.js`). It deliberately has no Compose plugin: agents drive the Docker Engine API, and the control-plane host runs the control container with the host's own Compose (`control setup` checks for it). It is based on `node:24-alpine` (musl), which keeps the image about half the size of a Debian one; the Docker client comes from Alpine's `docker-cli` package, so the Docker daemon, containerd and runc are not installed.
+
+Run `npm run test:smoke:image` to build the actual shipped runtime image and execute its Docker client and Node binaries, the compiled control/agent database initialization (a writable snapshot and read-only validation) and `tools/scripts/verify-mtls-runtime.js`, which issues certificates with `@peculiar/x509`, serves mTLS with the production TLS options and calls it through the undici transport, including the loopback connect-target override that still verifies the generated control name. Nothing from the host socket or CLI plugins is mounted. To test an already-built image, set `TUG_RUNTIME_IMAGE=<tag>`; to check another Dockerfile, set `TUG_RUNTIME_DOCKERFILE=<path>`. CI runs this packaging smoke before the integration test, so the fake-VPS image cannot mask missing runtime tools.
+
+## Continuous integration and releases
+
+`.github/workflows/ci.yml` runs on pushes to `main` and on pull requests: typecheck and lint on Node 24 (the version in `.nvmrc`), unit tests and the dist smoke test on the exact minimum Node 22.18.0 and current Node 22 and 24, plus the Docker integration tests (fake VPS containers, registry-free image delivery, plain-address cross-node networking and rolling rollouts) as separate jobs. `.github/workflows/release.yml` versions and publishes; see [Releases](releases.md). Every user-facing change needs a changeset (`npm run changeset`).
+
+### CI caching
+
+The `.github/actions/setup` composite action installs Node.js with the npm download cache (keyed on the lockfile) and runs `npm ci`; every job uses it. The integration jobs prebuild their images with `.github/actions/build-images` (Buildx and the GitHub Actions layer cache, scopes `runtime` and `fake-vps`) and set `TUG_COMPOSE_BUILD=` so the integration scripts run `docker compose up` (on the CI host, to start the test stack) without `--build`; the Compose files tag those images `tug-runtime:integration` and `tug-fake-vps:integration`. Locally the variable is unset and the scripts keep passing `--build`. Only pushes to `main` write the cache, because a pull request can read the default branch's cache but not another pull request's. The release workflow's `edge` job exports the `runtime` scope and the versioned `image` job reads it. The Dockerfiles pin the Node.js major (`node:24-*`), so an upstream LTS bump cannot silently invalidate the cache or change the shipped runtime.
+
+Published images are `ghcr.io/wpelevator/tug`. To test an unreleased CLI against a checkout, use `--build-local-image` on `control setup` and `node add` (builds `tug:local` from the working directory and copies it over SSH). The dev Compose files (`docker-compose*.yml`) and the integration tests always build the image from the checkout with `build: .` and never pull from GHCR.
+
+## Compose stack
+
+`docker-compose.yml` defines:
+
+- `control`: the control plane on its single mTLS port `8443`, with SQLite, the master key, the private CA and saved image archives in a named volume, plus the Docker socket for builds. It requires `TUG_CONTROL_URL`, the HTTPS origin clients connect to.
+- `agent`: an optional profile service (applying nothing, `TUG_APPLY=false`); run it with `--profile agent` once you have an enrollment credential for it.
+
+```bash
+export TUG_CONTROL_URL=https://localhost:8443
+docker compose up --build control
+# issue the root operator credential on the control-plane host (fresh state only):
+docker compose exec control node dist/main.js control issue-credential --name root --role admin > tug-root.credential.json
+node src/cli.ts init tug-root.credential.json
+# prepare a node, then start the agent with the join key it prints:
+node src/cli.ts node enroll --name dev-node --agent-control-url https://127.0.0.1:8443
+TUG_JOIN_KEY=... docker compose --profile agent up agent
+```
+
+Control does not print or store a credential on startup; `issue-credential` writes the credential to stdout and keeps no private key. The agent keeps its identity in the `agent-state` volume, so it ignores the join key after the first start.
+
+## Host mode
+
+```bash
+export TUG_CONTROL_URL=https://localhost:8443
+npm run start:control
+node src/main.ts control issue-credential --name root --role admin > tug-root.credential.json
+node src/cli.ts init tug-root.credential.json
+```
+
+Host mode shares `./data` (or `TUG_DATA_DIR`) between control and `issue-credential`. Register a development node the way `node add` does, with the node making its own key:
+
+```bash
+export TUG_STATE_DIR=$PWD/data/agent TUG_NODE_NAME=dev-node
+node src/main.ts agent identity > dev-node.csr
+node src/cli.ts node register dev-node --csr dev-node.csr --address 127.0.0.1 | node src/main.ts agent install-cert
+```
+
+Start an agent that reports status without applying Docker changes:
+
+```bash
+export TUG_APPLY=false
+npm run start:agent
+```
+
+Deploy a Compose project without uploading a build context:
+
+```bash
+node src/cli.ts deploy -f docker-compose.yml
+node src/cli.ts status
+```
+
+Services with `build:` upload the current directory and control builds and stores compressed saved images for agents. `deploy --build-local` instead builds on the operator daemon and uploads archives; the platform must match target nodes. Published-image-only applications upload no context and control pins their registry digests before publication.
+
+## Integration test
+
+`docker-compose.sample.yml` is a three-service sample app with explicit placement: `web` on `vps-a`, `worker` on `vps-b`, `cache` on `vps-c`.
+
+`docker-compose.integration.yml` starts a control plane and three fake VPSes. On macOS/Docker Desktop each fake VPS is a privileged Linux container with its own Docker daemon and Tug agent, so agents apply containers into isolated Docker environments through their own Engine API rather than the host daemon.
+
+```bash
+npm run test:integration
+```
+
+The test starts control, runs a provisioner that issues the root credential and each fake VPS's node identity (the `agent identity` / `agent install-cert` flow, written to the shared `/tokens` volume), sets a sample secret, deploys `docker-compose.sample.yml`, waits for all three agents to report the desired revision, verifies published images run by the digest pinned on control, and tears the stack down. Set `KEEP_INTEGRATION_STACK=true` to keep containers and volumes for inspection. The integration network defaults to subnet `10.89.240.0/24`; override it with `TUG_TEST_SUBNET` if that overlaps with local Docker or VPN networks. For concurrent runs, set a unique `COMPOSE_PROJECT_NAME`, `TUG_TEST_PORT` and matching `TUG_CONTROL_URL` (an `https://` connect target; the credential still verifies `https://control:8443`).
+
+### Registry-free image integration
+
+```bash
+npm run test:integration:no-registry
+```
+
+The Node-driven `tools/scripts/integration-no-registry.js` uses `docker-compose.integration-no-registry.yml` with isolated control and node Docker daemons and no registry service or registry configuration. It verifies remote builds with changed application contents under a reused display version, rollback, agent restart, local application builds and backup/restore with a fresh image download. Separate daemons ensure a node cannot accidentally reuse control's already-built image instead of downloading the archive. The API port defaults to `18081` (`TUG_IMAGE_TEST_PORT` overrides it); use a unique `COMPOSE_PROJECT_NAME` when running alongside other tests. `KEEP_INTEGRATION_STACK=true` preserves the stack for inspection. These container tests do not establish real SSH installation, public ACME certificate issuance or a real-host private-network/NAT baseline.
+
+### Rolling rollout integration test
+
+```bash
+npm run test:integration:rollout
+```
+
+Layers `docker-compose.integration-rollout.yml` over the same stack with a single fake VPS (`vps-a`) that runs the app and is the Caddy ingress node, then runs `tools/scripts/integration-rollout.js`. It deploys v1 of a rolling nginx service next to an unchanged recreated service, keeps requesting the app through Caddy from inside the VPS every ~50 ms while v2 rolls out (zero failed requests, v1 then v2 observed, the old container removed after the drain, the recreated service's container id unchanged), then deploys v3 whose health check fails and asserts that v2 kept serving the whole time, v3 never received a request, control recorded `deployment.auto_rollback` and the rollback deployment succeeded. It uses the compose project `tug-it-rollout` and the same host ports as the other integration tests, so run them one at a time; `KEEP_INTEGRATION_STACK=true` keeps the stack.
+
+## Test VPS with Lima
+
+`template-vps.yml` is a [Lima](https://lima-vm.io/) template for an Ubuntu VM with Docker and Node.js installed, useful as a real SSH target for `control setup` and `node add`:
+
+```bash
+limactl start --name=vps template-vps.yml
+```
+
+### Networking integration test
+
+```bash
+npm run test:integration:network
+```
+
+Layers `docker-compose.integration-network.yml` over the same stack with plain numeric node `address` values and `vps-c` marked as an ingress node, deploys `docker-compose.network-sample.yml` (`web` on `vps-a`, `client` on `vps-b`) and runs `tools/scripts/integration-verify-network.js`: a container on `vps-b` fetches `http://web:8080` by its cross-node service name, Caddy on `vps-c` serves `Host: app.test` from `vps-a`, and generated published-port bindings use the intended private node addresses rather than every interface. It uses the compose project `tug-it-net` and the same ports as the plain integration test, so run them one at a time.
+
+## Running the agent outside Docker
+
+Run the agent in a container, as the installers do (`--network host`, the Docker socket mounted). An agent started as a plain host process, for example `node src/main.ts agent` on macOS, can create containers but cannot reach their IP addresses, so every TCP or HTTP health check times out and the deploy fails with a message such as `tcp timeout 172.21.0.2:3306` that does not mention where the agent runs. On Docker Desktop the Docker daemon lives in a VM, so container addresses are only reachable from inside it. When the agent is not in a container, a failed TCP or HTTP health check says so in its error and points at `docker run`; `doctor` does not check it separately.