@wpelevator/tug 0.9.0 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -10,14 +10,15 @@ Deploy Docker Compose applications to one or more ordinary Linux VPSes through a
10
10
  - Nodes join over SSH (`node-install`) or enroll themselves with a single `docker run` after `node-prepare-enrollment`, each with its own certificate and an immutable node ID. Agents only dial out, so nodes behind NAT work.
11
11
  - Agents pull desired state from control, reconcile containers through the Docker Engine API, run TCP/HTTP/Docker health checks and report status. An outbound WebSocket wakes them as soon as control queues work and carries live `exec`, followed logs, `ps` and `inspect` sessions, with long-poll and periodic-poll fallbacks for ordinary work.
12
12
  - `x-tug` placement of services and volumes on nodes, with immutable desired-state records per deployment.
13
- - Build application images on control from uploaded source, or locally with `deploy --build-local`. Agents download saved images directly from control. Published images are pinned to registry digests at deployment creation.
13
+ - Build application images on control from uploaded source, or locally with `deploy --build-local`. Both use a filtered source snapshot that excludes known Tug credentials; operators must still exclude other secrets. Agents download saved images directly from control. Published images are pinned to registry digests at deployment creation.
14
14
  - Secrets encrypted at rest, released only to the nodes whose services reference them, with targeted service recreation when only a secret changes.
15
15
  - Deployment convergence, per-project operation locks, cancellation, rollback, per-service rollout events and log fetching through the agents.
16
16
  - Opt-in zero-downtime rolling rollouts for ingress-fronted services (`x-tug.rollout`) and automatic rollback of failed deployments (`x-tug.rollback: auto`). See [Compose extensions](docs/compose.md#rolling-rollouts).
17
17
  - Private published images use encrypted project credentials for digest resolution and Engine API pulls.
18
- - Optional WireGuard mesh between nodes, health-aware cross-node service names (hosts-file entries injected into containers) and a managed Caddy ingress with automatic HTTPS, each opt-in. See [Networking](docs/networking.md).
19
- - Private-by-default published ports: generated Compose binds them to the overlay IP, loopback or the node address, never every interface unless you write `0.0.0.0`. See [Networking](docs/networking.md#published-ports-private-by-default).
18
+ - Health-aware cross-node service names (hosts-file entries injected into containers), on by default with a `TUG_SERVICE_NAMES=false` opt-out, and an optional managed Caddy ingress with automatic HTTPS. Both use each node's private `address` on your existing network. See [Networking](docs/networking.md).
19
+ - Private-by-default published ports: agent-generated container definitions bind them to the node address or loopback, never every interface unless you write `0.0.0.0`. Tug does not encrypt node-to-node workload traffic; your network owns reachability, firewalling and encryption. See [Networking](docs/networking.md#published-ports-private-by-default).
20
20
  - Workload backups of service commands and volumes with restic, a separate control-plane backup and restore, and `tug doctor` diagnostics for certificate expiry, node connectivity and clock skew.
21
+ - `tug status --human` summarizes deployments, node reports and failures; the default JSON output remains available for automation.
21
22
 
22
23
  ## How it fits together
23
24
 
@@ -38,17 +39,19 @@ Deploy Docker Compose applications to one or more ordinary Linux VPSes through a
38
39
  | |
39
40
  +------+------------+ +------+------------------+
40
41
  | Node app1 | | Node app2 (ingress node)|
41
- | Agent |<->| Agent |
42
+ | Agent | | Agent |
42
43
  | Docker Engine | | Docker Engine |
43
44
  | services | | services |
44
45
  | | | managed Caddy ingress |
45
46
  +-------------------+ +-------------------------+
46
- <-> optional WireGuard mesh between nodes
47
+ \_________________________/
48
+ workload traffic between node addresses
49
+ over the operator's private network
47
50
 
48
51
  SSH is used only by the CLI to install the control plane and nodes.
49
52
  ```
50
53
 
51
- Operators and CI talk to the control API. The control plane stores desired state and builds images; each agent dials out to fetch the work for its node, applies it through the Docker Engine API and reports health back, so nodes behind NAT need no inbound ports. SSH is used only to install the control plane and its nodes. The optional WireGuard mesh and Caddy ingress are described in [Networking](docs/networking.md), and every flow in detail in [Architecture](docs/architecture.md).
54
+ Operators and CI talk to the control API. The control plane stores desired state and builds images; each agent dials out to fetch the work for its node, applies it through the Docker Engine API and reports health back, so the control channel works through NAT without inbound ports. SSH is used only to install the control plane and its nodes. Cross-node service names and Caddy ingress use each node's `address` over the operator's private network; Tug provides no overlay or encryption for that workload traffic. See [Networking](docs/networking.md), and [Architecture](docs/architecture.md) for every flow in detail.
52
55
 
53
56
  ## Quick start
54
57
 
@@ -56,14 +59,16 @@ Install the CLI (Node >= 22.18), or run it without installing through `npx @wpel
56
59
 
57
60
  ```bash
58
61
  npm install -g @wpelevator/tug
59
- tug control-install root@203.0.113.5 --advertise-url https://203.0.113.5:8443 --install-docker
62
+ tug control-install root@203.0.113.5 --install-docker
60
63
  tug init tug-root.credential.json
61
- tug node-install root@198.51.100.7 --name app1 --install-docker
64
+ tug node-install root@198.51.100.7 --name app1 --address 10.0.0.7 --install-docker
62
65
  tug deploy -f docker-compose.yml
63
66
  ```
64
67
 
65
68
  The CLI installs the matching `ghcr.io/wpelevator/tug:<version>` image on your hosts. Control serves mutual TLS directly on its port (`8443`): every operator, CI job and node authenticates with a certificate from a private CA created at install, and clients verify a name generated from that CA, so the address can be a bare IP and can change later. Do not terminate TLS in front of it.
66
69
 
70
+ A `controlUrl` in `tug.yml` is the one name for where control is: `control-install` advertises it, and every other command connects to it. Pass `--control-url` (or set `TUG_CONTROL_URL`) to override it for one invocation. See [CLI configuration](docs/cli.md#config-file) for precedence and per-node defaults.
71
+
67
72
  The full walkthrough, including nodes you cannot SSH to (`tug node-prepare-enrollment`), CI credentials (`tug credential-create`), moving the control plane and recovery, is in [Getting started](docs/getting-started.md).
68
73
 
69
74
  ## Documentation
@@ -72,11 +77,11 @@ The full walkthrough, including nodes you cannot SSH to (`tug node-prepare-enrol
72
77
  - [Terminology](docs/terminology.md): core concepts, the naming review (which recommendations are implemented), industry comparisons and the remaining proposed interface improvements.
73
78
  - [Architecture](docs/architecture.md): components, reconciliation, workflow diagrams, secrets and runtime paths.
74
79
  - [Compose extensions](docs/compose.md): `x-tug` placement, health checks, volumes and limitations.
75
- - [Control plane](docs/control-plane.md): install, mutual TLS and the private CA, image delivery, control-plane backup and restore.
80
+ - [Control plane](docs/control-plane.md): install, mutual TLS and the private CA, image delivery, control-plane backup and recovery on another host.
76
81
  - [Backups](docs/backups.md): scheduled workload backups with `x-tug.backup` (restic), restore, recipes for common databases, and a restore drill. See also the [WordPress example](examples/wordpress/README.md).
77
82
  - [Access control](docs/access-control.md): certificate identity classes, scoped operator credentials, roles, audit trail and a GitHub Actions deploy example.
78
83
  - [Security](docs/security.md): deployment trust boundaries, security audit findings and validation limits.
79
- - [Networking](docs/networking.md): WireGuard mesh, cross-node service names, ingress and TLS, firewall notes.
84
+ - [Networking](docs/networking.md): bring-your-own private networking, cross-node service names, ingress and TLS, firewall notes.
80
85
  - [Nodes](docs/nodes.md): node names and IDs, SSH install, self-service enrollment, certificate renewal and recovery, registration and status.
81
86
  - [CLI reference](docs/cli.md): every `tug` command and option.
82
87
  - [Configuration](docs/configuration.md): CLI, control and agent environment variables.
@@ -86,6 +91,7 @@ The full walkthrough, including nodes you cannot SSH to (`tug node-prepare-enrol
86
91
  - [Releases](docs/releases.md): changesets, npm and Docker image publishing.
87
92
  - [Standalone binary](docs/standalone-binary.md): proposed Bun-compiled CLI binaries and Homebrew distribution, with the mTLS dependency findings.
88
93
  - [Roadmap](docs/roadmap.md): goals, principles, the reference WordPress topology and planned milestones.
94
+ - [Docker Swarm comparison](docs/swarm-comparison.md): overlapping capabilities, workflow advantages, scheduling and networking limits, failure behavior and workload fit.
89
95
 
90
96
  ## Development
91
97
 
package/dist/agent.d.ts CHANGED
@@ -6,7 +6,7 @@ import type { ContainerSummary, DockerApi } from './docker-api.ts';
6
6
  import type { AgentLiveCommand, AgentLiveFrame } from './protocol.ts';
7
7
  import type { LiveEngineOptions } from './live.ts';
8
8
  import { parseDuration } from './util.ts';
9
- import type { HostRunner, HttpClient } from './runners.ts';
9
+ import type { HttpClient } from './runners.ts';
10
10
  import type { Command, CommandResult, DesiredState, HealthCheck, ProjectStatus, ImageArtifact, PortBinding, ServiceEndpoint } from './protocol.ts';
11
11
  export interface AgentContext {
12
12
  docker: DockerApi;
@@ -17,10 +17,6 @@ export interface AgentContext {
17
17
  controlUrl: string;
18
18
  /** The node credential presented to control; absent only in tests with a fake `http`. */
19
19
  credential?: Credential;
20
- /** Runs `wg` / `ip` for the mesh. */
21
- host?: HostRunner;
22
- /** Manage the WireGuard mesh (generate a key, report it, reconcile `wg0`). */
23
- mesh?: boolean;
24
20
  caddyImage?: string;
25
21
  /** Host port range (`LOW-HIGH`) for rolling-rollout revisions; default 20000-20999. */
26
22
  rolloutPorts?: string;
@@ -96,8 +92,10 @@ export declare function resolveBindAddress(address: string | null, lookup?: (hos
96
92
  address: string;
97
93
  }>): Promise<string>;
98
94
  export declare function reconcile(desiredState: DesiredState, ctx?: AgentContext, endpoints?: Record<string, ServiceEndpoint[]>, ports?: PortBinding, held?: string[]): Promise<ProjectStatus>;
99
- /** Reports this node's WireGuard public key when control has not seen the current one yet. */
100
- export declare function reportWgKey(ctx: AgentContext): Promise<void>;
95
+ /** Docker `extra_hosts` values must be IPs, so resolve hostname node addresses on this agent. */
96
+ export declare function endpointAddressesForAgent(endpoints?: Record<string, ServiceEndpoint[]>, localServices?: Set<string>, lookup?: (host: string) => Promise<{
97
+ address: string;
98
+ }>): Promise<Record<string, ServiceEndpoint[]>>;
101
99
  export declare function pollOnce(ctx?: AgentContext): Promise<ProjectStatus[]>;
102
100
  /** Loads the persisted identity, or enrolls with a join key. Without either the agent refuses to start. */
103
101
  export declare function loadOrEnroll(): Promise<AgentDocument>;
package/dist/agent.js CHANGED
@@ -13,7 +13,7 @@ import { agentIdentityCsr, createTransport, describeMissingIdentity, installIden
13
13
  import { clientTlsOptions, parseCredential } from "./pki.js";
14
14
  import { createHash } from 'node:crypto';
15
15
  import { addressBoundServices, isRolling, serviceSecretVersions } from "./compose.js";
16
- import { LABEL_PROJECT, LABEL_SERVICE, containerSpecs, networkName, volumeName, } from "./container-spec.js";
16
+ import { LABEL_PROJECT, LABEL_SERVICE, assertVolumeOwnership, containerSpecs, networkName, volumeName, } from "./container-spec.js";
17
17
  import { API_VERSION, DockerEngine, isNotFound } from "./docker-api.js";
18
18
  import { JobRunner } from "./agent-jobs.js";
19
19
  import { LIVE_MAX_FRAME_BYTES, LIVE_START_TIMEOUT_MS } from "./protocol.js";
@@ -21,17 +21,13 @@ import { LiveOperationEngine } from "./live.js";
21
21
  import { activeRollingStates, clearRolloutState, ensureProjectResources, loadRolloutState, projectContainers, reconcileRecreate, reconcileRolling, sweepOrphans, } from "./reconciler.js";
22
22
  import { ensureDir, errMessage, isPublicAddress, now, parseDuration } from "./util.js";
23
23
  import { openAgentDb } from "./db.js";
24
- import { createHostRunner } from "./runners.js";
25
24
  import { CADDY_CONTAINER, reconcileIngress, removeIngress } from "./ingress.js";
26
- import { WG_INTERFACE, ensureWgKeys, readReportedKey, reconcileMesh, rotateWgKeys, writeReportedKey, } from "./mesh.js";
27
25
  // Connect-only override (for example an SSH tunnel); the certificate is still verified against the credential's public URL.
28
26
  const CONTROL_URL_OVERRIDE = process.env.TUG_CONTROL_URL;
29
27
  let NODE_NAME = process.env.TUG_NODE_NAME || process.env.HOSTNAME || 'node1';
30
28
  const STATE_DIR = process.env.TUG_STATE_DIR || '/var/lib/tug';
31
29
  const APPLY = process.env.TUG_APPLY !== 'false';
32
30
  const INTERVAL = Number(process.env.TUG_POLL_INTERVAL_MS || 5000);
33
- // Opt-in WireGuard mesh: needs `wg`/`ip` in the agent image, host networking and NET_ADMIN.
34
- const MESH = process.env.TUG_MESH === 'true';
35
31
  let activeDocument;
36
32
  export function defaultContext(doc = activeDocument) {
37
33
  if (!doc)
@@ -45,8 +41,6 @@ export function defaultContext(doc = activeDocument) {
45
41
  apply: APPLY,
46
42
  controlUrl: transport.base,
47
43
  credential: doc,
48
- host: createHostRunner(),
49
- mesh: MESH,
50
44
  caddyImage: process.env.TUG_CADDY_IMAGE,
51
45
  rolloutPorts: process.env.TUG_ROLLOUT_PORTS,
52
46
  };
@@ -591,7 +585,7 @@ async function publicPortsCheck(ctx) {
591
585
  name: 'ports.public',
592
586
  ok: true,
593
587
  stdout: stdout.slice(-4000),
594
- warning: `${listeners.length} published port${listeners.length === 1 ? ' of Tug projects listens' : 's of Tug projects listen'} on a public address; bind them to 127.0.0.1, the overlay IP or a private address unless that is intended`,
588
+ warning: `${listeners.length} published port${listeners.length === 1 ? ' of Tug projects listens' : 's of Tug projects listen'} on a public address; bind them to 127.0.0.1 or a private node address unless that is intended`,
595
589
  }
596
590
  : {
597
591
  name: 'ports.public',
@@ -633,10 +627,6 @@ export async function executeCommand(command, ctx = defaultContext()) {
633
627
  }
634
628
  }
635
629
  const extra = [];
636
- if (ctx.mesh && ctx.host) {
637
- const host = ctx.host;
638
- extra.push(['wireguard.version', () => host('wg', ['--version'])], ['wireguard.interface', () => host('wg', ['show', WG_INTERFACE])]);
639
- }
640
630
  if (await fs.stat(path.join(ctx.stateDir, 'ingress')).then(() => true, () => false)) {
641
631
  extra.push([
642
632
  'ingress.caddy',
@@ -700,28 +690,21 @@ export async function executeCommand(command, ctx = defaultContext()) {
700
690
  if (command.type === 'volume.delete') {
701
691
  const { project, volume } = payload;
702
692
  // Volumes are never removed by reconciliation; this explicit command is the only way one goes away.
703
- await ctx.docker.removeVolume(volumeName(project, volume)).catch((e) => {
704
- if (!isNotFound(e))
705
- throw e;
706
- });
693
+ const name = volumeName(project, volume);
694
+ const existing = await ctx.docker.inspectVolume(name);
695
+ if (existing)
696
+ assertVolumeOwnership(existing, project, volume);
697
+ if (existing)
698
+ await ctx.docker.removeVolume(name).catch((e) => {
699
+ if (!isNotFound(e))
700
+ throw e;
701
+ });
707
702
  return {
708
703
  commandId: command.id,
709
704
  state: 'succeeded',
710
705
  result: { project, volume, deleted: true },
711
706
  };
712
707
  }
713
- if (command.type === 'wg.rotate-key') {
714
- if (!ctx.mesh)
715
- throw new Error('the WireGuard mesh is not enabled on this agent (TUG_MESH=true)');
716
- const { publicKey } = await rotateWgKeys(ctx.stateDir);
717
- await api(ctx, '/agent/v1/wireguard/key', {
718
- method: 'POST',
719
- headers: { 'content-type': 'application/json' },
720
- body: JSON.stringify({ publicKey }),
721
- });
722
- await writeReportedKey(ctx.stateDir, publicKey);
723
- return { commandId: command.id, state: 'succeeded', result: { publicKey } };
724
- }
725
708
  return {
726
709
  commandId: command.id,
727
710
  state: 'failed',
@@ -1051,7 +1034,7 @@ async function warnPublicPorts(ctx, desiredState, serviceNames, bind) {
1051
1034
  ? addressBoundServices(desiredState.compose, serviceNames, bind.loopback)
1052
1035
  : [];
1053
1036
  const message = exposed.length
1054
- ? `published ports of ${exposed.join(', ')} listen on public address ${bind.address}; enable the mesh, give the node a private address, or write the host IP in ports to make the exposure explicit`
1037
+ ? `published ports of ${exposed.join(', ')} listen on public address ${bind.address}; give the node a private address, or write the host IP in ports to make the exposure explicit`
1055
1038
  : '';
1056
1039
  if ((publicPortWarnings.get(key) ?? '') === message)
1057
1040
  return;
@@ -1070,44 +1053,44 @@ async function warnPublicPorts(ctx, desiredState, serviceNames, bind) {
1070
1053
  }),
1071
1054
  }).catch(() => { });
1072
1055
  }
1073
- /** Reports this node's WireGuard public key when control has not seen the current one yet. */
1074
- export async function reportWgKey(ctx) {
1075
- const { publicKey } = await ensureWgKeys(ctx.stateDir);
1076
- if ((await readReportedKey(ctx.stateDir)) === publicKey)
1077
- return;
1078
- await api(ctx, '/agent/v1/wireguard/key', {
1079
- method: 'POST',
1080
- headers: { 'content-type': 'application/json' },
1081
- body: JSON.stringify({ publicKey }),
1082
- });
1083
- await writeReportedKey(ctx.stateDir, publicKey);
1084
- }
1085
- let lastNetworkError = '';
1086
- async function networkStep(label, fn) {
1087
- try {
1088
- await fn();
1089
- lastNetworkError = '';
1090
- }
1091
- catch (e) {
1092
- const message = `${label}: ${errMessage(e)}`;
1093
- if (message !== lastNetworkError)
1094
- console.error(message);
1095
- lastNetworkError = message;
1096
- }
1056
+ /** Docker `extra_hosts` values must be IPs, so resolve hostname node addresses on this agent. */
1057
+ export async function endpointAddressesForAgent(endpoints = {}, localServices = new Set(), lookup = (host) => dns.lookup(host)) {
1058
+ const cache = new Map();
1059
+ const resolve = (address) => {
1060
+ if (net.isIP(address))
1061
+ return Promise.resolve(address);
1062
+ let resolved = cache.get(address);
1063
+ if (!resolved) {
1064
+ resolved = lookup(address).then(({ address: ip }) => ip, (error) => {
1065
+ throw new Error(`cannot resolve node address ${address} for cross-node service names: ${errMessage(error)}`, { cause: error });
1066
+ });
1067
+ cache.set(address, resolved);
1068
+ }
1069
+ return resolved;
1070
+ };
1071
+ return Object.fromEntries(await Promise.all(Object.entries(endpoints)
1072
+ .filter(([service]) => !localServices.has(service))
1073
+ .map(async ([service, list]) => [
1074
+ service,
1075
+ (await Promise.all(list.map(async (ep) => {
1076
+ try {
1077
+ return { ...ep, address: await resolve(ep.address) };
1078
+ }
1079
+ catch (error) {
1080
+ console.error(errMessage(error));
1081
+ return null;
1082
+ }
1083
+ }))).filter((ep) => ep !== null),
1084
+ ])));
1097
1085
  }
1098
1086
  export async function pollOnce(ctx = defaultContext()) {
1099
- if (ctx.mesh)
1100
- await networkStep('wireguard key report failed', () => reportWgKey(ctx));
1101
1087
  const work = await api(ctx, '/agent/v1/work');
1102
- // The mesh comes up first so cross-node service endpoints are routable when containers start.
1103
- if (ctx.mesh && work.network) {
1104
- const network = work.network;
1105
- await networkStep('wireguard reconcile failed', () => reconcileMesh(ctx.host ?? createHostRunner(), ctx.stateDir, network));
1106
- }
1107
1088
  const projects = [];
1108
1089
  for (const desired of work.desiredStates || (work.desiredState ? [work.desiredState] : [])) {
1109
1090
  try {
1110
- projects.push(await reconcile(desired, ctx, work.serviceEndpoints?.[desired.project], work.ports, work.holds?.[desired.project]));
1091
+ projects.push(await reconcile(desired, ctx, await endpointAddressesForAgent(work.serviceEndpoints?.[desired.project], new Set(Object.entries(desired.services)
1092
+ .filter(([, spec]) => spec.placement?.nodes?.includes(ctx.nodeName))
1093
+ .map(([name]) => name))), work.ports, work.holds?.[desired.project]));
1111
1094
  }
1112
1095
  catch (e) {
1113
1096
  projects.push({
@@ -1198,12 +1181,9 @@ async function enroll(joinKey) {
1198
1181
  const transport = createTransport(decoded.credential, CONTROL_URL_OVERRIDE || decoded.connectUrl);
1199
1182
  console.log(`enrolling with ${decoded.credential.url} as ${decoded.name}`);
1200
1183
  const csr = await agentIdentityCsr(STATE_DIR);
1201
- const wgPublicKey = MESH ? (await ensureWgKeys(STATE_DIR)).publicKey : undefined;
1202
1184
  const body = {
1203
1185
  csr,
1204
1186
  address: process.env.TUG_NODE_ADDRESS,
1205
- publicAddress: process.env.TUG_NODE_PUBLIC_ADDRESS,
1206
- wgPublicKey,
1207
1187
  };
1208
1188
  try {
1209
1189
  const res = await transport.http(`${transport.base}/agent/v1/enroll`, {
@@ -1214,8 +1194,6 @@ async function enroll(joinKey) {
1214
1194
  if (!res.ok)
1215
1195
  throw new Error(`enroll failed: ${res.status} ${await res.text()}`);
1216
1196
  const doc = await installIdentity(STATE_DIR, (await res.json()));
1217
- if (wgPublicKey)
1218
- await writeReportedKey(STATE_DIR, wgPublicKey);
1219
1197
  console.log(`enrolled as node ${doc.nodeName}`);
1220
1198
  return doc;
1221
1199
  }
@@ -1,6 +1,6 @@
1
1
  import { Transform } from 'node:stream';
2
2
  import { redactSecrets } from "./backup-destination.js";
3
- import { LABEL_JOB, LABEL_PROJECT, LABEL_SERVICE } from "./container-spec.js";
3
+ import { LABEL_JOB, LABEL_PROJECT, LABEL_SERVICE, assertVolumeOwnership, volumeName, } from "./container-spec.js";
4
4
  /** Docker volume holding restic's cache across jobs, so repeated runs do not re-read the repository index. */
5
5
  export const RESTIC_CACHE_VOLUME = 'tug_restic_cache';
6
6
  const TAIL_BYTES = 4096;
@@ -148,14 +148,26 @@ export async function runBackupJob(payload, commandId, ctx, kind = 'backup.run')
148
148
  if (scratch) {
149
149
  if (!dockerVolume)
150
150
  throw new JobError('a scratch restore needs a scratch volume name');
151
- if (!(await docker.inspectVolume(dockerVolume)))
151
+ let existing = await docker.inspectVolume(dockerVolume);
152
+ if (!existing) {
152
153
  await docker.createVolume(dockerVolume, {
153
154
  labels: { 'tug.project': payload.project, 'tug.restore': payload.runId },
154
155
  });
156
+ existing = await docker.inspectVolume(dockerVolume);
157
+ }
158
+ if (existing?.Labels?.[LABEL_PROJECT] !== payload.project ||
159
+ existing.Labels?.['tug.restore'] !== payload.runId)
160
+ throw new JobError(`scratch volume ${dockerVolume} is not owned by this restore`);
161
+ }
162
+ else {
163
+ if (dockerVolume !== volumeName(payload.project, payload.volume.name))
164
+ throw new JobError('volume name does not match the project and target');
165
+ const existing = await docker.inspectVolume(dockerVolume);
166
+ if (!existing)
167
+ // Docker would silently create an empty volume for a missing name and the backup would succeed with nothing in it.
168
+ throw new JobError(`volume ${dockerVolume} does not exist on this node; a volume is created when a service that uses it is placed here`);
169
+ assertVolumeOwnership(existing, payload.project, payload.volume.name);
155
170
  }
156
- else if (!(await docker.inspectVolume(dockerVolume)))
157
- // Docker would silently create an empty volume for a missing name and the backup would succeed with nothing in it.
158
- throw new JobError(`volume ${dockerVolume} does not exist on this node; a volume is created when a service that uses it is placed here`);
159
171
  binds.push(`${dockerVolume}:/data${kind === 'backup.restore' ? '' : ':ro'}`);
160
172
  }
161
173
  const restic = await helper('restic', {
@@ -1,5 +1,5 @@
1
1
  import { id } from "./util.js";
2
- import { volumeName } from "./container-spec.js";
2
+ import { scratchVolumeName } from "./container-spec.js";
3
3
  const MINUTE = 60_000;
4
4
  const LOCK_TTL_MS = 15 * MINUTE;
5
5
  const HOLD_TIMEOUT_MS = 5 * MINUTE;
@@ -77,6 +77,8 @@ export function createRestoreService(deps) {
77
77
  }
78
78
  /** Validates a request and describes what a restore would do; nothing is changed. */
79
79
  function plan(request) {
80
+ // The most recent deployment may have moved the target since the last scheduler tick.
81
+ backups.sync();
80
82
  const desired = deployed(request.project);
81
83
  if (!desired)
82
84
  return { error: `project ${request.project} has no successful deployment`, status: 409 };
@@ -115,9 +117,7 @@ export function createRestoreService(deps) {
115
117
  snapshotId: snapshot.snapshotId,
116
118
  snapshotTime: snapshot.time,
117
119
  mode,
118
- ...(mode === 'scratch'
119
- ? { scratchVolume: volumeName(request.project, `restore-${id('r').slice(-6)}`) }
120
- : {}),
120
+ ...(mode === 'scratch' ? { scratchVolume: scratchVolumeName(request.project, id('r')) } : {}),
121
121
  writers,
122
122
  safetyBackup: mode === 'in-place' && request.safetyBackup !== false,
123
123
  };
@@ -397,6 +397,8 @@ export function createBackupService(deps) {
397
397
  }
398
398
  /** Queues the `backup.restore` job for a target on its node. */
399
399
  function queueRestore(project, target, restore, runId) {
400
+ // Placement can change after planning; never queue a restore using a stale schedule.
401
+ sync();
400
402
  const schedule = db.get('SELECT * FROM backup_schedules WHERE project_id=? AND target=? AND enabled=1', project, target);
401
403
  const current = deployed(project)[0];
402
404
  if (!schedule || !current)
@@ -11,5 +11,7 @@ export interface LocalBuildDependencies {
11
11
  /** Credential exclusions are mandatory and cannot be re-included by source ignore rules. The tracked config names are excluded at the context root only, so an app's own nested `tug.json` is kept. */
12
12
  export declare function sourceArchiveFilter(entry: string, configPaths: string[], ignores: string[]): boolean;
13
13
  export declare function createSourceArchive(contextDir: string, archive: string, configFiles: string[]): Promise<void>;
14
+ /** Build from the same filtered, validated source snapshot used for remote builds. */
15
+ export declare function createLocalBuildSnapshot(contextDir: string, directory: string, configFiles: string[]): Promise<string>;
14
16
  /** Build only application build services; callers own temporary files and upload transport. */
15
17
  export declare function prepareLocalImages(compose: ComposeFile, contextDir: string, directory: string, dependencies?: LocalBuildDependencies): Promise<LocalImageUpload[]>;
package/dist/cli-build.js CHANGED
@@ -2,7 +2,7 @@ import path from 'node:path';
2
2
  import fs from 'node:fs/promises';
3
3
  import { randomUUID } from 'node:crypto';
4
4
  import * as tar from 'tar';
5
- import { buildImage, run } from "./build.js";
5
+ import { buildImage, run, unpackContext } from "./build.js";
6
6
  import { saveImageArchive } from "./artifacts.js";
7
7
  import { CONFIG_NAMES } from "./cli-config.js";
8
8
  /** At any depth: the git directory and every file that can hold a token (the untracked config overlays). */
@@ -122,6 +122,19 @@ export async function createSourceArchive(contextDir, archive, configFiles) {
122
122
  filter: (entry) => sourceArchiveFilter(entry, configPaths, ignores),
123
123
  }, ['.']);
124
124
  }
125
+ /** Build from the same filtered, validated source snapshot used for remote builds. */
126
+ export async function createLocalBuildSnapshot(contextDir, directory, configFiles) {
127
+ const archive = path.join(directory, 'source.tgz');
128
+ const snapshot = path.join(directory, 'source');
129
+ try {
130
+ await createSourceArchive(contextDir, archive, configFiles);
131
+ await unpackContext(archive, snapshot);
132
+ return snapshot;
133
+ }
134
+ finally {
135
+ await fs.rm(archive, { force: true });
136
+ }
137
+ }
125
138
  /** Build only application build services; callers own temporary files and upload transport. */
126
139
  export async function prepareLocalImages(compose, contextDir, directory, dependencies = {
127
140
  build: buildImage,
@@ -1,5 +1,5 @@
1
1
  import type { Credential } from './pki.ts';
2
- export type SettingType = 'string' | 'boolean' | 'port' | 'list' | 'path';
2
+ export type SettingType = 'string' | 'boolean' | 'list' | 'path';
3
3
  export interface Setting {
4
4
  type: SettingType;
5
5
  /** The flag or positional argument the key stands for; the key is the camelCase of the long flag. */
@@ -28,6 +28,9 @@ export interface ResolvedCliConfig {
28
28
  localFile: string;
29
29
  /** Where connections go: the credential's public URL unless overridden (an override changes the connect target only). */
30
30
  controlUrl: string;
31
+ /** Shared config address for installation, before connection-only flags, environment and credential defaults. */
32
+ /** The control URL the operator asked for (flag, environment or config), without credential or default fallbacks. */
33
+ requestedControlUrl?: string;
31
34
  credential?: Credential;
32
35
  credentialFile?: string;
33
36
  project: string;
@@ -8,7 +8,18 @@ import { errMessage } from "./util.js";
8
8
  export const COMMAND_SETTINGS = {
9
9
  file: { type: 'path', flag: '--file', commands: ['deploy'] },
10
10
  buildLocal: { type: 'boolean', flag: '--build-local', commands: ['deploy'] },
11
- wait: { type: 'boolean', flag: '--no-wait', commands: ['deploy', 'rollback'] },
11
+ wait: {
12
+ type: 'boolean',
13
+ flag: '--no-wait',
14
+ commands: [
15
+ 'deploy',
16
+ 'rollback',
17
+ 'backup-run',
18
+ 'node-install',
19
+ 'node-replace-identity',
20
+ 'node-repair-identity',
21
+ ],
22
+ },
12
23
  };
13
24
  /** Per-node connection data (`nodes.<name>`); `nodeDefaults` takes the subset that is not node-specific. */
14
25
  export const NODE_SETTINGS = {
@@ -17,32 +28,28 @@ export const NODE_SETTINGS = {
17
28
  flag: '<ssh-target>',
18
29
  commands: ['node-install', 'node-replace-identity', 'node-repair-identity'],
19
30
  },
20
- address: { type: 'string', flag: '--address', commands: ['node-install', 'node-register'] },
21
- publicAddress: {
31
+ address: {
22
32
  type: 'string',
23
- flag: '--public-address',
24
- commands: ['node-install', 'node-register'],
33
+ flag: '--address',
34
+ commands: [
35
+ 'node-install',
36
+ 'node-replace-identity',
37
+ 'node-repair-identity',
38
+ 'node-register',
39
+ 'node-prepare-enrollment',
40
+ ],
25
41
  },
26
42
  label: {
27
43
  type: 'list',
28
44
  flag: '--label',
29
- commands: ['node-install', 'node-register', 'node-prepare-enrollment'],
30
- },
31
- mesh: {
32
- type: 'boolean',
33
- flag: '--mesh',
34
45
  commands: [
35
46
  'node-install',
36
47
  'node-replace-identity',
37
48
  'node-repair-identity',
49
+ 'node-register',
38
50
  'node-prepare-enrollment',
39
51
  ],
40
52
  },
41
- wgPort: {
42
- type: 'port',
43
- flag: '--wg-port',
44
- commands: ['node-install', 'node-replace-identity', 'node-repair-identity'],
45
- },
46
53
  installDocker: {
47
54
  type: 'boolean',
48
55
  flag: '--install-docker',
@@ -51,6 +58,7 @@ export const NODE_SETTINGS = {
51
58
  'node-replace-identity',
52
59
  'node-repair-identity',
53
60
  'node-prepare-enrollment',
61
+ 'control-install',
54
62
  ],
55
63
  },
56
64
  image: {
@@ -61,6 +69,7 @@ export const NODE_SETTINGS = {
61
69
  'node-replace-identity',
62
70
  'node-repair-identity',
63
71
  'node-prepare-enrollment',
72
+ 'control-install',
64
73
  ],
65
74
  },
66
75
  agentControlUrl: {
@@ -74,7 +83,7 @@ export const NODE_SETTINGS = {
74
83
  ],
75
84
  },
76
85
  };
77
- const NODE_SPECIFIC_KEYS = ['ssh', 'address', 'publicAddress'];
86
+ const NODE_SPECIFIC_KEYS = ['ssh', 'address'];
78
87
  const SETTINGS = { ...COMMAND_SETTINGS, ...NODE_SETTINGS };
79
88
  /** Default file names, in lookup order; more than one present is an error. */
80
89
  export const CONFIG_NAMES = ['tug.yml', 'tug.yaml', 'tug.json', '.tug.json'];
@@ -118,16 +127,14 @@ export async function resolveCliConfig(options, env, cwd, creating = false) {
118
127
  if (!creating)
119
128
  throw new Error(`cannot use credential ${credentialFile ?? 'from TUG_CREDENTIAL'}: ${errMessage(e)}`, { cause: e });
120
129
  }
121
- const controlUrl = options.controlUrl ||
122
- env.TUG_CONTROL_URL ||
123
- config.controlUrl ||
124
- credential?.url ||
125
- 'https://localhost:8443';
130
+ const requestedControlUrl = options.controlUrl || env.TUG_CONTROL_URL || config.controlUrl;
131
+ const controlUrl = requestedControlUrl || credential?.url || 'https://localhost:8443';
126
132
  return {
127
133
  file,
128
134
  fileExists: Boolean(base),
129
135
  localFile,
130
136
  controlUrl: normalizeControlUrl(controlUrl),
137
+ requestedControlUrl: requestedControlUrl ? normalizeControlUrl(requestedControlUrl) : undefined,
131
138
  credential,
132
139
  credentialFile,
133
140
  project: options.project || env.TUG_PROJECT || config.project || 'site',
@@ -214,8 +221,6 @@ function settingValue(setting, value, where, dir) {
214
221
  if (typeof value !== 'boolean')
215
222
  throw new Error(`${where} must be true or false`);
216
223
  return value;
217
- case 'port':
218
- return String(integerOption(String(value), where, 1, 65535));
219
224
  case 'list':
220
225
  return (Array.isArray(value) ? value : [value]).map((item, index) => text(item, Array.isArray(value) ? `${where}[${index}]` : where));
221
226
  }
@@ -1,17 +1,20 @@
1
1
  /** The agent dials container IPs for health checks; from the default bridge they are unreachable on current Docker (29.x), so it always shares the host network namespace. */
2
2
  export declare const HOST_NETWORK_FLAG = "--network host";
3
- export declare const MESH_RUN_FLAGS = "--network host --cap-add NET_ADMIN";
4
3
  /** Docker platform for the architecture a host reports with `uname -m`. */
5
4
  export declare function dockerPlatform(uname: string): string;
6
5
  /** Build for the host the image will run on, not the machine that builds it (an arm64 laptop and an amd64 VPS differ). */
7
6
  export declare function localImageBuildArgs(imageTag: string, uname: string): string[];
8
7
  export declare function shellQuote(value: unknown): string;
9
- export declare function dockerRunSnippet(image: string, joinKey: string, mesh?: boolean): string;
8
+ export declare function dockerRunSnippet(image: string, joinKey: string, address?: string): string;
10
9
  /** The bootstrap runs in its own shell so its idempotent early exit cannot skip enrollment. */
11
10
  export declare function enrollmentCloudInit(bootstrap: string, command: string): string;
12
11
  /** Name of the container `control-install` creates (Compose project `tug-control`, service `control`). */
13
12
  export declare const CONTROL_CONTAINER = "tug-control-control-1";
14
13
  /** A TCP port is free, or is held by the control container this install would replace, so installing again upgrades in place. */
15
14
  export declare function controlPortCheck(port: number): string;
16
- /** An occupied mesh port is reusable only when the installed Tug agent owns the live key. */
17
- export declare function meshPortCheck(port: number): string;
15
+ /**
16
+ * The default control URL for `control-install`: the SSH target's host and the published port. Accepts
17
+ * `host`, `user@host`, `ssh://user@host:2222` and bracketed IPv6. An SSH alias from ~/.ssh/config is used as written,
18
+ * so pass `--control-url` when the alias is not a name or address clients can reach.
19
+ */
20
+ export declare function defaultControlUrl(sshTarget: string, port: number | string): string;