@wpelevator/tug 0.2.0 → 0.3.0

package/README.md

@@ -2,13 +2,13 @@
 
 The name: tugboats pull ships into place, like the pull-based agents here; `tug` is the CLI.
 
-Deploy Docker Compose projects to one or more ordinary Linux VPSes through a small self-hosted control plane and a pull-based agent on each node, without Kubernetes or a hosted PaaS. The original specification is in [plan.md](plan.md).
+Deploy Docker Compose projects to one or more ordinary Linux VPSes through a small self-hosted control plane and a pull-based agent on each node, without Kubernetes or a hosted PaaS. What it is for, the deployment shapes it targets and the planned milestones are in the [Roadmap](docs/roadmap.md).
 
 ## Features
 
 - Control API with bearer-token auth, scoped `read`/`deploy`/`admin` API tokens for CI and teammates, an audit trail naming who acted, and SQLite metadata.
 - Nodes join over SSH (`node-install`) or enroll themselves with a single `docker run` (`node-enroll`), with per-node agent and registry tokens. Agents only dial out, so nodes behind NAT work.
-- Agents pull desired state from control, apply a generated per-node Compose file, run TCP/HTTP/Docker health checks and report status.
+- Agents pull desired state from control, apply a generated per-node Compose file, run TCP/HTTP/Docker health checks and report status. An outbound WebSocket wakes them as soon as control queues work and carries live `exec`, followed logs, `ps` and `inspect` sessions, with long-poll and periodic-poll fallbacks for ordinary work.
 - `x-tug` placement of services and volumes on nodes, with immutable desired-state records per deployment.
 - Build application images on control from uploaded source, or locally with `deploy --build-local`. Agents download saved images directly from control by default; a bring-your-own registry is optional.
 - Secrets encrypted at rest, released only to the nodes whose services reference them, with targeted service recreation when only a secret changes.
@@ -16,8 +16,41 @@ Deploy Docker Compose projects to one or more ordinary Linux VPSes through a sma
 - Opt-in zero-downtime rolling rollouts for ingress-fronted services (`x-tug.rollout`) and automatic rollback of failed deployments (`x-tug.rollback: auto`). See [Compose extensions](docs/compose.md#rolling-rollouts).
 - Per-node registry pull tokens enforced by an htpasswd-backed registry you provide, with rotation and revocation.
 - Optional WireGuard mesh between nodes, health-aware cross-node service names and a managed Caddy ingress with automatic HTTPS, each opt-in. See [Networking](docs/networking.md).
+- Private-by-default published ports: generated Compose binds them to the overlay IP, loopback or the node address, never every interface unless you write `0.0.0.0`. See [Networking](docs/networking.md#published-ports-private-by-default).
 - Control-plane backup and restore, and `tug doctor` diagnostics.
 
+## How it fits together
+
+```text
+  tug CLI / CI (API token)
+       |
+       |  HTTPS: deploy, status, logs, exec
+       v
++-------------------------------------+
+| Control plane                       |
+|   Control API (role-checked)        |
+|   SQLite: desired states, secrets   |
+|   image builds, saved images        |
+|   registry (optional)               |
++-------------------------------------+
+       ^                       ^
+       |  agents dial out: work, status,
+       |  WebSocket wake and live sessions
+       |                       |
++------+------------+   +------+------------------+
+| Node app1         |   | Node app2 (ingress node)|
+|   Agent           |<->|   Agent                 |
+|   Docker Compose  |   |   Docker Compose        |
+|   services        |   |   services              |
+|                   |   |   managed Caddy ingress |
++-------------------+   +-------------------------+
+        <-> optional WireGuard mesh between nodes
+
+  SSH is used only by the CLI to install the control plane and nodes.
+```
+
+Operators and CI talk to the control API. The control plane stores desired state and builds images; each agent dials out to fetch the work for its node, applies it with Docker Compose and reports health back, so nodes behind NAT need no inbound ports. SSH is used only to install control and agents. The optional WireGuard mesh and Caddy ingress are described in [Networking](docs/networking.md), and every flow in detail in [Architecture](docs/architecture.md).
+
 ## Quick start
 
 Install the CLI (Node >= 22.18), or run it without installing through `npx @wpelevator/tug`:
@@ -33,13 +66,14 @@ The CLI installs the matching `ghcr.io/wpelevator/tug:<version>` image on your h
 3. Set the secrets your services need: `tug secret-set DB_PASSWORD`.
 4. Add nodes with `tug node-install root@app1.example.com --name app1 --install-docker`, or `tug node-enroll --name app1` and run the printed Docker command. For a bare cloud host, `tug node-enroll --name app1 --install-docker` prints cloud-init user-data.
 5. Deploy: `tug deploy -f docker-compose.yml`. It waits until every node reports the new version healthy.
-6. Operate with `tug status`, `events`, `logs`, `restart`, `rollback` and `doctor`.
+6. Operate with `tug status`, `events`, `exec`, `logs`, `ps`, `inspect`, `restart`, `rollback` and `doctor`.
 
 ## Documentation
 
 - [Architecture](docs/architecture.md): components, reconciliation, workflow diagrams, secrets and runtime paths.
 - [Compose extensions](docs/compose.md): `x-tug` placement, health checks, volumes and limitations.
 - [Control plane](docs/control-plane.md): install, TLS, registry credentials, backup and restore.
+- [Backups](docs/backups.md): interim WordPress backup sidecar with restic, per-node and NFS variants, and a restore drill until `x-tug.backup` (M11).
 - [Access control](docs/access-control.md): scoped API tokens, roles, audit trail and a GitHub Actions deploy example.
 - [Networking](docs/networking.md): WireGuard mesh, service DNS, ingress and TLS, firewall notes.
 - [Nodes](docs/nodes.md): SSH install, self-service enrollment, registration and status.
@@ -47,9 +81,10 @@ The CLI installs the matching `ghcr.io/wpelevator/tug:<version>` image on your h
 - [Configuration](docs/configuration.md): CLI, control and agent environment variables.
 - [Development](docs/development.md): local setup, CI, Compose stack, host mode and the integration test.
 - [Testing](docs/testing.md): test layers, the numbers rollouts are held to, and the backlog of mocked tests.
+- [Observability](docs/observability.md): what can be observed today, and the proposed logs, metrics and alerting.
 - [Releases](docs/releases.md): changesets, npm and Docker image publishing.
 - [Standalone binary](docs/standalone-binary.md): proposed Bun-compiled CLI binaries and Homebrew distribution.
-- [Roadmap](docs/roadmap.md): planned milestones toward parity with and beyond uncloud.
+- [Roadmap](docs/roadmap.md): goals, principles, the reference WordPress topology and planned milestones.
 
 ## Development
 

package/dist/agent.d.ts

@@ -1,7 +1,10 @@
 #!/usr/bin/env node
+import { WebSocket } from 'ws';
+import type { AgentLiveCommand, AgentLiveFrame } from './protocol.ts';
+import type { LiveEngineOptions } from './live.ts';
 import { parseDuration } from './util.ts';
 import type { DockerRunner, HostRunner, HttpClient } from './runners.ts';
-import type { Command, CommandResult, DesiredState, HealthCheck, ProjectStatus, ImageArtifact, ServiceEndpoint } from './protocol.ts';
+import type { Command, CommandResult, DesiredState, HealthCheck, ProjectStatus, ImageArtifact, PortBinding, ServiceEndpoint } from './protocol.ts';
 export interface AgentContext {
     docker: DockerRunner;
     http: HttpClient;
@@ -19,13 +22,71 @@ export interface AgentContext {
     rolloutPorts?: string;
 }
 export declare function defaultContext(): AgentContext;
+export type AgentLiveControlFrame = AgentLiveCommand;
+export type AgentLiveOutputFrame = AgentLiveFrame;
+interface LiveChannelHandlers {
+    onLive?: (frame: AgentLiveControlFrame) => void | Promise<void>;
+    onDisconnect?: () => void;
+}
+type WebSocketFactory = (url: string, token: string) => WebSocket;
+/** Keeps one outbound stream open and falls back to a bounded HTTP wait when an intermediary rejects WebSockets. */
+export declare class AgentWakeChannel {
+    private readonly ctx;
+    private readonly createSocket;
+    private socket?;
+    private cursor;
+    private pending;
+    private waiter?;
+    private readonly handlers;
+    constructor(ctx: AgentContext, createSocket?: WebSocketFactory, handlers?: LiveChannelHandlers);
+    private notify;
+    private connect;
+    private waitOnSocket;
+    private longPoll;
+    wait(timeoutMs: number): Promise<void>;
+    send(frame: AgentLiveOutputFrame): boolean;
+    close(): void;
+}
+/** Resolves a service once before starting a process so rollouts cannot silently retarget an open session. */
+export declare function resolveLiveContainer(ctx: AgentContext, request: {
+    project: string;
+    service?: string;
+}): Promise<string>;
+export declare class AgentLiveOperations {
+    private readonly ctx;
+    private readonly emit;
+    private readonly engine;
+    private readonly sessions;
+    constructor(ctx: AgentContext, emit: (frame: AgentLiveOutputFrame) => void, options?: Omit<LiveEngineOptions, 'emit'>);
+    handle(frame: AgentLiveControlFrame): Promise<void>;
+    dispose(): void;
+    private fail;
+    private open;
+}
 export declare function containerIp(ctx: AgentContext, project: string, service: string, cwd: string, composeFile?: string): Promise<string>;
 export declare function waitForHealth(ctx: AgentContext, project: string, service: string, health: HealthCheck | null | undefined, cwd: string, composeFile?: string): Promise<void>;
 export { parseDuration };
+export interface PublicListener {
+    project: string;
+    container: string;
+    listen: string;
+}
+/**
+ * Containers of Tug projects (`projects`, plus their rolling revision projects `<project>-…`) that
+ * listen on all interfaces or on a public IP, from `docker ps --format '{{.Label "com.docker.compose.project"}}\t{{.Names}}\t{{.Ports}}'`.
+ */
+export declare function publicListeners(psOutput: string, projects: readonly string[]): PublicListener[];
 export declare function executeCommand(command: Command, ctx?: AgentContext): Promise<CommandResult>;
 export declare function ensureArtifact(ctx: AgentContext, artifact: ImageArtifact): Promise<void>;
 export declare function pruneManagedImages(ctx: AgentContext, retained: string[]): Promise<void>;
-export declare function reconcile(desiredState: DesiredState, ctx?: AgentContext, endpoints?: Record<string, ServiceEndpoint[]>): Promise<ProjectStatus>;
+/**
+ * The IP that published ports without a host IP listen on: `127.0.0.1` when the node has no
+ * address, the address itself when it is an IP, otherwise what it resolves to on this node.
+ */
+export declare function resolveBindAddress(address: string | null, lookup?: (host: string) => Promise<{
+    address: string;
+}>): Promise<string>;
+export declare function reconcile(desiredState: DesiredState, ctx?: AgentContext, endpoints?: Record<string, ServiceEndpoint[]>, ports?: PortBinding): Promise<ProjectStatus>;
 /** Reports this node's WireGuard public key when control has not seen the current one yet. */
 export declare function reportWgKey(ctx: AgentContext): Promise<void>;
 export declare function pollOnce(ctx?: AgentContext): Promise<ProjectStatus[]>;