@studiometa/trafic-agent 0.1.49 → 0.1.50

package/README.md

@@ -1,6 +1,6 @@
 # @studiometa/trafic-agent
 
-Server agent for DDEV preview environments — auth, scale-to-zero, and auto-start.
+Server agent for DDEV preview environments — auth, scale-to-zero, auto-start, scheduled removals and backups.
 
 Part of [Trafic](https://github.com/studiometa/trafic), a tool for managing DDEV preview environments on Linux servers.
 
@@ -10,6 +10,9 @@ Part of [Trafic](https://github.com/studiometa/trafic), a tool for managing DDEV
 - **Scale-to-zero** — Automatically stop idle DDEV projects to save RAM
 - **Auto-start** — Show a waiting page and restart stopped projects on request
 - **Per-project config** — Override auth and idle timeout per project
+- **Backups on removal** — Dump the database and archive the upload directories before a project is removed
+- **Scheduled removal** — Remove a project after a delay; a new deploy cancels it
+- **Running limit** — `max_running_projects` stops the least recently used projects
 
 ## Installation
 
@@ -19,24 +22,47 @@ npm install -g @studiometa/trafic-agent
 
 ## Commands
 
-### `trafic-agent serve`
+### `trafic-agent start`
 
-Start the agent server.
+Start the agent server. Setup installs a systemd service that runs this command.
 
 ```bash
-trafic-agent serve
+trafic-agent start
 # or with custom config
-trafic-agent serve --config /etc/trafic/config.toml
+trafic-agent start --config /etc/trafic/config.toml
+# or on another port than the one in the config
+trafic-agent start --port 9877
 ```
 
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-c`, `--config` | Path to the config file | first of `/etc/trafic/config.toml`, `./config.toml`, `./trafic.toml` |
+| `-p`, `--port` | Port of the agent HTTP server. Replaces `port` from the config | `port` from the config |
+
 ### `trafic-agent setup`
 
-Interactive server setup — installs Docker, DDEV, configures DNS, and sets up the agent as a systemd service.
+Server setup, run as root. It installs Docker, DDEV and Node.js, configures DDEV and Traefik for forward auth, writes `/etc/trafic/config.toml`, installs the agent as a systemd service and hardens the server. You configure the wildcard DNS yourself. Setup takes its options from flags only: it asks no questions.
 
 ```bash
-trafic-agent setup --tld previews.example.com
+sudo trafic-agent setup --tld=previews.example.com
 ```
 
+`trafic setup` runs this command over SSH. See the [CLI README](../trafic-cli/README.md#trafic-setup).
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `--tld` | TLD for DDEV projects. Required on the first run; a re-run reuses `tld` from the config | `tld` from the config |
+| `--email` | Email for Let's Encrypt certificates. Without it, projects use a mkcert certificate | - |
+| `--dns-provider` | DNS-01 provider (lego name, e.g. `cloudflare`), for one wildcard `*.<tld>` certificate. Requires `--email`. A re-run reuses `[tls] dns_provider` from the config, and then `--email` is required again. See [TLS](#tls) | `[tls] dns_provider` from the config |
+| `--dns-env` | Credential for that provider as `KEY=VALUE`, repeatable | `[tls.dns_env]` from the config |
+| `--ssh-users` | SSH users to allow, comma-separated | `ddev` |
+| `--trusted-proxy-hops` | Proxies in front of the agent (`2` behind a CDN). Written to a new config only; ignored when the config exists | `1` |
+| `--no-hardening` | Skip server hardening | `false` |
+| `--no-root-ssh` | Disable root SSH login (recovery via rescue mode only) | `false` |
+| `--no-docker` | Skip Docker installation | `false` |
+| `--no-ddev` | Skip DDEV installation | `false` |
+| `--dry-run` | Show what would be done without making changes | `false` |
+
 **Requirements:**
 - Ubuntu 24.04 LTS
 - Root access (for initial setup)
@@ -44,6 +70,90 @@ trafic-agent setup --tld previews.example.com
 
 **Automatic security updates.** Setup installs `unattended-upgrades` and writes `/etc/apt/apt.conf.d/50unattended-upgrades`, allowing Ubuntu's release, security and ESM pockets plus Docker Engine (`origin=Docker`, from `download.docker.com`). Docker security releases are therefore applied daily with the rest. Note that a Docker Engine update restarts the daemon and stops running preview containers; the waiting page starts a project again on its next request. DDEV is deliberately excluded — it is updated by `trafic-agent upgrade` instead. The server never reboots on its own (`Unattended-Upgrade::Automatic-Reboot "false"`).
 
+### `trafic-agent audit`
+
+Check the security state of the server: SSH, firewall, Fail2ban, automatic updates, the agent service, Docker, disk usage, the permissions of the config, the owner of the projects directory and, with a `[tls]` section, the wildcard certificate and the permissions of its credentials. The command exits with code 1 when a check fails. Run it as root: `ufw status` and the files under `/var/lib/docker` need root.
+
+```bash
+sudo trafic-agent audit
+```
+
+### `trafic-agent version` / `trafic-agent help`
+
+`trafic-agent version` (also `-v`, `--version`) prints the version. `trafic-agent help` (also `-h`, `--help`) prints the help with every option.
+
+### `trafic-agent skill`
+
+Print the [`trafic-agent` agent skill](../../skills/trafic-agent/SKILL.md) of this version to stdout. A coding agent reads it to operate the server. To install the skill from the repository instead, run `npx skills add studiometa/trafic` or `gh skill install studiometa/trafic`.
+
+```bash
+trafic-agent skill
+```
+
+### `trafic-agent destroy`
+
+Remove a project from the server. Run it as the `ddev` user, the user the service runs as. The command refuses to run as root: DDEV does not run as root, and root could leave files in the database directory that the service cannot write to.
+
+```bash
+trafic-agent destroy preview-42--my-app
+# or with custom config
+trafic-agent destroy preview-42--my-app --config /etc/trafic/config.toml
+# back up the project first
+trafic-agent destroy preview-42--my-app --backup
+# give the backup up to one hour
+trafic-agent destroy preview-42--my-app --backup --timeout 1h
+# remove the project in 48 hours, after a backup
+trafic-agent destroy preview-42--my-app --delay 48h --backup
+# cancel that removal
+trafic-agent destroy preview-42--my-app --cancel
+# list the scheduled removals
+trafic-agent destroy --list
+```
+
+Steps:
+1. **Backup** (with `--backup` only) — see [Backups](#backups). If the backup fails, the command removes nothing and exits with an error
+2. **DDEV** — runs `ddev delete -Oy` in the project directory (containers and volumes, no snapshot). A failure is reported as a warning and does not stop the removal
+3. **Directory** — removes the project directory
+4. **Database** — deletes the project's rows from the agent database (`projects`, `access_logs` and `scheduled_destroys`)
+
+The project directory is the `approot` from DDEV's `project_list.yaml`, or `<projects_dir>/<name>` when DDEV no longer knows the project. When the directory does not exist, the command prints a warning, does not run DDEV and only deletes the project's rows from the agent database, so a retry after a failed run cleans them up. The command refuses a directory that is not inside `projects_dir`, also after it follows symbolic links, and refuses a project directory that is a symbolic link. It also refuses an empty name, `.`, `..` or a name with a `/`.
+
+The command checks the config as `start` does, and exits with an error before it changes anything when the config is not valid (for example a relative `[backup] dir`).
+
+`trafic destroy` runs this command over SSH.
+
+### Scheduled removals
+
+`trafic-agent destroy <name> --delay <duration>` removes nothing. It records the removal date (now plus the delay) in the agent database and exits. The delay uses hours, minutes and seconds, in this order and without spaces: `48h`, `1h30m`, `90m`. The command refuses any other delay, for example `2d`, `1h30` or `48h 30m`. With `--backup`, the agent backs up the project before the scheduled removal, and keeps `--timeout` as the time limit of that backup. A new `--delay` for the same project replaces the date, the backup choice and the timeout, unless the removal has started.
+
+The agent service checks the scheduled removals every `idle_check_interval` (default `5m`). It removes each project whose date is past with the same steps as `trafic-agent destroy`. When a removal fails (for example, the backup fails), the agent logs the error, keeps the removal scheduled and tries again on the next check. A removal that fails after its backup (for example, the service stops during `ddev delete`) does not back up again on the next check: the backup is already made. A failure on one project does not stop the removal of the others.
+
+The service makes the backup of a scheduled removal itself, inside its systemd sandbox (`ProtectSystem=strict`). The sandbox lets it write to `/var/backups/trafic`, `/home/ddev` and `/var/lib/trafic` only. When `[backup] dir` is another directory, each scheduled backup fails with a `Cannot write to the backup directory` error that names the sandbox, and the project is never removed. Keep `[backup] dir` inside one of these directories.
+
+`trafic deploy` cancels the scheduled removal of the project it deploys. To cancel it by hand, run `trafic-agent destroy <name> --cancel`. A cancel stops a removal that is due but not started, or that is still in its backup. After the backup, the agent marks the removal as started, just before `ddev delete`. From then on, `--cancel` fails with `the removal of <name> is in progress` and the project is removed, so a `trafic deploy` fails at its first step instead of deploying a project that is about to go. `trafic-agent destroy --list` shows the scheduled removals, soonest first, and marks a started removal as `in progress`.
+
+The date is kept in the agent database, not in `.ddev/config.trafic.yaml`: the agent finds it even when DDEV no longer knows the project, and a deploy cancels it with one command.
+
+### Backups
+
+`trafic-agent destroy --backup` writes a backup to `<backup.dir>/<project>/<timestamp>/`, for example `/var/backups/trafic/preview-42--my-app/2026-10-14T07-40-24Z/`:
+
+- `db.sql.gz` — the `ddev export-db` dump. Skipped when the project has no database (`omit_containers: [db]`)
+- `files.tar.gz` — the upload directories, with paths relative to the project directory. Skipped when the project has none
+
+The upload directories are the ones DDEV uses: `upload_dirs` from the project's `.ddev/config.yaml`, or the default of the project type (for example `wp-content/uploads` for WordPress). The agent reads them from `DDEV_FILES_DIRS` in the web container. Both steps need the project to run, so the agent starts a stopped project and stops it again after the backup. While it removes a project, the idle sweep and [`max_running_projects`](#running-project-limit) do not stop it, and the waiting page does not start it again (it shows an error page): the project has the status `destroying` in the agent database. When the removal is killed (for example, the CI job is cancelled or the SSH connection drops), the agent sets the status back to `running` once the `trafic-agent destroy` process is gone, at the next idle sweep or request. The idle sweep then stops the project when it is idle.
+
+The backup has one time limit for the dump and the archive together: 40 minutes by default. `--timeout` sets it, as a duration such as `90s`, `10m` or `1h` (a bare number is minutes). The start and the stop of the project are not part of it. A backup that takes longer fails, and nothing is removed. `trafic destroy --backup` passes its own `--timeout` (default `10m`). A file that changes while the agent archives it (the project runs during the backup) does not fail the backup: the archive has the file as `tar` read it, and the agent prints a warning.
+
+The agent service removes backups older than `retain_days` once an hour. It can only write to `/var/backups/trafic`, `/home/ddev` and `/var/lib/trafic`: if you set `dir` to another path, remove old backups yourself, and do not use `--delay` with `--backup` (see [Scheduled removals](#scheduled-removals)). `dir` must not be `projects_dir`, be inside it or contain it (so not `/home/ddev` itself, with the default `projects_dir`), because the removal of a project would remove its backup: the agent refuses such a config. Setup creates `/var/backups/trafic`, owned by `ddev` with mode `700`.
+
+To restore a backup into a project:
+
+```bash
+ddev import-db --file=/var/backups/trafic/<project>/<timestamp>/db.sql.gz
+tar -xzf /var/backups/trafic/<project>/<timestamp>/files.tar.gz -C <project directory>
+```
+
 ### `trafic-agent upgrade` / `trafic-agent update`
 
 Upgrade the server to the latest version of `trafic-agent` in one command. `update` is an alias for `upgrade`.
@@ -81,7 +191,9 @@ Migration state is stored in `/etc/trafic/.migrations.json` and updated after ea
 
 ## Configuration
 
-Create `/etc/trafic/config.toml`:
+Setup writes `/etc/trafic/config.toml`. Edit it, then run `sudo systemctl restart trafic-agent`. See [examples/config.toml](../../examples/config.toml) for every key.
+
+Without `--config`, the agent reads the first file it finds of `/etc/trafic/config.toml`, `./config.toml` and `./trafic.toml`. When there is none, it uses the defaults.
 
 ```toml
 # Required: TLD for DDEV projects
@@ -90,12 +202,26 @@ tld = "previews.example.com"
 # Agent HTTP server port (default: 9876)
 port = 9876
 
-# Scale-to-zero: stop idle projects after this duration
+# Must match trafic deploy --projects-dir (default: /home/ddev/www)
+projects_dir = "/home/ddev/www"
+
+# Scale-to-zero: stop idle projects after this duration (default: 30m; setup writes 4h)
 idle_timeout = "4h"
 
+# Idle, running-limit and scheduled-removal sweeps (default: 5m)
+idle_check_interval = "5m"
+
+# Most projects running at the same time (default: 0, no limit)
+max_running_projects = 8
+
+# Backups made by `trafic-agent destroy --backup`
+[backup]
+dir = "/var/backups/trafic"  # default
+retain_days = 14             # default; older backups are removed
+
 # Authentication
 [auth]
-default_policy = "basic"  # allow, deny, basic, or token
+default_policy = "basic"  # allow, deny, basic, or token; basic also accepts the tokens
 
 # Proxies in front of the agent (default: 1 = ddev-router/Traefik alone).
 # Raise by one for each extra proxy: a CDN or load balancer ahead of
@@ -109,10 +235,10 @@ allowed_ips = ["192.168.1.0/24", "10.0.0.0/8"]
 # Bearer tokens for CI/API access
 tokens = ["your-ci-token"]
 
-# Basic auth credentials
+# Basic auth credentials, as "user:password" or { username, password } tables
 basic_auth = ["user:password"]
 
-# Per-hostname rules
+# Per-hostname rules (first match wins, after the global allowed_ips)
 [[auth.rules]]
 match = "*.public.*"
 policy = "allow"
@@ -120,6 +246,7 @@ policy = "allow"
 [[auth.rules]]
 match = "admin.*"
 policy = "basic"
+allowed_ips = ["203.0.113.0/24"]  # optional: these IPs skip basic auth
 
 # Optional: one wildcard certificate instead of one per preview.
 # See the TLS section below.
@@ -142,6 +269,20 @@ auth_policy: allow  # allow, deny, basic, or token
 idle_timeout: never
 ```
 
+- `idle_timeout` replaces the global `idle_timeout` for this project, shorter or longer. The idle sweep reads it at each check.
+- `auth_policy` replaces only `default_policy`. The global `allowed_ips` and a matching `[[auth.rules]]` still apply first.
+- The agent reads `auth_policy` when it starts and when DDEV's `project_list.yaml` changes. After you change it, restart the agent: `sudo systemctl restart trafic-agent`.
+
+## Running project limit
+
+`max_running_projects` limits how many projects run at the same time, so that a burst of traffic on many previews does not use all the memory of the server. Unset or `0` means no limit.
+
+- Before the waiting page starts a stopped project, the agent counts the running projects: the projects that `ddev list` shows up, and the projects the agent is starting. The project to start is one of them. Over the limit, it stops the running projects with the oldest access first. When many previews start at the same time, each start frees its own slot.
+- `trafic deploy` runs `ddev start` without the agent. The idle sweep (every `idle_check_interval`) stops the projects with the oldest access until the server is back under the limit. A project that a deploy started counts as accessed when the sweep first sees it, so it is not the first one stopped.
+- Projects with `idle_timeout: never` are never stopped by this rule. When no other project can be stopped, the agent starts the project anyway and logs a warning.
+- A project that `trafic-agent destroy` is removing (also from a scheduled removal) is never stopped by this rule or by the idle sweep: its backup starts the project, and a stop would fail the backup. It still counts as running.
+- A start that the agent service did not finish (the service restarted during `ddev start`) does not keep its slot: when the service starts again, it sets the project back to `running`, and the idle sweep checks it against DDEV.
+
 ## How it works
 
 ```
@@ -158,6 +299,7 @@ Traefik (DDEV router)
      └─► 502 error → errors middleware → trafic-agent
                           │
                           ├─► known project → Waiting page + auto-start
+                          ├─► project being removed → Error page (503)
                           └─► unknown → Error page
 ```
 
@@ -167,6 +309,7 @@ The agent:
 3. Tracks project activity for scale-to-zero
 4. Starts stopped projects when requested
 5. Shows a waiting page while projects start
+6. Removes projects whose [scheduled removal](#scheduled-removals) is due
 
 ## Endpoints
 
@@ -248,7 +391,7 @@ router container and starts one running project. DDEV regenerates the static
 config on any project start, but reads `router-compose.*.yaml` only when it
 recreates the router, so the removal is what gets the credentials into the
 container. The router is back within seconds. With no project running, the
-next deploy applies it. `trafic-agent audit` reports whether the certificate
+next deploy applies it. `sudo trafic-agent audit` reports whether the certificate
 was issued.
 
 The first order takes about a minute after the router starts: lego waits for

package/dist/cli.js

@@ -1,6 +1,6 @@
 #!/usr/bin/env node
-import { B as info, D as closeDb, G as success, K as warn, R as exec, V as isRoot, W as step, ct as validateConfig, ot as loadConfig, s as startServer, z as execSilent } from "./server-DZAo8Clg.js";
-import { i as runPendingMigrations, n as setup, r as listMigrations, s as startIdleScheduler, t as audit } from "./setup-DWgb6dAD.js";
+import { $ as info, A as cancelScheduledDestroy, Ct as validateConfig, M as closeDb, Q as execSilent, St as parseTimeout, Z as exec, at as warn, bt as loadConfig, et as isRoot, it as success, l as startServer, rt as step, z as getScheduledDestroys } from "./server-B_iFvS6v.js";
+import { _ as startIdleScheduler, d as destroyProject, f as loadDestroyConfig, g as startBackupSweepScheduler, i as runPendingMigrations, l as scheduleProjectDestroy, n as setup, p as openDestroyDb, r as listMigrations, s as startAccessLogSweepScheduler, t as audit, u as startScheduledDestroyScheduler } from "./setup-BbXL1ZFY.js";
 import { readFileSync } from "node:fs";
 import { execSync, spawnSync } from "node:child_process";
 import { parseArgs } from "node:util";
@@ -113,7 +113,7 @@ function upgradeDdev(dryRun) {
 /** The real collaborators, used unless a caller passes their own. */
 function nodeUpgradeIo() {
 	return {
-		currentVersion: "0.1.49",
+		currentVersion: "0.1.50",
 		alreadyReExeced: process.env["TRAFIC_UPGRADE_REEXEC"] === "1",
 		isRoot,
 		fetchLatestVersion,
@@ -192,6 +192,9 @@ function upgradeDdevStep(dryRun, io) {
 	else success("DDEV up to date");
 }
 //#endregion
+//#region ../../skills/trafic-agent/SKILL.md?raw
+var SKILL_default = "---\nname: trafic-agent\ndescription: Operate a Trafic preview server with trafic-agent (@studiometa/trafic-agent), the service that runs next to DDEV and Traefik. Covers setup flags, the trafic-agent systemd service and its logs, upgrade and migrations, audit, destroy (--backup, --timeout, --delay, --cancel, --list), the keys of /etc/trafic/config.toml (auth, idle timeout, max_running_projects, [backup], [tls]) and the per-project .ddev/config.trafic.yaml. Use when you install, configure, upgrade, debug or clean up a server that hosts DDEV previews deployed with trafic, or when you edit its config.toml.\nlicense: MIT\n---\n\n# trafic-agent\n\n`trafic-agent` runs on the server that hosts the DDEV previews. It does these jobs:\n\n- Forward authentication for Traefik: IP allowlist, bearer tokens, basic auth, rules per hostname.\n- Scale-to-zero: it stops idle projects, and starts a stopped project on its next request behind a waiting page.\n- A limit on the number of running projects (`max_running_projects`).\n- Project removal, with an optional backup, now or at a scheduled date.\n\nThis file can be older or newer than the agent on the server. `trafic-agent skill` prints the copy of this skill that matches the installed version: run it on the server and prefer its content when it differs.\n\n## Users and files\n\n| Item | Value |\n|---|---|\n| Service | `trafic-agent.service`, runs `trafic-agent start --config /etc/trafic/config.toml` as user `ddev` |\n| Config | `/etc/trafic/config.toml`, mode `640`, owner `root:ddev`. Edit it as root |\n| Database | `/var/lib/trafic/db.sqlite` (projects, access logs, scheduled removals), owned by `ddev` |\n| Migration state | `/etc/trafic/.migrations.json` |\n| Projects | `/home/ddev/www/<project>` (`projects_dir`) |\n| Backups | `/var/backups/trafic/<project>/<timestamp>/` (`[backup] dir`), owner `ddev`, mode `700` |\n| HTTP port | `9876` (`port`), open to the Docker bridge only |\n\nWhich user runs which command:\n\n- As root (or with `sudo`): `setup`, `upgrade`, `audit`, edits of `config.toml`, `systemctl`. The `ddev` user has no general sudo.\n- As `ddev`: `destroy`. It refuses to run as root, because DDEV does not run as root and root could leave files in the database directory that the service cannot write.\n- As any user: `upgrade --list`, `version`, `help`, `skill`.\n\n## Service and logs\n\n```bash\nsudo systemctl status trafic-agent\nsudo systemctl restart trafic-agent       # after each edit of config.toml\njournalctl -u trafic-agent -f             # follow the logs\njournalctl -u trafic-agent -n 200 --no-pager\ncurl -s http://127.0.0.1:9876/__health__  # agent health and version\n```\n\nThe unit has a systemd sandbox (`ProtectSystem=strict`). The service can write only to `/var/lib/trafic`, `/home/ddev` and `/var/backups/trafic`. When the config is not valid, `start` prints `Configuration errors:` with one line per error, exits with code 1, and systemd restarts it every 5 seconds. Read the journal, fix the config, restart.\n\n## `trafic-agent setup`\n\nRun as root on a fresh Ubuntu 24.04 server, or remotely with `trafic setup` from the CLI. It installs Docker, DDEV and Node.js, configures DDEV and Traefik for forward auth, writes `/etc/trafic/config.toml` (only when it does not exist), creates the systemd service and hardens the server (SSH, UFW, Fail2ban, unattended-upgrades, `DOCKER-USER` firewall rules). It asks no questions. A second run is safe: each step is skipped when the server is already in the target state.\n\n```bash\nsudo trafic-agent setup --tld=previews.example.com --email=admin@example.com\n```\n\n| Flag | Use |\n|---|---|\n| `--tld` | Domain of the previews. Required on the first run. A second run reads `tld` from the config |\n| `--email` | Let's Encrypt email. Without it, projects use a mkcert certificate |\n| `--dns-provider <lego name>` | One wildcard `*.<tld>` certificate through DNS-01. Requires `--email`, also on a second run that reads it from the config |\n| `--dns-env KEY=VALUE` | Credential of the DNS provider, repeatable |\n| `--ssh-users <list>` | Users that SSH allows, comma-separated. Default `ddev` |\n| `--trusted-proxy-hops <n>` | `2` when a CDN or load balancer is in front of Traefik. Written to a new config only |\n| `--no-hardening`, `--no-root-ssh`, `--no-docker`, `--no-ddev` | Skip or change these steps. With `--no-root-ssh`, only the rescue mode of the provider can recover root access |\n| `--dry-run` | Show what setup would do |\n\nSetup does not create the wildcard DNS record (`*.<tld>` to the server IP) and does not add SSH keys. Add CI keys to `/home/ddev/.ssh/authorized_keys`.\n\nA second run applies the flags you give it, not the flags of the first run. Pass the same `--email`, `--ssh-users`, `--no-root-ssh` and `--no-*` flags again: without `--email`, it turns Let's Encrypt off and deletes the issued certificates; without `--ssh-users`, SSH allows only `ddev`, root and the user who ran `sudo`. It reads only `tld` and `[tls]` from the config.\n\nWhen the config exists, setup does not change it. With a new `--dns-provider`, it prints the `[tls]` section to add by hand.\n\n## `trafic-agent upgrade`\n\n`update` is an alias.\n\n```bash\ntrafic-agent upgrade --list        # migrations and their status, no change\nsudo trafic-agent upgrade --dry-run\nsudo trafic-agent upgrade\n```\n\nSteps: install the latest `@studiometa/trafic-agent` from npm when it is newer, run the pending migrations, upgrade the `ddev` apt package (a failure is only a warning), restart `trafic-agent`.\n\n- Upgrade the agent **before** you raise the CLI version in CI. `trafic deploy` and `trafic destroy` call `trafic-agent destroy` on the server, and an older agent does not know the new options.\n- Migrations fix the state of existing servers. They run once per server, in order, and the state file records each one after it succeeds. A fresh `setup` marks all of them as applied.\n- DDEV is upgraded only here, never by unattended-upgrades, so that a new major version does not break previews without an operator.\n\n## `trafic-agent audit`\n\n```bash\nsudo trafic-agent audit\n```\n\nRun it as root. `ufw status` and the files under `/var/lib/docker` need root, so as another user the firewall and wildcard certificate checks fail. It checks SSH, the firewall, Fail2ban, automatic updates, the agent service, Docker, disk usage, the permissions of the config, the owner of the projects directory and, with a `[tls]` section, the wildcard certificate and the permissions of its credentials. It exits with code 1 when a check fails, and each failed check prints a fix.\n\n## `trafic-agent destroy`\n\nRun it as `ddev`. `trafic destroy` from the CLI runs the same command over SSH.\n\n```bash\ntrafic-agent destroy preview-42--my-app\ntrafic-agent destroy preview-42--my-app --backup\ntrafic-agent destroy preview-42--my-app --backup --timeout 1h\ntrafic-agent destroy preview-42--my-app --delay 48h --backup\ntrafic-agent destroy preview-42--my-app --cancel\ntrafic-agent destroy --list\n```\n\nSteps: backup (with `--backup`; if it fails, nothing is removed), `ddev delete -Oy` (a failure is only a warning), removal of the project directory, removal of the project's rows in the agent database.\n\n| Flag | Use |\n|---|---|\n| `-c`, `--config` | Config file. Default: first of `/etc/trafic/config.toml`, `./config.toml`, `./trafic.toml` |\n| `--backup` | Dump the database (`db.sql.gz`) and archive the upload directories (`files.tar.gz`) to `<backup dir>/<project>/<timestamp>/` first. The agent starts a stopped project for the backup and stops it again |\n| `--timeout <duration>` | Time limit of the dump and the archive together. Default `40m`. `10m`, `90s`, `1h`; a bare number is minutes |\n| `--delay <duration>` | Remove nothing now. Record a removal at now plus the delay. Hours, minutes and seconds, in that order, no spaces: `48h`, `1h30m`, `90m` |\n| `--cancel` | Cancel the scheduled removal of the project |\n| `--list` | List the scheduled removals, soonest first. `in progress` marks a removal that has started |\n\nSafety rules of the command:\n\n- It refuses a project directory outside `projects_dir`, also after it follows symbolic links, and a project directory that is a symbolic link. It refuses an empty name, `.`, `..` and names with `/`.\n- When the directory does not exist, it only deletes the database rows. So you can run it again after a failed removal.\n- It checks the config as `start` does, and stops before any change when the config is not valid.\n\nScheduled removals:\n\n- The service checks them every `idle_check_interval`, and removes each project whose date is past with the same steps.\n- A failed scheduled removal is logged in the journal and tried again at the next check. A removal that fails after its backup does not back up again.\n- `trafic deploy` cancels the scheduled removal of the project it deploys. After the backup, the agent marks the removal as started, just before `ddev delete`. From then on, `--cancel` and a deploy fail with `the removal of <project> is in progress`.\n- The service makes the backup of a scheduled removal inside its sandbox. If `[backup] dir` is outside `/var/backups/trafic`, `/home/ddev` and `/var/lib/trafic`, each scheduled backup fails with `Cannot write to the backup directory` and the project is never removed.\n\nRestore a backup:\n\n```bash\ncd <project directory>\nddev import-db --file=/var/backups/trafic/<project>/<timestamp>/db.sql.gz\ntar -xzf /var/backups/trafic/<project>/<timestamp>/files.tar.gz -C <project directory>\n```\n\n## `config.toml`\n\nEdit `/etc/trafic/config.toml` as root, then `sudo systemctl restart trafic-agent`. Values not in the file take their default.\n\n| Key | Default | Use |\n|---|---|---|\n| `tld` | none, required | Domain of the previews |\n| `port` | `9876` | Agent HTTP port |\n| `db_path` | `/var/lib/trafic/db.sqlite` | Agent database |\n| `project_list_path` | `/home/ddev/.ddev/project_list.yaml` | DDEV project list, watched for changes |\n| `projects_dir` | `/home/ddev/www` | Must be the same as `trafic deploy --projects-dir` (default `~/www`). `destroy` never removes a directory outside it |\n| `idle_timeout` | `30m` (setup writes `4h`) | Stop a project after this time without requests |\n| `idle_check_interval` | `5m` | Interval of the idle sweep, the running limit and the scheduled removals |\n| `max_running_projects` | `0` (no limit) | Most projects that run at the same time. See below |\n| `[backup] dir` | `/var/backups/trafic` | Absolute path. Must not be `projects_dir`, be inside it or contain it (so not `/home/ddev`) |\n| `[backup] retain_days` | `14` | The service removes older backups once an hour. Whole number, 1 or more |\n| `[auth] default_policy` | `basic` | `allow`, `deny`, `basic` or `token`. `basic` also accepts the tokens |\n| `[auth] trusted_proxy_hops` | `1` | Proxies in front of the agent. `2` with a CDN. Set it before `allowed_ips` |\n| `[auth] allowed_ips` | `[]` | IPs or CIDR ranges that skip authentication |\n| `[auth] tokens` | `[]` | Bearer tokens, sent as `Authorization: Bearer <token>` |\n| `[auth] basic_auth` | `[]` | `\"user:password\"` strings, or `{ username, password }` tables |\n| `[[auth.rules]]` | none | `match` (hostname glob), `policy`, optional `tokens` and `allowed_ips`. First match wins, after the global `allowed_ips` |\n| `[tls] dns_provider`, `ca_server`, `[tls.dns_env]` | none | Wildcard certificate through DNS-01. A restart does not apply a change: migration `0016__wildcard_dns_challenge` applies a first `[tls]` on a server set up before it (`sudo trafic-agent upgrade`); otherwise run `sudo trafic-agent setup` again with the flags of the first run |\n\nExample:\n\n```toml\ntld = \"previews.example.com\"\nprojects_dir = \"/home/ddev/www\"\nidle_timeout = \"4h\"\nidle_check_interval = \"5m\"\nmax_running_projects = 8\n\n[backup]\ndir = \"/var/backups/trafic\"\nretain_days = 14\n\n[auth]\ndefault_policy = \"basic\"\ntrusted_proxy_hops = 1\nallowed_ips = [\"203.0.113.0/24\"]\ntokens = [\"ci-token\"]\nbasic_auth = [\"preview:change-me\"]\n\n[[auth.rules]]\nmatch = \"*.public.*\"\npolicy = \"allow\"\n```\n\nValidation errors that stop `start` and `destroy`: `tld is required`, `port must be between 1 and 65535`, `max_running_projects must be a whole number, 0 or more`, `auth.default_policy must be ...`, `backup.dir must be an absolute path`, `backup.dir must not be projects_dir, be inside it or contain it`, `backup.retain_days must be a whole number of 1 or more`, and errors on `tls.dns_env` keys.\n\n## `max_running_projects`\n\n- Before the waiting page starts a stopped project, the agent counts the running projects and the projects it is starting. Over the limit, it stops the projects with the oldest access first.\n- `trafic deploy` starts projects without the agent. The idle sweep then stops the projects with the oldest access until the server is under the limit.\n- Projects with `idle_timeout: never` are never stopped by this rule. If no other project can stop, the agent starts the project anyway and logs a warning.\n- A project that `destroy` is removing is never stopped by this rule or by the idle sweep.\n\n## Per-project config: `.ddev/config.trafic.yaml`\n\nCommitted in the project repository. Flat `key: value` lines only.\n\n```yaml\nauth_policy: allow   # allow, deny, basic or token\nidle_timeout: never  # or a duration such as 2h\n```\n\n- `idle_timeout` replaces the global value for this project, shorter or longer. The idle sweep reads it at each check.\n- `auth_policy` replaces only `default_policy`. The global `allowed_ips` and a matching `[[auth.rules]]` still apply first. The agent reads it when it starts and when DDEV's `project_list.yaml` changes; after a change, restart the service.\n\n## Troubleshooting\n\n| Symptom | Check and fix |\n|---|---|\n| Service restarts in a loop | `journalctl -u trafic-agent -n 50`. Fix the `Configuration errors:` lines, then restart |\n| Preview asks for a password | `default_policy = \"basic\"`. Add the client IP to `allowed_ips`, a rule, or `auth_policy: allow` in the project |\n| `allowed_ips` does not match | `trusted_proxy_hops` is wrong. Use `2` behind a CDN |\n| Preview stays on the waiting page | `ddev list`, then `ddev start` in the project directory to see the DDEV error. Check `max_running_projects` and the journal |\n| Project stops too soon | Raise `idle_timeout`, or set `idle_timeout: never` in the project |\n| `cannot read ... (EACCES): connect as the ddev user` | Run `destroy` as `ddev`, not as another user |\n| `Refusing to remove ...: it is not inside the projects directory` | Set `projects_dir` to the directory that `trafic deploy` uses |\n| `The backup timed out ...` | Run again with a larger `--timeout`, for example `1h` |\n| `Cannot write to the backup directory` | Put `[backup] dir` in `/var/backups/trafic`, `/home/ddev` (not `projects_dir`) or `/var/lib/trafic`, owned by `ddev` |\n| Certificate warning on previews | `sudo trafic-agent audit`. Without `--email` at setup, projects use mkcert. Per-host certificates are limited to 50 new ones per domain per week; use `[tls]` for one wildcard |\n| Ports 8025, 8026, 8142, 8143 reachable | The `DOCKER-USER` rules are missing: `sudo iptables -S DOCKER-USER`, `sudo systemctl status trafic-docker-firewall` |\n\n## CI side\n\nFor the CI jobs that call this server (`trafic deploy`, `trafic destroy`), use the `trafic-cli` skill, or run `trafic skill` with the CLI version that CI uses.\n";
+//#endregion
 //#region src/cli.ts
 var HELP = `
 trafic-agent — Server agent for DDEV preview environments
@@ -202,14 +205,18 @@ Usage:
 Commands:
   start                   Start the agent server
   setup                   Setup a new server (Docker, DDEV, hardening)
-  upgrade                 Check for a new version, install it, run migrations, restart the service
+  upgrade                 Install the latest agent, run migrations, upgrade DDEV, restart the service
   update                  Alias for upgrade
   audit                   Run security audit checks
-  version                 Show version
-  help                    Show this help
+  destroy <name>          Remove a project: DDEV project, directory, database records,
+                          now or after a delay
+  skill                   Print the agent skill (SKILL.md) of this version
+  version                 Show version (also -v, --version)
+  help                    Show this help (also -h, --help)
 
 Start options:
-  -c, --config <path>     Path to config file (default: /etc/trafic/config.toml)
+  -c, --config <path>     Path to config file (default: first of /etc/trafic/config.toml,
+                          ./config.toml, ./trafic.toml)
   -p, --port <port>       Override port from config
 
 Setup options:
@@ -227,6 +234,20 @@ Setup options:
                           Written to the generated config; ignored if one exists
   --dry-run               Show what would be done without making changes
 
+Destroy options:
+  -c, --config <path>     Path to config file (default: first of /etc/trafic/config.toml,
+                          ./config.toml, ./trafic.toml)
+  --backup                Back up the database and upload directories first, to
+                          [backup] dir. If the backup fails, nothing is removed
+  --timeout <duration>    Time limit of the backup, the database dump and the files
+                          archive together: 10m, 90s, 1h; a bare number is minutes
+                          (default: 40m)
+  --delay <duration>      Remove nothing now: schedule the removal after this delay
+                          (hours, minutes, seconds only: 48h, 1h30m). The agent
+                          removes the project when it is due
+  --cancel                Cancel the scheduled removal of the project
+  --list                  List the scheduled removals
+
 Upgrade/update options:
   --dry-run               Show what would be done without making changes
   --list                  List all migrations and their status (no install or restart)
@@ -247,30 +268,42 @@ Examples:
 
   sudo trafic-agent setup --tld=previews.example.com --no-hardening --dry-run
 
-  # Upgrade: install latest version, run migrations, restart service
+  # Upgrade: install the latest agent, run migrations, upgrade DDEV, restart the service
   sudo trafic-agent upgrade
   sudo trafic-agent upgrade --dry-run
   trafic-agent upgrade --list
   sudo trafic-agent update          # alias for upgrade
 
   # Run security audit
-  trafic-agent audit
+  sudo trafic-agent audit
+
+  # Remove a project (as the ddev user, never as root)
+  trafic-agent destroy preview-42--my-app
+  trafic-agent destroy preview-42--my-app --backup
+  trafic-agent destroy preview-42--my-app --backup --timeout 1h
+  trafic-agent destroy preview-42--my-app --delay 48h --backup
+  trafic-agent destroy preview-42--my-app --cancel
+  trafic-agent destroy --list
 `;
 function printHelp() {
 	console.log(HELP.trim());
 }
 function printVersion() {
-	console.log("0.1.49");
+	console.log("0.1.50");
 }
-async function runStart(values) {
-	const config = loadConfig(values.config);
-	if (values.port) config.port = parseInt(values.port, 10);
+/** Print the configuration errors and exit with code 1, if there are any. */
+function exitOnConfigErrors(config) {
 	const errors = validateConfig(config);
 	if (errors.length > 0) {
 		console.error("Configuration errors:");
 		for (const error of errors) console.error(`  - ${error}`);
 		process.exit(1);
 	}
+}
+async function runStart(values) {
+	const config = loadConfig(values.config);
+	if (values.port) config.port = parseInt(values.port, 10);
+	exitOnConfigErrors(config);
 	const shutdown = () => {
 		console.log("\nShutting down...");
 		closeDb();
@@ -280,6 +313,75 @@ async function runStart(values) {
 	process.on("SIGTERM", shutdown);
 	startServer(config);
 	startIdleScheduler(config);
+	startBackupSweepScheduler(config);
+	startAccessLogSweepScheduler();
+	startScheduledDestroyScheduler(config);
+}
+/**
+* Remove a project. Runs as the ddev user, next to the running service: both
+* open the same database, so the config is read the same way `start` reads it.
+*/
+async function runDestroy(values, name) {
+	if (isRoot()) {
+		console.error("Error: run trafic-agent destroy as the ddev user, not root");
+		process.exit(1);
+	}
+	if (!name && !values.list) {
+		console.error("Error: a project name is required");
+		console.error("Example: trafic-agent destroy preview-42--my-app");
+		process.exit(1);
+	}
+	const timeout = values.timeout;
+	const timeoutMs = timeout === void 0 ? void 0 : parseTimeout(timeout);
+	if (timeout !== void 0 && timeoutMs === void 0) {
+		console.error(`Error: invalid --timeout "${timeout}". Expected a duration such as 10m, 90s or 1h`);
+		process.exit(1);
+	}
+	const backup = values.backup;
+	const delay = values.delay;
+	try {
+		const config = loadDestroyConfig(values.config);
+		exitOnConfigErrors(config);
+		openDestroyDb(config.dbPath);
+		if (values.list) printScheduledDestroys();
+		else if (values.cancel) cancelDestroy(name);
+		else if (delay !== void 0) {
+			const destroyAt = scheduleProjectDestroy(name, delay, config, {
+				backup,
+				timeoutMs
+			});
+			console.log(`Scheduled the removal of ${name} at ${destroyAt.toISOString()}${backup ? ", after a backup" : ""}`);
+		} else await destroyProject(name, config, {
+			backup,
+			timeoutMs
+		});
+	} catch (error) {
+		console.error(`Error: ${error.message}`);
+		process.exitCode = 1;
+	} finally {
+		closeDb();
+	}
+}
+/**
+* Cancel the scheduled removal of a project. Fails when the sweep already
+* removes the project, so a deploy that cancels first stops there.
+*/
+function cancelDestroy(name) {
+	const result = cancelScheduledDestroy(name);
+	if (result === "removing") throw new Error(`the removal of ${name} is in progress, it is too late to cancel it`);
+	console.log(result === "cancelled" ? `Cancelled the scheduled removal of ${name}` : `No removal scheduled for ${name}`);
+}
+/** Print the scheduled removals, soonest first. */
+function printScheduledDestroys() {
+	const scheduled = getScheduledDestroys();
+	if (scheduled.length === 0) {
+		console.log("No removal scheduled");
+		return;
+	}
+	for (const { name, destroyAt, backup, removing } of scheduled) {
+		const notes = [backup && "backup", removing && "in progress"].filter(Boolean);
+		console.log(`${new Date(destroyAt).toISOString()}  ${name}${notes.length ? `  (${notes.join(", ")})` : ""}`);
+	}
 }
 function handleUpgrade(values) {
 	const dryRun = values["dry-run"];
@@ -374,6 +476,10 @@ async function main() {
 				type: "boolean",
 				short: "h"
 			},
+			version: {
+				type: "boolean",
+				short: "v"
+			},
 			config: {
 				type: "string",
 				short: "c"
@@ -396,7 +502,11 @@ async function main() {
 			},
 			"trusted-proxy-hops": { type: "string" },
 			"dry-run": { type: "boolean" },
-			list: { type: "boolean" }
+			list: { type: "boolean" },
+			backup: { type: "boolean" },
+			timeout: { type: "string" },
+			delay: { type: "string" },
+			cancel: { type: "boolean" }
 		}
 	});
 	const command = positionals[0] ?? "help";
@@ -404,6 +514,10 @@ async function main() {
 		printHelp();
 		process.exit(0);
 	}
+	if (values.version) {
+		printVersion();
+		return;
+	}
 	switch (command) {
 		case "help":
 			printHelp();
@@ -411,6 +525,9 @@ async function main() {
 		case "version":
 			printVersion();
 			break;
+		case "skill":
+			process.stdout.write(SKILL_default);
+			break;
 		case "start":
 			await runStart(values);
 			break;
@@ -424,6 +541,9 @@ async function main() {
 		case "audit":
 			await audit();
 			break;
+		case "destroy":
+			await runDestroy(values, positionals[1]);
+			break;
 		default:
 			console.error(`Unknown command: ${command}`);
 			printHelp();