lts-backup-cli 2.0.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- lts_backup_cli/__init__.py +3 -0
- lts_backup_cli/__main__.py +3 -0
- lts_backup_cli/apache.py +45 -0
- lts_backup_cli/cli.py +559 -0
- lts_backup_cli/config.py +308 -0
- lts_backup_cli/doctor.py +278 -0
- lts_backup_cli/mariadb.py +157 -0
- lts_backup_cli/restic.py +122 -0
- lts_backup_cli/runner.py +172 -0
- lts_backup_cli/schedule.py +111 -0
- lts_backup_cli/services.py +604 -0
- lts_backup_cli/sshconfig.py +142 -0
- lts_backup_cli/validation.py +189 -0
- lts_backup_cli/wizard.py +419 -0
- lts_backup_cli-2.0.0.dist-info/METADATA +522 -0
- lts_backup_cli-2.0.0.dist-info/RECORD +19 -0
- lts_backup_cli-2.0.0.dist-info/WHEEL +4 -0
- lts_backup_cli-2.0.0.dist-info/entry_points.txt +2 -0
- lts_backup_cli-2.0.0.dist-info/licenses/LICENSE +29 -0
|
@@ -0,0 +1,522 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: lts-backup-cli
|
|
3
|
+
Version: 2.0.0
|
|
4
|
+
Summary: LTS backup utility: per-server restic backups of Apache vhosts, MariaDB databases and folders over SSH/SFTP, with discovery and scheduled rotation.
|
|
5
|
+
Author-email: LTS <joaomartins@lts.pt>
|
|
6
|
+
License: BSD-3-Clause
|
|
7
|
+
License-File: LICENSE
|
|
8
|
+
Keywords: apache,backup,cli,mariadb,mysql,restic,restore,sftp,systemd
|
|
9
|
+
Classifier: Development Status :: 4 - Beta
|
|
10
|
+
Classifier: Environment :: Console
|
|
11
|
+
Classifier: Intended Audience :: System Administrators
|
|
12
|
+
Classifier: License :: OSI Approved :: BSD License
|
|
13
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: Topic :: System :: Archiving :: Backup
|
|
21
|
+
Requires-Python: >=3.9
|
|
22
|
+
Requires-Dist: click<9,>=8.1
|
|
23
|
+
Requires-Dist: questionary<3,>=2.0
|
|
24
|
+
Requires-Dist: tomli-w>=1.0
|
|
25
|
+
Requires-Dist: tomli>=2.0; python_version < '3.11'
|
|
26
|
+
Provides-Extra: dev
|
|
27
|
+
Requires-Dist: behave>=1.2.6; extra == 'dev'
|
|
28
|
+
Requires-Dist: build>=1.0; extra == 'dev'
|
|
29
|
+
Requires-Dist: mypy>=1.8; extra == 'dev'
|
|
30
|
+
Requires-Dist: pytest-cov>=4; extra == 'dev'
|
|
31
|
+
Requires-Dist: pytest>=7; extra == 'dev'
|
|
32
|
+
Requires-Dist: ruff>=0.5; extra == 'dev'
|
|
33
|
+
Requires-Dist: twine>=5; extra == 'dev'
|
|
34
|
+
Description-Content-Type: text/markdown
|
|
35
|
+
|
|
36
|
+
# lts-backup-cli — `ltsbu`
|
|
37
|
+
|
|
38
|
+
`ltsbu` gives every **server** its own set of [restic](https://restic.net) backups, stored on SFTP
|
|
39
|
+
storage and made **incrementally on a schedule, with automatic rotation**:
|
|
40
|
+
|
|
41
|
+
- **one repo per Apache/httpd vhost**: its DocumentRoot
|
|
42
|
+
- **one database repo per server**: any number of MariaDB databases, each in its own snapshot
|
|
43
|
+
- **one repo per extra folder**: anything else on the server, such as `/etc/apache2`
|
|
44
|
+
|
|
45
|
+
`ltsbu discover <host>` logs in to a server over SSH, finds its vhosts and databases, lets you pick
|
|
46
|
+
what to back up and installs the systemd timers. Run it again whenever you want to change anything.
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
┌──── admin box (ltsbu) ────┐ ┌──────── web server ────────┐ ┌─── storage ───┐
|
|
50
|
+
│ config, restic passwords, │ ssh │ restic · mariadb-dump │ SFTP │ /srv/restic/ │
|
|
51
|
+
│ systemd timers │ ─────▶ │ vhost DocumentRoots │ ─────▶ │ <server>/ │
|
|
52
|
+
│ ltsbu discover/backup/... │ │ MariaDB (localhost) │ │ <repo>/ │
|
|
53
|
+
└───────────────────────────┘ └────────────────────────────┘ └───────────────┘
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
- **Central pull.** ltsbu runs on one admin box. Commands run on each web server over SSH, so the
|
|
57
|
+
web servers only need `restic`, the MariaDB client and an SSH login. If the admin box itself is
|
|
58
|
+
the server, use the host `localhost` to run everything locally.
|
|
59
|
+
- **Incremental, every 2 hours, plus a weekly run** at a staggered night slot. After each run ltsbu
|
|
60
|
+
rotates (`restic forget`), and the weekly run also prunes. Two backups never run at the same time.
|
|
61
|
+
- **Databases are streamed** (`mariadb-dump | restic backup --stdin`): no dump file is ever written to disk.
|
|
62
|
+
- **Verified before use.** The MariaDB admin login is checked during `discover` and saved on the
|
|
63
|
+
server (`0600`). `ltsbu doctor` re-checks it with every database, the paths, the storage route,
|
|
64
|
+
the repos and the timers.
|
|
65
|
+
- **Secrets never appear in a process list or a log.**
|
|
66
|
+
|
|
67
|
+
> **Upgrading from 0.x?** Version 2 is a clean break. The old model, one snapshot per label holding
|
|
68
|
+
> files plus a dump, is gone, and so is `ltsbu configure`. ltsbu refuses a v1 `config.toml` with a
|
|
69
|
+
> hint: move it aside and run `ltsbu discover <host>`. Old snapshots remain readable with plain restic.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## Contents
|
|
74
|
+
|
|
75
|
+
1. [Requirements](#requirements)
|
|
76
|
+
2. [Installation](#installation)
|
|
77
|
+
3. [Quick start](#quick-start)
|
|
78
|
+
4. [Commands](#commands): `discover` · `doctor` · `list` · `backup` · `snapshot` · `restore` · `forget` · `credentials`
|
|
79
|
+
5. [Common options](#common-options)
|
|
80
|
+
6. [How backups work](#how-backups-work)
|
|
81
|
+
7. [Schedule and rotation](#schedule-and-rotation)
|
|
82
|
+
8. [Files and configuration](#files-and-configuration)
|
|
83
|
+
9. [Managed SSH config blocks](#managed-ssh-config-blocks)
|
|
84
|
+
10. [Restoring](#restoring)
|
|
85
|
+
11. [Moving to another admin box / disaster recovery](#moving-to-another-admin-box--disaster-recovery)
|
|
86
|
+
12. [Troubleshooting](#troubleshooting)
|
|
87
|
+
13. [Development](#development)
|
|
88
|
+
14. [License](#license)
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## Requirements
|
|
93
|
+
|
|
94
|
+
| Where | Component | Version | Notes |
|
|
95
|
+
|---|---|---|---|
|
|
96
|
+
| admin box | OS | Rocky Linux 8.10, Ubuntu 22.04 | with systemd (timers) |
|
|
97
|
+
| admin box | Python | ≥ 3.9 | Rocky 8: `python39` package; Ubuntu 22.04: system 3.10 |
|
|
98
|
+
| admin box | OpenSSH client | any | key-based login to each server |
|
|
99
|
+
| web server | restic | ≥ 0.12 | Rocky 8: official release binary (not in EPEL 8); Ubuntu: `apt` (0.12.1) |
|
|
100
|
+
| web server | MariaDB client | 10.4+ | **only if it has a database repo**: `mariadb`/`mysql` and `mariadb-dump`/`mysqldump` |
|
|
101
|
+
| web server | OpenSSH client + bash | any | restic reaches the storage over SFTP |
|
|
102
|
+
| storage | SSH server with SFTP | any | the default on any Linux or NAS |
|
|
103
|
+
|
|
104
|
+
## Installation
|
|
105
|
+
|
|
106
|
+
On the **admin box**:
|
|
107
|
+
|
|
108
|
+
```bash
|
|
109
|
+
# Rocky Linux 8.10
|
|
110
|
+
sudo dnf install -y python39 python39-pip openssh-clients
|
|
111
|
+
python3.9 -m pip install --user lts-backup-cli
|
|
112
|
+
|
|
113
|
+
# Ubuntu 22.04
|
|
114
|
+
sudo apt install -y python3-pip openssh-client
|
|
115
|
+
python3 -m pip install --user lts-backup-cli
|
|
116
|
+
|
|
117
|
+
ltsbu --version # make sure ~/.local/bin is on PATH (or use pipx)
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
On every **web server**:
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
# Rocky Linux 8.10: restic is not packaged in EPEL 8, install the official static binary
|
|
124
|
+
curl -fsSL https://github.com/restic/restic/releases/download/v0.16.4/restic_0.16.4_linux_amd64.bz2 \
|
|
125
|
+
| bunzip2 | sudo tee /usr/local/bin/restic >/dev/null && sudo chmod 755 /usr/local/bin/restic
|
|
126
|
+
sudo dnf install -y mariadb openssh-clients # MariaDB client only for database backups
|
|
127
|
+
|
|
128
|
+
# Ubuntu 22.04
|
|
129
|
+
sudo apt install -y restic mariadb-client openssh-client
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
## Quick start
|
|
133
|
+
|
|
134
|
+
1. **SSH keys.** The admin box must log in to the web server, and the web server must log in to the
|
|
135
|
+
storage:
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
# on the admin box
|
|
139
|
+
ssh-keygen -t ed25519 -f ~/.ssh/id_ltsbu -N '' && ssh-copy-id -i ~/.ssh/id_ltsbu.pub root@web1.example.com
|
|
140
|
+
# on the web server
|
|
141
|
+
ssh-keygen -t ed25519 -f ~/.ssh/id_storage -N '' && ssh-copy-id -i ~/.ssh/id_storage.pub restic@nas.example.com
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
2. **Discover the server** (interactive):
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
ltsbu discover web1.example.com
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
The wizard asks for, or detects:
|
|
151
|
+
- the SSH login. A host already defined in your `~/.ssh/config` is used as is.
|
|
152
|
+
- the storage host, user, base path and the key **on the server**
|
|
153
|
+
- the **MariaDB admin ("sa") login**, if no working one is saved yet. It is verified and stored
|
|
154
|
+
`0600` on the server.
|
|
155
|
+
- **vhosts** to back up, one repo each (multi-select, all checked by default)
|
|
156
|
+
- **databases** for the server's `db` repo (multi-select)
|
|
157
|
+
- extra **folders**, one repo each
|
|
158
|
+
- optionally a custom schedule and retention per repo (default: every 2h + weekly, rotated)
|
|
159
|
+
- a summary, then: initialise the restic repos and install the timers
|
|
160
|
+
|
|
161
|
+
3. **Check everything:** `ltsbu doctor`
|
|
162
|
+
|
|
163
|
+
4. **Done.** Backups now run on their own. To run one by hand and look at it:
|
|
164
|
+
|
|
165
|
+
```bash
|
|
166
|
+
ltsbu backup --server web1 --all
|
|
167
|
+
ltsbu snapshot --server web1 --repo db
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
> **Keep the restic passwords safe.** Without them the repos can't be decrypted. They live in
|
|
171
|
+
> `~/.config/ltsbu/servers/<server>/<repo>.pass` on the admin box, with a copy on the server.
|
|
172
|
+
> Keep an offline copy with `ltsbu credentials export` (see [below](#moving-to-another-admin-box--disaster-recovery)).
|
|
173
|
+
|
|
174
|
+
## Commands
|
|
175
|
+
|
|
176
|
+
Every command accepts the [common options](#common-options), before or after the command name.
|
|
177
|
+
|
|
178
|
+
### `ltsbu discover HOST`
|
|
179
|
+
|
|
180
|
+
Creates or edits the backups of one server. `HOST` is a hostname or IP address, or a `Host` alias
|
|
181
|
+
from your `~/.ssh/config` (then ltsbu writes no SSH block for it), or `localhost`.
|
|
182
|
+
|
|
183
|
+
Running it again for the same server is **edit mode**:
|
|
184
|
+
- current choices are the defaults
|
|
185
|
+
- the vhosts' DocumentRoots are refreshed
|
|
186
|
+
- new repos get a password and a staggered schedule
|
|
187
|
+
- the server-side secrets and timers are refreshed
|
|
188
|
+
|
|
189
|
+
Repos you uncheck are only removed after an explicit confirmation (`--remove` when non-interactive).
|
|
190
|
+
Removing a repo stops its timers and drops it from the config, **but never deletes restic data**.
|
|
191
|
+
|
|
192
|
+
For automation, pass the answers as options with `--non-interactive`:
|
|
193
|
+
|
|
194
|
+
```bash
|
|
195
|
+
export LTSBU_DB_PASSWORD='…' # used with --db-user; never on the command line
|
|
196
|
+
ltsbu discover web1.example.com --non-interactive \
|
|
197
|
+
--ssh-user root --identity ~/.ssh/id_ltsbu \
|
|
198
|
+
--storage-host nas.example.com --storage-user restic --storage-path /srv/restic \
|
|
199
|
+
--storage-identity /root/.ssh/id_storage \
|
|
200
|
+
--db-user root --vhosts all --databases shop_db,blog_db --folders "/etc/apache2"
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
| Option | Default |
|
|
204
|
+
|---|---|
|
|
205
|
+
| `--server LABEL` | derived from the host (`web1` for `web1.example.com`) |
|
|
206
|
+
| `--ssh-user`, `--ssh-port`, `--identity` | *(required for a new server unless HOST is an ssh alias)*, `22`, none |
|
|
207
|
+
| `--storage-host`, `--storage-user` | *(required for a new server)* |
|
|
208
|
+
| `--storage-port`, `--storage-path`, `--storage-identity` | `22`, `/srv/restic`, the server's default keys |
|
|
209
|
+
| `--db-user` / `--no-db-login` | password from `$LTSBU_DB_PASSWORD` / skip databases |
|
|
210
|
+
| `--vhosts`, `--databases` | `all`, `none` or a comma-separated list; new server: all; edit: unchanged |
|
|
211
|
+
| `--folders "/a /b"` | none |
|
|
212
|
+
| `--every-hours N`, `--weekly-day DAY`, `--weekly-time HH:MM` | `2`, staggered night slot |
|
|
213
|
+
| `--keep-hourly/-daily/-weekly/-monthly N` | `12 / 7 / 4 / 6` |
|
|
214
|
+
| `--remove` | off: unchecked repos are kept |
|
|
215
|
+
| `--no-restic-init`, `--no-timers` | off: repos are initialised and timers installed |
|
|
216
|
+
|
|
217
|
+
### `ltsbu doctor`
|
|
218
|
+
|
|
219
|
+
Runs health checks and exits with **3** if any check fails. Remote checks are skipped for a server
|
|
220
|
+
that ltsbu can't log in to.
|
|
221
|
+
|
|
222
|
+
| Check | What it verifies |
|
|
223
|
+
|---|---|
|
|
224
|
+
| local binaries | `ssh` exists (`systemctl` missing is only a warning) |
|
|
225
|
+
| config lint / ssh config lint | values are valid, local secrets are `0600`, managed SSH blocks are present and balanced |
|
|
226
|
+
| dns / tcp | each server's name resolves and its SSH port answers |
|
|
227
|
+
| ssh login | non-interactive key login to the server (`BatchMode=yes`) |
|
|
228
|
+
| remote binaries | `restic` on the server; the MariaDB client only if it has a database repo |
|
|
229
|
+
| paths | each vhost/folder path exists on the server |
|
|
230
|
+
| MariaDB | the saved login file is `0600`, the login works, and **every configured database exists** |
|
|
231
|
+
| storage | the server reaches the storage over SSH, and every restic repo opens with its password |
|
|
232
|
+
| timers | both timers of every repo are installed and enabled; user timers warn if linger is off |
|
|
233
|
+
|
|
234
|
+
### `ltsbu list`
|
|
235
|
+
|
|
236
|
+
```
|
|
237
|
+
SERVER REPO KIND WHAT SCHEDULE REPOSITORY
|
|
238
|
+
web1 blog-example-com vhost /var/www/blog every 2h, weekly Tue 03:30 sftp:ltsbu-storage-web1:/srv/restic/web1/blog-example-com
|
|
239
|
+
web1 db db shop_db, blog_db every 2h, weekly Sat 01:00 sftp:ltsbu-storage-web1:/srv/restic/web1/db
|
|
240
|
+
web1 shop-example-com vhost /var/www/shop every 2h, weekly Thu 02:00 sftp:ltsbu-storage-web1:/srv/restic/web1/shop-example-com
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
### `ltsbu backup`
|
|
244
|
+
|
|
245
|
+
Backs up repos, then rotates them. This is what the timers run.
|
|
246
|
+
|
|
247
|
+
```bash
|
|
248
|
+
ltsbu backup --server web1 --repo shop-example-com
|
|
249
|
+
ltsbu backup --repo db # --server is optional when the repo name is unique
|
|
250
|
+
ltsbu backup --server web1 --all # every repo of web1; --all alone: every server
|
|
251
|
+
ltsbu backup --repo db --weekly # tag "weekly" and prune after forget
|
|
252
|
+
ltsbu backup --repo shop-example-com --include-git
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
In a database repo each database is backed up separately. If one fails, the others still run and the
|
|
256
|
+
command exits 1, naming the database that failed.
|
|
257
|
+
|
|
258
|
+
### `ltsbu snapshot`
|
|
259
|
+
|
|
260
|
+
Lists a repo's snapshots (`--json` for machine output), selected by the `server:<s>` and
|
|
261
|
+
`repo:<r>` tags, never by hostname.
|
|
262
|
+
|
|
263
|
+
### `ltsbu restore [SNAPSHOT]`
|
|
264
|
+
|
|
265
|
+
Restores **on the server**. `SNAPSHOT` is `latest` (the default) or an ID or ID prefix.
|
|
266
|
+
|
|
267
|
+
| Repo kind | Options | Action |
|
|
268
|
+
|---|---|---|
|
|
269
|
+
| vhost / folder | `--target DIR` | restore under `DIR` on the server; `--target /` restores in place (asks first) |
|
|
270
|
+
| db | `--database NAME [--into OTHER]` | stream the dump into `NAME`, or into `OTHER` which is created if missing (asks first) |
|
|
271
|
+
|
|
272
|
+
`--target` with a db repo, or `--database` with a files repo, is an error. Confirmation is skipped
|
|
273
|
+
with `--yes` and refused without it when there is no terminal.
|
|
274
|
+
|
|
275
|
+
### `ltsbu forget`
|
|
276
|
+
|
|
277
|
+
Rotation runs automatically after every backup. Use `forget` to apply a different policy once.
|
|
278
|
+
|
|
279
|
+
```bash
|
|
280
|
+
ltsbu forget --repo shop-example-com --yes
|
|
281
|
+
ltsbu forget --repo db --keep-daily 30 --prune --yes # any --keep-* replaces the whole policy
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
An empty policy (everything 0) is refused, because it would delete every snapshot.
|
|
285
|
+
|
|
286
|
+
### `ltsbu credentials export [FILE]` / `import FILE`
|
|
287
|
+
|
|
288
|
+
Moves servers from one admin box to another: config, restic passwords, the admin SSH keys and the
|
|
289
|
+
**sites' database users** (user, host, `mysql_native_password` hash and database-level grants, read
|
|
290
|
+
from each database in a db repo). `--server` limits the export to one server. `--no-db-users` leaves
|
|
291
|
+
the users out.
|
|
292
|
+
|
|
293
|
+
`import` validates everything **before** writing anything. It:
|
|
294
|
+
- writes the secrets `0600` to this box's locations
|
|
295
|
+
- adds the SSH blocks
|
|
296
|
+
- recreates the database users on the server, building the SQL from validated fields and sending it on stdin
|
|
297
|
+
|
|
298
|
+
A server that already exists here needs `--force`, and then **keeps its local connection settings**
|
|
299
|
+
(see [disaster recovery](#moving-to-another-admin-box--disaster-recovery)).
|
|
300
|
+
|
|
301
|
+
> **The bundle is the key to your backups.** It can decrypt every snapshot and delete the repos.
|
|
302
|
+
> Pipe it over SSH (`ltsbu credentials export | ssh admin2 ltsbu credentials import -`), or
|
|
303
|
+
> `shred -u` every copy.
|
|
304
|
+
|
|
305
|
+
## Common options
|
|
306
|
+
|
|
307
|
+
| Option | Meaning |
|
|
308
|
+
|---|---|
|
|
309
|
+
| `--server LABEL`, `-s` | server to act on; optional when only one matches |
|
|
310
|
+
| `--repo NAME`, `-r` | repo on that server; optional when only one matches |
|
|
311
|
+
| `--dry-run` | print the commands that would change something instead of running them; read-only commands still run |
|
|
312
|
+
| `--verbose`, `-v` | log every external command (secrets are never on a command line) |
|
|
313
|
+
| `--config FILE` | alternative config file (default `~/.config/ltsbu/config.toml`); put it before the command |
|
|
314
|
+
|
|
315
|
+
Exit codes: `0` success, `1` error, `2` invalid usage, `3` doctor found failures.
|
|
316
|
+
|
|
317
|
+
## How backups work
|
|
318
|
+
|
|
319
|
+
- **Files** (vhost and folder repos):
|
|
320
|
+
|
|
321
|
+
```
|
|
322
|
+
ssh <server> restic -r sftp:ltsbu-storage-<server>:<path>/<server>/<repo> backup --host <server> --tag … -- <paths>
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
Nested `.git` directories are excluded unless the repo has `exclude_git = false`.
|
|
326
|
+
- **Databases** (db repo), one database at a time:
|
|
327
|
+
|
|
328
|
+
```
|
|
329
|
+
ssh <server> bash -o pipefail -c 'mariadb-dump --defaults-extra-file=<0600 my.cnf> --single-transaction --quick --routines --triggers <db> | restic backup --stdin --stdin-filename <db>.sql --tag db:<db> …'
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
The fixed file name `<db>.sql` makes each database its own restic group, so incremental dedup, parent
|
|
333
|
+
detection and rotation all work per database. The dump has no `CREATE DATABASE`, so it can be loaded
|
|
334
|
+
under any name.
|
|
335
|
+
- **Tags:** `ltsbu`, `server:<server>`, `repo:<repo>`, `db:<db>`, and `weekly` on weekly runs.
|
|
336
|
+
- **One at a time:** a lock (`~/.cache/ltsbu/backup.lock`) makes overlapping timer runs queue
|
|
337
|
+
instead of loading the servers in parallel.
|
|
338
|
+
- Every argument sent over SSH is shell-quoted. Secrets reach the server **on stdin** and are written
|
|
339
|
+
`0600` with `umask 077`. They never appear on a command line.
|
|
340
|
+
|
|
341
|
+
## Schedule and rotation
|
|
342
|
+
|
|
343
|
+
Each repo gets two systemd timers on the admin box:
|
|
344
|
+
|
|
345
|
+
| Timer | Default | Runs |
|
|
346
|
+
|---|---|---|
|
|
347
|
+
| `ltsbu-<server>-<repo>.timer` | `OnCalendar=*-*-* 00/2:MM:00`, where `MM` is a per-repo minute | `backup`, then `forget` |
|
|
348
|
+
| `ltsbu-<server>-<repo>-weekly.timer` | one night slot between 01:00 and 04:30, a different day/time for each repo on the same storage | `backup --weekly`, then `forget --prune` |
|
|
349
|
+
|
|
350
|
+
All timers have `Persistent=true`, so a run missed while the box was off happens at boot, and
|
|
351
|
+
`RandomizedDelaySec=5min`. The default keep policy is `hourly 12, daily 7, weekly 4, monthly 6`. It
|
|
352
|
+
uses restic's `--keep-*` options and can be changed per repo in `discover`. While a policy is not yet
|
|
353
|
+
full, restic ≥ 0.14 also keeps the oldest snapshot (restic 0.12 does not), so young repos may show
|
|
354
|
+
one snapshot more than you expect.
|
|
355
|
+
|
|
356
|
+
As a normal user, units go to `~/.config/systemd/user` and need `loginctl enable-linger $USER` to
|
|
357
|
+
run while you are logged out (doctor warns). As root they go to `/etc/systemd/system`.
|
|
358
|
+
|
|
359
|
+
## Files and configuration
|
|
360
|
+
|
|
361
|
+
| Where | Path | Mode | Contents |
|
|
362
|
+
|---|---|---|---|
|
|
363
|
+
| admin box | `~/.config/ltsbu/config.toml` | 0600 | servers and repos (no secrets) |
|
|
364
|
+
| admin box | `~/.config/ltsbu/servers/<server>/<repo>.pass` | 0600 | restic password (master copy) |
|
|
365
|
+
| admin box | `~/.config/systemd/user/ltsbu-*.{service,timer}` | | timers |
|
|
366
|
+
| server | `~/.config/ltsbu/my.cnf` | 0600 | MariaDB admin login (`[client]`, `host=localhost`) |
|
|
367
|
+
| server | `~/.config/ltsbu/<repo>.pass` | 0600 | restic password used on the server |
|
|
368
|
+
|
|
369
|
+
Example `config.toml`:
|
|
370
|
+
|
|
371
|
+
```toml
|
|
372
|
+
version = 2
|
|
373
|
+
|
|
374
|
+
[servers.web1]
|
|
375
|
+
ssh_host = "web1.example.com"
|
|
376
|
+
ssh_user = "root"
|
|
377
|
+
ssh_port = 22
|
|
378
|
+
ssh_identity = "/home/admin/.ssh/id_ltsbu"
|
|
379
|
+
secrets_dir = "/root/.config/ltsbu"
|
|
380
|
+
db_login = true
|
|
381
|
+
|
|
382
|
+
[servers.web1.storage]
|
|
383
|
+
host = "nas.example.com"
|
|
384
|
+
user = "restic"
|
|
385
|
+
port = 22
|
|
386
|
+
base_path = "/srv/restic"
|
|
387
|
+
identity = "/root/.ssh/id_storage"
|
|
388
|
+
|
|
389
|
+
[servers.web1.repos.db]
|
|
390
|
+
kind = "db"
|
|
391
|
+
databases = ["blog_db", "shop_db"]
|
|
392
|
+
password_file = "/home/admin/.config/ltsbu/servers/web1/db.pass"
|
|
393
|
+
|
|
394
|
+
[servers.web1.repos.db.schedule]
|
|
395
|
+
every_hours = 2
|
|
396
|
+
minute = 17
|
|
397
|
+
weekly_day = "Sat"
|
|
398
|
+
weekly_time = "01:00"
|
|
399
|
+
|
|
400
|
+
[servers.web1.repos.db.keep]
|
|
401
|
+
hourly = 12
|
|
402
|
+
daily = 7
|
|
403
|
+
weekly = 4
|
|
404
|
+
monthly = 6
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
The MariaDB admin login needs `SELECT`, `LOCK TABLES`, `SHOW VIEW` and `TRIGGER` for backups. Restores
|
|
408
|
+
also need `CREATE`, `DROP`, `INSERT`, `ALTER` and `INDEX`, and `credentials import` needs `CREATE USER`
|
|
409
|
+
and `GRANT OPTION`. On a Virtualmin server, use `root`.
|
|
410
|
+
|
|
411
|
+
## Managed SSH config blocks
|
|
412
|
+
|
|
413
|
+
ltsbu edits two `~/.ssh/config` files, and only between its tag comments:
|
|
414
|
+
|
|
415
|
+
- **On the admin box**, a block `ltsbu:<server>` with `Host ltsbu-<server>` for the login to the
|
|
416
|
+
server. None is written when `HOST` is your own alias.
|
|
417
|
+
- **On the server**, a block `ltsbu:storage-<server>` with `Host ltsbu-storage-<server>` for restic's
|
|
418
|
+
SFTP route to the storage.
|
|
419
|
+
|
|
420
|
+
```sshconfig
|
|
421
|
+
# >>> ltsbu:storage-web1 >>>
|
|
422
|
+
# Managed by lts-backup-cli. Edits inside this block are overwritten.
|
|
423
|
+
Host ltsbu-storage-web1
|
|
424
|
+
HostName nas.example.com
|
|
425
|
+
User restic
|
|
426
|
+
Port 22
|
|
427
|
+
IdentityFile /root/.ssh/id_storage
|
|
428
|
+
IdentitiesOnly yes
|
|
429
|
+
StrictHostKeyChecking accept-new
|
|
430
|
+
ServerAliveInterval 60
|
|
431
|
+
ServerAliveCountMax 240
|
|
432
|
+
# <<< ltsbu:storage-web1 <<<
|
|
433
|
+
```
|
|
434
|
+
|
|
435
|
+
Each edit is atomic and first saves a timestamped backup, `config.ltsbu-<timestamp>.bak`.
|
|
436
|
+
`StrictHostKeyChecking accept-new` trusts a host's key on first contact and rejects a changed key
|
|
437
|
+
afterwards, so unattended runs work on fresh machines.
|
|
438
|
+
|
|
439
|
+
## Restoring
|
|
440
|
+
|
|
441
|
+
```bash
|
|
442
|
+
ltsbu snapshot --repo shop-example-com # what's there?
|
|
443
|
+
ltsbu restore --repo shop-example-com --target /tmp/shop-restore # files -> /tmp/shop-restore/var/www/shop
|
|
444
|
+
ltsbu restore 4f2a9c1e --repo shop-example-com --target / --yes # in place, from a specific snapshot
|
|
445
|
+
ltsbu restore --repo db --database shop_db --into shop_check --yes # inspect a copy
|
|
446
|
+
ltsbu restore --repo db --database shop_db --yes # put it back
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
Plain restic works too, on the server:
|
|
450
|
+
`restic -r sftp:ltsbu-storage-web1:/srv/restic/web1/db --password-file ~/.config/ltsbu/db.pass snapshots`.
|
|
451
|
+
|
|
452
|
+
restic restores numeric uid/gid. If the site's unix user has a different uid on a rebuilt server,
|
|
453
|
+
fix ownership with `chown -R`.
|
|
454
|
+
|
|
455
|
+
## Moving to another admin box / disaster recovery
|
|
456
|
+
|
|
457
|
+
**New admin box, same servers:**
|
|
458
|
+
|
|
459
|
+
```bash
|
|
460
|
+
ltsbu credentials export | ssh admin2 ltsbu credentials import -
|
|
461
|
+
ssh admin2 'ltsbu discover web1.example.com && ltsbu doctor' # re-installs the timers there
|
|
462
|
+
```
|
|
463
|
+
|
|
464
|
+
Disable the timers on the old admin box afterwards (`systemctl --user disable --now 'ltsbu-*'`), so
|
|
465
|
+
the two boxes never run the same backups.
|
|
466
|
+
|
|
467
|
+
**Rebuilt or replacement web server** (same label, new machine):
|
|
468
|
+
|
|
469
|
+
1. `ltsbu discover new-web1.example.com --server web1`. This re-points `web1`, writes the server-side
|
|
470
|
+
secrets and the storage route, and keeps the repos and their passwords.
|
|
471
|
+
2. `ltsbu credentials import --force bundle.toml` (from an earlier export), if the sites' database
|
|
472
|
+
users must be recreated. The local connection settings from step 1 are kept.
|
|
473
|
+
3. `ltsbu restore --repo db --database shop_db --yes`, and `ltsbu restore --repo <vhost> --target / --yes`.
|
|
474
|
+
|
|
475
|
+
## Troubleshooting
|
|
476
|
+
|
|
477
|
+
| Symptom | Fix |
|
|
478
|
+
|---|---|
|
|
479
|
+
| `cannot log in to ltsbu-<server>` | `ssh-copy-id` your key to the server, then `ssh ltsbu-<server> true` |
|
|
480
|
+
| doctor: **storage ssh FAIL** | on the server: `ssh ltsbu-storage-<server> true`; authorise the server's key on the storage |
|
|
481
|
+
| doctor: **mariadb login FAIL** | re-run `ltsbu discover <host>` and enter the admin login again |
|
|
482
|
+
| doctor: **database X FAIL** `database not found` | the database was dropped or renamed: re-run `discover` and update the selection |
|
|
483
|
+
| doctor: **timers FAIL** | re-run `ltsbu discover <host>` (timers are refreshed); check `systemctl --user list-timers 'ltsbu-*'` |
|
|
484
|
+
| doctor: **systemd linger WARN** | `loginctl enable-linger $USER` |
|
|
485
|
+
| `v1 config detected` | ltsbu 2 is a clean break: move the old config aside and run `ltsbu discover <host>` |
|
|
486
|
+
| `Access denied … LOCK TABLES` during a backup | grant the admin login `LOCK TABLES` and `SHOW VIEW` |
|
|
487
|
+
| `refusing without --yes (no terminal)` | add `--yes` in scripts |
|
|
488
|
+
| Ubuntu: a restic flag is unsupported | Ubuntu's restic 0.12 is old: `sudo restic self-update` |
|
|
489
|
+
|
|
490
|
+
`--verbose` shows every command ltsbu runs.
|
|
491
|
+
|
|
492
|
+
---
|
|
493
|
+
|
|
494
|
+
## Development
|
|
495
|
+
|
|
496
|
+
```bash
|
|
497
|
+
python3 -m venv .venv && . .venv/bin/activate
|
|
498
|
+
pip install -e '.[dev]'
|
|
499
|
+
|
|
500
|
+
ruff check . && ruff format --check . && mypy src # lint + types
|
|
501
|
+
pytest -q --cov=lts_backup_cli # unit
|
|
502
|
+
behave --tags=~@e2e # BDD acceptance (no network)
|
|
503
|
+
|
|
504
|
+
# Smoke: the built wheel in a clean venv
|
|
505
|
+
python -m build --wheel --outdir /tmp/dist . && python -m venv /tmp/smoke \
|
|
506
|
+
&& /tmp/smoke/bin/pip install /tmp/dist/*.whl \
|
|
507
|
+
&& PATH=/tmp/smoke/bin:$PATH PYTHON=/tmp/smoke/bin/python sh tests/smoke/smoke.sh
|
|
508
|
+
|
|
509
|
+
# E2E: real SSH, restic, MariaDB 10.4 and SFTP on both supported distros (smoke runs first)
|
|
510
|
+
docker compose -f tests/docker/docker-compose.yml run --rm --build rocky8
|
|
511
|
+
docker compose -f tests/docker/docker-compose.yml run --rm --build ubuntu2204
|
|
512
|
+
docker compose -f tests/docker/docker-compose.yml down && rm -rf tests/docker/.e2e
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
Design principles, the testing strategy and the Bitbucket CI/CD pipeline are described in
|
|
516
|
+
[CLAUDE.md](CLAUDE.md). The requirements are in [docs/SPEC2.md](docs/SPEC2.md) (v2) and
|
|
517
|
+
[docs/SPEC.md](docs/SPEC.md) (v1). Pushing a `vX.Y.Z` tag that matches `__version__` publishes the
|
|
518
|
+
package to PyPI.
|
|
519
|
+
|
|
520
|
+
## License
|
|
521
|
+
|
|
522
|
+
BSD 3-Clause. Copyright (c) 2026, joaomartins@logitools.pt and LTS, Lda. See [LICENSE](LICENSE).
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
lts_backup_cli/__init__.py,sha256=1wJa_zDxVQV1XAkMEkXIPy7q0jhxbQGwkSYgrNuolGQ,78
|
|
2
|
+
lts_backup_cli/__main__.py,sha256=bYt9eEaoRQWdejEHFD8REx9jxVEdZptECFsV7F49Ink,30
|
|
3
|
+
lts_backup_cli/apache.py,sha256=E3EGKxJhnOZj1KkrVl6s0KLS1hHK9A2bAuzurF9Sejk,1732
|
|
4
|
+
lts_backup_cli/cli.py,sha256=TG5olfkk4RfQ4Fj-x4C5Yg0gCYNmcJ145DeDezFVgk0,20781
|
|
5
|
+
lts_backup_cli/config.py,sha256=nHUBkYoSjBXf4LUsP8_kOlhkL8V6I4aKIWjfG4R5nCE,10187
|
|
6
|
+
lts_backup_cli/doctor.py,sha256=Xpfa0NRBWkealLqkxPvgUw0A9rqL548MTXhRb0uP9YI,9910
|
|
7
|
+
lts_backup_cli/mariadb.py,sha256=fkImT8LBAokGbKR9MCwP0vpGZDV6aLz2ejY4xYDtZFo,6841
|
|
8
|
+
lts_backup_cli/restic.py,sha256=f5j8UzRIVrr371F9okx1SURceUXUa-YUgbnpQwUfyLA,4017
|
|
9
|
+
lts_backup_cli/runner.py,sha256=yrlrTBTlessi4-N-MrSsKnebei7Q6hhCAnw5dSGdiWk,5246
|
|
10
|
+
lts_backup_cli/schedule.py,sha256=2ZNkES5niGreEYsWwCDXkhXmJa6Nx_Htda46Ovp6NlQ,4164
|
|
11
|
+
lts_backup_cli/services.py,sha256=VBC_K40s-Me6hOjFhUD27lu9YchPbzjA6Ynk-gTaP3w,24719
|
|
12
|
+
lts_backup_cli/sshconfig.py,sha256=XrPwdY2pxSZYSPoJ_7xvU0qTUQF2nOjXvKHC8NUNVpk,4475
|
|
13
|
+
lts_backup_cli/validation.py,sha256=KT0D-YDt_CnUT5wjgsPAuM8WeBvLRiBIZGF3OTqgHMU,6663
|
|
14
|
+
lts_backup_cli/wizard.py,sha256=W7BEVbYcdVJRyLeUZ_TGLnsJgzQRTf3YAFlaxwI9VzQ,15998
|
|
15
|
+
lts_backup_cli-2.0.0.dist-info/METADATA,sha256=hLcGKrgHaeccoMQwovGVpyXo5BkAft74vXrbGfQLny0,23890
|
|
16
|
+
lts_backup_cli-2.0.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
|
|
17
|
+
lts_backup_cli-2.0.0.dist-info/entry_points.txt,sha256=7ioQfvD_zedwcOfaxP1q9ETs3Bsjdz3PJr1ZvVBSudc,50
|
|
18
|
+
lts_backup_cli-2.0.0.dist-info/licenses/LICENSE,sha256=mZaiHrvtRpGQYJBJCg8t2dyqoBCoGo8ixbSQ91yA3qU,1546
|
|
19
|
+
lts_backup_cli-2.0.0.dist-info/RECORD,,
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
BSD 3-Clause License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026, joaomartins@logitools.pt and LTS, Lda.
|
|
4
|
+
All rights reserved.
|
|
5
|
+
|
|
6
|
+
Redistribution and use in source and binary forms, with or without
|
|
7
|
+
modification, are permitted provided that the following conditions are met:
|
|
8
|
+
|
|
9
|
+
1. Redistributions of source code must retain the above copyright notice, this
|
|
10
|
+
list of conditions and the following disclaimer.
|
|
11
|
+
|
|
12
|
+
2. Redistributions in binary form must reproduce the above copyright notice,
|
|
13
|
+
this list of conditions and the following disclaimer in the documentation
|
|
14
|
+
and/or other materials provided with the distribution.
|
|
15
|
+
|
|
16
|
+
3. Neither the name of the copyright holder nor the names of its
|
|
17
|
+
contributors may be used to endorse or promote products derived from
|
|
18
|
+
this software without specific prior written permission.
|
|
19
|
+
|
|
20
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
|
21
|
+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
22
|
+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
23
|
+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
|
24
|
+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
25
|
+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
|
26
|
+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
|
27
|
+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
|
28
|
+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|
29
|
+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|