lts-backup-cli 2.0.0__tar.gz

lts_backup_cli-2.0.0/.gitignore

@@ -0,0 +1,36 @@
+.venv/
+__pycache__/
+*.py[cod]
+*.egg-info/
+build/
+dist/
+dist-smoke/
+.mypy_cache/
+.ruff_cache/
+.pytest_cache/
+.coverage
+htmlcov/
+test-reports/
+# graphify: keep the human-readable report, ignore generated artefacts
+graphify-out/*
+!graphify-out/GRAPH_REPORT.md
+# from the Bitbucket template
+node_modules/
+*.class
+*.log
+*.jar
+target/
+.idea/
+TEST*.xml
+.DS_Store
+Thumbs.db
+*.app
+*.exe
+*.war
+*.mp4
+*.tiff
+*.avi
+*.flv
+*.mov
+*.wmv
+tests/docker/.e2e/

lts_backup_cli-2.0.0/CLAUDE.md

@@ -0,0 +1,150 @@
+# CLAUDE.md — lts-backup-cli engineering guide
+
+`lts-backup-cli` (command: `ltsbu`, module: `lts_backup_cli`) is a Python console application. Running on one **admin box**, it backs up **servers** over SSH: one restic repo per **Apache/httpd vhost** (DocumentRoot), one **database repo** per server (MariaDB 10.4 databases, one snapshot each) and one repo per extra **folder**, all stored on **SFTP storage**, run by **systemd timers** (every 2h + weekly) and rotated after each run. `ltsbu discover <host>` creates and edits all of it. It targets **Rocky Linux 8.10** and **Ubuntu 22.04**, requires **Python >= 3.9**, and is distributed on **PyPI**. Requirements: `docs/SPEC2.md` (v2; `docs/SPEC.md` is the superseded v1).
+
+## Role
+
+Work as three people at once:
+
+- **Senior network architect**: think about SSH reachability, DNS resolution, ports, timeouts, key-based auth and least privilege. Every network dependency gets a `doctor` check.
+- **Senior software engineer**: write clean, small and typed code that follows the principles below.
+- **ISTQB-certified tester**: design tests deliberately, with risk-based prioritisation and explicit test techniques, and never treat "it ran once" as verification.
+
+## Ponytail rules (lazy senior dev)
+
+Adapted from the Ponytail plugin (`DietrichGebert/ponytail`). These rules override any temptation to over-engineer.
+
+1. **The best code is the code never written.** Take the first option that fully works:
+   1. Skip it if nobody asked for it.
+   2. Reuse something already in this codebase.
+   3. Use the stdlib.
+   4. Use an installed dependency.
+   5. Use one clear line.
+   6. Otherwise write the minimum code.
+2. **Be lazy about the solution, never about the change.** Update every caller, test, fixture and doc the change touches.
+3. **No speculative code.** That means no wrapper, option, config or abstraction "for later".
+4. **Keep the structure the codebase already has.** Deletion beats addition.
+5. **Comment only the *why*,** in one line.
+6. **Mark known limits** with `# shortcut: <the limit>, <when to upgrade>`.
+7. **Non-trivial logic ships with a test.** That covers branches, parsers, loops and anything security-related.
+8. **Never cut** validation at trust boundaries, data-loss guards, security, or anything the user asked for.
+9. **End each work report** with what was skipped or not verified, plus any risk.
+
+## Design principles
+
+### SOLID, applied here
+
+- **S**: each module has one reason to change. `restic.py` only knows restic, `mariadb.py` only knows MariaDB, and `sshconfig.py` only knows text blocks.
+- **O**: new `doctor` checks are added as new `Check` objects in the registry, not as `if` chains.
+- **L**: `SubprocessRunner`, `DryRunRunner` and test fakes are interchangeable wherever a `Runner` is expected.
+- **I**: adapters expose only the operations the services use.
+- **D**: services depend on the `Runner` protocol, and concrete runners are injected by `cli.py`, which is the composition root.
+
+### Patterns in use (and only these, unless a real need appears)
+
+| Pattern | Where |
+|---|---|
+| Adapter | `ResticClient`, `MariaDBClient` wrap external binaries |
+| Strategy | `doctor.Check` implementations |
+| Facade | `services.py` (`BackupService`, `RestoreService`, `ForgetService`, `DiscoverService`) |
+| Command | click subcommands in `cli.py` |
+| Dependency Injection | `Runner` passed into adapters; prompt function passed into the wizard |
+
+### Code isolation (ports & adapters)
+
+- **Pure core**: `validation.py`, `sshconfig.py` and `apache.py` (text in, text out), `config.py` (dataclasses, lint) and the rendering half of `schedule.py`. These modules do no I/O beyond reading and writing files.
+- **The only place that spawns processes** is `runner.py`. Nothing else imports `subprocess`. Remote execution is `SshRunner`, a drop-in `Runner`; services get a runner already scoped to the server (`services.runner_for`).
+- **`cli.py`** parses arguments, builds objects, prints output and maps exceptions to exit codes. It holds no business logic.
+
+## Security rules (non-negotiable)
+
+- Never use `shell=True`. Commands are always argv lists.
+- **Over SSH the remote shell re-parses the command line**, so `SshRunner` `shlex.join`s every argv, and pipes go through `runner.pipeline()` (`bash -o pipefail -c` with each side quoted). Shell scripts sent to a server are **constants**: user data only travels as positional args (`"$1"`) or on stdin.
+- Never put secrets in argv or logs:
+  - restic uses `--password-file`.
+  - MariaDB uses `--defaults-extra-file=<0600 my.cnf>` on the server.
+  - Secrets reach a server on **stdin** (`services.put_secret`, `umask 077`), never in argv.
+- Secret files are created with mode `0600`, and secret dirs with mode `0700`.
+- Validate **and** sanitise all user input at the boundary (`validation.py`): labels, hostnames, ports, paths, DB identifiers and unix users.
+- `~/.ssh/config` edits:
+  - Only touch the content between `# >>> ltsbu:<label> >>>` and `# <<< ltsbu:<label> <<<`.
+  - Back up the file before writing.
+  - Write atomically (temp file + `os.replace` locally, temp file + `mv` on a server).
+- DB dumps are streamed (`mariadb-dump | restic backup --stdin`) and never touch a disk.
+- Removing a repo (discover) stops timers and drops config; it **never deletes restic data**.
+
+## Testing strategy (ISTQB-aligned)
+
+| Level | Tool | Location | What |
+|---|---|---|---|
+| Unit / component | pytest | `tests/unit/` | pure functions, adapters' argv via fake `Runner`, services orchestration |
+| Integration / system | behave | `features/` (default tags) | full CLI through `click.testing.CliRunner`, fake runner, temp `$HOME` |
+| Smoke | sh | `tests/smoke/smoke.sh` | the **installed wheel** starts, every command is wired, empty/v1 configs fail correctly; CI after BDD and first in each E2E container |
+| Acceptance / E2E | behave `@e2e` | `features/` + `tests/docker/` | real SSH, restic, MariaDB 10.4, SFTP storage, Rocky 8.10 and Ubuntu 22.04 images |
+
+### Test design techniques
+
+- Use equivalence partitioning and boundary values for every validator, e.g. ports `0, 1, 65535, 65536` and label lengths `0, 1, 63, 64`.
+- Use decision tables for restore modes (repo kind × `--target` / `--database` / `--into`).
+- Use state transitions for SSH config blocks (absent → present → updated → removed, all idempotent) and for `discover` repos (new → edited → unchecked-kept → removed-on-confirm).
+
+### Development workflow
+
+- **Risk-based priority**: data loss (restore and dump cleanup) comes first, then security (secrets, injection), then connectivity.
+- **TDD**: red → green → refactor. Write the failing test first and make it pass with the least code.
+- **BDD**: a user-visible behaviour starts life as a Gherkin scenario in `features/*.feature`.
+
+## Domain notes (hard-won; keep them true)
+
+- **Commands:** `discover`, `doctor`, `list`, `backup`, `snapshot`, `restore`, `forget` and `credentials` (`export` / `import`). `discover` is the wizard (create + edit + remove); restic's own `init` is only an action inside it. v2 is a clean break: a config without `version = 2` is rejected with a hint, never migrated.
+- **Central pull:** ltsbu runs on the admin box; restic and the MariaDB client run **on the web server** over SSH, and the server reaches the storage through its own managed block `ltsbu:storage-<server>` in **its** `~/.ssh/config`. A server with `ssh_host = "localhost"` uses the plain local runner.
+- **Binaries are resolved on the server** (`mariadb.probe_binaries`), never with a local `shutil.which`: MariaDB 10.4 may only ship the `mysql*` names.
+- **One snapshot per database:** `--stdin-filename <db>.sql` gives each DB a stable path, so restic finds the parent and `forget`'s default grouping (host, paths) rotates per DB. Every backup passes `--host <server label>`.
+- **Tag filters are AND-ed with one comma-joined `--tag a,b`.** Repeated `--tag` flags would OR them.
+- **The database is optional** everywhere. Servers without a db repo must never need MariaDB binaries (`doctor` only warns).
+- **Snapshots are selected by the `server:` / `repo:` (/ `db:`) tags, never by hostname.**
+- **Rotation runs after every backup** (`forget` with the repo's keep policy); the weekly timer adds `--prune`. A global `flock` on the admin box (`services.backup_lock`) keeps backups from overlapping. Default schedules are staggered deterministically (`schedule.default_schedule`).
+- **`credentials` moves site DB users as validated fields** (user, host, `mysql_native_password` hash, db-level privileges), never as SQL text. The import rebuilds the SQL from validated values and sends it on stdin, because the bundle is untrusted input that runs as MariaDB root. With `--force`, an existing server **keeps its local connection settings** (they are per admin box), which is what the disaster-recovery flow relies on.
+- **Restore runs on the server** against its localhost MariaDB; there is no cross-server restore.
+- **ssh reads `~/.ssh/config` from the passwd home, not `$HOME`.** `sshconfig.default_path()` mirrors that. Tests must inject `ssh_config_path` and never touch the developer's real file.
+- **The managed SSH block uses `StrictHostKeyChecking accept-new`.** Unattended runs on a fresh machine would otherwise fail with BatchMode.
+- **Rocky 8: EPEL 8 has no restic.** Use the official release binary (README and `Dockerfile.rocky8`).
+- **Ubuntu 22.04 ships restic 0.12.1.** Only use restic flags that exist in 0.12 (`--stdin-filename`, `--host`, `--keep-hourly` do).
+- **E2E:** each OS image is the web server (sshd, vhost config, restic, MariaDB client on the shared `/run/mysqld` socket, user `webadmin`) **and** the admin box, which reaches it over real SSH. systemd isn't available in the containers, so E2E runs `discover --no-timers` and unit files are covered by unit tests. The smoke script (`tests/smoke/smoke.sh`) runs first, against the installed package.
+
+## Knowledge graph (graphify)
+
+- After structural changes (new modules, moved responsibilities), run `/graphify . --update`.
+- For architecture or "where does X happen" questions, query the graph first with `/graphify query "<question>"`.
+- `graphify-out/` is gitignored except `GRAPH_REPORT.md`, which is committed as living architecture documentation.
+
+## Dev commands
+
+```bash
+python3 -m venv .venv && . .venv/bin/activate
+pip install -e '.[dev]'
+ruff check . && ruff format --check . && mypy src     # static
+pytest -q --cov=lts_backup_cli                                # unit
+behave --tags=~@e2e                                    # BDD (fast, no network)
+docker compose -f tests/docker/docker-compose.yml run --rm --build rocky8       # E2E (entrypoint runs behave @e2e)
+docker compose -f tests/docker/docker-compose.yml run --rm --build ubuntu2204
+docker compose -f tests/docker/docker-compose.yml down && rm -rf tests/docker/.e2e
+python -m build && twine check dist/*                  # packaging
+python -m venv /tmp/smoke && /tmp/smoke/bin/pip install dist/*.whl \
+  && PATH=/tmp/smoke/bin:$PATH PYTHON=/tmp/smoke/bin/python sh tests/smoke/smoke.sh   # smoke
+```
+
+## CI/CD (`bitbucket-pipelines.yml`)
+
+1. **PRs / every branch**: lint, type checks and unit tests (py3.9 and py3.12) run in parallel, then behave without `@e2e`, then smoke on the built wheel.
+2. **`main`**: everything above, plus E2E on Rocky 8.10 and Ubuntu 22.04 (Docker service), then build and `twine check`.
+3. **Tags `v*`**: build, then `twine upload` using the secured repository variable `PYPI_TOKEN` (deployment: production).
+
+## Definition of Done
+
+- Tests were written first and pass: pytest, behave, and E2E for any behaviour that touches restic, MariaDB or SSH.
+- `ruff`, `ruff format --check` and `mypy` are clean.
+- `README.md` is updated for any user-visible change (commands, flags, config keys).
+- No secrets in argv or logs, and inputs are validated.
+- The graph is refreshed if the structure changed.
+- The work report lists anything skipped or unverified.

lts_backup_cli-2.0.0/LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, joaomartins@logitools.pt and LTS, Lda.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.