kamal-backup 0.1.0.pre.2 → 0.1.0.pre.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f90e6e9f02fd3ddb958b6f2c41de4219b36fa3e8ff4b84ea779c7decf74379b5
-  data.tar.gz: ab72fc616b324bcceee78c3d0060900e747334bf2ef8f3e962c0d81e0d7674ca
+  metadata.gz: 8ed4e21bfee7ac0e789c3a0706626062055c92701128b5999a3828f5f3190057
+  data.tar.gz: 5983618fb0fb16db51cfa82f4ff28398c00568d7a5082fd494a7063c944b111d
 SHA512:
-  metadata.gz: a94b5efc6f8c9727a176899572a758247f1db0150af45f71b6a0f830c95a9294f885d26e67d7b649308f604087e9e318f427033b8428ed663671b0b1cdbda87f
-  data.tar.gz: 97cfef328b321fefca4d95c36caf25d3a168e49b97e5de02879d3907a88d61842aa233d6dacb4367e1ac5c6866d2636586e67db91e04f2acfa02057dae318c4b
+  metadata.gz: 227a34dc536cd9d5ca13cb2853fcac13c03c5defbe87e406374c1ebc7b94e7cc094693903aab1e7b6addb2b2a11913bae419767cb0e242f56d86818171bacb79
+  data.tar.gz: 16788fa7108df55c2afa70f900511ce6574c931be8459774649ebcb533c415873e4424d8172e0df987804f6509f4aea7fb398676cd3741ff0c59fddf6d7e9a74

data/README.md

@@ -1,40 +1,60 @@
 # kamal-backup
 
-`kamal-backup` is a small Docker image for Kamal accessories. It creates encrypted, restic-backed backups for self-hosted apps by backing up database dumps and mounted application files together.
+`kamal-backup` gives Rails apps a clean backup and restore workflow for Kamal.
 
-The Docker image is the normal production interface. The Ruby gem packages the same `kamal-backup` executable so the image can install it cleanly, and so operators can optionally run the CLI from a laptop for restore drills when they have restic, database clients, and the right environment configured.
+It backs up:
 
-It is aimed at common Kamal backup needs:
+- PostgreSQL, MySQL/MariaDB, or SQLite
+- file-backed Active Storage and other mounted app files
 
-- Kamal Postgres backup
-- Kamal MySQL and MariaDB backup
-- Kamal Active Storage backup
-- Kamal restic backup
-- Restore drills and evidence for security reviews such as CASA
+It restores in two clear modes:
 
-## What It Backs Up
+- `restore local`: pull a production backup onto your machine
+- `restore production`: restore back into live production
 
-`kamal-backup` handles two data surfaces:
+And it drills in two clear modes:
 
-1. A logical database dump from PostgreSQL, MySQL/MariaDB, or SQLite.
-2. Mounted application files such as Rails Active Storage.
+- `drill local`: prove the backup works on your machine
+- `drill production`: restore into scratch targets on production infrastructure, run checks, and record evidence
 
-Database backups are logical dumps, not raw database data directories. File backups use one `restic backup` snapshot per run containing all configured mounted paths, so `restore-files latest` restores all file paths from that run.
+Under the hood it uses [restic](https://restic.net/) for encrypted backup storage and repository management.
 
-Database dump snapshots are tagged with `kamal-backup`, `app:<name>`, `type:database`, `adapter:<adapter>`, and `run:<timestamp>`. The dump object uses a flat restic stdin filename such as `databases-chatwithwork-postgres-20260422T120000Z.pgdump` because restic stdin backups do not support nested virtual paths consistently.
+## Why Rails teams use it
 
-## Quick Start With Kamal
+`kamal-backup` is aimed at the common self-hosted Rails setup where:
 
-Add a backup accessory to your Kamal deploy config:
+- the app is deployed with Kamal
+- the database is PostgreSQL, MySQL/MariaDB, or SQLite
+- file data lives on a mounted volume
+- you need real restore drills and evidence for CASA or another security review
 
-```yaml
-aliases:
-  backup: accessory exec backup "kamal-backup backup"
-  backup-list: accessory exec backup "kamal-backup list"
-  backup-check: accessory exec backup "kamal-backup check"
-  backup-evidence: accessory exec backup "kamal-backup evidence"
-  backup-logs: accessory logs backup -f
+If your app already stores Active Storage blobs directly in S3, there may be no local file path for `BACKUP_PATHS` to capture. In that case, `kamal-backup` still covers the database side, but object-storage backups are a separate concern.
+
+## Quick Start
+
+Add the gem in your Rails app:
+
+```ruby
+group :development do
+  gem "kamal-backup"
+end
+```
+
+Install it and generate the local config stubs:
+
+```sh
+bundle install
+bundle exec kamal-backup init
+```
+
+That creates:
 
+- `config/kamal-backup.yml`
+- `config/kamal-backup.local.yml`
+
+Then add the backup accessory to `config/deploy.yml`:
+
+```yaml
 accessories:
   backup:
     image: ghcr.io/crmne/kamal-backup:latest
@@ -64,294 +84,206 @@ bin/kamal accessory boot backup
 bin/kamal accessory logs backup
 ```
 
-Run manual commands:
+Run the first backup from your app checkout with the local gem and Kamal-style destination selection:
 
 ```sh
-bin/kamal backup
-bin/kamal backup-list
-bin/kamal backup-check
-bin/kamal backup-evidence
-bin/kamal backup-logs
+bundle exec kamal-backup -d production backup
+bundle exec kamal-backup -d production list
+bundle exec kamal-backup -d production evidence
 ```
 
-Alias reference:
-
-| Alias | Expands to | Use |
-|---|---|---|
-| `bin/kamal backup` | `accessory exec backup "kamal-backup backup"` | Run one backup immediately. |
-| `bin/kamal backup-list` | `accessory exec backup "kamal-backup list"` | Show restic snapshots for the configured app. |
-| `bin/kamal backup-check` | `accessory exec backup "kamal-backup check"` | Run `restic check` and store the latest check result. |
-| `bin/kamal backup-evidence` | `accessory exec backup "kamal-backup evidence"` | Print redacted backup evidence JSON. |
-| `bin/kamal backup-logs` | `accessory logs backup -f` | Tail the backup accessory logs. |
-
-## Commands
-
-Commands usually run inside the production backup accessory with `bin/kamal accessory exec backup "kamal-backup <command>"`, or through Kamal aliases such as `bin/kamal backup`. A local gem install is useful when you intentionally want the operator laptop to run restic and database client commands directly.
+If you keep multiple deploy configs, pass `-c` the same way Kamal does:
 
 ```sh
-kamal-backup backup
-kamal-backup restore-db [snapshot-or-latest]
-kamal-backup restore-files [snapshot-or-latest] [target-dir]
-kamal-backup list
-kamal-backup check
-kamal-backup evidence
-kamal-backup schedule
-kamal-backup version
+bundle exec kamal-backup -c config/deploy.staging.yml -d staging backup
 ```
 
-| Command | What it does |
-|---|---|
-| `backup` | Runs one immediate backup, creating one database snapshot and one file snapshot for all `BACKUP_PATHS`. |
-| `restore-db [snapshot-or-latest]` | Restores a database dump. Defaults to `latest` and requires explicit restore environment. |
-| `restore-files [snapshot-or-latest] [target-dir]` | Restores file paths from a file snapshot. Defaults to `latest /restore/files`. |
-| `list` | Lists restic snapshots for the configured app tags. |
-| `check` | Runs `restic check` and records the latest result for evidence output. |
-| `evidence` | Prints redacted JSON with backup configuration, latest snapshots, check status, and tool versions. |
-| `schedule` | Runs the foreground scheduler loop used by the container default command. |
-| `version` | Prints the gem version. `--version` and `-v` do the same. |
+Examples live in:
 
-The default container command is:
+- [examples/kamal-accessory.yml](examples/kamal-accessory.yml)
+- [examples/kamal-backup.yml.example](examples/kamal-backup.yml.example)
+- [examples/kamal-backup.local.yml.example](examples/kamal-backup.local.yml.example)
 
-```sh
-kamal-backup schedule
-```
+## What Restic Does Here
 
-## Configuration
+`kamal-backup` uses restic as the backup engine and repository format.
 
-Required common environment:
+In the normal Kamal setup, you do not install restic on the Rails app host. The backup accessory image already includes it. You only point the accessory at a restic repository, usually:
 
-```sh
-APP_NAME=chatwithwork
-DATABASE_ADAPTER=postgres
-RESTIC_REPOSITORY=s3:https://s3.example.com/chatwithwork-backups
-RESTIC_PASSWORD=change-me
-BACKUP_PATHS=/data/storage
-```
+- S3-compatible object storage
+- a restic REST server
+- a filesystem path for local development
 
-`BACKUP_PATHS` accepts colon-separated or newline-separated paths. Every path must exist. Suspicious broad paths such as `/`, `/var`, `/etc`, and `/root` are refused unless `KAMAL_BACKUP_ALLOW_SUSPICIOUS_PATHS=true`.
+If you choose a `rest:` repository, `kamal-backup` does not install or operate that server for you. It is a separate service.
 
-PostgreSQL:
+## Commands
 
-```sh
-DATABASE_ADAPTER=postgres
-DATABASE_URL=postgres://app@app-db:5432/app_production
-PGPASSWORD=change-me
-```
-
-MySQL/MariaDB:
+The operator-facing command surface is:
 
 ```sh
-DATABASE_ADAPTER=mysql
-DATABASE_URL=mysql2://app@app-mysql:3306/app_production
-MYSQL_PWD=change-me
+kamal-backup init
+kamal-backup backup
+kamal-backup restore local [snapshot-or-latest]
+kamal-backup restore production [snapshot-or-latest]
+kamal-backup drill local [snapshot-or-latest]
+kamal-backup drill production [snapshot-or-latest]
+kamal-backup list
+kamal-backup check
+kamal-backup evidence
+kamal-backup schedule
+kamal-backup version
 ```
 
-SQLite:
+Production-side commands shell out through Kamal when you pass `-d` or `-c`. Local commands run on your machine.
 
-```sh
-DATABASE_ADAPTER=sqlite
-SQLITE_DATABASE_PATH=/data/db/production.sqlite3
-```
-
-Retention defaults:
+Common examples:
 
 ```sh
-RESTIC_KEEP_LAST=7
-RESTIC_KEEP_DAILY=7
-RESTIC_KEEP_WEEKLY=4
-RESTIC_KEEP_MONTHLY=6
-RESTIC_KEEP_YEARLY=2
-RESTIC_FORGET_AFTER_BACKUP=true
+bundle exec kamal-backup -d production backup
+bundle exec kamal-backup -d production check
+bundle exec kamal-backup -d production evidence
+bundle exec kamal-backup -d production restore production latest
+bundle exec kamal-backup -d production drill production latest --database app_restore_20260423 --files /restore/files
+bundle exec kamal-backup -d production version
+bundle exec kamal-backup restore local latest
+bundle exec kamal-backup drill local latest --check "bin/rails runner 'puts User.count'"
 ```
 
-Set `RESTIC_FORGET_AFTER_BACKUP=false` for append-only repositories, such as a restic REST server started with `--append-only`. In that mode, run retention and prune from the backup server or another trusted maintenance process with delete permissions.
+Use `kamal-backup help`, `kamal-backup help restore`, or `kamal-backup help drill` for task-specific usage.
 
-Scheduler and checks:
+## How a Backup Run Works
 
-```sh
-BACKUP_SCHEDULE_SECONDS=86400
-BACKUP_START_DELAY_SECONDS=0
-RESTIC_CHECK_AFTER_BACKUP=false
-RESTIC_CHECK_READ_DATA_SUBSET=5%
-```
+When `kamal-backup backup` runs, it does five things:
 
-For S3-compatible restic repositories, provide the standard restic/AWS variables as Kamal secrets:
+1. Validates the app name, restic repository, database settings, and `BACKUP_PATHS`.
+2. Creates a database backup with the database-native export tool.
+3. Streams that database backup into restic with tags such as `type:database`, `adapter:<adapter>`, and `run:<timestamp>`.
+4. Runs one `restic backup` for all configured `BACKUP_PATHS`, tagged as `type:files` with the same `run:<timestamp>`.
+5. Optionally runs `restic forget --prune` and `restic check`.
 
-```sh
-AWS_ACCESS_KEY_ID=...
-AWS_SECRET_ACCESS_KEY=...
-AWS_DEFAULT_REGION=...
-```
+That shared `run:<timestamp>` tag lets you match the database backup and file backup from the same run.
 
-## Restore Drills
+## Restore
 
-Restores are intentionally hard to run by accident. Every restore command requires:
+`restore` means "put data back."
 
-```sh
-KAMAL_BACKUP_ALLOW_RESTORE=true
-```
+`restore local` runs on your machine. With `-d` or `-c`, it asks Kamal for the backup accessory config and uses that as the source of truth for:
 
-Database restores use restore-specific environment by default. They do not restore to `DATABASE_URL`.
+- `APP_NAME`
+- `DATABASE_ADAPTER`
+- `RESTIC_REPOSITORY`
+- `LOCAL_RESTORE_SOURCE_PATHS` from the accessory `BACKUP_PATHS`
 
-PostgreSQL restore:
+You still provide the local targets yourself in `config/kamal-backup.local.yml` or env:
 
-```sh
-bin/kamal accessory exec backup \
-  --env KAMAL_BACKUP_ALLOW_RESTORE=true \
-  --env RESTORE_DATABASE_URL=postgres://app@app-db:5432/app_restore \
-  "kamal-backup restore-db latest"
-```
+- `DATABASE_URL` or `SQLITE_DATABASE_PATH`
+- `BACKUP_PATHS`
+- local secrets such as `RESTIC_PASSWORD` and DB passwords
 
-MySQL/MariaDB restore:
+Example:
 
 ```sh
-bin/kamal accessory exec backup \
-  --env KAMAL_BACKUP_ALLOW_RESTORE=true \
-  --env RESTORE_DATABASE_URL=mysql2://app@app-mysql:3306/app_restore \
-  "kamal-backup restore-db latest"
+bundle exec kamal-backup -d production restore local latest
 ```
 
-SQLite restore:
+`restore production` is the emergency path back into the live production database and live production file paths:
 
 ```sh
-bin/kamal accessory exec backup \
-  --env KAMAL_BACKUP_ALLOW_RESTORE=true \
-  --env RESTORE_SQLITE_DATABASE_PATH=/restore/db/restore.sqlite3 \
-  "kamal-backup restore-db latest"
+bundle exec kamal-backup -d production restore production latest
 ```
 
-File restore:
+It prompts locally, then shells out through Kamal to the backup accessory.
 
-```sh
-bin/kamal accessory exec backup \
-  --env KAMAL_BACKUP_ALLOW_RESTORE=true \
-  "kamal-backup restore-files latest /restore/files"
-```
+## Restore Drills
+
+`drill` means "restore, check, and record the result."
 
-Restore targets that look production-like are refused unless:
+`drill local` is often the fastest proof for a small app:
 
 ```sh
-KAMAL_BACKUP_ALLOW_PRODUCTION_RESTORE=true
+bundle exec kamal-backup -d production drill local latest --check "bin/rails runner 'puts User.count'"
 ```
 
-File restores to configured backup paths are refused unless:
+`drill production` restores into scratch targets on production infrastructure. It does not touch the live production database:
 
 ```sh
-KAMAL_BACKUP_ALLOW_IN_PLACE_FILE_RESTORE=true
+bundle exec kamal-backup -d production drill production latest \
+  --database app_restore_20260423 \
+  --files /restore/files \
+  --check "test -d /restore/files/data/storage"
 ```
 
-## Evidence
+Every drill writes `last_restore_drill.json` under `KAMAL_BACKUP_STATE_DIR`, and `kamal-backup evidence` includes that latest result.
 
-`kamal-backup evidence` prints a redacted JSON summary suitable for operational evidence:
-
-- app name
-- current time
-- database adapter
-- redacted restic repository
-- configured file backup paths
-- whether client-side forget/prune is enabled
-- retention policy
-- latest database and file snapshots
-- last tracked `restic check` result
-- image version
-- installed tool versions
+## Evidence for CASA and Similar Reviews
 
-Secrets, passwords, access keys, and database URL credentials are redacted.
+`evidence` is the JSON summary you can attach to an ops record or security review.
 
-Run:
-
-```sh
-bin/kamal accessory exec backup "kamal-backup evidence"
-```
+It includes:
 
-## Local Development
-
-Run tests:
+- latest database and file snapshots
+- latest `restic check` result
+- latest restore drill result
+- retention settings
+- tool versions
 
-```sh
-bin/test
-```
+For many reviews, the useful sequence is:
 
-Run docs locally:
+1. scheduled backups
+2. repository checks
+3. a real restore drill
+4. `kamal-backup evidence`
 
-```sh
-cd docs
-bundle install
-bundle exec jekyll serve --livereload
-```
+That reads much better to a reviewer than "the backup job is green."
 
-Published docs are configured for `https://kamal-backup.dev`.
+## Configuration Highlights
 
-Build the image:
+Core accessory environment:
 
 ```sh
-docker build -t kamal-backup .
+APP_NAME=chatwithwork
+DATABASE_ADAPTER=postgres
+RESTIC_REPOSITORY=s3:https://s3.example.com/chatwithwork-backups
+RESTIC_PASSWORD=change-me
+BACKUP_PATHS=/data/storage
 ```
 
-CI publishes container images to `ghcr.io/crmne/kamal-backup`. Pull requests build the image without pushing; branch, tag, SHA, default-branch `latest`, and default-branch version tags are pushed on non-PR builds. The version tag comes from `lib/kamal_backup/version.rb`, matching the gem version.
-
-The CLI is packaged as the `kamal-backup` gem. The Docker image builds and installs that gem, which is why `kamal-backup` is on `PATH` inside the container. On default-branch CI, a new gem version is published to RubyGems and GitHub Packages when it does not already exist. The RubyGems publish step expects the repository secret `RUBYGEMS_AUTH_TOKEN`.
-
-For local Ruby use:
+PostgreSQL:
 
 ```sh
-gem build kamal-backup.gemspec
-gem install ./kamal-backup-*.gem
-kamal-backup --help
+DATABASE_ADAPTER=postgres
+DATABASE_URL=postgres://app@app-db:5432/app_production
+PGPASSWORD=change-me
 ```
 
-In normal Kamal use, you do not need to install the gem on the app host. Run the command inside the accessory:
+MySQL/MariaDB:
 
 ```sh
-bin/kamal accessory exec backup "kamal-backup evidence"
+DATABASE_ADAPTER=mysql
+DATABASE_URL=mysql2://app@app-mysql:3306/app_production
+MYSQL_PWD=change-me
 ```
 
-Run a local backup against a filesystem restic repository:
+SQLite:
 
 ```sh
-export APP_NAME=local-app
-export DATABASE_ADAPTER=sqlite
-export SQLITE_DATABASE_PATH=/tmp/app.sqlite3
-export BACKUP_PATHS=/tmp/app-files
-export RESTIC_REPOSITORY=/tmp/kamal-backup-restic
-export RESTIC_PASSWORD=local-password
-export RESTIC_INIT_IF_MISSING=true
-
-kamal-backup backup
-kamal-backup list
-kamal-backup evidence
+DATABASE_ADAPTER=sqlite
+SQLITE_DATABASE_PATH=/data/db/production.sqlite3
 ```
 
-An example Docker Compose setup for local integration work is in `examples/docker-compose.integration.yml`.
-
-## Container Contents
-
-The image is based on Debian slim Ruby and includes:
-
-- Ruby runtime
-- `pg_dump` and `pg_restore`
-- `mariadb-dump` or `mysqldump`, plus `mariadb` or `mysql`
-- `sqlite3`
-- `restic`
-- CA certificates
-- `tini`
-
-## Security Notes
+Local config files:
 
-- Subprocesses are executed with argument arrays, not shell interpolation.
-- The CLI redacts secrets in errors and evidence output.
-- Database backups use logical dump tools.
-- File data should be mounted read-only in the backup accessory.
-- Restores require explicit environment flags.
-- Object storage credentials should be least-privilege for the backup bucket or prefix.
+- `config/kamal-backup.yml`
+- `config/kamal-backup.local.yml`
 
-## Non-Goals
+Keep secrets such as `RESTIC_PASSWORD`, cloud credentials, and local DB passwords in environment variables, not in YAML files.
 
-- Not a hosted backup service.
-- Not a replacement for database point-in-time recovery.
-- Not a physical replication tool.
-- Not a secret manager.
+## Docs
 
-## License
+Full docs live in [`docs/`](docs/):
 
-MIT
+- [`docs/_guide/getting-started.md`](docs/_guide/getting-started.md)
+- [`docs/_guide/configuration.md`](docs/_guide/configuration.md)
+- [`docs/_guide/restore.md`](docs/_guide/restore.md)
+- [`docs/_guide/restore-drills.md`](docs/_guide/restore-drills.md)
+- [`docs/_reference/commands.md`](docs/_reference/commands.md)

data/exe/kamal-backup

@@ -1,5 +1,9 @@
 #!/usr/bin/env ruby
 
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)
+
+require "bundler/setup"
+
 $LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
 
 require "kamal_backup"