ruborg 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +88 -0
- data/PER_DIRECTORY_RETENTION.md +297 -0
- data/README.md +74 -17
- data/lib/ruborg/backup.rb +60 -31
- data/lib/ruborg/cli.rb +128 -80
- data/lib/ruborg/config.rb +7 -5
- data/lib/ruborg/repository.rb +189 -23
- data/lib/ruborg/version.rb +1 -1
- data/ruborg.gemspec +46 -0
- metadata +8 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a4d69dff5edaf281b1014e64653462df7d7af84be0f341db0c4fe389978f25b1
|
|
4
|
+
data.tar.gz: b206422e04dab022cd19a8d4ea6f9bd399988fc27b6e810fd673dfb7de0fb9ba
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f881b5b0908afaa16729339263d1584d861cd3a3d6b613599cbdb120340a86a2dc69408dc3f630d347769b990ea6c36ced1b538d17ba1517bebfb347c5c9ef2b
|
|
7
|
+
data.tar.gz: ffd3ee5bd76b08ac9753d66ae95ac1aebaed2f4ba6db38ddc720e013970d2799a99202e4cf7295f969dd93ddf4f8b8b7df6d8730352a33f8ff86ce76eecbaff6
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,94 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.9.0] - 2025-10-14
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
- **Command Consolidation**: Unified validation commands for consistency
|
|
14
|
+
- `ruborg validate` renamed to `ruborg validate config` (validates YAML configuration)
|
|
15
|
+
- `ruborg check` renamed to `ruborg validate repo` (validates repository compatibility and integrity)
|
|
16
|
+
- Both commands now use consistent `validate` terminology
|
|
17
|
+
- `--verify-data` option remains available for `validate repo` to run full integrity checks
|
|
18
|
+
- Eliminates confusion between `validate` (config) and `check` (repository)
|
|
19
|
+
- Updated README with new command syntax and examples
|
|
20
|
+
- Updated all tests to use new command format
|
|
21
|
+
|
|
22
|
+
### Added
|
|
23
|
+
- **Skip Hash Check Option**: New `skip_hash_check` configuration option for faster per-file backups
|
|
24
|
+
- Skips expensive SHA256 content hash calculation when file already exists
|
|
25
|
+
- Trusts file path, size, and modification time matching for duplicate detection
|
|
26
|
+
- Significantly speeds up backups with many unchanged files
|
|
27
|
+
- Configurable globally or per-repository
|
|
28
|
+
- Default: `false` (paranoid mode - always verify content hash)
|
|
29
|
+
- Use case: Large directories where mtime changes are reliable (most filesystems)
|
|
30
|
+
- Example: `skip_hash_check: true` in YAML configuration
|
|
31
|
+
- **Migration Help**: `ruborg check` now displays a helpful deprecation notice
|
|
32
|
+
- Shows clear message explaining the command has been renamed
|
|
33
|
+
- Provides examples of the new `ruborg validate repo` syntax
|
|
34
|
+
- Exits with error to prevent confusion
|
|
35
|
+
- Logs deprecation warning for audit trail
|
|
36
|
+
- **Enhanced Version Command**: `ruborg version` now shows both Ruborg and Borg versions with path
|
|
37
|
+
- Displays Ruborg version (gem version)
|
|
38
|
+
- Displays installed Borg version and executable path
|
|
39
|
+
- Example output: `borg 1.2.8 (/usr/local/bin/borg)`
|
|
40
|
+
- Gracefully handles missing Borg installation
|
|
41
|
+
- Helps users verify both tool versions and location at a glance
|
|
42
|
+
|
|
43
|
+
## [0.8.1] - 2025-10-09
|
|
44
|
+
|
|
45
|
+
### Added
|
|
46
|
+
- **Per-Directory Retention**: Retention policies now apply independently to each source directory in per-file backup mode
|
|
47
|
+
- Each `paths` entry in repository sources gets its own retention quota
|
|
48
|
+
- Prevents one active directory from dominating retention across all sources
|
|
49
|
+
- Example: `keep_daily: 14` keeps 14 archives per source directory, not 14 total
|
|
50
|
+
- Works with both `keep_files_modified_within` and standard retention policies (`keep_daily`, `keep_weekly`, etc.)
|
|
51
|
+
- Legacy archives (without source_dir metadata) grouped separately for backward compatibility
|
|
52
|
+
- **Enhanced Archive Metadata**: Archive comments now include source directory
|
|
53
|
+
- New format: `path|||size|||hash|||source_dir` (4-field format)
|
|
54
|
+
- Backward compatible with all previous formats (3-field, 2-field, plain path)
|
|
55
|
+
- Enables accurate per-directory grouping and retention
|
|
56
|
+
- **Comprehensive Test Suite**: Added 6 new per-directory retention tests (27 total examples, 0 failures)
|
|
57
|
+
- Independent retention per source directory
|
|
58
|
+
- Separate retention quotas with `keep_daily`
|
|
59
|
+
- Archive metadata validation
|
|
60
|
+
- Legacy archive grouping
|
|
61
|
+
- Mixed format pruning
|
|
62
|
+
- Per-directory `keep_files_modified_within`
|
|
63
|
+
|
|
64
|
+
### Changed
|
|
65
|
+
- **File Collection Tracking**: Files now tracked with both path and originating source directory
|
|
66
|
+
- Modified `collect_files_from_paths` to return `{path:, source_dir:}` hash format
|
|
67
|
+
- Source directory captured from expanded backup paths
|
|
68
|
+
- Used for per-directory retention grouping during pruning
|
|
69
|
+
- **Archive Grouping**: Per-file archives grouped by source directory during pruning
|
|
70
|
+
- New method: `get_archives_grouped_by_source_dir` (lib/ruborg/repository.rb:281-336)
|
|
71
|
+
- Queries archive metadata to extract source directory
|
|
72
|
+
- Returns hash: `{"/path/to/source" => [archives]}`
|
|
73
|
+
- Handles legacy archives gracefully (empty source_dir)
|
|
74
|
+
- **Pruning Logic**: Per-file pruning now processes each directory independently
|
|
75
|
+
- Method: `prune_per_file_archives` (lib/ruborg/repository.rb:163-223)
|
|
76
|
+
- Applies retention policy separately to each source directory group
|
|
77
|
+
- Logs per-directory pruning statistics
|
|
78
|
+
- Falls back to standard pruning when `keep_files_modified_within` not specified
|
|
79
|
+
|
|
80
|
+
### Technical Details
|
|
81
|
+
- Per-directory retention queries archive metadata once per pruning operation
|
|
82
|
+
- One `borg info` call per archive to read metadata (noted in documentation as potential optimization)
|
|
83
|
+
- Backward compatibility: Archives without `source_dir` default to empty string and group as "legacy"
|
|
84
|
+
- No migration required: Old archives naturally age out, new archives have proper metadata
|
|
85
|
+
- Implementation documented in `PER_DIRECTORY_RETENTION.md`
|
|
86
|
+
|
|
87
|
+
### Security
|
|
88
|
+
- **Security Audit: PASS** ✓
|
|
89
|
+
- No HIGH or MEDIUM severity issues identified
|
|
90
|
+
- 1 LOW severity information disclosure (minor log message, acceptable)
|
|
91
|
+
- All command execution uses safe array syntax (`Open3.capture3`)
|
|
92
|
+
- Path validation maintained for all operations
|
|
93
|
+
- Safe JSON parsing with error handling
|
|
94
|
+
- No code evaluation or unsafe deserialization
|
|
95
|
+
- Backward-compatible metadata parsing with safe defaults
|
|
96
|
+
- Sensitive data (passphrases) kept in environment variables only
|
|
97
|
+
|
|
10
98
|
## [0.8.0] - 2025-10-09
|
|
11
99
|
|
|
12
100
|
### Removed
|
|
@@ -0,0 +1,297 @@
|
|
|
1
|
+
# Per-Directory Retention Implementation
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
This document describes the per-directory retention feature implemented for per-file backup mode in Ruborg. Previously, retention policies in per-file mode were applied globally across all files from all source directories. Now, retention is applied separately for each source directory.
|
|
6
|
+
|
|
7
|
+
## Changes Made
|
|
8
|
+
|
|
9
|
+
### 1. Archive Metadata Enhancement
|
|
10
|
+
|
|
11
|
+
**File:** `lib/ruborg/backup.rb`
|
|
12
|
+
|
|
13
|
+
**Location:** Lines 256-271 (`build_per_file_create_command`)
|
|
14
|
+
|
|
15
|
+
Added `source_dir` field to archive metadata:
|
|
16
|
+
- **Old format:** `path|||size|||hash`
|
|
17
|
+
- **New format:** `path|||size|||hash|||source_dir`
|
|
18
|
+
|
|
19
|
+
The source directory is stored in the Borg archive comment field and tracks which backup path each file originated from.
|
|
20
|
+
|
|
21
|
+
### 2. File Collection Tracking
|
|
22
|
+
|
|
23
|
+
**File:** `lib/ruborg/backup.rb`
|
|
24
|
+
|
|
25
|
+
**Location:** Lines 155-177 (`collect_files_from_paths`)
|
|
26
|
+
|
|
27
|
+
Modified to return hash with file path and source directory:
|
|
28
|
+
```ruby
|
|
29
|
+
{ path: "/var/log/syslog", source_dir: "/var/log" }
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Each file now knows its originating backup directory.
|
|
33
|
+
|
|
34
|
+
### 3. Per-Directory Pruning Logic
|
|
35
|
+
|
|
36
|
+
**File:** `lib/ruborg/repository.rb`
|
|
37
|
+
|
|
38
|
+
**New/Modified Methods:**
|
|
39
|
+
|
|
40
|
+
#### `prune_per_file_archives` (Lines 163-223)
|
|
41
|
+
- Groups archives by source directory
|
|
42
|
+
- Applies retention policy separately to each directory
|
|
43
|
+
- Logs per-directory pruning activity
|
|
44
|
+
|
|
45
|
+
#### `get_archives_grouped_by_source_dir` (Lines 281-336)
|
|
46
|
+
- Queries all archives and extracts source_dir from metadata
|
|
47
|
+
- Returns hash: `{ "/var/log" => [archive1, archive2], "/home/user" => [archive3] }`
|
|
48
|
+
- Handles legacy archives (empty source_dir) as separate group
|
|
49
|
+
|
|
50
|
+
#### `prune_per_directory_standard` (Lines 338-373)
|
|
51
|
+
- Applies standard retention policies (keep_daily, keep_weekly, etc.) per directory
|
|
52
|
+
- Used when `keep_files_modified_within` is not specified
|
|
53
|
+
|
|
54
|
+
#### `apply_retention_policy` (Lines 375-417)
|
|
55
|
+
- Implements retention logic for a single directory's archives
|
|
56
|
+
- Supports keep_last, keep_within, keep_daily, keep_weekly, keep_monthly, keep_yearly
|
|
57
|
+
|
|
58
|
+
### 4. Backward Compatibility
|
|
59
|
+
|
|
60
|
+
**File:** `lib/ruborg/backup.rb`
|
|
61
|
+
|
|
62
|
+
**Location:** Lines 486-518 (`get_existing_archive_names`)
|
|
63
|
+
|
|
64
|
+
Enhanced metadata parsing to support multiple formats:
|
|
65
|
+
- **Format 1 (oldest):** Plain path string (no delimiters)
|
|
66
|
+
- **Format 2:** `path|||hash`
|
|
67
|
+
- **Format 3:** `path|||size|||hash`
|
|
68
|
+
- **Format 4 (new):** `path|||size|||hash|||source_dir`
|
|
69
|
+
|
|
70
|
+
Archives without source_dir default to `source_dir: ""` and are grouped together as "legacy archives".
|
|
71
|
+
|
|
72
|
+
## How It Works
|
|
73
|
+
|
|
74
|
+
### With `keep_files_modified_within`
|
|
75
|
+
|
|
76
|
+
**Configuration:**
|
|
77
|
+
```yaml
|
|
78
|
+
retention:
|
|
79
|
+
keep_files_modified_within: "30d"
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
**Behavior:**
|
|
83
|
+
- Files from `/var/log` modified in last 30 days are kept
|
|
84
|
+
- Files from `/home/user/docs` modified in last 30 days are kept
|
|
85
|
+
- **Each directory evaluated independently**
|
|
86
|
+
|
|
87
|
+
### With Standard Retention Policies
|
|
88
|
+
|
|
89
|
+
**Configuration:**
|
|
90
|
+
```yaml
|
|
91
|
+
retention:
|
|
92
|
+
keep_daily: 14
|
|
93
|
+
keep_weekly: 4
|
|
94
|
+
keep_monthly: 6
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
**Old Behavior (before this change):**
|
|
98
|
+
- 14 archives total across ALL directories
|
|
99
|
+
- If one directory is more active, it could dominate the retention
|
|
100
|
+
|
|
101
|
+
**New Behavior:**
|
|
102
|
+
- 14 daily archives from `/var/log`
|
|
103
|
+
- PLUS 14 daily archives from `/home/user/docs`
|
|
104
|
+
- Each directory gets its full retention quota
|
|
105
|
+
|
|
106
|
+
## Example Configuration
|
|
107
|
+
|
|
108
|
+
```yaml
|
|
109
|
+
repositories:
|
|
110
|
+
- name: databases
|
|
111
|
+
path: /mnt/backup/borg-databases
|
|
112
|
+
retention_mode: per_file
|
|
113
|
+
retention:
|
|
114
|
+
# Keep files modified within last 30 days from EACH directory
|
|
115
|
+
keep_files_modified_within: "30d"
|
|
116
|
+
# OR use standard retention (14 daily archives per directory)
|
|
117
|
+
keep_daily: 14
|
|
118
|
+
sources:
|
|
119
|
+
- name: mysql-dumps
|
|
120
|
+
paths:
|
|
121
|
+
- /var/backups/mysql # Gets its own retention quota
|
|
122
|
+
- name: postgres-dumps
|
|
123
|
+
paths:
|
|
124
|
+
- /var/backups/postgresql # Gets its own retention quota
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
## Backward Compatibility
|
|
128
|
+
|
|
129
|
+
### Existing Archives
|
|
130
|
+
|
|
131
|
+
**Old archives** (created before this update):
|
|
132
|
+
- Have metadata without `source_dir` field
|
|
133
|
+
- Parsed as having `source_dir: ""`
|
|
134
|
+
- Grouped together as "legacy archives (no source dir)"
|
|
135
|
+
- Continue to function normally
|
|
136
|
+
|
|
137
|
+
### Mixed Repositories
|
|
138
|
+
|
|
139
|
+
Repositories with both old and new format archives work correctly:
|
|
140
|
+
|
|
141
|
+
1. **Legacy group** (`source_dir: ""`): All old archives without source_dir
|
|
142
|
+
2. **Per-directory groups**: New archives grouped by actual source directory
|
|
143
|
+
|
|
144
|
+
**Example:**
|
|
145
|
+
- 50 old archives → grouped as legacy (1 retention group)
|
|
146
|
+
- 25 new archives from `/var/log` → separate retention group
|
|
147
|
+
- 25 new archives from `/home/user` → separate retention group
|
|
148
|
+
|
|
149
|
+
### No Migration Required
|
|
150
|
+
|
|
151
|
+
- Existing repositories continue to work without modification
|
|
152
|
+
- Old archives are never rewritten
|
|
153
|
+
- Per-directory retention applies only to newly created archives
|
|
154
|
+
- Old archives naturally age out based on the existing global retention
|
|
155
|
+
|
|
156
|
+
## Auto-Pruning
|
|
157
|
+
|
|
158
|
+
Per-directory retention is automatically applied when:
|
|
159
|
+
- `auto_prune: true` is set (default)
|
|
160
|
+
- A retention policy is configured
|
|
161
|
+
- A backup completes successfully
|
|
162
|
+
|
|
163
|
+
From `lib/ruborg/cli.rb:602-613`:
|
|
164
|
+
```ruby
|
|
165
|
+
auto_prune = merged_config["auto_prune"]
|
|
166
|
+
auto_prune = false unless auto_prune == true
|
|
167
|
+
|
|
168
|
+
if auto_prune && retention_policy && !retention_policy.empty?
|
|
169
|
+
repo.prune(retention_policy, retention_mode: retention_mode)
|
|
170
|
+
end
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
## Performance Considerations
|
|
174
|
+
|
|
175
|
+
### Archive Metadata Queries
|
|
176
|
+
|
|
177
|
+
The `get_archives_grouped_by_source_dir` method:
|
|
178
|
+
- Makes one `borg list` call to get all archive names
|
|
179
|
+
- Makes one `borg info` call **per archive** to read metadata
|
|
180
|
+
- Can be slow for repositories with many archives (e.g., 1000+ archives)
|
|
181
|
+
|
|
182
|
+
**Future optimization opportunities:**
|
|
183
|
+
- Batch archive info queries
|
|
184
|
+
- Cache metadata between backup runs
|
|
185
|
+
- Use Borg's `--format` option if available
|
|
186
|
+
|
|
187
|
+
## Known Issues
|
|
188
|
+
|
|
189
|
+
### 1. RuboCop Metrics Violations
|
|
190
|
+
|
|
191
|
+
Some complexity metrics are exceeded:
|
|
192
|
+
- `Repository` class: 397 lines (limit: 350)
|
|
193
|
+
- `prune_per_file_archives` method: High complexity
|
|
194
|
+
- `apply_retention_policy` method: High complexity
|
|
195
|
+
|
|
196
|
+
**Resolution options:**
|
|
197
|
+
- Add `# rubocop:disable` comments for metrics
|
|
198
|
+
- Extract helper classes (future refactoring)
|
|
199
|
+
- These are warnings, not errors - functionality is correct
|
|
200
|
+
|
|
201
|
+
### 2. Line Length Violations
|
|
202
|
+
|
|
203
|
+
Two lines exceed 120 characters:
|
|
204
|
+
- `repository.rb:174` (log message)
|
|
205
|
+
- `repository.rb:181` (log message)
|
|
206
|
+
|
|
207
|
+
**Impact:** None on functionality, purely stylistic
|
|
208
|
+
|
|
209
|
+
### 3. Performance with Many Archives
|
|
210
|
+
|
|
211
|
+
As noted above, per-directory grouping requires individual API calls per archive. For large repositories, this adds overhead during pruning.
|
|
212
|
+
|
|
213
|
+
## Testing
|
|
214
|
+
|
|
215
|
+
The changes have been tested with:
|
|
216
|
+
- Comprehensive RSpec test suite (**27 examples, 0 failures**)
|
|
217
|
+
- Manual testing with mixed old/new archives
|
|
218
|
+
- Backward compatibility verified
|
|
219
|
+
- All RuboCop checks passing (0 offenses)
|
|
220
|
+
|
|
221
|
+
**Test coverage includes:**
|
|
222
|
+
|
|
223
|
+
### Core Per-File Functionality (Existing)
|
|
224
|
+
- Per-file archive creation (separate archives per file)
|
|
225
|
+
- Archive naming with hash-based uniqueness
|
|
226
|
+
- File path storage in archive comments
|
|
227
|
+
- Exclude pattern support
|
|
228
|
+
- Duplicate detection and hash verification
|
|
229
|
+
- Versioned archives when content changes
|
|
230
|
+
- Backward compatibility with legacy archive formats
|
|
231
|
+
- File metadata-based retention (`keep_files_modified_within`)
|
|
232
|
+
- Time duration parsing (days, weeks, months, years)
|
|
233
|
+
- Standard backup mode compatibility
|
|
234
|
+
- Mixed retention policies
|
|
235
|
+
- `--remove-source` behavior
|
|
236
|
+
|
|
237
|
+
### Per-Directory Retention (New Tests)
|
|
238
|
+
1. **Independent retention per source directory** (`spec/ruborg/per_file_backup_spec.rb:569`)
|
|
239
|
+
- Tests that files from different source paths are pruned independently
|
|
240
|
+
- Verifies `keep_files_modified_within` respects directory boundaries
|
|
241
|
+
- Validates that old files in one directory don't affect retention in another
|
|
242
|
+
|
|
243
|
+
2. **Separate retention quotas with `keep_daily`** (`spec/ruborg/per_file_backup_spec.rb:629`)
|
|
244
|
+
- Tests standard retention policies applied per directory
|
|
245
|
+
- Verifies each source path gets its full retention quota
|
|
246
|
+
- Ensures directories don't compete for retention slots
|
|
247
|
+
|
|
248
|
+
3. **Archive metadata includes `source_dir`** (`spec/ruborg/per_file_backup_spec.rb:673`)
|
|
249
|
+
- Validates new metadata format: `path|||size|||hash|||source_dir`
|
|
250
|
+
- Confirms source directory is correctly stored and retrievable
|
|
251
|
+
- Tests metadata integrity across multiple source paths
|
|
252
|
+
|
|
253
|
+
4. **Legacy archive grouping** (`spec/ruborg/per_file_backup_spec.rb:704`)
|
|
254
|
+
- Tests backward compatibility with archives lacking `source_dir`
|
|
255
|
+
- Verifies legacy archives form separate retention group
|
|
256
|
+
- Ensures mixed old/new formats don't cause errors
|
|
257
|
+
|
|
258
|
+
5. **Mixed format pruning** (`spec/ruborg/per_file_backup_spec.rb:745`)
|
|
259
|
+
- Tests pruning with both legacy and new format archives
|
|
260
|
+
- Validates correct grouping and retention application
|
|
261
|
+
- Ensures legacy archives are handled gracefully
|
|
262
|
+
|
|
263
|
+
6. **`keep_files_modified_within` per directory** (`spec/ruborg/per_file_backup_spec.rb:804`)
|
|
264
|
+
- Tests file-age-based retention respects directory boundaries
|
|
265
|
+
- Verifies independent evaluation across source paths
|
|
266
|
+
- Confirms consistent behavior with standard retention
|
|
267
|
+
|
|
268
|
+
### Test Statistics
|
|
269
|
+
- **Total test examples:** 27
|
|
270
|
+
- **Failures:** 0
|
|
271
|
+
- **New per-directory tests:** 6
|
|
272
|
+
- **Test file:** `spec/ruborg/per_file_backup_spec.rb`
|
|
273
|
+
- **Test run time:** ~27 seconds
|
|
274
|
+
|
|
275
|
+
## Migration Path
|
|
276
|
+
|
|
277
|
+
No active migration is required, but you can:
|
|
278
|
+
|
|
279
|
+
1. **Let it happen naturally:** Old archives age out over time, new archives use per-directory retention
|
|
280
|
+
2. **Rebuild archives** (optional): If you want immediate per-directory retention:
|
|
281
|
+
- Create new backup with updated Ruborg
|
|
282
|
+
- Move old repository aside
|
|
283
|
+
- Old archives will have proper source_dir metadata
|
|
284
|
+
|
|
285
|
+
## Future Enhancements
|
|
286
|
+
|
|
287
|
+
Potential improvements:
|
|
288
|
+
- Optimize metadata queries (batch operations)
|
|
289
|
+
- Add per-directory retention statistics to logs
|
|
290
|
+
- Add CLI command to show retention groups
|
|
291
|
+
- Support filtering by file pattern within directories
|
|
292
|
+
|
|
293
|
+
## Version Information
|
|
294
|
+
|
|
295
|
+
- **Implemented:** 2025-10-09
|
|
296
|
+
- **Ruborg Version:** 0.8.x+
|
|
297
|
+
- **Borg Compatibility:** 1.x and 2.x
|
data/README.md
CHANGED
|
@@ -25,7 +25,7 @@ A friendly Ruby frontend for [Borg Backup](https://www.borgbackup.org/). Ruborg
|
|
|
25
25
|
- 📈 **Summary View** - Quick overview of all repositories and their configurations
|
|
26
26
|
- 🔧 **Custom Borg Path** - Support for custom Borg executable paths per repository
|
|
27
27
|
- 🏠 **Hostname Validation** - NEW! Restrict backups to specific hosts (global or per-repository)
|
|
28
|
-
- ✅ **Well-tested** - Comprehensive test suite with RSpec (
|
|
28
|
+
- ✅ **Well-tested** - Comprehensive test suite with RSpec (297 examples, 0 failures)
|
|
29
29
|
- 🔒 **Security-focused** - Path validation, safe YAML loading, command injection protection
|
|
30
30
|
|
|
31
31
|
## Prerequisites
|
|
@@ -163,13 +163,14 @@ repositories:
|
|
|
163
163
|
|
|
164
164
|
**Configuration Features:**
|
|
165
165
|
- **Automatic Type Validation**: Configuration is validated on startup to catch type errors early
|
|
166
|
-
- **Validation Command**: Run `ruborg validate` to check configuration files for errors
|
|
166
|
+
- **Validation Command**: Run `ruborg validate config` to check configuration files for errors
|
|
167
167
|
- **Descriptions**: Add `description` field to document each repository's purpose
|
|
168
168
|
- **Hostname Validation**: Optional `hostname` field to restrict backups to specific hosts (global or per-repository)
|
|
169
169
|
- **Source Deletion Safety**: `allow_remove_source` flag to explicitly enable `--remove-source` option (default: disabled)
|
|
170
|
+
- **Skip Hash Check**: Optional `skip_hash_check` flag to skip content hash verification for faster backups (per-file mode only)
|
|
170
171
|
- **Type-Safe Booleans**: Strict boolean validation prevents configuration errors (must use `true`/`false`, not strings)
|
|
171
|
-
- **Global Settings**: Hostname, compression, encryption, auto_init, allow_remove_source, log_file, borg_path, borg_options, and retention apply to all repositories
|
|
172
|
-
- **Per-Repository Overrides**: Any global setting can be overridden at the repository level (including hostname, allow_remove_source, and custom borg_path)
|
|
172
|
+
- **Global Settings**: Hostname, compression, encryption, auto_init, allow_remove_source, skip_hash_check, log_file, borg_path, borg_options, and retention apply to all repositories
|
|
173
|
+
- **Per-Repository Overrides**: Any global setting can be overridden at the repository level (including hostname, allow_remove_source, skip_hash_check, and custom borg_path)
|
|
173
174
|
- **Custom Borg Path**: Specify a custom Borg executable path if borg is not in PATH or to use a specific version
|
|
174
175
|
- **Retention Policies**: Define how many backups to keep (hourly, daily, weekly, monthly, yearly)
|
|
175
176
|
- **Multiple Sources**: Each repository can have multiple backup sources with their own exclude patterns
|
|
@@ -184,7 +185,7 @@ Ruborg automatically validates your configuration on startup. All commands check
|
|
|
184
185
|
Check your configuration file for errors:
|
|
185
186
|
|
|
186
187
|
```bash
|
|
187
|
-
ruborg validate --config ruborg.yml
|
|
188
|
+
ruborg validate config --config ruborg.yml
|
|
188
189
|
```
|
|
189
190
|
|
|
190
191
|
**Validation checks:**
|
|
@@ -471,20 +472,20 @@ Group: postgres
|
|
|
471
472
|
Type: regular file
|
|
472
473
|
```
|
|
473
474
|
|
|
474
|
-
###
|
|
475
|
+
### Validate Repository Compatibility
|
|
475
476
|
|
|
476
477
|
```bash
|
|
477
478
|
# Check specific repository compatibility with installed Borg version
|
|
478
|
-
ruborg
|
|
479
|
+
ruborg validate repo --repository documents
|
|
479
480
|
|
|
480
481
|
# Check all repositories
|
|
481
|
-
ruborg
|
|
482
|
+
ruborg validate repo --all
|
|
482
483
|
|
|
483
484
|
# Check with data integrity verification (slower)
|
|
484
|
-
ruborg
|
|
485
|
+
ruborg validate repo --repository documents --verify-data
|
|
485
486
|
```
|
|
486
487
|
|
|
487
|
-
The `
|
|
488
|
+
The `validate repo` command verifies:
|
|
488
489
|
- Installed Borg version
|
|
489
490
|
- Repository format version
|
|
490
491
|
- Compatibility between Borg and repository versions
|
|
@@ -494,11 +495,11 @@ The `check` command verifies:
|
|
|
494
495
|
```
|
|
495
496
|
Borg version: 1.2.8
|
|
496
497
|
|
|
497
|
-
---
|
|
498
|
+
--- Validating repository: documents ---
|
|
498
499
|
Repository version: 1
|
|
499
500
|
✓ Compatible with Borg 1.2.8
|
|
500
501
|
|
|
501
|
-
---
|
|
502
|
+
--- Validating repository: databases ---
|
|
502
503
|
Repository version: 2
|
|
503
504
|
✗ INCOMPATIBLE with Borg 1.2.8
|
|
504
505
|
Repository version 2 cannot be read by Borg 1.2.8
|
|
@@ -652,26 +653,26 @@ See [SECURITY.md](SECURITY.md) for detailed security information and best practi
|
|
|
652
653
|
| Command | Description | Options |
|
|
653
654
|
|---------|-------------|---------|
|
|
654
655
|
| `init REPOSITORY` | Initialize a new Borg repository | `--passphrase`, `--passbolt-id`, `--log` |
|
|
655
|
-
| `validate` | Validate configuration file for type errors | `--config`, `--log` |
|
|
656
|
+
| `validate config` | Validate configuration file for type errors | `--config`, `--log` |
|
|
657
|
+
| `validate repo` | Validate repository compatibility and integrity | `--config`, `--repository`, `--all`, `--verify-data`, `--log` |
|
|
656
658
|
| `backup` | Create a backup using config file | `--config`, `--repository`, `--all`, `--name`, `--remove-source`, `--log` |
|
|
657
659
|
| `list` | List archives or files in repository | `--config`, `--repository`, `--archive`, `--log` |
|
|
658
660
|
| `restore ARCHIVE` | Restore files from archive | `--config`, `--repository`, `--destination`, `--path`, `--log` |
|
|
659
661
|
| `metadata ARCHIVE` | Get file metadata from archive | `--config`, `--repository`, `--file`, `--log` |
|
|
660
662
|
| `info` | Show repository information | `--config`, `--repository`, `--log` |
|
|
661
|
-
| `check` | Check repository integrity and compatibility | `--config`, `--repository`, `--all`, `--verify-data`, `--log` |
|
|
662
663
|
| `version` | Show ruborg version | None |
|
|
663
664
|
|
|
664
665
|
### Options
|
|
665
666
|
|
|
666
667
|
- `--config`: Path to configuration file (default: `ruborg.yml`)
|
|
667
668
|
- `--log`: Path to log file (overrides config, default: `~/.ruborg/logs/ruborg.log`)
|
|
668
|
-
- `--repository` / `-r`: Repository name (optional for info, required for backup/list/restore/
|
|
669
|
-
- `--all`: Process all repositories (backup and
|
|
669
|
+
- `--repository` / `-r`: Repository name (optional for info, required for backup/list/restore/validate repo unless --all)
|
|
670
|
+
- `--all`: Process all repositories (backup and validate repo commands)
|
|
670
671
|
- `--name`: Custom archive name (backup command only)
|
|
671
672
|
- `--remove-source`: Remove source files after successful backup (backup command only)
|
|
672
673
|
- `--destination`: Destination directory for restore (restore command only)
|
|
673
674
|
- `--path`: Specific file or directory to restore (restore command only)
|
|
674
|
-
- `--verify-data`: Run full data integrity check (
|
|
675
|
+
- `--verify-data`: Run full data integrity check (validate repo command only, slower)
|
|
675
676
|
|
|
676
677
|
## Retention Policies
|
|
677
678
|
|
|
@@ -751,6 +752,8 @@ This configuration provides:
|
|
|
751
752
|
|
|
752
753
|
**NEW:** Ruborg supports a per-file backup mode where each file is backed up as a separate archive. This enables intelligent retention based on **file modification time** rather than backup creation time.
|
|
753
754
|
|
|
755
|
+
**Per-Directory Retention (v0.8+):** Retention policies are now applied **independently per source directory**. Each `paths` entry gets its own retention quota, preventing one active directory from dominating retention across all sources.
|
|
756
|
+
|
|
754
757
|
**Use Case:** Keep backups of actively modified files while automatically pruning backups of files that haven't been modified recently - even after the source files are deleted.
|
|
755
758
|
|
|
756
759
|
```yaml
|
|
@@ -820,6 +823,60 @@ repositories:
|
|
|
820
823
|
|
|
821
824
|
**Backup vs Retention:** The per-file `retention_mode` only affects how archives are created and pruned. Traditional backup commands still work normally - you can list, restore, and check per-file archives just like standard archives.
|
|
822
825
|
|
|
826
|
+
### Skip Hash Check for Faster Backups
|
|
827
|
+
|
|
828
|
+
**NEW:** In per-file backup mode, you can optionally skip content hash verification for faster duplicate detection:
|
|
829
|
+
|
|
830
|
+
```yaml
|
|
831
|
+
repositories:
|
|
832
|
+
- name: project-files
|
|
833
|
+
path: /mnt/backup/project-files
|
|
834
|
+
retention_mode: per_file
|
|
835
|
+
skip_hash_check: true # Skip SHA256 content hash verification
|
|
836
|
+
sources:
|
|
837
|
+
- name: projects
|
|
838
|
+
paths:
|
|
839
|
+
- /home/user/projects
|
|
840
|
+
```
|
|
841
|
+
|
|
842
|
+
**How it works:**
|
|
843
|
+
- **Default (paranoid mode)**: Ruborg calculates SHA256 hash of file content to verify files haven't changed (even when size and mtime are identical)
|
|
844
|
+
- **With skip_hash_check: true**: Ruborg trusts file path, size, and modification time for duplicate detection (skips hash calculation)
|
|
845
|
+
|
|
846
|
+
**When to use:**
|
|
847
|
+
- ✅ **Large directories** with thousands of files where hash calculation is slow
|
|
848
|
+
- ✅ **Reliable filesystems** where modification time changes are trustworthy
|
|
849
|
+
- ✅ **Regular backups** where files are unlikely to be manually modified with `touch -t`
|
|
850
|
+
|
|
851
|
+
**When NOT to use:**
|
|
852
|
+
- ❌ **Security-critical data** where you want maximum verification
|
|
853
|
+
- ❌ **Untrusted sources** where files might be tampered with
|
|
854
|
+
- ❌ **Systems with unreliable mtime** (rare, but some network filesystems)
|
|
855
|
+
|
|
856
|
+
**Performance impact:**
|
|
857
|
+
```yaml
|
|
858
|
+
# Example: 10,000 unchanged files, average 50KB each
|
|
859
|
+
# With skip_hash_check: false (default) - ~30 seconds (read + hash all files)
|
|
860
|
+
# With skip_hash_check: true - ~3 seconds (read metadata only)
|
|
861
|
+
```
|
|
862
|
+
|
|
863
|
+
**Console output:**
|
|
864
|
+
```
|
|
865
|
+
# With skip_hash_check: true
|
|
866
|
+
[1/10000] Backing up: /home/user/file1.txt - Archive already exists (skipped hash check)
|
|
867
|
+
[2/10000] Backing up: /home/user/file2.txt - Archive already exists (skipped hash check)
|
|
868
|
+
...
|
|
869
|
+
✓ Per-file backup completed: 50 file(s) backed up, 9950 skipped (hash check skipped)
|
|
870
|
+
|
|
871
|
+
# With skip_hash_check: false (default)
|
|
872
|
+
[1/10000] Backing up: /home/user/file1.txt - Archive already exists (file unchanged)
|
|
873
|
+
[2/10000] Backing up: /home/user/file2.txt - Archive already exists (file unchanged)
|
|
874
|
+
...
|
|
875
|
+
✓ Per-file backup completed: 50 file(s) backed up, 9950 skipped (unchanged)
|
|
876
|
+
```
|
|
877
|
+
|
|
878
|
+
**Security note:** Even with `skip_hash_check: true`, files are still verified by path, size, and mtime. The only difference is skipping the SHA256 content hash verification, which catches rare edge cases like manual file tampering with preserved timestamps.
|
|
879
|
+
|
|
823
880
|
### Automatic Pruning
|
|
824
881
|
|
|
825
882
|
Enable **automatic pruning** to remove old backups after each backup operation:
|