dar-backup 1.0.0.1__py3-none-any.whl → 1.0.2__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: dar-backup
3
- Version: 1.0.0.1
3
+ Version: 1.0.2
4
4
  Summary: A script to do full, differential and incremental backups using dar. Some files are restored from the backups during verification, after which par2 redundancy files are created. The script also has a cleanup feature to remove old backups and par2 files.
5
5
  Project-URL: GPG Public Key, https://keys.openpgp.org/search?q=dar-backup@pm.me
6
6
  Project-URL: Homepage, https://github.com/per2jensen/dar-backup/tree/main/v2
@@ -686,9 +686,9 @@ Classifier: Development Status :: 5 - Production/Stable
686
686
  Classifier: Intended Audience :: End Users/Desktop
687
687
  Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
688
688
  Classifier: Operating System :: POSIX :: Linux
689
- Classifier: Programming Language :: Python :: 3.9
689
+ Classifier: Programming Language :: Python :: 3.11
690
690
  Classifier: Topic :: System :: Archiving :: Backup
691
- Requires-Python: >=3.9
691
+ Requires-Python: >=3.11
692
692
  Requires-Dist: argcomplete>=3.6.2
693
693
  Requires-Dist: inputimeout>=1.0.4
694
694
  Requires-Dist: jinja2>=3.1.6
@@ -722,17 +722,19 @@ Description-Content-Type: text/markdown
722
722
  **Reliable DAR backups, automated in clean Python**
723
723
 
724
724
  [![Codecov](https://codecov.io/gh/per2jensen/dar-backup/branch/main/graph/badge.svg)](https://codecov.io/gh/per2jensen/dar-backup)
725
- [![Snyk Vuln findings](https://snyk.io/test/github/per2jensen/dar-backup/badge.svg)](https://snyk.io/test/github/per2jensen/dar-backup)
725
+ [![Snyk Vuln findings](https://snyk.io/test/github/per2jensen/dar-backup/badge.svg)](https://security.snyk.io/vuln/?search=dar-backup)
726
726
  ![CI](https://github.com/per2jensen/dar-backup/actions/workflows/py-tests.yml/badge.svg)
727
727
  [![PyPI version](https://img.shields.io/pypi/v/dar-backup.svg)](https://pypi.org/project/dar-backup/)
728
728
  [![PyPI downloads](https://img.shields.io/badge/dynamic/json?color=blue&label=PyPI%20downloads&query=total&url=https%3A%2F%2Fraw.githubusercontent.com%2Fper2jensen%2Fdar-backup%2Fmain%2Fdownloads.json)](https://pypi.org/project/dar-backup/)
729
- [![# clones](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/badge_clones.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png)
730
- [![Milestone](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/v2/doc/badges/milestone_badge.json)](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/weekly_clones.png) <sub>🎯 Stats powered by [ClonePulse](https://github.com/per2jensen/clonepulse)</sub>
729
+ [![# clones](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/clonepulse/badge_clones.json)](https://github.com/per2jensen/dar-backup/blob/main/clonepulse/weekly_clones.png)
730
+ [![Milestone](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/per2jensen/dar-backup/main/clonepulse/milestone_badge.json)](https://github.com/per2jensen/dar-backup/blob/main/clonepulse/weekly_clones.png) <sub>🎯 Stats powered by [ClonePulse](https://github.com/per2jensen/clonepulse)</sub>
731
731
 
732
732
  The wonderful 'dar' [Disk Archiver](https://github.com/Edrusb/DAR) is used for
733
733
  the heavy lifting, together with the [parchive](https://github.com/Parchive/par2cmdline) suite in these scripts.
734
734
 
735
- This is the `Python` based [**version 2**](https://github.com/per2jensen/dar-backup/tree/main/v2) of `dar-backup`.
735
+ This is the `Python` based [**version 2**](v2) of `dar-backup`.
736
+
737
+ You can see the [v2 Changelog](v2/Changelog.md) for details on features and progress.
736
738
 
737
739
  ## TL;DR
738
740
 
@@ -746,6 +748,9 @@ Version **1.0.0** was reached on October 9, 2025.
746
748
  - [TL;DR](#tldr)
747
749
  - [Table of Contents](#table-of-contents)
748
750
  - [My use case](#my-use-case)
751
+ - [My setup](#my-setup)
752
+ - [Why PAR2 is especially good for portable / offsite copies](#why-par2-is-especially-good-for-portable--offsite-copies)
753
+ - [Design choices](#design-choices)
749
754
  - [Features](#features)
750
755
  - [License](#license)
751
756
  - [Quick Guide](#quick-guide)
@@ -789,6 +794,8 @@ Version **1.0.0** was reached on October 9, 2025.
789
794
  - [restore test fails with exit code 5](#restore-test-fails-with-exit-code-5)
790
795
  - [Par2](#par2)
791
796
  - [Par2 to verify/repair](#par2-to-verifyrepair)
797
+ - [Par2 files kept with archives](#par2-files-kept-with-archives)
798
+ - [Par2 files in separate directory](#par2-files-in-separate-directory)
792
799
  - [Par2 create redundancy files](#par2-create-redundancy-files)
793
800
  - [Points of interest](#points-of-interest)
794
801
  - [Limitations on File Names with Special Characters](#limitations-on-file-names-with-special-characters)
@@ -806,8 +813,8 @@ Version **1.0.0** was reached on October 9, 2025.
806
813
  - [Performance tip due to par2](#performance-tip-due-to-par2)
807
814
  - [.darrc sets -vd -vf (since v0.6.4)](#darrc-sets--vd--vf-since-v064)
808
815
  - [Separate log file for command output](#separate-log-file-for-command-output)
816
+ - [Trace Logging (Debug details)](#trace-logging-debug-details)
809
817
  - [Skipping cache directories](#skipping-cache-directories)
810
- - [Progress bar and current directory](#progress-bar-and-current-directory)
811
818
  - [Shell autocompletion](#shell-autocompletion)
812
819
  - [Use it](#use-it)
813
820
  - [Archive name completion (smart, context-aware)](#archive-name-completion-smart-context-aware)
@@ -821,12 +828,22 @@ Version **1.0.0** was reached on October 9, 2025.
821
828
  - [CLI Tools Overview](#cli-tools-overview)
822
829
  - [test coverage](#test-coverage)
823
830
  - [Dar-backup options](#dar-backup-options)
831
+ - [Dar-backup exit codes](#dar-backup-exit-codes)
832
+ - [Dar-backup env vars](#dar-backup-env-vars)
824
833
  - [Manager Options](#manager-options)
825
- - [Cleanup options](#cleanup-options)
834
+ - [Cleanup env vars](#cleanup-env-vars)
826
835
  - [Clean-log options](#clean-log-options)
827
836
  - [Dar-backup-systemd options](#dar-backup-systemd-options)
828
837
  - [Installer options](#installer-options)
829
838
  - [Demo options](#demo-options)
839
+ - [Config changes](#config-changes)
840
+ - [1.0.1](#101)
841
+ - [DISCORD WEBHOOK](#discord-webhook)
842
+ - [Restore test config](#restore-test-config)
843
+ - [Par2](#par2-1)
844
+ - [1.0.2](#102)
845
+ - [Trace Logging](#trace-logging)
846
+ - [Command output Capture](#command-output-capture)
830
847
 
831
848
  ## My use case
832
849
 
@@ -836,7 +853,7 @@ I needed the following:
836
853
  - Backup primarily photos, home made video and different types of documents
837
854
  - I have cloud storage mounted on a directory within my home dir. The filesystem is [FUSE based](https://www.kernel.org/doc/html/latest/filesystems/fuse.html), which gives it a few special features
838
855
 
839
- - Backup cloud storage (cloud is convenient, but I want control over my backups)
856
+ - Backup my cloud storage (cloud is convenient, but I want control over my backups)
840
857
  - A non-privileged user can perform a mount
841
858
  - A privileged user cannot look into the filesystem --> a backup script running as root is not suitable
842
859
 
@@ -847,6 +864,48 @@ I needed the following:
847
864
 
848
865
  I do not need the encryption features of dar, as all storage is already encrypted.
849
866
 
867
+ ## My setup
868
+
869
+ 1. Primary backup to server with an ext4 file system on mdadm RAID1
870
+
871
+ 2. Secondary copies to multiple USB disks / cloud
872
+
873
+ 3. Archive integrity verification anywhere using [Par2](#par2) and `dar -t`.
874
+
875
+ 4. Archive repair anywhere if needed. By default `dar-backup` creates par2 redundancy files with 5% coverage. Enough to fix localized bitrot.
876
+
877
+ 5. No dependency on original system
878
+
879
+ ### Why PAR2 is especially good for portable / offsite copies
880
+
881
+ PAR2 parity is:
882
+
883
+ > Self-contained (travels with the data)
884
+ >
885
+ >Format-agnostic (works on any filesystem)
886
+ >
887
+ >Location-agnostic (local disk, USB, cloud object storage)
888
+ >
889
+ >Tool-stable (PAR2 spec has not changed in years)
890
+ >
891
+ >That means:
892
+ >
893
+ >**Integrity protection moves with the archive**.
894
+
895
+ ### Design choices
896
+
897
+ My design choices are boring, proven and pragmatic:
898
+ >
899
+ >mdadm handles disks
900
+ >
901
+ >PAR2 handles data integrity
902
+ >
903
+ >You control when and how verification happens
904
+ >
905
+ >Errors have a fair chance of being diagnosed and fixed, due to well known tooling.
906
+ >
907
+ >No hidden magic, no lock-in
908
+
850
909
  ## Features
851
910
 
852
911
  - The battle tested [dar](https://github.com/Edrusb/DAR) Disk Archiver is used for the actual backups - it comes highly recommended.
@@ -869,7 +928,7 @@ I needed the following:
869
928
  ## License
870
929
 
871
930
  These scripts are licensed under the GPLv3 license.
872
- Read more here: [GNU GPL3.0](https://www.gnu.org/licenses/gpl-3.0.en.html), or have a look at the ["LICENSE"](https://github.com/per2jensen/dar-backup/blob/main/LICENSE) file in this repository.
931
+ Read more here: [GNU GPL3.0](https://www.gnu.org/licenses/gpl-3.0.en.html), or have a look at the ["LICENSE"](LICENSE) file in this repository.
873
932
 
874
933
  ## Quick Guide
875
934
 
@@ -1214,7 +1273,7 @@ This python version is v2 of dar-backup, v1 is made in bash.
1214
1273
 
1215
1274
  ## Community
1216
1275
 
1217
- Please review the [Code of Conduct](https://github.com/per2jensen/dar-backup/blob/main/CODE_OF_CONDUCT.md) to help keep this project welcoming and focused.
1276
+ Please review the [Code of Conduct](CODE_OF_CONDUCT.md) to help keep this project welcoming and focused.
1218
1277
 
1219
1278
  ## Requirements
1220
1279
 
@@ -1234,7 +1293,7 @@ On Ubuntu, install the requirements this way:
1234
1293
 
1235
1294
  ### dar-backup overview
1236
1295
 
1237
- ![dar-backup overview](https://github.com/per2jensen/dar-backup/blob/main/v2/doc/dar-backup-overview.svg)
1296
+ [![dar-backup overview](v2/doc/dar-backup-overview-small.png)](v2/doc/dar-backup-overview.png)
1238
1297
 
1239
1298
  ### dar-backup
1240
1299
 
@@ -1263,6 +1322,15 @@ The `cleanup` application deletes DIFF and INCR if the archives are older than t
1263
1322
 
1264
1323
  `cleanup` will only remove FULL archives if the option `--cleanup-specific-archives` is used. It requires the user to confirm deletion of FULL archives.
1265
1324
 
1325
+ Use `--dry-run` to preview which archives, PAR2 files, and catalogs would be removed without deleting anything.
1326
+
1327
+ Examples:
1328
+
1329
+ ```bash
1330
+ cleanup --dry-run -d media-files --log-stdout
1331
+ cleanup --dry-run --cleanup-specific-archives -d media-files media-files_INCR_2025-12-22
1332
+ ```
1333
+
1266
1334
  ### manager
1267
1335
 
1268
1336
  `dar`has the concept of catalogs which can be exported and optionally be added to a catalog database. That database makes it much easier to restore the correct version of a backed up file if for example a target date has been set.
@@ -1406,7 +1474,7 @@ deactivate
1406
1474
 
1407
1475
  The configuration file's default location is: ~/.config/dar-backup/dar-backup.conf
1408
1476
 
1409
- If you have your config file somewhere else, use the `--config` option to point to it.
1477
+ If you have your config file somewhere else, use the `--config-file` option to point to it.
1410
1478
 
1411
1479
  Tilde `~` and environment variables can be used in the paths for various file locations.
1412
1480
 
@@ -1442,6 +1510,17 @@ INCR_AGE = 40
1442
1510
  [PAR2]
1443
1511
  ERROR_CORRECTION_PERCENT = 5
1444
1512
  ENABLED = True
1513
+ # Optional PAR2 configuration
1514
+ # PAR2_DIR = /path/to/par2-store
1515
+ # PAR2_RATIO_FULL = 10
1516
+ # PAR2_RATIO_DIFF = 5
1517
+ # PAR2_RATIO_INCR = 5
1518
+ # PAR2_RUN_VERIFY = false
1519
+
1520
+ # Optional per-backup overrides (section name = backup definition)
1521
+ [media-files]
1522
+ PAR2_DIR = /mnt/par2/media-files
1523
+ PAR2_RATIO_FULL = 10
1445
1524
 
1446
1525
  # scripts to run before the backup to setup the environment
1447
1526
  [PREREQ]
@@ -1453,6 +1532,16 @@ SCRIPT_1 = df -h
1453
1532
  #SCRIPT_2 = another_script.sh
1454
1533
  ```
1455
1534
 
1535
+ PAR2 notes:
1536
+
1537
+ - If `PAR2_DIR` is unset, par2 files are created next to the archive slices (legacy behavior) and no manifest is written
1538
+ - When `PAR2_DIR` is set, dar-backup writes a manifest next to the par2 set:
1539
+ `archive_base.par2.manifest.ini`
1540
+ - When generating a par2 set, par2 reads all archive slices before writing any output files; for large backups, this initial read can take hours
1541
+ - Verify or repair using:
1542
+ `par2 verify -B <archive_dir> <par2_set.par2>`
1543
+ `par2 repair -B <archive_dir> <par2_set.par2>`
1544
+
1456
1545
  ### .darrc
1457
1546
 
1458
1547
  The package includes a default `darrc` file which configures `dar`.
@@ -1774,6 +1863,25 @@ This happens when the shell splits the quoted string or interprets globs before
1774
1863
 
1775
1864
  > 💡 **Tip:** See [dar's documentation](http://dar.linux.free.fr/doc/man/dar.html#COMMANDS%20AND%20OPTIONS)
1776
1865
 
1866
+ >
1867
+ > 💡💡 **Tip:** To filter all the empty directories away that `dar` emits when listing contents, append this grep:
1868
+ >
1869
+ > ```bash
1870
+ > |grep -vE '\s+d[rwx-]{9}\s'
1871
+ >```
1872
+ >
1873
+ >Example using the grep to discard directory noise from `dar's` output:
1874
+ >
1875
+ > ```bash
1876
+ > dar-backup --list-contents media-files_INCR_2025-05-10 --selection="-I '*Z50*' -X '*.xmp'" | grep -vE '\s+d[rwx-]{9}\s'
1877
+ >[Saved][ ] [-L-][ 0%][X] -rw-rw-r-- user user 26 Mio Fri May 9 11:26:16 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0633.NEF
1878
+ >[Saved][ ] [-L-][ 0%][X] -rw-rw-r-- user user 26 Mio Fri May 9 11:26:16 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0632.NEF
1879
+ >[Saved][ ] [-L-][ 0%][X] -rw-rw-r-- user user 28 Mio Fri May 9 11:09:04 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0631.NEF
1880
+ >[Saved][ ] [-L-][ 0%][X] -rw-rw-r-- user user 29 Mio Fri May 9 11:09:03 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0630.NEF
1881
+ >[Saved][ ] [-L-][ 0%][X] -rw-rw-r-- user user 29 Mio Fri May 9 11:09:03 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0629.NEF
1882
+ >...
1883
+ >```
1884
+
1777
1885
  ### select a directory
1778
1886
 
1779
1887
  Select files and sub directories in `home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling`
@@ -1900,9 +2008,39 @@ If you need to use this option, un-comment it in the [.darrc](#darrc) file (loca
1900
2008
 
1901
2009
  ## Par2
1902
2010
 
2011
+ Why keep PAR2 on a different storage device:
2012
+
2013
+ - Reduces single-disk failure impact: bitrot on the archive disk does not affect the parity.
2014
+ - Easier offsite rotation: you can sync only the PAR2 sets to a different failure domain.
2015
+
2016
+ Redundancy guidance:
2017
+
2018
+ - FULL backups: 10% is a practical default for larger data sets and longer retention.
2019
+ - DIFF/INCR: 5% is often enough because the delta is smaller and easier to re-create.
2020
+ - Increase the ratio if the storage is flaky or the backup is hard to re-run.
2021
+
2022
+ Rule of thumb table:
2023
+
2024
+ | Backup type | Suggested PAR2 ratio | Notes |
2025
+ |-------------|----------------------|-------|
2026
+ | FULL | 10% | Longer retention, larger data set |
2027
+ | DIFF | 5% | Smaller delta |
2028
+ | INCR | 5% | Smaller delta |
2029
+
2030
+ >
2031
+ >For large, contiguous archives on reliable local storage, 7–8% has proven sufficient in practice; 10% remains a conservative default.
2032
+ >
2033
+
2034
+ Cloud sync / air-gap note:
2035
+
2036
+ - Syncing PAR2 sets to a different device or remote store protects against bitrot and small corruption, but it cannot recover a completely lost archive.
2037
+ - An air-gapped PAR2 store is useful when the archive disk is exposed to ransomware or accidental deletion.
2038
+
1903
2039
  ### Par2 to verify/repair
1904
2040
 
1905
- You can run a par2 verification on an archive like this:
2041
+ #### Par2 files kept with archives
2042
+
2043
+ If PAR2 files are stored next to the archives (legacy per-slice behavior), you can verify like this:
1906
2044
 
1907
2045
  ```bash
1908
2046
  for file in <archive>*.dar.par2; do
@@ -1916,6 +2054,14 @@ if there are problems with a slice, try to repair it like this:
1916
2054
  par2 repair <archive>.<slice number>.dar.par2
1917
2055
  ```
1918
2056
 
2057
+ #### Par2 files in separate directory
2058
+
2059
+ See [docs on disk layout matters](v2/doc/portable-par2-layout.md)
2060
+
2061
+ >Test case proving this flow:
2062
+ >
2063
+ >[tests/test_par2_manifest.py](v2/tests/test_par2_manifest.py)
2064
+
1919
2065
  ### Par2 create redundancy files
1920
2066
 
1921
2067
  If you have merged archives, you will need to create the .par2 redundency files manually.
@@ -1929,6 +2075,15 @@ done
1929
2075
 
1930
2076
  where "c" is create, -r5 is 5% redundency and -n1 is 1 redundency file
1931
2077
 
2078
+ If you want to create a single parity set for all slices in an archive:
2079
+
2080
+ ```bash
2081
+ par2 create -B <archive_dir> -r5 <par2_dir>/<archive_base>.par2 <archive_dir>/<archive_base>.*.dar
2082
+ ```
2083
+
2084
+ **OBSERVE** [docs on disk layout matters](v2/doc/portable-par2-layout.md)
2085
+
2086
+
1932
2087
  ## Points of interest
1933
2088
 
1934
2089
  ### Limitations on File Names with Special Characters
@@ -2083,23 +2238,29 @@ In order to not clutter that log file with the output of commands being run, a n
2083
2238
 
2084
2239
  The secondary log file can get quite cluttered, if you want to remove the clutter, run the `clean-log`script with the `--file` option, or simply delete it.
2085
2240
 
2086
- ### Skipping cache directories
2241
+ ### Trace Logging (Debug details)
2087
2242
 
2088
- The author uses the `--cache-directory-tagging` option in his [backup definitions](#backup-definition-example).
2243
+ To keep the main log file clean while preserving essential debugging information, `dar-backup` creates a separate trace log file (e.g., `dar-backup.trace.log`) alongside the main log.
2089
2244
 
2090
- The effect is that directories with the [CACHEDIR.TAG](https://bford.info/cachedir/) file are not backed up. Those directories contain content fetched from the net, which is of an ephemeral nature and probably not what you want to back up.
2245
+ - **Main Log (`dar-backup.log`)**: Contains clean, human-readable INFO/ERROR messages. Stack traces are suppressed here.
2246
+ - **Trace Log (`dar-backup.trace.log`)**: Captures ALL messages at `DEBUG` level, including full exception stack traces. Use this file for debugging crashes or unexpected behavior.
2091
2247
 
2092
- If the option is not in the backup definition, the cache directories are backed up as any other.
2248
+ You can configure the rotation of this file in `[MISC]`:
2249
+
2250
+ ```ini
2251
+ [MISC]
2252
+ # ... other settings ...
2253
+ TRACE_LOG_MAX_BYTES = 10485760 # 10 MB default
2254
+ TRACE_LOG_BACKUP_COUNT = 1 # Keep 1 old trace file (default)
2255
+ ```
2093
2256
 
2094
- ### Progress bar and current directory
2257
+ ### Skipping cache directories
2095
2258
 
2096
- If you run dar-backup interactively in a "normal" console on your computer,
2097
- dar-backup displays 2 visual artifacts to show progress.
2259
+ The author uses the `--cache-directory-tagging` option in his [backup definitions](#backup-definition-example).
2098
2260
 
2099
- 1. a progress bar that fills up and starts over
2100
- 2. a status line showing the directory being backed up. If the directory is big and takes time to backup, the line is not changing, but you will probably know there is a lot to backup.
2261
+ The effect is that directories with the [CACHEDIR.TAG](https://bford.info/cachedir/) file are not backed up. Those directories contain content fetched from the net, which is of an ephemeral nature and probably not what you want to back up.
2101
2262
 
2102
- The indicators are not shown if dar-backup is run from systemd or if it is used in terminal multiplexers like `tmux` or `screen`. So no polluting of journald logs.
2263
+ If the option is not in the backup definition, the cache directories are backed up as any other.
2103
2264
 
2104
2265
  ### Shell autocompletion
2105
2266
 
@@ -2284,7 +2445,6 @@ pytest # run the test suite
2284
2445
 
2285
2446
  - Perhaps look into pre-processing backup definitions. As `dar` does not expand env vars
2286
2447
  `dar-backup` could do so and feed the result to `dar`.
2287
- - When run interactively, a progress bar during test and par2 generation would be nice.
2288
2448
  - Look into a way to move the .par2 files away from the `dar` slices, to maximize chance of good redundancy.
2289
2449
  - Add option to dar-backup to use the `dar` option `--fsa-scope none`
2290
2450
 
@@ -2309,15 +2469,15 @@ One backup definition per file
2309
2469
 
2310
2470
  ### CLI Tools Overview
2311
2471
 
2312
- | Command | Description |
2313
- |----------------------|-------------------------------------------|
2314
- | [dar-backup](#dar-backup-options)| Perform full, differential, or incremental backups with verification and restore testing |
2315
- | [manager](#manager-options) | Maintain and query catalog databases for archives |
2316
- | [cleanup](#cleanup-options) | Remove outdated DIFF/INCR archives (and optionally FULLs) |
2317
- | [clean-log](#clean-log-options) | Clean up excessive log output from dar command logs |
2472
+ | Command | Description |
2473
+ | --- | --- |
2474
+ | [dar-backup](#dar-backup-options) | Perform full, differential, or incremental backups with verification and restore testing |
2475
+ | [manager](#manager-options) | Maintain and query catalog databases for archives |
2476
+ | [cleanup](#cleanup-options) | Remove outdated DIFF/INCR archives (and optionally FULLs) |
2477
+ | [clean-log](#clean-log-options) | Clean up excessive log output from dar command logs |
2318
2478
  | [dar-backup-systemd](#dar-backup-systemd-options) | Generate (and optionally install) systemd timers and services for automated backups |
2319
- | [installer](#installer-options) | Set up directories and optionally create catalog databases according to a config file |
2320
- | [demo](#demo-options) | Set up required directories and config files for a demo|
2479
+ | [installer](#installer-options) | Set up directories and optionally create catalog databases according to a config file |
2480
+ | [demo](#demo-options) | Set up required directories and config files for a demo|
2321
2481
 
2322
2482
  ### test coverage
2323
2483
 
@@ -2372,6 +2532,7 @@ Available options:
2372
2532
  --examples Show examples of using dar-backup.py.
2373
2533
  -l, --list List available backups.
2374
2534
  --list-contents <archive> List the contents of a specified archive.
2535
+ --list-definitions List backup definitions from BACKUP.D_DIR.
2375
2536
  --selection <params> Define file selection for listing/restoring.
2376
2537
  --restore <archive> Restore a specified archive.
2377
2538
  -r, --restore <archive> Restore archive.
@@ -2381,6 +2542,7 @@ Available options:
2381
2542
  --log-level <level> `debug` or `trace`, default is `info`.
2382
2543
  --log-stdout Also print log messages to stdout.
2383
2544
  --do-not-compare Do not compare restores to file system.
2545
+ --preflight-check Run preflight checks and exit (runs automatically; this flag just exits after checks).
2384
2546
  --examples Show examples of using dar-backup.
2385
2547
  --readme Print README.md and exit
2386
2548
  --readme-pretty Print README.md with Markdown styling and exit
@@ -2389,6 +2551,22 @@ Available options:
2389
2551
  -v, --version Show version and license information.
2390
2552
  ```
2391
2553
 
2554
+ #### Dar-backup exit codes
2555
+
2556
+ - 0: Success.
2557
+ - 1: Error (backup/restore/preflight failure).
2558
+ - 2: Warning (restore test failed or backup already exists and is skipped).
2559
+ - 127: Typically an error during startup, file or config value missing
2560
+ - if the `dar -t` test fails, exit code 1 is emitted
2561
+ - restore tests could fail if the source file has changed after the backup
2562
+
2563
+ #### Dar-backup env vars
2564
+
2565
+ | Env var | Value | Description |
2566
+ | --- | --- | --- |
2567
+ | DAR_BACKUP_CONFIG_FILE | Full path to config file | Overrides built-in default, overridden by --config-file |
2568
+ | DAR_BACKUP_DISCORD_WEBHOOK_URL | https://discord.com/api/webhooks/\<userID\>/\<webhook UUID\> | The full url |
2569
+
2392
2570
  ### Manager Options
2393
2571
 
2394
2572
  This script manages `dar` databases and catalogs.
@@ -2408,7 +2586,12 @@ Available options:
2408
2586
  --find-file <file> Search catalogs for a specific file.
2409
2587
  --verbose Enable verbose output.
2410
2588
  --log-level <level> Set log level (`debug` or `trace`, default is `info`).
2411
- ```
2589
+
2590
+ #### Manager env vars
2591
+
2592
+ | Env var | Value | Description |
2593
+ | --- | --- | --- |
2594
+ | DAR_BACKUP_CONFIG_FILE | path to the config file | Default is $HOME/.config/dar-backup/dar-backup.conf |```
2412
2595
 
2413
2596
  ### Cleanup options
2414
2597
 
@@ -2425,12 +2608,19 @@ Supported options:
2425
2608
  --alternate-archive-dir Clean up in this directory instead of the default one.
2426
2609
  --cleanup-specific-archives "<archive>, <>, ..." Comma separated list of archives to cleanup.
2427
2610
  -l, --list List available archives (filter using the -d option).
2611
+ --dry-run Show what would be deleted without removing files.
2428
2612
  --verbose Print various status messages to screen.
2429
2613
  --log-level <level> `debug` or `trace`, default is `info`", default="info".
2430
2614
  --log-stdout Print log messages to stdout.
2431
2615
  --test-mode This is used when running pytest test cases
2432
2616
  ```
2433
2617
 
2618
+ #### Cleanup env vars
2619
+
2620
+ | Env var | Value | Description |
2621
+ | --- | --- | --- |
2622
+ | DAR_BACKUP_CONFIG_FILE | path to the config file | Default is $HOME/.config/dar-backup/dar-backup.conf |
2623
+
2434
2624
  ### Clean-log options
2435
2625
 
2436
2626
  This script removes excessive logging output from `dar` logs, improving readability and efficiency. Available options:
@@ -2480,8 +2670,8 @@ Create directories:
2480
2670
  - ~/.config/dar-backup/
2481
2671
  - ~/.config/dar-backup/backup.d/
2482
2672
  - ~/dar-backup/
2483
- - ~/dar-backup/backups
2484
- - ~/dar-backup/restore
2673
+ - ~/dar-backup/backups/
2674
+ - ~/dar-backup/restore/
2485
2675
 
2486
2676
  Sets up demo config files:
2487
2677
 
@@ -2500,3 +2690,134 @@ Sets up demo config files:
2500
2690
  -v, --version Display version and licensing information.
2501
2691
  -h, --help Displays usage info
2502
2692
  ```
2693
+
2694
+ ### Config changes
2695
+
2696
+ #### 1.0.1
2697
+
2698
+ ##### DISCORD WEBHOOK
2699
+
2700
+ For Discord notifications use the `DAR_BACKUP_DISCORD_WEBHOOK_URL` environment variable. It should not be placed in the config file.
2701
+
2702
+ DAR_BACKUP_DISCORD_WEBHOOK_URL is the entire endpoint like this:
2703
+
2704
+ ```bash
2705
+ export DAR_BACKUP_DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/\<userId\>/\<uuid\>
2706
+ ```
2707
+
2708
+ ##### Restore test config
2709
+
2710
+ Restore tests choose random files from the archive and compare them with the live filesystem.
2711
+ To avoid noisy paths (caches, temp files, logs), you can exclude candidates before the random
2712
+ selection happens. All matching is case-insensitive.
2713
+
2714
+ Config keys (in [MISC]):
2715
+
2716
+ - RESTORETEST_EXCLUDE_PREFIXES: comma-separated path prefixes to skip. Matches from the start of
2717
+ the path (after trimming a leading "/"). Use trailing "/" for directories.
2718
+ - RESTORETEST_EXCLUDE_SUFFIXES: comma-separated filename suffixes to skip.
2719
+ - RESTORETEST_EXCLUDE_REGEX: optional regex to skip anything matching the path.
2720
+
2721
+ Example:
2722
+
2723
+ ```ini
2724
+ [MISC]
2725
+ RESTORETEST_EXCLUDE_PREFIXES = .cache/, .local/share/Trash/, .mozilla/, snap/firefox/common/.mozilla/
2726
+ RESTORETEST_EXCLUDE_SUFFIXES = .sqlite-wal, .sqlite-shm, .log, .tmp, .lock, .journal
2727
+ RESTORETEST_EXCLUDE_REGEX = (^|/)(Cache|cache|Logs|log)/
2728
+ ```
2729
+
2730
+ Regex tips (case-insensitive):
2731
+
2732
+ - Match common cache/log directories anywhere:
2733
+ `(^|/)(cache|logs)/`
2734
+ - Skip thumbnails and temp dirs:
2735
+ `(^|/)(thumbnails|tmp|temp)/`
2736
+ - Exclude browser profile noise while keeping other files:
2737
+ `(^|/)\.mozilla/|/snap/firefox/common/\.mozilla/`
2738
+
2739
+ ##### Par2
2740
+
2741
+ New optional PAR2 settings were added to the config file. If none of these keys are added, dar-backup behaves exactly as before (PAR2 files next to archives, per-slice parity).
2742
+
2743
+ | Name | Description | When it is in effect | Suggested value |
2744
+ |------|-------------|----------------------|-----------------|
2745
+ | PAR2_DIR | Directory to store .par2 and .vol*.par2 files | When set | A different device or mount from BACKUP_DIR |
2746
+ | PAR2_RATIO_FULL | Redundancy percent for FULL | When set | 10 (%) |
2747
+ | PAR2_RATIO_DIFF | Redundancy percent for DIFF | When set | 5 (%)|
2748
+ | PAR2_RATIO_INCR | Redundancy percent for INCR | When set | 5 (%)|
2749
+ | PAR2_RUN_VERIFY | Verify after create | When set | false |
2750
+
2751
+ Notes:
2752
+
2753
+ - PAR2_RATIO_*, and PAR2_RUN_VERIFY apply even if PAR2_DIR is not set (i.e. par2 output stays next to the archives).
2754
+
2755
+ Per-backup overrides use a section named after the backup definition with the same PAR2_* keys:
2756
+
2757
+ ```text
2758
+
2759
+ ######################################################################
2760
+ # Per-backup configuration example overrides
2761
+ ######################################################################
2762
+
2763
+ # --------------------------------------------------------------------
2764
+ # Per-backup overrides (section name must match backup.d filename stem)
2765
+ # Example: backup.d/home.conf -> [home]
2766
+ # --------------------------------------------------------------------
2767
+
2768
+ #[home]
2769
+ # Disable PAR2 entirely for this backup definition
2770
+ PAR2_ENABLED = false
2771
+ #
2772
+ #[media]
2773
+ # Store PAR2 files in a separate location for this backup definition
2774
+ #PAR2_DIR = /samba/par2/media
2775
+ # Raise redundancy only for FULL
2776
+ #
2777
+ [documents]
2778
+ # Run verify par2 sets after creation
2779
+ PAR2_RUN_VERIFY = true
2780
+ #
2781
+ #[etc]
2782
+ # Keep global PAR2 settings but tweak ratios for this backup definition
2783
+ # RATIO is given in percent (%)
2784
+ #PAR2_RATIO_FULL = 15
2785
+ #PAR2_RATIO_DIFF = 8
2786
+ #PAR2_RATIO_INCR = 8
2787
+ ```
2788
+
2789
+ [Per-backup override test case: `tests/test_par2_overrides.py`](v2/tests/test_par2_overrides.py)
2790
+
2791
+ #### 1.0.2
2792
+
2793
+ ##### Trace Logging
2794
+
2795
+ To support debugging without cluttering the main log file, a secondary trace log is now created (e.g., `dar-backup.trace.log`).
2796
+ This file captures all `DEBUG` level messages and full exception stack traces.
2797
+
2798
+ You can configure its rotation in the `[MISC]` section:
2799
+
2800
+ - `TRACE_LOG_MAX_BYTES`: Max size of the trace log file in bytes. Default is `10485760` (10 MB).
2801
+ - `TRACE_LOG_BACKUP_COUNT`: Number of rotated trace log files to keep. Default is `1`.
2802
+
2803
+ Example:
2804
+
2805
+ ```ini
2806
+ [MISC]
2807
+ TRACE_LOG_MAX_BYTES = 10485760
2808
+ TRACE_LOG_BACKUP_COUNT = 1
2809
+ ```
2810
+
2811
+ ##### Command output Capture
2812
+
2813
+ - New optional `[MISC]` setting: `COMMAND_CAPTURE_MAX_BYTES` (default 102400).
2814
+ - Limits how much stdout/stderr is kept in memory per command while still logging full output.
2815
+ - Set to `0` to disable buffering entirely. Command output is still streamed to dar-backup-commands.log
2816
+ - If set to `0`, the calling function cannot rely on output from the executed command. The exit value is the only result provided.
2817
+
2818
+ Example:
2819
+
2820
+ ```ini
2821
+ [MISC]
2822
+ COMMAND_CAPTURE_MAX_BYTES = 102400
2823
+ ```
@@ -0,0 +1,25 @@
1
+ dar_backup/.darrc,sha256=-aerqivZmOsW_XBCh9IfbYTUvw0GkzDSr3Vx4GcNB1g,2113
2
+ dar_backup/Changelog.md,sha256=gIAlrXElafjmReneKZlhaPd5FcXU7SJ58_P-huzLTO8,19436
3
+ dar_backup/README.md,sha256=b9w97562AlVTE7ob_5kmf1pc9EiUi7swv8g11M4NOrE,78420
4
+ dar_backup/__about__.py,sha256=0lLpLB4_DxcwxlIZF1F8V8O6WVfjzKepl2PPulVwn2Q,370
5
+ dar_backup/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
6
+ dar_backup/clean_log.py,sha256=Uf6MjpS_v1IVZdfx7ogRo6j5EIscdxjLImn8GKZo_rc,7038
7
+ dar_backup/cleanup.py,sha256=aC7ogT68u570ufXrzFgpCcxWgzN5dX0o5LIgZhgbQKo,19147
8
+ dar_backup/command_runner.py,sha256=ktK_LmVUZATZlfXPniGLC4W_8x187OzKreK7Dhb5LUA,13123
9
+ dar_backup/config_settings.py,sha256=PKe6iTVeSsN1DW4kTgFbUHsl8S5E-f0I3Lhn4P9a4hs,12723
10
+ dar_backup/dar-backup.conf,sha256=ejucmhEbSdqFU4JEVfrWbFj9H6U87TWSnhex3LraEUY,2124
11
+ dar_backup/dar-backup.conf.j2,sha256=kD13msmP2IXLjnJ9lZmqtLTnsUZq1EkPZAI1LlOkYXo,3852
12
+ dar_backup/dar_backup.py,sha256=997dp6zjXluoenldo19KYtDVt91uMJxoHOI6cIVzohs,69805
13
+ dar_backup/dar_backup_systemd.py,sha256=PwAc2H2J3hQLWpnC6Ib95NZYtB2G2NDgkSblfLj1n10,3875
14
+ dar_backup/demo.py,sha256=KaW-KOqP1sBX4TYJoO-Wr84e81ILI2kN2iV57QU6rbg,8243
15
+ dar_backup/demo_backup_def.j2,sha256=hQW2Glp0QGV3Kt8cwjS0mpOCdyzjVlpgbgL6LpXTKJA,1793
16
+ dar_backup/exceptions.py,sha256=dBQwgKUilgfb1Tnlm_m1Dl1IqoY75L0n1uRkm0DBVTk,158
17
+ dar_backup/installer.py,sha256=yJ0xXi24C1Vo6gnj2BlhV1X55ZIG5t21t8tNKTYSclM,7616
18
+ dar_backup/manager.py,sha256=rctD1WNdSwyEiDw81wnhPyzIVKjrax2QU71PRSx4MqQ,34995
19
+ dar_backup/rich_progress.py,sha256=SfwFxebBl6jnDQMUQr4McknkW1yQWaJVo1Ju1OD3okA,3221
20
+ dar_backup/util.py,sha256=wVyHzdZIQSnXPSup2m0Wx1w-PKx_ornsg4MMEW5AoKs,38608
21
+ dar_backup-1.0.2.dist-info/METADATA,sha256=7_shfQw0tpdJbnlfR46Abd03LxkTzQZj_Xv1mtjNAIM,121126
22
+ dar_backup-1.0.2.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
23
+ dar_backup-1.0.2.dist-info/entry_points.txt,sha256=pOK9M8cHeAcGIatrYzkm_1O89kPk0enyYONALYjFBx4,286
24
+ dar_backup-1.0.2.dist-info/licenses/LICENSE,sha256=OXLcl0T2SZ8Pmy2_dmlvKuetivmyPd5m1q-Gyd-zaYY,35149
25
+ dar_backup-1.0.2.dist-info/RECORD,,
@@ -1,4 +1,4 @@
1
1
  Wheel-Version: 1.0
2
- Generator: hatchling 1.27.0
2
+ Generator: hatchling 1.28.0
3
3
  Root-Is-Purelib: true
4
4
  Tag: py3-none-any