dar-backup 1.0.1__tar.gz → 1.0.2__tar.gz

{dar_backup-1.0.1 → dar_backup-1.0.2}/.gitignore

@@ -16,6 +16,7 @@ v2/old_dar_backup
 v2/dist
 v2/build
 v2/.coverage
+v2/.coverage*
 v2/dar-backup.tar
 v2/coverage.xml
 v2/packages/*

{dar_backup-1.0.1/src/dar_backup → dar_backup-1.0.2}/Changelog.md

@@ -1,6 +1,35 @@
 <!-- markdownlint-disable MD024 -->
 # dar-backup Changelog
 
+## v2-1.0.2 - 2026-01-25
+
+### Added
+
+- Streaming restore-test sampling using reservoir sampling to avoid holding full file lists in memory.
+- Configurable command output capture cap (`COMMAND_CAPTURE_MAX_BYTES`, default 100 KB) to limit in-memory stdout/stderr while still logging full output.
+- Streaming list output for `dar-backup --list-contents` and `manager --list-archive-contents` to avoid large in-memory buffers.
+- Test coverage additions for config parsing, util helpers, restore-test sampling edge cases, par2 slice helpers, and get_backed_up_files error paths.
+- Tests for missing source files during restore verification and for par2 generation order after verification.
+- CommandRunner test coverage for sanitize failure notes, text/binary output handling, timeouts, Popen failures, and TTY restore logic.
+- Tests for COMMAND_CAPTURE_MAX_BYTES defaults (0 and 1k) and binary stdout/stderr capture with truncation and log_output disabled.
+- Manager test coverage for create-db guardrails and catalog listing parsing/sorting across runner/subprocess paths.
+- Cleanup now reports PREREQ/POSTREQ failures cleanly and sends Discord failure notifications when configured.
+- New trace logger that always logs at DEBUG and captures stacktraces if they happens. Default max size is 10MB + 1 rollover file.
+
+### Changed
+
+- BUGFIX: Ensure existing files are removed before restore verification to prevent false positives.
+- Clears out restore-test directory on program start to ensure a clean slate.
+- Restore-test selection now streams DAR XML listings and samples candidates without loading all entries into RAM.
+- `get_backed_up_files` uses incremental XML parsing to reduce memory use for large archives.
+- Restore verification now logs a warning and continues when a source or restored file is missing during comparison.
+- CommandRunner supports per-command capture cap overrides (disable cap with `capture_output_limit_bytes=-1`).
+- Cleanup now rejects unsafe archive names when `--cleanup-specific-archives` is used to prevent accidental deletions.
+- Removed deprecated PAR2 layout/mode settings and simplified PAR2 cleanup to delete all matching .par2 artifacts.
+- Config templates/docs updated to drop PAR2_LAYOUT/PAR2_MODE references.
+- [Snyk] Python 3.11 required in pyproject.toml. Snyk has flagged a vulnerabilly in an xml parser that requires the bump to 3.11.
+- Config parsing errors now emit concise messages (no stack trace) and trigger Discord failure notifications in CLI tools.
+
 ## v2-1.0.1 - 2026-01-09
 
 ### Added

{dar_backup-1.0.1 → dar_backup-1.0.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dar-backup
-Version: 1.0.1
+Version: 1.0.2
 Summary: A script to do full, differential and incremental backups using dar.     Some files are restored from the backups during verification, after which par2 redundancy files are created.     The script also has a cleanup feature to remove old backups and par2 files.
 Project-URL: GPG Public Key, https://keys.openpgp.org/search?q=dar-backup@pm.me
 Project-URL: Homepage, https://github.com/per2jensen/dar-backup/tree/main/v2
@@ -686,9 +686,9 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: End Users/Desktop
 Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
 Classifier: Operating System :: POSIX :: Linux
-Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.11
 Classifier: Topic :: System :: Archiving :: Backup
-Requires-Python: >=3.9
+Requires-Python: >=3.11
 Requires-Dist: argcomplete>=3.6.2
 Requires-Dist: inputimeout>=1.0.4
 Requires-Dist: jinja2>=3.1.6
@@ -813,8 +813,8 @@ Version **1.0.0** was reached on October 9, 2025.
     - [Performance tip due to par2](#performance-tip-due-to-par2)
     - [.darrc sets -vd -vf (since v0.6.4)](#darrc-sets--vd--vf-since-v064)
     - [Separate log file for command output](#separate-log-file-for-command-output)
+    - [Trace Logging (Debug details)](#trace-logging-debug-details)
     - [Skipping cache directories](#skipping-cache-directories)
-    - [Progress bar and current directory](#progress-bar-and-current-directory)
     - [Shell autocompletion](#shell-autocompletion)
       - [Use it](#use-it)
       - [Archive name completion (smart, context-aware)](#archive-name-completion-smart-context-aware)
@@ -841,6 +841,9 @@ Version **1.0.0** was reached on October 9, 2025.
         - [DISCORD WEBHOOK](#discord-webhook)
         - [Restore test config](#restore-test-config)
         - [Par2](#par2-1)
+      - [1.0.2](#102)
+        - [Trace Logging](#trace-logging)
+        - [Command output Capture](#command-output-capture)
   
 ## My use case
 
@@ -1509,7 +1512,6 @@ ERROR_CORRECTION_PERCENT = 5
 ENABLED = True
 # Optional PAR2 configuration
 # PAR2_DIR = /path/to/par2-store
-# PAR2_MODE = per-slice
 # PAR2_RATIO_FULL = 10
 # PAR2_RATIO_DIFF = 5
 # PAR2_RATIO_INCR = 5
@@ -1518,7 +1520,6 @@ ENABLED = True
 # Optional per-backup overrides (section name = backup definition)
 [media-files]
 PAR2_DIR = /mnt/par2/media-files
-PAR2_MODE = per-archive
 PAR2_RATIO_FULL = 10
 
 # scripts to run before the backup to setup the environment
@@ -1536,6 +1537,7 @@ PAR2 notes:
 - If `PAR2_DIR` is unset, par2 files are created next to the archive slices (legacy behavior) and no manifest is written
 - When `PAR2_DIR` is set, dar-backup writes a manifest next to the par2 set:
   `archive_base.par2.manifest.ini`
+- When generating a par2 set, par2 reads all archive slices before writing any output files; for large backups, this initial read can take hours
 - Verify or repair using:
   `par2 verify -B <archive_dir> <par2_set.par2>`
   `par2 repair -B <archive_dir> <par2_set.par2>`
@@ -1861,6 +1863,25 @@ This happens when the shell splits the quoted string or interprets globs before
 
 > 💡 **Tip:** See [dar's documentation](http://dar.linux.free.fr/doc/man/dar.html#COMMANDS%20AND%20OPTIONS)
 
+>
+> 💡💡 **Tip:** To filter all the empty directories away that `dar` emits when listing  contents, append this grep:
+>
+> ```bash
+> |grep -vE '\s+d[rwx-]{9}\s'
+>```
+>
+>Example using the grep to discard directory noise from `dar's` output:
+>
+> ```bash
+> dar-backup --list-contents media-files_INCR_2025-05-10 --selection="-I '*Z50*' -X '*.xmp'" | grep -vE '\s+d[rwx-]{9}\s'
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 26  Mio Fri May  9 11:26:16 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0633.NEF
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 26  Mio Fri May  9 11:26:16 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0632.NEF
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 28  Mio Fri May  9 11:09:04 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0631.NEF
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 29  Mio Fri May  9 11:09:03 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0630.NEF
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 29  Mio Fri May  9 11:09:03 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0629.NEF
+>...
+>```
+
 ### select a directory
 
 Select files and sub directories in `home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling`
@@ -2217,23 +2238,29 @@ In order to not clutter that log file with the output of commands being run, a n
 
 The secondary log file can get quite cluttered, if you want to remove the clutter, run the `clean-log`script with the `--file` option, or simply delete it.
 
-### Skipping cache directories
+### Trace Logging (Debug details)
 
-The author uses the `--cache-directory-tagging` option in his [backup definitions](#backup-definition-example).
+To keep the main log file clean while preserving essential debugging information, `dar-backup` creates a separate trace log file (e.g., `dar-backup.trace.log`) alongside the main log.
 
-The effect is that directories with the [CACHEDIR.TAG](https://bford.info/cachedir/) file are not backed up. Those directories contain content fetched from the net, which is of an ephemeral nature and probably not what you want to back up.
+- **Main Log (`dar-backup.log`)**: Contains clean, human-readable INFO/ERROR messages. Stack traces are suppressed here.
+- **Trace Log (`dar-backup.trace.log`)**: Captures ALL messages at `DEBUG` level, including full exception stack traces. Use this file for debugging crashes or unexpected behavior.
 
-If the option is not in the backup definition, the cache directories are backed up as any other.
+You can configure the rotation of this file in `[MISC]`:
 
-### Progress bar and current directory
+```ini
+[MISC]
+# ... other settings ...
+TRACE_LOG_MAX_BYTES = 10485760  # 10 MB default
+TRACE_LOG_BACKUP_COUNT = 1      # Keep 1 old trace file (default)
+```
 
-If you run dar-backup interactively in a "normal" console on your computer,
-dar-backup displays 2 visual artifacts to show progress.
+### Skipping cache directories
 
-1. a progress bar that fills up and starts over
-2. a status line showing the directory being backed up. If the directory is big and takes time to backup, the line is not changing, but you will probably know there is a lot to backup.
+The author uses the `--cache-directory-tagging` option in his [backup definitions](#backup-definition-example).
 
-The indicators are not shown if dar-backup is run from systemd or if it is used in terminal multiplexers like `tmux` or `screen`. So no polluting of journald logs.
+The effect is that directories with the [CACHEDIR.TAG](https://bford.info/cachedir/) file are not backed up. Those directories contain content fetched from the net, which is of an ephemeral nature and probably not what you want to back up.
+
+If the option is not in the backup definition, the cache directories are backed up as any other.
 
 ### Shell autocompletion
 
@@ -2418,7 +2445,6 @@ pytest                   # run the test suite
 
 - Perhaps look into pre-processing backup definitions. As `dar` does not expand env vars
   `dar-backup` could do so and feed the result to `dar`.
-- When run interactively, a progress bar during test and par2 generation would be nice.
 - Look into a way to move the .par2 files away from the `dar` slices, to maximize chance of good redundancy.
 - Add option to dar-backup to use the `dar` option `--fsa-scope none`
 
@@ -2754,10 +2780,44 @@ PAR2_RUN_VERIFY = true
 #
 #[etc]
 # Keep global PAR2 settings but tweak ratios for this backup definition
-# RATIO is i percent number
+# RATIO is given in percent (%)
 #PAR2_RATIO_FULL = 15  
 #PAR2_RATIO_DIFF = 8
 #PAR2_RATIO_INCR = 8
 ```
 
 [Per-backup override test case: `tests/test_par2_overrides.py`](v2/tests/test_par2_overrides.py)
+
+#### 1.0.2
+
+##### Trace Logging
+
+To support debugging without cluttering the main log file, a secondary trace log is now created (e.g., `dar-backup.trace.log`).
+This file captures all `DEBUG` level messages and full exception stack traces.
+
+You can configure its rotation in the `[MISC]` section:
+
+- `TRACE_LOG_MAX_BYTES`: Max size of the trace log file in bytes. Default is `10485760` (10 MB).
+- `TRACE_LOG_BACKUP_COUNT`: Number of rotated trace log files to keep. Default is `1`.
+
+Example:
+
+```ini
+[MISC]
+TRACE_LOG_MAX_BYTES = 10485760
+TRACE_LOG_BACKUP_COUNT = 1
+```
+
+##### Command output Capture
+
+- New optional `[MISC]` setting: `COMMAND_CAPTURE_MAX_BYTES` (default 102400).
+  - Limits how much stdout/stderr is kept in memory per command while still logging full output.
+  - Set to `0` to disable buffering entirely. Command output is still streamed to dar-backup-commands.log
+  - If set to `0`, the calling function cannot rely on output from the executed command. The exit value is the only result provided.
+
+Example:
+
+```ini
+[MISC]
+COMMAND_CAPTURE_MAX_BYTES = 102400
+```

{dar_backup-1.0.1 → dar_backup-1.0.2}/README.md

@@ -95,8 +95,8 @@ Version **1.0.0** was reached on October 9, 2025.
     - [Performance tip due to par2](#performance-tip-due-to-par2)
     - [.darrc sets -vd -vf (since v0.6.4)](#darrc-sets--vd--vf-since-v064)
     - [Separate log file for command output](#separate-log-file-for-command-output)
+    - [Trace Logging (Debug details)](#trace-logging-debug-details)
     - [Skipping cache directories](#skipping-cache-directories)
-    - [Progress bar and current directory](#progress-bar-and-current-directory)
     - [Shell autocompletion](#shell-autocompletion)
       - [Use it](#use-it)
       - [Archive name completion (smart, context-aware)](#archive-name-completion-smart-context-aware)
@@ -123,6 +123,9 @@ Version **1.0.0** was reached on October 9, 2025.
         - [DISCORD WEBHOOK](#discord-webhook)
         - [Restore test config](#restore-test-config)
         - [Par2](#par2-1)
+      - [1.0.2](#102)
+        - [Trace Logging](#trace-logging)
+        - [Command output Capture](#command-output-capture)
   
 ## My use case
 
@@ -791,7 +794,6 @@ ERROR_CORRECTION_PERCENT = 5
 ENABLED = True
 # Optional PAR2 configuration
 # PAR2_DIR = /path/to/par2-store
-# PAR2_MODE = per-slice
 # PAR2_RATIO_FULL = 10
 # PAR2_RATIO_DIFF = 5
 # PAR2_RATIO_INCR = 5
@@ -800,7 +802,6 @@ ENABLED = True
 # Optional per-backup overrides (section name = backup definition)
 [media-files]
 PAR2_DIR = /mnt/par2/media-files
-PAR2_MODE = per-archive
 PAR2_RATIO_FULL = 10
 
 # scripts to run before the backup to setup the environment
@@ -818,6 +819,7 @@ PAR2 notes:
 - If `PAR2_DIR` is unset, par2 files are created next to the archive slices (legacy behavior) and no manifest is written
 - When `PAR2_DIR` is set, dar-backup writes a manifest next to the par2 set:
   `archive_base.par2.manifest.ini`
+- When generating a par2 set, par2 reads all archive slices before writing any output files; for large backups, this initial read can take hours
 - Verify or repair using:
   `par2 verify -B <archive_dir> <par2_set.par2>`
   `par2 repair -B <archive_dir> <par2_set.par2>`
@@ -1143,6 +1145,25 @@ This happens when the shell splits the quoted string or interprets globs before
 
 > 💡 **Tip:** See [dar's documentation](http://dar.linux.free.fr/doc/man/dar.html#COMMANDS%20AND%20OPTIONS)
 
+>
+> 💡💡 **Tip:** To filter all the empty directories away that `dar` emits when listing  contents, append this grep:
+>
+> ```bash
+> |grep -vE '\s+d[rwx-]{9}\s'
+>```
+>
+>Example using the grep to discard directory noise from `dar's` output:
+>
+> ```bash
+> dar-backup --list-contents media-files_INCR_2025-05-10 --selection="-I '*Z50*' -X '*.xmp'" | grep -vE '\s+d[rwx-]{9}\s'
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 26  Mio Fri May  9 11:26:16 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0633.NEF
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 26  Mio Fri May  9 11:26:16 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0632.NEF
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 28  Mio Fri May  9 11:09:04 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0631.NEF
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 29  Mio Fri May  9 11:09:03 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0630.NEF
+>[Saved][ ]       [-L-][   0%][X]  -rw-rw-r--   user user 29  Mio Fri May  9 11:09:03 2025 home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling/Z50_0629.NEF
+>...
+>```
+
 ### select a directory
 
 Select files and sub directories in `home/user/data/2025/2025-05-09-Roskilde-Nordisk-udstilling`
@@ -1499,23 +1520,29 @@ In order to not clutter that log file with the output of commands being run, a n
 
 The secondary log file can get quite cluttered, if you want to remove the clutter, run the `clean-log`script with the `--file` option, or simply delete it.
 
-### Skipping cache directories
+### Trace Logging (Debug details)
 
-The author uses the `--cache-directory-tagging` option in his [backup definitions](#backup-definition-example).
+To keep the main log file clean while preserving essential debugging information, `dar-backup` creates a separate trace log file (e.g., `dar-backup.trace.log`) alongside the main log.
 
-The effect is that directories with the [CACHEDIR.TAG](https://bford.info/cachedir/) file are not backed up. Those directories contain content fetched from the net, which is of an ephemeral nature and probably not what you want to back up.
+- **Main Log (`dar-backup.log`)**: Contains clean, human-readable INFO/ERROR messages. Stack traces are suppressed here.
+- **Trace Log (`dar-backup.trace.log`)**: Captures ALL messages at `DEBUG` level, including full exception stack traces. Use this file for debugging crashes or unexpected behavior.
 
-If the option is not in the backup definition, the cache directories are backed up as any other.
+You can configure the rotation of this file in `[MISC]`:
 
-### Progress bar and current directory
+```ini
+[MISC]
+# ... other settings ...
+TRACE_LOG_MAX_BYTES = 10485760  # 10 MB default
+TRACE_LOG_BACKUP_COUNT = 1      # Keep 1 old trace file (default)
+```
 
-If you run dar-backup interactively in a "normal" console on your computer,
-dar-backup displays 2 visual artifacts to show progress.
+### Skipping cache directories
 
-1. a progress bar that fills up and starts over
-2. a status line showing the directory being backed up. If the directory is big and takes time to backup, the line is not changing, but you will probably know there is a lot to backup.
+The author uses the `--cache-directory-tagging` option in his [backup definitions](#backup-definition-example).
 
-The indicators are not shown if dar-backup is run from systemd or if it is used in terminal multiplexers like `tmux` or `screen`. So no polluting of journald logs.
+The effect is that directories with the [CACHEDIR.TAG](https://bford.info/cachedir/) file are not backed up. Those directories contain content fetched from the net, which is of an ephemeral nature and probably not what you want to back up.
+
+If the option is not in the backup definition, the cache directories are backed up as any other.
 
 ### Shell autocompletion
 
@@ -1700,7 +1727,6 @@ pytest                   # run the test suite
 
 - Perhaps look into pre-processing backup definitions. As `dar` does not expand env vars
   `dar-backup` could do so and feed the result to `dar`.
-- When run interactively, a progress bar during test and par2 generation would be nice.
 - Look into a way to move the .par2 files away from the `dar` slices, to maximize chance of good redundancy.
 - Add option to dar-backup to use the `dar` option `--fsa-scope none`
 
@@ -2036,10 +2062,44 @@ PAR2_RUN_VERIFY = true
 #
 #[etc]
 # Keep global PAR2 settings but tweak ratios for this backup definition
-# RATIO is i percent number
+# RATIO is given in percent (%)
 #PAR2_RATIO_FULL = 15  
 #PAR2_RATIO_DIFF = 8
 #PAR2_RATIO_INCR = 8
 ```
 
 [Per-backup override test case: `tests/test_par2_overrides.py`](v2/tests/test_par2_overrides.py)
+
+#### 1.0.2
+
+##### Trace Logging
+
+To support debugging without cluttering the main log file, a secondary trace log is now created (e.g., `dar-backup.trace.log`).
+This file captures all `DEBUG` level messages and full exception stack traces.
+
+You can configure its rotation in the `[MISC]` section:
+
+- `TRACE_LOG_MAX_BYTES`: Max size of the trace log file in bytes. Default is `10485760` (10 MB).
+- `TRACE_LOG_BACKUP_COUNT`: Number of rotated trace log files to keep. Default is `1`.
+
+Example:
+
+```ini
+[MISC]
+TRACE_LOG_MAX_BYTES = 10485760
+TRACE_LOG_BACKUP_COUNT = 1
+```
+
+##### Command output Capture
+
+- New optional `[MISC]` setting: `COMMAND_CAPTURE_MAX_BYTES` (default 102400).
+  - Limits how much stdout/stderr is kept in memory per command while still logging full output.
+  - Set to `0` to disable buffering entirely. Command output is still streamed to dar-backup-commands.log
+  - If set to `0`, the calling function cannot rely on output from the executed command. The exit value is the only result provided.
+
+Example:
+
+```ini
+[MISC]
+COMMAND_CAPTURE_MAX_BYTES = 102400
+```

{dar_backup-1.0.1 → dar_backup-1.0.2}/doc/Configfile.svg

@@ -370,7 +370,6 @@
      x="975"
      y="618"
      id="text48">If section [name] is absent → uses global [PAR2] values</text>
-  <!-- Bottom right: PAR2_LAYOUT effects (merged) -->
   <!-- Arrows -->
   <path
      class="arrow"

dar_backup-1.0.2/doc/Gemini3-assessment.md

@@ -0,0 +1,46 @@
+# Assessment of dar-backup (v2)
+
+## Gemini3-preview assessment
+
+Date: January 23, 2026
+
+### 1. Code Quality
+
+**Rating: Excellent**
+
+The codebase demonstrates professional engineering standards and best practices for a Python CLI tool.
+
+*   **Structure & Organization:** The project follows a clear modular design.
+    *   `dar_backup.py` acts as the orchestrator.
+    *   `util.py` handles shared logic (logging, config parsing).
+    *   `command_runner.py` encapsulates system command execution.
+*   **Type Hinting:** Extensive use of Python type hints (e.g., `-> List[str]`, `config_settings: ConfigSettings`) ensures code clarity and enables static analysis.
+*   **Documentation:** Functions feature comprehensive docstrings detailing arguments, return values, and exceptions.
+*   **Error Handling:** Robust exception handling is in place. Custom exceptions like `BackupError` are used, and `subprocess` errors are caught and logged gracefully.
+*   **Logging:** A sophisticated dual-logger setup (`main_logger` and `secondary_logger`) separates application logic from the verbose output of the underlying `dar` commands.
+
+### 2. Test Coverage
+**Rating: Excellent**
+
+The testing strategy is rigorous, verified by the reported **89% code coverage**.
+
+*   **Framework:** Uses `pytest`, the industry standard for Python testing.
+*   **Mocking:** The tests in `v2/tests/` (e.g., `test_dar_backup.py`) make extensive use of `unittest.mock`.
+    *   **Isolation:** File system operations and external shell commands (like `dar` and `par2`) are mocked effectively. This ensures tests are fast, deterministic, and safe to run without modifying the user's system.
+*   **Scenarios:** The test suite covers:
+    *   **Success paths:** Verifying correct command construction.
+    *   **Failure paths:** Handling non-zero exit codes from subprocesses.
+    *   **Edge cases:** Missing files, permission errors, and configuration mismatches.
+
+### 3. README.md
+**Rating: Excellent**
+
+The documentation is exhaustive and user-friendly.
+
+*   **Completeness:** It covers every aspect of the tool's lifecycle: Installation (pip/venv), Configuration (detailed `ini` examples), Usage, and Automation (systemd generation).
+*   **Context:** The "My Use Case" and "Design Choices" sections provide valuable context, explaining *why* the tool exists and the problems it solves (e.g., bitrot protection with PAR2).
+*   **Quick Start:** A "TL;DR" and "Quick Guide" allow users to get up and running immediately.
+*   **Transparency:** Includes live badges for build status, PyPI version, and test coverage, increasing trust.
+
+### Summary
+`dar-backup` v2 is a high-quality, production-ready tool. It is well-architected, thoroughly tested, and exceptionally documented, making it a reliable choice for critical backup operations.

{dar_backup-1.0.1 → dar_backup-1.0.2}/doc/dev.md

@@ -37,6 +37,13 @@ python3 -m build && pip install --force-reinstall dist/dar_backup-${VERSION}-py3
 
 ## Howto use pytest in venv
 
+The simplest way to run the test suite
+
+```` bash
+# assumes the venc is activated, see above
+pytest
+````
+
 A pytest.ini is located in the v2 directory, so that pytest writes out captures to  console.
 
 That is useful when working with a single test and is the default

{dar_backup-1.0.1 → dar_backup-1.0.2}/pyproject.toml

@@ -66,11 +66,11 @@ description = """A script to do full, differential and incremental backups using
 readme = "README.md"
 readme-content-type = "text/markdown"
 license = { file="LICENSE", name="GPL-3.0-or-later" }
-requires-python = ">=3.9"
+requires-python = ">=3.11"
 classifiers = [
     "Development Status :: 5 - Production/Stable",
     "Intended Audience :: End Users/Desktop",
-    'Programming Language :: Python :: 3.9',
+    'Programming Language :: Python :: 3.11',
     "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
     "Operating System :: POSIX :: Linux",
     "Topic :: System :: Archiving :: Backup",
@@ -110,6 +110,9 @@ packaging = [
  "twine>=6.1.0",
 ]
 
+[tool.coverage.run]
+data_file = "/tmp/.coverage"
+
 [project.urls]
 "GPG Public Key" = "https://keys.openpgp.org/search?q=dar-backup@pm.me"
 Homepage = "https://github.com/per2jensen/dar-backup/tree/main/v2"

{dar_backup-1.0.1 → dar_backup-1.0.2/src/dar_backup}/Changelog.md

@@ -1,6 +1,35 @@
 <!-- markdownlint-disable MD024 -->
 # dar-backup Changelog
 
+## v2-1.0.2 - 2026-01-25
+
+### Added
+
+- Streaming restore-test sampling using reservoir sampling to avoid holding full file lists in memory.
+- Configurable command output capture cap (`COMMAND_CAPTURE_MAX_BYTES`, default 100 KB) to limit in-memory stdout/stderr while still logging full output.
+- Streaming list output for `dar-backup --list-contents` and `manager --list-archive-contents` to avoid large in-memory buffers.
+- Test coverage additions for config parsing, util helpers, restore-test sampling edge cases, par2 slice helpers, and get_backed_up_files error paths.
+- Tests for missing source files during restore verification and for par2 generation order after verification.
+- CommandRunner test coverage for sanitize failure notes, text/binary output handling, timeouts, Popen failures, and TTY restore logic.
+- Tests for COMMAND_CAPTURE_MAX_BYTES defaults (0 and 1k) and binary stdout/stderr capture with truncation and log_output disabled.
+- Manager test coverage for create-db guardrails and catalog listing parsing/sorting across runner/subprocess paths.
+- Cleanup now reports PREREQ/POSTREQ failures cleanly and sends Discord failure notifications when configured.
+- New trace logger that always logs at DEBUG and captures stacktraces if they happens. Default max size is 10MB + 1 rollover file.
+
+### Changed
+
+- BUGFIX: Ensure existing files are removed before restore verification to prevent false positives.
+- Clears out restore-test directory on program start to ensure a clean slate.
+- Restore-test selection now streams DAR XML listings and samples candidates without loading all entries into RAM.
+- `get_backed_up_files` uses incremental XML parsing to reduce memory use for large archives.
+- Restore verification now logs a warning and continues when a source or restored file is missing during comparison.
+- CommandRunner supports per-command capture cap overrides (disable cap with `capture_output_limit_bytes=-1`).
+- Cleanup now rejects unsafe archive names when `--cleanup-specific-archives` is used to prevent accidental deletions.
+- Removed deprecated PAR2 layout/mode settings and simplified PAR2 cleanup to delete all matching .par2 artifacts.
+- Config templates/docs updated to drop PAR2_LAYOUT/PAR2_MODE references.
+- [Snyk] Python 3.11 required in pyproject.toml. Snyk has flagged a vulnerabilly in an xml parser that requires the bump to 3.11.
+- Config parsing errors now emit concise messages (no stack trace) and trigger Discord failure notifications in CLI tools.
+
 ## v2-1.0.1 - 2026-01-09
 
 ### Added