duplicity 3.2.2__tar.gz → 4.0.0.dev6__tar.gz

{duplicity-3.2.2 → duplicity-4.0.0.dev6}/CHANGELOG.md

@@ -1,8 +1,138 @@
 
-(Unreleased) / 2026-10-14
+(Unreleased) / 2026-11-10
 =========================
 
+ * [`6560ed00`](https://gitlab.com/duplicity/duplicity/-/commit/6560ed00):fix: fix verify and compare notice messages in restore.py
 
+rel.4.0.0.dev5 / 2026-11-09
+===========================
+
+ * [`e40e119b`](https://gitlab.com/duplicity/duplicity/-/commit/e40e119b):new: add colorized help output and convert boolean CLI flags
+
+rel.4.0.0.dev4 / 2026-11-09
+===========================
+
+ * [`6b72705e`](https://gitlab.com/duplicity/duplicity/-/commit/6b72705e):chg: rename compare alias and fix notice message, enable pylint checks
+ * [`28817b3e`](https://gitlab.com/duplicity/duplicity/-/commit/28817b3e):chg: remove unused imports, dead classes, and unreferenced functions
+
+rel.4.0.0.dev3 / 2026-11-08
+===========================
+
+ * [`68a98c71`](https://gitlab.com/duplicity/duplicity/-/commit/68a98c71):new: add --gpghome CLI option and improve GnuPG home processing
+ * [`b51fea66`](https://gitlab.com/duplicity/duplicity/-/commit/b51fea66):new: compress local cache metadata files and improve sync/collection handling
+ * [`03c26356`](https://gitlab.com/duplicity/duplicity/-/commit/03c26356):new: add 'compare' action to compare backed-up files with current local copies
+ * [`196924fb`](https://gitlab.com/duplicity/duplicity/-/commit/196924fb):fix: swap verify and compare operational scope
+ * [`493ca237`](https://gitlab.com/duplicity/duplicity/-/commit/493ca237):fix: remove force_print from startup parameters log message
+ * [`72eb6f14`](https://gitlab.com/duplicity/duplicity/-/commit/72eb6f14):fix: refine backend pool shutdown, gpgme file closing, and test GNUPGHOME setup
+ * [`4fa4837b`](https://gitlab.com/duplicity/duplicity/-/commit/4fa4837b):fix: check whether encryption/sign keys require passwords and simplify passphrase handling
+ * [`c66d71d5`](https://gitlab.com/duplicity/duplicity/-/commit/c66d71d5):fix: Resolve backend pool cleanup race condition
+ * [`f0182755`](https://gitlab.com/duplicity/duplicity/-/commit/f0182755):fix: always log startup parameters.
+ * [`03bb64d5`](https://gitlab.com/duplicity/duplicity/-/commit/03bb64d5):fix: accurately report ElapsedTime across full backup duration
+ * [`42015644`](https://gitlab.com/duplicity/duplicity/-/commit/42015644):fix: ensure backend pool worker processes terminate on shutdown and exit.
+ * [`3e6c3943`](https://gitlab.com/duplicity/duplicity/-/commit/3e6c3943):fix: add 'make wheels' capability
+ * [`735cfca9`](https://gitlab.com/duplicity/duplicity/-/commit/735cfca9):fix: some fixes to tools/makechangelog
+ * [`036715dd`](https://gitlab.com/duplicity/duplicity/-/commit/036715dd):fix: add repository root to sys.path in src/__main__.py for direct execution
+ * [`16a17a00`](https://gitlab.com/duplicity/duplicity/-/commit/16a17a00):fix: handle read-only filename attributes on signature file objects
+ * [`b0345f45`](https://gitlab.com/duplicity/duplicity/-/commit/b0345f45):fix: handle multi-volume signature parts in get_combined_path_iter
+ * [`635aa869`](https://gitlab.com/duplicity/duplicity/-/commit/635aa869):fix: use tempdir path directly in check_resources and add unit tests
+
+rel.4.0.0.dev2 / 2026-11-02
+===========================
+
+ * [`d3506eda`](https://gitlab.com/duplicity/duplicity/-/commit/d3506eda):new: implement concurrent volume prefetcher for restore and verify
+ * [`679831f7`](https://gitlab.com/duplicity/duplicity/-/commit/679831f7):new: Add a comprehensive benchmark tool for 3.x to 4.x comparison
+ * [`ddf54b4c`](https://gitlab.com/duplicity/duplicity/-/commit/ddf54b4c):chg: isolate backend CLI options, URL formats, and config
+ * [`1512150c`](https://gitlab.com/duplicity/duplicity/-/commit/1512150c):chg: simplify CLI argument definitions with direct add_argument calls
+ * [`cabdc68b`](https://gitlab.com/duplicity/duplicity/-/commit/cabdc68b):chg: Run test-template on push events only
+ * [`c5c4f555`](https://gitlab.com/duplicity/duplicity/-/commit/c5c4f555):chg: Build wheels with apt's python3-gpg, not PyPI's gpg.
+ * [`9a2d27e4`](https://gitlab.com/duplicity/duplicity/-/commit/9a2d27e4):chg: Port issue #926 zlib-ng compression acceleration to duplicity4
+ * [`4ac3086d`](https://gitlab.com/duplicity/duplicity/-/commit/4ac3086d):chg: Remove redundant log message after backup completion in `dup_main.py` to streamline output.
+ * [`a6704d39`](https://gitlab.com/duplicity/duplicity/-/commit/a6704d39):chg: Add explicit logging error codes to improve error tracing; update tests and dependencies to align with changes
+ * [`04cb9144`](https://gitlab.com/duplicity/duplicity/-/commit/04cb9144):chg: Add GPGME detection and enhanced GPG recompilation logic; refactor setup to improve maintainability and error handling.
+ * [`ccb50af3`](https://gitlab.com/duplicity/duplicity/-/commit/ccb50af3):chg: Add sparse file support for efficient backup and restore with --sparse and --no-sparse options, functional tests, and documentation updates.
+ * [`4837111c`](https://gitlab.com/duplicity/duplicity/-/commit/4837111c):chg: Refactor backend file handling to use context managers, improve error management, and add unit tests for code review fixes.
+ * [`4887508b`](https://gitlab.com/duplicity/duplicity/-/commit/4887508b):chg: Port latest fixes and pylint check enhancements from dev to duplicity4
+ * [`2686dcb6`](https://gitlab.com/duplicity/duplicity/-/commit/2686dcb6):chg: Port backend missing file handling and backup set deletion improvements from dev
+ * [`40851660`](https://gitlab.com/duplicity/duplicity/-/commit/40851660):chg: Add support for preserving and restoring hard links during backup operations, including --hardlinks and --no-hardlinks options, tests, and documentation updates.
+ * [`a16fed38`](https://gitlab.com/duplicity/duplicity/-/commit/a16fed38):chg: Update project name in Sphinx configuration to "duplicity4"
+ * [`e9908e97`](https://gitlab.com/duplicity/duplicity/-/commit/e9908e97):chg: Enable real-time progress reporting during concurrent backend operations using shared memory counter
+ * [`06c38f07`](https://gitlab.com/duplicity/duplicity/-/commit/06c38f07):chg: Centralize progress reporting logic and simplify backend-specific implementations
+ * [`587eb3a8`](https://gitlab.com/duplicity/duplicity/-/commit/587eb3a8):chg: Update macOS deployment target and improve GitLab CI rules for wheel builds
+ * [`5c368d26`](https://gitlab.com/duplicity/duplicity/-/commit/5c368d26):chg: Enhance help footer to clarify usage of main and command-specific help
+ * [`5f6fd4cb`](https://gitlab.com/duplicity/duplicity/-/commit/5f6fd4cb):chg: Remove obsolete --help-<command> options in favor of 'duplicity command -h' help
+ * [`62fe605b`](https://gitlab.com/duplicity/duplicity/-/commit/62fe605b):chg: Restrict help output to options valid for each command
+ * [`3912fd11`](https://gitlab.com/duplicity/duplicity/-/commit/3912fd11):chg: Move `pytest` and `gpgsm-locking` plans to `plans_done` folder
+ * [`76fb450c`](https://gitlab.com/duplicity/duplicity/-/commit/76fb450c):chg: Update codebase docstrings and sphinx configuration for 100% doc coverage
+ * [`60bfeb34`](https://gitlab.com/duplicity/duplicity/-/commit/60bfeb34):chg: remove optimization check and replace asserts with explicit error handling
+ * [`8e5018fe`](https://gitlab.com/duplicity/duplicity/-/commit/8e5018fe):chg: Break out restore, cleanup, restart, and sync logic from dup_main.py into dedicated modules
+ * [`58f0840b`](https://gitlab.com/duplicity/duplicity/-/commit/58f0840b):chg: Break out full and inc backup logic from dup_main.py into backup.py
+ * [`95ff6ab2`](https://gitlab.com/duplicity/duplicity/-/commit/95ff6ab2):chg: Add `_FDWrapper` for safer file descriptor handling in GPG operations
+ * [`e60db0e3`](https://gitlab.com/duplicity/duplicity/-/commit/e60db0e3):chg: Improve Par2 backend robustness and introduce enhanced testing
+ * [`c824ba98`](https://gitlab.com/duplicity/duplicity/-/commit/c824ba98):fix: accelerate backup/restore pipeline and enforce full backup on legacy repos
+ * [`84d9df65`](https://gitlab.com/duplicity/duplicity/-/commit/84d9df65):fix: Avoid mutating global tempfile.tempdir to prevent AF_UNIX socket length errors
+ * [`88617c9b`](https://gitlab.com/duplicity/duplicity/-/commit/88617c9b):fix: Speed up frontend tarfile build
+ * [`71c7e0c2`](https://gitlab.com/duplicity/duplicity/-/commit/71c7e0c2):fix: Speed up by moving encryption and compression into the backend pool tasks
+ * [`c08d8906`](https://gitlab.com/duplicity/duplicity/-/commit/c08d8906):fix: Default to 3.2.2 for now.
+ * [`76b30bf2`](https://gitlab.com/duplicity/duplicity/-/commit/76b30bf2):fix: Fix perms on tools/benchmark*.py
+ * [`af0c0f24`](https://gitlab.com/duplicity/duplicity/-/commit/af0c0f24):fix: Dynamic MACOSX_DEPLOYMENT_TARGET for cibuildwheel macOS builds
+ * [`3ef55348`](https://gitlab.com/duplicity/duplicity/-/commit/3ef55348):fix: Fix CI configuration and cibuildwheel setup for wheels-* jobs
+ * [`588bad93`](https://gitlab.com/duplicity/duplicity/-/commit/588bad93):fix: Add `include_deferred` option to `filter_restore_path_iter` for improved restore and verify behavior
+ * [`8f9d00d0`](https://gitlab.com/duplicity/duplicity/-/commit/8f9d00d0):fix: Deprecate `--path-to-restore`; add warnings and alternatives for restore operations
+ * [`00b47437`](https://gitlab.com/duplicity/duplicity/-/commit/00b47437):fix: Add robust hardlink handling and data verification logic; implement comprehensive unit and functional tests.
+ * [`f8443a22`](https://gitlab.com/duplicity/duplicity/-/commit/f8443a22):fix: Add detailed tracking for `NewFileSize` in sparse and hardlinked file stats; enhance unit and functional tests for verification
+ * [`380eab54`](https://gitlab.com/duplicity/duplicity/-/commit/380eab54):fix: Add statistics for sparse and hardlinked files in dry-run and live backups; implement unit and functional tests for consistency verification.
+ * [`879ae803`](https://gitlab.com/duplicity/duplicity/-/commit/879ae803):fix: Add support for `--path-to-restore` in `verify` action; enhance selective file and directory verification handling.
+ * [`eb691a53`](https://gitlab.com/duplicity/duplicity/-/commit/eb691a53):fix: Prevent buffered stream desync during sparse reads on Python 3.14+
+ * [`1c9901a3`](https://gitlab.com/duplicity/duplicity/-/commit/1c9901a3):fix: Optimize restore and deep verify for efficiency; implement sequential restore, signature-based verification, and stream-based delta patching with bounded resource usage.
+ * [`56ee5f79`](https://gitlab.com/duplicity/duplicity/-/commit/56ee5f79):fix: Move completed plan files to correct directory structure (`plans/done`); add `fix-verify-excessive-disk-space.md`
+ * [`7539adae`](https://gitlab.com/duplicity/duplicity/-/commit/7539adae):fix: Improve disk space efficiency in `verify`; add sequential volume verification and explicit resource cleanup
+
+rel.4.0.0.dev1 / 2026-10-05
+===========================
+
+ * [`24d9b668`](https://gitlab.com/duplicity/duplicity/-/commit/24d9b668):new: Extend Par2 backend with transactional support and aggregate recovery file handling
+ * [`c652bbaf`](https://gitlab.com/duplicity/duplicity/-/commit/c652bbaf):chg: Add logging configuration options and improve logging integration
+ * [`77cdf36f`](https://gitlab.com/duplicity/duplicity/-/commit/77cdf36f):chg: Remove `_legacy_transfer` from Par2 backend and simplify upload/move logic
+ * [`fc0f36e0`](https://gitlab.com/duplicity/duplicity/-/commit/fc0f36e0):chg: Add `put_validated` method with retry logic and refactor upload validation
+ * [`a3ccdbf9`](https://gitlab.com/duplicity/duplicity/-/commit/a3ccdbf9):chg: Remove `get_data_files` from `setup.py` and simplify extension build logic
+ * [`06098dd1`](https://gitlab.com/duplicity/duplicity/-/commit/06098dd1):chg: Remove `src/lazy.py` and associated tests to simplify the codebase
+ * [`28f252e8`](https://gitlab.com/duplicity/duplicity/-/commit/28f252e8):chg: Add Python 3.15 support across the project
+ * [`ad59a65d`](https://gitlab.com/duplicity/duplicity/-/commit/ad59a65d):chg: Clarify `flynt` usage for Python 3.10 compatibility and adjust string concatenation in `manifest.py`
+ * [`4d596288`](https://gitlab.com/duplicity/duplicity/-/commit/4d596288):chg: Refactor manifest handling to adopt JSON Lines format for improved structure and extensibility
+ * [`12309c12`](https://gitlab.com/duplicity/duplicity/-/commit/12309c12):chg: Update `CHANGELOG.md` with new release entries and modify `makechangelog` to ignore its own changes.
+ * [`041c451a`](https://gitlab.com/duplicity/duplicity/-/commit/041c451a):chg: Simplify Makefile test command and update coverage settings
+
+rel.4.0.0.dev0 / 2026-09-23
+===========================
+
+ * [`57eb00be`](https://gitlab.com/duplicity/duplicity/-/commit/57eb00be):chg: Add support for split signature and manifest volumes with atomic completion records
+ * [`e1df4b06`](https://gitlab.com/duplicity/duplicity/-/commit/e1df4b06):chg: Move remaining plans to 'plans_done' and resolve outdated references
+ * [`0466fea6`](https://gitlab.com/duplicity/duplicity/-/commit/0466fea6):chg: Remove legacy Pyrax and CloudFiles backends
+ * [`ded09476`](https://gitlab.com/duplicity/duplicity/-/commit/ded09476):chg: Restored "NOTICE" log level and updated verbosity handling
+ * [`3b327d9f`](https://gitlab.com/duplicity/duplicity/-/commit/3b327d9f):chg: Improve string formatting and update pylint job configuration
+ * [`d71e71a6`](https://gitlab.com/duplicity/duplicity/-/commit/d71e71a6):chg: Add `swig` to CI dependencies in `.gitlab-ci.yml`
+ * [`77fa2670`](https://gitlab.com/duplicity/duplicity/-/commit/77fa2670):chg: Remove legacy `gpginterface.py` and associated tests
+ * [`9cf4557a`](https://gitlab.com/duplicity/duplicity/-/commit/9cf4557a):chg: Remove redundant parentheses from class declarations
+ * [`5c7b8ad0`](https://gitlab.com/duplicity/duplicity/-/commit/5c7b8ad0):chg: Remove redundant `object` inheritance across classes for Python 3
+ * [`30264c15`](https://gitlab.com/duplicity/duplicity/-/commit/30264c15):chg: Standardize logging levels and refactor verbosity handling
+ * [`e28a3c87`](https://gitlab.com/duplicity/duplicity/-/commit/e28a3c87):chg: Replace `duplicity` references with `src` across codebase
+
+rel.3.2.3.dev2 / 2026-11-05
+===========================
+
+ * [`4cf4e13c`](https://gitlab.com/duplicity/duplicity/-/commit/4cf4e13c):chg: Move wheels build from CI to local.
+ * [`87cc7510`](https://gitlab.com/duplicity/duplicity/-/commit/87cc7510):fix: Fix check resources() in dup_main.py
+ * [`28f1a91b`](https://gitlab.com/duplicity/duplicity/-/commit/28f1a91b):fix: giobackend retries do not attempt to remount remote_file
+
+rel.3.2.3.dev1 / 2026-10-21
+===========================
+
+ * [`2617372a`](https://gitlab.com/duplicity/duplicity/-/commit/2617372a):chg: Add archive verification and improve file handling
+
+rel.3.2.3.dev0 / 2026-10-14
+===========================
+
+ * [`042cd4a9`](https://gitlab.com/duplicity/duplicity/-/commit/042cd4a9):chg: Set version to 3.2.3.dev0 for LP.
 
 rel.3.2.2 / 2026-10-14
 ======================
@@ -122,21 +252,6 @@ rel.3.2.0 / 2026-08-10
  * [`c823ed36`](https://gitlab.com/duplicity/duplicity/-/commit/c823ed36):fix: Could not release lockfile
  * [`3d9eab94`](https://gitlab.com/duplicity/duplicity/-/commit/3d9eab94):fix: Comprehensive fix for backup, restart, and cleanup logic.
 
-rel.4.0.0.dev0 / 2026-09-23
-===========================
-
- * [`8427bb00`](https://gitlab.com/duplicity/duplicity/-/commit/8427bb00):chg: Add support for split signature and manifest volumes with atomic completion records
- * [`2b0c76e7`](https://gitlab.com/duplicity/duplicity/-/commit/2b0c76e7):chg: Move remaining plans to 'plans_done' and resolve outdated references
- * [`1d07db2f`](https://gitlab.com/duplicity/duplicity/-/commit/1d07db2f):chg: Remove legacy Pyrax and CloudFiles backends
- * [`b04ca97b`](https://gitlab.com/duplicity/duplicity/-/commit/b04ca97b):chg: Restored "NOTICE" log level and updated verbosity handling
- * [`f9b6ed8d`](https://gitlab.com/duplicity/duplicity/-/commit/f9b6ed8d):chg: Improve string formatting and update pylint job configuration
- * [`5437ccf2`](https://gitlab.com/duplicity/duplicity/-/commit/5437ccf2):chg: Add `swig` to CI dependencies in `.gitlab-ci.yml`
- * [`50e0e15f`](https://gitlab.com/duplicity/duplicity/-/commit/50e0e15f):chg: Remove legacy `gpginterface.py` and associated tests
- * [`c5e53820`](https://gitlab.com/duplicity/duplicity/-/commit/c5e53820):chg: Remove redundant parentheses from class declarations
- * [`f2fa175e`](https://gitlab.com/duplicity/duplicity/-/commit/f2fa175e):chg: Remove redundant `object` inheritance across classes for Python 3
- * [`74f7d3a9`](https://gitlab.com/duplicity/duplicity/-/commit/74f7d3a9):chg: Standardize logging levels and refactor verbosity handling
- * [`d090ab43`](https://gitlab.com/duplicity/duplicity/-/commit/d090ab43):chg: Replace `duplicity` references with `src` across codebase
-
 rel.3.2.1.dev3 / 2026-09-23
 ===========================
 

duplicity-4.0.0.dev6/PKG-INFO

@@ -0,0 +1,140 @@
+Metadata-Version: 2.4
+Name: duplicity
+Version: 4.0.0.dev6
+Summary: Encrypted backup using rsync algorithm
+Author-email: Kenneth Loafman <kenneth@loafman.com>
+Maintainer: Edgar Soldin, Thomas Laubrock
+Project-URL: Homepage, http://duplicity.us
+Project-URL: Download, https://gitlab.com/AlphaPrime/duplicity4/-/releases
+Project-URL: Issues, https://gitlab.com/AlphaPrime/duplicity4/-/issues
+Project-URL: Contact, https://lists.nongnu.org/mailman/listinfo/duplicity-talk
+Project-URL: Documentation, http://duplicity.us
+Project-URL: Repository, https://gitlab.com/AlphaPrime/duplicity4.git
+Project-URL: Changelog, https://gitlab.com/AlphaPrime/duplicity4/-/blob/duplicity4/CHANGELOG.md
+Classifier: Development Status :: 6 - Mature
+Classifier: Environment :: Console
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: POSIX
+Classifier: Programming Language :: C
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Classifier: Topic :: System :: Archiving :: Backup
+Requires-Python: <3.16,>=3.10
+Description-Content-Type: text/markdown
+License-File: COPYING
+License-File: AUTHORS.md
+Requires-Dist: fasteners>=0.19
+Requires-Dist: httplib2
+Requires-Dist: pysocks
+Requires-Dist: requests
+Requires-Dist: requests-oauthlib
+Requires-Dist: zlib_ng>=1.0.0
+Requires-Dist: azure-storage-blob
+Requires-Dist: b2sdk
+Requires-Dist: boto3
+Requires-Dist: botocore
+Requires-Dist: boxsdk[jwt]
+Requires-Dist: dropbox
+Requires-Dist: gdata-python3
+Requires-Dist: google-api-python-client
+Requires-Dist: google-auth-oauthlib
+Requires-Dist: jottalib
+Requires-Dist: mediafire
+Requires-Dist: megatools
+Requires-Dist: paramiko
+Requires-Dist: pexpect
+Requires-Dist: pydrive2
+Requires-Dist: python-swiftclient
+Dynamic: license-file
+
+# Duplicity 4
+
+Duplicity 4 (`duplicity4`) backs directories by producing encrypted, tar-format volumes and uploading them to a remote or local storage backend. By using librsync and the rsync algorithm, Duplicity generates bandwidth- and space-efficient incremental archives that only record binary deltas of changed files. Archives are encrypted and signed using GnuPG (via GPGME) to ensure privacy, integrity, and authenticity.
+
+## Key Features
+
+- **Concurrent Backend Worker Pool**: Multi-worker asynchronous volume compression, encryption, and upload pipeline (`src/backend_pool.py`) for significantly faster backups.
+- **Concurrent Restore Prefetching**: Asynchronous volume prefetching for restore and verification operations.
+- **Hardware-Accelerated Compression**: High-throughput compression using `zlib-ng`.
+- **Advanced File Handling**: Native sparse file detection and efficient chunking (`--sparse`) alongside preserved hardlink reconstruction (`--hardlinks`).
+- **Encrypted & Signed Backups**: Uses GnuPG via GPGME to encrypt and sign archives with standard OpenPGP or CMS (X.509) keys.
+- **Bandwidth & Space Efficient**: Employs librsync to compute binary deltas for changed files in incremental backups.
+- **Standard Archive Format**: Volumes are standard POSIX `tar` format archives encrypted with OpenPGP/CMS.
+- **Extensive Storage Backend Support**: Backups can be stored locally or uploaded to cloud and remote services (SSH/SFTP, rsync, Amazon S3, Backblaze B2, Google Cloud Storage, Microsoft Azure Blob, Google Drive, Microsoft OneDrive, Dropbox, WebDAV, Swift, Mega, and more).
+- **Restartable Transfers**: Asynchronous volume generation and transfer resume support for reliable operation over interrupted connections.
+- **Standardized Logging**: Python standard logging integration with named severity levels (`error`, `warning`, `notice`, `info`, `debug`, `critical`) and machine-readable output options.
+
+## Quick Start Examples
+
+### 1. Perform a Backup
+```shell
+# Backup local directory to a remote SFTP destination
+duplicity /home/user/data pex+sftp://user@example.com/backup
+
+# Force a full backup
+duplicity full /home/user/data pex+sftp://user@example.com/backup
+```
+
+### 2. Restore Files
+```shell
+# Restore the entire latest backup to a destination folder
+duplicity restore pex+sftp://user@example.com/backup /home/user/restore
+
+# Restore a specific file from 3 days ago
+duplicity restore --file-to-restore document.txt --time 3D pex+sftp://user@example.com/backup /home/user/document.txt
+```
+
+### 3. Verify and Compare Backup Integrity
+```shell
+# Compare remote backup contents against the local filesystem
+duplicity compare pex+sftp://user@example.com/backup /home/user/data
+
+# Verify remote backup volume checksums and integrity
+duplicity verify pex+sftp://user@example.com/backup /home/user/data
+```
+
+### 4. Check Backup Status and List Files
+```shell
+# Show collection status (backup sets and volume chains)
+duplicity collection-status pex+sftp://user@example.com/backup
+
+# List all files currently contained in the backup archive
+duplicity list-current-files pex+sftp://user@example.com/backup
+```
+
+## System Requirements
+
+- **Python**: Python >= 3.10 (tested through Python 3.15).
+- **C Libraries**: `librsync` (>= 0.9.7, 2.x recommended), `libgpgme` (GPGME C library), `zlib-ng`.
+- **GnuPG**: `gpg` (2.x), `gpgsm`, and `python3-gpg` (GPGME Python bindings).
+- **Build Tools**: C compiler (`gcc` or `clang`), `make`, `python3-dev`, `swig`, `pkg-config`.
+
+## Documentation & Guides
+
+Comprehensive guides for users, packagers, and developers are organized in the `README/` directory and documentation:
+
+- [Installation Guide](README/README-INSTALL.md) — Comprehensive installation instructions (pipx, apt, pip venv, source).
+- [Architecture & Design](README/README-ARCH.md) — Internal design, volume formats, signature chains, and encryption workflow.
+- [Repository & Development Guide](README/README-REPO.md) — Source tree setup, building C extensions, and running from source.
+- [Testing Guide](README/README-TESTING.md) — Running tests with Docker, pytest, parallel workers, and test isolation guidelines.
+- [Call for Testing](README/README-TESTING-CALL.md) — Call for testing Duplicity 4.0 pre-releases, focus areas, and feedback channels.
+- [Logging Subsystem & Protocol](README/README-LOG.md) — Named log levels, command-line flags, and machine-readable output protocol.
+- [Release Process](README/README-RELEASE.md) — Release checklists, versioning guidelines, Launchpad PPAs, snaps, and PyPI packaging.
+- [Snap Packaging](README/README-SNAP.md) — Building and testing multi-architecture Snap packages.
+- [Sphinx Documentation Guide](docs/README.md) — Building HTML documentation locally and ReadTheDocs configuration.
+- [Backend Development Guide](src/backends/README.md) — Guide to implementing and testing new storage backends.
+- [Test Keyring Reference](testing/gnupg/README.md) — Test GnuPG keys, fingerprints, and passphrases.
+
+## Help & Community
+
+- **Official Website**: <https://duplicity.us>
+- **Project Repository**: <https://gitlab.com/AlphaPrime/duplicity4.git>
+- **Issue Tracker**: <https://gitlab.com/AlphaPrime/duplicity4/-/issues>
+- **Online Documentation**: <https://duplicity4.readthedocs.io/>
+- **Mailing List**: <https://lists.nongnu.org/mailman/listinfo/duplicity-talk>
+- **Translation Portal**: <https://crowdin.com/project/duplicity>

duplicity-4.0.0.dev6/README/README-ARCH.md

@@ -0,0 +1,134 @@
+# Duplicity 4 Architecture and Design
+
+This document details the architectural layout, core subsystems, data formats, and execution flows of Duplicity 4 (`duplicity4`).
+
+## System Component Diagram
+
+```mermaid
+graph TD
+    subgraph Interface
+        Main["__main__.py (Entry Point)"]
+    end
+
+    subgraph Orchestration
+        DupMain["dup_main.py (Orchestration)"]
+    end
+
+    subgraph Support_Config ["Support & Config"]
+        CLI["cli_main.py (Argument Parsing)"]
+        Config["config.py (Global Settings)"]
+        LogUtil["log.py / util.py (Logging/Support)"]
+    end
+
+    subgraph Core_Logic ["Core Logic"]
+        Selection["selection.py (File Selection)"]
+        Backend["backend.py (Storage Layer)"]
+        Errors["errors.py (Exception Mgmt)"]
+    end
+
+    subgraph Operation
+        DiffDir["diffdir.py (Backup Logic)"]
+        Backends["backends/ (Storage Drivers)"]
+        PatchDir["patchdir.py (Restore Logic)"]
+    end
+
+    subgraph Engines_Metadata ["Engines & Metadata"]
+        LibRsync["librsync.py (rsync delta)"]
+        GPGME["gpgme/ (GPGME Crypto Engine)"]
+        ZlibNG["zlib-ng (Compression Engine)"]
+        Manifest["manifest.py (Metadata)"]
+    end
+
+    Main --> DupMain
+    DupMain --> CLI
+    DupMain --> Config
+    DupMain --> LogUtil
+
+    CLI --> Selection
+    Config --> Backend
+    LogUtil ~~~ Errors
+    Errors -.-> LogUtil
+
+    Selection --> DiffDir
+    Backend --> Backends
+
+    DiffDir --> Backends
+    PatchDir --> Backends
+
+    DiffDir --> LibRsync
+    DiffDir --> GPGME
+    DiffDir --> ZlibNG
+    DiffDir --> Manifest
+
+    PatchDir --> LibRsync
+    PatchDir --> GPGME
+    PatchDir --> ZlibNG
+    PatchDir --> Manifest
+
+    Backends --> GPGME
+    Backends --> ZlibNG
+```
+
+## Architectural Overview
+
+Duplicity operates on a pipelined, streaming architecture that turns a filesystem directory into encrypted, compressed, multi-volume remote backup sets.
+
+### 1. Interface & Orchestration Layer
+- **`src/__main__.py`**: Top-level entry point when running from source or binary packages.
+- **`src/cli_main.py`**: Command-line interface parser and validator based on standard Python `argparse`. Translates CLI arguments into application configuration.
+- **`src/dup_main.py`**: Top-level workflow coordinator. Dispatches commands (`backup`, `restore`, `verify`, `collection-status`, `list-current-files`, `cleanup`, `remove-older-than`, etc.).
+
+### 2. Core Processing Engines
+- **Selection Engine (`src/selection.py`)**: Traverses source directories and evaluates file inclusion/exclusion rules based on globbing, regular expressions, file lists, or size/time criteria.
+- **Delta Engine (`src/librsync.py`, `src/_librsyncmodule.c`)**: Interfaces with the C `librsync` library. Computes rolling checksums (signatures) of existing files and binary differences (deltas) against previous versions.
+- **Archive Generator (`src/diffdir.py`)**: Bundles file deltas or full file contents into POSIX standard tar archives (`difftar`). Uses `zlib-ng` for high-throughput hardware-accelerated compression.
+- **Archive Restorer (`src/patchdir.py`)**: Reconstructs directory trees by streaming full archives and applying successive incremental deltas in chronological order.
+- **Manifest Engine (`src/manifest.py`)**: Records file listings, SHA1/SHA256 checksums, and volume block allocations for each backup set.
+
+### 3. Cryptography Subsystem (`src/gpgme/`)
+Duplicity uses GPGME (GnuPG Made Easy) bindings via `python3-gpg` (`src/gpgme/`):
+- **`src/gpgme/context.py`**: Manages `gpg.Context` lifecycles, protocol selection (OpenPGP vs. CMS/X.509), loopback pinentry, and passphrase callbacks.
+- **`src/gpgme/file.py`**: Provides asynchronous streaming encryption, decryption, signing, and verification wrapper streams (`GPGFile`).
+- **`src/gpgme/keyring.py`**: Interrogates GnuPG keyrings for key existence, fingerprints, and validation.
+- **`src/gpgme/profile.py`**: Encapsulates encryption parameters, recipient keys, signing keys, and passphrases.
+
+### 4. Storage Backend Layer (`src/backend.py` & `src/backends/`)
+- **`src/backend.py` (`BackendWrapper`)**: Provides a uniform interface for all operations (`put`, `get`, `list`, `delete`, `query_info`, `move`) with automatic retry logic, network backoff, and volume validation.
+- **`src/backends/`**: Modular storage drivers implementing target-specific protocols and cloud APIs (SFTP, S3, B2, Azure, Google Drive, OneDrive, WebDAV, rsync, Swift, etc.).
+
+## Archive Volume Structure and Naming Conventions
+
+Backup sets consist of several file types uploaded to the storage backend:
+
+| File Type | Pattern | Description |
+|-----------|---------|-------------|
+| **Full Volume** | `duplicity-full.<time>.vol<n>.difftar[.gpg]` | POSIX tar archive containing full snapshot data of volume `n`. |
+| **Incremental Volume** | `duplicity-inc.<start_time>.to.<end_time>.vol<n>.difftar[.gpg]` | POSIX tar archive containing binary librsync deltas for changed files. |
+| **Full Signatures** | `duplicity-full-signatures.<time>[.vol<n>].sigtar[.gpg]` | Signatures (librsync checksum blocks) for all files in the full backup. |
+| **Incremental Signatures** | `duplicity-new-signatures.<start_time>.to.<end_time>[.vol<n>].sigtar[.gpg]` | Signatures for new/modified files in the incremental backup. |
+| **Manifest** | `duplicity-(full\|inc).<time>.manifest[.gpg]` | Metadata listing all archived files, volumes, and file checksums. |
+| **Completion Record** | `duplicity-(full\|inc).<time>.manifest.complete[.gpg]` | Atomic commit record specifying complete manifest and signature part sets. |
+| **Statistics** | `duplicity-(full\|inc).<time>.jsonstat[.gz\|.gpg]` | Machine-readable JSON backup statistics and transfer metrics. |
+
+- `<time>` format: ISO 8601 UTC timestamp string, formatted as `YYYYMMDDTHHMMSSZ`.
+- Extensions: Volumes are unencrypted (e.g. `.difftar`) or encrypted with GnuPG (`.difftar.gpg` or `.difftar.p7m`), depending on encryption settings.
+
+## Data Flow
+
+### Full Backup
+1. **Cataloging & Signatures**: Duplicity scans the source tree via `selection.py`.
+2. **Tar Volume Generation**: Files are packed into `difftar` streams up to `--volsize` (default: 200MB).
+3. **Encryption & Signing**: Volume streams pass through `GPGFile` for encryption/signing.
+4. **Upload**: Volumes are transferred to the backend and validated for size/hash integrity.
+5. **Signatures & Manifest**: A signature archive (`sigtar`) and manifest are generated, encrypted, and uploaded.
+
+### Incremental Backup
+1. **Signature Retrieval**: Duplicity downloads the signature file (`sigtar`) from the most recent backup.
+2. **Delta Computation**: The source tree is scanned; files that have changed are compared against cached signatures to generate librsync deltas.
+3. **Volume Generation & Upload**: Incremental `difftar` volumes containing only deltas and new files are encrypted and uploaded.
+4. **Signature & Manifest Update**: New signature chains and manifests are uploaded.
+
+### Restore
+1. **Collection Status**: Remote backend is scanned to locate the full backup and necessary incremental chain covering the requested target time.
+2. **Manifest Verification**: The manifest is read to determine which volumes contain the required file(s).
+3. **Stream Reconstruction**: Full `difftar` volumes are streamed and unpacked, followed by sequential delta application from each incremental volume until the target state is restored.