PyHardLinkBackup 1.0.0rc0__tar.gz → 1.0.1__tar.gz

pyhardlinkbackup-1.0.1/.run/only DocWrite.run.xml

@@ -0,0 +1,24 @@
+<component name="ProjectRunConfigurationManager">
+  <configuration default="false" name="only DocWrite" type="tests" factoryName="Unittests">
+    <module name="PyHardLinkBackup" />
+    <option name="ENV_FILES" value="" />
+    <option name="INTERPRETER_OPTIONS" value="" />
+    <option name="PARENT_ENVS" value="true" />
+    <envs>
+      <env name="PYTHONUNBUFFERED" value="1" />
+      <env name="PYTHONWARNINGS" value="always" />
+    </envs>
+    <option name="SDK_HOME" value="" />
+    <option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
+    <option name="IS_MODULE_SDK" value="true" />
+    <option name="ADD_CONTENT_ROOTS" value="false" />
+    <option name="ADD_SOURCE_ROOTS" value="false" />
+    <EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
+    <option name="RUN_TOOL" value="" />
+    <option name="_new_pattern" value="&quot;&quot;" />
+    <option name="_new_additionalArguments" value="&quot;&quot;" />
+    <option name="_new_target" value="&quot;PyHardLinkBackup.tests.test_doc_write&quot;" />
+    <option name="_new_targetType" value="&quot;PYTHON&quot;" />
+    <method v="2" />
+  </configuration>
+</component>

{pyhardlinkbackup-1.0.0rc0 → pyhardlinkbackup-1.0.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: PyHardLinkBackup
-Version: 1.0.0rc0
+Version: 1.0.1
 Summary: HardLink/Deduplication Backups with Python
 Project-URL: Documentation, https://github.com/jedie/PyHardLinkBackup
 Project-URL: Source, https://github.com/jedie/PyHardLinkBackup
@@ -23,7 +23,71 @@ Description-Content-Type: text/markdown
 
 HardLink/Deduplication Backups with Python
 
-**WIP:** v1.0.0 is a complete rewrite of PyHardLinkBackup. The new version is not usable, yet!
+**WIP:** v1.0.0 is a complete rewrite of PyHardLinkBackup.
+
+## installation
+
+You can use [pipx](https://pipx.pypa.io/stable/installation/) to install and use PyHardLinkBackup, e.g.:
+
+```bash
+sudo apt install pipx
+
+pipx install PyHardLinkBackup
+```
+
+After this you can call the CLI via `phlb` command.
+The main command is `phlb backup <source> <destination>`:
+
+[comment]: <> (✂✂✂ auto generated backup help start ✂✂✂)
+```
+usage: phlb backup [-h] source destination [--excludes [STR [STR ...]]] [-v]
+
+Backup the source directory to the destination directory using hard links for deduplication.
+
+╭─ positional arguments ───────────────────────────────────────────────────────────────────────────────────────────────╮
+│ source       Source directory to back up. (required)                                                                 │
+│ destination  Destination directory for the backup. (required)                                                        │
+╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ -h, --help   show this help message and exit                                                                         │
+│ --excludes [STR [STR ...]]                                                                                           │
+│              List of directory or file names to exclude from backup. (default: __pycache__ .cache .temp .tmp .tox    │
+│              .nox)                                                                                                   │
+│ -v, --verbosity                                                                                                      │
+│              Verbosity level; e.g.: -v, -vv, -vvv, etc. (repeatable)                                                 │
+╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```
+[comment]: <> (✂✂✂ auto generated backup help end ✂✂✂)
+
+
+
+complete help for main CLI app:
+
+[comment]: <> (✂✂✂ auto generated main help start ✂✂✂)
+```
+usage: phlb [-h] {backup,version}
+
+
+
+╭─ options ─────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ -h, --help   show this help message and exit                                                              │
+╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ subcommands ─────────────────────────────────────────────────────────────────────────────────────────────╮
+│ (required)                                                                                                │
+│   • backup   Backup the source directory to the destination directory using hard links for deduplication. │
+│   • version  Print version and exit                                                                       │
+╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```
+[comment]: <> (✂✂✂ auto generated main help end ✂✂✂)
+
+
+### update
+
+If you use pipx, just call:
+```bash
+pipx upgrade PyHardLinkBackup
+```
+see: https://pipx.pypa.io/stable/docs/#pipx-upgrade
 
 
 ## concept
@@ -83,70 +147,18 @@ e.g.: hash like `abcdef123...` stored in: `{destination}/.phlb/hash-lookup/ab/cd
 The file contains only the relative path to the first hardlink of this file content.
 
 
-## CLI - backup command
-
-The main command is `backup`:
-
-[comment]: <> (✂✂✂ auto generated backup help start ✂✂✂)
-```
-usage: ./cli.py backup [-h] source destination [--excludes STR|{[STR [STR ...]]}] [-v]
-
-Backup the source directory to the destination directory using hard links for deduplication.
-
-╭─ positional arguments ───────────────────────────────────────────────────────────────────────────────────────────────╮
-│ source       Source directory to back up. (required)                                                                 │
-│ destination  Destination directory for the backup. (required)                                                        │
-╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
-╭─ options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ -h, --help   show this help message and exit                                                                         │
-│ --excludes STR|{[STR [STR ...]]}                                                                                     │
-│              List of directory or file names to exclude from backup. (default: __pycache__ .cache .temp .tmp .tox    │
-│              .nox)                                                                                                   │
-│ -v, --verbosity                                                                                                      │
-│              Verbosity level; e.g.: -v, -vv, -vvv, etc. (repeatable)                                                 │
-╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
-```
-[comment]: <> (✂✂✂ auto generated backup help end ✂✂✂)
-
-
-## CLI - main app help
-
-[comment]: <> (✂✂✂ auto generated main help start ✂✂✂)
-```
-usage: ./cli.py [-h] {backup,benchmark-hashes,version}
-
+## start development
 
-
-╭─ options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ -h, --help            show this help message and exit                                                                │
-╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
-╭─ subcommands ────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ (required)                                                                                                           │
-│   • backup            Backup the source directory to the destination directory using hard links for deduplication.   │
-│   • benchmark-hashes  Benchmark different file hashing algorithms on the given path Example output:                  │
-│                                                                                                                      │
-│                       Total files hashed: 220, total size: 1187.7 MiB                                                │
-│                                                                                                                      │
-│                       Results: Total file content read time: 1.7817s                                                 │
-│                                                                                                                      │
-│                       sha1       | Total: 0.6827s | 0.4x hash/read sha256     | Total: 0.7189s | 0.4x hash/read      │
-│                       sha224     | Total: 0.7375s | 0.4x hash/read sha384     | Total: 1.6552s | 0.9x hash/read      │
-│                       blake2b    | Total: 1.6708s | 0.9x hash/read md5        | Total: 1.6870s | 0.9x hash/read      │
-│                       sha512     | Total: 1.7269s | 1.0x hash/read shake_128  | Total: 1.9834s | 1.1x hash/read      │
-│                       sha3_224   | Total: 2.3006s | 1.3x hash/read sha3_256   | Total: 2.3856s | 1.3x hash/read      │
-│                       shake_256  | Total: 2.4375s | 1.4x hash/read blake2s    | Total: 2.5219s | 1.4x hash/read      │
-│                       sha3_384   | Total: 3.2596s | 1.8x hash/read sha3_512   | Total: 4.5328s | 2.5x hash/read      │
-│   • version           Print version and exit                                                                         │
-╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```bash
+~$ git clone https://github.com/jedie/PyHardLinkBackup.git
+~$ cd PyHardLinkBackup
+~/PyHardLinkBackup$ ./cli.py --help
+~/PyHardLinkBackup$ ./dev-cli.py --help
 ```
-[comment]: <> (✂✂✂ auto generated main help end ✂✂✂)
-
-
-## dev CLI
 
 [comment]: <> (✂✂✂ auto generated dev help start ✂✂✂)
 ```
-usage: ./dev-cli.py [-h] {coverage,install,lint,mypy,nox,pip-audit,publish,shell-completion,test,update,update-readme-history,update-test-snapshot-files,version}
+usage: ./dev-cli.py [-h] {benchmark-hashes,coverage,install,lint,mypy,nox,pip-audit,publish,shell-completion,test,update,update-readme-history,update-test-snapshot-files,version}
 
 
 
@@ -155,6 +167,8 @@ usage: ./dev-cli.py [-h] {coverage,install,lint,mypy,nox,pip-audit,publish,shell
 ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ╭─ subcommands ────────────────────────────────────────────────────────────────────────────────────────────────────────╮
 │ (required)                                                                                                           │
+│   • benchmark-hashes                                                                                                 │
+│                Benchmark different file hashing algorithms on the given path.                                        │
 │   • coverage   Run tests and show coverage report.                                                                   │
 │   • install    Install requirements and 'PyHardLinkBackup' via pip as editable.                                      │
 │   • lint       Check/fix code style by run: "ruff check --fix"                                                       │
@@ -190,7 +204,16 @@ v1 is a complete rewrite of PyHardLinkBackup.
 
 [comment]: <> (✂✂✂ auto generated history start ✂✂✂)
 
-* [v1.0.0rc0](https://github.com/jedie/PyHardLinkBackup/compare/v0.13.0...v1.0.0rc0)
+* [v1.0.1](https://github.com/jedie/PyHardLinkBackup/compare/v1.0.0...v1.0.1)
+  * 2026-01-13 - Store SHA256SUMS files in backup directories
+* [v1.0.0](https://github.com/jedie/PyHardLinkBackup/compare/v0.13.0...v1.0.0)
+  * 2026-01-13 - Change "./cli.py" to "phlb" (because it's the name installed via pipx)
+  * 2026-01-13 - Update README
+  * 2026-01-13 - Fix benchmark moved to dev CLI ;)
+  * 2026-01-13 - Remove tyro warning
+  * 2026-01-13 - Move "benchmark_hashes" from app to dev cli (It's more for testing)
+  * 2026-01-13 - Rename [project.scripts] hooks
+  * 2026-01-13 - Add DocWrite, handle broken symlinks, keep file meta, handle missing hardlink sources
   * 2026-01-12 - First working iteration with rich progess bar
   * 2026-01-08 - Rewrite everything
 * [v0.13.0](https://github.com/jedie/PyHardLinkBackup/compare/v0.12.3...v0.13.0)
@@ -220,6 +243,9 @@ v1 is a complete rewrite of PyHardLinkBackup.
   * 2020-03-16 - just warn if used directly (needfull for devlopment to call this directly ;)
   * 2020-03-16 - update requirements
   * 2020-03-16 - +pytest-randomly
+
+<details><summary>Expand older history entries ...</summary>
+
 * [v0.12.2](https://github.com/jedie/PyHardLinkBackup/compare/v0.12.1...v0.12.2)
   * 2020-03-06 - repare v0.12.2 release
   * 2020-03-06 - enhance log file content
@@ -228,9 +254,6 @@ v1 is a complete rewrite of PyHardLinkBackup.
   * 2020-03-05 - Fix #40 by decrease log level
   * 2020-03-05 - Update boot_pyhardlinkbackup.cmd
   * 2020-03-05 - Update boot_pyhardlinkbackup.sh
-
-<details><summary>Expand older history entries ...</summary>
-
 * [v0.12.1](https://github.com/jedie/PyHardLinkBackup/compare/v0.12.0...v0.12.1)
   * 2020-03-05 - update tests and set version to 0.12.1
   * 2020-03-05 - less verbose pytest output

{pyhardlinkbackup-1.0.0rc0 → pyhardlinkbackup-1.0.1}/PyHardLinkBackup/__init__.py

@@ -1,8 +1,7 @@
-"""
-    PyHardLinkBackup
+"""DocWrite: README.md # PyHardLinkBackup
     HardLink/Deduplication Backups with Python
 """
 
 # See https://packaging.python.org/en/latest/specifications/version-specifiers/
-__version__ = '1.0.0rc0'
+__version__ = '1.0.1'
 __author__ = 'Jens Diemer <PyHardLinkBackup@jensdiemer.de>'

{pyhardlinkbackup-1.0.0rc0 → pyhardlinkbackup-1.0.1}/PyHardLinkBackup/backup.py

@@ -40,6 +40,22 @@ class BackupResult:
     copied_small_size: int
 
 
+def store_hash(file_path: Path, file_hash: str):
+    """DocWrite: README.md ## SHA256SUMS
+    A `SHA256SUMS` file is stored in each backup directory containing the SHA256 hashes of all files in that directory.
+    It's the same format as e.g.: `sha256sum * > SHA256SUMS` command produces.
+    So it's possible to verify the integrity of the backup files later.
+    e.g.:
+    ```bash
+    cd .../your/backup/foobar/20240101_120000/
+    sha256sum -c SHA256SUMS
+    ```
+    """
+    hash_file_path = file_path.parent / 'SHA256SUMS'
+    with hash_file_path.open('a') as f:
+        f.write(f'{file_hash}  {file_path.name}\n')
+
+
 def backup_tree(*, src_root: Path, backup_root: Path, excludes: set[str]) -> BackupResult:
     src_root = src_root.resolve()
     if not src_root.is_dir():
@@ -84,22 +100,37 @@ def backup_tree(*, src_root: Path, backup_root: Path, excludes: set[str]) -> Bac
         next_update = 0
         for entry in iter_scandir_files(src_root, excludes=excludes):
             backup_count += 1
-            size = entry.stat().st_size
+            src_path = Path(entry.path)
+
+            dst_path = backup_dir / src_path.relative_to(src_root)
+            dst_dir_path = dst_path.parent
+            if not dst_dir_path.exists():
+                dst_dir_path.mkdir(parents=True, exist_ok=False)
+
+            try:
+                size = entry.stat().st_size
+            except FileNotFoundError:
+                # e.g.: Handle broken symlink
+                target = os.readlink(src_path)
+                dst_path.symlink_to(target)
+                symlink_files += 1
+                continue
+
             backup_size += size
 
+            if entry.name == 'SHA256SUMS':
+                # Skip existing SHA256SUMS files in source tree,
+                # because we create our own SHA256SUMS files.
+                logger.debug('Skip existing SHA256SUMS file: %s', src_path)
+                continue
+
             now = time.monotonic()
             if now >= next_update:
                 progress.update(backup_count=backup_count, backup_size=backup_size)
                 next_update = now + 0.5
 
-            src_path = Path(entry.path)
-            dst_path = backup_dir / src_path.relative_to(src_root)
-
-            dst_path.parent.mkdir(parents=True, exist_ok=True)
-
             if entry.is_symlink():
                 logger.debug('Copy symlink: %s to %s', src_path, dst_path)
-                # Copy symlinks as-is
                 target = os.readlink(src_path)
                 dst_path.symlink_to(target)
                 symlink_files += 1
@@ -114,10 +145,12 @@ def backup_tree(*, src_root: Path, backup_root: Path, excludes: set[str]) -> Bac
                 # Small file -> always copy without deduplication
                 logger.info('Copy small file: %s to %s', src_path, dst_path)
                 shutil.copy2(src_path, dst_path)
+                file_hash = copy_and_hash(src_path, dst_path)
                 copied_files += 1
                 copied_size += size
                 copied_small_files += 1
                 copied_small_size += size
+                store_hash(dst_path, file_hash)
                 continue
 
             if size in size_db:
@@ -152,10 +185,12 @@ def backup_tree(*, src_root: Path, backup_root: Path, excludes: set[str]) -> Bac
                         hardlinked_size += size
                     else:
                         logger.info('Copy unique file: %s to %s', src_path, dst_path)
-                        shutil.copy2(src_path, dst_path)
                         hash_db[file_hash] = dst_path
                         copied_files += 1
                         copied_size += size
+
+                # Keep original file metadata (permission bits, time stamps, and flags)
+                shutil.copy2(src_path, dst_path)
             else:
                 # A file with this size not backuped before -> Can't be duplicate -> copy and hash
                 file_hash = copy_and_hash(src_path, dst_path)
@@ -164,6 +199,9 @@ def backup_tree(*, src_root: Path, backup_root: Path, excludes: set[str]) -> Bac
                 copied_files += 1
                 copied_size += size
 
+            store_hash(dst_path, file_hash)
+
+        # Finalize progress indicator values:
         progress.update(backup_count=backup_count, backup_size=backup_size)
 
     print(f'\nBackup complete: {backup_dir} (total size {human_filesize(backup_size)})\n')

{pyhardlinkbackup-1.0.0rc0 → pyhardlinkbackup-1.0.1}/PyHardLinkBackup/cli_app/phlb.py

@@ -32,7 +32,7 @@ def backup(
     ],
     /,
     excludes: Annotated[
-        tuple[str],
+        tuple,
         tyro.conf.arg(
             help='List of directory or file names to exclude from backup.',
         ),

{pyhardlinkbackup-1.0.0rc0/PyHardLinkBackup/cli_app → pyhardlinkbackup-1.0.1/PyHardLinkBackup/cli_dev}/benchmark.py

@@ -9,7 +9,7 @@ from cli_base.cli_tools.verbosity import setup_logging
 from cli_base.tyro_commands import TyroVerbosityArgType
 from rich import print  # noqa
 
-from PyHardLinkBackup.cli_app import app
+from PyHardLinkBackup.cli_dev import app
 from PyHardLinkBackup.utilities.filesystem import iter_scandir_files
 
 
@@ -26,30 +26,29 @@ def benchmark_hashes(
     verbosity: TyroVerbosityArgType = 1,
 ) -> None:
     """
-    Benchmark different file hashing algorithms on the given path
-
-    Example output:
-
-    Total files hashed: 220, total size: 1187.7 MiB
-
-    Results:
-    Total file content read time: 1.7817s
-
-    sha1       | Total: 0.6827s | 0.4x hash/read
-    sha256     | Total: 0.7189s | 0.4x hash/read
-    sha224     | Total: 0.7375s | 0.4x hash/read
-    sha384     | Total: 1.6552s | 0.9x hash/read
-    blake2b    | Total: 1.6708s | 0.9x hash/read
-    md5        | Total: 1.6870s | 0.9x hash/read
-    sha512     | Total: 1.7269s | 1.0x hash/read
-    shake_128  | Total: 1.9834s | 1.1x hash/read
-    sha3_224   | Total: 2.3006s | 1.3x hash/read
-    sha3_256   | Total: 2.3856s | 1.3x hash/read
-    shake_256  | Total: 2.4375s | 1.4x hash/read
-    blake2s    | Total: 2.5219s | 1.4x hash/read
-    sha3_384   | Total: 3.2596s | 1.8x hash/read
-    sha3_512   | Total: 4.5328s | 2.5x hash/read
+    Benchmark different file hashing algorithms on the given path.
     """
+    # Example output:
+    #
+    # Total files hashed: 220, total size: 1187.7 MiB
+    #
+    # Results:
+    # Total file content read time: 1.7817s
+    #
+    # sha1       | Total: 0.6827s | 0.4x hash/read
+    # sha256     | Total: 0.7189s | 0.4x hash/read
+    # sha224     | Total: 0.7375s | 0.4x hash/read
+    # sha384     | Total: 1.6552s | 0.9x hash/read
+    # blake2b    | Total: 1.6708s | 0.9x hash/read
+    # md5        | Total: 1.6870s | 0.9x hash/read
+    # sha512     | Total: 1.7269s | 1.0x hash/read
+    # shake_128  | Total: 1.9834s | 1.1x hash/read
+    # sha3_224   | Total: 2.3006s | 1.3x hash/read
+    # sha3_256   | Total: 2.3856s | 1.3x hash/read
+    # shake_256  | Total: 2.4375s | 1.4x hash/read
+    # blake2s    | Total: 2.5219s | 1.4x hash/read
+    # sha3_384   | Total: 3.2596s | 1.8x hash/read
+    # sha3_512   | Total: 4.5328s | 2.5x hash/read
     setup_logging(verbosity=verbosity)
     assert_is_dir(base_path)
     print(f'Benchmarking file hashes under: {base_path}')