PyHardLinkBackup 1.2.0__tar.gz → 1.3.0__tar.gz

{pyhardlinkbackup-1.2.0 → pyhardlinkbackup-1.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: PyHardLinkBackup
-Version: 1.2.0
+Version: 1.3.0
 Summary: HardLink/Deduplication Backups with Python
 Project-URL: Documentation, https://github.com/jedie/PyHardLinkBackup
 Project-URL: Source, https://github.com/jedie/PyHardLinkBackup
@@ -63,22 +63,36 @@ Backup the source directory to the destination directory using hard links for de
 
 
 
+Running a backup looks like:
+
+![2026-01-15-phlb1.png](https://raw.githubusercontent.com/jedie/jedie.github.io/main/screenshots/PyHardLinkBackup/2026-01-15-phlb1.png "2026-01-15-phlb1.png")
+
+
+
+If it's finished it display a summary:
+
+![2026-01-15-phlb2.png](https://raw.githubusercontent.com/jedie/jedie.github.io/main/screenshots/PyHardLinkBackup/2026-01-15-phlb2.png "2026-01-15-phlb2.png")
+
+
+
 complete help for main CLI app:
 
 [comment]: <> (✂✂✂ auto generated main help start ✂✂✂)
 ```
-usage: phlb [-h] {backup,version}
+usage: phlb [-h] {backup,rebuild,version}
 
 
 
-╭─ options ─────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ -h, --help   show this help message and exit                                                              │
-╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
-╭─ subcommands ─────────────────────────────────────────────────────────────────────────────────────────────╮
-│ (required)                                                                                                │
-│   • backup   Backup the source directory to the destination directory using hard links for deduplication. │
-│   • version  Print version and exit                                                                       │
-╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ -h, --help   show this help message and exit                                                                         │
+╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ subcommands ────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ (required)                                                                                                           │
+│   • backup   Backup the source directory to the destination directory using hard links for deduplication.            │
+│   • rebuild  Rebuild the file hash and size database by scanning all backup files. And also verify SHA256SUMS and/or │
+│              store missing hashes in SHA256SUMS files.                                                               │
+│   • version  Print version and exit                                                                                  │
+╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 [comment]: <> (✂✂✂ auto generated main help end ✂✂✂)
 
@@ -214,6 +228,12 @@ Overview of main changes:
 
 [comment]: <> (✂✂✂ auto generated history start ✂✂✂)
 
+* [v1.3.0](https://github.com/jedie/PyHardLinkBackup/compare/v1.2.0...v1.3.0)
+  * 2026-01-15 - Verify SHA256SUMS files in "rebuild" command, too.
+  * 2026-01-15 - Code cleanup: use more generic names for and in BackupProgress
+  * 2026-01-15 - Add tests for rebuild
+  * 2026-01-15 - Add command to "rebuld" the size and hash filesystem database
+  * 2026-01-15 - Add screenshots in the README
 * [v1.2.0](https://github.com/jedie/PyHardLinkBackup/compare/v1.1.0...v1.2.0)
   * 2026-01-15 - Add error handling: Log exception but continue with the backup
   * 2026-01-15 - Check permission and hadlink support on destination path
@@ -225,6 +245,9 @@ Overview of main changes:
   * 2026-01-14 - Add "Overview of main changes" to README
 * [v1.0.1](https://github.com/jedie/PyHardLinkBackup/compare/v1.0.0...v1.0.1)
   * 2026-01-13 - Store SHA256SUMS files in backup directories
+
+<details><summary>Expand older history entries ...</summary>
+
 * [v1.0.0](https://github.com/jedie/PyHardLinkBackup/compare/v0.13.0...v1.0.0)
   * 2026-01-13 - Change "./cli.py" to "phlb" (because it's the name installed via pipx)
   * 2026-01-13 - Update README
@@ -235,9 +258,6 @@ Overview of main changes:
   * 2026-01-13 - Add DocWrite, handle broken symlinks, keep file meta, handle missing hardlink sources
   * 2026-01-12 - First working iteration with rich progess bar
   * 2026-01-08 - Rewrite everything
-
-<details><summary>Expand older history entries ...</summary>
-
 * [v0.13.0](https://github.com/jedie/PyHardLinkBackup/compare/v0.12.3...v0.13.0)
   * 2020-03-18 - release v0.13.0
   * 2020-03-17 - deactivate pypy tests in travis, because of SQLite errors, like:

{pyhardlinkbackup-1.2.0 → pyhardlinkbackup-1.3.0}/PyHardLinkBackup/__init__.py

@@ -3,5 +3,5 @@
 """
 
 # See https://packaging.python.org/en/latest/specifications/version-specifiers/
-__version__ = '1.2.0'
+__version__ = '1.3.0'
 __author__ = 'Jens Diemer <PyHardLinkBackup@jensdiemer.de>'

{pyhardlinkbackup-1.2.0 → pyhardlinkbackup-1.3.0}/PyHardLinkBackup/backup.py

@@ -21,7 +21,8 @@ from PyHardLinkBackup.utilities.filesystem import (
     supports_hardlinks,
 )
 from PyHardLinkBackup.utilities.humanize import human_filesize
-from PyHardLinkBackup.utilities.rich_utils import BackupProgress
+from PyHardLinkBackup.utilities.rich_utils import DisplayFileTreeProgress
+from PyHardLinkBackup.utilities.sha256sums import store_hash
 
 
 logger = logging.getLogger(__name__)
@@ -151,22 +152,6 @@ def backup_one_file(
     store_hash(dst_path, file_hash)
 
 
-def store_hash(file_path: Path, file_hash: str):
-    """DocWrite: README.md ## SHA256SUMS
-    A `SHA256SUMS` file is stored in each backup directory containing the SHA256 hashes of all files in that directory.
-    It's the same format as e.g.: `sha256sum * > SHA256SUMS` command produces.
-    So it's possible to verify the integrity of the backup files later.
-    e.g.:
-    ```bash
-    cd .../your/backup/foobar/20240101_120000/
-    sha256sum -c SHA256SUMS
-    ```
-    """
-    hash_file_path = file_path.parent / 'SHA256SUMS'
-    with hash_file_path.open('a') as f:
-        f.write(f'{file_hash}  {file_path.name}\n')
-
-
 def backup_tree(*, src_root: Path, backup_root: Path, excludes: set[str]) -> BackupResult:
     src_root = src_root.resolve()
     if not src_root.is_dir():
@@ -202,7 +187,7 @@ def backup_tree(*, src_root: Path, backup_root: Path, excludes: set[str]) -> Bac
 
     print(f'\nBackup to {backup_dir}...\n')
 
-    with BackupProgress(src_file_count, src_total_size) as progress:
+    with DisplayFileTreeProgress(src_file_count, src_total_size) as progress:
         # "Databases" for deduplication
         size_db = FileSizeDatabase(phlb_conf_dir)
         hash_db = FileHashDatabase(backup_root, phlb_conf_dir)
@@ -226,11 +211,13 @@ def backup_tree(*, src_root: Path, backup_root: Path, excludes: set[str]) -> Bac
             else:
                 now = time.monotonic()
                 if now >= next_update:
-                    progress.update(backup_count=backup_result.backup_count, backup_size=backup_result.backup_size)
+                    progress.update(
+                        completed_file_count=backup_result.backup_count, completed_size=backup_result.backup_size
+                    )
                     next_update = now + 0.5
 
         # Finalize progress indicator values:
-        progress.update(backup_count=backup_result.backup_count, backup_size=backup_result.backup_size)
+        progress.update(completed_file_count=backup_result.backup_count, completed_size=backup_result.backup_size)
 
     print(f'\nBackup complete: {backup_dir} (total size {human_filesize(backup_result.backup_size)})\n')
     print(f'  Total files processed: {backup_result.backup_count}')

{pyhardlinkbackup-1.2.0 → pyhardlinkbackup-1.3.0}/PyHardLinkBackup/cli_app/phlb.py

@@ -7,6 +7,7 @@ from cli_base.cli_tools.verbosity import setup_logging
 from cli_base.tyro_commands import TyroVerbosityArgType
 from rich import print  # noqa
 
+from PyHardLinkBackup import rebuild_databases
 from PyHardLinkBackup.backup import backup_tree
 from PyHardLinkBackup.cli_app import app
 
@@ -48,3 +49,23 @@ def backup(
         backup_root=dst,
         excludes=set(excludes),
     )
+
+
+@app.command
+def rebuild(
+    backup_root: Annotated[
+        Path,
+        tyro.conf.arg(
+            metavar='backup-directory',
+            help='Root directory of the the backups.',
+        ),
+    ],
+    /,
+    verbosity: TyroVerbosityArgType = 2,
+) -> None:
+    """
+    Rebuild the file hash and size database by scanning all backup files. And also verify SHA256SUMS
+    and/or store missing hashes in SHA256SUMS files.
+    """
+    setup_logging(verbosity=verbosity)
+    rebuild_databases.rebuild(backup_root=backup_root)

pyhardlinkbackup-1.3.0/PyHardLinkBackup/rebuild_databases.py

@@ -0,0 +1,147 @@
+import dataclasses
+import logging
+import os
+import sys
+import time
+from pathlib import Path
+
+from PyHardLinkBackup.utilities.file_hash_database import FileHashDatabase
+from PyHardLinkBackup.utilities.file_size_database import FileSizeDatabase
+from PyHardLinkBackup.utilities.filesystem import hash_file, humanized_fs_scan, iter_scandir_files
+from PyHardLinkBackup.utilities.humanize import human_filesize
+from PyHardLinkBackup.utilities.rich_utils import DisplayFileTreeProgress
+from PyHardLinkBackup.utilities.sha256sums import check_sha256sums, store_hash
+
+
+logger = logging.getLogger(__name__)
+
+
+@dataclasses.dataclass
+class RebuildResult:
+    process_count: int = 0
+    process_size: int = 0
+    #
+    added_size_count: int = 0
+    added_hash_count: int = 0
+    #
+    error_count: int = 0
+    #
+    hash_verified_count: int = 0
+    hash_mismatch_count: int = 0
+    hash_not_found_count: int = 0
+
+
+def rebuild_one_file(
+    *,
+    entry: os.DirEntry,
+    size_db: FileSizeDatabase,
+    hash_db: FileHashDatabase,
+    rebuild_result: RebuildResult,
+):
+    rebuild_result.process_count += 1
+
+    if entry.name == 'SHA256SUMS':
+        # Skip existing SHA256SUMS files
+        return
+
+    size = entry.stat().st_size
+    rebuild_result.process_size += size
+
+    if size < size_db.MIN_SIZE:
+        # Small files will never deduplicate, skip them
+        return
+
+    file_path = Path(entry.path)
+    file_hash = hash_file(file_path)
+
+    if size not in size_db:
+        size_db.add(size)
+        rebuild_result.added_size_count += 1
+
+    if file_hash not in hash_db:
+        hash_db[file_hash] = file_path
+        rebuild_result.added_hash_count += 1
+
+    # We have calculated the current hash of the file,
+    # Let's check if we can verify it, too:
+    file_path = Path(entry.path)
+    compare_result = check_sha256sums(
+        file_path=file_path,
+        file_hash=file_hash,
+    )
+    if compare_result is True:
+        rebuild_result.hash_verified_count += 1
+    elif compare_result is False:
+        rebuild_result.hash_mismatch_count += 1
+    elif compare_result is None:
+        rebuild_result.hash_not_found_count += 1
+        store_hash(
+            file_path=file_path,
+            file_hash=file_hash,
+        )
+
+
+def rebuild(backup_root: Path) -> RebuildResult:
+    backup_root = backup_root.resolve()
+    if not backup_root.is_dir():
+        print(f'Error: Backup directory "{backup_root}" does not exist!')
+        sys.exit(1)
+
+    phlb_conf_dir = backup_root / '.phlb'
+    if not phlb_conf_dir.is_dir():
+        print(
+            f'Error: Backup directory "{backup_root}" seems to be wrong:'
+            f' Our hidden ".phlb" configuration directory is missing!'
+        )
+        sys.exit(1)
+
+    file_count, total_size = humanized_fs_scan(backup_root, excludes={'.phlb'})
+
+    with DisplayFileTreeProgress(file_count, total_size) as progress:
+        # "Databases" for deduplication
+        size_db = FileSizeDatabase(phlb_conf_dir)
+        hash_db = FileHashDatabase(backup_root, phlb_conf_dir)
+
+        rebuild_result = RebuildResult()
+
+        next_update = 0
+        for entry in iter_scandir_files(backup_root, excludes={'.phlb'}):
+            try:
+                rebuild_one_file(
+                    entry=entry,
+                    size_db=size_db,
+                    hash_db=hash_db,
+                    rebuild_result=rebuild_result,
+                )
+            except Exception as err:
+                logger.exception(f'Backup {entry.path} {err.__class__.__name__}: {err}')
+                rebuild_result.error_count += 1
+            else:
+                now = time.monotonic()
+                if now >= next_update:
+                    progress.update(
+                        completed_file_count=rebuild_result.process_count, completed_size=rebuild_result.process_size
+                    )
+                    next_update = now + 0.5
+
+        # Finalize progress indicator values:
+        progress.update(completed_file_count=rebuild_result.process_count, completed_size=rebuild_result.process_size)
+
+    print(f'\nRebuild "{backup_root}" completed:')
+    print(f'  Total files processed: {rebuild_result.process_count}')
+    print(f'  Total size processed: {human_filesize(rebuild_result.process_size)}')
+
+    print(f'  Added file size information entries: {rebuild_result.added_size_count}')
+    print(f'  Added file hash entries: {rebuild_result.added_hash_count}')
+
+    if rebuild_result.error_count > 0:
+        print(f'  Errors during rebuild: {rebuild_result.error_count} (see log for details)')
+
+    print('\nSHA256SUMS verification results:')
+    print(f'  Successfully verified files: {rebuild_result.hash_verified_count}')
+    print(f'  File hash mismatches: {rebuild_result.hash_mismatch_count}')
+    print(f'  File hashes not found, newly stored: {rebuild_result.hash_not_found_count}')
+
+    print()
+
+    return rebuild_result

pyhardlinkbackup-1.3.0/PyHardLinkBackup/tests/test_rebuild_database.py

@@ -0,0 +1,204 @@
+import logging
+import tempfile
+import textwrap
+from pathlib import Path
+from unittest.mock import patch
+
+from bx_py_utils.test_utils.redirect import RedirectOut
+from cli_base.cli_tools.test_utils.base_testcases import BaseTestCase
+
+from PyHardLinkBackup import rebuild_databases
+from PyHardLinkBackup.rebuild_databases import RebuildResult, rebuild, rebuild_one_file
+from PyHardLinkBackup.utilities.file_size_database import FileSizeDatabase
+
+
+def sorted_rglob_paths(path: Path):
+    return sorted([str(p.relative_to(path)) for p in path.rglob('*')])
+
+
+def sorted_rglob_files(path: Path):
+    return sorted([str(p.relative_to(path)) for p in path.rglob('*') if p.is_file()])
+
+
+class RebuildDatabaseTestCase(BaseTestCase):
+    def test_happy_path(self):
+        with tempfile.TemporaryDirectory() as temp_dir:
+            temp_path = Path(temp_dir)
+
+            backup_root = temp_path / 'backup'
+
+            with self.assertRaises(SystemExit), RedirectOut() as redirected_out:
+                rebuild(backup_root)
+
+            self.assertEqual(redirected_out.stderr, '')
+            self.assertEqual(redirected_out.stdout, f'Error: Backup directory "{backup_root}" does not exist!\n')
+
+            backup_root.mkdir()
+
+            with self.assertRaises(SystemExit), RedirectOut() as redirected_out:
+                rebuild(backup_root)
+
+            self.assertEqual(redirected_out.stderr, '')
+            self.assertIn('hidden ".phlb" configuration directory is missing', redirected_out.stdout)
+
+            phlb_conf_dir = backup_root / '.phlb'
+            phlb_conf_dir.mkdir()
+
+            #######################################################################################
+            # Run on empty backup directory:
+
+            self.assertEqual(sorted_rglob_paths(backup_root), ['.phlb'])
+
+            with self.assertLogs('PyHardLinkBackup', level=logging.DEBUG), RedirectOut() as redirected_out:
+                rebuild_result = rebuild(backup_root)
+            self.assertEqual(
+                rebuild_result,
+                RebuildResult(
+                    process_count=0,
+                    process_size=0,
+                    added_size_count=0,
+                    added_hash_count=0,
+                    error_count=0,
+                ),
+            )
+            self.assertEqual(
+                sorted_rglob_paths(backup_root),
+                [
+                    '.phlb',
+                    '.phlb/hash-lookup',
+                    '.phlb/size-lookup',
+                ],
+            )
+            self.assertEqual(redirected_out.stderr, '')
+
+            #######################################################################################
+            # Add one backuped file and run again:
+
+            snapshot_path = backup_root / 'source-name' / '2026-01-15-181709'
+            snapshot_path.mkdir(parents=True)
+
+            minimum_file_content = 'X' * FileSizeDatabase.MIN_SIZE
+            (snapshot_path / 'file1.txt').write_text(minimum_file_content)
+
+            with self.assertLogs('PyHardLinkBackup', level=logging.DEBUG), RedirectOut() as redirected_out:
+                rebuild_result = rebuild(backup_root)
+            self.assertEqual(
+                sorted_rglob_paths(backup_root),
+                [
+                    '.phlb',
+                    '.phlb/hash-lookup',
+                    '.phlb/hash-lookup/bb',
+                    '.phlb/hash-lookup/bb/c4',
+                    '.phlb/hash-lookup/bb/c4/bbc4de2ca238d1ec41fb622b75b5cf7d31a6d2ac92405043dd8f8220364fefc8',
+                    '.phlb/size-lookup',
+                    '.phlb/size-lookup/10',
+                    '.phlb/size-lookup/10/00',
+                    '.phlb/size-lookup/10/00/1000',
+                    'source-name',
+                    'source-name/2026-01-15-181709',
+                    'source-name/2026-01-15-181709/SHA256SUMS',
+                    'source-name/2026-01-15-181709/file1.txt',
+                ],
+            )
+            self.assertEqual(
+                sorted_rglob_files(backup_root),
+                [
+                    '.phlb/hash-lookup/bb/c4/bbc4de2ca238d1ec41fb622b75b5cf7d31a6d2ac92405043dd8f8220364fefc8',
+                    '.phlb/size-lookup/10/00/1000',
+                    'source-name/2026-01-15-181709/SHA256SUMS',
+                    'source-name/2026-01-15-181709/file1.txt',
+                ],
+            )
+            self.assertEqual(redirected_out.stderr, '')
+            self.assertEqual(
+                rebuild_result,
+                RebuildResult(
+                    process_count=1,
+                    process_size=1000,
+                    added_size_count=1,
+                    added_hash_count=1,
+                    error_count=0,
+                    hash_verified_count=0,
+                    hash_mismatch_count=0,
+                    hash_not_found_count=1,
+                ),
+            )
+            self.assertEqual(
+                (snapshot_path / 'SHA256SUMS').read_text(),
+                'bbc4de2ca238d1ec41fb622b75b5cf7d31a6d2ac92405043dd8f8220364fefc8  file1.txt\n',
+            )
+
+            #######################################################################################
+            # Add a file with same content and run again:
+
+            minimum_file_content = 'X' * FileSizeDatabase.MIN_SIZE
+            (snapshot_path / 'same_content.txt').write_text(minimum_file_content)
+
+            with self.assertLogs('PyHardLinkBackup', level=logging.DEBUG) as logs, RedirectOut() as redirected_out:
+                rebuild_result = rebuild(backup_root)
+            # No new hash of size entries, just the new file:
+            self.assertEqual(
+                sorted_rglob_files(backup_root),
+                [
+                    '.phlb/hash-lookup/bb/c4/bbc4de2ca238d1ec41fb622b75b5cf7d31a6d2ac92405043dd8f8220364fefc8',
+                    '.phlb/size-lookup/10/00/1000',
+                    'source-name/2026-01-15-181709/SHA256SUMS',
+                    'source-name/2026-01-15-181709/file1.txt',
+                    'source-name/2026-01-15-181709/same_content.txt',
+                ],
+            )
+            self.assertEqual(redirected_out.stderr, '')
+            self.assertEqual(
+                rebuild_result,
+                RebuildResult(
+                    process_count=3,
+                    process_size=2000,
+                    added_size_count=0,
+                    added_hash_count=0,
+                    error_count=0,
+                    hash_verified_count=1,  # Existing file verified successfully
+                    hash_mismatch_count=0,
+                    hash_not_found_count=1,  # One file added
+                ),
+                '\n'.join(logs.output) + redirected_out.stdout,
+            )
+            self.assertEqual(
+                (snapshot_path / 'SHA256SUMS').read_text(),
+                textwrap.dedent("""\
+                    bbc4de2ca238d1ec41fb622b75b5cf7d31a6d2ac92405043dd8f8220364fefc8  file1.txt
+                    bbc4de2ca238d1ec41fb622b75b5cf7d31a6d2ac92405043dd8f8220364fefc8  same_content.txt
+                """),
+            )
+
+            #######################################################################################
+            # Test error handling
+
+            def rebuild_one_file_mock(*, entry, **kwargs):
+                if entry.name == 'file1.txt':
+                    raise IOError('Bam!')
+                return rebuild_one_file(entry=entry, **kwargs)
+
+            with (
+                self.assertLogs('PyHardLinkBackup', level=logging.ERROR) as logs,
+                RedirectOut() as redirected_out,
+                patch.object(rebuild_databases, 'rebuild_one_file', rebuild_one_file_mock),
+            ):
+                rebuild_result = rebuild(backup_root)
+            logs = ''.join(logs.output)
+            self.assertIn(f'Backup {snapshot_path}/file1.txt OSError: Bam!\n', logs)
+            self.assertIn('\nTraceback (most recent call last):\n', logs)
+            self.assertEqual(redirected_out.stderr, '')
+
+            self.assertEqual(
+                rebuild_result,
+                RebuildResult(
+                    process_count=2,
+                    process_size=1000,
+                    added_size_count=0,
+                    added_hash_count=0,
+                    error_count=1,  # <<< one file caused error
+                    hash_verified_count=1,
+                    hash_mismatch_count=0,
+                    hash_not_found_count=0,
+                ),
+            )

{pyhardlinkbackup-1.2.0 → pyhardlinkbackup-1.3.0}/PyHardLinkBackup/utilities/file_hash_database.py

@@ -32,6 +32,10 @@ class FileHashDatabase:
         hash_path = self.base_path / first_dir_name / second_dir_name / hash
         return hash_path
 
+    def __contains__(self, hash: str) -> bool:
+        hash_path = self._get_hash_path(hash)
+        return hash_path.exists()
+
     def get(self, hash: str) -> Path | None:
         hash_path = self._get_hash_path(hash)
         try:

{pyhardlinkbackup-1.2.0 → pyhardlinkbackup-1.3.0}/PyHardLinkBackup/utilities/rich_utils.py

@@ -32,8 +32,8 @@ class HumanFileSizeColumn(ProgressColumn):
         return Text(human_filesize(file_size))
 
 
-class BackupProgress:
-    def __init__(self, src_file_count: int, src_total_size: int):
+class DisplayFileTreeProgress:
+    def __init__(self, total_file_count: int, total_size: int):
         percentage_format = '[progress.percentage]{task.percentage:>3.1f}%'
         self.overall_progress = Progress(
             TaskProgressColumn(text_format=percentage_format),
@@ -50,7 +50,7 @@ class BackupProgress:
             BarColumn(bar_width=50),
             TextColumn('{task.completed}/{task.total} Files'),
         )
-        self.file_count_progress_task_id = self.file_count_progress.add_task(description='', total=src_file_count)
+        self.file_count_progress_task_id = self.file_count_progress.add_task(description='', total=total_file_count)
         self.file_count_progress_task = self.file_count_progress.tasks[0]
 
         self.file_size_progress = Progress(
@@ -60,11 +60,11 @@ class BackupProgress:
             '|',
             TransferSpeedColumn(),
         )
-        self.file_size_progress_task_id = self.file_size_progress.add_task(description='', total=src_total_size)
+        self.file_size_progress_task_id = self.file_size_progress.add_task(description='', total=total_size)
         self.file_size_progress_task = self.file_size_progress.tasks[0]
 
         progress_table = Table.grid()
-        progress_table.add_row(Panel(self.overall_progress, title='[b]Overall Backup Progress', border_style='green'))
+        progress_table.add_row(Panel(self.overall_progress, title='[b]Overall Progress', border_style='green'))
         progress_table.add_row(Panel(self.file_count_progress, title='Total files saved'))
         progress_table.add_row(Panel(self.file_size_progress, title='Total file size processed'))
 
@@ -74,15 +74,15 @@ class BackupProgress:
         self.live.__enter__()
         return self
 
-    def update(self, backup_count: int, backup_size: int):
+    def update(self, completed_file_count: int, completed_size: int):
         self.file_count_progress.update(
             task_id=self.file_count_progress_task_id,
-            completed=backup_count,
+            completed=completed_file_count,
             refresh=True,
         )
         self.file_size_progress.update(
             task_id=self.file_size_progress_task_id,
-            completed=backup_size,
+            completed=completed_size,
             refresh=True,
         )
         self.overall_progress.update(

pyhardlinkbackup-1.3.0/PyHardLinkBackup/utilities/sha256sums.py

@@ -0,0 +1,61 @@
+import logging
+from pathlib import Path
+
+
+logger = logging.getLogger(__name__)
+
+
+def get_sha256sums_path(file_path: Path):
+    """
+    >>> get_sha256sums_path(Path('foo/bar/baz.txt'))
+    PosixPath('foo/bar/SHA256SUMS')
+    """
+    hash_file_path = file_path.parent / 'SHA256SUMS'
+    return hash_file_path
+
+
+def store_hash(file_path: Path, file_hash: str):
+    """DocWrite: README.md ## SHA256SUMS
+    A `SHA256SUMS` file is stored in each backup directory containing the SHA256 hashes of all files in that directory.
+    It's the same format as e.g.: `sha256sum * > SHA256SUMS` command produces.
+    So it's possible to verify the integrity of the backup files later.
+    e.g.:
+    ```bash
+    cd .../your/backup/foobar/20240101_120000/
+    sha256sum -c SHA256SUMS
+    ```
+    """
+    hash_file_path = get_sha256sums_path(file_path)
+    with hash_file_path.open('a') as f:
+        f.write(f'{file_hash}  {file_path.name}\n')
+
+
+def check_sha256sums(
+    *,
+    file_path: Path,
+    file_hash: str,
+) -> bool | None:
+    hash_file_path = get_sha256sums_path(file_path=file_path)
+    if not hash_file_path.is_file():
+        return None  # Nothing to verify against
+
+    with hash_file_path.open('r') as f:
+        for line in f:
+            try:
+                expected_hash, filename = line.split(' ', maxsplit=1)
+            except ValueError:
+                logger.exception(f'Invalid line in "{hash_file_path}": {line!r}')
+            else:
+                filename = filename.strip()
+                if filename == file_path.name:
+                    if not expected_hash == file_hash:
+                        logger.error(
+                            f'Hash {file_hash} from file {file_path} does not match hash in {hash_file_path} !'
+                        )
+                        return False
+                    else:
+                        logger.debug(f'{file_path} hash verified successfully from {hash_file_path}.')
+                        return True
+
+    logger.info('No SHA256SUMS entry found for file: %s', file_path)
+    return None

{pyhardlinkbackup-1.2.0 → pyhardlinkbackup-1.3.0}/PyHardLinkBackup/utilities/tests/test_file_hash_database.py

@@ -72,8 +72,10 @@ class FileHashDatabaseTestCase(BaseTestCase):
             file_a_path.touch()
 
             self.assertIs(hash_db.get('12345678abcdef'), None)
+            self.assertIs('12345678abcdef' in hash_db, False)
             hash_db['12345678abcdef'] = file_a_path
             self.assertEqual(hash_db.get('12345678abcdef'), file_a_path)
+            self.assertIs('12345678abcdef' in hash_db, True)
             with self.assertLogs('PyHardLinkBackup', level=logging.DEBUG):
                 self.assertEqual(
                     get_hash_db_filenames(hash_db),

{pyhardlinkbackup-1.2.0 → pyhardlinkbackup-1.3.0}/README.md

@@ -48,22 +48,36 @@ Backup the source directory to the destination directory using hard links for de
 
 
 
+Running a backup looks like:
+
+![2026-01-15-phlb1.png](https://raw.githubusercontent.com/jedie/jedie.github.io/main/screenshots/PyHardLinkBackup/2026-01-15-phlb1.png "2026-01-15-phlb1.png")
+
+
+
+If it's finished it display a summary:
+
+![2026-01-15-phlb2.png](https://raw.githubusercontent.com/jedie/jedie.github.io/main/screenshots/PyHardLinkBackup/2026-01-15-phlb2.png "2026-01-15-phlb2.png")
+
+
+
 complete help for main CLI app:
 
 [comment]: <> (✂✂✂ auto generated main help start ✂✂✂)
 ```
-usage: phlb [-h] {backup,version}
+usage: phlb [-h] {backup,rebuild,version}
 
 
 
-╭─ options ─────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ -h, --help   show this help message and exit                                                              │
-╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
-╭─ subcommands ─────────────────────────────────────────────────────────────────────────────────────────────╮
-│ (required)                                                                                                │
-│   • backup   Backup the source directory to the destination directory using hard links for deduplication. │
-│   • version  Print version and exit                                                                       │
-╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ -h, --help   show this help message and exit                                                                         │
+╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ subcommands ────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ (required)                                                                                                           │
+│   • backup   Backup the source directory to the destination directory using hard links for deduplication.            │
+│   • rebuild  Rebuild the file hash and size database by scanning all backup files. And also verify SHA256SUMS and/or │
+│              store missing hashes in SHA256SUMS files.                                                               │
+│   • version  Print version and exit                                                                                  │
+╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 [comment]: <> (✂✂✂ auto generated main help end ✂✂✂)
 
@@ -199,6 +213,12 @@ Overview of main changes:
 
 [comment]: <> (✂✂✂ auto generated history start ✂✂✂)
 
+* [v1.3.0](https://github.com/jedie/PyHardLinkBackup/compare/v1.2.0...v1.3.0)
+  * 2026-01-15 - Verify SHA256SUMS files in "rebuild" command, too.
+  * 2026-01-15 - Code cleanup: use more generic names for and in BackupProgress
+  * 2026-01-15 - Add tests for rebuild
+  * 2026-01-15 - Add command to "rebuld" the size and hash filesystem database
+  * 2026-01-15 - Add screenshots in the README
 * [v1.2.0](https://github.com/jedie/PyHardLinkBackup/compare/v1.1.0...v1.2.0)
   * 2026-01-15 - Add error handling: Log exception but continue with the backup
   * 2026-01-15 - Check permission and hadlink support on destination path
@@ -210,6 +230,9 @@ Overview of main changes:
   * 2026-01-14 - Add "Overview of main changes" to README
 * [v1.0.1](https://github.com/jedie/PyHardLinkBackup/compare/v1.0.0...v1.0.1)
   * 2026-01-13 - Store SHA256SUMS files in backup directories
+
+<details><summary>Expand older history entries ...</summary>
+
 * [v1.0.0](https://github.com/jedie/PyHardLinkBackup/compare/v0.13.0...v1.0.0)
   * 2026-01-13 - Change "./cli.py" to "phlb" (because it's the name installed via pipx)
   * 2026-01-13 - Update README
@@ -220,9 +243,6 @@ Overview of main changes:
   * 2026-01-13 - Add DocWrite, handle broken symlinks, keep file meta, handle missing hardlink sources
   * 2026-01-12 - First working iteration with rich progess bar
   * 2026-01-08 - Rewrite everything
-
-<details><summary>Expand older history entries ...</summary>
-
 * [v0.13.0](https://github.com/jedie/PyHardLinkBackup/compare/v0.12.3...v0.13.0)
   * 2020-03-18 - release v0.13.0
   * 2020-03-17 - deactivate pypy tests in travis, because of SQLite errors, like: