hbkit 0.4.2__tar.gz → 0.4.4__tar.gz

{hbkit-0.4.2 → hbkit-0.4.4}/.gitignore

@@ -8,3 +8,6 @@ build/
 .pytest_cache/
 .DS_Store
 *.part
+
+# Local-only fixtures: hashes of source files that only exist on this machine.
+.local/

{hbkit-0.4.2 → hbkit-0.4.4}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: hbkit
-Version: 0.4.2
+Version: 0.4.4
 Summary: Recover files from Synology Hyper Backup (.hbk) archives without Synology software
 Project-URL: Homepage, https://github.com/YordiLorenzo/hbkit
 Project-URL: Source, https://github.com/YordiLorenzo/hbkit
@@ -40,8 +40,7 @@ a tree, and pull out what you want. Works headless on Linux and macOS, including
 own Hyper Backup Explorer is awkward or unavailable.
 
 ```sh
-brew install lz4          # or: sudo apt install liblz4-1
-pip install hbkit
+brew install YordiLorenzo/tap/hbkit     # macOS / Linux, pulls in liblz4 for you
 
 hbk /Volumes/Backup doctor              # can this archive be recovered?
 hbk /Volumes/Backup doctor -p secret    # encrypted? add a password
@@ -49,8 +48,20 @@ hbk-tui /Volumes/Backup                 # browse and select interactively
 hbk /Volumes/Backup get "/Photos/*" ~/restore
 ```
 
-`liblz4` is a runtime requirement — chunks are raw LZ4 blocks. It is present on most
-systems already; set `HBK_LZ4` if yours lives somewhere unusual.
+### Other ways to install
+
+```sh
+pip install hbkit          # any platform; also needs liblz4 (see below)
+pipx install hbkit         # same, kept in its own environment
+```
+
+Arch users can build from [`packaging/aur/PKGBUILD`](packaging/aur/PKGBUILD); a nixpkgs
+derivation lives in [`packaging/nix/package.nix`](packaging/nix/package.nix).
+
+`liblz4` is a runtime requirement — chunks are raw LZ4 blocks, loaded with `dlopen`, so a
+missing library only shows up when you extract something. The Homebrew formula installs it
+for you. Otherwise: `brew install lz4`, `sudo apt install liblz4-1`, or
+`sudo pacman -S lz4`. Set `HBK_LZ4` if yours lives somewhere unusual.
 
 ---
 
@@ -277,6 +288,8 @@ Read this before trusting it with the only copy of anything.
   bytes, which is the one thing a recovery tool must never do.
 - **Whole-file dedup** (`off_virtual_file = -1`, files living in `Pool/file_pool`) is not
   decoded. One file in 501,278 in the reference archive.
+- **Windows is untested for extraction.** CI installs the package and checks that it
+  imports, but no archive has been rebuilt on Windows. Linux and macOS are verified.
 
 ## The format
 

{hbkit-0.4.2 → hbkit-0.4.4}/README.md

@@ -11,8 +11,7 @@ a tree, and pull out what you want. Works headless on Linux and macOS, including
 own Hyper Backup Explorer is awkward or unavailable.
 
 ```sh
-brew install lz4          # or: sudo apt install liblz4-1
-pip install hbkit
+brew install YordiLorenzo/tap/hbkit     # macOS / Linux, pulls in liblz4 for you
 
 hbk /Volumes/Backup doctor              # can this archive be recovered?
 hbk /Volumes/Backup doctor -p secret    # encrypted? add a password
@@ -20,8 +19,20 @@ hbk-tui /Volumes/Backup                 # browse and select interactively
 hbk /Volumes/Backup get "/Photos/*" ~/restore
 ```
 
-`liblz4` is a runtime requirement — chunks are raw LZ4 blocks. It is present on most
-systems already; set `HBK_LZ4` if yours lives somewhere unusual.
+### Other ways to install
+
+```sh
+pip install hbkit          # any platform; also needs liblz4 (see below)
+pipx install hbkit         # same, kept in its own environment
+```
+
+Arch users can build from [`packaging/aur/PKGBUILD`](packaging/aur/PKGBUILD); a nixpkgs
+derivation lives in [`packaging/nix/package.nix`](packaging/nix/package.nix).
+
+`liblz4` is a runtime requirement — chunks are raw LZ4 blocks, loaded with `dlopen`, so a
+missing library only shows up when you extract something. The Homebrew formula installs it
+for you. Otherwise: `brew install lz4`, `sudo apt install liblz4-1`, or
+`sudo pacman -S lz4`. Set `HBK_LZ4` if yours lives somewhere unusual.
 
 ---
 
@@ -248,6 +259,8 @@ Read this before trusting it with the only copy of anything.
   bytes, which is the one thing a recovery tool must never do.
 - **Whole-file dedup** (`off_virtual_file = -1`, files living in `Pool/file_pool`) is not
   decoded. One file in 501,278 in the reference archive.
+- **Windows is untested for extraction.** CI installs the package and checks that it
+  imports, but no archive has been rebuilt on Windows. Linux and macOS are verified.
 
 ## The format
 

hbkit-0.4.4/packaging/README.md

@@ -0,0 +1,59 @@
+# Packaging
+
+Recipes for package repositories that keep their definitions outside this repo. Each one
+pins a released sdist from PyPI, so bumping a version means changing the version string and
+the hash, nothing else.
+
+| Target | Files here | Lives at |
+| ------ | ---------- | -------- |
+| Homebrew | — | [YordiLorenzo/homebrew-tap](https://github.com/YordiLorenzo/homebrew-tap) |
+| AUR | `aur/PKGBUILD`, `aur/.SRCINFO` | `ssh://aur@aur.archlinux.org/hbkit.git` |
+| nixpkgs | `nix/package.nix` | `pkgs/by-name/hb/hbkit/package.nix` ([PR #550225](https://github.com/NixOS/nixpkgs/pull/550225)) |
+
+## A note on liblz4
+
+hbkit `dlopen()`s liblz4 rather than linking it, so a package that forgets the dependency
+still builds and installs cleanly, then fails the first time someone extracts a file. Every
+recipe here must declare it, and should ideally set `HBK_LZ4` to an absolute path so the
+library resolves regardless of the loader search path.
+
+Homebrew and Nix both wrap the two entry points with `HBK_LZ4` for exactly this reason —
+Nix has no global library directory at all, so the name alone resolves to nothing. Arch
+puts `liblz4.so.1` on the default loader path, so a plain dependency is enough there.
+
+The same shape caused a worse bug: workers are spawned as `sys.executable -m hbkit.runner`,
+and a wrapper script leaves `sys.executable` pointing at an interpreter that cannot import
+hbkit. Fixed in 0.4.3 by passing the parent's `sys.path` down. Any recipe that wraps the
+entry points needs at least 0.4.3.
+
+## AUR
+
+```sh
+git clone ssh://aur@aur.archlinux.org/hbkit.git
+cp packaging/aur/PKGBUILD packaging/aur/.SRCINFO hbkit/
+cd hbkit && git add PKGBUILD .SRCINFO && git commit -m "hbkit 0.4.3" && git push
+```
+
+`.SRCINFO` must match `PKGBUILD`; regenerate it on Arch with `makepkg --printsrcinfo >
+.SRCINFO`. The AUR rejects a push where the two disagree.
+
+## nixpkgs
+
+`nix/package.nix` goes to `pkgs/by-name/hb/hbkit/package.nix` in a nixpkgs fork. It
+references a maintainer that must exist first, so `maintainers/maintainer-list.nix` needs:
+
+```nix
+  yordilorenzo = {
+    email = "yordilorenzo@gmail.com";
+    github = "YordiLorenzo";
+    githubId = 7012112;
+    name = "Yordi de Kleijn";
+  };
+```
+
+Build it before opening the PR:
+
+```sh
+nix-build -A hbkit
+nix run -f . hbkit -- --version
+```

{hbkit-0.4.2 → hbkit-0.4.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "hbkit"
-version = "0.4.2"
+version = "0.4.4"
 description = "Recover files from Synology Hyper Backup (.hbk) archives without Synology software"
 readme = "README.md"
 requires-python = ">=3.10"

{hbkit-0.4.2 → hbkit-0.4.4}/src/hbkit/__init__.py

@@ -1,6 +1,6 @@
 """hbkit - recover files from Synology Hyper Backup (.hbk) archives without Synology software."""
 
-__version__ = "0.4.2"
+__version__ = "0.4.4"
 
 from .archive import (Archive, NeedPassword, UnsupportedArchive,  # noqa: F401
                       find_archive_root, is_archive)

{hbkit-0.4.2 → hbkit-0.4.4}/src/hbkit/archive.py

@@ -210,6 +210,16 @@ class _LazyCats(dict):
     def values(self):
         return [v for v in dict.values(self)]
 
+    @property
+    def known(self) -> list[int]:
+        """Every shard id the archive has, opened or not.
+
+        Iterating the dict yields only what something has already touched, so a
+        caller that just wants to *describe* the archive - `doctor` - has to ask
+        for this instead, or it reports an archive with shards as having none.
+        """
+        return sorted(self._dirs)
+
 
 def is_archive(d: str) -> bool:
     return os.path.isdir(os.path.join(d, "Pool")) and os.path.isdir(os.path.join(d, "Config"))

{hbkit-0.4.2 → hbkit-0.4.4}/src/hbkit/cli.py

@@ -87,12 +87,17 @@ def all_files(db):
     """)
 
 
+def resolve_password(password):
+    """-p wins, then HBK_PASSWORD. None means "prompt if the archive turns out to need it"."""
+    return password or os.environ.get("HBK_PASSWORD")
+
+
 def open_archive(archive, password=None):
     """Open, prompting for a password only if the archive turns out to be encrypted."""
     try:
         return hbk.Archive(archive, password=password)
     except hbk.NeedPassword:
-        pw = password or os.environ.get("HBK_PASSWORD")
+        pw = resolve_password(password)
         if not pw:
             if not sys.stdin.isatty():
                 sys.exit("archive is encrypted - pass --password or set HBK_PASSWORD")
@@ -226,7 +231,7 @@ def main() -> int:
 
     if cmd == "doctor":
         from . import doctor
-        r = doctor.diagnose(archive, password=password)
+        r = doctor.diagnose(archive, password=resolve_password(password))
         print(doctor.render(r))
         return 0 if (r.ok and not r.blockers) else 1
 

{hbkit-0.4.2 → hbkit-0.4.4}/src/hbkit/doctor.py

@@ -88,7 +88,7 @@ def diagnose(path: str, sample: int = 10, seed: int = 0, password: str | None =
     r.fact("virtual_file record", f"{arc.vf.record_size} B")
     r.fact("chunk_index record", f"{arc.ci.record_size} B "
                                  f"({'v3' if arc.ci.record_size == 29 else 'v1/v2'})")
-    r.fact("file_chunk shards", ", ".join(str(k) for k in sorted(arc.fc)))
+    r.fact("file_chunk shards", ", ".join(str(k) for k in arc.fc.known))
     r.check("virtual_file layout known", arc.vf.record_size == 56, f"{arc.vf.record_size} B")
     r.check("chunk_index layout known", arc.ci.record_size in (16, 29), f"{arc.ci.record_size} B")
 

{hbkit-0.4.2 → hbkit-0.4.4}/src/hbkit/runner.py

@@ -140,6 +140,12 @@ class Runner:
                 json.dump({"root": self.root, "dest": self.dest, "verify": self.verify,
                            "check": self.check, "strict": self.strict, "items": slice_}, fh)
             env = dict(os.environ)
+            # sys.executable is the *unwrapped* interpreter. Where hbkit reaches
+            # sys.path through a wrapper script rather than site-packages - a Nix
+            # build, a PYTHONPATH-based distro package - that interpreter cannot
+            # import hbkit, and every worker dies before extracting a byte. Hand
+            # the parent's own path down so the child resolves what the parent did.
+            env["PYTHONPATH"] = os.pathsep.join(p for p in sys.path if p)
             if self.password is not None:      # env, not the job file - no secret on disk
                 env["HBK_PASSWORD"] = self.password
             p = subprocess.Popen(

{hbkit-0.4.2 → hbkit-0.4.4}/tests/test_hbkit.py

@@ -415,3 +415,76 @@ def test_cache_dir_prefers_new_name_but_honours_the_old_one(tmp_path, monkeypatc
     assert hbki._default_cache().endswith("/.cache/hbk-recovery")   # legacy honoured
     (home / ".cache" / "hbkit").mkdir(parents=True)
     assert hbki._default_cache().endswith("/.cache/hbkit")          # new wins once present
+
+
+def test_runner_hands_its_import_path_to_workers(tmp_path, monkeypatch):
+    """Workers are spawned as `sys.executable -m hbkit.runner`, and sys.executable is the
+    *unwrapped* interpreter. Where hbkit reaches sys.path through a wrapper script instead
+    of site-packages - a Nix build, a PYTHONPATH-based distro package - that interpreter
+    cannot import hbkit and every worker dies before extracting a byte. Caught by packaging
+    for Nix, invisible to a pip install."""
+    import io
+
+    from hbkit import runner as hbk_runner
+
+    seen = {}
+
+    class FakePopen:
+        def __init__(self, cmd, env=None, **kw):
+            seen["cmd"], seen["env"] = cmd, env
+            self.stdout, self.stderr = io.StringIO(""), io.StringIO("")
+
+        def poll(self):
+            return 0
+
+        def wait(self):
+            return 0
+
+    monkeypatch.setattr(hbk_runner.subprocess, "Popen", FakePopen)
+    r = hbk_runner.Runner("/archive", str(tmp_path), [("/a", 0, 1, 0)], n_workers=1)
+    r.start()
+    r.cleanup()
+
+    assert seen["cmd"][1:3] == ["-m", "hbkit.runner"]
+    paths = seen["env"]["PYTHONPATH"].split(os.pathsep)
+    assert any(os.path.isdir(os.path.join(p, "hbkit")) for p in paths), \
+        f"no path in PYTHONPATH provides hbkit: {paths}"
+
+
+@pytest.mark.parametrize(("flag", "env", "want"), [
+    (None, "env", "env"),          # the bug: doctor ignored HBK_PASSWORD entirely
+    ("flag", "env", "flag"),       # an explicit -p still wins
+    (None, None, None),            # nothing set: leave it to the prompt
+])
+def test_doctor_honours_the_password_environment_variable(monkeypatch, flag, env, want):
+    """`-p` and HBK_PASSWORD are documented as equivalent, but doctor forwarded only `-p`,
+    so `HBK_PASSWORD=... hbk <archive> doctor` called a readable encrypted archive
+    unrecoverable."""
+    from hbkit import cli, doctor
+
+    seen = {}
+
+    class Result:
+        ok, blockers = True, []
+
+    monkeypatch.setattr(doctor, "diagnose",
+                        lambda a, password=None: (seen.update(pw=password), Result())[1])
+    monkeypatch.setattr(doctor, "render", lambda r: "")
+    monkeypatch.delenv("HBK_PASSWORD", raising=False)
+    if env:
+        monkeypatch.setenv("HBK_PASSWORD", env)
+
+    argv = ["hbk", "/some/archive", "doctor"] + (["-p", flag] if flag else [])
+    monkeypatch.setattr(sys, "argv", argv)
+    assert cli.main() == 0
+    assert seen["pw"] == want
+
+
+def test_lazycats_lists_shards_before_any_are_opened():
+    """`doctor` describes an archive without extracting from it, so nothing has opened a
+    file_chunk shard by the time it prints them. Iterating the dict reported an empty list
+    and made a healthy archive look malformed."""
+    cats = hbk._LazyCats({4: "/nonexistent/file_chunk4.index",
+                          0: "/nonexistent/file_chunk0.index"})
+    assert cats.known == [0, 4]
+    assert len(cats) == 0, "listing shards must not open them"