fs-explorer 0.1.0__tar.gz

fs_explorer-0.1.0/.gitattributes

@@ -0,0 +1,4 @@
+# Fixtures are binary images and exact byte streams: never convert line endings.
+tests/fixtures/** -text -diff
+*.py text eol=lf
+*.sh text eol=lf

fs_explorer-0.1.0/.gitignore

@@ -0,0 +1,23 @@
+# Python artifacts
+__pycache__/
+*.py[cod]
+build/
+dist/
+*.egg-info/
+.venv/
+
+# Tool caches
+.mypy_cache/
+.ruff_cache/
+.pytest_cache/
+.coverage
+htmlcov/
+
+# OS files
+.DS_Store
+
+# Local data
+samples/
+tests/fixtures/_build/
+extract-out/
+/local

fs_explorer-0.1.0/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, Arthur Gouhier
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

fs_explorer-0.1.0/PKG-INFO

@@ -0,0 +1,290 @@
+Metadata-Version: 2.5
+Name: fs_explorer
+Version: 0.1.0
+Summary: List and read disk images, filesystems, and archives using only the Python standard library
+Author-email: Arthur Gouhier <ajgouhier@gmail.com>
+License-Expression: BSD-3-Clause
+License-File: LICENSE
+Requires-Python: >=3.10
+Provides-Extra: dev
+Requires-Dist: mypy; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# fs-explorer
+
+List and read the contents of disk images, filesystems, and archives using only the Python standard
+library.
+
+```
+pip install fs-explorer        # distribution
+import fs_explorer             # package
+fs-explorer ls disk.dmg        # command (also: python -m fs_explorer)
+```
+
+## Guarantees
+
+- **No dependencies.** Python 3.10+, standard library only. System libraries (`libzstd`, `liblz4`,
+  `liblzo2`, `libcrypto`, Apple's `libcompression`/CommonCrypto) are used through `ctypes` when
+  present, always with a pure-Python fallback.
+- **Listing reads metadata only.** Directory records, inode tables, B-tree nodes, central directories,
+  and headers. File data is not read. Formats that can only be listed by decompressing everything
+  (e.g. `.tar.gz`) are detected and refused with an explicit reason, unless you opt in with
+  `full_read_limit`.
+- **Reading touches only what the file needs.** Only the blocks, chunks, extents, or solid-block prefix
+  that the requested file depends on are read and decoded.
+- **Streaming.** Entries are yielded as they are parsed. Memory is bounded by the largest single
+  directory plus the cache budget (default 64 MiB), not by image size.
+- **Read-only.** No mounting, no subprocesses, no writes to the source. `extract()` writes only under its
+  destination directory.
+
+## Supported formats
+
+### Containers and partition maps
+
+| Format | Notes | What is read to list |
+|---|---|---|
+| Raw images (`.img`, `.iso`, `.bin`) | detection by content | superblocks |
+| UDIF / DMG | raw, zero, zlib, bzip2, LZMA, ADC, LZFSE chunks | koly trailer + plist chunk table |
+| Encrypted DMG v1 / v2 | AES-128/256, password | header, key blobs; data decrypted per 4 KiB chunk |
+| `.sparseimage`, `.sparsebundle` | bands read lazily; missing bands are zeros | header / `Info.plist` |
+| VHD (fixed, dynamic, differencing), VHDX (incl. log replay, differencing) | parents via sibling files | footer/headers, BAT |
+| VMDK (monolithicSparse, streamOptimized, split, flat, snapshots) | | grain directory/tables |
+| QCOW2 (v2/v3, compressed deflate/zstd, backing files, extended L2) | | L1/L2 tables |
+| VDI, Android sparse, BIN/CUE (Mode 1, Mode 2 Form 1) | | block map / chunk list / cue sheet |
+| GPT, APM, MBR with EBR chains | volumes named `p1`…`pN` | partition tables |
+
+### Filesystems
+
+| Filesystem | Reading extras |
+|---|---|
+| HFS+ / HFSX | decmpfs compression (zlib, LZVN, LZFSE), hard links, resource forks, xattrs |
+| APFS | decmpfs, clones, sparse files, **native encryption** (password or recovery key); volumes `v1`…`vN` |
+| ISO 9660 + Joliet + Rock Ridge | multi-extent files, zisofs |
+| UDF (1.02–2.60, sparable and metadata partitions) | embedded data, all allocation descriptor kinds |
+| FAT12/16/32, exFAT | long names, NoFatChain |
+| NTFS | LZNT1 compression, sparse files, named streams, reparse symlinks/junctions, WOF (XPRESS/LZX) |
+| ext2/3/4 | extents, indirect blocks, inline data, holes, xattrs |
+| SquashFS 4 | gzip, lzma, lzo, xz, lz4, zstd; fragments; xattrs |
+| Btrfs (single, DUP, RAID1, RAID1C3/4) | subvolumes, compressed extents (zlib, zstd, LZO), holes |
+
+### Archives and streams
+
+| Format | Listing | Reading |
+|---|---|---|
+| zip (Zip64) | central directory only | stored, deflate, Deflate64, bzip2, LZMA, xz, zstd, PPMd; ZipCrypto, WinZip AES |
+| tar (ustar, GNU, PAX, sparse) | headers only | direct views; compressed tar readable forward-only |
+| 7z | header (incl. encrypted header) | LZMA, LZMA2, BCJ/ARM/PPC/SPARC/IA64, Delta, BCJ2, bzip2, Deflate, Deflate64, PPMd, 7zAES |
+| RAR5, RAR4 | headers (incl. encrypted headers) | **stored members only** (with decryption); compressed members raise `MissingCodecError` |
+| xar and `.pkg` (via Bom) | TOC + Bom `Paths` tree | zlib/bzip2/xz members; Bom-backed payloads through pbzx + cpio |
+| cpio (newc, crc, odc, binary), ar / `.deb`, RPM, CAB | headers | CAB: stored, MSZIP, LZX; RPM: payload (gzip, bzip2, xz, zstd) forward-only |
+| gzip, xz, zstd, bzip2, lz4 streams | one entry: original name and size when stored | decompressed contents |
+
+### Codec tiers
+
+| Tier | Codecs | Implementation |
+|---|---|---|
+| core | zlib/deflate, bzip2, LZMA/xz, BCJ, Delta | standard library |
+| core | zstd | `compression.zstd` (3.14+) → `libzstd` → pure Python |
+| core | LZFSE, LZVN, ADC, LZ4, pbzx, CRC32C | `libcompression`/`liblz4` where available → pure Python |
+| common | LZO1X, LZNT1, MSZIP, BCJ2, Deflate64, zisofs, ZipCrypto, WinZip AES | `liblzo2` → pure Python |
+| heavy | PPMd H and I, LZX, XPRESS Huffman | pure Python (slow: PPMd ≈ 0.1 MB/s) |
+| crypto | AES (ECB, CBC, CTR, XTS), 3DES, RFC 3394, PBKDF2, 7z/RAR KDFs | `libcrypto` / CommonCrypto → pure Python |
+
+### Refused or out of scope
+
+- **Refused for listing** (detected; allowed with `full_read_limit`): compressed tar and cpio, `.deb` data
+  payloads, `.pkg` payloads without a Bom, disk images wrapped in a compression stream (`.img.xz`).
+  Reading individual members still works by streaming.
+- **Refused for reading:** NTFS EFS, ext4 fscrypt, APFS per-file (hardware-bound) keys, QCOW2
+  encryption, VDI differencing images, RAR compressed members.
+- **Not detected:** NDIF, Disk Copy 4.2, E01, WIM/ESD, XFS, f2fs, multi-device Btrfs striping,
+  multi-volume archives, BitLocker, LUKS, StuffIt, plain (non-wrapped) HFS.
+
+## Command line
+
+```
+fs-explorer ls      [OPTIONS] SOURCE [TARGET]
+fs-explorer cat     [OPTIONS] SOURCE PATH
+fs-explorer extract [OPTIONS] SOURCE [PATH] -o DEST
+fs-explorer probe   [OPTIONS] SOURCE
+```
+
+```console
+$ fs-explorer probe Install.dmg
+UDIF (UDZO) > GPT (512-byte sectors) > p2 "Installer" > HFS+
+
+$ fs-explorer ls -l disk.img /p2/Users -r 0
+drwxr-xr-x      501       20              2024-05-01 10:12 /p2/Users/alice
+...
+
+$ fs-explorer ls -n 1 backup.tar /old          # descend into nested containers
+$ fs-explorer cat disk.vhdx /p1/Windows/win.ini
+$ fs-explorer cat --offset 1M --length 4096 image.qcow2 /p1/big.bin | xxd
+$ fs-explorer extract release.7z /docs -o out/ -v
+$ fs-explorer ls --json encrypted.dmg --ask-password
+$ fs-explorer ls outer.zip /disk.dmg/p1/Applications     # paths cross containers
+```
+
+Common options: `--password PW`, `--password-file PATH`, `--ask-password` (prompts once per
+encrypted layer, showing the APFS hint), `$FS_EXPLORER_PASSWORD`, `--format NAME`,
+`--backend auto|pure|system`, `--cache-size 64M`, `--full-read-limit SIZE`,
+`--errors raise|yield|skip` (default `yield`: errors go to stderr, listing continues), `--stats`.
+
+`ls`: `-r N` recursion depth (default unlimited, 0 = direct children), `-n N` nested container
+descent, `--no-stat`, `-i/-x PATTERN` include/exclude (excluded directories are pruned),
+`-T f,d,l,v` entry types, `-a` system files, `--no-flatten`, `--sort`, `-l`, `--json` (JSON Lines),
+`-0`, `--relative`.
+
+`cat`: `--offset`, `--length`, `--no-follow`, `--verify`.
+`extract`: `-o DEST`, `-r/-i/-x/-T`, `--preserve mode,mtime,owner,symlinks,hardlinks,xattrs,resource_forks`,
+`--on-conflict error|skip|overwrite|rename`, `--max-size`, `--max-ratio`, `--no-verify`, `--no-sparse`, `-v`.
+
+Exit codes: 0 success; 1 some entries had errors; 2 usage error; 3 unsupported format or missing
+codec; 4 missing or wrong password; 5 target not found; 6 unsafe path or limit exceeded.
+
+## Python API
+
+```python
+import fs_explorer as fx
+
+for e in fx.iter_entries("disk.dmg", target="/p2/Users", recursive=1):
+    print(e.path, e.type.name, e.size, e.mtime)
+
+data = fx.read_file("backup.7z", "/docs/report.pdf", password="secret")
+
+with fx.open_file("image.vhdx", "/p1/pagefile.sys") as f:   # io.RawIOBase
+    f.seek(1 << 30)
+    chunk = f.read(4096)
+
+for e in fx.extract("archive.zip", "/", "out/", on_conflict="rename"):
+    print("wrote", e.path)
+
+print(" > ".join(map(str, fx.probe("Install.dmg"))))
+
+# Reuse parsed layers across calls:
+with fx.open_source("disk.img", password=lambda req: ask_user(req.description, req.hint)) as src:
+    names = [e.name for e in src.entries(stat=False, recursive=0)]
+    blob = src.read_file("/p1/etc/hostname")
+    print(src.stats.as_dict())      # bytes read, cache hits, decompressed, decrypted, backends
+```
+
+`iter_entries(source, *, recursive=-1, target="/", password=None, nested=0, stat=True, include=(),
+exclude=(), types=None, format_hint=None, errors="raise", system_files=False, flatten=True,
+sort=False, full_read_limit=0, resolver=None, cache_size=64<<20, backend="auto")`.
+A source is a path, a binary file object (seekable), `bytes`, or a directory (`.sparsebundle`).
+
+`Entry` fields: `path`, `name`, `type` (`FILE`, `DIR`, `SYMLINK`, `HARDLINK`, `CHAR_DEVICE`,
+`BLOCK_DEVICE`, `FIFO`, `SOCKET`, `VOLUME`, `OTHER`), `depth`, `size`, `stored_size`, `mode`, `uid`,
+`gid`, `owner`, `group`, `mtime_ns`/`ctime_ns`/`atime_ns`/`btime_ns` (ns since the Unix epoch, UTC;
+`.mtime` etc. give `datetime`s), `nlink`, `inode`, `link_target`, `format`, `container`, `extra`,
+`error`. `entry.to_dict()` is JSON-ready.
+
+Errors derive from `FsExplorerError`: `UnsupportedFormatError`, `PasswordRequiredError`,
+`WrongPasswordError`, `CorruptDataError`, `MissingCodecError`, `TargetNotFoundError`,
+`AmbiguousTargetError`, `NotAFileError`, `ChecksumError`, `UnsafePathError`, `LimitExceededError`.
+
+## Paths, volumes, and depth
+
+- Paths are `/`-separated from the source root. Containers are unwrapped until a filesystem or archive
+  is reached. When a partition map, APFS container, or multi-volume layer has several listable
+  volumes, the root lists them as `VOLUME` entries with canonical names: `p1`…`pN` for partitions
+  (1-based, like `disk2s1`/`sda1`; MBR logical partitions start at `p5`) and `v1`…`vN` for APFS
+  volumes. Single-child layers are collapsed (`flatten=True`).
+- Labels (filesystem volume name, else the GPT/APM partition name) are in `extra["label"]` and are
+  accepted as aliases in targets: `/Macintosh HD/Users`. Exact match first, then case-insensitive;
+  canonical names always win; a label shared by several volumes raises `AmbiguousTargetError`.
+  Labels are read lazily (only for `stat=True`, i.e. `ls -l`/`--json`, and alias lookups).
+- Unrecognized partitions (swap, empty) appear as `VOLUME` entries with `error` set.
+- A path segment that names a container file descends into it: `/Downloads/x.dmg/p1/Applications`.
+  As a listing target, `/x.dmg` yields the file entry itself; `/x.dmg/` lists its contents.
+- Btrfs subvolumes appear as directories where they are linked; the root is the default subvolume.
+- `depth` is relative to the target (0 = direct children). Each directory, volume boundary, and
+  (with `nested > 0`) container boundary adds one level.
+- HFS+ names are converted from NFD to NFC (`keep_raw_names` keeps them); targets are normalized
+  before lookup. Symlinks inside images are followed within the image (loop limit 40); a link that
+  escapes the image root raises `TargetNotFoundError`.
+
+## Passwords and crypto
+
+`password` may be a `str`/`bytes`, or a callable receiving a `PasswordRequest(description, hint,
+attempt)` and returning a candidate or `None` to give up. Supported: encrypted DMG (v1/v2),
+APFS native encryption (user passwords and the recovery key; the stored hint is passed to the
+callable), 7z (content and header), RAR5 and RAR4 (stored members and encrypted headers), zip
+(ZipCrypto, WinZip AES-128/192/256).
+
+AES and 3DES use `libcrypto` (OpenSSL 1.1/3, including the copy shipped with CPython on Windows) or
+CommonCrypto on macOS when available; otherwise a pure-Python T-table AES (~1 MB/s). For encrypted
+APFS volumes and bulk reads from encrypted DMGs the native backend matters more than anything else;
+`--stats` shows which backend served each codec and cipher. Setting `FS_EXPLORER_NO_NATIVE=1` disables
+every system library, as if none were installed.
+
+## `full_read_limit` and spooling
+
+Some members have no random access: a DMG inside a deflated zip member, a zip inside a `.tar.gz`, an
+`.img.xz`. When such a member's stored size is within `full_read_limit`, it is spooled to a
+`SpooledTemporaryFile` and opened normally; otherwise listing inside it raises `LimitExceededError`
+or `UnsupportedFormatError` with the reason. The same limit enables listing formats that need full
+decompression (compressed tar/cpio). Reading a single member of a compressed tar never needs it.
+
+## Extraction safety
+
+- `..` components, absolute paths, drive letters, and NUL bytes are rejected (`UnsafePathError`);
+  member names that start with `/` are extracted relative to the destination.
+- Symlinks are created last, and a symlink whose target is absolute or resolves outside the
+  destination is refused; files are never written through a symlink, pre-existing or created during
+  the same extraction.
+- `max_size` (total bytes) and `max_ratio` (per member, default 1000×) are enforced while writing.
+- Case-insensitive collisions, Windows-invalid characters, and reserved names are handled
+  (`on_conflict`); holes are skipped with `seek` (`sparse=True`); checksums are verified by default.
+- Metadata: mode and mtime by default; `owner` (root only), `xattrs`, `resource_forks` (the named fork
+  on macOS, AppleDouble `._` files elsewhere), `symlinks` and `hardlinks` within the extraction set.
+- Solid blocks (7z, CAB folders, RAR solid, pbzx, RPM payloads) are processed in storage order, so
+  extraction is linear.
+
+## Performance notes
+
+Listing is dominated by B-tree walking and header parsing and typically runs at thousands to tens of
+thousands of entries per second. Reading throughput depends on the codec:
+
+| Path | Throughput (Python 3.11) |
+|---|---|
+| stdlib zlib, bzip2, xz; native zstd/lz4/lzo; libcrypto AES | hundreds of MB/s |
+| pure LZ4, LZO | 15–25 MB/s |
+| pure zstd, LZFSE, LZVN, Deflate64 | 8–15 MB/s |
+| pure LZX, XPRESS | 5–7 MB/s |
+| pure AES-CBC / XTS | ~1 MB/s |
+| pure PPMd | 0.03–0.15 MB/s |
+
+`tools/bench.py IMAGE --read-largest 3 --backend pure|system` measures entries/s and MB/s on your
+own images.
+
+## Limitations
+
+- RAR decompression is not implemented (stored members only).
+- APFS: live tree only (no snapshot selection); sealed volumes are untested.
+- HFS+: no journal replay; case-insensitive comparison approximates Apple's FastUnicodeCompare.
+- Btrfs: no log-tree replay, no RAID0/10/5/6, no seed devices or zoned mode.
+- UDF: VAT (virtual) partitions are refused.
+- FAT timestamps are local time and reported as if UTC.
+- Encrypted DMG handling was validated against images built by our fixture generator and reference
+  decryptors, not Apple-made images.
+
+## Development
+
+```
+pip install -e .[dev]
+pytest -q                      # ~950 tests, about a minute
+ruff check src tests tools
+mypy
+```
+
+Fixtures live in `tests/fixtures/` (xz-compressed images decompressed on first use into
+`tests/fixtures/_build/`, with JSON oracles recording paths, types, sizes, and SHA-256 of every
+file). `tools/make_fixtures_linux.sh` regenerates them from the standard tree
+(`tools/fixtures/tree.py`) with `mkfs.*`, `mksquashfs`, `xorriso`, `qemu-img`, `7z`, `rar`, `gcab`,
+`rpmbuild`, and Python builders for Apple formats (HFS+, APFS, UDIF, encrypted DMG, sparse images,
+pkg) that Linux tools cannot write. `tools/make_fixtures_macos.sh` builds Apple-native images with
+`hdiutil`. Some reference images come from the dfVFS project (Apache 2.0, see
+`tests/fixtures/dfvfs/NOTICE`).

fs_explorer-0.1.0/README.md

@@ -0,0 +1,276 @@
+# fs-explorer
+
+List and read the contents of disk images, filesystems, and archives using only the Python standard
+library.
+
+```
+pip install fs-explorer        # distribution
+import fs_explorer             # package
+fs-explorer ls disk.dmg        # command (also: python -m fs_explorer)
+```
+
+## Guarantees
+
+- **No dependencies.** Python 3.10+, standard library only. System libraries (`libzstd`, `liblz4`,
+  `liblzo2`, `libcrypto`, Apple's `libcompression`/CommonCrypto) are used through `ctypes` when
+  present, always with a pure-Python fallback.
+- **Listing reads metadata only.** Directory records, inode tables, B-tree nodes, central directories,
+  and headers. File data is not read. Formats that can only be listed by decompressing everything
+  (e.g. `.tar.gz`) are detected and refused with an explicit reason, unless you opt in with
+  `full_read_limit`.
+- **Reading touches only what the file needs.** Only the blocks, chunks, extents, or solid-block prefix
+  that the requested file depends on are read and decoded.
+- **Streaming.** Entries are yielded as they are parsed. Memory is bounded by the largest single
+  directory plus the cache budget (default 64 MiB), not by image size.
+- **Read-only.** No mounting, no subprocesses, no writes to the source. `extract()` writes only under its
+  destination directory.
+
+## Supported formats
+
+### Containers and partition maps
+
+| Format | Notes | What is read to list |
+|---|---|---|
+| Raw images (`.img`, `.iso`, `.bin`) | detection by content | superblocks |
+| UDIF / DMG | raw, zero, zlib, bzip2, LZMA, ADC, LZFSE chunks | koly trailer + plist chunk table |
+| Encrypted DMG v1 / v2 | AES-128/256, password | header, key blobs; data decrypted per 4 KiB chunk |
+| `.sparseimage`, `.sparsebundle` | bands read lazily; missing bands are zeros | header / `Info.plist` |
+| VHD (fixed, dynamic, differencing), VHDX (incl. log replay, differencing) | parents via sibling files | footer/headers, BAT |
+| VMDK (monolithicSparse, streamOptimized, split, flat, snapshots) | | grain directory/tables |
+| QCOW2 (v2/v3, compressed deflate/zstd, backing files, extended L2) | | L1/L2 tables |
+| VDI, Android sparse, BIN/CUE (Mode 1, Mode 2 Form 1) | | block map / chunk list / cue sheet |
+| GPT, APM, MBR with EBR chains | volumes named `p1`…`pN` | partition tables |
+
+### Filesystems
+
+| Filesystem | Reading extras |
+|---|---|
+| HFS+ / HFSX | decmpfs compression (zlib, LZVN, LZFSE), hard links, resource forks, xattrs |
+| APFS | decmpfs, clones, sparse files, **native encryption** (password or recovery key); volumes `v1`…`vN` |
+| ISO 9660 + Joliet + Rock Ridge | multi-extent files, zisofs |
+| UDF (1.02–2.60, sparable and metadata partitions) | embedded data, all allocation descriptor kinds |
+| FAT12/16/32, exFAT | long names, NoFatChain |
+| NTFS | LZNT1 compression, sparse files, named streams, reparse symlinks/junctions, WOF (XPRESS/LZX) |
+| ext2/3/4 | extents, indirect blocks, inline data, holes, xattrs |
+| SquashFS 4 | gzip, lzma, lzo, xz, lz4, zstd; fragments; xattrs |
+| Btrfs (single, DUP, RAID1, RAID1C3/4) | subvolumes, compressed extents (zlib, zstd, LZO), holes |
+
+### Archives and streams
+
+| Format | Listing | Reading |
+|---|---|---|
+| zip (Zip64) | central directory only | stored, deflate, Deflate64, bzip2, LZMA, xz, zstd, PPMd; ZipCrypto, WinZip AES |
+| tar (ustar, GNU, PAX, sparse) | headers only | direct views; compressed tar readable forward-only |
+| 7z | header (incl. encrypted header) | LZMA, LZMA2, BCJ/ARM/PPC/SPARC/IA64, Delta, BCJ2, bzip2, Deflate, Deflate64, PPMd, 7zAES |
+| RAR5, RAR4 | headers (incl. encrypted headers) | **stored members only** (with decryption); compressed members raise `MissingCodecError` |
+| xar and `.pkg` (via Bom) | TOC + Bom `Paths` tree | zlib/bzip2/xz members; Bom-backed payloads through pbzx + cpio |
+| cpio (newc, crc, odc, binary), ar / `.deb`, RPM, CAB | headers | CAB: stored, MSZIP, LZX; RPM: payload (gzip, bzip2, xz, zstd) forward-only |
+| gzip, xz, zstd, bzip2, lz4 streams | one entry: original name and size when stored | decompressed contents |
+
+### Codec tiers
+
+| Tier | Codecs | Implementation |
+|---|---|---|
+| core | zlib/deflate, bzip2, LZMA/xz, BCJ, Delta | standard library |
+| core | zstd | `compression.zstd` (3.14+) → `libzstd` → pure Python |
+| core | LZFSE, LZVN, ADC, LZ4, pbzx, CRC32C | `libcompression`/`liblz4` where available → pure Python |
+| common | LZO1X, LZNT1, MSZIP, BCJ2, Deflate64, zisofs, ZipCrypto, WinZip AES | `liblzo2` → pure Python |
+| heavy | PPMd H and I, LZX, XPRESS Huffman | pure Python (slow: PPMd ≈ 0.1 MB/s) |
+| crypto | AES (ECB, CBC, CTR, XTS), 3DES, RFC 3394, PBKDF2, 7z/RAR KDFs | `libcrypto` / CommonCrypto → pure Python |
+
+### Refused or out of scope
+
+- **Refused for listing** (detected; allowed with `full_read_limit`): compressed tar and cpio, `.deb` data
+  payloads, `.pkg` payloads without a Bom, disk images wrapped in a compression stream (`.img.xz`).
+  Reading individual members still works by streaming.
+- **Refused for reading:** NTFS EFS, ext4 fscrypt, APFS per-file (hardware-bound) keys, QCOW2
+  encryption, VDI differencing images, RAR compressed members.
+- **Not detected:** NDIF, Disk Copy 4.2, E01, WIM/ESD, XFS, f2fs, multi-device Btrfs striping,
+  multi-volume archives, BitLocker, LUKS, StuffIt, plain (non-wrapped) HFS.
+
+## Command line
+
+```
+fs-explorer ls      [OPTIONS] SOURCE [TARGET]
+fs-explorer cat     [OPTIONS] SOURCE PATH
+fs-explorer extract [OPTIONS] SOURCE [PATH] -o DEST
+fs-explorer probe   [OPTIONS] SOURCE
+```
+
+```console
+$ fs-explorer probe Install.dmg
+UDIF (UDZO) > GPT (512-byte sectors) > p2 "Installer" > HFS+
+
+$ fs-explorer ls -l disk.img /p2/Users -r 0
+drwxr-xr-x      501       20              2024-05-01 10:12 /p2/Users/alice
+...
+
+$ fs-explorer ls -n 1 backup.tar /old          # descend into nested containers
+$ fs-explorer cat disk.vhdx /p1/Windows/win.ini
+$ fs-explorer cat --offset 1M --length 4096 image.qcow2 /p1/big.bin | xxd
+$ fs-explorer extract release.7z /docs -o out/ -v
+$ fs-explorer ls --json encrypted.dmg --ask-password
+$ fs-explorer ls outer.zip /disk.dmg/p1/Applications     # paths cross containers
+```
+
+Common options: `--password PW`, `--password-file PATH`, `--ask-password` (prompts once per
+encrypted layer, showing the APFS hint), `$FS_EXPLORER_PASSWORD`, `--format NAME`,
+`--backend auto|pure|system`, `--cache-size 64M`, `--full-read-limit SIZE`,
+`--errors raise|yield|skip` (default `yield`: errors go to stderr, listing continues), `--stats`.
+
+`ls`: `-r N` recursion depth (default unlimited, 0 = direct children), `-n N` nested container
+descent, `--no-stat`, `-i/-x PATTERN` include/exclude (excluded directories are pruned),
+`-T f,d,l,v` entry types, `-a` system files, `--no-flatten`, `--sort`, `-l`, `--json` (JSON Lines),
+`-0`, `--relative`.
+
+`cat`: `--offset`, `--length`, `--no-follow`, `--verify`.
+`extract`: `-o DEST`, `-r/-i/-x/-T`, `--preserve mode,mtime,owner,symlinks,hardlinks,xattrs,resource_forks`,
+`--on-conflict error|skip|overwrite|rename`, `--max-size`, `--max-ratio`, `--no-verify`, `--no-sparse`, `-v`.
+
+Exit codes: 0 success; 1 some entries had errors; 2 usage error; 3 unsupported format or missing
+codec; 4 missing or wrong password; 5 target not found; 6 unsafe path or limit exceeded.
+
+## Python API
+
+```python
+import fs_explorer as fx
+
+for e in fx.iter_entries("disk.dmg", target="/p2/Users", recursive=1):
+    print(e.path, e.type.name, e.size, e.mtime)
+
+data = fx.read_file("backup.7z", "/docs/report.pdf", password="secret")
+
+with fx.open_file("image.vhdx", "/p1/pagefile.sys") as f:   # io.RawIOBase
+    f.seek(1 << 30)
+    chunk = f.read(4096)
+
+for e in fx.extract("archive.zip", "/", "out/", on_conflict="rename"):
+    print("wrote", e.path)
+
+print(" > ".join(map(str, fx.probe("Install.dmg"))))
+
+# Reuse parsed layers across calls:
+with fx.open_source("disk.img", password=lambda req: ask_user(req.description, req.hint)) as src:
+    names = [e.name for e in src.entries(stat=False, recursive=0)]
+    blob = src.read_file("/p1/etc/hostname")
+    print(src.stats.as_dict())      # bytes read, cache hits, decompressed, decrypted, backends
+```
+
+`iter_entries(source, *, recursive=-1, target="/", password=None, nested=0, stat=True, include=(),
+exclude=(), types=None, format_hint=None, errors="raise", system_files=False, flatten=True,
+sort=False, full_read_limit=0, resolver=None, cache_size=64<<20, backend="auto")`.
+A source is a path, a binary file object (seekable), `bytes`, or a directory (`.sparsebundle`).
+
+`Entry` fields: `path`, `name`, `type` (`FILE`, `DIR`, `SYMLINK`, `HARDLINK`, `CHAR_DEVICE`,
+`BLOCK_DEVICE`, `FIFO`, `SOCKET`, `VOLUME`, `OTHER`), `depth`, `size`, `stored_size`, `mode`, `uid`,
+`gid`, `owner`, `group`, `mtime_ns`/`ctime_ns`/`atime_ns`/`btime_ns` (ns since the Unix epoch, UTC;
+`.mtime` etc. give `datetime`s), `nlink`, `inode`, `link_target`, `format`, `container`, `extra`,
+`error`. `entry.to_dict()` is JSON-ready.
+
+Errors derive from `FsExplorerError`: `UnsupportedFormatError`, `PasswordRequiredError`,
+`WrongPasswordError`, `CorruptDataError`, `MissingCodecError`, `TargetNotFoundError`,
+`AmbiguousTargetError`, `NotAFileError`, `ChecksumError`, `UnsafePathError`, `LimitExceededError`.
+
+## Paths, volumes, and depth
+
+- Paths are `/`-separated from the source root. Containers are unwrapped until a filesystem or archive
+  is reached. When a partition map, APFS container, or multi-volume layer has several listable
+  volumes, the root lists them as `VOLUME` entries with canonical names: `p1`…`pN` for partitions
+  (1-based, like `disk2s1`/`sda1`; MBR logical partitions start at `p5`) and `v1`…`vN` for APFS
+  volumes. Single-child layers are collapsed (`flatten=True`).
+- Labels (filesystem volume name, else the GPT/APM partition name) are in `extra["label"]` and are
+  accepted as aliases in targets: `/Macintosh HD/Users`. Exact match first, then case-insensitive;
+  canonical names always win; a label shared by several volumes raises `AmbiguousTargetError`.
+  Labels are read lazily (only for `stat=True`, i.e. `ls -l`/`--json`, and alias lookups).
+- Unrecognized partitions (swap, empty) appear as `VOLUME` entries with `error` set.
+- A path segment that names a container file descends into it: `/Downloads/x.dmg/p1/Applications`.
+  As a listing target, `/x.dmg` yields the file entry itself; `/x.dmg/` lists its contents.
+- Btrfs subvolumes appear as directories where they are linked; the root is the default subvolume.
+- `depth` is relative to the target (0 = direct children). Each directory, volume boundary, and
+  (with `nested > 0`) container boundary adds one level.
+- HFS+ names are converted from NFD to NFC (`keep_raw_names` keeps them); targets are normalized
+  before lookup. Symlinks inside images are followed within the image (loop limit 40); a link that
+  escapes the image root raises `TargetNotFoundError`.
+
+## Passwords and crypto
+
+`password` may be a `str`/`bytes`, or a callable receiving a `PasswordRequest(description, hint,
+attempt)` and returning a candidate or `None` to give up. Supported: encrypted DMG (v1/v2),
+APFS native encryption (user passwords and the recovery key; the stored hint is passed to the
+callable), 7z (content and header), RAR5 and RAR4 (stored members and encrypted headers), zip
+(ZipCrypto, WinZip AES-128/192/256).
+
+AES and 3DES use `libcrypto` (OpenSSL 1.1/3, including the copy shipped with CPython on Windows) or
+CommonCrypto on macOS when available; otherwise a pure-Python T-table AES (~1 MB/s). For encrypted
+APFS volumes and bulk reads from encrypted DMGs the native backend matters more than anything else;
+`--stats` shows which backend served each codec and cipher. Setting `FS_EXPLORER_NO_NATIVE=1` disables
+every system library, as if none were installed.
+
+## `full_read_limit` and spooling
+
+Some members have no random access: a DMG inside a deflated zip member, a zip inside a `.tar.gz`, an
+`.img.xz`. When such a member's stored size is within `full_read_limit`, it is spooled to a
+`SpooledTemporaryFile` and opened normally; otherwise listing inside it raises `LimitExceededError`
+or `UnsupportedFormatError` with the reason. The same limit enables listing formats that need full
+decompression (compressed tar/cpio). Reading a single member of a compressed tar never needs it.
+
+## Extraction safety
+
+- `..` components, absolute paths, drive letters, and NUL bytes are rejected (`UnsafePathError`);
+  member names that start with `/` are extracted relative to the destination.
+- Symlinks are created last, and a symlink whose target is absolute or resolves outside the
+  destination is refused; files are never written through a symlink, pre-existing or created during
+  the same extraction.
+- `max_size` (total bytes) and `max_ratio` (per member, default 1000×) are enforced while writing.
+- Case-insensitive collisions, Windows-invalid characters, and reserved names are handled
+  (`on_conflict`); holes are skipped with `seek` (`sparse=True`); checksums are verified by default.
+- Metadata: mode and mtime by default; `owner` (root only), `xattrs`, `resource_forks` (the named fork
+  on macOS, AppleDouble `._` files elsewhere), `symlinks` and `hardlinks` within the extraction set.
+- Solid blocks (7z, CAB folders, RAR solid, pbzx, RPM payloads) are processed in storage order, so
+  extraction is linear.
+
+## Performance notes
+
+Listing is dominated by B-tree walking and header parsing and typically runs at thousands to tens of
+thousands of entries per second. Reading throughput depends on the codec:
+
+| Path | Throughput (Python 3.11) |
+|---|---|
+| stdlib zlib, bzip2, xz; native zstd/lz4/lzo; libcrypto AES | hundreds of MB/s |
+| pure LZ4, LZO | 15–25 MB/s |
+| pure zstd, LZFSE, LZVN, Deflate64 | 8–15 MB/s |
+| pure LZX, XPRESS | 5–7 MB/s |
+| pure AES-CBC / XTS | ~1 MB/s |
+| pure PPMd | 0.03–0.15 MB/s |
+
+`tools/bench.py IMAGE --read-largest 3 --backend pure|system` measures entries/s and MB/s on your
+own images.
+
+## Limitations
+
+- RAR decompression is not implemented (stored members only).
+- APFS: live tree only (no snapshot selection); sealed volumes are untested.
+- HFS+: no journal replay; case-insensitive comparison approximates Apple's FastUnicodeCompare.
+- Btrfs: no log-tree replay, no RAID0/10/5/6, no seed devices or zoned mode.
+- UDF: VAT (virtual) partitions are refused.
+- FAT timestamps are local time and reported as if UTC.
+- Encrypted DMG handling was validated against images built by our fixture generator and reference
+  decryptors, not Apple-made images.
+
+## Development
+
+```
+pip install -e .[dev]
+pytest -q                      # ~950 tests, about a minute
+ruff check src tests tools
+mypy
+```
+
+Fixtures live in `tests/fixtures/` (xz-compressed images decompressed on first use into
+`tests/fixtures/_build/`, with JSON oracles recording paths, types, sizes, and SHA-256 of every
+file). `tools/make_fixtures_linux.sh` regenerates them from the standard tree
+(`tools/fixtures/tree.py`) with `mkfs.*`, `mksquashfs`, `xorriso`, `qemu-img`, `7z`, `rar`, `gcab`,
+`rpmbuild`, and Python builders for Apple formats (HFS+, APFS, UDIF, encrypted DMG, sparse images,
+pkg) that Linux tools cannot write. `tools/make_fixtures_macos.sh` builds Apple-native images with
+`hdiutil`. Some reference images come from the dfVFS project (Apache 2.0, see
+`tests/fixtures/dfvfs/NOTICE`).