dupefind 0.2.0__tar.gz

dupefind-0.2.0/.gitattributes

@@ -0,0 +1 @@
+* text=auto eol=lf

dupefind-0.2.0/.github/FUNDING.yml

@@ -0,0 +1 @@
+buy_me_a_coffee: jlianesglrs

dupefind-0.2.0/.gitignore

@@ -0,0 +1,12 @@
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+build/
+dist/
+*.egg-info/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/

dupefind-0.2.0/CHANGELOG.md

@@ -0,0 +1,52 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
+[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.2.0]
+
+### Added
+
+- `scan` and `iter_groups` take several roots (directories or files) and return
+  typed results: `ScanReport`, `DuplicateGroup`, `DuplicateFile`, `ScanIssue`,
+  `ScanStats`, `Progress`.
+- `ScanOptions`: hash algorithm, partial hash size, size bounds, empty files,
+  include and exclude globs, link following and hard-link reporting.
+- Progress callback for the walk, partial and full stages.
+- Files that change while being read, unreadable files and missing paths are
+  reported as issues instead of being dropped silently.
+- `plan_actions`, `apply_actions` and `choose_keeper`: delete or hard link extra
+  copies with a keep policy (first, oldest, newest, shortest). Dry run by
+  default; every action re-checks size, identity and byte-for-byte equality,
+  stats both files again just before acting, moves the target aside and checks it
+  once more before removing it, and refuses links, junctions, read-only targets
+  and targets with extra NTFS data streams.
+- `report_to_json` and `report_to_dict` with a versioned schema.
+- The `dupefind` command line tool and `python -m dupefind`.
+- GUIDE.md.
+
+### Changed
+
+- Hard links to one file are no longer reported as duplicates of each other;
+  they are merged into one file and listed as `aliases`. Pass
+  `ScanOptions(hardlinks=True)` for the previous view.
+- Files no larger than the partial hash size are read once instead of twice.
+
+### Fixed
+
+- Windows junctions were walked even with `follow_symlinks=False`, so a junction
+  into the tree listed files as duplicates of themselves.
+- Overlapping or repeated roots no longer list a file twice.
+
+## [0.1.0]
+
+- First release.
+- `find_duplicates` returns duplicate files grouped by their full content hash.
+- `iter_duplicates` streams the same groups one at a time.
+- Staged matching compares files by size, then a partial hash, then a full hash,
+  so unique files are never read in full.
+- Read-only by design: the search reads file contents and never modifies, moves
+  or deletes anything on disk.

dupefind-0.2.0/CONTRIBUTING.md

@@ -0,0 +1,19 @@
+# Contributing
+
+Thanks for taking the time. This project keeps a small, predictable shape.
+
+## Setup
+
+```
+pip install -r requirements-dev.txt
+```
+
+## Before you open a pull request
+
+- Keep public API in `src/dupefind/__init__.py`; internals live in `_`-prefixed modules.
+- Add a test for the behaviour you change. Tests inject time, disk and network; they
+  never touch the real thing.
+- Run `scripts/check.ps1` (format, lint, types, tests) and make sure it passes.
+- One logical change per commit, imperative subject line.
+
+Open an issue first if the change is large or shifts the public API.

dupefind-0.2.0/GUIDE.md

@@ -0,0 +1,181 @@
+# dupefind guide
+
+## How a scan works
+
+`scan(*roots, options=None, on_progress=None)` returns a `ScanReport`.
+`iter_groups` takes the same arguments and streams `DuplicateGroup` objects
+instead. Both read files and never change them.
+
+1. **Walk.** Every root is used as given, even when it is a link. A root may be a
+   directory (walked depth first, entries in name order) or a single file. Links
+   met during the walk are skipped unless `follow_symlinks=True`. Directories are
+   recognised by their real location, so nothing is walked twice.
+2. **Size.** Files are grouped by byte size. A unique size cannot have a duplicate,
+   so such a file is never opened.
+3. **Identity.** Each same-size file is stat'ed again. Two paths that resolve to
+   the same location are one entry. Two names with the same device and file id are
+   hard links to one file: they are merged into one `DuplicateFile` whose
+   `aliases` lists the other names, unless `hardlinks=True`. If the size or
+   modification time differs from the walk, the file is reported as `changed`.
+4. **Partial hash.** The first `partial_size` bytes are hashed. A file no larger
+   than that is now fully hashed and is never read again.
+5. **Full hash.** Files still sharing a partial digest are hashed in full. A file
+   whose byte count differs from its size while it is read, or whose size or
+   modification time changed by the time its group is confirmed, is dropped and
+   reported as `changed`.
+
+Groups of larger files are yielded first.
+
+## Options
+
+| Field | Default | Meaning |
+| --- | --- | --- |
+| `algorithm` | `"sha256"` | Any fixed-size `hashlib` algorithm. The digest is of the whole file. |
+| `partial_size` | `4096` | Bytes hashed before any full read. Must be at least 1. |
+| `min_size` | `0` | Ignore files smaller than this. |
+| `max_size` | `None` | Ignore files larger than this. |
+| `include_empty` | `True` | Compare zero-length files (they all form one group). |
+| `include` | `()` | Globs; when given, only matching files are compared. |
+| `exclude` | `()` | Globs for files and directories to skip. |
+| `follow_symlinks` | `False` | Walk into symlinked directories and junctions. |
+| `hardlinks` | `False` | List every hard-link name as a separate copy. |
+
+Invalid values raise `ValueError` when the options are built (`TypeError` for a
+bare string given as a pattern list).
+
+Glob rules: a pattern without `/` matches the base name; a pattern with `/`
+matches the path relative to the root it was found under, written with `/`.
+Matching follows the platform case rules, so it is case-insensitive on Windows.
+
+Empty files: fdupes compares them by default, jdupes needs `-z`, and rmlint leaves
+them out. `dupefind` keeps its `0.1.0` behaviour and compares them; pass
+`include_empty=False` or `min_size=1` (`--no-empty` or `--min-size 1`) before any
+clean-up.
+
+## Results
+
+- `DuplicateGroup`: `digest`, `algorithm`, `size`, `files` (sorted by path, at
+  least two), `paths`, `wasted_bytes` (size times copies minus one).
+- `DuplicateFile`: `path`, `size`, `mtime_ns`, `aliases`.
+- `ScanIssue`: `path`, `kind` (`permission`, `missing`, `changed`, `error`),
+  `message`. The path is not part of any group.
+- `ScanStats`: `files_seen`, `candidates`, `partial_hashed`, `full_hashed`,
+  `bytes_read`, `groups`, `duplicate_files`, `wasted_bytes`.
+- `Progress`: `stage` (`walk`, `partial`, `full`), `done`, `total`, `bytes_read`.
+
+## Clean-up actions
+
+`plan_actions(groups, action="delete", keep="oldest")` returns `PlannedAction`
+items. One file per group survives with its aliases; every other file and every
+alias of it gets an action, so a delete frees the space.
+
+A regular file is always preferred as the survivor over a symlink (which can only
+appear with `follow_symlinks`), so the real data is what is kept.
+
+| `keep` | Survivor (ties broken by path) |
+| --- | --- |
+| `first` | first path in sort order |
+| `oldest` | oldest modification time |
+| `newest` | newest modification time |
+| `shortest` | shortest path |
+
+`apply_actions(plan, dry_run=True)` returns one `ActionResult` per item with
+`status` `dry-run`, `done`, `skipped` or `failed`. Every item, in a dry run too, is
+checked right before it would run:
+
+- the target, looked at without following links, is a regular file of its own:
+  a symlink, a junction or a directory found at the target path is refused, so
+  with `follow_symlinks` the symlinks in a group are reported but never acted on;
+- the target is writable (see read-only files below);
+- the kept file exists and is a regular file;
+- they are not the same file (same device and file id), so a hard link, junction
+  path, symlink or other spelling of the kept file is never removed;
+- both still have the planned size;
+- for `hardlink`, both are on the same device;
+- the two files are equal byte for byte;
+- the target is not a kept file anywhere in the plan.
+
+- on Windows, the target has no extra NTFS data streams (the download marker
+  `Zone.Identifier` is allowed), since only the main stream is compared and the
+  others would be lost.
+
+Then the act itself is guarded so the data always exists under some name:
+
+1. The kept file is opened and held open until the act is over. On Windows this
+   stops any other process deleting or renaming it in the meantime.
+2. Both paths are stat'ed again and must still show the same device, file id,
+   size, modification time and mode as the files that were just compared.
+3. The target is renamed aside to a random name next to it
+   (`.NAME.RANDOM.dupefind`) and that aside file is checked once more to be the
+   compared file, and the kept file to be unchanged. A target swapped for another
+   file or a link, or a kept file removed or rewritten, is refused and the aside
+   file is put back under its name without overwriting anything.
+4. `delete` then removes the aside file. `hardlink` creates a hard link to the real
+   file behind the kept path at the freed name (which fails if anything appeared
+   there), checks it is the kept file, and only then removes the aside file.
+
+A refusal never removes either file, so a group always keeps at least the kept
+copy. During step 4 the target name is briefly missing, but its data is still on
+disk under the aside name; if the process is killed there, that file remains. If
+putting a file back fails, the result says where it is. What remains is the few
+instructions between the last check of the kept file and the removal of the aside
+copy, during which, on POSIX only (no open-file lock there), another process could
+delete the kept file.
+
+Read-only files: a target that is read-only (the Windows read-only attribute, or
+no owner write bit or no write access on POSIX, such as mode 0444) is `skipped`
+with the reason `target is read-only`, for both actions, in a dry run too, and is
+left exactly as it was. Clear the flag first if you want it handled. A read-only
+kept file is fine: it is only read, and a hard link to it shares its attributes.
+If a hard link to a read-only kept file has to be undone, the flag is cleared just
+long enough to remove that extra name and is then put back. If the operating
+system refuses the act itself (for example a busy target, or a directory you
+cannot write to), the result is `failed` and the target is left in place.
+
+Windows allows at most 1023 extra hard links per file; beyond that the action
+fails and is reported.
+
+Only file contents are compared. Metadata that is not part of the content, such as
+permissions, owners and extended attributes, is not compared and goes with a
+removed or replaced copy.
+
+## JSON
+
+`report_to_json(report, actions=None, indent=2)` and `report_to_dict` produce:
+
+```json
+{
+  "schema": 1,
+  "groups": [
+    {"digest": "...", "algorithm": "sha256", "size": 6000, "wasted_bytes": 6000,
+     "files": [{"path": "a.jpg", "mtime_ns": 0, "aliases": []}]}
+  ],
+  "issues": [{"path": "...", "kind": "permission", "message": "..."}],
+  "stats": {"files_seen": 0, "candidates": 0, "partial_hashed": 0, "full_hashed": 0,
+            "bytes_read": 0, "groups": 0, "duplicate_files": 0, "wasted_bytes": 0},
+  "actions": null
+}
+```
+
+`actions` is a list of `{action, target, keep, size, status, reason}` when
+actions were planned. `schema` changes only if a field is removed or changes
+meaning.
+
+## Command line
+
+```
+dupefind PATH [PATH ...] [--json] [--algorithm NAME] [--partial-size N]
+         [--min-size N] [--max-size N] [--no-empty] [--include GLOB]...
+         [--exclude GLOB]... [--follow-symlinks] [--hardlinks]
+         [--delete | --hardlink] [--keep first|oldest|newest|shortest]
+         [--apply] [--empty] [-q]
+```
+
+With `--delete` or `--hardlink`, empty files are left out unless `--empty` is given
+(a plain report still lists them, unless `--no-empty`).
+
+Sizes accept `K`, `M`, `G` and `T` suffixes (powers of 1024). Text output lists
+each group as `# SIZE bytes x N files, ALGORITHM:DIGEST`, its paths (aliases
+indented with `=`), and a blank line. With `--delete` or `--hardlink` one line per
+action follows. Skipped paths and a summary go to stderr unless `-q`. Errors exit
+with status 1, as does any failed action. `python -m dupefind` works too.

dupefind-0.2.0/IMPROVEMENTS-round.md

@@ -0,0 +1,95 @@
+# dupefind improvement round
+
+Progress:
+- [x] Brainstorm ranked improvements (this file)
+- [x] Typed results, options and scan issues (`ScanOptions`, `DuplicateGroup`, `ScanReport`)
+- [x] Walker: hard-link identity, junctions and symlinks not followed, globs, size limits
+- [x] Staging: configurable algorithm and partial size, change detection, progress callback
+- [x] Safe actions: plan and apply delete or hardlink, dry run by default, keep policies
+- [x] JSON output and CLI (`dupefind PATH...`)
+- [x] Docs: README, GUIDE, CHANGELOG, docstrings
+- [x] Gates: ruff format, ruff check, mypy --strict, pytest
+- [x] Contracts, brute-force oracle on seeded trees, real wheel e2e
+- [x] Pull request opened (#2)
+- [x] Independent review findings fixed, guard clean
+- [x] Read-only targets refused, documented and tested (Windows attribute, POSIX 0444 in WSL)
+- [x] Swaps between scan and apply (TOCTOU) refused, with a last re-stat before acting, tested
+- [x] Second independent review (junction loops and roots, repeated, cased and nested roots) fixed
+- [x] Gates, guard and PR body refreshed
+
+Every item is additive. `find_duplicates` and `iter_duplicates` keep their names,
+their positional `root` argument, `follow_symlinks`, and their return types.
+
+## Starting point
+
+`0.1.0` grouped files by size, a 4 KiB SHA-256 prefix and a full SHA-256. It had no
+options beyond `follow_symlinks`, no way to see what was skipped, and two safety
+gaps found while reading the walker:
+
+- Hard links to one file were reported as duplicates of each other. Any tool that
+  deleted "all but one" of such a group would free no space at best.
+- On Windows a directory junction is not a symlink to `os.scandir`, so junctions
+  were walked even with `follow_symlinks=False`. A junction pointing back into the
+  tree made every file under it look like its own duplicate (the "same directory
+  given twice" data-loss caveat both fdupes and jdupes document).
+
+## Sources (fetched 2026-10-12)
+
+- fdupes man page: https://raw.githubusercontent.com/adrianlopezroche/fdupes/master/fdupes.1
+  (size, MD5, then byte-by-byte; `-H` "files that point to the same disk area are
+  treated as non-duplicates" by default; `-s` follows symlinked directories only on
+  request; `-G`/`-L` size bounds; caveat on listing a file as its own duplicate).
+- jdupes README: https://codeberg.org/jbruchon/jdupes/raw/branch/master/README.md
+  (linked files are non-duplicates "for safety" unless `-H`; zero-length files only
+  match with `-z`; the change check before acting on a file guards against TOCTTOU;
+  Windows allows at most 1023 hard links per file).
+- rmlint man page: https://raw.githubusercontent.com/sahib/rmlint/master/docs/rmlint.1.rst
+  (`--no-hardlinked` keeps one file of a hard-linked set; `-F` ignores symlinks;
+  `--size` excludes empty files by default; rank criteria such as oldest mtime and
+  path for the original to keep).
+
+## Ranked improvements
+
+1. **Hard-link identity.** Files sharing a device and file id (`st_dev`, `st_ino`,
+   which Python fills from the NTFS file id on Windows) are one file. By default a
+   hard-linked set is reported once, with the other names kept as `aliases`; a set
+   made only of links to one file is never a duplicate group. `hardlinks=True`
+   restores the old view (fdupes and jdupes `-H`).
+2. **Links are not followed by default, junctions included.** Symlinks and Windows
+   junctions (reparse tag `IO_REPARSE_TAG_MOUNT_POINT`) are skipped unless
+   `follow_symlinks=True`. Every path is also de-duplicated by its normalised real
+   path, so overlapping roots or a root given twice (also in another letter case on
+   a case-insensitive filesystem) never list a file as its own duplicate.
+3. **Typed results.** `DuplicateGroup` (digest, algorithm, size, `files`, `paths`,
+   `wasted_bytes`), `DuplicateFile` (path, size, mtime, aliases) and `ScanReport`
+   (groups, issues, stats). Frozen dataclasses, fully typed.
+4. **Issues instead of silent drops.** Permission errors, vanished files and files
+   that changed while being read are recorded as `ScanIssue(path, kind, message)`
+   and left out of the results. Change detection re-stats size and mtime after the
+   full hash, as jdupes does before acting.
+5. **`ScanOptions`.** `algorithm` (any fixed-size `hashlib` algorithm, default
+   `sha256` so keys are unchanged), `partial_size`, `min_size`, `max_size`,
+   `include_empty`, `include`/`exclude` globs, `follow_symlinks`, `hardlinks`.
+   Validated at construction with typed `ValueError`s.
+6. **Staging that reads less.** A file no larger than the partial size is hashed
+   once and its partial digest is its full digest, so it is never read twice.
+7. **Multiple roots, streaming and progress.** `scan(*roots)` and
+   `iter_groups(*roots)` take several paths (directories or files). Groups stream
+   as they are confirmed. `on_progress` receives a `Progress(stage, done, total)`.
+8. **Safe actions.** `plan_actions(groups, action="delete"|"hardlink", keep=...)`
+   with keep policies `first`, `oldest`, `newest`, `shortest`; `apply_actions(plan,
+   dry_run=True)` acts only with `dry_run=False`. Before each action it re-checks
+   that both files exist, are not the same file, still have the scanned size, and
+   are byte-for-byte equal. Hard links are created next to the target and moved
+   over it atomically. Nothing is ever deleted without the explicit flag.
+9. **JSON output.** `report_to_json(report)` with a versioned, documented schema.
+10. **CLI.** `dupefind PATH...` prints fdupes-style groups, `--json`, the scan
+    options, and `--delete`/`--hardlink` that only print a plan unless `--apply`.
+11. **Docs.** Docstrings on every public name, a GUIDE covering policies, actions,
+    JSON schema and the CLI, and a CHANGELOG entry.
+
+## Out of scope
+
+Byte-for-byte confirmation during the scan itself (hash equality is the scan
+verdict; actions do the byte comparison), a hash cache database, and fuzzy
+matching of media files.

dupefind-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Javier Lianes
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

dupefind-0.2.0/PKG-INFO

@@ -0,0 +1,154 @@
+Metadata-Version: 2.5
+Name: dupefind
+Version: 0.2.0
+Summary: Find duplicate files by staged hashing: group by size, then partial hash, then full hash
+Project-URL: Homepage, https://lianes8studio.com
+Project-URL: Repository, https://github.com/JaviLianes8/dupefind
+Project-URL: Issues, https://github.com/JaviLianes8/dupefind/issues
+Project-URL: Changelog, https://github.com/JaviLianes8/dupefind/blob/main/CHANGELOG.md
+Project-URL: LinkedIn, https://www.linkedin.com/in/jlianes/
+Project-URL: Funding, https://buymeacoffee.com/jlianesglrs
+Author-email: Javier Lianes <javier@lianes8studio.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: cli,dedupe,duplicates,fdupes,files,filesystem,hardlink,hash
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: System :: Filesystems
+Classifier: Topic :: Utilities
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Provides-Extra: dev
+Requires-Dist: build; extra == 'dev'
+Requires-Dist: mypy; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Requires-Dist: twine; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# dupefind
+
+Find duplicate files by staged hashing: group by size, then partial hash, then full hash.
+
+A small, fully typed library with no dependencies (standard `hashlib` only) and a
+command line tool. Scanning only reads files. Clean-up is a separate, explicit step
+that is a dry run unless you say otherwise.
+
+## Install
+
+```
+pip install dupefind
+```
+
+## Quick start
+
+```python
+from dupefind import scan
+
+report = scan("photos/import", "photos/library")
+for group in report.groups:
+    print(group.size, group.digest)
+    for path in group.paths:
+        print("  ", path)
+for issue in report.issues:  # unreadable, vanished or changed while reading
+    print("skipped", issue.path, issue.kind)
+```
+
+The original interface still works and returns `dict[str, list[Path]]` keyed by
+the SHA-256 of the content:
+
+```python
+from dupefind import find_duplicates, iter_duplicates
+
+groups = find_duplicates("path/to/tree")
+for digest, paths in iter_duplicates("path/to/tree"):
+    ...
+```
+
+## Options
+
+```python
+from dupefind import ScanOptions, iter_groups
+
+options = ScanOptions(
+    algorithm="blake2b",  # any fixed-size hashlib algorithm, default sha256
+    partial_size=64 * 1024,  # bytes hashed before any full read
+    min_size=1,  # leave empty files alone
+    exclude=(".git", "*.tmp"),
+)
+for group in iter_groups("data", options=options, on_progress=print):
+    ...
+```
+
+Safe defaults, in line with fdupes, jdupes and rmlint:
+
+- **Hard links are one file.** Several names of the same file are merged and kept
+  in `aliases`; a set made only of hard links is never a duplicate group. Set
+  `hardlinks=True` to list every name.
+- **Links are not followed.** Symlinks and Windows junctions found while walking
+  are skipped unless `follow_symlinks=True`. Paths you pass in are always used.
+- **A file is never its own duplicate.** Overlapping roots, a root given twice, a
+  root in another letter case, or a link loop never list one file twice.
+- **Changes are caught.** A file whose size or modification time changes while it
+  is read is left out and reported.
+- **Errors are reported, not raised.** Permission errors and vanished files become
+  `ScanIssue` entries.
+
+## Cleaning up
+
+```python
+from dupefind import apply_actions, plan_actions, scan
+
+plan = plan_actions(scan("photos").groups, action="delete", keep="oldest")
+for result in apply_actions(plan):  # dry run: nothing changes
+    print(result.status, result.planned.target)
+apply_actions(plan, dry_run=False)  # now it happens
+```
+
+`keep` is `first`, `oldest`, `newest` or `shortest`. `action="hardlink"` replaces
+each copy with a hard link to the kept file instead. Right before acting, every
+pair is checked again: both files exist, are not the same file, have the scanned
+size, and are equal byte for byte; then both are stat'ed once more just before the
+act, the target is moved aside and checked again, and the kept file is held open.
+Links, junctions, read-only files and (on Windows) files with extra data streams
+are never removed or replaced. Anything else is skipped and reported.
+
+## Command line
+
+```
+dupefind ~/Pictures                       # list duplicate groups
+dupefind a b --min-size 1M --json         # two trees, JSON report
+dupefind . --exclude .git --exclude "*.tmp"
+dupefind . --delete --keep oldest         # show the plan (dry run)
+dupefind . --delete --keep oldest --apply # really delete the other copies
+dupefind . --hardlink --apply             # link copies to one file
+```
+
+Nothing changes unless `--apply` is given with `--delete` or `--hardlink`. Those
+two leave empty files alone unless `--empty` is added. A summary and any skipped paths go to stderr. Exit status is 0 on success and 1 on
+an error or a failed action.
+
+See [GUIDE.md](GUIDE.md) for the staging, every option, the JSON schema and the
+safety checks in detail.
+
+## Why
+
+Finding duplicate files is a common chore, but the Python packages for it are
+mostly stale: `fastdupes` last shipped in 2015, `dupfinder` in 2010, and `dupe` in
+2021. The few that are maintained are command-line tools, not a typed library you
+can import. `dupefind` is both: a typed API with safe link handling and explicit,
+verified clean-up, and a CLI for the shell.
+
+## License
+
+MIT. See [LICENSE](LICENSE).
+
+## Links
+
+- Website: https://lianes8studio.com
+- GitHub: https://github.com/JaviLianes8
+- LinkedIn: https://www.linkedin.com/in/jlianes/

dupefind-0.2.0/README.md

@@ -0,0 +1,122 @@
+# dupefind
+
+Find duplicate files by staged hashing: group by size, then partial hash, then full hash.
+
+A small, fully typed library with no dependencies (standard `hashlib` only) and a
+command line tool. Scanning only reads files. Clean-up is a separate, explicit step
+that is a dry run unless you say otherwise.
+
+## Install
+
+```
+pip install dupefind
+```
+
+## Quick start
+
+```python
+from dupefind import scan
+
+report = scan("photos/import", "photos/library")
+for group in report.groups:
+    print(group.size, group.digest)
+    for path in group.paths:
+        print("  ", path)
+for issue in report.issues:  # unreadable, vanished or changed while reading
+    print("skipped", issue.path, issue.kind)
+```
+
+The original interface still works and returns `dict[str, list[Path]]` keyed by
+the SHA-256 of the content:
+
+```python
+from dupefind import find_duplicates, iter_duplicates
+
+groups = find_duplicates("path/to/tree")
+for digest, paths in iter_duplicates("path/to/tree"):
+    ...
+```
+
+## Options
+
+```python
+from dupefind import ScanOptions, iter_groups
+
+options = ScanOptions(
+    algorithm="blake2b",  # any fixed-size hashlib algorithm, default sha256
+    partial_size=64 * 1024,  # bytes hashed before any full read
+    min_size=1,  # leave empty files alone
+    exclude=(".git", "*.tmp"),
+)
+for group in iter_groups("data", options=options, on_progress=print):
+    ...
+```
+
+Safe defaults, in line with fdupes, jdupes and rmlint:
+
+- **Hard links are one file.** Several names of the same file are merged and kept
+  in `aliases`; a set made only of hard links is never a duplicate group. Set
+  `hardlinks=True` to list every name.
+- **Links are not followed.** Symlinks and Windows junctions found while walking
+  are skipped unless `follow_symlinks=True`. Paths you pass in are always used.
+- **A file is never its own duplicate.** Overlapping roots, a root given twice, a
+  root in another letter case, or a link loop never list one file twice.
+- **Changes are caught.** A file whose size or modification time changes while it
+  is read is left out and reported.
+- **Errors are reported, not raised.** Permission errors and vanished files become
+  `ScanIssue` entries.
+
+## Cleaning up
+
+```python
+from dupefind import apply_actions, plan_actions, scan
+
+plan = plan_actions(scan("photos").groups, action="delete", keep="oldest")
+for result in apply_actions(plan):  # dry run: nothing changes
+    print(result.status, result.planned.target)
+apply_actions(plan, dry_run=False)  # now it happens
+```
+
+`keep` is `first`, `oldest`, `newest` or `shortest`. `action="hardlink"` replaces
+each copy with a hard link to the kept file instead. Right before acting, every
+pair is checked again: both files exist, are not the same file, have the scanned
+size, and are equal byte for byte; then both are stat'ed once more just before the
+act, the target is moved aside and checked again, and the kept file is held open.
+Links, junctions, read-only files and (on Windows) files with extra data streams
+are never removed or replaced. Anything else is skipped and reported.
+
+## Command line
+
+```
+dupefind ~/Pictures                       # list duplicate groups
+dupefind a b --min-size 1M --json         # two trees, JSON report
+dupefind . --exclude .git --exclude "*.tmp"
+dupefind . --delete --keep oldest         # show the plan (dry run)
+dupefind . --delete --keep oldest --apply # really delete the other copies
+dupefind . --hardlink --apply             # link copies to one file
+```
+
+Nothing changes unless `--apply` is given with `--delete` or `--hardlink`. Those
+two leave empty files alone unless `--empty` is added. A summary and any skipped paths go to stderr. Exit status is 0 on success and 1 on
+an error or a failed action.
+
+See [GUIDE.md](GUIDE.md) for the staging, every option, the JSON schema and the
+safety checks in detail.
+
+## Why
+
+Finding duplicate files is a common chore, but the Python packages for it are
+mostly stale: `fastdupes` last shipped in 2015, `dupfinder` in 2010, and `dupe` in
+2021. The few that are maintained are command-line tools, not a typed library you
+can import. `dupefind` is both: a typed API with safe link handling and explicit,
+verified clean-up, and a CLI for the shell.
+
+## License
+
+MIT. See [LICENSE](LICENSE).
+
+## Links
+
+- Website: https://lianes8studio.com
+- GitHub: https://github.com/JaviLianes8
+- LinkedIn: https://www.linkedin.com/in/jlianes/