duplicate-cleaner 0.1.0__tar.gz

duplicate_cleaner-0.1.0/PKG-INFO

@@ -0,0 +1,110 @@
+Metadata-Version: 2.3
+Name: duplicate-cleaner
+Version: 0.1.0
+Summary: Cleans specified directories from duplicates based on hash comparison
+Author: Konstantin Kovalenko
+Author-email: Konstantin Kovalenko <konstantin.l.kovalenko@gmail.com>
+License: MIT
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Environment :: Console
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/despy49/duplicate-cleaner
+Project-URL: Repository, https://github.com/despy49/duplicate-cleaner
+Description-Content-Type: text/markdown
+
+# 🚀 Duplicate File Cleaner (with 100% Test Coverage)
+
+A fast and secure Python command-line utility (CLI) for finding and removing duplicate files. This project is built with strict type annotations, system error handling, and full test coverage.
+
+## 🌟 Features
+
+* **100% Code Coverage:** Every part of the application, including empty directory cleanup, interactive choices, and edge cases, is tested.
+* **Isolated Testing:** Robust handling of system exceptions such as symlink loops (`ELOOP`) and permission errors (`EACCES`) via  mocking.
+* **Scan workflow:** Identifies duplicate files efficiently by grouping them by size first, then calculating SHA-256 chunked hashes for verification.
+
+---
+
+## 🛠 Stack & Tools
+
+* **Language:** Python 3.12+ (with strict `typing` annotations)
+* **Environment & Package Manager:** `uv`
+* **Testing Framework:** `pytest`
+* **Coverage Tool:** `pytest-cov`
+* **Reporting:** `allure-pytest`
+
+---
+
+## 📦 Code Coverage Metrics
+
+Run the coverage tool directly using the customized default test runner. The project has zero coverage gaps:
+
+| File | Stmts | Miss | Branch | BrPart | Cover |
+| :--- | :---: | :---: | :---: | :---: | :---: |
+| `src/duplicate_cleaner/__init__.py` | 8 | 0 | 0 | 0 | **100.0%** |
+| `src/duplicate_cleaner/duplicate_cleaner.py` | 151 | 0 | 60 | 0 | **100.0%** |
+| **TOTAL** | **159** | **0** | **60** | **0** | **100.0%** |
+
+### Verified Edge Cases:
+1. **Circular Symlinks (`ELOOP`):** Validated protection loops against recursive symlinks during depth detection and link flattening.
+2. **Relative Symlink Targets:** Correct resolution of target paths when symlinks point to relative locations (`../`).
+3. **Permission Errors (`EACCES`):** Ensures locked or system files are safely skipped and logged without crashing the tool.
+4. **Interactive Prompts and Flags:** Complete parameter mapping for CLI combinations like `--no-confirm`, `--remove-empty`, and user inputs (`y/N`).
+
+---
+
+## 🚀 Quick Start
+
+### 1. Installation and Setup
+Project dependencies are managed via `uv`. Clone the repository and run the setup command in the project root:
+```bash
+uv sync
+```
+*This command creates an isolated virtual environment and automatically installs all dependencies.*
+
+### 2. Running the Utility
+Execute the application entry point using `uv run`:
+```bash
+# Basic run (scans user home directory by default)
+uv run duplicate-cleaner
+
+# Scan a specific directory recursively with auto-confirmation and empty directory cleanup
+uv run duplicate-cleaner --target . --recurse --no-confirm --remove-empty
+```
+
+---
+
+## 🧪 Testing & Reporting
+
+Thanks to pre-configured `addopts` project settings, short logs, test execution coverage metrics collection, and Allure report indexing run automatically.
+
+### Run Tests and Collect Coverage
+```bash
+uv run pytest
+```
+*This instantly executes tests, renders an HTML report in `htmlcov/`, and updates text metrics.*
+
+### Generate Allure Reports
+Build and view the dashboard in your default browser (requires a local installation of `allure-cli`):
+```bash
+allure serve allure-results
+```
+
+---
+
+## ⚙️ Code Quality & Linters
+
+The codebase complies with modern standard formatting guidelines. Code styles are validated with **Ruff**:
+```bash
+# Check code style, formatting, and imports
+uv run ruff check
+
+# Automatically format the code to PEP 8 standards
+uv run ruff format
+```
+
+Run static type checking using **mypy**:
+```bash
+uv run mypy src/ tests/
+```

duplicate_cleaner-0.1.0/README.md

@@ -0,0 +1,94 @@
+# 🚀 Duplicate File Cleaner (with 100% Test Coverage)
+
+A fast and secure Python command-line utility (CLI) for finding and removing duplicate files. This project is built with strict type annotations, system error handling, and full test coverage.
+
+## 🌟 Features
+
+* **100% Code Coverage:** Every part of the application, including empty directory cleanup, interactive choices, and edge cases, is tested.
+* **Isolated Testing:** Robust handling of system exceptions such as symlink loops (`ELOOP`) and permission errors (`EACCES`) via  mocking.
+* **Scan workflow:** Identifies duplicate files efficiently by grouping them by size first, then calculating SHA-256 chunked hashes for verification.
+
+---
+
+## 🛠 Stack & Tools
+
+* **Language:** Python 3.12+ (with strict `typing` annotations)
+* **Environment & Package Manager:** `uv`
+* **Testing Framework:** `pytest`
+* **Coverage Tool:** `pytest-cov`
+* **Reporting:** `allure-pytest`
+
+---
+
+## 📦 Code Coverage Metrics
+
+Run the coverage tool directly using the customized default test runner. The project has zero coverage gaps:
+
+| File | Stmts | Miss | Branch | BrPart | Cover |
+| :--- | :---: | :---: | :---: | :---: | :---: |
+| `src/duplicate_cleaner/__init__.py` | 8 | 0 | 0 | 0 | **100.0%** |
+| `src/duplicate_cleaner/duplicate_cleaner.py` | 151 | 0 | 60 | 0 | **100.0%** |
+| **TOTAL** | **159** | **0** | **60** | **0** | **100.0%** |
+
+### Verified Edge Cases:
+1. **Circular Symlinks (`ELOOP`):** Validated protection loops against recursive symlinks during depth detection and link flattening.
+2. **Relative Symlink Targets:** Correct resolution of target paths when symlinks point to relative locations (`../`).
+3. **Permission Errors (`EACCES`):** Ensures locked or system files are safely skipped and logged without crashing the tool.
+4. **Interactive Prompts and Flags:** Complete parameter mapping for CLI combinations like `--no-confirm`, `--remove-empty`, and user inputs (`y/N`).
+
+---
+
+## 🚀 Quick Start
+
+### 1. Installation and Setup
+Project dependencies are managed via `uv`. Clone the repository and run the setup command in the project root:
+```bash
+uv sync
+```
+*This command creates an isolated virtual environment and automatically installs all dependencies.*
+
+### 2. Running the Utility
+Execute the application entry point using `uv run`:
+```bash
+# Basic run (scans user home directory by default)
+uv run duplicate-cleaner
+
+# Scan a specific directory recursively with auto-confirmation and empty directory cleanup
+uv run duplicate-cleaner --target . --recurse --no-confirm --remove-empty
+```
+
+---
+
+## 🧪 Testing & Reporting
+
+Thanks to pre-configured `addopts` project settings, short logs, test execution coverage metrics collection, and Allure report indexing run automatically.
+
+### Run Tests and Collect Coverage
+```bash
+uv run pytest
+```
+*This instantly executes tests, renders an HTML report in `htmlcov/`, and updates text metrics.*
+
+### Generate Allure Reports
+Build and view the dashboard in your default browser (requires a local installation of `allure-cli`):
+```bash
+allure serve allure-results
+```
+
+---
+
+## ⚙️ Code Quality & Linters
+
+The codebase complies with modern standard formatting guidelines. Code styles are validated with **Ruff**:
+```bash
+# Check code style, formatting, and imports
+uv run ruff check
+
+# Automatically format the code to PEP 8 standards
+uv run ruff format
+```
+
+Run static type checking using **mypy**:
+```bash
+uv run mypy src/ tests/
+```

duplicate_cleaner-0.1.0/pyproject.toml

@@ -0,0 +1,75 @@
+[project]
+name = "duplicate-cleaner"
+version = "0.1.0"
+description = "Cleans specified directories from duplicates based on hash comparison"
+readme = "README.md"
+requires-python = ">=3.12"
+classifiers = [
+    "Programming Language :: Python :: 3.12",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Environment :: Console",
+]
+
+[project.license]
+text = "MIT"
+
+[[project.authors]]
+name = "Konstantin Kovalenko"
+email = "konstantin.l.kovalenko@gmail.com"
+
+[project.urls]
+Homepage = "https://github.com/despy49/duplicate-cleaner"
+Repository = "https://github.com/despy49/duplicate-cleaner"
+
+[project.scripts]
+duplicate-cleaner = "duplicate_cleaner.duplicate_cleaner:main"
+
+[build-system]
+requires = ["uv_build>=0.12.2,<0.13.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "allure-pytest>=2.16.2",
+    "mypy>=2.4.1",
+    "pytest>=9.2.0",
+    "pytest-cov>=7.1.0",
+]
+
+[tool.pytest.ini_options]
+minversion = "8.0"
+testpaths = ["tests"]
+disable_test_id_escaping_and_forfeit_all_rights_to_community_support = true
+markers = [
+    "unit: Isolated tests using mocks or parsing logic without disk I/O",
+    "functional: Core logic tests interacting with temporary file trees",
+    "integration: Component integration tests invoking the main execution flow",
+]
+addopts = "-v --tb=short --cov --cov-report=term --cov-report=html --alluredir=allure-results"
+norecursedirs = [
+    ".git",
+    ".venv",
+    "build",
+    "dist",
+]
+python_files = "test_*.py"
+python_functions = "test_*"
+
+[tool.coverage.run]
+source = ["src"]
+branch = true
+parallel = true
+
+[tool.coverage.report]
+show_missing = true
+skip_empty = true
+sort = "Cover"
+exclude_also = [
+    "pragma: no cover",
+    "if __name__ == .__main__.:",
+    "raise NotImplementedError",
+]
+
+[tool.coverage.html]
+directory = "htmlcov"

duplicate_cleaner-0.1.0/pyproject.toml.orig

@@ -0,0 +1,75 @@
+[project]
+name = "duplicate-cleaner"
+version = "0.1.0"
+description = "Cleans specified directories from duplicates based on hash comparison"
+readme = "README.md"
+requires-python = ">=3.12"
+license = { text = "MIT" }
+authors = [
+    { name = "Konstantin Kovalenko", email = "konstantin.l.kovalenko@gmail.com" }
+]
+classifiers = [
+    "Programming Language :: Python :: 3.12",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Environment :: Console",
+]
+[project.urls]
+Homepage = "https://github.com/despy49/duplicate-cleaner"
+Repository = "https://github.com/despy49/duplicate-cleaner"
+
+
+[project.scripts]
+duplicate-cleaner = "duplicate_cleaner.duplicate_cleaner:main"
+
+[build-system]
+requires = ["uv_build>=0.12.2,<0.13.0"]
+build-backend = "uv_build"
+
+
+[dependency-groups]
+dev = [
+    "allure-pytest>=2.16.2",
+    "mypy>=2.4.1",
+    "pytest>=9.2.0",
+    "pytest-cov>=7.1.0",
+]
+
+
+[tool.pytest.ini_options]
+minversion = "8.0"
+testpaths = ["tests"]
+disable_test_id_escaping_and_forfeit_all_rights_to_community_support = true
+markers = [
+    "unit: Isolated tests using mocks or parsing logic without disk I/O",
+    "functional: Core logic tests interacting with temporary file trees",
+    "integration: Component integration tests invoking the main execution flow"
+]
+
+addopts = "-v --tb=short --cov --cov-report=term --cov-report=html --alluredir=allure-results"
+norecursedirs = [
+    ".git",
+    ".venv",
+    "build",
+    "dist"
+]
+python_files = "test_*.py"
+python_functions = "test_*"
+
+[tool.coverage.run]
+source = ["src"]
+branch = true
+parallel = true
+
+[tool.coverage.report]
+show_missing = true
+skip_empty = true
+sort = "Cover"
+exclude_also = [
+    "pragma: no cover",
+    "if __name__ == .__main__.:",
+    "raise NotImplementedError"
+]
+
+[tool.coverage.html]
+directory = "htmlcov" 

duplicate_cleaner-0.1.0/src/duplicate_cleaner/__init__.py

@@ -0,0 +1,8 @@
+from .duplicate_cleaner import calculate_file_hash  # noqa: F401, I001
+from .duplicate_cleaner import find_duplicate_files  # noqa: F401
+from .duplicate_cleaner import flatten_symlink_chain  # noqa: F401
+from .duplicate_cleaner import parse_args  # noqa: F401
+from .duplicate_cleaner import remove_duplicate_files  # noqa: F401
+from .duplicate_cleaner import remove_empty_directories  # noqa: F401
+from .duplicate_cleaner import symlink_depth  # noqa: F401
+from .duplicate_cleaner import B_RED, GREEN, RST, color_print  # noqa: F401

duplicate_cleaner-0.1.0/src/duplicate_cleaner/duplicate_cleaner.py

@@ -0,0 +1,231 @@
+import argparse
+import errno
+import hashlib
+import os
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+B_RED = "\033[1;31m"
+GREEN = "\033[0;32m"
+RST = "\033[0m"
+
+
+def color_print(color: str, msg: str) -> None:
+    if color not in [B_RED, GREEN]:
+        raise RuntimeError(f"ERROR: color '{color!r}' undefined")
+    print(f"{color}{msg}{RST}")
+
+
+def calculate_file_hash(file_path: Path) -> str | None:
+    hasher = hashlib.sha256()
+    try:
+        with open(file_path, "rb") as file:
+            for chunk in iter(lambda: file.read(65536), b""):
+                hasher.update(chunk)
+        return hasher.hexdigest()
+    except (PermissionError, FileNotFoundError, OSError) as e:
+        color_print(B_RED, f"Failed reading '{file_path}': {e}")
+        return None
+
+
+def find_duplicate_files(root_dir: Path, recurse: bool) -> dict[str, list]:
+    size_groups = defaultdict(list)
+    duplicates = defaultdict(list)
+
+    color_print(GREEN, "Scanning directories structures for file sizes...")
+
+    search_pattern = "**/*" if recurse else "*"
+    for fpath in root_dir.glob(search_pattern):
+        try:
+            if fpath.is_dir() and not fpath.is_symlink():
+                continue
+
+            fsize = fpath.stat().st_size
+            size_groups[fsize].append(fpath)
+        except OSError as e:
+            if os.path.islink(fpath):
+                if e.errno == errno.ENOENT:
+                    color_print(B_RED, f"[Broken Symlink] '{fpath}' is broken.")
+                elif e.errno == errno.ELOOP:
+                    color_print(B_RED, f"[Symlink Loop] '{fpath}' forms a loop")
+                else:
+                    color_print(
+                        B_RED, f"[OSError Symlink] Failed to access link '{fpath}': {e}"
+                    )
+            else:
+                color_print(
+                    B_RED, f"[OSError File] Failed to get size for '{fpath}': {e}"
+                )
+            continue
+
+    color_print(GREEN, "Calculating file hashes...")
+    for paths in size_groups.values():
+        if len(paths) < 2:
+            continue
+
+        for file_path in paths:
+            file_hash = calculate_file_hash(file_path)
+            if file_hash:
+                duplicates[file_hash].append(file_path)
+    result = {h: paths for h, paths in duplicates.items() if len(paths) > 1}
+    for file_paths in result.values():
+        file_paths.sort(key=lambda p: len(str(p)))
+    return result
+
+
+def symlink_depth(path: Path) -> int:
+    path = path.absolute()
+    depth = 0
+    seen = set()
+
+    while path.is_symlink():
+        if path in seen:
+            raise RuntimeError("Symlink loop detected")
+        seen.add(path)
+
+        target = Path(path.readlink())
+        if not target.is_absolute():
+            target = path.parent / target
+
+        path = target.absolute()
+        depth += 1
+    return depth
+
+
+def flatten_symlink_chain(first: Path) -> None:
+    first = first.absolute()
+    current = first
+    links = []
+    seen = set()
+
+    while current.is_symlink():
+        current = current.absolute()
+        if current in seen:
+            raise RuntimeError("Symlink loop detected")
+        seen.add(current)
+        links.append(current)
+
+        target = Path(current.readlink())
+        current = target if target.is_absolute() else current.parent / target
+
+    final_target = current.resolve(strict=True)  # existence guaranteed
+
+    for link in links[1:]:
+        link.unlink()
+
+    first.unlink()
+    first.symlink_to(final_target)
+
+
+def remove_empty_directories(root_folder: Path):
+    color_print(GREEN, "Cleaning up empty directories...")
+
+    all_dirs = [p for p in root_folder.rglob("*") if p.is_dir() and not p.is_symlink()]
+    all_dirs.sort(key=lambda p: len(p.parts), reverse=True)
+
+    for dir_path in all_dirs:
+        if not any(dir_path.iterdir()):
+            try:
+                dir_path.rmdir()
+                color_print(GREEN, "[Removed Empty Dir] -> {dir_path}")
+            except OSError as e:
+                color_print(B_RED, f"ERROR: failed removing a dir:{e}")
+
+
+def remove_duplicate_files(duplicates: dict) -> None:
+    for file_paths in duplicates.values():
+        color_print(GREEN, f"Duplicates found (keeping: {file_paths[0]}):")
+        for to_delete in file_paths[1:]:
+            color_print(GREEN, f"[Deleting] -> {to_delete}")
+            try:
+                if to_delete.is_symlink():
+                    flatten_symlink_chain(to_delete)
+                    to_delete.unlink(missing_ok=True)
+                else:
+                    to_delete.unlink(missing_ok=True)
+            except OSError as e:
+                color_print(B_RED, f"[Error] Failed deleting {to_delete}: {e}")
+
+
+def parse_args():
+    parser = argparse.ArgumentParser("Duplicate cleaner script")
+    parser.add_argument(
+        "-t",
+        "--target",
+        dest="root_dir",
+        type=Path,
+        default=Path.home(),
+        help="Path to target directory",
+    )
+
+    parser.add_argument(
+        "-r",
+        "--recurse",
+        dest="recursive",
+        action="store_true",
+        help="Scan subdirectories recursively",
+    )
+
+    parser.add_argument(
+        "-nc",
+        "--no-confirm",
+        dest="no_confirm",
+        action="store_true",
+        help="Delete duplicates without confirmation prompt",
+    )
+
+    parser.add_argument(
+        "-re",
+        "--remove-empty",
+        dest="remove_empty_dirs",
+        action="store_true",
+        help="Remove empty directories after cleanup",
+    )
+
+    def check_root_dir(rd: Path) -> None:
+        if not rd.exists():
+            color_print(B_RED, f"Error: path '{rd}' not found")
+            parser.print_help()
+            sys.exit(-1)
+
+        if not rd.is_dir():
+            color_print(B_RED, f"Error: '{rd.__str__()}' is not a directory. Aborted")
+            parser.print_help()
+            sys.exit(-1)
+
+    try:
+        args = parser.parse_args()
+        args.root_dir = args.root_dir.resolve()
+        check_root_dir(args.root_dir)
+        return args
+    except argparse.ArgumentError as e:
+        color_print(B_RED, f"Error parsing args: {e}")
+        parser.print_help()
+        sys.exit(-1)
+
+
+def main() -> None:
+    args = parse_args()
+
+    duplicates: dict = find_duplicate_files(args.root_dir, args.recursive)
+
+    if not duplicates:
+        color_print(GREEN, "Duplicates not found")
+    else:
+        if args.no_confirm:
+            remove_duplicate_files(duplicates)
+            if args.remove_empty_dirs:
+                remove_empty_directories(args.root_dir)
+        else:
+            duplicate_count = len(duplicates)
+            decision = input(
+                f"{GREEN}\nFound {duplicate_count} duplicate groups. "
+                f"Remove them?(y/N): "
+            )
+            if decision.lower() == "y":
+                remove_duplicate_files(duplicates)
+                if args.remove_empty_dirs:
+                    remove_empty_directories(args.root_dir)
+            else:
+                color_print(B_RED, "File deletion aborted")