merger-cli 1.0.0__tar.gz

merger_cli-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Diogo Losacco Toporcov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

merger_cli-1.0.0/MANIFEST.in

@@ -0,0 +1,6 @@
+include LICENSE
+include README.md
+include pyproject.toml
+
+recursive-include merger *.py
+recursive-include examples *.py

merger_cli-1.0.0/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: merger-cli
+Version: 1.0.0
+Summary: Merger is a tool that scans a directory, filters files using customizable patterns, and merges readable content into a single output file.
+Author-email: Diogo Toporcov <diogotoporcov@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/diogotoporcov/merger-cli
+Project-URL: Documentation, https://github.com/diogotoporcov/merger-cli
+Keywords: merger,file system,concatenation,automation,development
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: chardet>=5.2.0
+Dynamic: license-file
+
+# Merger CLI
+
+[![Python](https://img.shields.io/badge/python-3.8%2B-blue.svg)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+Merger is a command-line utility for developers that scans a directory, filters files using customizable ignore patterns, and merges all readable content into a single structured output file. It supports custom file readers and validators, making it easily extendable for formats such as `.ipynb`, `.pdf`, or any specific format.
+
+---
+
+## Summary
+
+1. [Core Features](#core-features)
+2. [Dependencies](#dependencies)
+3. [Installation](#installation)
+4. [Usage](#usage)
+5. [Custom Readers](#custom-readers)
+6. [CLI Options](#cli-options)
+7. [License](#license)
+
+---
+
+## Core Features
+
+* **Recursive merge** of all readable text files under a root directory.
+* **Glob-based ignore patterns** using `.gitignore`-style syntax.
+* **Automatic encoding detection**.
+* **Custom file readers and validators** for non-text formats.
+* **CLI support** for installation, removal, and listing of custom readers.
+* **Human-readable merged output**, including a directory tree header and file delimiters.
+
+---
+
+## Dependencies
+
+| Component   | Version / Type | Notes                       |
+|-------------|----------------|-----------------------------|
+| **Python**  | ≥ 3.8          | Required                    |
+
+All dependencies are listed in [`requirements.txt`](requirements.txt).
+
+---
+
+## Installation
+
+### 1. Clone the repository
+
+```bash
+git clone https://github.com/diogotoporcov/merger-cli.git
+cd merger-cli
+```
+
+### 2. Create and activate a virtual environment
+
+**Linux / macOS**
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+```
+
+**Windows (PowerShell)**
+
+```powershell
+python -m venv .venv
+.venv\Scripts\Activate.ps1
+```
+
+### 3. Install dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+### 4. Install as CLI tool
+
+```bash
+pip install .
+```
+
+This registers the `merger` command globally.
+
+---
+
+## Usage
+
+### Basic merge
+
+```bash
+merger ./src ./merged.txt
+```
+
+### Custom ignore patterns
+
+```bash
+merger "C:\Users\USER\Desktop\project" "C:\Users\USER\Desktop\project\output.txt" --ignore "*.log" "__pycache__" "*.tmp"
+```
+
+### Custom ignore file
+
+```bash
+merger . ./output.txt -p ./merger.ignore
+```
+
+### Include empty files
+
+```bash
+merger ./data ./output.txt --empty
+```
+
+### Verbose output
+
+```bash
+merger ./src ./merged.txt --log-level DEBUG
+```
+
+---
+
+## Custom Readers
+
+You can extend Merger to handle new file formats.
+
+### Installing a custom reader
+
+```bash
+merger --install .ipynb path/to/ipynb.py
+```
+
+Where `ipynb.py` must define:
+
+*   ```python
+    validator: Callable[[Path], bool]
+    ```
+*   ```python
+    reader: Callable[[Path], str]
+    ```
+
+To uninstall:
+
+```bash
+merger --uninstall .ipynb
+```
+
+List installed readers:
+
+```bash
+merger --list-installed
+```
+
+An example `.ipynb` reader can be found in
+[`examples/custom_readers/ipynb.py`](examples/custom_readers/ipynb.py).
+
+---
+
+## CLI Options
+
+| Option                  | Description                                                                    |
+|-------------------------|--------------------------------------------------------------------------------|
+| `--ignore`              | List of glob-style ignore patterns.                                            |
+| `-f, --ignore-file`     | Path to file containing ignore patterns (Default: `<input_dir>/merger.ignore`. |
+| `-i, --install`         | Install a custom reader for an extension.                                      |
+| `-u, --uninstall`       | Remove a custom reader (`*` removes all).                                      |
+| `--list-installed`      | Show installed readers.                                                        |
+| `-l, --log-level`       | Set logging verbosity (`DEBUG`, `INFO`, etc.).                                 |
+| `--empty`               | Include empty files in merged output.                                          |
+| `--prefix` / `--suffix` | Customize file delimiters in output.                                           |
+| `--overrides`           | Load override reader definitions from a Python module.                         |
+| `--no-tree`             | Do not include the generated directory tree in the output file.                |
+
+
+---
+
+## License
+
+This project is licensed under the MIT License — see [LICENSE](LICENSE) for details.

merger_cli-1.0.0/README.md

@@ -0,0 +1,174 @@
+# Merger CLI
+
+[![Python](https://img.shields.io/badge/python-3.8%2B-blue.svg)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+Merger is a command-line utility for developers that scans a directory, filters files using customizable ignore patterns, and merges all readable content into a single structured output file. It supports custom file readers and validators, making it easily extendable for formats such as `.ipynb`, `.pdf`, or any specific format.
+
+---
+
+## Summary
+
+1. [Core Features](#core-features)
+2. [Dependencies](#dependencies)
+3. [Installation](#installation)
+4. [Usage](#usage)
+5. [Custom Readers](#custom-readers)
+6. [CLI Options](#cli-options)
+7. [License](#license)
+
+---
+
+## Core Features
+
+* **Recursive merge** of all readable text files under a root directory.
+* **Glob-based ignore patterns** using `.gitignore`-style syntax.
+* **Automatic encoding detection**.
+* **Custom file readers and validators** for non-text formats.
+* **CLI support** for installation, removal, and listing of custom readers.
+* **Human-readable merged output**, including a directory tree header and file delimiters.
+
+---
+
+## Dependencies
+
+| Component   | Version / Type | Notes                       |
+|-------------|----------------|-----------------------------|
+| **Python**  | ≥ 3.8          | Required                    |
+
+All dependencies are listed in [`requirements.txt`](requirements.txt).
+
+---
+
+## Installation
+
+### 1. Clone the repository
+
+```bash
+git clone https://github.com/diogotoporcov/merger-cli.git
+cd merger-cli
+```
+
+### 2. Create and activate a virtual environment
+
+**Linux / macOS**
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+```
+
+**Windows (PowerShell)**
+
+```powershell
+python -m venv .venv
+.venv\Scripts\Activate.ps1
+```
+
+### 3. Install dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+### 4. Install as CLI tool
+
+```bash
+pip install .
+```
+
+This registers the `merger` command globally.
+
+---
+
+## Usage
+
+### Basic merge
+
+```bash
+merger ./src ./merged.txt
+```
+
+### Custom ignore patterns
+
+```bash
+merger "C:\Users\USER\Desktop\project" "C:\Users\USER\Desktop\project\output.txt" --ignore "*.log" "__pycache__" "*.tmp"
+```
+
+### Custom ignore file
+
+```bash
+merger . ./output.txt -p ./merger.ignore
+```
+
+### Include empty files
+
+```bash
+merger ./data ./output.txt --empty
+```
+
+### Verbose output
+
+```bash
+merger ./src ./merged.txt --log-level DEBUG
+```
+
+---
+
+## Custom Readers
+
+You can extend Merger to handle new file formats.
+
+### Installing a custom reader
+
+```bash
+merger --install .ipynb path/to/ipynb.py
+```
+
+Where `ipynb.py` must define:
+
+*   ```python
+    validator: Callable[[Path], bool]
+    ```
+*   ```python
+    reader: Callable[[Path], str]
+    ```
+
+To uninstall:
+
+```bash
+merger --uninstall .ipynb
+```
+
+List installed readers:
+
+```bash
+merger --list-installed
+```
+
+An example `.ipynb` reader can be found in
+[`examples/custom_readers/ipynb.py`](examples/custom_readers/ipynb.py).
+
+---
+
+## CLI Options
+
+| Option                  | Description                                                                    |
+|-------------------------|--------------------------------------------------------------------------------|
+| `--ignore`              | List of glob-style ignore patterns.                                            |
+| `-f, --ignore-file`     | Path to file containing ignore patterns (Default: `<input_dir>/merger.ignore`. |
+| `-i, --install`         | Install a custom reader for an extension.                                      |
+| `-u, --uninstall`       | Remove a custom reader (`*` removes all).                                      |
+| `--list-installed`      | Show installed readers.                                                        |
+| `-l, --log-level`       | Set logging verbosity (`DEBUG`, `INFO`, etc.).                                 |
+| `--empty`               | Include empty files in merged output.                                          |
+| `--prefix` / `--suffix` | Customize file delimiters in output.                                           |
+| `--overrides`           | Load override reader definitions from a Python module.                         |
+| `--no-tree`             | Do not include the generated directory tree in the output file.                |
+
+
+---
+
+## License
+
+This project is licensed under the MIT License — see [LICENSE](LICENSE) for details.

merger_cli-1.0.0/examples/custom_readers/ipynb.py

@@ -0,0 +1,82 @@
+import json
+from pathlib import Path
+from typing import Final, Callable
+
+import chardet
+
+
+def is_ipynb_file(file_path: Path) -> bool:
+    """
+    Checks whether the given file is a valid Jupyter Notebook (.ipynb) file.
+
+    Args:
+        file_path (Path): Path to the file.
+
+    Returns:
+        bool: True if the file is a readable notebook with at least one cell, False otherwise.
+    """
+    try:
+        if not file_path.suffix == ".ipynb":
+            return False
+
+        with open(file_path, "rb") as file:
+            chunk = file.read(1024)
+
+        result = chardet.detect(chunk)
+        encoding = result.get("encoding")
+        confidence = result.get("confidence", 0)
+
+        if not encoding or confidence < 0.8:
+            encoding = "utf-8"
+
+        with file_path.open(encoding=encoding) as f:
+            notebook = json.load(f)
+
+        return isinstance(notebook, dict) and "cells" in notebook and isinstance(notebook["cells"], list)
+
+    except Exception:
+        return False
+
+
+def extract_ipynb_content(file_path: Path, include_markdown: bool = True, include_code: bool = True) -> str:
+    """
+    Extracts code and markdown content from a Jupyter Notebook (.ipynb) file.
+
+    Args:
+        file_path (Path): Path to the notebook file.
+        include_markdown (bool): Whether to include markdown cells.
+        include_code (bool): Whether to include code cells.
+
+    Returns:
+        str: Extracted content with cells separated by double newlines.
+    """
+    result = []
+
+    with file_path.open(encoding="utf-8") as f:
+        notebook = json.load(f)
+
+    for cell in notebook.get("cells", []):
+        cell_type = cell.get("cell_type")
+        lines = cell.get("source", [])
+
+        if isinstance(lines, str):
+            lines = lines.splitlines()
+
+        if cell_type == "code" and include_code:
+            block = [line.rstrip() for line in lines]
+            if block:
+                result.append("\n".join(block))
+
+        elif cell_type == "markdown" and include_markdown:
+            block = [line.rstrip() for line in lines]
+            if block:
+                result.append("```markdown\n" + "\n".join(block) + "\n```")
+
+    return "\n\n".join(result)
+
+
+validator: Final[Callable[[Path], bool]] = is_ipynb_file
+reader: Final[Callable[[Path], str]] = extract_ipynb_content
+
+
+__all__ = ["validator", "reader"]

merger_cli-1.0.0/examples/main.example.py

@@ -0,0 +1,29 @@
+from pathlib import Path
+
+from merger.files import merge
+import examples.custom_readers.ipynb as ipynb
+
+if __name__ == "__main__":
+    root = Path("path/to/dir")
+    ignore_patterns = [
+        "README.md",
+        ".idea",
+        "__pycache__",
+        ".env",
+        "./example/path",  # File or folder named 'path' relative ./example/, where '.' is the root dir
+        "C:/Users/User/Desktop/path/to/dir/2",
+        "output.txt",
+        ".venv",
+        "*.docx",  # Any file with extension .docx
+        "*cache*",  # Any file of folder that contains 'cache' in its name or path
+        "__*__"  # Any file or folder that starts with '__' and ends with '__'
+    ]
+    output_path = Path("./output.txt")
+
+    merge(
+        root,
+        ignore_patterns,
+        output_path,
+        validation_func_override={".ipynb": ipynb.validator},
+        read_func_override={".ipynb": ipynb.reader}
+    )

merger_cli-1.0.0/merger/cli.py

@@ -0,0 +1,143 @@
+import argparse
+import logging
+from pathlib import Path
+
+from .files import merge, read_ignore_file
+from .logger import logger, setup_logger
+from .registry import register_reader, unregister_reader, list_readers, load_installed_readers, load_custom_readers
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Merge readable files in a directory with support for ignore patterns and custom file readers."
+    )
+
+    # Required positional args
+    parser.add_argument("input_dir", type=Path, nargs="?", help="Root directory to scan for files")
+    parser.add_argument("output_file", type=Path, nargs="?",
+                        help="File to save merged output (default: <input_dir>/merger.txt)")
+
+    # Reader management
+    parser.add_argument("-i", "--install", nargs=2, metavar=("EXT", "MODULE_PATH"),
+                        help="Install a custom reader for a given extension (e.g., .pdf)")
+
+    parser.add_argument("-u", "--uninstall", metavar="EXT",
+                        help="Uninstall a custom reader by extension (use '*' to remove all)")
+
+    parser.add_argument("--list-installed", action="store_true",
+                        help="List all installed custom readers")
+
+    # Logging
+    parser.add_argument(
+        "-l", "--log-level",
+        type=str,
+        choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
+        default="INFO",
+        help="Set the logging level (default: INFO)"
+    )
+
+    # Ignore logic
+    parser.add_argument("--ignore", nargs="*", default=[],
+                        help="Glob-style patterns to ignore (e.g., '*.log', '__pycache__')")
+
+    parser.add_argument(
+        "-f", "--ignore-file", type=Path,
+        help="File containing glob-style patterns to ignore (default: <input_dir>/merger.ignore)")
+
+    # Custom reader overrides
+    parser.add_argument("--overrides", type=Path,
+                        help="Path to Python module with `validators` and `readers` dictionaries for custom overrides")
+
+    parser.add_argument("--empty", action="store_true", default=False,
+                        help="Include empty files in the merged output")
+
+    parser.add_argument("--prefix", type=str, default="<<FILE_START: {path}>>\n",
+                        help="Format string for file start marker (set empty string to disable)")
+
+    parser.add_argument("--suffix", type=str, default="\n<<FILE_END: {path}>>\n\n",
+                        help="Format string for file end marker (set empty string to disable)")
+
+    # Tree
+    parser.add_argument(
+        "--no-tree",
+        action="store_true",
+        default=False,
+        help="Do not include the generated directory tree in the output file"
+    )
+
+    # CLI Logic
+    args = parser.parse_args()
+
+    setup_logger(level=getattr(logging, args.log_level.upper()))
+
+    if args.install:
+        ext, path = args.install
+        register_reader(ext, path)
+        logger.info(f"Installed reader for '{ext}' from '{path}'")
+        return
+
+    if args.uninstall:
+        if args.uninstall == "*":
+            installed = list_readers()
+            if not installed:
+                logger.info("No custom readers to uninstall.")
+            else:
+                for ext in list(installed.keys()):
+                    unregister_reader(ext)
+                    logger.info(f"Uninstalled reader for '{ext}'")
+        else:
+            unregister_reader(args.uninstall)
+            logger.info(f"Uninstalled reader for '{args.uninstall}'")
+        return
+
+    if args.list_installed:
+        installed = list_readers()
+        if not installed:
+            logger.info("No custom readers installed.")
+        else:
+            logger.info("Installed Custom Readers:")
+            for ext, mod_path in installed.items():
+                logger.info(f"  {ext}: {mod_path}")
+        return
+
+    # Handle default output file
+    if not args.input_dir:
+        parser.error("input_dir is required unless installing/uninstalling/listing readers.")
+
+    if not args.output_file:
+        args.output_file = args.input_dir / "merger.txt"
+
+    if not args.ignore_file and args.input_dir:
+        default_ignore = args.input_dir / "merger.ignore"
+        if default_ignore.exists():
+            args.ignore_file = default_ignore
+            logger.info("Found default ignore file 'merger.ignore' in input directory. Using it for ignore patterns.")
+
+    ignore_patterns = args.ignore.copy()
+    if args.ignore_file:
+        ignore_patterns.extend(read_ignore_file(args.ignore_file))
+
+    readers, validators = load_installed_readers()
+
+    if args.overrides:
+        r, v = load_custom_readers(args.overrides)
+        readers.update(r)
+        validators.update(v)
+
+    merge(
+        dir_path=args.input_dir,
+        ignore_patterns=ignore_patterns,
+        output_path=args.output_file,
+        validation_func_override=validators,
+        read_func_override=readers,
+        write_if_empty=args.empty,
+        prefix_format=args.prefix,
+        suffix_format=args.suffix,
+        include_tree=not args.no_tree
+    )
+
+    logger.info(f"Saved to {args.output_file}")
+
+
+if __name__ == "__main__":
+    main()

merger_cli-1.0.0/merger/files.py

@@ -0,0 +1,139 @@
+from pathlib import Path
+from typing import Callable, Dict, Optional, List
+
+import chardet
+
+from .filtering import filter_files_by_patterns
+from .tree import generate_tree_visualizer
+from .logger import logger
+
+
+def is_text_file(
+    file_path: Path,
+    chunk_size: int = 1024,
+    *,
+    validation_func_override: Optional[Dict[str, Callable[[Path], bool]]] = None,
+    min_encoding_detection_confidence: float = 0.8
+) -> bool:
+    logger.debug(f"Checking if file is text: {file_path}")
+    try:
+        if validation_func_override and callable(validation_override_func := validation_func_override.get(file_path.suffix)):
+            result = validation_override_func(file_path)
+            logger.debug(f"Custom validator used for {file_path.suffix}: {result}")
+            return result
+
+        with open(file_path, "rb") as file:
+            chunk = file.read(chunk_size)
+
+        result = chardet.detect(chunk)
+        encoding = result.get("encoding")
+        confidence = result.get("confidence", 0)
+        logger.debug(f"Detected encoding for {file_path}: {encoding} (confidence: {confidence})")
+
+        if not encoding or confidence < min_encoding_detection_confidence:
+            encoding = "utf-8"
+            logger.debug(f"Low confidence for {file_path}. Falling back to utf-8")
+
+        chunk.decode(encoding)
+        return True
+
+    except Exception as e:
+        logger.error(f"Failed to decode {file_path}: {e}")
+        return False
+
+
+def append_content(
+    root: Path,
+    file_path: Path,
+    output_path: Path,
+    *,
+    prefix: Optional[str] = "<<FILE_START: {path}>>\n",
+    suffix: Optional[str] = "\n<<FILE_END: {path}>>\n\n",
+    read_func_override: Optional[Dict[str, Callable[[Path], str]]] = None,
+    write_if_empty: bool = False
+) -> None:
+    relative_path = ".\\" + str(file_path.relative_to(root))
+    formatted_prefix = prefix.format(path=relative_path) if prefix else ""
+    formatted_suffix = suffix.format(path=relative_path) if suffix else ""
+
+    try:
+        if read_func_override and callable(read_override_func := read_func_override.get(file_path.suffix)):
+            content = read_override_func(file_path)
+            logger.debug(f"Used custom reader for {file_path}")
+        else:
+            content = file_path.read_text(encoding="utf-8")
+            logger.debug(f"Read text from {file_path} using utf-8")
+
+        if not write_if_empty and not content:
+            logger.debug(f"Skipping empty file: {file_path}")
+            return
+
+        with output_path.open("a", encoding="utf-8") as f:
+            f.write(formatted_prefix + content + formatted_suffix)
+            logger.debug(f"Merged: {file_path}")
+
+    except Exception as e:
+        logger.error(f"Failed to append content from {file_path}: {e}")
+
+
+def merge(
+    dir_path: Path,
+    ignore_patterns: List[str],
+    output_path: Path,
+    *,
+    read_func_override: Optional[Dict[str, Callable[[Path], str]]] = None,
+    validation_func_override: Optional[Dict[str, Callable[[Path], bool]]] = None,
+    min_encoding_detection_confidence: float = 0.8,
+    write_if_empty: bool = False,
+    prefix_format: Optional[str] = "<<FILE_START: {path}>>\n",
+    suffix_format: Optional[str] = "\n<<FILE_END: {path}>>\n\n",
+    include_tree: bool = True
+) -> None:
+    logger.debug(f"Starting merge from: {dir_path}")
+    paths = filter_files_by_patterns(dir_path, ignore_patterns, True)
+    paths = [path for path in paths if path.resolve() != output_path.resolve()]
+    logger.debug(f"Filtered paths: {len(paths)} files to process")
+
+    if include_tree:
+        try:
+            with open(output_path, "w", encoding="utf-8") as f:
+                tree = generate_tree_visualizer(dir_path, paths)
+                f.write(f"{tree}\n")
+                logger.debug(f"Directory tree written to {output_path}")
+
+        except Exception as e:
+            logger.error(f"Failed to write tree header to {output_path}: {e}")
+            return
+
+    else:
+        output_path.write_text("", encoding="utf-8")
+
+    paths = [path for path in paths if not path.is_dir()]
+
+    for path in paths:
+        if not is_text_file(
+            path,
+            validation_func_override=validation_func_override,
+            min_encoding_detection_confidence=min_encoding_detection_confidence
+        ):
+            logger.debug(f"Skipped non-text file: {path}")
+            continue
+
+        append_content(
+            root=dir_path,
+            file_path=path,
+            output_path=output_path,
+            prefix=prefix_format,
+            suffix=suffix_format,
+            read_func_override=read_func_override,
+            write_if_empty=write_if_empty
+        )
+
+
+def read_ignore_file(file_path: Path) -> list[str]:
+    if not file_path.is_file():
+        logger.error(f"Ignore file not found: {file_path}")
+        raise FileNotFoundError(f"Ignore file not found: {file_path}")
+
+    logger.debug(f"Reading ignore patterns from: {file_path}")
+    return [line.strip() for line in file_path.read_text(encoding="utf-8").splitlines() if line.strip()]

merger_cli-1.0.0/merger/filtering.py

@@ -0,0 +1,47 @@
+from pathlib import Path
+import fnmatch
+from typing import List
+
+from .logger import logger  # Importa o logger customizado
+
+
+def filter_files_by_patterns(dir_path: Path, ignore_patterns: List[str], recursive: bool) -> List[Path]:
+    """
+    Filters files and directories under a root directory, excluding any that match ignore patterns.
+
+    Args:
+        dir_path (Path): Directory to scan files from.
+        ignore_patterns (List[str]): List of glob-style patterns to exclude files and directories.
+        recursive (bool): Whether to scan directories recursively.
+
+    Returns:
+        List[Path]: A list of matching file and directory paths.
+    """
+
+    def matches_any_pattern(path: Path) -> bool:
+        rel_path = path.relative_to(dir_path).as_posix()
+        for pattern in ignore_patterns:
+            pat = pattern.rstrip("/")
+            if fnmatch.fnmatch(rel_path, pat) or fnmatch.fnmatch(path.name, pat):
+                logger.debug(f"Ignoring path '{rel_path}' matched by pattern '{pat}'")
+                return True
+        return False
+
+    results = []
+
+    def scan_dir(directory: Path):
+        logger.debug(f"Scanning directory: {directory}")
+        try:
+            for entry in directory.iterdir():
+                if matches_any_pattern(entry):
+                    continue
+
+                results.append(entry)
+                if recursive and entry.is_dir():
+                    scan_dir(entry)
+        except Exception as e:
+            logger.warning(f"Failed to scan directory {directory}: {e}")
+
+    scan_dir(dir_path)
+    logger.debug(f"Total matched files and folders: {len(results)}")
+    return results

merger_cli-1.0.0/merger/logger.py

@@ -0,0 +1,22 @@
+import logging
+import sys
+
+
+def setup_logger(name: str = "merger", level: int = logging.INFO) -> logging.Logger:
+    logger = logging.getLogger(name)
+    logger.setLevel(level)
+
+    if not logger.handlers:
+        handler = logging.StreamHandler(sys.stdout)
+        formatter = logging.Formatter("[%(levelname)s] %(message)s")
+        handler.setFormatter(formatter)
+        logger.addHandler(handler)
+
+    else:
+        for handler in logger.handlers:
+            handler.setLevel(level)
+
+    return logger
+
+
+logger = setup_logger()

merger_cli-1.0.0/merger/registry.py

@@ -0,0 +1,136 @@
+import importlib.util
+import os
+import platform
+import shutil
+from pathlib import Path
+from typing import Tuple, Dict, Callable
+
+from .logger import logger  # Adiciona o logger
+
+
+def get_readers_folder() -> Path:
+    system = platform.system()
+    if system == "Windows":
+        base = os.getenv("LOCALAPPDATA") or os.getenv("APPDATA") or str(Path.home())
+        readers_dir = Path(base) / "Merger" / "installed_readers"
+    elif system == "Darwin":
+        readers_dir = Path.home() / "Library" / "Application Support" / "Merger" / "installed_readers"
+    else:
+        readers_dir = Path.home() / ".local" / "share" / "Merger" / "installed_readers"
+
+    readers_dir.mkdir(parents=True, exist_ok=True)
+    logger.debug(f"Readers directory resolved to: {readers_dir}")
+    return readers_dir
+
+
+def register_reader(extension: str, module_path: str):
+    if not extension.startswith("."):
+        raise ValueError("Extension must start with a dot, e.g. '.pdf'")
+
+    module_path = Path(module_path).resolve()
+    logger.debug(f"Registering reader for {extension} from {module_path}")
+
+    try:
+        spec = importlib.util.spec_from_file_location("temp_custom_reader", module_path)
+        module = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(module)
+
+        expected_all = ["reader"]
+        actual_all = getattr(module, "__all__", None)
+
+        if actual_all != expected_all:
+            raise ValueError(f"Invalid module: __all__ must be exactly {expected_all}, got {actual_all}")
+
+        if not hasattr(module, "validator") or not callable(module.validator):
+            raise ValueError("Module must define a callable 'validator'")
+
+        if not hasattr(module, "reader") or not callable(module.reader):
+            raise ValueError("Module must define a callable 'reader'")
+
+        readers_dir = get_readers_folder()
+        dest_path = readers_dir / f"{extension[1:]}.py"
+        shutil.copy(module_path, dest_path)
+
+        logger.info(f"Reader for '{extension}' registered successfully at '{dest_path}'")
+
+    except Exception as e:
+        logger.error(f"Failed to register reader for '{extension}': {e}")
+        raise
+
+
+def unregister_reader(extension: str):
+    if not extension.startswith("."):
+        raise ValueError("Extension must start with a dot, e.g. '.pdf'")
+
+    readers_dir = get_readers_folder()
+    target = readers_dir / f"{extension[1:]}.py"
+
+    if target.exists():
+        target.unlink()
+        logger.info(f"Reader for '{extension}' unregistered (removed): {target}")
+    else:
+        logger.warning(f"No reader found to unregister for extension: '{extension}'")
+
+
+def list_readers() -> Dict[str, str]:
+    readers_dir = get_readers_folder()
+    readers = {
+        f".{f.stem}": str(f.resolve())
+        for f in readers_dir.glob("*.py")
+    }
+
+    logger.debug(f"Listing installed readers: {readers}")
+    return readers
+
+
+def load_installed_readers() -> Tuple[Dict[str, Callable], Dict[str, Callable]]:
+    readers_folder = get_readers_folder()
+    validators = {}
+    readers = {}
+
+    for file in readers_folder.glob("*.py"):
+        ext = f".{file.stem}"
+        logger.debug(f"Loading reader module: {file}")
+
+        try:
+            spec = importlib.util.spec_from_file_location(f"reader_{file.stem}", file)
+            module = importlib.util.module_from_spec(spec)
+            spec.loader.exec_module(module)
+
+            all_attr = getattr(module, "__all__", [])
+            reader = getattr(module, "reader", None)
+            validator = getattr(module, "validator", None)
+
+            if "reader" not in all_attr or not callable(reader):
+                logger.warning(f"Skipping invalid reader module '{file.name}': missing or invalid 'reader'")
+                continue
+
+            if "validator" in all_attr and not callable(validator):
+                logger.warning(f"Skipping invalid validator in '{file.name}'")
+                continue
+
+            readers[ext] = reader
+            validators[ext] = validator
+            logger.debug(f"Loaded reader for {ext}")
+
+        except Exception as e:
+            logger.error(f"Failed to load reader from '{file}': {e}")
+
+    return readers, validators
+
+
+def load_custom_readers(module_path: Path) -> Tuple[Dict[str, Callable], Dict[str, Callable]]:
+    logger.debug(f"Loading custom readers from: {module_path}")
+    try:
+        spec = importlib.util.spec_from_file_location("custom_readers_module", module_path)
+        module = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(module)
+
+        readers = getattr(module, "readers", {})
+        validators = getattr(module, "validators", {})
+        logger.info(f"Custom readers loaded successfully from: {module_path}")
+        return readers, validators
+
+    except Exception as e:
+        logger.error(f"Failed to load custom readers from {module_path}: {e}")
+        raise

merger_cli-1.0.0/merger/tree.py

@@ -0,0 +1,59 @@
+from pathlib import Path
+from collections import defaultdict
+from typing import Dict, Union, List
+
+_FileTree = Dict[str, Union["_FileTree", None]]
+
+
+def generate_tree_visualizer(root_path: Path, paths: List[Path]) -> str:
+    """
+    Generates a visual tree structure of the provided paths relative to a root directory.
+
+    Args:
+        root_path (Path): The root directory.
+        paths (List[Path]): List of file and directory paths relative to the root to include in the tree.
+
+    Returns:
+        str: A formatted string representing the directory structure.
+    """
+    root_path = root_path.resolve()
+
+    def tree() -> _FileTree:
+        return defaultdict(tree)
+
+    file_tree: _FileTree = tree()
+
+    for path in paths:
+        try:
+            relative_parts = path.resolve().relative_to(root_path).parts
+        except ValueError:
+            continue
+
+        current = file_tree
+        for part in relative_parts:
+            current = current[part]
+
+    def build_tree_str(d: _FileTree, prefix: str = "", current_path: Path = root_path) -> str:
+        tree_str = []
+        entries = sorted(d.keys())
+        for i, key in enumerate(entries, 1):
+            is_last = i == len(entries)
+            connector = "└── " if is_last else "├── "
+
+            full_path = current_path / key
+
+            if full_path.is_dir():
+                display_name = f"{key}/"
+            else:
+                display_name = key
+
+            tree_str.append(f"{prefix}{connector}{display_name}\n")
+
+            extension = "    " if is_last else "│   "
+            tree_str.append(build_tree_str(d[key], prefix + extension, full_path))
+
+        return "".join(tree_str)
+
+    tree_output = f"{root_path.name}/\n{build_tree_str(file_tree)}"
+    return tree_output
+

merger_cli-1.0.0/merger_cli.egg-info/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: merger-cli
+Version: 1.0.0
+Summary: Merger is a tool that scans a directory, filters files using customizable patterns, and merges readable content into a single output file.
+Author-email: Diogo Toporcov <diogotoporcov@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/diogotoporcov/merger-cli
+Project-URL: Documentation, https://github.com/diogotoporcov/merger-cli
+Keywords: merger,file system,concatenation,automation,development
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: chardet>=5.2.0
+Dynamic: license-file
+
+# Merger CLI
+
+[![Python](https://img.shields.io/badge/python-3.8%2B-blue.svg)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+Merger is a command-line utility for developers that scans a directory, filters files using customizable ignore patterns, and merges all readable content into a single structured output file. It supports custom file readers and validators, making it easily extendable for formats such as `.ipynb`, `.pdf`, or any specific format.
+
+---
+
+## Summary
+
+1. [Core Features](#core-features)
+2. [Dependencies](#dependencies)
+3. [Installation](#installation)
+4. [Usage](#usage)
+5. [Custom Readers](#custom-readers)
+6. [CLI Options](#cli-options)
+7. [License](#license)
+
+---
+
+## Core Features
+
+* **Recursive merge** of all readable text files under a root directory.
+* **Glob-based ignore patterns** using `.gitignore`-style syntax.
+* **Automatic encoding detection**.
+* **Custom file readers and validators** for non-text formats.
+* **CLI support** for installation, removal, and listing of custom readers.
+* **Human-readable merged output**, including a directory tree header and file delimiters.
+
+---
+
+## Dependencies
+
+| Component   | Version / Type | Notes                       |
+|-------------|----------------|-----------------------------|
+| **Python**  | ≥ 3.8          | Required                    |
+
+All dependencies are listed in [`requirements.txt`](requirements.txt).
+
+---
+
+## Installation
+
+### 1. Clone the repository
+
+```bash
+git clone https://github.com/diogotoporcov/merger-cli.git
+cd merger-cli
+```
+
+### 2. Create and activate a virtual environment
+
+**Linux / macOS**
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+```
+
+**Windows (PowerShell)**
+
+```powershell
+python -m venv .venv
+.venv\Scripts\Activate.ps1
+```
+
+### 3. Install dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+### 4. Install as CLI tool
+
+```bash
+pip install .
+```
+
+This registers the `merger` command globally.
+
+---
+
+## Usage
+
+### Basic merge
+
+```bash
+merger ./src ./merged.txt
+```
+
+### Custom ignore patterns
+
+```bash
+merger "C:\Users\USER\Desktop\project" "C:\Users\USER\Desktop\project\output.txt" --ignore "*.log" "__pycache__" "*.tmp"
+```
+
+### Custom ignore file
+
+```bash
+merger . ./output.txt -p ./merger.ignore
+```
+
+### Include empty files
+
+```bash
+merger ./data ./output.txt --empty
+```
+
+### Verbose output
+
+```bash
+merger ./src ./merged.txt --log-level DEBUG
+```
+
+---
+
+## Custom Readers
+
+You can extend Merger to handle new file formats.
+
+### Installing a custom reader
+
+```bash
+merger --install .ipynb path/to/ipynb.py
+```
+
+Where `ipynb.py` must define:
+
+*   ```python
+    validator: Callable[[Path], bool]
+    ```
+*   ```python
+    reader: Callable[[Path], str]
+    ```
+
+To uninstall:
+
+```bash
+merger --uninstall .ipynb
+```
+
+List installed readers:
+
+```bash
+merger --list-installed
+```
+
+An example `.ipynb` reader can be found in
+[`examples/custom_readers/ipynb.py`](examples/custom_readers/ipynb.py).
+
+---
+
+## CLI Options
+
+| Option                  | Description                                                                    |
+|-------------------------|--------------------------------------------------------------------------------|
+| `--ignore`              | List of glob-style ignore patterns.                                            |
+| `-f, --ignore-file`     | Path to file containing ignore patterns (Default: `<input_dir>/merger.ignore`. |
+| `-i, --install`         | Install a custom reader for an extension.                                      |
+| `-u, --uninstall`       | Remove a custom reader (`*` removes all).                                      |
+| `--list-installed`      | Show installed readers.                                                        |
+| `-l, --log-level`       | Set logging verbosity (`DEBUG`, `INFO`, etc.).                                 |
+| `--empty`               | Include empty files in merged output.                                          |
+| `--prefix` / `--suffix` | Customize file delimiters in output.                                           |
+| `--overrides`           | Load override reader definitions from a Python module.                         |
+| `--no-tree`             | Do not include the generated directory tree in the output file.                |
+
+
+---
+
+## License
+
+This project is licensed under the MIT License — see [LICENSE](LICENSE) for details.

merger_cli-1.0.0/merger_cli.egg-info/SOURCES.txt

@@ -0,0 +1,22 @@
+LICENSE
+MANIFEST.in
+README.md
+pyproject.toml
+setup.cfg
+examples/__init__.py
+examples/main.example.py
+examples/custom_readers/__init__.py
+examples/custom_readers/ipynb.py
+merger/__init__.py
+merger/cli.py
+merger/files.py
+merger/filtering.py
+merger/logger.py
+merger/registry.py
+merger/tree.py
+merger_cli.egg-info/PKG-INFO
+merger_cli.egg-info/SOURCES.txt
+merger_cli.egg-info/dependency_links.txt
+merger_cli.egg-info/entry_points.txt
+merger_cli.egg-info/requires.txt
+merger_cli.egg-info/top_level.txt

merger_cli-1.0.0/merger_cli.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

merger_cli-1.0.0/merger_cli.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+merger = merger.cli:main

merger_cli-1.0.0/merger_cli.egg-info/requires.txt

@@ -0,0 +1 @@
+chardet>=5.2.0

merger_cli-1.0.0/merger_cli.egg-info/top_level.txt

@@ -0,0 +1 @@
+merger

merger_cli-1.0.0/pyproject.toml

@@ -0,0 +1,33 @@
+[project]
+name = "merger-cli"
+version = "1.0.0"
+description = "Merger is a tool that scans a directory, filters files using customizable patterns, and merges readable content into a single output file."
+keywords = ["merger", "file system", "concatenation", "automation", "development"]
+authors = [
+    { name = "Diogo Toporcov", email = "diogotoporcov@gmail.com" }
+]
+license = { text = "MIT" }
+readme = "README.md"
+requires-python = ">=3.8"
+dependencies = [
+    "chardet>=5.2.0"
+]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Intended Audience :: Developers",
+    "Topic :: Software Development :: Libraries :: Python Modules"
+]
+
+
+[project.urls]
+Homepage = "https://github.com/diogotoporcov/merger-cli"
+Documentation = "https://github.com/diogotoporcov/merger-cli"
+
+[build-system]
+requires = ["setuptools>=80.9.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project.scripts]
+merger = "merger.cli:main"

merger_cli-1.0.0/setup.cfg

@@ -0,0 +1,7 @@
+[metadata]
+description-file = README.md
+
+[egg_info]
+tag_build = 
+tag_date = 0
+