pdfalyzer 1.16.14__tar.gz → 1.17.0__tar.gz

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/CHANGELOG.md

@@ -1,7 +1,11 @@
 # NEXT RELEASE
 
+# 1.17.0
+* Add `extract_pdf_pages` command line tool (imported from `clown_sort`)
+* Add `extract_text_from_pdfs` command line tool (imported from `clown_sort`)
+
 ### 1.16.14
-* Bump `yaralyzer` to v1.0.9
+* Bump `yaralyzer` to v1.0.9, handle `FileNotFoundError` which is now raised instead of `TypeError`
 * Drop support for python 3.9
 
 ### 1.16.13

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pdfalyzer
-Version: 1.16.14
+Version: 1.17.0
 Summary: PDF analysis tool. Scan a PDF with YARA rules, visualize its inner tree-like data structure in living color (lots of colors), force decodes of suspicious font binaries, and more.
 Home-page: https://github.com/michelcrypt4d4mus/pdfalyzer
 License: GPL-3.0-or-later
@@ -21,20 +21,23 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Artistic Software
 Classifier: Topic :: Scientific/Engineering :: Visualization
 Classifier: Topic :: Security
+Provides-Extra: extract
+Requires-Dist: PyMuPDF (>=1.26.4,<2.0.0) ; extra == "extract"
 Requires-Dist: anytree (>=2.13,<3.0)
 Requires-Dist: pypdf (>=6.0.0,<7.0.0)
+Requires-Dist: pytesseract (>=0.3.13,<0.4.0) ; extra == "extract"
 Requires-Dist: yaralyzer (>=1.0.9,<2.0.0)
 Project-URL: Changelog, https://github.com/michelcrypt4d4mus/pdfalyzer/blob/master/CHANGELOG.md
 Project-URL: Documentation, https://github.com/michelcrypt4d4mus/pdfalyzer
 Project-URL: Repository, https://github.com/michelcrypt4d4mus/pdfalyzer
 Description-Content-Type: text/markdown
 
-<!-- ![Tests](https://img.shields.io/github/workflow/status/michelcrypt4d4mus/pdfalyzer/tests?label=tests)  -->
-![Python Version](https://img.shields.io/pypi/pyversions/pdfalyzer)
 [![GithubRelease](https://img.shields.io/github/v/release/michelcrypt4d4mus/pdfalyzer?sort=semver)](https://pypi.org/project/pdfalyzer/)
-[![GitHub last commit](https://img.shields.io/github/last-commit/michelcrypt4d4mus/pdfalyzer)](https://github.com/michelcrypt4d4mus/pdfalyzer)
 ![PyPiRelease](https://img.shields.io/pypi/v/pdfalyzer)
+[![GitHub last commit](https://img.shields.io/github/last-commit/michelcrypt4d4mus/pdfalyzer)](https://github.com/michelcrypt4d4mus/pdfalyzer)
+![Python Version](https://img.shields.io/pypi/pyversions/pdfalyzer)
 ![Downloads](https://img.shields.io/pypi/dm/pdfalyzer)
+[![Tests](https://github.com/michelcrypt4d4mus/pdfalyzer/actions/workflows/python-package.yml/badge.svg)](https://github.com/michelcrypt4d4mus/pdfalyzer/actions/workflows/python-package.yml)
 
 
 # THE PDFALYZER
@@ -242,9 +245,27 @@ Things like, say, a hidden binary `/F` (PDF instruction meaning "URL") followed
 
 -------------
 
+
 # PDF Resources
 ## Included PDF Tools
-The Pdfalyzer ships with a command line tool `combine_pdfs` that combines multiple PDFs into a single PDF. Run `combine_pdfs --help` to see the options.
+The Pdfalyzer comes with a few command line tools:
+
+#### `combine_pdfs`
+Combines multiple PDFs into a single PDF. Run `combine_pdfs --help` to see the options.
+
+#### `extract_pdf_pages`
+Extracts page ranges (e.g. "10-25") from a PDF and writes them to a new PDF. Run `extract_pdf_pages --help` to see the options.
+![](doc/extract_pages_from_pdf_help.png)
+
+#### `extract_text_from_pdfs`
+Extracts text from a PDF, including applying OCR to all embedded images. Requires that you install The Pdfalyzer's optional dependencies:
+
+```bash
+pipx install pdfalyzer[extract]
+```
+
+Run `extract_text_from_pdfs --help` to see the options.
+
 
 ## 3rd Party PDF Tools
 ### Installing Didier Stevens's PDF Analysis Tools

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/README.md

@@ -1,9 +1,9 @@
-<!-- ![Tests](https://img.shields.io/github/workflow/status/michelcrypt4d4mus/pdfalyzer/tests?label=tests)  -->
-![Python Version](https://img.shields.io/pypi/pyversions/pdfalyzer)
 [![GithubRelease](https://img.shields.io/github/v/release/michelcrypt4d4mus/pdfalyzer?sort=semver)](https://pypi.org/project/pdfalyzer/)
-[![GitHub last commit](https://img.shields.io/github/last-commit/michelcrypt4d4mus/pdfalyzer)](https://github.com/michelcrypt4d4mus/pdfalyzer)
 ![PyPiRelease](https://img.shields.io/pypi/v/pdfalyzer)
+[![GitHub last commit](https://img.shields.io/github/last-commit/michelcrypt4d4mus/pdfalyzer)](https://github.com/michelcrypt4d4mus/pdfalyzer)
+![Python Version](https://img.shields.io/pypi/pyversions/pdfalyzer)
 ![Downloads](https://img.shields.io/pypi/dm/pdfalyzer)
+[![Tests](https://github.com/michelcrypt4d4mus/pdfalyzer/actions/workflows/python-package.yml/badge.svg)](https://github.com/michelcrypt4d4mus/pdfalyzer/actions/workflows/python-package.yml)
 
 
 # THE PDFALYZER
@@ -211,9 +211,27 @@ Things like, say, a hidden binary `/F` (PDF instruction meaning "URL") followed
 
 -------------
 
+
 # PDF Resources
 ## Included PDF Tools
-The Pdfalyzer ships with a command line tool `combine_pdfs` that combines multiple PDFs into a single PDF. Run `combine_pdfs --help` to see the options.
+The Pdfalyzer comes with a few command line tools:
+
+#### `combine_pdfs`
+Combines multiple PDFs into a single PDF. Run `combine_pdfs --help` to see the options.
+
+#### `extract_pdf_pages`
+Extracts page ranges (e.g. "10-25") from a PDF and writes them to a new PDF. Run `extract_pdf_pages --help` to see the options.
+![](doc/extract_pages_from_pdf_help.png)
+
+#### `extract_text_from_pdfs`
+Extracts text from a PDF, including applying OCR to all embedded images. Requires that you install The Pdfalyzer's optional dependencies:
+
+```bash
+pipx install pdfalyzer[extract]
+```
+
+Run `extract_text_from_pdfs --help` to see the options.
+
 
 ## 3rd Party PDF Tools
 ### Installing Didier Stevens's PDF Analysis Tools

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/pdfalyzer/__init__.py

@@ -1,5 +1,6 @@
 import code
 import sys
+from argparse import Namespace
 from os import environ, getcwd, path
 
 from dotenv import load_dotenv
@@ -24,13 +25,14 @@ from yaralyzer.output.file_export import invoke_rich_export
 from yaralyzer.output.rich_console import console
 from yaralyzer.util.logging import log_and_print
 
+from pdfalyzer.decorators.pdf_file import PdfFile
 from pdfalyzer.helpers.filesystem_helper import file_size_in_mb, set_max_open_files
 from pdfalyzer.helpers.rich_text_helper import print_highlighted
 from pdfalyzer.output.pdfalyzer_presenter import PdfalyzerPresenter
 from pdfalyzer.output.styles.rich_theme import PDFALYZER_THEME_DICT
 from pdfalyzer.pdfalyzer import Pdfalyzer
 from pdfalyzer.util.argument_parser import (MAX_QUALITY, ask_to_proceed, output_sections, parse_arguments,
-     parse_combine_pdfs_args)
+     parse_combine_pdfs_args, parse_pdf_page_extraction_args, parse_text_extraction_args)
 from pdfalyzer.util.pdf_parser_manager import PdfParserManager
 
 # For the table shown by running pdfalyzer_show_color_theme
@@ -132,7 +134,19 @@ def combine_pdfs():
     print_highlighted(txt)
 
 
-# TODO: migrate this functionality from clown_sort
-# def extract_pages_from_pdf() -> None:
-#     args = parse_pdf_page_extraction_args()
-#     PdfFile(args.pdf_file).extract_page_range(args.page_range, destination_dir=args.destination_dir)
+def extract_pdf_pages() -> None:
+    args = parse_pdf_page_extraction_args()
+    PdfFile(args.pdf_file).extract_page_range(args.page_range, destination_dir=args.destination_dir)
+
+
+def extract_text_from_pdfs() -> None:
+    """
+    Extract text from a single file or from all files in a given directory. Can accept
+    multiple paths as arguments on the command line.
+    """
+    args: Namespace = parse_text_extraction_args()
+    console.line()
+
+    for file_path in args.files_to_process:
+        PdfFile(file_path).print_extracted_text(args.page_range, args.print_as_parsed)
+        console.line(2)

pdfalyzer-1.17.0/pdfalyzer/decorators/pdf_file.py

@@ -0,0 +1,212 @@
+import io
+from logging import Logger
+from os import path
+from pathlib import Path
+from typing import List, Optional, Union
+
+from pypdf import PdfReader, PdfWriter
+from pypdf.errors import DependencyError, EmptyFileError, PdfStreamError
+from rich.console import Console
+from rich.markup import escape
+from rich.panel import Panel
+from rich.text import Text
+from yaralyzer.output.rich_console import console
+from yaralyzer.util.logging import log as yaralyzer_log
+
+from pdfalyzer.helpers.filesystem_helper import create_dir_if_it_does_not_exist, insert_suffix_before_extension
+from pdfalyzer.helpers.image_helper import ocr_text
+from pdfalyzer.helpers.rich_text_helper import (attention_getting_panel, error_text, mild_warning,
+     print_error, stderr_console)
+from pdfalyzer.helpers.string_helper import exception_str
+from pdfalyzer.util.page_range import PageRange
+
+DEFAULT_PDF_ERRORS_DIR = Path.cwd().joinpath('pdf_errors')
+MIN_PDF_SIZE_TO_LOG_PROGRESS_TO_STDERR = 1024 * 1024 * 20
+
+
+class PdfFile:
+    """
+    Wrapper for a PDF file path that provides useful methods and properties.
+
+    Attributes:
+        file_path (Path): The path to the PDF file.
+        dirname (Path): The directory containing the PDF file.
+        basename (str): The base name of the PDF file (with extension).
+        basename_without_ext (str): The base name of the PDF file (without extension).
+        extname (str): The file extension of the PDF file.
+    """
+
+    def __init__(self, file_path: Union[str, Path]) -> None:
+        """
+        Args:
+            file_path (Union[str, Path]): Path to the PDF file.
+        """
+        self.file_path: Path = Path(file_path)
+
+        if not self.file_path.exists():
+            raise FileNotFoundError(f"File '{file_path}' does not exist.")
+
+        self.dirname = self.file_path.parent
+        self.basename: str = path.basename(file_path)
+        self.basename_without_ext: str = str(Path(self.basename).with_suffix(''))
+        self.extname: str = self.file_path.suffix
+        self.file_size = self.file_path.stat().st_size
+
+    def extract_page_range(
+            self,
+            page_range: PageRange,
+            destination_dir: Optional[Path] = None,
+            extra_file_suffix: Optional[str] = None
+        ) -> Path:
+        """
+        Extract a range of pages to a new PDF file.
+
+        Args:
+            page_range (PageRange): Range of pages to extract.
+            destination_dir (Optional[Path]): Directory to save the new PDF file. Defaults to the same
+                directory as the source PDF.
+            extra_file_suffix (Optional[str]): An optional suffix to append to the new PDF's filename.
+                Defaults to the page range suffix.
+
+        Returns:
+            Path: The path to the newly created PDF file containing the extracted pages.
+        """
+        destination_dir = destination_dir or self.dirname
+        create_dir_if_it_does_not_exist(destination_dir)
+
+        if extra_file_suffix is None:
+            file_suffix = page_range.file_suffix()
+        else:
+            file_suffix = f"{page_range.file_suffix()}__{extra_file_suffix}"
+
+        extracted_pages_pdf_basename = insert_suffix_before_extension(self.file_path, file_suffix).name
+        extracted_pages_pdf_path = destination_dir.joinpath(extracted_pages_pdf_basename)
+        console.print(f"Extracting {page_range.file_suffix()} from '{self.file_path}' to '{extracted_pages_pdf_path}'...")
+        pdf_writer = PdfWriter()
+
+        with open(self.file_path, 'rb') as source_pdf:
+            pdf_writer.append(fileobj=source_pdf, pages=page_range.to_tuple())
+
+            with open(extracted_pages_pdf_path, 'wb') as extracted_pages_pdf:
+                pdf_writer.write(extracted_pages_pdf)
+
+        console.print(f"Extracted pages to new PDF: '{extracted_pages_pdf_path}'.")
+        return extracted_pages_pdf_path
+
+    def extract_text(
+            self,
+            page_range: Optional[PageRange] = None,
+            logger: Optional[Logger] = None,
+            print_as_parsed: bool = False
+        ) -> Optional[str]:
+        """
+        Use PyPDF to extract text page by page and use Tesseract to OCR any embedded images.
+
+        Args:
+            page_range (Optional[PageRange]): If provided, only extract text from pages in this range.
+                Page numbers are 1-indexed. If not provided, extract text from all pages.
+            log (Optional[Logger]): If provided, log progress to this logger. Otherwise use default logger.
+            print_as_parsed (bool): If True, print each page's text to STDOUT as it is parsed.
+
+        Returns:
+            Optional[str]: The extracted text, or None if extraction failed.
+        """
+        from PIL import Image  # Imported here to avoid hard dependency if not using this method
+        log = logger or yaralyzer_log
+        log.debug(f"Extracting text from '{self.file_path}'...")
+        self._page_numbers_of_errors: List[int] = []
+        extracted_pages = []
+
+        try:
+            pdf_reader = PdfReader(self.file_path)
+            page_count = len(pdf_reader.pages)
+            log.debug(f"PDF Page count: {page_count}")
+
+            for page_number, page in enumerate(pdf_reader.pages, start=1):
+                if page_range and not page_range.in_range(page_number):
+                    self._log_to_stderr(f"Skipping page {page_number}...")
+                    continue
+
+                self._log_to_stderr(f"Parsing page {page_number}...")
+                page_buffer = Console(file=io.StringIO())
+                page_buffer.print(Panel(f"PAGE {page_number}", padding=(0, 15), expand=False))
+                page_buffer.print(escape(page.extract_text().strip()))
+                image_number = 1
+
+                # Extracting images is a bit fraught (lots of PIL and pypdf exceptions have come from here)
+                try:
+                    for image_number, image in enumerate(page.images, start=1):
+                        image_name = f"Page {page_number}, Image {image_number}"
+                        self._log_to_stderr(f"   Processing {image_name}...", "dim")
+                        page_buffer.print(Panel(image_name, expand=False))
+                        image_obj = Image.open(io.BytesIO(image.data))
+                        image_text = ocr_text(image_obj, f"{self.file_path} ({image_name})")
+                        page_buffer.print((image_text or '').strip())
+                except (OSError, NotImplementedError, TypeError, ValueError) as e:
+                    error_str = exception_str(e)
+                    msg = f"{error_str} while parsing embedded image {image_number} on page {page_number}..."
+                    mild_warning(msg)
+
+                    # Dump an error PDF and encourage user to report to pypdf team.
+                    if 'JBIG2Decode' not in str(e):
+                        stderr_console.print_exception()
+
+                        if page_number not in self._page_numbers_of_errors:
+                            self._handle_extraction_error(page_number, error_str)
+                            self._page_numbers_of_errors.append(page_number)
+
+                page_text = page_buffer.file.getvalue()
+                extracted_pages.append(page_text)
+                log.debug(page_text)
+
+                if print_as_parsed:
+                    print(f"{page_text}")
+        except DependencyError:
+            log.error("Pdfalyzer is missing an optional dependency required to extract text. Try 'pip install pdfalyzer[extract]'")
+        except EmptyFileError:
+            log.warning("Skipping empty file!")
+        except PdfStreamError as e:
+            print_error(f"Error parsing PDF file '{self.file_path}': {e}")
+            stderr_console.print_exception()
+
+        return "\n\n".join(extracted_pages).strip()
+
+    def print_extracted_text(self, page_range: Optional[PageRange] = None, print_as_parsed: bool = False) -> None:
+        """Fancy wrapper for printing the extracted text to the screen."""
+        console.print(Panel(str(self.file_path), expand=False, style='bright_white reverse'))
+        txt = self.extract_text(page_range=page_range, print_as_parsed=print_as_parsed)
+
+        if not print_as_parsed:
+            console.print(txt)
+
+    def _handle_extraction_error(self, page_number: int, error_msg: str) -> None:
+        """Rip the offending page to a new file and suggest that user report bug to PyPDF."""
+        destination_dir = DEFAULT_PDF_ERRORS_DIR
+
+        try:
+            extracted_file = self.extract_page_range(PageRange(str(page_number)), destination_dir, error_msg)
+        except Exception as e:
+            stderr_console.print(error_text(f"Failed to extract a page for submission to PyPDF team."))
+            extracted_file = None
+
+        blink_txt = Text('', style='bright_white')
+        blink_txt.append("An error (", style='blink color(154)').append(error_msg, style='color(11) blink')
+        blink_txt.append(') ', style='blink color(154)')
+        blink_txt.append("was encountered while processing a PDF file.\n\n", style='blink color(154)')
+
+        txt = Text(f"The error was of a type such that it probably came from a bug in ", style='bright_white')
+        txt.append('PyPDF', style='underline bright_green').append('. It was encountered processing the file ')
+        txt.append(str(self.file_path), style='file').append('. You should see a stack trace above this box.\n\n')
+
+        txt.append('The offending page will be extracted to ', style='bright_white')
+        txt.append(str(extracted_file), style='file').append('.\n\n')
+        txt.append(f"Please visit 'https://github.com/py-pdf/pypdf/issues' to report a bug. ", style='bold')
+        txt.append(f"Providing the devs with the extracted page and the stack trace help improve pypdf.")
+        stderr_console.print(attention_getting_panel(blink_txt + txt, title='PyPDF Error'))
+
+    def _log_to_stderr(self, msg: str, style: Optional[str] = None) -> None:
+        """When parsing very large PDFs it can be useful to log progress and other messages to STDERR."""
+        if self.file_size < MIN_PDF_SIZE_TO_LOG_PROGRESS_TO_STDERR:
+            return
+
+        stderr_console.print(msg, style=style or "")

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/pdfalyzer/helpers/filesystem_helper.py

@@ -6,6 +6,7 @@ from pathlib import Path
 from typing import Optional, Union
 
 from yaralyzer.output.rich_console import console
+from yaralyzer.util.logging import log
 
 from pdfalyzer.helpers.rich_text_helper import print_highlighted
 
@@ -18,9 +19,20 @@ PDF_EXT = '.pdf'
 # type StrOrPath = Union[str, Path]
 
 
-def with_pdf_extension(file_path: Union[str, Path]) -> str:
-    """Append '.pdf' to 'file_path' if it doesn't already end with '.pdf'."""
-    return str(file_path) + ('' if is_pdf(file_path) else PDF_EXT)
+def create_dir_if_it_does_not_exist(dir: Path) -> None:
+    """Like it says on the tin."""
+    if dir.exists():
+        return
+
+    console.warning(f"Need to create '{dir}'")
+    dir.mkdir(parents=True, exist_ok=True)
+
+
+def insert_suffix_before_extension(file_path: Path, suffix: str, separator: str = '__') -> Path:
+    """Inserting 'page 1' suffix in 'path/to/file.jpg' -> '/path/to/file__page_1.jpg'."""
+    suffix = strip_bad_chars(suffix).replace(' ', '_')
+    file_path_without_extension = file_path.with_suffix('')
+    return Path(f"{file_path_without_extension}{separator}{suffix}{file_path.suffix}")
 
 
 def is_pdf(file_path: Union[str, Path]) -> bool:
@@ -100,3 +112,15 @@ def set_max_open_files(num_filehandles: int = DEFAULT_MAX_OPEN_FILES) -> tuple[O
                 soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
 
     return (soft, hard)
+
+
+def strip_bad_chars(text: str) -> str:
+    """Remove chars that don't work well in filenames."""
+    text = ' '.join(text.splitlines()).replace('\\s+', ' ')
+    text = re.sub('’', "'", text).replace('|', 'I').replace(',', ',')
+    return re.sub('[^-0-9a-zA-Z@.,?_:=#\'\\$" ()]+', '_', text).replace('  ', ' ')
+
+
+def with_pdf_extension(file_path: Union[str, Path]) -> str:
+    """Append `".pdf"` to `file_path` if it doesn't already end with `".pdf"`."""
+    return str(file_path) + ('' if is_pdf(file_path) else PDF_EXT)

pdfalyzer-1.17.0/pdfalyzer/helpers/image_helper.py

@@ -0,0 +1,31 @@
+from typing import Optional
+
+from PIL import Image
+from yaralyzer.output.rich_console import console
+
+from pdfalyzer.helpers.rich_text_helper import warning_text
+
+
+def ocr_text(image: Image.Image, image_name: str) -> Optional[str]:
+    """Use pytesseract to OCR the text in the image and return it as a string."""
+    import pytesseract
+    text = None
+
+    try:
+        text = pytesseract.image_to_string(image)
+    except pytesseract.pytesseract.TesseractError as e:
+        console.print_exception()
+        console.print(warning_text(f"Tesseract OCR failure '{image_name}'! No OCR text extracted..."))
+    except OSError as e:
+        if 'truncated' in str(e):
+            console.print(warning_text(f"Truncated image file '{image_name}'!"))
+        else:
+            console.print_exception()
+            console.print(f"Error while extracting '{image_name}'!", style='bright_red')
+            raise e
+    except Exception as e:
+        console.print_exception()
+        console.print(f"Error while extracting '{image_name}'!", style='bright_red')
+        raise e
+
+    return None if text is None else text.strip()

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/pdfalyzer/helpers/rich_text_helper.py

@@ -1,17 +1,24 @@
 """
 Functions for miscellaneous Rich text/string pretty printing operations.
 """
-from typing import List, Union
+from sys import stderr
+from typing import List, Optional, Union
 
 from pypdf.generic import PdfObject
 from rich.console import Console
+from rich.panel import Panel
+from rich.padding import Padding
 from rich.text import Text
+from yaralyzer.output.rich_console import console
 
 from pdfalyzer.helpers.pdf_object_helper import pypdf_class_name
 from pdfalyzer.output.styles.node_colors import get_label_style, get_class_style_italic
 
+ARROW_BULLET = '➤ '
+
 # Usually we use the yaralyzer console but that has no highlighter
 pdfalyzer_console = Console(color_system='256')
+stderr_console = Console(color_system='256', file=stderr)
 
 
 def print_highlighted(msg: Union[str, Text], **kwargs) -> None:
@@ -32,6 +39,21 @@ def quoted_text(
     return txt
 
 
+def indented_bullet(msg: Union[str, Text], style: Optional[str] = None) -> Text:
+    return Text('  ') + bullet_text(msg, style)
+
+
+def bullet_text(msg: Union[str, Text], style: Optional[str] = None) -> Text:
+    if isinstance(msg, str):
+        msg = Text(msg, style=style)
+
+    return Text(ARROW_BULLET).append(msg)
+
+
+def mild_warning(msg: str) -> None:
+    console.print(indented_bullet(Text(msg, style='mild_warning')))
+
+
 def node_label(idnum: int, label: str, pdf_object: PdfObject, underline: bool = True) -> Text:
     """Colored text representation of a PDF node. Example: <5:FontDescriptor(Dictionary)>."""
     text = Text('<', style='white')
@@ -58,3 +80,30 @@ def pct_txt(_number: int, total: int, digits: int = 1) -> Text:
     """Return nicely formatted percentage, e.g. '(80%)'."""
     pct = (100 * float(_number) / float(total)).__round__(digits)
     return Text(f"({pct}%)", style='blue')
+
+
+def warning_text(text: Union[str, Text]) -> Text:
+    msg = Text('').append(f"WARNING", style='bright_yellow').append(": ")
+
+    if isinstance(text, Text):
+        return msg + text
+    else:
+        return msg.append(text)
+
+
+def error_text(text: Union[str, Text]) -> Text:
+    msg = Text('').append(f"ERROR", style='bright_red').append(": ")
+
+    if isinstance(text, Text):
+        return msg + text
+    else:
+        return msg.append(text)
+
+
+def attention_getting_panel(text: Text, title: str, style: str = 'white on red') -> Padding:
+    p = Panel(text, padding=(2), title=title, style=style)
+    return Padding(p, pad=(1, 10, 2, 10))
+
+
+def print_error(text: Union[str, Text]) -> Text:
+    console.print(error_text(text))

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/pdfalyzer/helpers/string_helper.py

@@ -35,10 +35,15 @@ def count_pattern_matches_in_text(pattern: str, text: str) -> int:
 
 
 def count_regex_matches_in_text(regex: Pattern, text: str) -> int:
-    """For use when you precompile the regex"""
+    """For use when you precompile the regex."""
     return sum(1 for _ in regex.finditer(text))
 
 
+def exception_str(e: Exception) -> str:
+    """A string with the type and message."""
+    return f"{type(e).__name__}: {e}"
+
+
 def root_address(_string: str) -> str:
     """Strip the bracketed part off an address, e.g. '/Root[1]' => '/Root'."""
     return _string.split('[')[0]

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/pdfalyzer/output/styles/rich_theme.py

@@ -51,7 +51,8 @@ PDFALYZER_THEME_DICT.update({
     'warn.harsh': 'reverse bright_yellow',
     # error log events
     'fail': 'bold reverse red',
-    'milderror': 'red',
+    'mild_error': 'red',  # TODO: unused?
+    'mild_warning': 'color(228) dim',
     'red_alert': 'blink bold red reverse on white',
 })
 

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/pdfalyzer/util/argument_parser.py

@@ -7,19 +7,23 @@ from collections import namedtuple
 from functools import partial, update_wrapper
 from importlib.metadata import version
 from os import getcwd, path
+from pathlib import Path
 from typing import List, Optional
 
 from rich_argparse_plus import RichHelpFormatterPlus
 from rich.prompt import Confirm
 from rich.text import Text
+from yaralyzer.helpers.file_helper import files_in_dir
 from yaralyzer.util.argument_parser import export, parser, parse_arguments as parse_yaralyzer_args, source
 from yaralyzer.util.logging import log, log_and_print, log_argparse_result, log_current_config, log_invocation
 
+
 from pdfalyzer.config import ALL_STREAMS, PDFALYZER, PdfalyzerConfig
 from pdfalyzer.detection.constants.binary_regexes import QUOTE_PATTERNS
 from pdfalyzer.helpers.filesystem_helper import (do_all_files_exist, extract_page_number, file_exists, is_pdf,
      with_pdf_extension)
 from pdfalyzer.helpers.rich_text_helper import print_highlighted
+from pdfalyzer.util.page_range import PageRangeArgumentValidator
 
 # NamedTuple to keep our argument selection orderly
 OutputSection = namedtuple('OutputSection', ['argument', 'method'])
@@ -258,6 +262,90 @@ def parse_combine_pdfs_args() -> Namespace:
     return args
 
 
+###########################################
+# Parse args for extract_pdf_pages() #
+###########################################
+page_range_validator = PageRangeArgumentValidator()
+
+extract_pdf_parser = ArgumentParser(
+    formatter_class=RichHelpFormatterPlus,
+    description="Extract pages from one PDF into a new PDF.",
+)
+
+extract_pdf_parser.add_argument('pdf_file', metavar='PDF_FILE', help='PDF to extract pages from')
+
+extract_pdf_parser.add_argument('--page-range', '-r',
+                                type=page_range_validator,
+                                help=page_range_validator.HELP_MSG,
+                                required=True)
+
+extract_pdf_parser.add_argument('--destination-dir', '-d',
+                                help="directory to write the new PDF to",
+                                default=Path.cwd())
+
+extract_pdf_parser.add_argument('--debug', action='store_true', help='turn on debug level logging')
+
+
+def parse_pdf_page_extraction_args() -> Namespace:
+    args = extract_pdf_parser.parse_args()
+
+    if not is_pdf(args.pdf_file):
+        log.error(f"'{args.pdf_file}' is not a PDF.")
+        sys.exit(-1)
+    elif not Path(args.destination_dir).exists():
+        log.error(f"Destination dir '{args.destination_dir}' does not exist.")
+        sys.exit(1)
+
+    return args
+
+
+############################################
+# Parse args for extract_text_from_pdfs() #
+############################################
+extract_text_parser = ArgumentParser(
+    formatter_class=RichHelpFormatterPlus,
+    description="Extract the text from one or more files or directories.",
+    epilog="If any of the FILE_OR_DIRs is a directory all files in that directory will be extracted."
+)
+
+extract_text_parser.add_argument('file_or_dir', nargs='+', metavar='FILE_OR_DIR')
+extract_text_parser.add_argument('--debug', action='store_true', help='turn on debug level logging')
+
+extract_text_parser.add_argument('--page-range', '-r',
+                                 type=page_range_validator,
+                                 help=f"[PDFs only] {page_range_validator.HELP_MSG}")
+
+extract_text_parser.add_argument('--print-as-parsed', '-p',
+                                 action='store_true',
+                                 help='print pages as they are parsed instead of waiting until document is fully parsed')
+
+
+def parse_text_extraction_args() -> Namespace:
+    args = extract_text_parser.parse_args()
+    args.files_to_process = []
+
+    for file_or_dir in args.file_or_dir:
+        file_path = Path(file_or_dir)
+
+        if not file_path.exists():
+            log.error(f"File '{file_path}' doesn't exist!")
+            sys.exit(-1)
+        elif file_path.is_dir():
+            args.files_to_process.extend(files_in_dir(file_path, 'pdf'))
+        else:
+            args.files_to_process.append(file_path)
+
+    if args.page_range and (len(args.files_to_process) > 1 or not is_pdf(args.files_to_process[0])):
+        log.error(f"--page-range can only be specified for a single PDF")
+        sys.exit(-1)
+
+    return args
+
+
+#############
+#  Helpers  #
+#############
+
 def ask_to_proceed() -> None:
     """Exit if user doesn't confirm they want to proceed."""
     if not Confirm.ask(Text("Proceed anyway?")):

pdfalyzer-1.17.0/pdfalyzer/util/page_range.py

@@ -0,0 +1,54 @@
+"""
+A range of page numbers. Copied from clown_sort repo.
+"""
+import re
+from argparse import ArgumentTypeError
+from dataclasses import dataclass
+from typing import Tuple
+
+PAGE_RANGE_REGEX = re.compile('\\d(-\\d)?')
+
+
+@dataclass
+class PageRange:
+    page_range: str
+
+    def __post_init__(self):
+        if not PAGE_RANGE_REGEX.match(self.page_range):
+            raise ValueError(f"Invalid page range '{self.page_range}'")
+
+        if '-' in self.page_range:
+            (self.first_page, self.last_page) = (int(p) for p in self.page_range.split('-'))
+        else:
+            self.first_page = int(self.page_range)
+            self.last_page = self.first_page + 1
+
+        if self.last_page <= self.first_page:
+            raise ValueError(f"Invalid page range {self.__repr__()}")
+
+    def in_range(self, page_number) -> bool:
+        """Returns `True` if `page_number` is in this range."""
+        return page_number >= self.first_page and page_number < self.last_page
+
+    def file_suffix(self) -> str:
+        """String that can be used as file suffix."""
+        if self.first_page + 1 == self.last_page:
+            return f"page_{self.first_page}"
+        else:
+            return f"pages_{self.first_page}-{self.last_page}"
+
+    def to_tuple(self) -> Tuple[int, int]:
+        return (self.first_page, self.last_page)
+
+    def __repr__(self) -> str:
+        return f"PageRange({self.first_page}, {self.last_page})"
+
+
+class PageRangeArgumentValidator(object):
+    HELP_MSG = "a single digit ('11') or a range ('11-15') (WILL NOT extract the last page)"
+
+    def __call__(self, value):
+        if not PAGE_RANGE_REGEX.match(value):
+            raise ArgumentTypeError("Argument has to match '{}'".format(PAGE_RANGE_REGEX.pattern))
+
+        return PageRange(value)

{pdfalyzer-1.16.14 → pdfalyzer-1.17.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "pdfalyzer"
-version = "1.16.14"
+version = "1.17.0"
 description = "PDF analysis tool. Scan a PDF with YARA rules, visualize its inner tree-like data structure in living color (lots of colors), force decodes of suspicious font binaries, and more."
 authors = ["Michel de Cryptadamus <michel@cryptadamus.com>"]
 license = "GPL-3.0-or-later"
@@ -68,8 +68,13 @@ packages = [
 python = "^3.10"
 anytree = "~=2.13"
 pypdf = "^6.0.0"
+PyMuPDF = {version = "^1.26.4", optional = true}
+pytesseract = {version = "^0.3.13", optional = true}
 yaralyzer = "^1.0.9"
 
+[tool.poetry.extras]
+extract = ["PyMuPDF", "pytesseract"]
+
 [tool.poetry.group.dev.dependencies]
 flake8 = "^7.3.0"
 pytest = "^7.1.2"
@@ -81,6 +86,8 @@ pytest-skip-slow = "^0.0.3"
 #############
 [tool.poetry.scripts]
 combine_pdfs = 'pdfalyzer:combine_pdfs'
+extract_pdf_pages = 'pdfalyzer:extract_pdf_pages'
+extract_text_from_pdfs = 'pdfalyzer:extract_text_from_pdfs'
 pdfalyze = 'pdfalyzer:pdfalyze'
 pdfalyzer_show_color_theme = 'pdfalyzer:pdfalyzer_show_color_theme'
 
@@ -107,3 +114,8 @@ requires = ["poetry-core>=1.0.0"]
 addopts = [
     "--import-mode=importlib",
 ]
+
+
+# Poetry 2.x handles optional dependencies like this:
+# [project.optional-dependencies]
+# extract = ["PyMuPDF", "pytesseract"]

pdfalyzer-1.16.14/pdfalyzer/decorators/pdf_file.py

@@ -1,50 +0,0 @@
-from os import path
-from pathlib import Path
-from typing import List, Optional, Union
-
-
-class PdfFile:
-    """
-    Wrapper for a PDF file path that provides useful methods and properties.
-    """
-    def __init__(self, file_path: Union[str, Path]) -> None:
-        self.file_path: Path = Path(file_path)
-
-        if not self.file_path.exists():
-            raise FileNotFoundError(f"File '{file_path}' does not exist.")
-
-        self.dirname = self.file_path.parent
-        self.basename: str = path.basename(file_path)
-        self.basename_without_ext: str = str(Path(self.basename).with_suffix(''))
-        self.extname: str = self.file_path.suffix
-        self.text_extraction_attempted: bool = False
-
-    def extract_page_range(
-            self,
-            page_range: PageRange,
-            destination_dir: Optional[Path] = None,
-            extra_file_suffix: Optional[str] = None
-        ) -> Path:
-        """Extract a range of pages to a new PDF file (or 1 page if last_page_number not provided.)"""
-        destination_dir = destination_dir or DEFAULT_PDF_ERRORS_DIR
-        create_dir_if_it_does_not_exist(destination_dir)
-
-        if extra_file_suffix is None:
-            file_suffix = page_range.file_suffix()
-        else:
-            file_suffix = f"{page_range.file_suffix()}__{extra_file_suffix}"
-
-        extracted_pages_pdf_basename = insert_suffix_before_extension(self.file_path, file_suffix).name
-        extracted_pages_pdf_path = destination_dir.joinpath(extracted_pages_pdf_basename)
-        stderr_console.print(f"Extracting {page_range.file_suffix()} from '{self.file_path}' to '{extracted_pages_pdf_path}'...")
-        pdf_writer = PdfWriter()
-
-        with open(self.file_path, 'rb') as source_pdf:
-            pdf_writer.append(fileobj=source_pdf, pages=page_range.to_tuple())
-
-        if SortableFile.confirm_file_overwrite(extracted_pages_pdf_path):
-            with open(extracted_pages_pdf_path, 'wb') as extracted_pages_pdf:
-                pdf_writer.write(extracted_pages_pdf)
-
-        stderr_console.print(f"Wrote new PDF '{extracted_pages_pdf_path}'.")
-        return extracted_pages_pdf_path