yaralyzer 1.0.8__tar.gz → 1.0.9__tar.gz

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/CHANGELOG.md

@@ -1,5 +1,8 @@
 # NEXT RELEASE
 
+### 1.0.9
+* Raise `FileNotFoundError` instead of `ValueError` if provided YARA rules files or dirs don't exist
+
 ### 1.0.8
 * Bump `python-dotenv` to v1.1.1
 * Use `mkdocs` and `lazydocs` to build automatic API documentation at https://michelcrypt4d4mus.github.io/yaralyzer/

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: yaralyzer
-Version: 1.0.8
+Version: 1.0.9
 Summary: Visualize and force decode YARA and regex matches found in a file or byte stream with colors. Lots of colors.
 Home-page: https://github.com/michelcrypt4d4mus/yaralyzer
 License: GPL-3.0-or-later
@@ -78,7 +78,7 @@ YARA just tells you the byte position and the matched string but it can't tell y
 
 Enter **The Yaralyzer**, which lets you quickly scan the regions around matches while also showing you what those regions would look like if they were forced into various character encodings.
 
-It's important to note that **The Yaralyzer** isn't a full on malware reversing tool. It can't do all the things a tool like [CyberChef](https://gchq.github.io/CyberChef/) does and it doesn't try to. It's more intended to give you a quick visual overview of suspect regions in the binary so you can hone in on the areas you might want to inspect with a more serious tool like [CyberChef](https://gchq.github.io/CyberChef/).
+**The Yaralyzer** isn't a malware reversing tool. It can't do all the things a tool like [CyberChef](https://gchq.github.io/CyberChef/) does and it doesn't try to. It's more intended to give you a quick visual overview of suspect regions in the binary so you can hone in on the areas you might want to inspect with a more serious tool like [CyberChef](https://gchq.github.io/CyberChef/).
 
 # Installation
 Install it with [`pipx`](https://pypa.github.io/pipx/) or `pip3`. `pipx` is a marginally better solution as it guarantees any packages installed with it will be isolated from the rest of your local python environment. Of course if you don't really have a local python environment this is a moot point and you can feel free to install with `pip`/`pip3`.
@@ -86,6 +86,7 @@ Install it with [`pipx`](https://pypa.github.io/pipx/) or `pip3`. `pipx` is a ma
 pipx install yaralyzer
 ```
 
+
 # Usage
 Run `yaralyze -h` to see the command line options (screenshot below).
 
@@ -99,7 +100,7 @@ If you place a file called `.yaralyzer` in your home directory or the current wo
 Only one `.yaralyzer` file will be loaded and the working directory's `.yaralyzer` takes precedence over the home directory's `.yaralyzer`.
 
 ### As A Library
-[`Yaralyzer`](yaralyzer/yaralyzer.py) is the main class. It has a variety of constructors supporting:
+[`Yaralyzer`](yaralyzer/yaralyzer.py) is the main class. Auto generated documentation for `Yaralyzer`'s various classes and methods can be found [here](https://michelcrypt4d4mus.github.io/yaralyzer/). It has a variety of [alternate constructors](https://michelcrypt4d4mus.github.io/yaralyzer/api/yaralyzer/) supporting:
 
 1. Precompiled YARA rules
 1. Creating a YARA rule from a string
@@ -108,7 +109,7 @@ Only one `.yaralyzer` file will be loaded and the working directory's `.yaralyze
 1. Scanning `bytes`
 1. Scanning a file
 
-Should you want to iterate over the `BytesMatch` (like a `re.Match` object for a YARA match) and `BytesDecoder` (tracks decoding attempt stats) objects returned by The Yaralyzer, you can do so like this:
+Should you want to iterate over the [`BytesMatch`](https://michelcrypt4d4mus.github.io/yaralyzer/api/bytes_match/) (like a `re.Match` object for a YARA match) and [`BytesDecoder`](https://michelcrypt4d4mus.github.io/yaralyzer/api/bytes_decoder/) (tracks decoding attempt stats) objects used by The Yaralyzer, you can do so like this:
 
 ```python
 from yaralyzer.yaralyzer import Yaralyzer
@@ -119,8 +120,6 @@ for bytes_match, bytes_decoder in yaralyzer.match_iterator():
     do_stuff()
 ```
 
-#### API Documentation
-Auto generated documentation for Yaralyzer's various classes and methods can be found [here](https://michelcrypt4d4mus.github.io/yaralyzer/).
 
 # Example Output
 The Yaralyzer can export visualizations to HTML, ANSI colored text, and SVG vector images using the file export functionality that comes with [Rich](https://github.com/Textualize/rich) as well as a (somewhat limited) plain text JSON format. SVGs can be turned into `png` format images with a tool like [Inkscape](https://inkscape.org/) or `cairosvg`. In our experience they both work though we've seen some glitchiness with `cairosvg`.

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/README.md

@@ -45,7 +45,7 @@ YARA just tells you the byte position and the matched string but it can't tell y
 
 Enter **The Yaralyzer**, which lets you quickly scan the regions around matches while also showing you what those regions would look like if they were forced into various character encodings.
 
-It's important to note that **The Yaralyzer** isn't a full on malware reversing tool. It can't do all the things a tool like [CyberChef](https://gchq.github.io/CyberChef/) does and it doesn't try to. It's more intended to give you a quick visual overview of suspect regions in the binary so you can hone in on the areas you might want to inspect with a more serious tool like [CyberChef](https://gchq.github.io/CyberChef/).
+**The Yaralyzer** isn't a malware reversing tool. It can't do all the things a tool like [CyberChef](https://gchq.github.io/CyberChef/) does and it doesn't try to. It's more intended to give you a quick visual overview of suspect regions in the binary so you can hone in on the areas you might want to inspect with a more serious tool like [CyberChef](https://gchq.github.io/CyberChef/).
 
 # Installation
 Install it with [`pipx`](https://pypa.github.io/pipx/) or `pip3`. `pipx` is a marginally better solution as it guarantees any packages installed with it will be isolated from the rest of your local python environment. Of course if you don't really have a local python environment this is a moot point and you can feel free to install with `pip`/`pip3`.
@@ -53,6 +53,7 @@ Install it with [`pipx`](https://pypa.github.io/pipx/) or `pip3`. `pipx` is a ma
 pipx install yaralyzer
 ```
 
+
 # Usage
 Run `yaralyze -h` to see the command line options (screenshot below).
 
@@ -66,7 +67,7 @@ If you place a file called `.yaralyzer` in your home directory or the current wo
 Only one `.yaralyzer` file will be loaded and the working directory's `.yaralyzer` takes precedence over the home directory's `.yaralyzer`.
 
 ### As A Library
-[`Yaralyzer`](yaralyzer/yaralyzer.py) is the main class. It has a variety of constructors supporting:
+[`Yaralyzer`](yaralyzer/yaralyzer.py) is the main class. Auto generated documentation for `Yaralyzer`'s various classes and methods can be found [here](https://michelcrypt4d4mus.github.io/yaralyzer/). It has a variety of [alternate constructors](https://michelcrypt4d4mus.github.io/yaralyzer/api/yaralyzer/) supporting:
 
 1. Precompiled YARA rules
 1. Creating a YARA rule from a string
@@ -75,7 +76,7 @@ Only one `.yaralyzer` file will be loaded and the working directory's `.yaralyze
 1. Scanning `bytes`
 1. Scanning a file
 
-Should you want to iterate over the `BytesMatch` (like a `re.Match` object for a YARA match) and `BytesDecoder` (tracks decoding attempt stats) objects returned by The Yaralyzer, you can do so like this:
+Should you want to iterate over the [`BytesMatch`](https://michelcrypt4d4mus.github.io/yaralyzer/api/bytes_match/) (like a `re.Match` object for a YARA match) and [`BytesDecoder`](https://michelcrypt4d4mus.github.io/yaralyzer/api/bytes_decoder/) (tracks decoding attempt stats) objects used by The Yaralyzer, you can do so like this:
 
 ```python
 from yaralyzer.yaralyzer import Yaralyzer
@@ -86,8 +87,6 @@ for bytes_match, bytes_decoder in yaralyzer.match_iterator():
     do_stuff()
 ```
 
-#### API Documentation
-Auto generated documentation for Yaralyzer's various classes and methods can be found [here](https://michelcrypt4d4mus.github.io/yaralyzer/).
 
 # Example Output
 The Yaralyzer can export visualizations to HTML, ANSI colored text, and SVG vector images using the file export functionality that comes with [Rich](https://github.com/Textualize/rich) as well as a (somewhat limited) plain text JSON format. SVGs can be turned into `png` format images with a tool like [Inkscape](https://inkscape.org/) or `cairosvg`. In our experience they both work though we've seen some glitchiness with `cairosvg`.

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "yaralyzer"
-version = "1.0.8"
+version = "1.0.9"
 description = "Visualize and force decode YARA and regex matches found in a file or byte stream with colors. Lots of colors."
 authors = ["Michel de Cryptadamus <michel@cryptadamus.com>"]
 readme = "README.md"

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/yaralyzer/__init__.py

@@ -27,7 +27,7 @@ def yaralyze():
     """
     Entry point for yaralyzer when invoked as a script.
 
-    Args are parsed from the command line and environment variables. See yaralyzer --help for details.
+    Args are parsed from the command line and environment variables. See `yaralyze --help` for details.
     """
     args = parse_arguments()
     output_basepath = None
@@ -55,13 +55,10 @@ def yaralyze():
 
     if args.export_txt:
         invoke_rich_export(console.save_text, output_basepath)
-
     if args.export_html:
         invoke_rich_export(console.save_html, output_basepath)
-
     if args.export_svg:
         invoke_rich_export(console.save_svg, output_basepath)
-
     if args.export_json:
         export_json(yaralyzer, output_basepath)
 

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/yaralyzer/bytes_match.py

@@ -1,4 +1,6 @@
-"""BytesMatch class for tracking regex and YARA matches against binary data."""
+"""
+`BytesMatch` class for tracking regex and YARA matches against binary data.
+"""
 import re
 from typing import Iterator, Optional
 
@@ -16,11 +18,8 @@ class BytesMatch:
     """
     Simple class to keep track of regex matches against binary data.
 
-    Basically an re.match object with some (not many) extra bells and whistles, most notably
-    the surrounding_bytes property.
-
-    pre_capture_len and post_capture_len refer to the regex sections before and after the capture group,
-    e.g. a regex like '123(.*)x:' would have pre_capture_len of 3 and post_capture_len of 2.
+    Basically a Regex `re.match` object with some (not many) extra bells and whistles, most notably
+    the `surrounding_bytes` property.
     """
 
     def __init__(
@@ -34,15 +33,15 @@ class BytesMatch:
         highlight_style: str = YaralyzerConfig.HIGHLIGHT_STYLE
     ) -> None:
         """
-        Initialize a BytesMatch object representing a match against binary data.
+        Initialize a `BytesMatch` object representing a match against binary data.
 
         Args:
             matched_against (bytes): The full byte sequence that was searched.
             start_idx (int): Start index of the match in the byte sequence.
             length (int): Length of the match in bytes.
             label (str): Label for the match (e.g., regex or YARA rule name).
-            ordinal (int): The Nth match for this pattern.
-            match (Optional[re.Match]): Regex match object, if available.
+            ordinal (int): This was the Nth match for this pattern (used for labeling only).
+            match (Optional[re.Match]): Regex `match` object, if available.
             highlight_style (str): Style to use for highlighting the match.
         """
         self.matched_against: bytes = matched_against
@@ -71,16 +70,16 @@ class BytesMatch:
         highlight_style: str = YaralyzerConfig.HIGHLIGHT_STYLE
     ) -> 'BytesMatch':
         """
-        Create a BytesMatch from a regex match object.
+        Alternate constructor to build a `BytesMatch` from a regex match object.
 
         Args:
             matched_against (bytes): The bytes searched.
-            match (re.Match): The regex match object.
-            ordinal (int): The Nth match for this pattern.
+            match (re.Match): The regex `match` object.
+            ordinal (int): This was the Nth match for this pattern (used for labeling only).
             highlight_style (str): Style for highlighting.
 
         Returns:
-            BytesMatch: The constructed BytesMatch instance.
+            BytesMatch: The constructed `BytesMatch` instance.
         """
         return cls(matched_against, match.start(), len(match[0]), match.re.pattern, ordinal, match, highlight_style)
 
@@ -95,7 +94,7 @@ class BytesMatch:
         highlight_style: str = YaralyzerConfig.HIGHLIGHT_STYLE
     ) -> 'BytesMatch':
         """
-        Build a BytesMatch from a YARA string match instance.
+        Alternate constructor to build a `BytesMatch` from a YARA string match instance.
 
         Args:
             matched_against (bytes): The bytes searched.
@@ -132,7 +131,7 @@ class BytesMatch:
         highlight_style: str = YaralyzerConfig.HIGHLIGHT_STYLE
     ) -> Iterator['BytesMatch']:
         """
-        Yield a BytesMatch for each string returned as part of a YARA match result dict.
+        Yield a `BytesMatch` for each string returned as part of a YARA match result dict.
 
         Args:
             matched_against (bytes): The bytes searched.
@@ -160,7 +159,7 @@ class BytesMatch:
 
     def style_at_position(self, idx) -> str:
         """
-        Get the style for the byte at position idx within the matched bytes.
+        Get the style for the byte at position `idx` within the matched bytes.
 
         Args:
             idx (int): Index within the surrounding bytes.
@@ -175,7 +174,7 @@ class BytesMatch:
 
     def location(self) -> Text:
         """
-        Get a styled Text object describing the start and end index of the match.
+        Get a styled `Text` object describing the start and end index of the match.
 
         Returns:
             Text: Rich Text object like '(start idx: 348190, end idx: 348228)'.
@@ -196,11 +195,11 @@ class BytesMatch:
         """
         Determine if the matched bytes should be decoded.
 
-        Whether the bytes are decodable depends on whether SUPPRESS_DECODES_TABLE is set
-        and whether the match length is between MIN/MAX_DECODE_LENGTH.
+        Whether the bytes are decodable depends on whether `SUPPRESS_DECODES_TABLE` is set
+        and whether the match length is between `MIN`/`MAX_DECODE_LENGTH`.
 
         Returns:
-            bool: True if decodable, False otherwise.
+            bool: `True` if decodable, `False` otherwise.
         """
         return self.match_length >= YaralyzerConfig.args.min_decode_length \
            and self.match_length <= YaralyzerConfig.args.max_decode_length \
@@ -211,7 +210,7 @@ class BytesMatch:
         Build a table of MD5/SHA hashes for the matched bytes.
 
         Returns:
-            Table: Rich Table object with hashes.
+            Table: Rich `Table` object with hashes.
         """
         return bytes_hashes_table(
             self.bytes,
@@ -224,7 +223,7 @@ class BytesMatch:
         Generate a message for when the match is too short or too long to decode.
 
         Returns:
-            Text: Rich Text object with the suppression notice.
+            Text: Rich `Text` object with the suppression notice.
         """
         txt = self.__rich__()
 
@@ -238,7 +237,7 @@ class BytesMatch:
 
     def to_json(self) -> dict:
         """
-        Convert this BytesMatch to a JSON-serializable dictionary.
+        Convert this `BytesMatch` to a JSON-serializable dictionary.
 
         Returns:
             dict: Dictionary representation of the match, suitable for JSON serialization.
@@ -275,7 +274,7 @@ class BytesMatch:
         self.surrounding_bytes: bytes = self.matched_against[self.surrounding_start_idx:self.surrounding_end_idx]
 
     def __rich__(self) -> Text:
-        """Get a rich Text representation of the match for display."""
+        """Get a rich `Text` representation of the match for display."""
         headline = prefix_with_style(str(self.match_length), style='number', root_style='decode.subheading')
         headline.append(f" bytes matching ")
         headline.append(f"{self.label} ", style=ALERT_STYLE if self.highlight_style == ALERT_STYLE else 'regex')

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/yaralyzer/config.py

@@ -18,19 +18,20 @@ MEGABYTE = 1024 * KILOBYTE
 
 def config_var_name(env_var: str) -> str:
     """
-    Get the name of env_var and strip off 'YARALYZER_' prefix.
+    Get the name of `env_var` and strip off `YARALYZER_` prefix.
 
     Example:
-        $ SURROUNDING_BYTES_ENV_VAR = 'YARALYZER_SURROUNDING_BYTES'
-        $ config_var_name(SURROUNDING_BYTES_ENV_VAR) => 'SURROUNDING_BYTES'
-
+        ```
+        SURROUNDING_BYTES_ENV_VAR = 'YARALYZER_SURROUNDING_BYTES'
+        config_var_name(SURROUNDING_BYTES_ENV_VAR) => 'SURROUNDING_BYTES'
+        ```
     """
     env_var = env_var.removeprefix("YARALYZER_")
     return f'{env_var=}'.partition('=')[0]
 
 
-def is_env_var_set_and_not_false(var_name):
-    """Return True if var_name is not empty and set to anything other than 'false' (capitalization agnostic)."""
+def is_env_var_set_and_not_false(var_name: str) -> bool:
+    """Return `True` if `var_name` is not empty and set to anything other than "false" (capitalization agnostic)."""
     if var_name in environ:
         var_value = environ[var_name]
         return var_value is not None and len(var_value) > 0 and var_value.lower() != 'false'
@@ -38,8 +39,8 @@ def is_env_var_set_and_not_false(var_name):
         return False
 
 
-def is_invoked_by_pytest():
-    """Return true if pytest is running."""
+def is_invoked_by_pytest() -> bool:
+    """Return `True` if invoked in a `pytest` context."""
     return is_env_var_set_and_not_false(PYTEST_FLAG)
 
 
@@ -84,13 +85,13 @@ class YaralyzerConfig:
 
     @classmethod
     def set_argument_parser(cls, parser: ArgumentParser) -> None:
-        """Sets the _argument_parser instance variable that will be used to parse command line args."""
+        """Sets the `_argument_parser` instance variable that will be used to parse command line args."""
         cls._argument_parser: ArgumentParser = parser
         cls._argparse_keys: List[str] = sorted([action.dest for action in parser._actions])
 
     @classmethod
     def set_args(cls, args: Namespace) -> None:
-        """Set the args class instance variable and update args with any environment variable overrides."""
+        """Set the `args` class instance variable and update args with any environment variable overrides."""
         cls.args = args
 
         for option in cls._argparse_keys:
@@ -115,11 +116,11 @@ class YaralyzerConfig:
 
     @classmethod
     def set_default_args(cls):
-        """Set args to their defaults as if parsed from the command line."""
+        """Set `self.args` to their defaults as if parsed from the command line."""
         cls.set_args(cls._argument_parser.parse_args(['dummy']))
 
     @classmethod
     def get_default_arg(cls, arg: str) -> Any:
-        """Return the default value for arg as defined by a DEFAULT_ style class variable."""
+        """Return the default value for `arg` as defined by a `DEFAULT_` style class variable."""
         default_var = f"DEFAULT_{arg.upper()}"
         return vars(cls).get(default_var)

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/yaralyzer/decoding/bytes_decoder.py

@@ -1,4 +1,6 @@
-"""BytesDecoder class for attempting to decode bytes with various encodings."""
+"""
+`BytesDecoder` class for attempting to decode bytes with various encodings.
+"""
 from collections import defaultdict
 from copy import deepcopy
 from operator import attrgetter
@@ -31,31 +33,32 @@ SCORE_SCALER = 100.0
 
 class BytesDecoder:
     """
-    Class to handle attempting to decode a chunk of bytes into strings with various possible encodings.
-
-    Leverages the chardet library to both guide what encodings are attempted as well as to rank decodings
-    in the results.
+    Handles decoding a chunk of bytes into strings using various possible encodings, ranking and displaying results.
+
+    This class leverages the `chardet` library and custom logic to try multiple encodings, track decoding outcomes,
+    and present the results in a rich, user-friendly format. It is used to analyze and display the possible
+    interpretations of a byte sequence, especially in the context of YARA matches or binary analysis.
+
+    Attributes:
+        bytes_match (BytesMatch): The `BytesMatch` instance being decoded.
+        bytes (bytes): The bytes (including surrounding context) to decode.
+        label (str): Label for this decoding attempt.
+        was_match_decodable (dict): Tracks successful decodes per encoding.
+        was_match_force_decoded (dict): Tracks forced decodes per encoding.
+        was_match_undecodable (dict): Tracks failed decodes per encoding.
+        decoded_strings (dict): Maps encoding to decoded string.
+        undecoded_rows (list): Stores undecoded table rows.
+        decodings (list): List of DecodingAttempt objects for each encoding tried.
+        encoding_detector (EncodingDetector): Used to detect and assess possible encodings.
     """
 
     def __init__(self, bytes_match: 'BytesMatch', label: Optional[str] = None) -> None:
         """
-        Initialize a BytesDecoder for attempting to decode a chunk of bytes using various encodings.
+        Initialize a `BytesDecoder` for attempting to decode a chunk of bytes using various encodings.
 
         Args:
-            bytes_match (BytesMatch): The BytesMatch object containing the bytes to decode and match metadata.
+            bytes_match (BytesMatch): The `BytesMatch` object containing the bytes to decode and match metadata.
             label (Optional[str], optional): Optional label for this decoding attempt. Defaults to the match label.
-
-        Attributes:
-            bytes_match (BytesMatch): The BytesMatch instance being decoded.
-            bytes (bytes): The bytes (including surrounding context) to decode.
-            label (str): Label for this decoding attempt.
-            was_match_decodable (dict): Tracks successful decodes per encoding.
-            was_match_force_decoded (dict): Tracks forced decodes per encoding.
-            was_match_undecodable (dict): Tracks failed decodes per encoding.
-            decoded_strings (dict): Maps encoding to decoded string.
-            undecoded_rows (list): Stores undecoded table rows.
-            decodings (list): List of DecodingAttempt objects for each encoding tried.
-            encoding_detector (EncodingDetector): Used to detect and assess possible encodings.
         """
         self.bytes_match = bytes_match
         self.bytes = bytes_match.surrounding_bytes
@@ -92,7 +95,12 @@ class BytesDecoder:
         yield Align(self.bytes_match.bytes_hashes_table(), CENTER, style='dim')
 
     def _build_decodings_table(self, suppress_decodes: bool = False) -> Table:
-        """First rows are the raw / hex views of the bytes, next rows are the attempted decodings."""
+        """
+        First rows are the raw / hex views of the bytes, next rows are the attempted decodings.
+
+        Args:
+            suppress_decodes (bool, optional): If `True` don't add decoding attempts to the table. Defaults to `False`.
+        """
         self.table = new_decoding_attempts_table(self.bytes_match)
 
         # Add the encoding rows to the table if not suppressed
@@ -124,15 +132,15 @@ class BytesDecoder:
         return self._undecoded_assessments(self.encoding_detector.force_display_assessments)
 
     def _undecoded_assessments(self, assessments: List[EncodingAssessment]) -> List[EncodingAssessment]:
-        """Filter out the already decoded assessments from a set of assessments"""
+        """Filter out the already decoded assessments from a set of assessments."""
         return [a for a in assessments if not self._was_decoded(a.encoding)]
 
     def _was_decoded(self, encoding: str) -> bool:
-        """Check whether a given encoding is in the table already"""
+        """Check whether a given encoding is in the table already."""
         return any(row.encoding == encoding for row in self.decodings)
 
     def _decode_attempt_subheading(self) -> Panel:
-        """Generate a rich.Panel for displaying decode attempts"""
+        """Generate a rich.Panel for displaying decode attempts."""
         headline = Text(f"Found ", style='decode.subheading') + self.bytes_match.__rich__()
         return Panel(headline, style='decode.subheading', expand=False)
 
@@ -149,7 +157,7 @@ class BytesDecoder:
                 self.was_match_force_decoded[decoding.encoding] += 1
 
     def _row_from_decoding_attempt(self, decoding: DecodingAttempt) -> DecodingTableRow:
-        """Create a DecodingAttemptTable row from a DecodingAttempt."""
+        """Create a `DecodingAttemptTable` row from a `DecodingAttempt`."""
         assessment = self.encoding_detector.get_encoding_assessment(decoding.encoding)
 
         # If the decoding can have a start offset add an appropriate extension to the encoding label
@@ -184,7 +192,7 @@ class BytesDecoder:
 
 
 def _build_encodings_metric_dict():
-    """One key for each key in ENCODINGS_TO_ATTEMPT, values are all 0."""
+    """One key for each key in `ENCODINGS_TO_ATTEMPT`, values are all 0."""
     metrics_dict = defaultdict(lambda: 0)
 
     for encoding in ENCODINGS_TO_ATTEMPT.keys():

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/yaralyzer/decoding/decoding_attempt.py

@@ -1,4 +1,6 @@
-"""Class to manage attempting to decode a chunk of bytes into strings with a given encoding."""
+"""
+Class to manage attempting to decode a chunk of bytes into strings with a given encoding.
+"""
 from sys import byteorder
 from typing import Optional
 
@@ -15,10 +17,33 @@ from yaralyzer.util.logging import log
 
 
 class DecodingAttempt:
-    """Class to manage attempting to decode a chunk of bytes into strings with a given encoding."""
+    """
+    Manages the process of attempting to decode a chunk of bytes into a string using a specified encoding.
+
+    This class tries to decode the bytes using the provided encoding, handling both standard and custom decoding
+    strategies (including multi-byte encodings and forced decoding attempts). It tracks the outcome, highlights
+    the decoded output, and provides information about the decoding process.
+
+    Attributes:
+        bytes (bytes): The bytes (including context) to decode.
+        bytes_match (BytesMatch): The `BytesMatch` object containing match and context info.
+        encoding (str): The encoding to attempt.
+        encoding_label (str): Label for the encoding (may include offset info).
+        start_offset (int): Byte offset used for decoding (for multi-byte encodings).
+        start_offset_label (Optional[str]): String label for the offset, if used.
+        was_force_decoded (bool): True if a forced decode was attempted.
+        failed_to_decode (bool): True if decoding failed.
+        decoded_string (Text): The decoded string as a Rich `Text` object (with highlighting).
+    """
 
     def __init__(self, bytes_match: 'BytesMatch', encoding: str) -> None:
-        # Args
+        """
+        Initialize a `DecodingAttempt` for a specific `encoding` on a given `BytesMatch`.
+
+        Args:
+            bytes_match (BytesMatch): The `BytesMatch` object containing the bytes to decode and match metadata.
+            encoding (str): The encoding to attempt for decoding the bytes.
+        """
         self.bytes = bytes_match.surrounding_bytes
         self.bytes_match = bytes_match
         self.encoding = encoding
@@ -30,15 +55,11 @@ class DecodingAttempt:
         self.failed_to_decode = False
         self.decoded_string = self._decode_bytes()
 
-    def is_wide_utf_encoding(self) -> bool:
-        """Returns True if the encoding is UTF-16 or UTF-32."""
-        return is_wide_utf(self.encoding)
-
     def _decode_bytes(self) -> Text:
         """
-        Tries builtin decode, hands off to other methods for harsher treatement
-        (byte shifting for UTF-16/32 and custom decode for the rest) if that fails.
-        Has side effect of setting 'self.decoded_string' value.
+        Tries builtin decode, hands off to other methods for harsher treatment (byte shifting for
+        UTF-16/32 and custom decode for the rest) if that fails. Has side effect of setting
+        `self.decoded_string` value.
         """
         try:
             decoded_string = self._to_rich_text(escape(self.bytes.decode(self.encoding)))
@@ -52,13 +73,15 @@ class DecodingAttempt:
 
         self.was_force_decoded = True
 
-        if self.is_wide_utf_encoding():
+        if is_wide_utf(self.encoding):
             return self._decode_utf_multibyte()
         else:
-            return self._custom_decode()
+            return self._custom_utf_decode()
 
-    def _custom_decode(self) -> Text:
-        """Returns a Text obj representing an attempt to force a UTF-8 encoding upon an array of bytes."""
+    def _custom_utf_decode(self) -> Text:
+        """
+        Returns a `Text` obj representing an attempt to force a UTF-8 encoding onto an array of bytes.
+        """
         log.info(f"Custom decoding {self.bytes_match} with {self.encoding}...")
         unprintable_char_map = ENCODINGS_TO_ATTEMPT.get(self.encoding)
         output = Text('', style='bytes.decoded')
@@ -116,7 +139,13 @@ class DecodingAttempt:
         return output
 
     def _decode_utf_multibyte(self) -> Text:
-        """UTF-16/32 are fixed width and multibyte and therefore depend on the position of the starting byte."""
+        """
+        UTF-16/32 are fixed width and multibyte and therefore depend on the position of the starting byte
+        so we try several offsets until we find one that at least kind of works.
+
+        Returns:
+            Text: Rich `Text` object representing the decoded string with highlighting.
+        """
         char_width = encoding_width(self.encoding)
         last_exception = None
         decoded_str = None
@@ -146,7 +175,15 @@ class DecodingAttempt:
             return self._failed_to_decode_msg_txt(last_exception)
 
     def _to_rich_text(self, _string: str, bytes_offset: int = 0) -> Text:
-        """Convert a decoded string to highlighted Text representation."""
+        """
+        Convert a decoded string to highlighted `Text` representation.
+
+        Args:
+            _string (str): The decoded string to convert.
+            bytes_offset (int): The byte offset used during decoding (for multi-byte encodings).
+        Returns:
+            Text: The rich `Text` representation of the decoded string with appropriate highlighting.
+        """
         # Adjust where we start the highlighting given the multibyte nature of the encodings
         log.debug(f"Stepping through {self.encoding} encoded string...")
         txt = Text('', style=self.bytes_match.style_at_position(0))
@@ -160,7 +197,7 @@ class DecodingAttempt:
             is_single_byte_encoding = False
             unprintable_chars = {}
 
-        for i, c in enumerate(_string):
+        for _i, c in enumerate(_string):
             char_bytes = bytes(c, self.encoding)
             char_width = len(char_bytes)
             style = self.bytes_match.style_at_position(current_byte_idx + bytes_offset)
@@ -180,6 +217,6 @@ class DecodingAttempt:
         return txt
 
     def _failed_to_decode_msg_txt(self, exception: Optional[Exception]) -> Text:
-        """Set failed_to_decode flag and return a Text object with the error message."""
+        """Set `self.failed_to_decode` flag and return a `Text` object with the error message."""
         self.failed_to_decode = True
         return prefix_with_style(f"(decode failed: {exception})", style='red dim italic')

{yaralyzer-1.0.8 → yaralyzer-1.0.9}/yaralyzer/encoding_detection/character_encodings.py

@@ -1,8 +1,11 @@
 """
 Constants related to character encodings.
 
-* https://www.mit.edu/people/kenta/two/iso8859.html
-* https://www.utf8-chartable.de/unicode-utf8-table.pl?utf8=dec
+Helpful links:
+
+* ISO-8859: [www.mit.edu/people/kenta/two/iso8859.html](https://www.mit.edu/people/kenta/two/iso8859.html)
+
+* UTF-8: [www.utf8-chartable.de/unicode-utf8-table.pl?utf8=dec](https://www.utf8-chartable.de/unicode-utf8-table.pl?utf8=dec)  # noqa: E501
 """
 
 # Bytes (TODO: why is this here?)
@@ -69,10 +72,10 @@ UNPRINTABLE_ASCII = {
 }
 
 
-def scrub_c1_control_chars(char_map):
+def scrub_c1_control_chars(char_map: dict) -> None:
     """
-    Fill in a dict with integer keys/values corresponding to where a given char encoding has no chars
-    because this range is for C1 control chars (AKA the 'undefined' part of the character map)
+    Fill in a `dict` with integer keys/values corresponding to where a given char encoding has no chars
+    because this range is for C1 control chars (AKA the "undefined" part of most character maps).
     """
     for i in range(128, 160):
         char_map[i] = f"C1.CHAR{i}"
@@ -164,7 +167,7 @@ WIDE_UTF_ENCODINGS = {
 
 
 def encoding_offsets(encoding: str) -> list:
-    """Get possible offsets for a given encoding. If the encoding is not in WIDE_UTF_ENCODINGS, return [0]."""
+    """Get possible offsets for a given encoding. If the encoding is not in `WIDE_UTF_ENCODINGS`, return `[0]`."""
     return WIDE_UTF_ENCODINGS.get(encoding, [0])