extract-msg 0.47.0__tar.gz → 0.48.1__tar.gz

{extract_msg-0.47.0 → extract_msg-0.48.1}/CHANGELOG.md

@@ -1,3 +1,19 @@
+**v0.48.1**
+* Added an option (`-s`, `--stdin`) to the command line to take an MSG file from stdin. This allows the user to pipe the MSG data from another program directly instead of having to write a middleman that uses the `extract-msg` library directly or having to write the file to the disk first.
+* Changed main function to allow for manual argument list to be passed to it.
+* Added attributes to `AttachmentBase` for creation and modification time. These can be accessed through `createdAt` or `creationTime` and `lastModificationTime` or `modifiedAt`.
+* Changed `OleWriter` tests to output the name of the test file being done if an error occurs.
+* Added tests for some command line stuff.
+
+**v0.48.0**
+* Adjusted error handling for named properties to handle critical streams being missing and to allow suppression of those errors.
+* Adjusted error handling for named properties to allow silencing of errors caused by invalid references to the name stream. If `ErrorBehavior.NAMED_NAME_STREAM` is provided to the `MSGFile` instance, a warning will be logged and that entry will simply be dropped.
+* Adjusted error handling for signed messages to better check for issues with the signed attachment. This should make errors from violating the standard much easier to understand. These errors can be ignored, but the attachment will *not* be parsed as a signed attachment.
+* Minor docstring updates.
+* Minor adjustments to `OleWriter` to prepare the code for being able to write version 4 files. Version 3 files are currently the only one's supported, but much of the code had hard-coded values that could be replaced with variables and small conditionals. This will have very little performance impact, and should not be noticeable.
+* Improved comments on `OleWriter` to make private sections more understandable.
+* Changed `MessageSignedBase._rawAttachments` to `MessageSignedBase.rawAttachments` to provide non-private access in a reliable way.
+
 **v0.47.0**
 * Changed the public API for `PropertiesStore` to improve the quality of its code. The properties type is now mandatory, and the intelligence field (and the related enum) has been removed.
     * Additionally, the `toBytes` and `__bytes__` methods will both generate based on the contents of this class, allowing for new properties to be created and for existing properties to be modified or removed if the class is set to writable on creation.

{extract_msg-0.47.0 → extract_msg-0.48.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: extract_msg
-Version: 0.47.0
+Version: 0.48.1
 Summary: Extracts emails and attachments saved in Microsoft Outlook's .msg files
 Home-page: https://github.com/TeamMsgExtractor/msg-extractor
 Download-URL: https://github.com/TeamMsgExtractor/msg-extractor/archives/master
@@ -79,7 +79,7 @@ refer to the usage information provided from the program's help dialog:
 
      usage: extract_msg [-h] [--use-content-id] [--json] [--file-logging] [-v] [--log LOG] [--config CONFIGPATH] [--out OUTPATH] [--use-filename] [--dump-stdout] [--html] [--pdf] [--wk-path WKPATH] [--wk-options [WKOPTIONS ...]]
                         [--prepared-html] [--charset CHARSET] [--raw] [--rtf] [--allow-fallback] [--skip-body-not-found] [--zip ZIP] [--save-header] [--attachments-only] [--skip-hidden] [--no-folders] [--skip-embedded] [--extract-embedded]
-                        [--overwrite-existing] [--skip-not-implemented] [--out-name OUTNAME | --glob] [--ignore-rtfde] [--progress]
+                        [--overwrite-existing] [--skip-not-implemented] [--out-name OUTNAME | --glob] [--ignore-rtfde] [--progress] [-s]
                         msg [msg ...]
 
      extract_msg: Extracts emails and attachments saved in Microsoft Outlook's .msg files. https://github.com/TeamMsgExtractor/msg-extractor
@@ -125,6 +125,7 @@ refer to the usage information provided from the program's help dialog:
        --glob, --wildcard    Interpret all paths as having wildcards. Incompatible with --out-name.
        --ignore-rtfde        Ignores all errors thrown from RTFDE when trying to save. Useful for allowing fallback to continue when an exception happens.
        --progress            Shows what file the program is currently working on during it's progress.
+       -s, --stdin           Read file from stdin (only works with one file at a time).
 
 **To use this in your own script**, start by using:
 
@@ -277,8 +278,8 @@ your access to the newest major version of extract-msg.
 .. |License: GPL v3| image:: https://img.shields.io/badge/License-GPLv3-blue.svg
    :target: LICENSE.txt
 
-.. |PyPI3| image:: https://img.shields.io/badge/pypi-0.47.0-blue.svg
-   :target: https://pypi.org/project/extract-msg/0.47.0/
+.. |PyPI3| image:: https://img.shields.io/badge/pypi-0.48.1-blue.svg
+   :target: https://pypi.org/project/extract-msg/0.48.1/
 
 .. |PyPI2| image:: https://img.shields.io/badge/python-3.8+-brightgreen.svg
    :target: https://www.python.org/downloads/release/python-3810/

{extract_msg-0.47.0 → extract_msg-0.48.1}/README.rst

@@ -61,7 +61,7 @@ refer to the usage information provided from the program's help dialog:
 
      usage: extract_msg [-h] [--use-content-id] [--json] [--file-logging] [-v] [--log LOG] [--config CONFIGPATH] [--out OUTPATH] [--use-filename] [--dump-stdout] [--html] [--pdf] [--wk-path WKPATH] [--wk-options [WKOPTIONS ...]]
                         [--prepared-html] [--charset CHARSET] [--raw] [--rtf] [--allow-fallback] [--skip-body-not-found] [--zip ZIP] [--save-header] [--attachments-only] [--skip-hidden] [--no-folders] [--skip-embedded] [--extract-embedded]
-                        [--overwrite-existing] [--skip-not-implemented] [--out-name OUTNAME | --glob] [--ignore-rtfde] [--progress]
+                        [--overwrite-existing] [--skip-not-implemented] [--out-name OUTNAME | --glob] [--ignore-rtfde] [--progress] [-s]
                         msg [msg ...]
 
      extract_msg: Extracts emails and attachments saved in Microsoft Outlook's .msg files. https://github.com/TeamMsgExtractor/msg-extractor
@@ -107,6 +107,7 @@ refer to the usage information provided from the program's help dialog:
        --glob, --wildcard    Interpret all paths as having wildcards. Incompatible with --out-name.
        --ignore-rtfde        Ignores all errors thrown from RTFDE when trying to save. Useful for allowing fallback to continue when an exception happens.
        --progress            Shows what file the program is currently working on during it's progress.
+       -s, --stdin           Read file from stdin (only works with one file at a time).
 
 **To use this in your own script**, start by using:
 
@@ -259,8 +260,8 @@ your access to the newest major version of extract-msg.
 .. |License: GPL v3| image:: https://img.shields.io/badge/License-GPLv3-blue.svg
    :target: LICENSE.txt
 
-.. |PyPI3| image:: https://img.shields.io/badge/pypi-0.47.0-blue.svg
-   :target: https://pypi.org/project/extract-msg/0.47.0/
+.. |PyPI3| image:: https://img.shields.io/badge/pypi-0.48.1-blue.svg
+   :target: https://pypi.org/project/extract-msg/0.48.1/
 
 .. |PyPI2| image:: https://img.shields.io/badge/python-3.8+-brightgreen.svg
    :target: https://www.python.org/downloads/release/python-3810/

{extract_msg-0.47.0 → extract_msg-0.48.1}/extract_msg/__init__.py

@@ -27,8 +27,8 @@ https://github.com/TeamMsgExtractor/msg-extractor
 #    along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 __author__ = 'Destiny Peterson & Matthew Walker'
-__date__ = '2023-12-09'
-__version__ = '0.47.0'
+__date__ = '2024-03-08'
+__version__ = '0.48.1'
 
 __all__ = [
     # Modules:

{extract_msg-0.47.0 → extract_msg-0.48.1}/extract_msg/__main__.py

@@ -10,12 +10,13 @@ import zipfile
 
 from extract_msg import __doc__, openMsg, utils
 from extract_msg.enums import ErrorBehavior
+from typing import List
 
 
-def main() -> None:
+def main(argv : List[str]) -> None:
     # Setup logging to stdout, indicate running from cli
     CLI_LOGGING = 'extract_msg_cli'
-    args = utils.getCommandArgs(sys.argv[1:])
+    args = utils.getCommandArgs(argv[1:])
 
     # Determine where to save the files to.
     currentDir = os.getcwd() # Store this in case the path changes.
@@ -111,4 +112,4 @@ def main() -> None:
         _zip.close()
 
 if __name__ == '__main__':
-    main()
+    main(sys.argv)

{extract_msg-0.47.0 → extract_msg-0.48.1}/extract_msg/attachments/attachment_base.py

@@ -290,7 +290,7 @@ class AttachmentBase(abc.ABC):
 
     def getNamedProp(self, propertyName: str, guid: str, default: _T = None) -> Union[Any, _T]:
         """
-        instance.namedProperties.get((propertyName, guid), default)
+        ``instance.namedProperties.get((propertyName, guid), default)``
 
         Can be overriden to create new behavior.
         """
@@ -400,7 +400,7 @@ class AttachmentBase(abc.ABC):
         the filename this function should receive should be "__substg1.0_001A".
 
         This should ALWAYS return a string if it was found, otherwise returns
-        None.
+        ``None``.
 
         :raises ReferenceError: The associated ``MSGFile`` instance has been
             garbage collected.
@@ -415,10 +415,11 @@ class AttachmentBase(abc.ABC):
         class if it is found.
 
         :param overrideClass: Class/function to use to morph the data that was
-            read. The data will be the first argument to the class's __init__
-            function or the function itself, if that is what is provided. If
-            the value is None, this function is not called. If you want it to
-            be called regardless, you should handle the data directly.
+            read. The data will be the first argument to the class's
+            ``__init__`` method or the function itself, if that is what is
+            provided. If the value is ``None``, this function is not called. If
+            you want it to be called regardless, you should handle the data
+            directly.
         """
         value = self.getStream(streamID)
 
@@ -472,7 +473,7 @@ class AttachmentBase(abc.ABC):
             the first item specifies what the second value will be.
         """
 
-    @functools.cached_property
+    @cached_property
     def attachmentEncoding(self) -> Optional[bytes]:
         """
         The encoding information about the attachment object.
@@ -482,7 +483,7 @@ class AttachmentBase(abc.ABC):
         """
         return self.getStream('__substg1.0_37020102')
 
-    @functools.cached_property
+    @cached_property
     def additionalInformation(self) -> Optional[str]:
         """
         The additional information about the attachment.
@@ -494,7 +495,7 @@ class AttachmentBase(abc.ABC):
         """
         return self.getStringStream('__substg1.0_370F')
 
-    @functools.cached_property
+    @cached_property
     def cid(self) -> Optional[str]:
         """
         Returns the Content ID of the attachment, if it exists.
@@ -525,8 +526,25 @@ class AttachmentBase(abc.ABC):
 
     @property
     def contentId(self) -> Optional[str]:
+        """
+        Alias of :attr:`cid`.
+        """
         return self.cid
 
+    @property
+    def createdAt(self) -> Optional[datetime.datetime]:
+        """
+        Alias of :attr:`creationTime`.
+        """
+        return self.creationTime
+
+    @cached_property
+    def creationTime(self) -> Optional[datetime.datetime]:
+        """
+        The time the attachment was created.
+        """
+        return self.getPropertyVal('30070040')
+
     @property
     @abc.abstractmethod
     def data(self) -> Optional[object]:
@@ -536,7 +554,7 @@ class AttachmentBase(abc.ABC):
         Returns ``None`` if there is no data to save.
         """
 
-    @functools.cached_property
+    @cached_property
     def dataType(self) -> Optional[Type[object]]:
         """
         The class that the data type will use, if it can be retrieved.
@@ -559,14 +577,14 @@ class AttachmentBase(abc.ABC):
         """
         return self.__dir
 
-    @functools.cached_property
+    @cached_property
     def displayName(self) -> Optional[str]:
         """
         Returns the display name of the folder.
         """
         return self.getStringStream('__substg1.0_3001')
 
-    @functools.cached_property
+    @cached_property
     def exceptionReplaceTime(self) -> Optional[datetime.datetime]:
         """
         The original date and time at which the instance in the recurrence
@@ -576,48 +594,62 @@ class AttachmentBase(abc.ABC):
         """
         return self.getPropertyVal('7FF90040')
 
-    @functools.cached_property
+    @cached_property
     def extension(self) -> Optional[str]:
         """
         The reported extension for the file.
         """
         return self.getStringStream('__substg1.0_3703')
 
-    @functools.cached_property
+    @cached_property
     def hidden(self) -> bool:
         """
         Indicates whether an Attachment object is hidden from the end user.
         """
         return bool(self.getPropertyVal('7FFE000B'))
 
-    @functools.cached_property
+    @cached_property
     def isAttachmentContactPhoto(self) -> bool:
         """
         Whether the attachment is a contact photo for a Contact object.
         """
         return bool(self.getPropertyVal('7FFF000B'))
 
-    @functools.cached_property
+    @cached_property
+    def lastModificationTime(self) -> Optional[datetime.datetime]:
+        """
+        The last time the attachment was modified.
+        """
+        return self.getPropertyVal('30080040')
+
+    @cached_property
     def longFilename(self) -> Optional[str]:
         """
         Returns the long file name of the attachment, if it exists.
         """
         return self.getStringStream('__substg1.0_3707')
 
-    @functools.cached_property
+    @cached_property
     def longPathname(self) -> Optional[str]:
         """
         The fully qualified path and file name with extension.
         """
         return self.getStringStream('__substg1.0_370D')
 
-    @functools.cached_property
+    @cached_property
     def mimetype(self) -> Optional[str]:
         """
         The content-type mime header of the attachment, if specified.
         """
         return tryGetMimetype(self, self.getStringStream('__substg1.0_370E'))
 
+    @property
+    def modifiedAt(self) -> Optional[datetime.datetime]:
+        """
+        Alias of :attr:`lastModificationTime`.
+        """
+        return self.lastModificationTime
+
     @property
     def msg(self) -> MSGFile:
         """
@@ -630,7 +662,7 @@ class AttachmentBase(abc.ABC):
             raise ReferenceError('The MSGFile for this Attachment instance has been garbage collected.')
         return msg
 
-    @functools.cached_property
+    @cached_property
     def name(self) -> Optional[str]:
         """
         The best name available for the file.
@@ -649,7 +681,7 @@ class AttachmentBase(abc.ABC):
         """
         return self.__namedProperties
 
-    @functools.cached_property
+    @cached_property
     def payloadClass(self) -> Optional[str]:
         """
         The class name of an object that can display the contents of the
@@ -664,17 +696,18 @@ class AttachmentBase(abc.ABC):
         """
         return self.__props
 
-    @functools.cached_property
+    @cached_property
     def renderingPosition(self) -> Optional[int]:
         """
         The offset, in rendered characters, to use when rendering the attachment
         within the main message text.
 
-        A value of 0xFFFFFFFF indicates a hidden attachment that is not to be rendered.
+        A value of ``0xFFFFFFFF`` indicates a hidden attachment that is not to
+        be rendered.
         """
         return self.getPropertyVal('370B0003')
 
-    @property
+    @cached_property
     def shortFilename(self) -> Optional[str]:
         """
         The short file name of the attachment, if it exists.

{extract_msg-0.47.0 → extract_msg-0.48.1}/extract_msg/encoding/utils.py

@@ -202,7 +202,7 @@ def variableByteDecode(codecName: str, data, errors: str, decodeTable: Dict[int,
     """
     Function for decoding variable-byte codecs that use one or two bytes per character.
 
-    Checks if a character is less than 0x80, mapping it directly if so.
+    Checks if a character is less than ``0x80``, mapping it directly if so.
     Otherwise, it reads the next byte and combines the two before looking up the
     new value.
 

{extract_msg-0.47.0 → extract_msg-0.48.1}/extract_msg/enums.py

@@ -674,6 +674,9 @@ class ErrorBehavior(enum.IntFlag):
     * STANDARDS_VIOLATION: Silences StandardViolationError where acceptable.
     * OLE_DEFECT_INCORRECT: Silences defects of type DEFECT_INCORRECT that are
       enabled by default. This can lead to strange bugs.
+    * NAMED_NAME_STREAM: Silences errors caused by invalid indexes into the
+      name stream for named properties. Any properties with an invalid name will
+      simply be dropped.
     * SUPPRESS_ALL: Silences all of the above.
     """
     THROW = 0b000000
@@ -688,8 +691,10 @@ class ErrorBehavior(enum.IntFlag):
     # General.
     STANDARDS_VIOLATION = 0b010000
     OLE_DEFECT_INCORRECT = 0b100000
+    # Named Properties
+    NAMED_NAME_STREAM = 0b1000000
 
-    SUPPRESS_ALL = 0b111111
+    SUPPRESS_ALL = 0b1111111
 
 
 

{extract_msg-0.47.0 → extract_msg-0.48.1}/extract_msg/msg_classes/message_signed_base.py

@@ -11,7 +11,7 @@ import re
 from typing import Generic, List, Optional, Type, TypeVar
 
 from ..attachments import AttachmentBase, SignedAttachment
-from ..enums import DeencapType, ErrorBehavior
+from ..enums import AttachmentType, DeencapType, ErrorBehavior
 from ..exceptions import StandardViolationError
 from .message_base import MessageBase
 from ..utils import inputToBytes, inputToString, unwrapMultipart
@@ -47,21 +47,42 @@ class MessageSignedBase(MessageBase, Generic[_T]):
         :raises StandardViolationError: The standard for signed messages was
             blatantly violated.
         """
-        atts = self._rawAttachments
+        # Set these to None to make error handling easier.
+        self._signedBody = None
+        self._signedHtmlBody = None
+
+        atts = self.rawAttachments
 
         if len(atts) != 1:
             if ErrorBehavior.STANDARDS_VIOLATION in self.errorBehavior:
                 if len(atts) == 0:
-                    logger.error('Signed message has no attachments, a violation of the standard.')
-                    self._sAttachments = []
-                    self._signedBody = None
-                    self._signedHtmlBody = None
+                    logger.error('Standards Violation: Signed message has no attachments.')
                     return []
-                # If there is at least one attachment, just try to use the
-                # first.
+                # If there is at least one attachment, log an error about it
+                # and then try to use the first one.
+                logger.error('Standards Violation: Signed message has more than one attachment. Attempting to use first one.')
             else:
                 raise StandardViolationError('Signed messages without exactly 1 (regular) attachment constitute a violation of the standard.')
 
+        # If we are here, validate that the found attachment is acceptable.
+        if atts[0].type is not AttachmentType.DATA:
+            # This is unacceptable and the attachment cannot be used.
+            if ErrorBehavior.STANDARDS_VIOLATION in self.errorBehavior:
+                logger.error('Standards Violation: Attachment on signed message is unacceptable (not binary).')
+                return []
+            raise StandardViolationError('Signed messages *must* have a binary attachment. Signed attachments cannot be parsed otherwise.')
+
+        # Check the mimetype *directly*. Detection of mimetype through data is
+        # unacceptable for this check.
+        #temp = atts[0].getStringStream('__substg1.0_370E')
+        #if temp == 'multipart/signed':
+            ## Do nothing, but this is how we detect that it is clear signed.
+            #pass
+        #else:
+            ## Do nothing, but this is how we detect that it is *not* clear
+            # signed.
+            #pass
+
         # We need to unwrap the multipart stream.
         unwrapped = unwrapMultipart(atts[0].data)
 
@@ -117,7 +138,7 @@ class MessageSignedBase(MessageBase, Generic[_T]):
         return htmlBody
 
     @functools.cached_property
-    def _rawAttachments(self) -> List[AttachmentBase]:
+    def rawAttachments(self) -> List[AttachmentBase]:
         """
         A property to allow access to the non-signed attachments.
         """

{extract_msg-0.47.0 → extract_msg-0.48.1}/extract_msg/msg_classes/msg.py

@@ -83,8 +83,8 @@ class MSGFile:
         :param errorBehavior: Optional, the behavior to use in the event of
             certain types of errors. Uses the ``ErrorBehavior`` enum.
         :param overrideEncoding: Optional, an encoding to use instead of the one
-            specified by the MSG file. If the value is "chardet" and you have
-            the chardet module installed, an attempt will be made to
+            specified by the MSG file. If the value is ``"chardet"`` and you
+            have the ``chardet`` module installed, an attempt will be made to
             auto-detect the encoding based on some of the string properties. Do
             not report encoding errors caused by this.
         :param treePath: Internal variable used for giving representation of the