mail-parser 4.8.0__tar.gz → 4.9.0__tar.gz

{mail_parser-4.8.0 → mail_parser-4.9.0}/.github/workflows/main.yml

@@ -12,7 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ['3.9', '3.10', '3.11', '3.12', '3.13', '3.14', '3.15']
+        python-version: ['3.10', '3.11', '3.12', '3.13', '3.14', '3.15']
 
     steps:
       - uses: actions/checkout@v4

mail_parser-4.9.0/GLOSSARY.md

@@ -0,0 +1,21 @@
+# Mail forensic evidence
+
+Language used when describing the evidence recovered from an email.
+
+## Language
+
+**Original evidence**:
+The message headers, MIME structure and payloads received for analysis, before
+display decoding or recovery.
+
+**Recovered value**:
+A usable interpretation of damaged message content that remains distinguishable
+from its original evidence.
+
+**Container representation**:
+A textual copy of the content enclosed by a multipart or attached message,
+which can overlap the evidence in its child attachments.
+
+**Partial recovery**:
+A result containing the safely recovered content and explicit diagnostics for
+representations that could not be produced within the analysis budget.

{mail_parser-4.8.0 → mail_parser-4.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mail-parser
-Version: 4.8.0
+Version: 4.9.0
 Summary: A tool that parses emails by enhancing the Python standard library, extracting all details into a comprehensive object.
 Project-URL: Homepage, https://github.com/SpamScope/mail-parser
 Project-URL: Source, https://github.com/SpamScope/mail-parser
@@ -398,6 +398,20 @@ An ambiguous trusted hop stops sender-IP attribution; older headers cannot
 supply a replacement IP. Balanced, unrecognized obsolete trace syntax remains
 raw without being declared invalid solely because it is obsolete.
 
+Envelope annotations (`envelope-from` and `envelope-sender`) are scanned
+without regex backtracking, preserving quoted delimiters, comments, domain
+literals and obsolete routes. Nested or unterminated envelope paths are not
+selected as addresses: `nested-envelope-address` or
+`unclosed-envelope-address` diagnostics retain the complete original field,
+including folding, while other trace clauses remain available. For these
+diagnostics, `recovered` indicates whether a later complete path was recovered
+from an affected clause after the outer path closes. Markers use ASCII
+case equivalence only; Unicode lookalikes remain literal evidence.
+Repeated reasons are aggregated per header to avoid
+duplicating the raw evidence for every malformed fragment. These annotations
+are lexical extractions, not full SMTP mailbox validation; the existing output
+for the valid null sender `<>` remains an absent envelope value.
+
 ### Date diagnostics
 
 Impossible calendar dates, clock components and numeric timezone minutes no
@@ -437,8 +451,272 @@ decoder that produces a non-Unicode scalar also triggers recovery: names retain
 their literal encoded form and text retains a safe byte-derived view, keeping
 JSON and CLI output encodable as UTF-8.
 
+Header decoding is strict. An unknown charset, invalid byte sequence or
+non-Unicode scalar leaves the complete original header value literal instead
+of dropping bytes or mixing a decoded prefix with the original. A
+`HeaderDecodeDefect` sets `has_defects`; `mail.header_decoding_defects` contains
+`reason`, `raw`, `header`, zero-based `occurrence`, zero-based MIME `part` and
+`recovered`. Here `recovered=True` means the literal evidence is available,
+not that decoding succeeded. This also covers nested non-address headers;
+address display names retain their existing context-aware diagnostics.
+Malformed encoded attachment names remain literal with `MimeHeaderDefect`.
+Raw Unicode text remains readable alongside encoded-words; decoding leaves
+literal spans intact and removes only ASCII folding whitespace between
+adjacent encoded-words. The optional RFC 2231 language suffix, such as
+`UTF-8*en`, is separated from the charset before strict decoding.
+
+### Resource limits for container representations
+
+Attached multipart containers and forwarded messages can repeat the same
+descendant payloads many times. Their optional textual representations share
+a per-message serialization budget. Preflight estimates reject expensive
+subtrees before copying; actual generator visits and writes to all internal
+text/byte buffers are also charged, including failed charset fallbacks.
+The character-work budget is 32 times the original tree's estimated size,
+with a floor of 1 Mi characters and a ceiling of 32 Mi characters. The estimate
+includes raw header/payload lengths and 256 characters of overhead per part.
+The node-visit budget is 16 times the number of MIME parts, with a floor of
+1,024 and a ceiling of 100,000. These are work units, not an input-size limit
+or an exact process-memory allowance.
+
+When a representation cannot fit, it is omitted entirely from `attachments`;
+no empty or truncated file is invented. `ResourceLimitDefect` sets
+`has_defects`, and `mail.resource_limit_defects` records its `reason`, MIME
+`part`, `filename` and `recovered=False`. Child attachments and neighboring
+body parts continue to be recovered under the existing disposition rules.
+The original container remains accessible at
+`list(mail.message.walk())[defect["part"]]` for ordinary MIME-tree parts.
+Recovered epilogue parts have indices after that original tree and remain
+traceable to the original epilogue text. Neither a resource limit nor a
+recovery diagnostic establishes malicious intent or invalid MIME syntax.
+
+Both new diagnostic properties always return lists. Nonempty lists appear
+in full and partial JSON output; literal wire headers colliding with those
+names remain accessible through `headers` and `message`. The serialization
+budget applies during parsing to container copies, not to input ingestion,
+individual leaf payloads, or an explicit later `message_as_string` request.
+Applications processing untrusted mail should still bound input size and
+isolate workers according to their own resource constraints.
+
 ## Defects and Their Critical Role in Email Security
 
+### Inspecting defects and recovered evidence
+
+A defect is a parsing anomaly, an inconsistent value, or a recovery/ambiguity
+diagnostic; it does not by itself establish malicious intent or an RFC violation.
+During parsing, mail-parser collects Python `email` defects from the MIME tree
+and adds its own address, trace, date, charset and MIME-metadata diagnostics:
+`mail.has_defects` becomes `True`, `mail.defects_categories` contains the category
+names, and `mail.defects` contains a list of dictionaries mapping a content type
+(for example `multipart/mixed`) or a diagnostic group (`address-headers`,
+`received-headers`, `date-headers`, `mime-headers`) to lists of textual
+descriptions. `AddressHeaderDefect`, `ReceivedHeaderDefect` and `DateHeaderDefect`
+also populate `mail.address_header_defects`, `mail.received_header_defects` and
+`mail.date_header_defects`, respectively; these properties always exist and are
+empty lists when there are no corresponding diagnostics. Their entries contain
+`reason`, `raw`, `header`, zero-based `occurrence` and `recovered`, plus address
+`candidates` or the selected ISO date `value` where applicable. All occurrences
+are examined, including repeated headers, while address properties and `mail.date`
+retain first-occurrence semantics. `recovered=True` means a value was recovered,
+not that the original input was valid; ambiguous address candidates remain
+evidence rather than selected mailboxes, and valid neighbours remain available.
+`CharsetDecodeDefect`, `MimeHeaderDefect` and collected standard-library defects
+have textual entries in `mail.defects`, without dedicated structured properties;
+the standard-library categories are described in
+[Python's defect reference](https://docs.python.org/3/library/email.errors.html),
+but this is not a guarantee that every defect Python can detect is collected
+(for example, transfer-decoding defects added after the MIME-tree collection may
+remain only in a part's `defects`). Full and partial output always include
+`has_defects`; when defects exist they include `defects` and
+`defects_categories`, with structured header diagnostics included when nonempty
+(JSON represents the category set as a list). Original evidence remains accessible
+through header properties such as `mail.from_raw`, `mail.message`, and individual
+MIME parts. A "bad epilogue" has no separate category or `epilogue_defects`
+property: `StartBoundaryNotFoundDefect` activates an attempt to recover a part
+between the declared boundary markers in the top-level `mail.message.epilogue`,
+and recovered content enters the usual text/attachment outputs without a separate
+recovery-success diagnostic. This is a limited recovery path, not a scan of every
+nested epilogue; an exception during this attempt is logged. An ordinary epilogue
+is permitted by [RFC 2046 section 5.1.1](https://www.rfc-editor.org/rfc/rfc2046.html#section-5.1.1)
+and is not itself a defect. The examples below cover each reporting family,
+successful and unsuccessful recovery, repeated headers, multiple simultaneous
+defects and valid controls; `has_defects=False` only means the implemented checks
+found no anomalies, not that the email passed a complete RFC validation.
+
+```python
+import base64
+import json
+import mailparser
+
+# 1. Recovered address: all four diagnostic views describe the same problem.
+mail = mailparser.parse_from_string(
+    "From: billing@trusted.example < billing@vendor.example >\r\n\r\n"
+)
+assert mail.from_ == [("billing@trusted.example", "billing@vendor.example")]
+assert mail.has_defects is True
+assert "AddressHeaderDefect" in mail.defects_categories
+assert mail.defects == [{"address-headers": [
+    "AddressHeaderDefect: from[0]: invalid-display-name"
+]}]
+assert mail.address_header_defects == [{
+    "reason": "invalid-display-name",
+    "raw": "billing@trusted.example < billing@vendor.example >",
+    "recovered": True,
+    "candidates": [{
+        "display_name": "billing@trusted.example",
+        "address": "billing@vendor.example",
+    }],
+    "header": "from",
+    "occurrence": 0,
+}]
+assert json.loads(mail.mail_json)["address_header_defects"] == (
+    mail.address_header_defects
+)
+
+# 2. Incomplete address: no selected mailbox; a valid neighbouring field survives.
+mail = mailparser.parse_from_string(
+    "From: Alice <alice@example.com\r\n"
+    "To: Bob <bob@example.com>\r\n\r\n"
+)
+assert mail.from_ == []
+assert mail.to == [("Bob", "bob@example.com")]
+assert mail.address_header_defects[0]["reason"] == "unclosed-delimiter"
+assert mail.address_header_defects[0]["recovered"] is False
+
+# 3. Repeated headers: diagnostics also cover occurrences after the first.
+mail = mailparser.parse_from_string(
+    "From: Alice <alice@example.com>\r\n"
+    "From: billing@trusted.example <billing@vendor.example>\r\n\r\n"
+)
+assert mail.from_ == [("Alice", "alice@example.com")]
+assert mail.address_header_defects[0]["occurrence"] == 1
+
+# 4. Received: ambiguous trace syntax stays raw, with structured diagnostics.
+trace = (
+    "from sender.example (unclosed; by mx.example; "
+    "1 Jan 2024 12:00:00 +0000"
+)
+mail = mailparser.parse_from_string(f"Received: {trace}\r\n\r\n")
+assert "ReceivedHeaderDefect" in mail.defects_categories
+assert mail.received_header_defects[0]["reason"] == "unclosed-comment"
+assert mail.received_header_defects[0]["recovered"] is False
+assert mail.received[0]["raw"] == trace
+assert "received-headers" in mail.defects[0]
+trace = (
+    "from sender.example by mx.example with ESMTP id token; "
+    "for <bob@example.com>; 1 Jan 2024 12:00:00 +0000"
+)
+mail = mailparser.parse_from_string(f"Received: {trace}\r\n\r\n")
+assert mail.received_header_defects[0]["reason"] == "misplaced-date-separator"
+assert mail.received_header_defects[0]["recovered"] is True
+assert mail.received[0]["for"] == "<bob@example.com>"
+
+# 5. Dates: an impossible date is not selected; a wrong weekday is recoverable.
+mail = mailparser.parse_from_string(
+    "Date: 31 Feb 2024 12:00:00 +0000\r\n\r\n"
+)
+assert "DateHeaderDefect" in mail.defects_categories
+assert mail.date is None
+assert mail.date_header_defects[0]["reason"] == "invalid-calendar-date"
+assert mail.date_header_defects[0]["recovered"] is False
+assert "date-headers" in mail.defects[0]
+mail = mailparser.parse_from_string(
+    "Date: Tue, 1 Jan 2024 12:00:00 +0000\r\n\r\n"
+)
+assert mail.date.isoformat() == "2024-01-01T12:00:00+00:00"
+assert mail.date_header_defects[0]["reason"] == "weekday-mismatch"
+assert mail.date_header_defects[0]["recovered"] is True
+# Date diagnostics also cover Resent-Date and timestamps inside Received.
+mail = mailparser.parse_from_string(
+    "Resent-Date: 31 Feb 2024 12:00:00 +0000\r\n"
+    "Received: from sender.example by mx.example; "
+    "1 Jan 2024 25:00:00 +0000\r\n\r\n"
+)
+assert {d["header"] for d in mail.date_header_defects} == {
+    "resent-date", "received"
+}
+
+# 6. Charset: failed declared decoding, then readable UTF-8 or replacement text.
+for payload, expected in [(b"caf\xc3\xa9", "café"), (b"a\xffb", "a\ufffdb")]:
+    mail = mailparser.parse_from_bytes(
+        b"Content-Type: text/plain; charset=ascii\r\n\r\n" + payload
+    )
+    assert "CharsetDecodeDefect" in mail.defects_categories
+    assert expected in mail.body
+    assert "CharsetDecodeDefect:" in mail.defects[0]["text/plain"][0]
+    assert mail.message.get_payload(decode=True) == payload
+
+# 7. MIME metadata: damaged filename bytes are retained in textual evidence.
+mail = mailparser.parse_from_bytes(
+    b"Content-Type: application/octet-stream\r\n"
+    b'Content-Disposition: attachment; filename="a\xffb.bin"\r\n'
+    b"Content-Transfer-Encoding: base64\r\n\r\nSGVsbG8="
+)
+assert "MimeHeaderDefect" in mail.defects_categories
+assert "\\xff" in " ".join(mail.defects[0]["mime-headers"])
+assert base64.b64decode(mail.attachments[0]["payload"]) == b"Hello"
+
+# 8. Standard-library structure defects: missing parameter/start/closing boundary.
+for raw, category in [
+    ("Content-Type: multipart/mixed\r\n\r\nHello",
+     "NoBoundaryInMultipartDefect"),
+    ('Content-Type: multipart/mixed; boundary="b"\r\n\r\nHello',
+     "StartBoundaryNotFoundDefect"),
+    ('Content-Type: multipart/mixed; boundary="b"\r\n\r\n'
+     '--b\r\nContent-Type: text/plain\r\n\r\nHello\r\n',
+     "CloseBoundaryNotFoundDefect"),
+    ("Subject: example\r\ninvalid header line\r\n\r\nHello",
+     "MissingHeaderBodySeparatorDefect"),
+]:
+    mail = mailparser.parse_from_string(raw)
+    assert mail.has_defects
+    assert category in mail.defects_categories
+    assert any(category in text for entry in mail.defects
+               for texts in entry.values() for text in texts)
+
+# 9. Bad epilogue: reusing the parent's boundary corrupts nested MIME structure.
+# The attachment after the first closing boundary lands in the outer epilogue.
+raw = (
+    'MIME-Version: 1.0\r\nContent-Type: multipart/mixed; boundary="b"\r\n\r\n'
+    '--b\r\nContent-Type: multipart/alternative; boundary="b"\r\n\r\n'
+    '--b\r\nContent-Type: text/plain\r\n\r\nHello\r\n--b--\r\n'
+    '--b\r\nContent-Type: application/octet-stream; name="evidence.txt"\r\n'
+    'Content-Disposition: attachment; filename="evidence.txt"\r\n'
+    'Content-Transfer-Encoding: base64\r\n\r\nSGVsbG8=\r\n--b--\r\n'
+)
+mail = mailparser.parse_from_string(raw)
+assert mail.has_defects
+assert "StartBoundaryNotFoundDefect" in mail.defects_categories
+assert "multipart/alternative" in mail.defects[0]
+assert "evidence.txt" in mail.message.epilogue
+assert mail.attachments[0]["filename"] == "evidence.txt"
+assert base64.b64decode(mail.attachments[0]["payload"]) == b"Hello"
+
+# 10. Multiple families coexist; inspect every entry rather than only defects[0].
+mail = mailparser.parse_from_string(
+    "From: billing@trusted.example <billing@vendor.example>\r\n"
+    "Date: 31 Feb 2024 12:00:00 +0000\r\n\r\n"
+)
+assert {"AddressHeaderDefect", "DateHeaderDefect"} <= mail.defects_categories
+assert mail.address_header_defects and mail.date_header_defects
+
+# 11. Valid controls: obsolete date syntax and an ordinary epilogue are accepted.
+for raw in [
+    "From: Alice <alice@example.com>\r\n"
+    "Date: 1 Jan 24 12:00:00 GMT\r\n\r\nHello",
+    'Content-Type: multipart/mixed; boundary="b"\r\n\r\n'
+    '--b\r\nContent-Type: text/plain\r\n\r\nHello\r\n'
+    '--b--\r\nOrdinary epilogue\r\n',
+]:
+    mail = mailparser.parse_from_string(raw)
+    assert mail.has_defects is False
+    assert mail.defects_categories == set()
+    assert mail.defects == []
+    assert mail.address_header_defects == []
+    assert mail.received_header_defects == []
+    assert mail.date_header_defects == []
+```
+
 Email structural defects are not merely technical curiosities—they represent **potential security
 vulnerabilities** that sophisticated attackers actively exploit to bypass spam filters, antivirus
 scanners, and email security gateways.

{mail_parser-4.8.0 → mail_parser-4.9.0}/README.md

@@ -366,6 +366,20 @@ An ambiguous trusted hop stops sender-IP attribution; older headers cannot
 supply a replacement IP. Balanced, unrecognized obsolete trace syntax remains
 raw without being declared invalid solely because it is obsolete.
 
+Envelope annotations (`envelope-from` and `envelope-sender`) are scanned
+without regex backtracking, preserving quoted delimiters, comments, domain
+literals and obsolete routes. Nested or unterminated envelope paths are not
+selected as addresses: `nested-envelope-address` or
+`unclosed-envelope-address` diagnostics retain the complete original field,
+including folding, while other trace clauses remain available. For these
+diagnostics, `recovered` indicates whether a later complete path was recovered
+from an affected clause after the outer path closes. Markers use ASCII
+case equivalence only; Unicode lookalikes remain literal evidence.
+Repeated reasons are aggregated per header to avoid
+duplicating the raw evidence for every malformed fragment. These annotations
+are lexical extractions, not full SMTP mailbox validation; the existing output
+for the valid null sender `<>` remains an absent envelope value.
+
 ### Date diagnostics
 
 Impossible calendar dates, clock components and numeric timezone minutes no
@@ -405,8 +419,272 @@ decoder that produces a non-Unicode scalar also triggers recovery: names retain
 their literal encoded form and text retains a safe byte-derived view, keeping
 JSON and CLI output encodable as UTF-8.
 
+Header decoding is strict. An unknown charset, invalid byte sequence or
+non-Unicode scalar leaves the complete original header value literal instead
+of dropping bytes or mixing a decoded prefix with the original. A
+`HeaderDecodeDefect` sets `has_defects`; `mail.header_decoding_defects` contains
+`reason`, `raw`, `header`, zero-based `occurrence`, zero-based MIME `part` and
+`recovered`. Here `recovered=True` means the literal evidence is available,
+not that decoding succeeded. This also covers nested non-address headers;
+address display names retain their existing context-aware diagnostics.
+Malformed encoded attachment names remain literal with `MimeHeaderDefect`.
+Raw Unicode text remains readable alongside encoded-words; decoding leaves
+literal spans intact and removes only ASCII folding whitespace between
+adjacent encoded-words. The optional RFC 2231 language suffix, such as
+`UTF-8*en`, is separated from the charset before strict decoding.
+
+### Resource limits for container representations
+
+Attached multipart containers and forwarded messages can repeat the same
+descendant payloads many times. Their optional textual representations share
+a per-message serialization budget. Preflight estimates reject expensive
+subtrees before copying; actual generator visits and writes to all internal
+text/byte buffers are also charged, including failed charset fallbacks.
+The character-work budget is 32 times the original tree's estimated size,
+with a floor of 1 Mi characters and a ceiling of 32 Mi characters. The estimate
+includes raw header/payload lengths and 256 characters of overhead per part.
+The node-visit budget is 16 times the number of MIME parts, with a floor of
+1,024 and a ceiling of 100,000. These are work units, not an input-size limit
+or an exact process-memory allowance.
+
+When a representation cannot fit, it is omitted entirely from `attachments`;
+no empty or truncated file is invented. `ResourceLimitDefect` sets
+`has_defects`, and `mail.resource_limit_defects` records its `reason`, MIME
+`part`, `filename` and `recovered=False`. Child attachments and neighboring
+body parts continue to be recovered under the existing disposition rules.
+The original container remains accessible at
+`list(mail.message.walk())[defect["part"]]` for ordinary MIME-tree parts.
+Recovered epilogue parts have indices after that original tree and remain
+traceable to the original epilogue text. Neither a resource limit nor a
+recovery diagnostic establishes malicious intent or invalid MIME syntax.
+
+Both new diagnostic properties always return lists. Nonempty lists appear
+in full and partial JSON output; literal wire headers colliding with those
+names remain accessible through `headers` and `message`. The serialization
+budget applies during parsing to container copies, not to input ingestion,
+individual leaf payloads, or an explicit later `message_as_string` request.
+Applications processing untrusted mail should still bound input size and
+isolate workers according to their own resource constraints.
+
 ## Defects and Their Critical Role in Email Security
 
+### Inspecting defects and recovered evidence
+
+A defect is a parsing anomaly, an inconsistent value, or a recovery/ambiguity
+diagnostic; it does not by itself establish malicious intent or an RFC violation.
+During parsing, mail-parser collects Python `email` defects from the MIME tree
+and adds its own address, trace, date, charset and MIME-metadata diagnostics:
+`mail.has_defects` becomes `True`, `mail.defects_categories` contains the category
+names, and `mail.defects` contains a list of dictionaries mapping a content type
+(for example `multipart/mixed`) or a diagnostic group (`address-headers`,
+`received-headers`, `date-headers`, `mime-headers`) to lists of textual
+descriptions. `AddressHeaderDefect`, `ReceivedHeaderDefect` and `DateHeaderDefect`
+also populate `mail.address_header_defects`, `mail.received_header_defects` and
+`mail.date_header_defects`, respectively; these properties always exist and are
+empty lists when there are no corresponding diagnostics. Their entries contain
+`reason`, `raw`, `header`, zero-based `occurrence` and `recovered`, plus address
+`candidates` or the selected ISO date `value` where applicable. All occurrences
+are examined, including repeated headers, while address properties and `mail.date`
+retain first-occurrence semantics. `recovered=True` means a value was recovered,
+not that the original input was valid; ambiguous address candidates remain
+evidence rather than selected mailboxes, and valid neighbours remain available.
+`CharsetDecodeDefect`, `MimeHeaderDefect` and collected standard-library defects
+have textual entries in `mail.defects`, without dedicated structured properties;
+the standard-library categories are described in
+[Python's defect reference](https://docs.python.org/3/library/email.errors.html),
+but this is not a guarantee that every defect Python can detect is collected
+(for example, transfer-decoding defects added after the MIME-tree collection may
+remain only in a part's `defects`). Full and partial output always include
+`has_defects`; when defects exist they include `defects` and
+`defects_categories`, with structured header diagnostics included when nonempty
+(JSON represents the category set as a list). Original evidence remains accessible
+through header properties such as `mail.from_raw`, `mail.message`, and individual
+MIME parts. A "bad epilogue" has no separate category or `epilogue_defects`
+property: `StartBoundaryNotFoundDefect` activates an attempt to recover a part
+between the declared boundary markers in the top-level `mail.message.epilogue`,
+and recovered content enters the usual text/attachment outputs without a separate
+recovery-success diagnostic. This is a limited recovery path, not a scan of every
+nested epilogue; an exception during this attempt is logged. An ordinary epilogue
+is permitted by [RFC 2046 section 5.1.1](https://www.rfc-editor.org/rfc/rfc2046.html#section-5.1.1)
+and is not itself a defect. The examples below cover each reporting family,
+successful and unsuccessful recovery, repeated headers, multiple simultaneous
+defects and valid controls; `has_defects=False` only means the implemented checks
+found no anomalies, not that the email passed a complete RFC validation.
+
+```python
+import base64
+import json
+import mailparser
+
+# 1. Recovered address: all four diagnostic views describe the same problem.
+mail = mailparser.parse_from_string(
+    "From: billing@trusted.example < billing@vendor.example >\r\n\r\n"
+)
+assert mail.from_ == [("billing@trusted.example", "billing@vendor.example")]
+assert mail.has_defects is True
+assert "AddressHeaderDefect" in mail.defects_categories
+assert mail.defects == [{"address-headers": [
+    "AddressHeaderDefect: from[0]: invalid-display-name"
+]}]
+assert mail.address_header_defects == [{
+    "reason": "invalid-display-name",
+    "raw": "billing@trusted.example < billing@vendor.example >",
+    "recovered": True,
+    "candidates": [{
+        "display_name": "billing@trusted.example",
+        "address": "billing@vendor.example",
+    }],
+    "header": "from",
+    "occurrence": 0,
+}]
+assert json.loads(mail.mail_json)["address_header_defects"] == (
+    mail.address_header_defects
+)
+
+# 2. Incomplete address: no selected mailbox; a valid neighbouring field survives.
+mail = mailparser.parse_from_string(
+    "From: Alice <alice@example.com\r\n"
+    "To: Bob <bob@example.com>\r\n\r\n"
+)
+assert mail.from_ == []
+assert mail.to == [("Bob", "bob@example.com")]
+assert mail.address_header_defects[0]["reason"] == "unclosed-delimiter"
+assert mail.address_header_defects[0]["recovered"] is False
+
+# 3. Repeated headers: diagnostics also cover occurrences after the first.
+mail = mailparser.parse_from_string(
+    "From: Alice <alice@example.com>\r\n"
+    "From: billing@trusted.example <billing@vendor.example>\r\n\r\n"
+)
+assert mail.from_ == [("Alice", "alice@example.com")]
+assert mail.address_header_defects[0]["occurrence"] == 1
+
+# 4. Received: ambiguous trace syntax stays raw, with structured diagnostics.
+trace = (
+    "from sender.example (unclosed; by mx.example; "
+    "1 Jan 2024 12:00:00 +0000"
+)
+mail = mailparser.parse_from_string(f"Received: {trace}\r\n\r\n")
+assert "ReceivedHeaderDefect" in mail.defects_categories
+assert mail.received_header_defects[0]["reason"] == "unclosed-comment"
+assert mail.received_header_defects[0]["recovered"] is False
+assert mail.received[0]["raw"] == trace
+assert "received-headers" in mail.defects[0]
+trace = (
+    "from sender.example by mx.example with ESMTP id token; "
+    "for <bob@example.com>; 1 Jan 2024 12:00:00 +0000"
+)
+mail = mailparser.parse_from_string(f"Received: {trace}\r\n\r\n")
+assert mail.received_header_defects[0]["reason"] == "misplaced-date-separator"
+assert mail.received_header_defects[0]["recovered"] is True
+assert mail.received[0]["for"] == "<bob@example.com>"
+
+# 5. Dates: an impossible date is not selected; a wrong weekday is recoverable.
+mail = mailparser.parse_from_string(
+    "Date: 31 Feb 2024 12:00:00 +0000\r\n\r\n"
+)
+assert "DateHeaderDefect" in mail.defects_categories
+assert mail.date is None
+assert mail.date_header_defects[0]["reason"] == "invalid-calendar-date"
+assert mail.date_header_defects[0]["recovered"] is False
+assert "date-headers" in mail.defects[0]
+mail = mailparser.parse_from_string(
+    "Date: Tue, 1 Jan 2024 12:00:00 +0000\r\n\r\n"
+)
+assert mail.date.isoformat() == "2024-01-01T12:00:00+00:00"
+assert mail.date_header_defects[0]["reason"] == "weekday-mismatch"
+assert mail.date_header_defects[0]["recovered"] is True
+# Date diagnostics also cover Resent-Date and timestamps inside Received.
+mail = mailparser.parse_from_string(
+    "Resent-Date: 31 Feb 2024 12:00:00 +0000\r\n"
+    "Received: from sender.example by mx.example; "
+    "1 Jan 2024 25:00:00 +0000\r\n\r\n"
+)
+assert {d["header"] for d in mail.date_header_defects} == {
+    "resent-date", "received"
+}
+
+# 6. Charset: failed declared decoding, then readable UTF-8 or replacement text.
+for payload, expected in [(b"caf\xc3\xa9", "café"), (b"a\xffb", "a\ufffdb")]:
+    mail = mailparser.parse_from_bytes(
+        b"Content-Type: text/plain; charset=ascii\r\n\r\n" + payload
+    )
+    assert "CharsetDecodeDefect" in mail.defects_categories
+    assert expected in mail.body
+    assert "CharsetDecodeDefect:" in mail.defects[0]["text/plain"][0]
+    assert mail.message.get_payload(decode=True) == payload
+
+# 7. MIME metadata: damaged filename bytes are retained in textual evidence.
+mail = mailparser.parse_from_bytes(
+    b"Content-Type: application/octet-stream\r\n"
+    b'Content-Disposition: attachment; filename="a\xffb.bin"\r\n'
+    b"Content-Transfer-Encoding: base64\r\n\r\nSGVsbG8="
+)
+assert "MimeHeaderDefect" in mail.defects_categories
+assert "\\xff" in " ".join(mail.defects[0]["mime-headers"])
+assert base64.b64decode(mail.attachments[0]["payload"]) == b"Hello"
+
+# 8. Standard-library structure defects: missing parameter/start/closing boundary.
+for raw, category in [
+    ("Content-Type: multipart/mixed\r\n\r\nHello",
+     "NoBoundaryInMultipartDefect"),
+    ('Content-Type: multipart/mixed; boundary="b"\r\n\r\nHello',
+     "StartBoundaryNotFoundDefect"),
+    ('Content-Type: multipart/mixed; boundary="b"\r\n\r\n'
+     '--b\r\nContent-Type: text/plain\r\n\r\nHello\r\n',
+     "CloseBoundaryNotFoundDefect"),
+    ("Subject: example\r\ninvalid header line\r\n\r\nHello",
+     "MissingHeaderBodySeparatorDefect"),
+]:
+    mail = mailparser.parse_from_string(raw)
+    assert mail.has_defects
+    assert category in mail.defects_categories
+    assert any(category in text for entry in mail.defects
+               for texts in entry.values() for text in texts)
+
+# 9. Bad epilogue: reusing the parent's boundary corrupts nested MIME structure.
+# The attachment after the first closing boundary lands in the outer epilogue.
+raw = (
+    'MIME-Version: 1.0\r\nContent-Type: multipart/mixed; boundary="b"\r\n\r\n'
+    '--b\r\nContent-Type: multipart/alternative; boundary="b"\r\n\r\n'
+    '--b\r\nContent-Type: text/plain\r\n\r\nHello\r\n--b--\r\n'
+    '--b\r\nContent-Type: application/octet-stream; name="evidence.txt"\r\n'
+    'Content-Disposition: attachment; filename="evidence.txt"\r\n'
+    'Content-Transfer-Encoding: base64\r\n\r\nSGVsbG8=\r\n--b--\r\n'
+)
+mail = mailparser.parse_from_string(raw)
+assert mail.has_defects
+assert "StartBoundaryNotFoundDefect" in mail.defects_categories
+assert "multipart/alternative" in mail.defects[0]
+assert "evidence.txt" in mail.message.epilogue
+assert mail.attachments[0]["filename"] == "evidence.txt"
+assert base64.b64decode(mail.attachments[0]["payload"]) == b"Hello"
+
+# 10. Multiple families coexist; inspect every entry rather than only defects[0].
+mail = mailparser.parse_from_string(
+    "From: billing@trusted.example <billing@vendor.example>\r\n"
+    "Date: 31 Feb 2024 12:00:00 +0000\r\n\r\n"
+)
+assert {"AddressHeaderDefect", "DateHeaderDefect"} <= mail.defects_categories
+assert mail.address_header_defects and mail.date_header_defects
+
+# 11. Valid controls: obsolete date syntax and an ordinary epilogue are accepted.
+for raw in [
+    "From: Alice <alice@example.com>\r\n"
+    "Date: 1 Jan 24 12:00:00 GMT\r\n\r\nHello",
+    'Content-Type: multipart/mixed; boundary="b"\r\n\r\n'
+    '--b\r\nContent-Type: text/plain\r\n\r\nHello\r\n'
+    '--b--\r\nOrdinary epilogue\r\n',
+]:
+    mail = mailparser.parse_from_string(raw)
+    assert mail.has_defects is False
+    assert mail.defects_categories == set()
+    assert mail.defects == []
+    assert mail.address_header_defects == []
+    assert mail.received_header_defects == []
+    assert mail.date_header_defects == []
+```
+
 Email structural defects are not merely technical curiosities—they represent **potential security
 vulnerabilities** that sophisticated attackers actively exploit to bypass spam filters, antivirus
 scanners, and email security gateways.

mail_parser-4.9.0/docs/adr/0001-bounded-forensic-recovery.md

@@ -0,0 +1,13 @@
+# Bound container representations while preserving original evidence
+
+Nested attached containers can require repeated copies of the same MIME
+subtrees. Keep existing representations while they fit a per-message work
+budget; omit expensive container representations with explicit diagnostics,
+continue recovering children, and retain the original MIME tree. Do not return
+empty or truncated attachment payloads. This preserves more evidence than
+rejecting the entire message and avoids changing ordinary attachment semantics.
+
+When an encoded-word cannot be decoded strictly, retain the complete literal
+header value and diagnose the failure, including its occurrence and MIME part.
+Do not discard undecodable bytes or combine a decoded prefix with the original
+header. A recovered result does not imply RFC compliance or malicious intent.

{mail_parser-4.8.0 → mail_parser-4.9.0}/src/mailparser/addresses.py

@@ -12,8 +12,9 @@ import re
 from dataclasses import dataclass, field
 from email.header import decode_header
 
+from mailparser.const import _ENCODED_WORD_RE as _ENCODED_WORD
+
 _FOLD = re.compile(r"\r?\n[ \t]+")
-_ENCODED_WORD = re.compile(r"=\?([^?\s]+)\?([bBqQ])\?([^?\s]*)\?=")
 # Anchored grammar checks: atom, quote and delimiter character sets do not
 # overlap. In particular, do not search for bare addresses in arbitrary text
 # or hand long malformed word sequences to headerregistry's quadratic parser.