mail-parser 4.6.0__tar.gz → 4.6.2__tar.gz

mail_parser-4.6.2/.claude/agents/security-reviewer.md

@@ -0,0 +1,161 @@
+---
+name: security-reviewer
+description: >
+  Security auditor for mail-parser and similar untrusted-input parsers. Reviews
+  code for real, reachable vulnerabilities — ReDoS, injection, path traversal,
+  unsafe subprocess/deserialization, MIME-structure and decode DoS,
+  parser-differential evasion, info leak — and also checks the safety invariants
+  (no network/XML/archive-extraction) still hold. PROVES each finding with a PoC
+  or measured blowup before reporting it. Use for
+  "security review", "audit for vulnerabilities", "check for security issues",
+  "is this exploitable". Threat model: input (email bytes/headers/attachments)
+  is fully attacker-controlled; the caller and filesystem are trusted.
+tools: [Read, Grep, Glob, Bash]
+model: opus
+---
+
+You audit code that parses attacker-controlled input. Every byte of an email —
+headers, folded whitespace, addresses, MIME structure, attachment names,
+payloads — is hostile. The calling application, its filesystem, and PATH are
+trusted. Judge each issue against that boundary; do not report threats that
+require an already-compromised host unless they escalate.
+
+## Prime directive: prove it, then report it
+
+A security finding you cannot demonstrate is a guess. Before you report:
+
+- **ReDoS / algorithmic complexity** — write a scaling probe. Feed the regex or
+  function graduated input sizes and measure. Report only if time grows
+  super-linearly (ratio ≥ ~3 per input doubling) AND the input is reachable
+  from the public API. Confirm reachability by driving it end-to-end
+  (`parse_from_string` / `parse_from_bytes`), not just the raw regex — a
+  pattern can be quadratic in isolation yet unreachable because an earlier step
+  normalizes the input. Guard each probe with `signal.setitimer` so a true
+  blowup fails fast instead of hanging.
+  **Complexity is not only regexes.** The dominant blowups in this codebase are
+  plain Python: a loop over attacker-chosen keys where each iteration rescans
+  the whole collection (`for k in keys: message.get_all(k)`) is O(keys × total)
+  with no regex involved. Vary the two axes *independently* — many distinct
+  header names vs. many repeats of one name — because a probe that only grows
+  the total byte count keeps the ratio flat and hides the quadratic term.
+- **Injection / traversal / write-primitive** — construct the malicious input
+  and show the resulting command, path, or file write. A `../` that
+  `os.path.basename` strips is not a finding.
+- **Resource / structure DoS** — build the crafted message (deeply nested or
+  very wide multipart, huge part count, oversized payload) and measure time and
+  memory end-to-end. Report only if consumption grows super-linearly or is
+  unbounded relative to input size. Note whether the MIME walk is iterative
+  (no stack overflow) before claiming a recursion bug.
+- **Parser-differential / evasion** — show two readings of the same bytes: what
+  the parser surfaces versus what the raw bytes contain. Dropped-undecodable
+  bytes (`errors="ignore"`) or Unicode normalization can hide an indicator a
+  downstream scanner would act on. The finding is the divergence, demonstrated.
+- If you cannot build a trigger, label it **UNCONFIRMED** and say what blocked
+  you. Never inflate an unproven concern to High.
+
+## What to examine (this codebase)
+
+| Surface                                               | Look for                                                                                                                                                                       |
+| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| Every `re.compile` / `re.*` on header or body text    | Nested/overlapping quantifiers, `(a+)+`, lazy-then-`\s*` overlap, unbounded `[^x]+`. Which normalization runs first (e.g. `JUNK_PATTERN` collapses `[\t\n]` but NOT spaces).   |
+| `subprocess` (`msgconvert`)                           | `shell=` must be false and args a list (no injection); missing `communicate(timeout=)` = hang DoS; `.decode()` without `errors=` = crash.                                      |
+| `tempfile` + `open(..., "w"/"wb")`                    | `mkstemp` (good) vs predictable names; leak on the exception path; symlink/TOCTOU before write.                                                                                |
+| Attachment filename → disk                            | `basename` + null-byte reject + realpath/`commonpath` containment + `islink`. Flag if the RAW (unsafe) name is also exposed to callers as a write target.                      |
+| Payload decode (`base64`, `get_payload(decode=True)`) | Unbounded in-memory amplification; malformed-input exceptions that crash the whole parse.                                                                                      |
+| MIME structure (`message.walk()`, `get_payload`)      | Part count, nesting depth, per-part size — no caps means a crafted multipart drives memory/time DoS. Measure scaling; confirm the walk is iterative before claiming recursion. |
+| Charset handling (`errors="ignore"`, `normalize`)     | Parser-differential / evasion: dropped bytes or NFC-normalized values make what the tool surfaces differ from the raw bytes a scanner sees. Show a mutation an analyst misses. |
+| Malformed-input robustness                            | One bad part must not abort the whole parse or kill the worker. Feed truncated/garbage MIME; an exception escaping the public API is a crash-DoS.                              |
+| Outlook `.msg` path (`extract_msg`, `msgconvert`)     | Untrusted `.msg` flows into third-party OLE/CFB + Perl parsers with their own CVE history — flag the transitive attack surface and the temp/subprocess handling around it.     |
+| Logging (`log.debug` of headers/filenames)            | Raw headers and attachment names are logged verbatim (PII / indicators). Data-handling note, not code-exec — report as Info.                                                   |
+| `eval`/`exec`/`pickle`/`yaml.load`/`__import__`       | Should be absent. Any occurrence is a finding until proven inert.                                                                                                              |
+| `getattr(self, X)` where `X` is a header/part name    | Reflective dispatch on an attacker-chosen name. See "Reflective dispatch" below — collision, recursion, and non-serializable leakage are all reachable from a 12-byte email.   |
+| `json.dumps` over a dynamically built dict            | Values arriving from reflection are not guaranteed JSON-safe. A bound method or `Message` object in the dict is an uncaught `TypeError` on a public property.                  |
+| `find()` / `in` / `split()` locating a security token | Naive substring search for a delimiter (`"by"`, a trust string) that an attacker also controls the *neighbouring* text of. See "Trust-boundary string parsing" below.          |
+
+## Reflective dispatch on attacker-controlled names
+
+`MailParser.__getattr__` exposes every header as an attribute, and `_make_mail`
+/ `headers` iterate `message.keys()` calling `getattr(self, name)`. The header
+name is attacker-chosen, so **the attacker picks which Python attribute is
+read**. Python resolves real class attributes *before* `__getattr__`, so any
+name in `dir(cls)` shadows the header path. Always run this probe:
+
+```python
+for a in [x for x in dir(MailParser) if not x.startswith("_")]:
+    try:
+        mailparser.parse_from_string(f"{a}: x\r\n\r\n").mail_json
+    except Exception as e:
+        print(a, type(e).__name__, e)
+```
+
+Three distinct bug classes fall out, and you must check for all three — finding
+one does not rule out the others:
+
+- **Method/property collision** — the dict gets a bound method or a live object
+  instead of a string. Downstream `json.dumps` raises `TypeError`. Crash-DoS on
+  a public API from a minimal message.
+- **Recursion cycle** — a property that itself iterates `message.keys()` and
+  calls `getattr` can re-enter itself when a header is named after that property
+  or one of its `_json` / `_raw` aliases. Check the cycle guard covers **every**
+  alias and is **case-insensitive**: `set(message.keys()) - {"headers"}` does not
+  exclude `Headers_json`, and `message.keys()` preserves the sender's casing.
+  A recursion cycle nested inside a per-key rescan multiplies the two costs —
+  measure it, it is usually the worst finding on the page.
+- **Side-effecting property** — reflection can *invoke* a property the caller
+  never asked for. Confirm no property in the collision set writes files, spawns
+  a subprocess, or mutates state.
+
+The fix to argue for is structural, not a denylist: header values must resolve
+through a lookup that never touches Python attributes. Reject patches that just
+add another name to an exclusion set — that is how the `headers_json` cycle
+survived the `headers` fix.
+
+## Trust-boundary string parsing
+
+`get_server_ipaddress(trust)` decides *which IP the mail came from* — its output
+is used for attribution and blocklisting, so a wrong answer is a security
+failure, not a cosmetic bug. Anywhere a security decision depends on locating a
+delimiter, check the search is anchored:
+
+- `header.find("by")` matches inside `derby.example.com` or `nearby.example.org`.
+  Hostnames come from the sender's HELO (no DNS control needed) and land in the
+  trusted MTA's own `Received` header.
+- Truncating the clause makes extraction fail on the *genuine* top hop, and the
+  loop then falls through to older, fully attacker-forged `Received` headers —
+  so the failure mode is not "returns nothing", it is "returns the attacker's
+  value". Always test the fall-through, not just the single-header case.
+- Use the existing anchored `const._CLAUSE_SPLITTER` rather than a bare `\bby\b`
+  (`\b` still matches inside `host.by.example`, since `.` is a non-word char).
+
+Prove these with a control matrix, not a single PoC: benign hostname, malicious
+hostname alone, forged header alone, and both together. If a benign hostname
+also misattributes, say so — it makes the finding a correctness bug too and
+raises the priority.
+
+## Invariants that must stay true (verify, don't assume)
+
+These hold in the current code and are cheap to re-check with a grep. A change
+that breaks one is a finding in itself — encode them so a future PR that adds
+the capability gets flagged instead of slipping in.
+
+- **No network access from parsing.** Nothing reached by parsed content may call
+  `urllib` / `socket` / `requests` / `smtplib`. A parser that fetches a URL, DTD,
+  or remote image is SSRF. `grep -rE "urllib|socket|requests|http" src/` must
+  stay free of live calls.
+- **No XML parser on mail content.** No `xml.*` / `lxml` / `etree`. Their absence
+  is what rules out XXE and entity-expansion (billion-laughs).
+- **No archive extraction.** Attachment code may read *names* (`.tar.gz`) but must
+  never `extractall` / open a payload with `zipfile` / `tarfile` — that would add
+  zip-slip and decompression-bomb surface.
+
+## Output
+
+Rank most-severe first. Per finding:
+
+- **Severity + CVSS 3.1 vector + CWE** (e.g. High — CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H = 7.5, CWE-1333).
+- **Location** `path:line`.
+- **Trigger** — the concrete input, and the measured evidence (size → time table, or the resulting command/path).
+- **Reachability** — the public call that reaches it.
+- **Fix** — the exact change (bound a quantifier, add `timeout=`, normalize first) and a failing regression test.
+
+No praise or filler. If nothing is exploitable, say so and list what you proved safe. You review; the caller fixes.

{mail_parser-4.6.0 → mail_parser-4.6.2}/.github/workflows/main.yml

@@ -69,7 +69,7 @@ jobs:
 
       - name: Publish to PyPI
         if: matrix.python-version == '3.10' && startsWith(github.ref, 'refs/tags/')
-        uses: pypa/gh-action-pypi-publish@v1.13.0
+        uses: pypa/gh-action-pypi-publish@v1.14.2
         with:
           user: ${{ secrets.PYPI_USERNAME }}
           password: ${{ secrets.PYPI_PASSWORD }}

{mail_parser-4.6.0 → mail_parser-4.6.2}/CLAUDE.md

@@ -61,6 +61,25 @@ Three access modes are supported for any attribute `X`:
 Address headers listed in `const.ADDRESSES_HEADERS` (`from`, `to`, `cc`, `bcc`, `reply-to`,
 `delivered-to`) return `list[tuple[str, str]]` (display_name, email) instead of plain strings.
 
+**Never resolve a header name with `getattr`, and never rewrite it**: `__getattr__` serves names
+the *caller* types, so it may fold `_`→`-`, honour the `_json`/`_raw` suffixes, and use `getattr`
+to reach computed parts (`attachments_json`). Names taken from a parsed message must go through
+`MailParser._header_value()`, which looks the name up literally in the header index and never
+touches a Python attribute. Both halves matter:
+
+- Python looks up real class attributes before `__getattr__`, so a sender naming a header `Parse`
+  or `Headers_json` otherwise picks which attribute is read — a bound method lands in the parsed
+  mail, or a property re-enters itself. Excluding individual names from a key set is not a fix;
+  that is what let `Headers_json` through after `headers` was excluded.
+- Suffix handling on a wire name is an amplifier and an evasion: each `_json` re-serializes the
+  previous result (five input bytes per doubling of the output), and `Subject_json:` reports the
+  value of `Subject` while silently dropping its own.
+
+**Header index**: `_build_header_index()` maps lowercased name → list of raw values once per
+parse. `Message.get_all()` is a linear scan, so the one call per distinct name previously made by
+`_make_mail()` cost O(distinct × total) on attacker-chosen names. Look up via the index inside any
+loop over header names.
+
 **RFC non-compliance fallback in `get_addresses()`**: Python's
 `email.utils.getaddresses(strict=True)` (hardened against CVE-2023-27043) rejects headers where
 the display name contains `@`. Since this is a forensics tool, `utils.py` applies
@@ -71,6 +90,28 @@ actually in the header.
 (`from`, `by`, `via`, `with`, `id`, `for`, `envelope-from`) using `const._CLAUSE_SPLITTER`.
 Output list is ordered first-hop first. Unparseable headers fall back to `{"raw": ...}`.
 
+**Sender-IP attribution fails closed**: `get_server_ipaddress()` walks trust-matching `Received`
+headers only while a hop names a *private* IP (an internal relay); a hop naming **no** IP ends the
+search with `None`. Never resume the walk on that case — older `Received` headers are written by
+the sender, so "extraction failed" would become "returns the sender's chosen IP".
+
+Candidate addresses come only from the `from` clause (`utils.get_from_clause()`, which ends at the
+next RFC 5321 keyword). Inside it, `_sender_ip_candidates()` applies one positional rule, because
+**text cannot be classified by what it looks like** — a closed `[...]` pair the sender wrote is
+byte-identical to one the MTA wrote, and `EHLO [8.8.8.8]` is a form RFC 5321 §4.1.3 requires:
+
+- the **first token** is the HELO name, whatever its shape, and is never a candidate;
+- an explicit **HELO marker inside a comment group** (`(helo=x)`, `(account a@b HELO x)`) is
+  sender text and is excluded, located with `const._HELO_RE`;
+- a candidate must sit **inside a `(`/`[` group** (`utils.group_spans()`), which is what makes a
+  clause truncated by a multi-word HELO fail closed — what it leaves behind is bare;
+- IPv4 and IPv6 matches are merged **positionally**, never family-first: choosing IPv4 first let
+  one private literal at EHLO suppress the IPv6 scan and hide the real sender.
+
+Only concession: `from [ip] (helo=x)` (Exim/CommuniGate), accepted when there is no other
+candidate and the marker is in a group. Do not add lookbehind guards to `_HELO_RE` to patch new
+cases — three rounds of that each reopened a hole the previous one closed.
+
 **Defect detection**: During `parse()`, every MIME part is walked and `_append_defects()` records
 RFC violations. `EPILOGUE_DEFECTS` triggers special epilogue extraction to recover hidden payloads
 in malformed boundaries.
@@ -87,6 +128,10 @@ Accessible as `parser.mail` / `parser.mail_partial`.
 To make a header always appear in partial output, add its lowercase name to `OTHERS_PARTS` in
 `const.py`. Address-type headers (returning parsed name/email tuples) go in `ADDRESSES_HEADERS`.
 
+If the new part is computed by a `MailParser` property rather than read off the wire, add it to
+`COMPUTED_PARTS` as well — that set is exactly what `_make_mail()` is allowed to resolve through
+attribute access.
+
 ## Workflow
 
 After every change:
@@ -95,3 +140,8 @@ After every change:
 1. Update README.md if the change affects usage, API, or setup.
 1. Stage changes and run pre-commit; fix all reported issues before proceeding.
 1. Run full test suite; fix all failures before reporting done.
+1. Run a security review of the change with the `security-reviewer` sub-agent
+   (`.claude/agents/security-reviewer.md`). All parsed input is
+   attacker-controlled, so any change to parsing, regexes, subprocess, temp
+   files, or attachment handling must be reviewed. Reproduce and fix every
+   High/Medium finding (with a regression test) before reporting done.

{mail_parser-4.6.0 → mail_parser-4.6.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mail-parser
-Version: 4.6.0
+Version: 4.6.2
 Summary: A tool that parses emails by enhancing the Python standard library, extracting all details into a comprehensive object.
 Author-email: Fedele Mantuano <mantuano.fedele@gmail.com>
 Maintainer-email: Fedele Mantuano <mantuano.fedele@gmail.com>
@@ -206,9 +206,16 @@ pipelines.
 - `body` - Complete message body
 - `text_html` - HTML body parts (list)
 - `text_plain` - Plain text body parts (list)
-- `headers` - All headers as a structured object
+- `headers` - All headers as a structured object. Keys are the header names exactly as they
+  appear in the message, and every header is reported — including names that match a
+  `MailParser` method or property (`Parse`, `Message`, `Headers_json`) and names containing
+  underscores (`X_Spam_Flag`). Such names are always resolved as headers, never as attributes,
+  and are never rewritten.
 - `attachments` - Complete attachment metadata and payloads
-- `get_server_ipaddress()` - Reliable sender IP extraction with trust levels
+- `get_server_ipaddress()` - Reliable sender IP extraction with trust levels. Only the `from`
+  clause of the first trusted `Received` header is searched, with the sender-supplied HELO name
+  removed, and the result is `None` when that hop names no public IP. Attribution never falls
+  back to older `Received` headers, which the sender is free to forge.
 - `to_domains` - Extracted recipient domains for analysis
 - `timezone` - Detected timezone information
 - `defects` - RFC compliance issues for security analysis
@@ -236,6 +243,12 @@ access the `X-MSMail-Priority` header:
 mail.X_MSMail_Priority
 ```
 
+This underscore-for-hyphen convenience, and the `_json` / `_raw` suffixes, apply only to
+attribute access written by you. Names read out of a message — the keys of `mail` and `headers` —
+are looked up literally, so a header genuinely named `X_Spam_Flag` or `Subject_json` keeps its own
+name and its own value. Attribute names beginning with an underscore are not headers and raise
+`AttributeError`.
+
 The `received` header is intelligently parsed into individual hops, revealing the complete email
 routing path. Each hop contains structured fields:
 

{mail_parser-4.6.0 → mail_parser-4.6.2}/README.md

@@ -179,9 +179,16 @@ pipelines.
 - `body` - Complete message body
 - `text_html` - HTML body parts (list)
 - `text_plain` - Plain text body parts (list)
-- `headers` - All headers as a structured object
+- `headers` - All headers as a structured object. Keys are the header names exactly as they
+  appear in the message, and every header is reported — including names that match a
+  `MailParser` method or property (`Parse`, `Message`, `Headers_json`) and names containing
+  underscores (`X_Spam_Flag`). Such names are always resolved as headers, never as attributes,
+  and are never rewritten.
 - `attachments` - Complete attachment metadata and payloads
-- `get_server_ipaddress()` - Reliable sender IP extraction with trust levels
+- `get_server_ipaddress()` - Reliable sender IP extraction with trust levels. Only the `from`
+  clause of the first trusted `Received` header is searched, with the sender-supplied HELO name
+  removed, and the result is `None` when that hop names no public IP. Attribution never falls
+  back to older `Received` headers, which the sender is free to forge.
 - `to_domains` - Extracted recipient domains for analysis
 - `timezone` - Detected timezone information
 - `defects` - RFC compliance issues for security analysis
@@ -209,6 +216,12 @@ access the `X-MSMail-Priority` header:
 mail.X_MSMail_Priority
 ```
 
+This underscore-for-hyphen convenience, and the `_json` / `_raw` suffixes, apply only to
+attribute access written by you. Names read out of a message — the keys of `mail` and `headers` —
+are looked up literally, so a header genuinely named `X_Spam_Flag` or `Subject_json` keeps its own
+name and its own value. Attribute names beginning with an underscore are not headers and raise
+`AttributeError`.
+
 The `received` header is intelligently parsed into individual hops, revealing the complete email
 routing path. Each hop contains structured fields:
 

{mail_parser-4.6.0 → mail_parser-4.6.2}/pyproject.toml

@@ -49,7 +49,9 @@ test = [
 ]
 
 [build-system]
-requires = ["hatchling"]
+# hatchling >=1.32 emits core metadata 2.5, which older twine/packaging
+# in the publishing toolchain cannot parse. Cap until PyPI tooling catches up.
+requires = ["hatchling>=1.27,<1.32"]
 build-backend = "hatchling.build"
 
 [tool.uv]

mail_parser-4.6.2/src/mailparser/const.py

@@ -0,0 +1,167 @@
+#!/usr/bin/env python
+
+"""
+Copyright 2018 Fedele Mantuano (https://twitter.com/fedelemantuano)
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import re
+
+# IPv4 pattern - validates octet range (0-255) per RFC 791
+REGXIP = re.compile(
+    r"(?:(?:25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}"
+    r"(?:25[0-5]|2[0-4]\d|[01]?\d\d?)"
+)
+
+# IPv6 pattern - matches standard and common compressed forms per RFC 5952
+# Alternation order matters: Python's ``re`` takes the first alternative
+# that matches, not the longest.  The "trailing ::" branch therefore has to
+# come *after* every branch that continues past the ``::`` — with it listed
+# early, ``2a00:1450:4864:20::32`` matched only as ``2a00:1450:4864:20::``,
+# reporting a different, valid, routable address as the sender.
+REGXIP6 = re.compile(
+    r"(?:(?:[0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}"  # full form
+    r"|[0-9a-fA-F]{1,4}:(?::[0-9a-fA-F]{1,4}){1,6}"  # 6 groups after ::
+    r"|(?:[0-9a-fA-F]{1,4}:){1,2}(?::[0-9a-fA-F]{1,4}){1,5}"
+    r"|(?:[0-9a-fA-F]{1,4}:){1,3}(?::[0-9a-fA-F]{1,4}){1,4}"
+    r"|(?:[0-9a-fA-F]{1,4}:){1,4}(?::[0-9a-fA-F]{1,4}){1,3}"
+    r"|(?:[0-9a-fA-F]{1,4}:){1,5}(?::[0-9a-fA-F]{1,4}){1,2}"
+    r"|(?:[0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}"  # 1 group after ::
+    r"|:(?::[0-9a-fA-F]{1,4}){1,7}"  # ::x:x...
+    r"|(?:[0-9a-fA-F]{1,4}:){1,7}:"  # trailing ::
+    r"|::)"  # just ::
+)
+
+# Normalize whitespace: collapse tabs and newlines to single space.
+# Parenthesized comments and bracketed IPs are preserved.
+JUNK_PATTERN = r"[\t\n]+"
+
+# ------------------------------------------------------------------ #
+# Received header parsing — RFC 5321 §4.4 grammar:
+#
+#   Received     = "Received:" *( received-token / comment ) ";" date-time
+#   received-token = "from" domain / "by" domain / "via" atom
+#                  / "with" atom  / "id"  atom   / "for" addr-spec
+#
+# Strategy: tokenize on clause keywords, then extract values per clause.
+# This eliminates the duplicated boundary lookaheads of the old
+# per-clause pattern list and matches the RFC grammar directly.
+# ------------------------------------------------------------------ #
+
+# Pattern that splits a received header into clause tokens.
+# Matches each RFC 5321 keyword at a word boundary followed by its value,
+# which extends up to the next keyword or semicolon.
+# The keywords are: from, by, via, with (not "with cipher"), id, for,
+# plus the non-standard envelope-from and envelope-sender.
+_CLAUSE_SPLITTER = re.compile(
+    r"(?:^|\s+)"
+    r"(from|by|via|with(?!\s+cipher)|id|for|envelope-from|envelope-sender)"
+    r"\s+",
+    re.I,
+)
+
+# Collapses any run of whitespace to a single space.  Applied to a received
+# header before ``_CLAUSE_SPLITTER`` so the splitter's ``\s+`` sub-patterns
+# only ever match a single character.  Without it a long run of spaces with no
+# following clause keyword drives the split into quadratic backtracking — an
+# ordinary parse of an attacker-supplied Received header becomes a denial of
+# service (CWE-1333).  Note the ``[\t\n]``-only normalization elsewhere does
+# not collapse spaces, so this guard must live at the point of use.
+_WS_RUN_RE = re.compile(r"\s+")
+
+# Matches the HELO/EHLO name an MTA records inside the ``from`` clause,
+# in the two shapes seen in the wild:
+#   from evil.example ([203.0.113.9]:45321 helo=[8.8.8.8]) by ...   (Exim)
+#   from [203.0.113.9] (account x@y HELO 8.8.8.8) by ...      (CommuniGate)
+# The HELO argument is chosen by the sender and may be an RFC 5321 address
+# literal, so it must be removed before scanning the clause for the sender
+# IP — otherwise the sender simply appends the IP they want reported.
+# Used to locate the span, never to delete it: the trailing \S+ would
+# otherwise swallow whatever follows, and a sender whose rDNS or HELO is
+# literally ``helo`` can place that word right before the MTA-written IP.
+# Two guards keep the span off MTA-written text.  The lookbehind: the
+# clause value *starts* with the HELO name, so at offset 0 the word is the
+# name itself, not a label introducing one.  The ``(?![\[(])`` on the
+# space-separated form: an MTA writes its own address inside brackets or
+# parentheses, so a following group is never part of the HELO argument.
+# Exim's ``helo=[8.8.8.8]`` is a genuine bracketed HELO argument and keeps
+# no such guard.
+_HELO_RE = re.compile(
+    r"(?<=[(\s])e?helo\s*=\S*"  # Exim "helo=value": after "(" or space
+    r"|(?<=\s)e?helo\s+\S+",  # space form: only after whitespace
+    re.I,
+)
+
+# RFC 5321 §4.1.3 tags an IPv6 literal as ``[IPv6:2a00:...]``.  REGXIP6
+# happily starts matching at the ``6`` of the tag and yields
+# ``6:2a00:1450:4864:20::`` — a different, valid, routable address that an
+# analyst would then act on.  Blank the tag before scanning.
+_IPV6_TAG_RE = re.compile(r"IPv6:", re.I)
+
+# Extracts envelope-from email: envelope-from <addr>
+_ENVELOPE_FROM_RE = re.compile(r"<([^>]+)>")
+
+# Date after semicolon (standard RFC 5321)
+_DATE_RE = re.compile(r";\s*(.*)", re.DOTALL)
+
+# SendGrid non-standard date format (no semicolon)
+_SENDGRID_DATE_RE = re.compile(
+    r"(\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\.\d{9}\s+\+0000\s+UTC)"
+    r"\s+m=\+\d+\.\d+",
+    re.I,
+)
+
+EPILOGUE_DEFECTS = {"StartBoundaryNotFoundDefect"}
+
+ADDRESSES_HEADERS = set(["bcc", "cc", "delivered-to", "from", "reply-to", "to"])
+
+# These parts are always returned
+OTHERS_PARTS = set(
+    [
+        "attachments",
+        "body",
+        "date",
+        "message-id",
+        "received",
+        "subject",
+        "timezone",
+        "to_domains",
+        "user-agent",
+        "x-mailer",
+        "x-original-to",
+    ]
+)
+
+# Subset of OTHERS_PARTS that MailParser computes itself: each maps to a
+# property, not to a header read off the wire.  ``_make_mail()`` resolves
+# only these names through attribute access; every other key it handles is
+# a header name chosen by the sender and goes through
+# ``MailParser._header_value()``, which never touches an attribute.  Keep
+# this set in sync with OTHERS_PARTS and with the properties in core.py.
+#
+# These names shadow a header of the same name in ``mail`` / ``mail_json``:
+# a message carrying a literal ``Body:`` header reports the computed body
+# there, not the header value.  The wire value is never lost — it is in
+# ``headers`` / ``headers_json`` under its own name — but a consumer
+# reading only ``mail_json`` will not see it.
+COMPUTED_PARTS = set(
+    [
+        "attachments",
+        "body",
+        "date",
+        "received",
+        "timezone",
+        "to_domains",
+    ]
+)