mailreport 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +9 -0
- data/LICENSE +21 -0
- data/README.md +146 -0
- data/lib/mailreport/archive.rb +118 -0
- data/lib/mailreport/dmarc/document.rb +180 -0
- data/lib/mailreport/dmarc/report.rb +57 -0
- data/lib/mailreport/dmarc.rb +16 -0
- data/lib/mailreport/tls_rpt.rb +134 -0
- data/lib/mailreport/version.rb +3 -0
- data/lib/mailreport.rb +8 -0
- data/mailreport.gemspec +37 -0
- metadata +83 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 4144dc7ef1d3210db021841accb097def4cf61d4edd582064920f994da16a262
|
|
4
|
+
data.tar.gz: cdeffe809666d65141cbbc169d110eadb3be17a30bfaf6311b5382af0ddd69ee
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: fe3ab048394c0dd88eb5d8ff99f1a0be42eff511c2add007753e39423e75d01d156a6815b07853a78469c8e7e839e01ffc265b404648a542494e21799b14f092
|
|
7
|
+
data.tar.gz: 8d59fb0ec330e5e5099ebeac2bd15420b9f53257a1a0f5ebb3b6c420de653ffcf26577396093f87aec4c86a26102ae1b4696a91824de7e7489addb9a1dc8cf62
|
data/CHANGELOG.md
ADDED
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 PostRider
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# mailreport
|
|
2
|
+
|
|
3
|
+
📧 Read the reports receivers send back about your domain
|
|
4
|
+
|
|
5
|
+
Answers one question: what are receivers seeing?.
|
|
6
|
+
|
|
7
|
+
## Features
|
|
8
|
+
|
|
9
|
+
- DMARC aggregate reports (RFC 7489 §7.2), gzipped, zipped, or bare
|
|
10
|
+
- SMTP TLS reports (RFC 8460), gzipped, zipped, or bare
|
|
11
|
+
- Bounded against hostile input, since the address these arrive at is published in DNS
|
|
12
|
+
|
|
13
|
+
> A report is only as truthful as the receiver that filed it. This library reports what a document says, not whether it is true.
|
|
14
|
+
|
|
15
|
+
## Contents
|
|
16
|
+
|
|
17
|
+
- [Getting Started](#getting-started)
|
|
18
|
+
- [DMARC Aggregate Reports](#dmarc-aggregate-reports)
|
|
19
|
+
- [SMTP TLS Reports](#smtp-tls-reports)
|
|
20
|
+
- [What Is Not Filled In](#what-is-not-filled-in)
|
|
21
|
+
- [Unreadable Documents](#unreadable-documents)
|
|
22
|
+
- [Testing](#testing)
|
|
23
|
+
- [Contributing](#contributing)
|
|
24
|
+
- [License](#license)
|
|
25
|
+
|
|
26
|
+
## Getting Started
|
|
27
|
+
|
|
28
|
+
```ruby
|
|
29
|
+
gem "mailreport"
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Requires Ruby >= 3.3.4
|
|
33
|
+
|
|
34
|
+
## DMARC Aggregate Reports
|
|
35
|
+
|
|
36
|
+
A domain asks for these with the `rua` tag of its DMARC record, and they arrive as attachments to ordinary mail. Hand over the attachment as it came:
|
|
37
|
+
|
|
38
|
+
```ruby
|
|
39
|
+
report = MailReport::Dmarc.parse(attachment_bytes)
|
|
40
|
+
|
|
41
|
+
report.organization # => "google.com"
|
|
42
|
+
report.report_id # => "14054835605949254452"
|
|
43
|
+
report.range # => 2026-07-27 00:00:00 UTC..2026-07-27 23:59:59 UTC
|
|
44
|
+
|
|
45
|
+
report.policy.domain # => "example.com"
|
|
46
|
+
report.policy.p # => "reject"
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Each record is one sending address over the window, with the count of messages it sent:
|
|
50
|
+
|
|
51
|
+
```ruby
|
|
52
|
+
report.records.each do |record|
|
|
53
|
+
record.source_ip # => "203.0.113.9"
|
|
54
|
+
record.count # => 42
|
|
55
|
+
record.disposition # => "none" - what the receiver did
|
|
56
|
+
record.dkim # => "pass" - aligned, as DMARC judged it
|
|
57
|
+
record.spf # => "pass"
|
|
58
|
+
end
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
`record.dkim` and `record.spf` are the **aligned** verdicts DMARC reached. Whether the checks themselves passed is a different question, and it's in `auth_results`:
|
|
62
|
+
|
|
63
|
+
```ruby
|
|
64
|
+
record.auth_results.dkim # => [#<DkimAuthResult domain: "example.com", selector: "s1", result: "pass", ...>]
|
|
65
|
+
record.auth_results.spf # => [#<SpfAuthResult domain: "example.com", scope: "mfrom", result: "pass">]
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
A message can carry a valid signature from a domain that doesn't align, and that pair is where it shows.
|
|
69
|
+
|
|
70
|
+
Where a receiver applied something milder than the published policy, it says why:
|
|
71
|
+
|
|
72
|
+
```ruby
|
|
73
|
+
record.reasons # => [#<Reason type: "forwarded", comment: "looks forwarded">]
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Worth reading before concluding anything from a `fail` - forwarding breaks SPF and often DKIM, and a receiver that noticed will say so here.
|
|
77
|
+
|
|
78
|
+
## SMTP TLS Reports
|
|
79
|
+
|
|
80
|
+
RFC 8460 reports arrive the same way. Parse the attachment bytes:
|
|
81
|
+
|
|
82
|
+
```ruby
|
|
83
|
+
report = MailReport::TlsRpt.parse(attachment_bytes)
|
|
84
|
+
|
|
85
|
+
report.organization # => "Company-X"
|
|
86
|
+
report.report_id # => "5065427c-23d3-47ca-b6e0-946ea0e8c4be"
|
|
87
|
+
report.range # => 2016-04-01 00:00:00 UTC..2016-04-01 23:59:59 UTC
|
|
88
|
+
|
|
89
|
+
report.policies.each do |policy|
|
|
90
|
+
policy.type # => "sts"
|
|
91
|
+
policy.domain # => "company-y.example"
|
|
92
|
+
policy.successes # => 5326
|
|
93
|
+
policy.failures # => 303
|
|
94
|
+
end
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## What Is Not Filled In
|
|
98
|
+
|
|
99
|
+
Elements the sender omitted come back `nil`, including ones the RFC gives defaults for:
|
|
100
|
+
|
|
101
|
+
```ruby
|
|
102
|
+
report.policy.adkim # => nil, where the sender said nothing
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
"The sender omitted `adkim`" and "the sender asked for relaxed alignment" are different facts, and only the first is in the document. Applying the RFC's defaults is left to the reader.
|
|
106
|
+
|
|
107
|
+
For the same reason, nothing here counts. Totals, alignment rates, and what counts as alarming are a consumer's to compute.
|
|
108
|
+
|
|
109
|
+
## Unreadable Documents
|
|
110
|
+
|
|
111
|
+
Nothing raises. A document that can't be read is `nil`:
|
|
112
|
+
|
|
113
|
+
```ruby
|
|
114
|
+
MailReport::Dmarc.parse(garbage) # => nil
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
This is deliberately different from a report carrying no records - that one is a receiver saying it saw no mail, and comes back as a report with an empty `records`.
|
|
118
|
+
|
|
119
|
+
A `rua` address is published in DNS, so anyone can send anything to it, and the limits assume as much:
|
|
120
|
+
|
|
121
|
+
- **Entity declarations are refused.** A report has none of its own, so the expansion bomb is refused with them. Escapes a real document needs (`&`, `é`) still read.
|
|
122
|
+
- **Decompression stops at 25 MB** rather than running to completion. Lower it where you'd rather hold less:
|
|
123
|
+
|
|
124
|
+
```ruby
|
|
125
|
+
MailReport::Archive.open(bytes, max_size: 1024 * 1024)
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
- **External references are never resolved** - no document reads a file, a URL, or a DNS name.
|
|
129
|
+
- **A zip entry is bounded as it inflates**, not on the size it claims.
|
|
130
|
+
|
|
131
|
+
Where an archive holds several entries, DMARC prefers a `.xml` name and TLS a `.json` name; otherwise the first non-empty file is read. Directories and empty placeholders are skipped.
|
|
132
|
+
|
|
133
|
+
## Testing
|
|
134
|
+
|
|
135
|
+
```sh
|
|
136
|
+
bundle install
|
|
137
|
+
rake
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
## Contributing
|
|
141
|
+
|
|
142
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md). Bug reports: [GitHub issues](https://github.com/mailpiece/mailreport/issues). Security: [SECURITY.md](SECURITY.md).
|
|
143
|
+
|
|
144
|
+
## License
|
|
145
|
+
|
|
146
|
+
MIT. See [LICENSE](LICENSE).
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
require "stringio"
|
|
2
|
+
require "zip_kit"
|
|
3
|
+
require "zlib"
|
|
4
|
+
|
|
5
|
+
module MailReport
|
|
6
|
+
# Attachment wrapping per RFC 7489 §7.2.1.1 / RFC 8460 §3. Detected from
|
|
7
|
+
# magic bytes, not the filename.
|
|
8
|
+
module Archive
|
|
9
|
+
GZIP = "\x1f\x8b".b
|
|
10
|
+
ZIP = "PK".b
|
|
11
|
+
|
|
12
|
+
MAX_SIZE = 25 * 1024 * 1024
|
|
13
|
+
CHUNK_SIZE = 64 * 1024
|
|
14
|
+
STORED = 0
|
|
15
|
+
|
|
16
|
+
UNREADABLE = [
|
|
17
|
+
ZipKit::FileReader::ReadError, ZipKit::FileReader::MissingEOCD,
|
|
18
|
+
ZipKit::FileReader::UnsupportedFeature, Zlib::Error
|
|
19
|
+
].freeze
|
|
20
|
+
|
|
21
|
+
class << self
|
|
22
|
+
# `name` prefers a zip entry whose filename matches (e.g. /\.xml\z/i).
|
|
23
|
+
def open(bytes, max_size: MAX_SIZE, name: nil)
|
|
24
|
+
bytes = bytes.to_s.b
|
|
25
|
+
|
|
26
|
+
if bytes.empty?
|
|
27
|
+
nil
|
|
28
|
+
elsif gzipped?(bytes)
|
|
29
|
+
inflated(bytes, max_size)
|
|
30
|
+
elsif zipped?(bytes)
|
|
31
|
+
unzipped(bytes, max_size, name)
|
|
32
|
+
elsif bytes.bytesize <= max_size
|
|
33
|
+
bytes
|
|
34
|
+
end
|
|
35
|
+
rescue *UNREADABLE
|
|
36
|
+
nil
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
private
|
|
40
|
+
def gzipped?(bytes)
|
|
41
|
+
bytes.start_with?(GZIP)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def inflated(bytes, max_size)
|
|
45
|
+
Zlib::GzipReader.wrap(StringIO.new(bytes)) do |gzip|
|
|
46
|
+
inflated = gzip.read(max_size + 1)
|
|
47
|
+
|
|
48
|
+
inflated if inflated && inflated.bytesize <= max_size
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def zipped?(bytes)
|
|
53
|
+
bytes.start_with?(ZIP)
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def unzipped(bytes, max_size, name)
|
|
57
|
+
archive = StringIO.new(bytes)
|
|
58
|
+
|
|
59
|
+
if entry = document_entry(archive, name)
|
|
60
|
+
archive.seek(entry.compressed_data_offset)
|
|
61
|
+
content(entry, archive, max_size)
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
# Prefer a report-shaped name when the caller gives one; otherwise the
|
|
66
|
+
# first non-empty file. Directories and empty placeholders are skipped.
|
|
67
|
+
def document_entry(archive, name)
|
|
68
|
+
entries = ZipKit::FileReader.read_zip_structure(io: archive).select do |entry|
|
|
69
|
+
!entry.filename.end_with?("/") && entry.uncompressed_size.positive?
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
if name
|
|
73
|
+
entries.find { |entry| entry.filename.match?(name) } || entries.first
|
|
74
|
+
else
|
|
75
|
+
entries.first
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
def content(entry, archive, max_size)
|
|
80
|
+
if entry.storage_mode == STORED
|
|
81
|
+
stored(entry, archive, max_size)
|
|
82
|
+
else
|
|
83
|
+
deflated(archive, max_size)
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
# Require the declared byte count so a truncated entry is not mistaken
|
|
88
|
+
# for a complete document.
|
|
89
|
+
def stored(entry, archive, max_size)
|
|
90
|
+
read = archive.read([ entry.compressed_size, max_size + 1 ].min)
|
|
91
|
+
|
|
92
|
+
read if read && read.bytesize == entry.compressed_size && read.bytesize <= max_size
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
# Raw deflate (zip), bounded on inflated output. Require finished? so a
|
|
96
|
+
# truncated stream is not mistaken for a complete document.
|
|
97
|
+
def deflated(archive, max_size)
|
|
98
|
+
inflater = Zlib::Inflate.new(-Zlib::MAX_WBITS)
|
|
99
|
+
inflated = +"".b
|
|
100
|
+
|
|
101
|
+
catch(:full) do
|
|
102
|
+
until inflater.finished?
|
|
103
|
+
piece = archive.read(CHUNK_SIZE) or break
|
|
104
|
+
|
|
105
|
+
inflater.inflate(piece) do |chunk|
|
|
106
|
+
inflated << chunk
|
|
107
|
+
throw :full if inflated.bytesize > max_size
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
inflated if inflater.finished? && inflated.bytesize <= max_size
|
|
113
|
+
ensure
|
|
114
|
+
inflater.close if inflater && !inflater.closed?
|
|
115
|
+
end
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
end
|
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
require "rexml/document"
|
|
2
|
+
|
|
3
|
+
require "mailreport/dmarc/report"
|
|
4
|
+
|
|
5
|
+
module MailReport
|
|
6
|
+
module Dmarc
|
|
7
|
+
# RFC 7489 Appendix C.
|
|
8
|
+
class Document
|
|
9
|
+
ROOT = "feedback"
|
|
10
|
+
|
|
11
|
+
PREDEFINED_ENTITIES = %w[ lt gt amp quot apos ].freeze
|
|
12
|
+
|
|
13
|
+
# REXML raises a bare RuntimeError when expansion limits trip; match the
|
|
14
|
+
# message so our own RuntimeErrors are not swallowed as unreadable input.
|
|
15
|
+
EXPANSION_ABANDONED = /entity expansions exceeded|entity expansion has grown too large/
|
|
16
|
+
|
|
17
|
+
def initialize(xml)
|
|
18
|
+
@xml = xml
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def report
|
|
22
|
+
if root = feedback
|
|
23
|
+
build(root)
|
|
24
|
+
end
|
|
25
|
+
rescue REXML::ParseException
|
|
26
|
+
nil
|
|
27
|
+
rescue RuntimeError => error
|
|
28
|
+
if error.message.match?(EXPANSION_ABANDONED)
|
|
29
|
+
nil
|
|
30
|
+
else
|
|
31
|
+
raise
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
private
|
|
36
|
+
def feedback
|
|
37
|
+
document = REXML::Document.new(@xml)
|
|
38
|
+
|
|
39
|
+
document.root if readable?(document) && document.root&.name == ROOT
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
# Refuse any declared entity before text is read (expansion timing).
|
|
43
|
+
def readable?(document)
|
|
44
|
+
declared_entities(document).empty?
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
def declared_entities(document)
|
|
48
|
+
document.doctype&.entities&.keys.to_a - PREDEFINED_ENTITIES
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def build(root)
|
|
52
|
+
metadata = child(root, "report_metadata")
|
|
53
|
+
|
|
54
|
+
Report.new(
|
|
55
|
+
organization: text(metadata, "org_name"),
|
|
56
|
+
email: text(metadata, "email"),
|
|
57
|
+
extra_contact_info: text(metadata, "extra_contact_info"),
|
|
58
|
+
report_id: text(metadata, "report_id"),
|
|
59
|
+
range: range(metadata),
|
|
60
|
+
errors: errors(metadata),
|
|
61
|
+
policy: policy(child(root, "policy_published")),
|
|
62
|
+
records: records(root)
|
|
63
|
+
)
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def errors(metadata)
|
|
67
|
+
elements(metadata, "error").filter_map { |element| text_of(element) }
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
# Epoch seconds (§7.2), as UTC.
|
|
71
|
+
def range(metadata)
|
|
72
|
+
dates = child(metadata, "date_range")
|
|
73
|
+
starts, ends = integer(dates, "begin"), integer(dates, "end")
|
|
74
|
+
|
|
75
|
+
Time.at(starts).utc..Time.at(ends).utc if starts && ends && starts <= ends
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
def policy(element)
|
|
79
|
+
Policy.new(
|
|
80
|
+
domain: text(element, "domain"),
|
|
81
|
+
adkim: text(element, "adkim"),
|
|
82
|
+
aspf: text(element, "aspf"),
|
|
83
|
+
p: text(element, "p"),
|
|
84
|
+
sp: text(element, "sp"),
|
|
85
|
+
pct: counted(element, "pct"),
|
|
86
|
+
fo: text(element, "fo")
|
|
87
|
+
)
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def records(root)
|
|
91
|
+
elements(root, "record").map { |element| record(element) }
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def record(element)
|
|
95
|
+
row = child(element, "row")
|
|
96
|
+
evaluated = child(row, "policy_evaluated")
|
|
97
|
+
|
|
98
|
+
Record.new(
|
|
99
|
+
source_ip: text(row, "source_ip"),
|
|
100
|
+
count: count_of(row),
|
|
101
|
+
disposition: text(evaluated, "disposition"),
|
|
102
|
+
dkim: text(evaluated, "dkim"),
|
|
103
|
+
spf: text(evaluated, "spf"),
|
|
104
|
+
reasons: reasons(evaluated),
|
|
105
|
+
identifiers: identifiers(child(element, "identifiers")),
|
|
106
|
+
auth_results: auth_results(child(element, "auth_results"))
|
|
107
|
+
)
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
def reasons(evaluated)
|
|
111
|
+
elements(evaluated, "reason").map do |element|
|
|
112
|
+
Reason.new(type: text(element, "type"), comment: text(element, "comment"))
|
|
113
|
+
end
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
def identifiers(element)
|
|
117
|
+
Identifiers.new(
|
|
118
|
+
header_from: text(element, "header_from"),
|
|
119
|
+
envelope_from: text(element, "envelope_from"),
|
|
120
|
+
envelope_to: text(element, "envelope_to")
|
|
121
|
+
)
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
def auth_results(element)
|
|
125
|
+
AuthResults.new(dkim: dkim_results(element), spf: spf_results(element))
|
|
126
|
+
end
|
|
127
|
+
|
|
128
|
+
def dkim_results(element)
|
|
129
|
+
elements(element, "dkim").map do |dkim|
|
|
130
|
+
DkimAuthResult.new(domain: text(dkim, "domain"), selector: text(dkim, "selector"),
|
|
131
|
+
result: text(dkim, "result"), human_result: text(dkim, "human_result"))
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
def spf_results(element)
|
|
136
|
+
elements(element, "spf").map do |spf|
|
|
137
|
+
SpfAuthResult.new(domain: text(spf, "domain"), scope: text(spf, "scope"), result: text(spf, "result"))
|
|
138
|
+
end
|
|
139
|
+
end
|
|
140
|
+
|
|
141
|
+
def child(element, name)
|
|
142
|
+
element.elements[name] if element
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
def elements(element, name)
|
|
146
|
+
if element
|
|
147
|
+
element.get_elements(name)
|
|
148
|
+
else
|
|
149
|
+
[]
|
|
150
|
+
end
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
def text(element, name)
|
|
154
|
+
text_of(child(element, name))
|
|
155
|
+
end
|
|
156
|
+
|
|
157
|
+
def text_of(element)
|
|
158
|
+
value = element&.text.to_s.strip
|
|
159
|
+
|
|
160
|
+
value unless value.empty?
|
|
161
|
+
end
|
|
162
|
+
|
|
163
|
+
def integer(element, name)
|
|
164
|
+
Integer(text(element, name), exception: false)
|
|
165
|
+
end
|
|
166
|
+
|
|
167
|
+
def counted(element, name)
|
|
168
|
+
value = integer(element, name)
|
|
169
|
+
|
|
170
|
+
value unless value.nil? || value.negative?
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
# A count the sender omitted is nil, not zero: "no messages" is a
|
|
174
|
+
# finding, and only the sender can report one.
|
|
175
|
+
def count_of(row)
|
|
176
|
+
counted(row, "count") || 0 if text(row, "count")
|
|
177
|
+
end
|
|
178
|
+
end
|
|
179
|
+
end
|
|
180
|
+
end
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
module MailReport
|
|
2
|
+
module Dmarc
|
|
3
|
+
# Named as RFC 7489 Appendix C.
|
|
4
|
+
|
|
5
|
+
DkimAuthResult = Data.define(:domain, :selector, :result, :human_result) do
|
|
6
|
+
def initialize(domain: nil, selector: nil, result: nil, human_result: nil)
|
|
7
|
+
super
|
|
8
|
+
end
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
SpfAuthResult = Data.define(:domain, :scope, :result) do
|
|
12
|
+
def initialize(domain: nil, scope: nil, result: nil)
|
|
13
|
+
super
|
|
14
|
+
end
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
AuthResults = Data.define(:dkim, :spf) do
|
|
18
|
+
def initialize(dkim: [], spf: [])
|
|
19
|
+
super
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
Identifiers = Data.define(:header_from, :envelope_from, :envelope_to) do
|
|
24
|
+
def initialize(header_from: nil, envelope_from: nil, envelope_to: nil)
|
|
25
|
+
super
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
# Override reasons from §7.2 (`forwarded`, `mailing_list`, …).
|
|
30
|
+
Reason = Data.define(:type, :comment) do
|
|
31
|
+
def initialize(type: nil, comment: nil)
|
|
32
|
+
super
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# `dkim` / `spf` are DMARC aligned verdicts; check results are in auth_results.
|
|
37
|
+
Record = Data.define(:source_ip, :count, :disposition, :dkim, :spf, :reasons, :identifiers, :auth_results) do
|
|
38
|
+
def initialize(source_ip: nil, count: nil, disposition: nil, dkim: nil, spf: nil,
|
|
39
|
+
reasons: [], identifiers: Identifiers.new, auth_results: AuthResults.new)
|
|
40
|
+
super
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
Policy = Data.define(:domain, :adkim, :aspf, :p, :sp, :pct, :fo) do
|
|
45
|
+
def initialize(domain: nil, adkim: nil, aspf: nil, p: nil, sp: nil, pct: nil, fo: nil)
|
|
46
|
+
super
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
Report = Data.define(:organization, :email, :extra_contact_info, :report_id, :range, :errors, :policy, :records) do
|
|
51
|
+
def initialize(organization: nil, email: nil, extra_contact_info: nil, report_id: nil,
|
|
52
|
+
range: nil, errors: [], policy: Policy.new, records: [])
|
|
53
|
+
super
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
require "mailreport/archive"
|
|
2
|
+
require "mailreport/dmarc/document"
|
|
3
|
+
require "mailreport/dmarc/report"
|
|
4
|
+
|
|
5
|
+
module MailReport
|
|
6
|
+
# RFC 7489 §7.2 aggregate reports.
|
|
7
|
+
module Dmarc
|
|
8
|
+
REPORT_NAME = /\.xml\z/i
|
|
9
|
+
|
|
10
|
+
def self.parse(bytes)
|
|
11
|
+
if xml = Archive.open(bytes, name: REPORT_NAME)
|
|
12
|
+
Document.new(xml).report
|
|
13
|
+
end
|
|
14
|
+
end
|
|
15
|
+
end
|
|
16
|
+
end
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
require "json"
|
|
2
|
+
require "time"
|
|
3
|
+
|
|
4
|
+
require "mailreport/archive"
|
|
5
|
+
|
|
6
|
+
module MailReport
|
|
7
|
+
# RFC 8460.
|
|
8
|
+
module TlsRpt
|
|
9
|
+
REPORT_NAME = /\.json\z/i
|
|
10
|
+
|
|
11
|
+
# §4.4.
|
|
12
|
+
Failure = Data.define(:result_type, :count, :sending_mta_ip, :receiving_mx_hostname,
|
|
13
|
+
:receiving_mx_helo, :receiving_ip, :additional_information, :failure_reason_code) do
|
|
14
|
+
def initialize(result_type: nil, count: nil, sending_mta_ip: nil, receiving_mx_hostname: nil,
|
|
15
|
+
receiving_mx_helo: nil, receiving_ip: nil, additional_information: nil, failure_reason_code: nil)
|
|
16
|
+
super
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
Policy = Data.define(:type, :domain, :mx_hosts, :policy_string, :successes, :failures, :failure_details) do
|
|
21
|
+
def initialize(type: nil, domain: nil, mx_hosts: [], policy_string: [], successes: nil, failures: nil,
|
|
22
|
+
failure_details: [])
|
|
23
|
+
super
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
Report = Data.define(:organization, :report_id, :contact, :started_at, :ended_at, :policies) do
|
|
28
|
+
def initialize(organization: nil, report_id: nil, contact: nil, started_at: nil, ended_at: nil, policies: [])
|
|
29
|
+
super
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def range
|
|
33
|
+
started_at..ended_at if started_at && ended_at && started_at <= ended_at
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
class << self
|
|
38
|
+
def parse(bytes)
|
|
39
|
+
json = Archive.open(bytes, name: REPORT_NAME)&.force_encoding(Encoding::UTF_8)
|
|
40
|
+
|
|
41
|
+
# A report is JSON, and JSON is UTF-8 (RFC 8460 §3, RFC 8259 §8.1).
|
|
42
|
+
document = JSON.parse(json) if json&.valid_encoding?
|
|
43
|
+
|
|
44
|
+
report_from(document) if document.is_a?(Hash)
|
|
45
|
+
rescue JSON::ParserError
|
|
46
|
+
nil
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
private
|
|
50
|
+
def report_from(document)
|
|
51
|
+
window = hash_at(document, "date-range")
|
|
52
|
+
|
|
53
|
+
Report.new(
|
|
54
|
+
organization: document["organization-name"],
|
|
55
|
+
report_id: document["report-id"],
|
|
56
|
+
contact: document["contact-info"],
|
|
57
|
+
started_at: time_at(window["start-datetime"]),
|
|
58
|
+
ended_at: time_at(window["end-datetime"]),
|
|
59
|
+
policies: array_at(document, "policies").filter_map { |entry| policy_from(entry) }
|
|
60
|
+
)
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def policy_from(entry)
|
|
64
|
+
if entry.is_a?(Hash)
|
|
65
|
+
policy, summary = hash_at(entry, "policy"), hash_at(entry, "summary")
|
|
66
|
+
|
|
67
|
+
Policy.new(
|
|
68
|
+
type: policy["policy-type"],
|
|
69
|
+
domain: policy["policy-domain"],
|
|
70
|
+
mx_hosts: listed(policy["mx-host"]),
|
|
71
|
+
policy_string: listed(policy["policy-string"]),
|
|
72
|
+
successes: count_at(summary, "total-successful-session-count"),
|
|
73
|
+
failures: count_at(summary, "total-failure-session-count"),
|
|
74
|
+
failure_details: array_at(entry, "failure-details").filter_map { |failure| failure_from(failure) }
|
|
75
|
+
)
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
def failure_from(entry)
|
|
80
|
+
if entry.is_a?(Hash)
|
|
81
|
+
Failure.new(
|
|
82
|
+
result_type: entry["result-type"],
|
|
83
|
+
count: count_at(entry, "failed-session-count"),
|
|
84
|
+
sending_mta_ip: entry["sending-mta-ip"],
|
|
85
|
+
receiving_mx_hostname: entry["receiving-mx-hostname"],
|
|
86
|
+
receiving_mx_helo: entry["receiving-mx-helo"],
|
|
87
|
+
receiving_ip: entry["receiving-ip"],
|
|
88
|
+
additional_information: entry["additional-information"],
|
|
89
|
+
failure_reason_code: entry["failure-reason-code"]
|
|
90
|
+
)
|
|
91
|
+
end
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
# Scalar or array — reporters send either.
|
|
95
|
+
def listed(value)
|
|
96
|
+
case value
|
|
97
|
+
when Array then value
|
|
98
|
+
when nil then []
|
|
99
|
+
else [ value ]
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
def hash_at(document, key)
|
|
104
|
+
document[key].is_a?(Hash) ? document[key] : {}
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
def array_at(document, key)
|
|
108
|
+
listed(document[key])
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
# A count the sender omitted is nil, not zero: "no sessions" is a
|
|
112
|
+
# finding, and only the sender can report one.
|
|
113
|
+
def count_at(document, key)
|
|
114
|
+
counted(document[key]) if document.key?(key)
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
def counted(given)
|
|
118
|
+
value = given.is_a?(Integer) ? given : Integer(given.to_s, exception: false)
|
|
119
|
+
|
|
120
|
+
if value && !value.negative?
|
|
121
|
+
value
|
|
122
|
+
else
|
|
123
|
+
0
|
|
124
|
+
end
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
def time_at(value)
|
|
128
|
+
Time.iso8601(value.to_s)
|
|
129
|
+
rescue ArgumentError
|
|
130
|
+
nil
|
|
131
|
+
end
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
end
|
data/lib/mailreport.rb
ADDED
data/mailreport.gemspec
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
$LOAD_PATH.push File.expand_path('lib', __dir__)
|
|
2
|
+
require_relative "lib/mailreport/version"
|
|
3
|
+
|
|
4
|
+
Gem::Specification.new do |spec|
|
|
5
|
+
spec.name = "mailreport"
|
|
6
|
+
spec.version = MailReport::VERSION
|
|
7
|
+
spec.platform = Gem::Platform::RUBY
|
|
8
|
+
spec.required_ruby_version = ">= 3.3.4"
|
|
9
|
+
spec.authors = [ "Simon Lev" ]
|
|
10
|
+
spec.email = [ "support@postrider.dev" ]
|
|
11
|
+
|
|
12
|
+
spec.summary = "DMARC aggregate and SMTP TLS reports, parsed in pure Ruby."
|
|
13
|
+
spec.description = "Answers one question about your domain: what are receivers seeing?"
|
|
14
|
+
|
|
15
|
+
spec.homepage = "https://github.com/mailpiece/mailreport"
|
|
16
|
+
spec.license = "MIT"
|
|
17
|
+
|
|
18
|
+
spec.metadata["source_code_uri"] = spec.homepage
|
|
19
|
+
spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/main/CHANGELOG.md"
|
|
20
|
+
spec.metadata["bug_tracker_uri"] = "#{spec.homepage}/issues"
|
|
21
|
+
spec.metadata["rubygems_mfa_required"] = "true"
|
|
22
|
+
|
|
23
|
+
# Dir, not `git ls-files`: an uncommitted checkout would package an empty gem.
|
|
24
|
+
spec.files = Dir[
|
|
25
|
+
"lib/**/*.rb",
|
|
26
|
+
"CHANGELOG.md",
|
|
27
|
+
"README.md",
|
|
28
|
+
"LICENSE",
|
|
29
|
+
"mailreport.gemspec"
|
|
30
|
+
]
|
|
31
|
+
spec.require_paths = [ "lib" ]
|
|
32
|
+
|
|
33
|
+
# CVE-2024-43398 (DoS, fixed 3.3.6); CVE-2024-49761 (ReDoS, fixed 3.3.9;
|
|
34
|
+
# Ruby 3.1-only in practice — we require >= 3.3.4).
|
|
35
|
+
spec.add_dependency "rexml", ">= 3.3.9"
|
|
36
|
+
spec.add_dependency "zip_kit", "~> 6.3"
|
|
37
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: mailreport
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Simon Lev
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: rexml
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: 3.3.9
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - ">="
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: 3.3.9
|
|
26
|
+
- !ruby/object:Gem::Dependency
|
|
27
|
+
name: zip_kit
|
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
|
29
|
+
requirements:
|
|
30
|
+
- - "~>"
|
|
31
|
+
- !ruby/object:Gem::Version
|
|
32
|
+
version: '6.3'
|
|
33
|
+
type: :runtime
|
|
34
|
+
prerelease: false
|
|
35
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
36
|
+
requirements:
|
|
37
|
+
- - "~>"
|
|
38
|
+
- !ruby/object:Gem::Version
|
|
39
|
+
version: '6.3'
|
|
40
|
+
description: 'Answers one question about your domain: what are receivers seeing?'
|
|
41
|
+
email:
|
|
42
|
+
- support@postrider.dev
|
|
43
|
+
executables: []
|
|
44
|
+
extensions: []
|
|
45
|
+
extra_rdoc_files: []
|
|
46
|
+
files:
|
|
47
|
+
- CHANGELOG.md
|
|
48
|
+
- LICENSE
|
|
49
|
+
- README.md
|
|
50
|
+
- lib/mailreport.rb
|
|
51
|
+
- lib/mailreport/archive.rb
|
|
52
|
+
- lib/mailreport/dmarc.rb
|
|
53
|
+
- lib/mailreport/dmarc/document.rb
|
|
54
|
+
- lib/mailreport/dmarc/report.rb
|
|
55
|
+
- lib/mailreport/tls_rpt.rb
|
|
56
|
+
- lib/mailreport/version.rb
|
|
57
|
+
- mailreport.gemspec
|
|
58
|
+
homepage: https://github.com/mailpiece/mailreport
|
|
59
|
+
licenses:
|
|
60
|
+
- MIT
|
|
61
|
+
metadata:
|
|
62
|
+
source_code_uri: https://github.com/mailpiece/mailreport
|
|
63
|
+
changelog_uri: https://github.com/mailpiece/mailreport/blob/main/CHANGELOG.md
|
|
64
|
+
bug_tracker_uri: https://github.com/mailpiece/mailreport/issues
|
|
65
|
+
rubygems_mfa_required: 'true'
|
|
66
|
+
rdoc_options: []
|
|
67
|
+
require_paths:
|
|
68
|
+
- lib
|
|
69
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
70
|
+
requirements:
|
|
71
|
+
- - ">="
|
|
72
|
+
- !ruby/object:Gem::Version
|
|
73
|
+
version: 3.3.4
|
|
74
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
75
|
+
requirements:
|
|
76
|
+
- - ">="
|
|
77
|
+
- !ruby/object:Gem::Version
|
|
78
|
+
version: '0'
|
|
79
|
+
requirements: []
|
|
80
|
+
rubygems_version: 4.0.3
|
|
81
|
+
specification_version: 4
|
|
82
|
+
summary: DMARC aggregate and SMTP TLS reports, parsed in pure Ruby.
|
|
83
|
+
test_files: []
|