dmarcator 1.0.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/ChangeLog.md +15 -0
- data/README.md +22 -15
- data/lib/dmarcator/internal/parser.rb +231 -273
- data/lib/dmarcator/query.rb +3 -2
- data/lib/dmarcator/record.rb +0 -28
- data/lib/dmarcator/resolver/plain.rb +119 -0
- data/lib/dmarcator/resolver.rb +3 -4
- data/lib/dmarcator/version.rb +1 -1
- metadata +2 -21
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 48797112128c5187471c9c2ad1f907c73450cf8ef0f6910e2a22815d74221222
|
|
4
|
+
data.tar.gz: 89727b4555cc67beddbcb53dbbfb4845dc69e6cb8c66e37b8549fa7a4ec2ec36
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b6305c8a45f78711592f6a68b61236241d68a45410de08eaeeaa8e158b6337421e9b5d6c78b21ace4b0b38109fa74f3eec198ba4b5bbe6c2a49e5127ff2e69ff
|
|
7
|
+
data.tar.gz: 20111fdcf9bd38ff20f48a2be10ffdf4aa946850643fca2a48696fa10c041f5b3a4d49d37f6deed32f9767ed758828fce1ade967a8c12fc52cd0c1adc2ede6a2
|
data/ChangeLog.md
CHANGED
|
@@ -1,3 +1,18 @@
|
|
|
1
|
+
### 1.2.0 / 2026-10-30
|
|
2
|
+
|
|
3
|
+
* Plaintext lookups now use DNS over TCP (`DMARCator::Resolver::Plain`) instead of `Resolv::DNS`. `Resolv` reads UDP replies into a 512-byte buffer, so a larger answer cannot be decoded, is discarded, and the resulting timeout is swallowed. Any TXT set over 512 bytes — common for SPF and for domains with several DKIM keys — was therefore returned as an empty array, which is indistinguishable from "no records published". TCP has no such limit, and failures raise `ResolutionError`.
|
|
4
|
+
* `DMARCATOR_DNS_TIMEOUT` now applies to plaintext lookups as well (default 5 seconds).
|
|
5
|
+
* `DMARCator::DNS_TIMEOUTS` is no longer used and is deprecated; it will be removed in 2.0.
|
|
6
|
+
* An injected `Resolv::DNS` still works, but carries the same 512-byte UDP limitation. Use `DMARCator::Resolver::Plain`, DoH, or DoT for SPF and DKIM accuracy.
|
|
7
|
+
|
|
8
|
+
### 1.1.0 / 2026-10-30
|
|
9
|
+
|
|
10
|
+
* Replace the Parslet grammar with a hand-written parser in `DMARCator::Internal`. Parsing behaviour is unchanged, verified differentially against the previous parser over 128 real-world and 47 edge-case records in both lenient and strict modes; record parsing is roughly 30x faster (0.84 ms to 0.03 ms per record).
|
|
11
|
+
* Remove the `parslet` runtime dependency. `nokogiri` is now the only runtime dependency.
|
|
12
|
+
* Messages for structurally malformed records now come from the parser itself (for example `invalid DMARC record: "..."`) and are truncated to 300 characters.
|
|
13
|
+
* `DMARCator::Internal::Parser::Transform` no longer exists. It was internal, so this is not covered by the compatibility promise.
|
|
14
|
+
* Reject records whose string encoding is invalid or not ASCII-compatible with `InvalidRecord`. Previously `Record.parse` could raise `ArgumentError` (invalid UTF-8) or `Encoding::CompatibilityError` (for example UTF-16), which escaped `rescue DMARCator::Error`. Binary strings from a DNS resolver, and other ASCII-compatible encodings such as ISO-8859-1, are unaffected.
|
|
15
|
+
|
|
1
16
|
### 1.0.0 / 2026-10-30
|
|
2
17
|
|
|
3
18
|
The public API is now frozen.
|
data/README.md
CHANGED
|
@@ -12,7 +12,7 @@ DMARCator is a Ruby library for parsing and querying [DMARC] records.
|
|
|
12
12
|
|
|
13
13
|
[DMARC] policies are published as DNS TXT records at `_dmarc.<domain>`. This
|
|
14
14
|
library parses those records into a structured `DMARCator::Record`, queries them
|
|
15
|
-
|
|
15
|
+
over DNS, and parses the XML aggregate
|
|
16
16
|
reports those policies generate.
|
|
17
17
|
|
|
18
18
|
## Installation
|
|
@@ -173,9 +173,9 @@ end
|
|
|
173
173
|
```
|
|
174
174
|
|
|
175
175
|
Notes for large scans: `Record.lookup` adds a parse per domain (~0.4 ms, far
|
|
176
|
-
more than the lookup on a fast resolver);
|
|
177
|
-
so
|
|
178
|
-
|
|
176
|
+
more than the lookup on a fast resolver); each lookup opens its own TCP
|
|
177
|
+
connection, so keep one resolver per thread when parallelising; and DoT/DoH add
|
|
178
|
+
a TLS handshake per lookup, so plaintext is the better choice for bulk work.
|
|
179
179
|
|
|
180
180
|
### SPF and DKIM
|
|
181
181
|
|
|
@@ -205,9 +205,8 @@ DMARCator.configure do |config|
|
|
|
205
205
|
end
|
|
206
206
|
```
|
|
207
207
|
|
|
208
|
-
A resolver object is shared by every thread,
|
|
209
|
-
|
|
210
|
-
factory instead:
|
|
208
|
+
A resolver object is shared by every thread, so threaded applications should
|
|
209
|
+
configure a factory instead:
|
|
211
210
|
|
|
212
211
|
```ruby
|
|
213
212
|
DMARCator.configure do |config|
|
|
@@ -251,8 +250,9 @@ report.records.first.source_ip
|
|
|
251
250
|
|
|
252
251
|
## DNS transport
|
|
253
252
|
|
|
254
|
-
By default,
|
|
255
|
-
can be selected with environment variables; no DNS provider
|
|
253
|
+
By default, lookups use plain DNS over **TCP** (`DMARCator::Resolver::Plain`).
|
|
254
|
+
Encrypted transports can be selected with environment variables; no DNS provider
|
|
255
|
+
is baked in:
|
|
256
256
|
|
|
257
257
|
```sh
|
|
258
258
|
# DNS over HTTPS (RFC 8484)
|
|
@@ -271,7 +271,15 @@ export DMARCATOR_DNS_TIMEOUT=5
|
|
|
271
271
|
|
|
272
272
|
Certificates are always verified. `DMARCATOR_DOH_IP` pins the address so the
|
|
273
273
|
DoH endpoint does not have to be resolved first; TLS still verifies the URL's
|
|
274
|
-
hostname.
|
|
274
|
+
hostname. `DMARCATOR_DNS_TIMEOUT` applies to every transport, including
|
|
275
|
+
plaintext (default 5 seconds).
|
|
276
|
+
|
|
277
|
+
Plaintext uses TCP rather than UDP deliberately. `Resolv::DNS` reads UDP replies
|
|
278
|
+
into a 512-byte buffer, so a larger answer cannot be decoded, is discarded, and
|
|
279
|
+
the resulting timeout is swallowed — meaning any TXT set over 512 bytes (common
|
|
280
|
+
for SPF, and for domains with several DKIM keys) came back as an empty result,
|
|
281
|
+
indistinguishable from "no records published". TCP has no such limit, and
|
|
282
|
+
failures raise `DMARCator::ResolutionError`.
|
|
275
283
|
|
|
276
284
|
Encrypted transports fail closed: a transport failure raises
|
|
277
285
|
`DMARCator::ResolutionError` instead of being reported as a domain without a
|
|
@@ -300,9 +308,10 @@ XML helper, the internal resolver plumbing — may change in a minor release.
|
|
|
300
308
|
* `DMARCator::Record`, `DMARCator::Report` and the parsed value objects are
|
|
301
309
|
read-only after construction and safe to share between threads.
|
|
302
310
|
* The memoized parser and transform are shared, and safe for concurrent use.
|
|
303
|
-
* Resolvers are not necessarily safe for concurrent lookups:
|
|
304
|
-
|
|
305
|
-
|
|
311
|
+
* Resolvers are not necessarily safe for concurrent lookups: each lookup opens
|
|
312
|
+
its own connection, but `Resolv::DNS` (if you inject one) does not serialise
|
|
313
|
+
queries. Use one resolver per thread, or configure a factory with
|
|
314
|
+
`DMARCator.configure` so each lookup builds its own.
|
|
306
315
|
* `DMARCator.configure` mutates process-wide state. Call it during boot rather
|
|
307
316
|
than per request.
|
|
308
317
|
|
|
@@ -348,7 +357,6 @@ for the reporting policy and scope.
|
|
|
348
357
|
## Requirements
|
|
349
358
|
|
|
350
359
|
* [Ruby] >= 3.3.13
|
|
351
|
-
* [parslet] >= 1.0.0, < 3.0.0
|
|
352
360
|
* [nokogiri] >= 1.10
|
|
353
361
|
|
|
354
362
|
## Testing
|
|
@@ -417,6 +425,5 @@ created by D Salko and released under the MIT License.
|
|
|
417
425
|
[dmarc-parser-gem]: https://rubygems.org/gems/dmarc_parser
|
|
418
426
|
[Trail of Bits]: https://www.trailofbits.com
|
|
419
427
|
[Ruby]: https://www.ruby-lang.org
|
|
420
|
-
[parslet]: https://kschiess.github.io/parslet/
|
|
421
428
|
[nokogiri]: https://nokogiri.org
|
|
422
429
|
[LICENSE.txt]: LICENSE.txt
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require 'strscan'
|
|
4
|
+
require 'uri'
|
|
5
|
+
|
|
3
6
|
require_relative '../exceptions'
|
|
4
7
|
require_relative '../uri'
|
|
5
8
|
|
|
6
|
-
require 'parslet'
|
|
7
|
-
require 'uri'
|
|
8
|
-
|
|
9
9
|
module DMARCator
|
|
10
10
|
#
|
|
11
11
|
# Implementation details of the library. Nothing here is covered by the
|
|
@@ -15,342 +15,300 @@ module DMARCator
|
|
|
15
15
|
#
|
|
16
16
|
module Internal
|
|
17
17
|
#
|
|
18
|
-
# DMARC
|
|
18
|
+
# Parses DMARC records (RFC 7489 section 6.4).
|
|
19
19
|
#
|
|
20
|
-
#
|
|
20
|
+
# A hand-written parser rather than a parser-combinator grammar: the
|
|
21
|
+
# language is small, and this keeps record parsing allocation-light and
|
|
22
|
+
# free of a runtime dependency.
|
|
21
23
|
#
|
|
22
|
-
# @
|
|
24
|
+
# @see https://www.rfc-editor.org/rfc/rfc7489#section-6.4
|
|
23
25
|
#
|
|
24
|
-
# @
|
|
26
|
+
# @api private
|
|
25
27
|
#
|
|
26
|
-
class Parser
|
|
27
|
-
|
|
28
|
+
class Parser
|
|
28
29
|
# Maximum accepted DMARC record size in bytes. Real records are far
|
|
29
30
|
# smaller; the cap bounds parsing work on hostile input.
|
|
30
|
-
#
|
|
31
31
|
MAX_RECORD_BYTES = 4096
|
|
32
32
|
|
|
33
|
-
|
|
33
|
+
# Maximum length of error messages echoed from untrusted input.
|
|
34
|
+
MAX_ERROR_MESSAGE_LENGTH = 300
|
|
34
35
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
36
|
+
# Tags whose value is one keyword from a fixed set.
|
|
37
|
+
KEYWORD_TAGS = {
|
|
38
|
+
'p' => %w[none quarantine reject],
|
|
39
|
+
'sp' => %w[none quarantine reject],
|
|
40
|
+
'adkim' => %w[r s],
|
|
41
|
+
'aspf' => %w[r s],
|
|
42
|
+
'rf' => %w[afrf iodef]
|
|
43
|
+
}.freeze
|
|
41
44
|
|
|
42
|
-
|
|
45
|
+
# Tags whose value is a decimal number of at most N digits.
|
|
46
|
+
DIGIT_TAGS = { 'ri' => 10, 'pct' => 3 }.freeze
|
|
43
47
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
str('=') >> wsp? >>
|
|
47
|
-
str('DMARC1').as(:v)
|
|
48
|
-
end
|
|
48
|
+
# Tags whose value is a list of mailto URIs.
|
|
49
|
+
URI_TAGS = %w[rua ruf].freeze
|
|
49
50
|
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
dmarc_srequest |
|
|
53
|
-
dmarc_auri |
|
|
54
|
-
dmarc_furi |
|
|
55
|
-
dmarc_adkim |
|
|
56
|
-
dmarc_aspf |
|
|
57
|
-
dmarc_ainterval |
|
|
58
|
-
dmarc_fo |
|
|
59
|
-
dmarc_rfmt |
|
|
60
|
-
dmarc_percent |
|
|
61
|
-
unknown_tag
|
|
62
|
-
end
|
|
51
|
+
# The `v=` tag, which may only appear first.
|
|
52
|
+
VERSION = /\Av[ \t]*=[ \t]*DMARC1\z/
|
|
63
53
|
|
|
64
|
-
#
|
|
65
|
-
|
|
66
|
-
# match the rule as `:invalid_tag` so strict mode can reject them.
|
|
67
|
-
#
|
|
68
|
-
# @param [Symbol] name
|
|
69
|
-
# The rule name, used as `dmarc_<name>`.
|
|
70
|
-
#
|
|
71
|
-
# @param [String] tag
|
|
72
|
-
# The DMARC tag name.
|
|
73
|
-
#
|
|
74
|
-
# @yieldreturn [Parslet::Atoms::Base]
|
|
75
|
-
# The parslet matching the tag's value.
|
|
76
|
-
#
|
|
77
|
-
# @api private
|
|
78
|
-
#
|
|
79
|
-
def self.tag_rule(name,tag,&block)
|
|
80
|
-
rule(:"dmarc_#{name}") do
|
|
81
|
-
(
|
|
82
|
-
str(tag) >> wsp? >> str('=') >> wsp? >>
|
|
83
|
-
instance_eval(&block).as(tag.to_sym)
|
|
84
|
-
) |
|
|
85
|
-
(
|
|
86
|
-
str(tag) >> wsp? >> str('=') >> wsp? >>
|
|
87
|
-
match["^; \t"].repeat(1)
|
|
88
|
-
).as(:invalid_tag)
|
|
89
|
-
end
|
|
90
|
-
end
|
|
54
|
+
# Characters allowed in a mailto local part and domain.
|
|
55
|
+
ADDRESS = /[A-Za-z0-9._%+-]+@[A-Za-z0-9._%+-]+/
|
|
91
56
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
end
|
|
57
|
+
# Characters allowed between tags.
|
|
58
|
+
WSP = /[ \t]*/
|
|
95
59
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
60
|
+
class << self
|
|
61
|
+
#
|
|
62
|
+
# Parses a DMARC record.
|
|
63
|
+
#
|
|
64
|
+
# @param [String] record
|
|
65
|
+
# The raw DMARC record to parse.
|
|
66
|
+
#
|
|
67
|
+
# @param [Boolean] strict (false)
|
|
68
|
+
# When `true`, raise on unknown tags or invalid tag values instead
|
|
69
|
+
# of silently ignoring them.
|
|
70
|
+
#
|
|
71
|
+
# @return [Hash{Symbol => Object}]
|
|
72
|
+
#
|
|
73
|
+
# @raise [InvalidRecord]
|
|
74
|
+
# The record is malformed, is too large, or (in strict mode)
|
|
75
|
+
# contains an unknown tag or invalid tag value.
|
|
76
|
+
#
|
|
77
|
+
def parse(record, strict: false)
|
|
78
|
+
unless record.is_a?(String)
|
|
79
|
+
raise InvalidRecord, "DMARC record must be a String, got #{record.class}"
|
|
80
|
+
end
|
|
99
81
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
82
|
+
if record.bytesize > MAX_RECORD_BYTES
|
|
83
|
+
raise InvalidRecord, "DMARC record exceeds maximum size of #{MAX_RECORD_BYTES} bytes"
|
|
84
|
+
end
|
|
103
85
|
|
|
104
|
-
|
|
86
|
+
# Malformed or non-ASCII-compatible encodings make the regular
|
|
87
|
+
# expressions below raise ArgumentError or Encoding::CompatibilityError
|
|
88
|
+
# instead of InvalidRecord, which would escape `rescue DMARCator::Error`.
|
|
89
|
+
unless record.encoding.ascii_compatible?
|
|
90
|
+
raise InvalidRecord,
|
|
91
|
+
"DMARC record must be ASCII-compatible text, got #{record.encoding}"
|
|
92
|
+
end
|
|
105
93
|
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
94
|
+
unless record.valid_encoding?
|
|
95
|
+
raise InvalidRecord, "DMARC record is not valid #{record.encoding} text"
|
|
96
|
+
end
|
|
109
97
|
|
|
110
|
-
|
|
111
|
-
fo_opt >> (wsp? >> str(':') >> wsp? >> fo_opt).repeat
|
|
112
|
-
end
|
|
98
|
+
tags = { v: :DMARC1 }
|
|
113
99
|
|
|
114
|
-
|
|
100
|
+
split(record).each do |segment|
|
|
101
|
+
status, key, value = parse_segment(segment)
|
|
115
102
|
|
|
116
|
-
|
|
103
|
+
case status
|
|
104
|
+
when :ok
|
|
105
|
+
raise InvalidRecord, "duplicate DMARC tag: #{key}" if tags.key?(key)
|
|
117
106
|
|
|
118
|
-
|
|
107
|
+
tags[key] = value
|
|
108
|
+
when :unknown
|
|
109
|
+
if strict
|
|
110
|
+
raise InvalidRecord, "unknown DMARC tag: #{truncate(segment).inspect}"
|
|
111
|
+
end
|
|
112
|
+
when :invalid
|
|
113
|
+
if strict
|
|
114
|
+
raise InvalidRecord, "invalid DMARC tag value: #{truncate(segment).inspect}"
|
|
115
|
+
end
|
|
116
|
+
end
|
|
117
|
+
end
|
|
119
118
|
|
|
120
|
-
|
|
121
|
-
|
|
119
|
+
validate!(tags)
|
|
120
|
+
end
|
|
122
121
|
|
|
123
|
-
|
|
122
|
+
private
|
|
124
123
|
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
124
|
+
#
|
|
125
|
+
# Splits a record into tag segments, validating the version tag.
|
|
126
|
+
#
|
|
127
|
+
def split(record)
|
|
128
|
+
separator = record.index(';')
|
|
129
|
+
version = (separator ? record[0, separator] : record).sub(/[ \t]+\z/, '')
|
|
130
|
+
rest = separator ? record[(separator + 1)..] : ''
|
|
132
131
|
|
|
133
|
-
|
|
132
|
+
unless version.match?(VERSION)
|
|
133
|
+
raise InvalidRecord, "invalid DMARC record: #{truncate(record).inspect}"
|
|
134
|
+
end
|
|
134
135
|
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
str(':') >> mailto_address
|
|
138
|
-
end
|
|
136
|
+
# At most one trailing separator, plus optional surrounding space.
|
|
137
|
+
rest = rest.sub(/[ \t]+\z/, '').sub(/;[ \t]*\z/, '').sub(/[ \t]+\z/, '')
|
|
139
138
|
|
|
140
|
-
|
|
141
|
-
mailto_local >> str('@') >> mailto_domain
|
|
142
|
-
end
|
|
139
|
+
return [] if rest.empty?
|
|
143
140
|
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
rule(:mailto_char) { match["A-Za-z0-9._%+-"] }
|
|
147
|
-
|
|
148
|
-
rule(:digit) { match('[0-9]') }
|
|
149
|
-
rule(:wsp) { str(' ') | str("\t") }
|
|
150
|
-
rule(:wsp?) { wsp.repeat }
|
|
151
|
-
|
|
152
|
-
#
|
|
153
|
-
# Converts the raw parse tree into tag attributes.
|
|
154
|
-
#
|
|
155
|
-
# @api private
|
|
156
|
-
#
|
|
157
|
-
class Transform < Parslet::Transform
|
|
158
|
-
|
|
159
|
-
rule(fo_opt: simple(:fo_opt)) { fo_opt }
|
|
160
|
-
rule(fo: simple(:fo_opt)) { {fo: [fo_opt]} }
|
|
161
|
-
|
|
162
|
-
rule(v: simple(:version)) { {v: version.to_sym} }
|
|
163
|
-
rule(p: simple(:p)) { {p: p.to_sym } }
|
|
164
|
-
rule(sp: simple(:sp)) { {sp: sp.to_sym} }
|
|
165
|
-
rule(rf: simple(:rf)) { {rf: rf.to_sym} }
|
|
166
|
-
rule(adkim: simple(:adkim)) { {adkim: adkim.to_sym} }
|
|
167
|
-
rule(aspf: simple(:aspf)) { {aspf: aspf.to_sym} }
|
|
168
|
-
|
|
169
|
-
rule(pct: simple(:pct)) { {pct: pct.to_i} }
|
|
170
|
-
rule(ri: simple(:ri)) { {ri: ri.to_i} }
|
|
171
|
-
|
|
172
|
-
rule(uri: simple(:uri), size: simple(:size), unit: simple(:unit)) do
|
|
173
|
-
Uri.new(
|
|
174
|
-
Transform.parse_uri(uri),
|
|
175
|
-
size.to_i,
|
|
176
|
-
unit.to_sym
|
|
177
|
-
)
|
|
178
|
-
end
|
|
179
|
-
rule(uri: simple(:uri), size: simple(:size)) do
|
|
180
|
-
Uri.new(
|
|
181
|
-
Transform.parse_uri(uri),
|
|
182
|
-
size.to_i
|
|
183
|
-
)
|
|
184
|
-
end
|
|
185
|
-
rule(uri: simple(:uri)) { Uri.new(Transform.parse_uri(uri)) }
|
|
186
|
-
rule(rua: subtree(:uris)) { {rua: Array(uris)} }
|
|
187
|
-
rule(ruf: subtree(:uris)) { {ruf: Array(uris)} }
|
|
141
|
+
rest.split(';', -1).map do |segment|
|
|
142
|
+
segment = segment.sub(/\A[ \t]+/, '').sub(/[ \t]+\z/, '')
|
|
188
143
|
|
|
189
|
-
|
|
190
|
-
|
|
144
|
+
if segment.empty?
|
|
145
|
+
raise InvalidRecord, "invalid DMARC record: #{truncate(record).inspect}"
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
segment
|
|
149
|
+
end
|
|
150
|
+
end
|
|
191
151
|
|
|
192
152
|
#
|
|
193
|
-
#
|
|
194
|
-
#
|
|
195
|
-
# @param [Parslet::Slice] value
|
|
196
|
-
#
|
|
197
|
-
# @return [URI::MailTo]
|
|
153
|
+
# Classifies and parses one `tag=value` segment.
|
|
198
154
|
#
|
|
199
|
-
# @
|
|
200
|
-
# The URI is not a valid `mailto:` URI.
|
|
155
|
+
# @return [Array(Symbol, Symbol, Object), Array(Symbol)]
|
|
201
156
|
#
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
def self.parse_uri(value)
|
|
205
|
-
uri = value.to_s
|
|
157
|
+
def parse_segment(segment)
|
|
158
|
+
name, value = segment.split('=', 2)
|
|
206
159
|
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
end
|
|
160
|
+
return [:unknown] if value.nil?
|
|
161
|
+
|
|
162
|
+
name = name.sub(/[ \t]+\z/, '').sub(/\A[ \t]+/, '')
|
|
163
|
+
value = value.sub(/\A[ \t]+/, '').sub(/[ \t]+\z/, '')
|
|
212
164
|
|
|
213
|
-
|
|
214
|
-
|
|
165
|
+
# A known tag with an empty value is not a tag at all.
|
|
166
|
+
return [:unknown] if value.empty?
|
|
167
|
+
|
|
168
|
+
if (keywords = KEYWORD_TAGS[name])
|
|
169
|
+
keyword(name, keywords, value)
|
|
170
|
+
elsif (max = DIGIT_TAGS[name])
|
|
171
|
+
digits(name, max, value)
|
|
172
|
+
elsif name == 'fo'
|
|
173
|
+
fo(value)
|
|
174
|
+
elsif URI_TAGS.include?(name)
|
|
175
|
+
uri_list(name, value, segment)
|
|
176
|
+
else
|
|
177
|
+
[:unknown]
|
|
215
178
|
end
|
|
179
|
+
end
|
|
216
180
|
|
|
217
|
-
|
|
181
|
+
def keyword(name, keywords, value)
|
|
182
|
+
return [:ok, name.to_sym, value.to_sym] if keywords.include?(value)
|
|
183
|
+
|
|
184
|
+
# A known keyword followed by junk is a malformed record, not an
|
|
185
|
+
# invalid value: the value grammar matched and the remainder did not.
|
|
186
|
+
malformed!(value) if keywords.any? { |keyword| value.start_with?(keyword) }
|
|
187
|
+
|
|
188
|
+
[:invalid]
|
|
218
189
|
end
|
|
219
190
|
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
# @param [String] message
|
|
224
|
-
#
|
|
225
|
-
# @return [String]
|
|
226
|
-
#
|
|
227
|
-
# @api private
|
|
228
|
-
#
|
|
229
|
-
def self.truncate(message)
|
|
230
|
-
message = message.to_s
|
|
191
|
+
def digits(name, max, value)
|
|
192
|
+
if value.match?(/\A\d+\z/)
|
|
193
|
+
return [:invalid] if value.length > max
|
|
231
194
|
|
|
232
|
-
|
|
233
|
-
message[0, MAX_ERROR_MESSAGE_LENGTH] + "..."
|
|
234
|
-
else
|
|
235
|
-
message
|
|
195
|
+
return [:ok, name.to_sym, value.to_i]
|
|
236
196
|
end
|
|
237
|
-
end
|
|
238
197
|
|
|
239
|
-
|
|
198
|
+
malformed!(value) if value.match?(/\A\d/)
|
|
240
199
|
|
|
241
|
-
|
|
242
|
-
# Parses a DMARC record.
|
|
243
|
-
#
|
|
244
|
-
# @param [String] record
|
|
245
|
-
# The raw DMARC record to parse.
|
|
246
|
-
#
|
|
247
|
-
# @param [Boolean] strict (false)
|
|
248
|
-
# When `true`, raise on unknown tags or invalid tag values instead of
|
|
249
|
-
# silently ignoring them.
|
|
250
|
-
#
|
|
251
|
-
# @return [Hash{Symbol => Object}]
|
|
252
|
-
# The Hash of tags within the record.
|
|
253
|
-
#
|
|
254
|
-
# @raise [InvalidRecord]
|
|
255
|
-
# The record is not a String, is malformed, is too large, or (in strict
|
|
256
|
-
# mode) contains an unknown tag or invalid tag value.
|
|
257
|
-
#
|
|
258
|
-
def self.parse(record, strict: false)
|
|
259
|
-
unless record.is_a?(String)
|
|
260
|
-
raise InvalidRecord, "DMARC record must be a String, got #{record.class}"
|
|
200
|
+
[:invalid]
|
|
261
201
|
end
|
|
262
202
|
|
|
263
|
-
|
|
264
|
-
|
|
203
|
+
def fo(value)
|
|
204
|
+
return [:ok, :fo, value.split(':')] if value.match?(/\A[01ds](?::[01ds])*\z/)
|
|
205
|
+
|
|
206
|
+
malformed!(value) if value.match?(/\A[01ds](?::[01ds])*/)
|
|
207
|
+
|
|
208
|
+
[:invalid]
|
|
265
209
|
end
|
|
266
210
|
|
|
267
|
-
|
|
211
|
+
def uri_list(name, value, _segment)
|
|
212
|
+
uris = []
|
|
213
|
+
scanner = StringScanner.new(value)
|
|
268
214
|
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
next unless ignored?(node)
|
|
215
|
+
loop do
|
|
216
|
+
uri = parse_uri(scanner)
|
|
272
217
|
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
218
|
+
# Any failure inside a URI makes the whole tag invalid.
|
|
219
|
+
return [:invalid] if uri.nil?
|
|
220
|
+
|
|
221
|
+
uris << uri
|
|
222
|
+
|
|
223
|
+
scanner.skip(WSP)
|
|
224
|
+
|
|
225
|
+
break if scanner.eos?
|
|
226
|
+
|
|
227
|
+
# A valid URI followed by junk is a malformed record.
|
|
228
|
+
malformed!(value) unless scanner.scan(/,/)
|
|
229
|
+
|
|
230
|
+
scanner.skip(WSP)
|
|
231
|
+
|
|
232
|
+
# A trailing comma leaves no URI to parse.
|
|
233
|
+
return [:invalid] if scanner.eos?
|
|
278
234
|
end
|
|
235
|
+
|
|
236
|
+
[:ok, name.to_sym, uris]
|
|
279
237
|
end
|
|
280
238
|
|
|
281
|
-
|
|
282
|
-
|
|
239
|
+
#
|
|
240
|
+
# Parses one `mailto:` URI, with its optional size and unit.
|
|
241
|
+
#
|
|
242
|
+
# @return [Uri, nil]
|
|
243
|
+
# The URI, or `nil` when the value is not a valid DMARC URI.
|
|
244
|
+
#
|
|
245
|
+
def parse_uri(scanner)
|
|
246
|
+
start = scanner.pos
|
|
283
247
|
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
248
|
+
return nil unless scanner.scan(/mailto:/i) && scanner.scan(ADDRESS)
|
|
249
|
+
|
|
250
|
+
uri_text = scanner.string.byteslice(start, scanner.pos - start)
|
|
251
|
+
size = unit = nil
|
|
252
|
+
|
|
253
|
+
if scanner.scan(/!/)
|
|
254
|
+
digits = scanner.scan(/\d{1,10}/)
|
|
255
|
+
|
|
256
|
+
# A bare `!` or more than ten digits is an invalid value.
|
|
257
|
+
return nil if digits.nil? || scanner.match?(/\d/)
|
|
258
|
+
|
|
259
|
+
size = digits.to_i
|
|
260
|
+
unit = scanner.scan(/[kmgt]/)&.to_sym
|
|
261
|
+
|
|
262
|
+
# A second `!` is an invalid value.
|
|
263
|
+
return nil if scanner.match?(/!/)
|
|
289
264
|
end
|
|
290
265
|
|
|
291
|
-
|
|
292
|
-
|
|
266
|
+
parsed = begin
|
|
267
|
+
URI.parse(uri_text)
|
|
268
|
+
rescue URI::Error
|
|
269
|
+
return nil
|
|
270
|
+
end
|
|
293
271
|
|
|
294
|
-
|
|
272
|
+
return nil unless parsed.is_a?(URI::MailTo)
|
|
295
273
|
|
|
296
|
-
|
|
297
|
-
|
|
274
|
+
Uri.new(parsed, size, unit)
|
|
275
|
+
end
|
|
298
276
|
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
#
|
|
306
|
-
# @return [Boolean]
|
|
307
|
-
#
|
|
308
|
-
def self.ignored?(node)
|
|
309
|
-
node.is_a?(Hash) && (node.key?(:unknown_tag) || node.key?(:invalid_tag))
|
|
310
|
-
end
|
|
277
|
+
#
|
|
278
|
+
# Raises for input whose valid prefix was followed by junk.
|
|
279
|
+
#
|
|
280
|
+
def malformed!(value)
|
|
281
|
+
raise InvalidRecord, "invalid DMARC record: #{truncate(value).inspect}"
|
|
282
|
+
end
|
|
311
283
|
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
#
|
|
320
|
-
def self.parser
|
|
321
|
-
@parser ||= new
|
|
322
|
-
end
|
|
284
|
+
#
|
|
285
|
+
# Validates parsed tag values that have a range.
|
|
286
|
+
#
|
|
287
|
+
def validate!(tags)
|
|
288
|
+
if tags[:pct] && !tags[:pct].between?(0, 100)
|
|
289
|
+
raise InvalidRecord, 'pct must be an integer between 0 and 100'
|
|
290
|
+
end
|
|
323
291
|
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
# @return [Transform]
|
|
328
|
-
#
|
|
329
|
-
# @api private
|
|
330
|
-
#
|
|
331
|
-
def self.transform
|
|
332
|
-
@transform ||= Transform.new
|
|
333
|
-
end
|
|
292
|
+
if tags[:ri] && !tags[:ri].between?(0, 2**32 - 1)
|
|
293
|
+
raise InvalidRecord, 'ri must be an integer between 0 and 4294967295'
|
|
294
|
+
end
|
|
334
295
|
|
|
335
|
-
|
|
336
|
-
# Validates the parsed tags.
|
|
337
|
-
#
|
|
338
|
-
# @param [Hash{Symbol => Object}] tags
|
|
339
|
-
# The parsed tags.
|
|
340
|
-
#
|
|
341
|
-
# @raise [InvalidRecord]
|
|
342
|
-
# One of the tags has an invalid value.
|
|
343
|
-
#
|
|
344
|
-
def self.validate!(tags)
|
|
345
|
-
if tags[:pct] && !(tags[:pct].is_a?(Integer) && tags[:pct].between?(0, 100))
|
|
346
|
-
raise InvalidRecord, "pct must be an integer between 0 and 100"
|
|
296
|
+
tags
|
|
347
297
|
end
|
|
348
298
|
|
|
349
|
-
|
|
350
|
-
|
|
299
|
+
#
|
|
300
|
+
# Truncates an error message to a bounded length.
|
|
301
|
+
#
|
|
302
|
+
def truncate(message)
|
|
303
|
+
message = message.to_s
|
|
304
|
+
|
|
305
|
+
if message.length > MAX_ERROR_MESSAGE_LENGTH
|
|
306
|
+
message[0, MAX_ERROR_MESSAGE_LENGTH] + '...'
|
|
307
|
+
else
|
|
308
|
+
message
|
|
309
|
+
end
|
|
351
310
|
end
|
|
352
311
|
end
|
|
353
|
-
|
|
354
312
|
end
|
|
355
313
|
end
|
|
356
314
|
end
|
data/lib/dmarcator/query.rb
CHANGED
|
@@ -18,8 +18,9 @@ module DMARCator
|
|
|
18
18
|
#
|
|
19
19
|
# DNS query timeouts in seconds, tried in order.
|
|
20
20
|
#
|
|
21
|
-
#
|
|
22
|
-
#
|
|
21
|
+
# @deprecated No longer used. Plaintext lookups use a single timeout
|
|
22
|
+
# ({Resolver::Plain}), configurable with `DMARCATOR_DNS_TIMEOUT`.
|
|
23
|
+
# Removed in 2.0.
|
|
23
24
|
#
|
|
24
25
|
DNS_TIMEOUTS = [2, 4, 8].freeze
|
|
25
26
|
|
data/lib/dmarcator/record.rb
CHANGED
|
@@ -284,36 +284,8 @@ module DMARCator
|
|
|
284
284
|
end
|
|
285
285
|
|
|
286
286
|
new(attributes)
|
|
287
|
-
rescue Parslet::ParseFailed => error
|
|
288
|
-
# Raising inside the rescue sets this error as the cause automatically.
|
|
289
|
-
raise InvalidRecord, truncate_error_message(error.message)
|
|
290
287
|
end
|
|
291
288
|
|
|
292
|
-
# Maximum length of error messages echoed from parsing untrusted input.
|
|
293
|
-
MAX_ERROR_MESSAGE_LENGTH = 500
|
|
294
|
-
|
|
295
|
-
#
|
|
296
|
-
# Truncates error messages to avoid echoing large amounts of untrusted
|
|
297
|
-
# input in exceptions.
|
|
298
|
-
#
|
|
299
|
-
# @param [String] message
|
|
300
|
-
# The error message to truncate.
|
|
301
|
-
#
|
|
302
|
-
# @return [String]
|
|
303
|
-
# The truncated error message.
|
|
304
|
-
#
|
|
305
|
-
def self.truncate_error_message(message)
|
|
306
|
-
message = message.to_s
|
|
307
|
-
|
|
308
|
-
if message.length > MAX_ERROR_MESSAGE_LENGTH
|
|
309
|
-
message[0, MAX_ERROR_MESSAGE_LENGTH] + "..."
|
|
310
|
-
else
|
|
311
|
-
message
|
|
312
|
-
end
|
|
313
|
-
end
|
|
314
|
-
|
|
315
|
-
private_class_method :truncate_error_message
|
|
316
|
-
|
|
317
289
|
#
|
|
318
290
|
# Looks up and parses the DMARC record for a domain.
|
|
319
291
|
#
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'resolv'
|
|
4
|
+
require 'socket'
|
|
5
|
+
|
|
6
|
+
require_relative '../exceptions'
|
|
7
|
+
require_relative 'encrypted'
|
|
8
|
+
|
|
9
|
+
module DMARCator
|
|
10
|
+
module Resolver
|
|
11
|
+
#
|
|
12
|
+
# Plain DNS over TCP ([RFC 1035] section 4.2.2).
|
|
13
|
+
#
|
|
14
|
+
# TCP rather than UDP on purpose. `Resolv::DNS` reads UDP replies into a
|
|
15
|
+
# 512-byte buffer, so a larger answer cannot be decoded, is discarded, and
|
|
16
|
+
# the resulting timeout is swallowed unless `raise_timeout_errors` is set.
|
|
17
|
+
# Large TXT sets - SPF records, several DKIM keys - therefore came back as
|
|
18
|
+
# an empty result, which looks exactly like "no records published". TCP has
|
|
19
|
+
# no such limit, and failures here raise {ResolutionError}.
|
|
20
|
+
#
|
|
21
|
+
# [RFC 1035]: https://www.rfc-editor.org/rfc/rfc1035#section-4.2.2
|
|
22
|
+
#
|
|
23
|
+
# @since 1.2.0
|
|
24
|
+
#
|
|
25
|
+
class Plain < Encrypted
|
|
26
|
+
# Default DNS port.
|
|
27
|
+
DEFAULT_PORT = 53
|
|
28
|
+
|
|
29
|
+
# Default connect/read timeout in seconds.
|
|
30
|
+
DEFAULT_TIMEOUT = 5
|
|
31
|
+
|
|
32
|
+
# @return [Array<Array(String, Integer)>] the nameservers to try.
|
|
33
|
+
attr_reader :nameservers
|
|
34
|
+
|
|
35
|
+
# @return [Integer] connect/read timeout in seconds.
|
|
36
|
+
attr_reader :timeout
|
|
37
|
+
|
|
38
|
+
#
|
|
39
|
+
# Builds a resolver from environment variables.
|
|
40
|
+
#
|
|
41
|
+
# @param [Hash] env
|
|
42
|
+
#
|
|
43
|
+
# @return [Plain]
|
|
44
|
+
#
|
|
45
|
+
# @raise [ArgumentError]
|
|
46
|
+
# `DMARCATOR_DNS_TIMEOUT` is not an integer.
|
|
47
|
+
#
|
|
48
|
+
def self.from_env(env = ENV)
|
|
49
|
+
new(timeout: timeout_from(env))
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def self.timeout_from(env)
|
|
53
|
+
value = env['DMARCATOR_DNS_TIMEOUT']
|
|
54
|
+
|
|
55
|
+
value.nil? || value.empty? ? DEFAULT_TIMEOUT : Integer(value, 10)
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
#
|
|
59
|
+
# The nameservers from the system resolver configuration.
|
|
60
|
+
#
|
|
61
|
+
# @return [Array<Array(String, Integer)>]
|
|
62
|
+
#
|
|
63
|
+
def self.system_nameservers
|
|
64
|
+
config = Resolv::DNS::Config.default_config_hash
|
|
65
|
+
|
|
66
|
+
if (ports = config[:nameserver_port]) && !ports.empty?
|
|
67
|
+
return ports.map { |host, port| [host, port || DEFAULT_PORT] }
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
servers = Array(config[:nameserver])
|
|
71
|
+
servers = ['127.0.0.1'] if servers.empty?
|
|
72
|
+
|
|
73
|
+
servers.map { |host| [host, DEFAULT_PORT] }
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
def initialize(nameservers: nil, timeout: DEFAULT_TIMEOUT)
|
|
77
|
+
@nameservers = (nameservers || self.class.system_nameservers).freeze
|
|
78
|
+
@timeout = timeout
|
|
79
|
+
|
|
80
|
+
raise ArgumentError, 'no nameservers configured' if @nameservers.empty?
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
private
|
|
84
|
+
|
|
85
|
+
def exchange(query)
|
|
86
|
+
wire = query.encode
|
|
87
|
+
last_error = nil
|
|
88
|
+
|
|
89
|
+
@nameservers.each do |host, port|
|
|
90
|
+
begin
|
|
91
|
+
return request(host, port, wire)
|
|
92
|
+
rescue StandardError => error
|
|
93
|
+
last_error = error
|
|
94
|
+
end
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
raise ResolutionError,
|
|
98
|
+
"DNS lookup failed: #{last_error&.message || 'no nameservers'}"
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
def request(host, port, wire)
|
|
102
|
+
socket = Socket.tcp(host, port, connect_timeout: @timeout)
|
|
103
|
+
socket.timeout = @timeout if socket.respond_to?(:timeout=)
|
|
104
|
+
|
|
105
|
+
socket.write([wire.bytesize].pack('n') + wire)
|
|
106
|
+
|
|
107
|
+
header = socket.read(2)
|
|
108
|
+
raise ResolutionError, 'short DNS response' if header.nil? || header.bytesize < 2
|
|
109
|
+
|
|
110
|
+
body = socket.read(header.unpack1('n'))
|
|
111
|
+
raise ResolutionError, 'short DNS response' if body.nil?
|
|
112
|
+
|
|
113
|
+
body
|
|
114
|
+
ensure
|
|
115
|
+
socket&.close
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
end
|
data/lib/dmarcator/resolver.rb
CHANGED
|
@@ -4,6 +4,7 @@ require 'resolv'
|
|
|
4
4
|
|
|
5
5
|
require_relative 'exceptions'
|
|
6
6
|
require_relative 'resolver/encrypted'
|
|
7
|
+
require_relative 'resolver/plain'
|
|
7
8
|
require_relative 'resolver/doh'
|
|
8
9
|
require_relative 'resolver/dot'
|
|
9
10
|
|
|
@@ -14,7 +15,7 @@ module DMARCator
|
|
|
14
15
|
# The transport is chosen with the `DMARCATOR_DNS` environment variable, or
|
|
15
16
|
# explicitly via {.build}:
|
|
16
17
|
#
|
|
17
|
-
# DMARCATOR_DNS=plaintext # default:
|
|
18
|
+
# DMARCATOR_DNS=plaintext # default: TCP (Resolver::Plain)
|
|
18
19
|
# DMARCATOR_DNS=doh # requires DMARCATOR_DOH_URL
|
|
19
20
|
# DMARCATOR_DNS=dot # requires DMARCATOR_DOT_SERVER
|
|
20
21
|
#
|
|
@@ -61,9 +62,7 @@ module DMARCator
|
|
|
61
62
|
|
|
62
63
|
case name
|
|
63
64
|
when 'plaintext'
|
|
64
|
-
|
|
65
|
-
resolver.timeouts = DNS_TIMEOUTS
|
|
66
|
-
resolver
|
|
65
|
+
Plain.from_env(env)
|
|
67
66
|
when 'doh'
|
|
68
67
|
DoH.from_env(env)
|
|
69
68
|
when 'dot'
|
data/lib/dmarcator/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: dmarcator
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Noah Berman
|
|
@@ -9,26 +9,6 @@ bindir: bin
|
|
|
9
9
|
cert_chain: []
|
|
10
10
|
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
11
|
dependencies:
|
|
12
|
-
- !ruby/object:Gem::Dependency
|
|
13
|
-
name: parslet
|
|
14
|
-
requirement: !ruby/object:Gem::Requirement
|
|
15
|
-
requirements:
|
|
16
|
-
- - ">="
|
|
17
|
-
- !ruby/object:Gem::Version
|
|
18
|
-
version: 1.0.0
|
|
19
|
-
- - "<"
|
|
20
|
-
- !ruby/object:Gem::Version
|
|
21
|
-
version: 3.0.0
|
|
22
|
-
type: :runtime
|
|
23
|
-
prerelease: false
|
|
24
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
25
|
-
requirements:
|
|
26
|
-
- - ">="
|
|
27
|
-
- !ruby/object:Gem::Version
|
|
28
|
-
version: 1.0.0
|
|
29
|
-
- - "<"
|
|
30
|
-
- !ruby/object:Gem::Version
|
|
31
|
-
version: 3.0.0
|
|
32
12
|
- !ruby/object:Gem::Dependency
|
|
33
13
|
name: nokogiri
|
|
34
14
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -71,6 +51,7 @@ files:
|
|
|
71
51
|
- lib/dmarcator/resolver/doh.rb
|
|
72
52
|
- lib/dmarcator/resolver/dot.rb
|
|
73
53
|
- lib/dmarcator/resolver/encrypted.rb
|
|
54
|
+
- lib/dmarcator/resolver/plain.rb
|
|
74
55
|
- lib/dmarcator/uri.rb
|
|
75
56
|
- lib/dmarcator/version.rb
|
|
76
57
|
homepage: https://github.com/bermannoah/dmarcator
|