homographic_spoofing 0.1.1 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +56 -0
- data/lib/homographic_spoofing/detector/detection.rb +9 -1
- data/lib/homographic_spoofing/detector/email_address.rb +382 -5
- data/lib/homographic_spoofing/detector/idn.rb +54 -5
- data/lib/homographic_spoofing/detector/rule/base.rb +1 -1
- data/lib/homographic_spoofing/detector/rule/context.rb +6 -2
- data/lib/homographic_spoofing/detector/rule/data/allowed_idn_characters.txt +1 -1
- data/lib/homographic_spoofing/detector/rule/data/digits.csv +80 -0
- data/lib/homographic_spoofing/detector/rule/disallowed_characters.rb +11 -4
- data/lib/homographic_spoofing/detector/rule/idn/base.rb +1 -1
- data/lib/homographic_spoofing/detector/rule/idn/context.rb +9 -3
- data/lib/homographic_spoofing/detector/rule/idn/dangerous_pattern.rb +15 -0
- data/lib/homographic_spoofing/detector/rule/idn/invisible_characters.rb +1 -1
- data/lib/homographic_spoofing/detector/rule/idn/script_confusable.rb +15 -3
- data/lib/homographic_spoofing/detector/rule/mixed_digits.rb +1 -1
- data/lib/homographic_spoofing/sanitizer/base.rb +142 -8
- data/lib/homographic_spoofing/sanitizer/idn.rb +30 -0
- data/lib/homographic_spoofing/version.rb +1 -1
- metadata +21 -7
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e62f8de9bed530122d993c518081b18c90ff10a28b6a1890daa2181beff984aa
|
|
4
|
+
data.tar.gz: efee11ffe57495378d48a1ed545ff6fc9d51189168df8766edf5a09779e27692
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 0e00f735b4ebcaae22a5c9f85ce5701da7184b44c474015e2f9ca4af661e426cb61e743ee6879de997a01dc41a869a100f3e92b2d702001686eb6716e9d2e893
|
|
7
|
+
data.tar.gz: bc297a9087a1fcc45f10c4c570c4675932733f3be961ea445796da6a44bfa4f344bbc8d3585492829fd394a7dcbab94ff327ade354acc3746ab35d2acb3214fa
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.2.0] - 2026-10-20
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- Sanitize uppercase and mixed-case confusable IDN domains.
|
|
15
|
+
- Scan each domain label independently for script confusables.
|
|
16
|
+
- Anchor allowed-TLD matching to whole TLD labels.
|
|
17
|
+
- Disallow the characters Chromium removes as quotation-mark and letter look-alikes: U+0138 (kra), U+02BB (turned comma), U+02BC (apostrophe) and U+02EC (voicing).
|
|
18
|
+
- Renamed `MOZILLA_DISALLOWED_CHARACTERS` to `DISALLOWED_CHARACTERS`, since the list now combines Mozilla's and Chromium's removals.
|
|
19
|
+
- Apply the mixed-script, mixed-digit, digit and script-specific rules to each subdomain label separately rather than to the whole subdomain chain.
|
|
20
|
+
- Flag a label that is only a CJK ideograph that looks like Latin or punctuation, such as `一` in `a.一.example.com`.
|
|
21
|
+
- Sanitize an email address only within the part each detection came from, so a benign mailbox that equals a spoofed domain label is left intact.
|
|
22
|
+
- Check the mailbox and each route domain of an address with an obsolete route separately instead of flagging the route itself.
|
|
23
|
+
|
|
24
|
+
### Security
|
|
25
|
+
|
|
26
|
+
- Pin GitHub Actions to commit SHAs and add an actionlint + zizmor audit job.
|
|
27
|
+
|
|
28
|
+
## [0.1.2] - 2025-07-22
|
|
29
|
+
|
|
30
|
+
### Added
|
|
31
|
+
|
|
32
|
+
- `csv` gem dependency (extracted from the standard library in Ruby 3.4).
|
|
33
|
+
|
|
34
|
+
### Changed
|
|
35
|
+
|
|
36
|
+
- Regenerated the allowed-IDN-character and digit tables.
|
|
37
|
+
- Renamed `MOZZILLA_DISALLOWED_CHARACTERS` to `MOZILLA_DISALLOWED_CHARACTERS`.
|
|
38
|
+
|
|
39
|
+
## [0.1.1] - 2024-06-20
|
|
40
|
+
|
|
41
|
+
### Added
|
|
42
|
+
|
|
43
|
+
- `homepage_uri` and `source_code_uri` gem metadata.
|
|
44
|
+
|
|
45
|
+
## [0.1.0] - 2024-06-20
|
|
46
|
+
|
|
47
|
+
### Added
|
|
48
|
+
|
|
49
|
+
- Initial release: detect and sanitize homographic spoofing attacks in URLs and
|
|
50
|
+
email addresses.
|
|
51
|
+
|
|
52
|
+
[Unreleased]: https://github.com/basecamp/homographic_spoofing/compare/v0.2.0...HEAD
|
|
53
|
+
[0.2.0]: https://github.com/basecamp/homographic_spoofing/compare/v0.1.2...v0.2.0
|
|
54
|
+
[0.1.2]: https://github.com/basecamp/homographic_spoofing/compare/v0.1.1...v0.1.2
|
|
55
|
+
[0.1.1]: https://github.com/basecamp/homographic_spoofing/compare/v0.1.0...v0.1.1
|
|
56
|
+
[0.1.0]: https://github.com/basecamp/homographic_spoofing/releases/tag/v0.1.0
|
|
@@ -1,2 +1,10 @@
|
|
|
1
|
-
|
|
1
|
+
# `span` is the [offset, length] of the address component the detection came
|
|
2
|
+
# from, in characters, within the field — so the sanitizer punycodes the
|
|
3
|
+
# offending label only inside that component and never rewrites a matching label
|
|
4
|
+
# in a sibling component (a local part equal to a spoofed domain label, say). It
|
|
5
|
+
# is derived from the parsed address, not by re-scanning the raw string for an
|
|
6
|
+
# "@", which trailing CFWS comments or a display name can carry spuriously. It
|
|
7
|
+
# is nil for detectors that run against a standalone string (a bare IDN or
|
|
8
|
+
# quoted string), where the whole field is the one component.
|
|
9
|
+
class HomographicSpoofing::Detector::Detection < Struct.new(:reason, :label, :span)
|
|
2
10
|
end
|
|
@@ -17,10 +17,12 @@ class HomographicSpoofing::Detector::EmailAddress
|
|
|
17
17
|
|
|
18
18
|
def detections
|
|
19
19
|
mail_address = mail_address_wrap(email_address)
|
|
20
|
+
spans = component_spans(mail_address)
|
|
20
21
|
[].tap do |result|
|
|
21
|
-
result.concat
|
|
22
|
-
result.concat
|
|
23
|
-
result.concat
|
|
22
|
+
result.concat detections_for(text: mail_address.name, type: "quoted_string", span: spans[:name])
|
|
23
|
+
result.concat detections_for(text: mailbox(mail_address), type: "local", span: spans[:local])
|
|
24
|
+
result.concat detections_for(text: mail_address.domain, type: "idn", span: spans[:domain])
|
|
25
|
+
route_domains.each { |domain, span| result.concat detections_for(text: domain, type: "idn", span:) }
|
|
24
26
|
end
|
|
25
27
|
rescue Mail::Field::FieldError
|
|
26
28
|
# Do not analyse invalid email addresses.
|
|
@@ -30,10 +32,385 @@ class HomographicSpoofing::Detector::EmailAddress
|
|
|
30
32
|
private
|
|
31
33
|
attr_reader :email_address
|
|
32
34
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
+
# Tag each detection with the [offset, length] of the component it came from
|
|
36
|
+
# so the sanitizer scopes the punycode replacement to that component alone —
|
|
37
|
+
# a domain-label spoof never rewrites a benign local part that merely equals
|
|
38
|
+
# the same string.
|
|
39
|
+
def detections_for(text:, type:, span:)
|
|
40
|
+
if text
|
|
41
|
+
detector_for(type:, text:).detections.each { |detection| detection.span = span }
|
|
42
|
+
else
|
|
43
|
+
[]
|
|
44
|
+
end
|
|
35
45
|
end
|
|
36
46
|
|
|
47
|
+
def detector_for(type:, text:)
|
|
48
|
+
"HomographicSpoofing::Detector::#{type.camelize}".constantize.new(text)
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
# Derive each component's [offset, length] span from parser token boundaries,
|
|
52
|
+
# not by searching the raw field for the stripped parts. The addr-spec built
|
|
53
|
+
# from the parser's *raw* local and domain — which keep the CFWS the stripped
|
|
54
|
+
# forms drop — is by construction a contiguous substring of the field, so its
|
|
55
|
+
# offset pins the mailbox and host spans exactly even when a comment or stray
|
|
56
|
+
# whitespace sits inside the address. The offset is taken inside the structural
|
|
57
|
+
# angle-address when there is one, so a display name or comment that repeats
|
|
58
|
+
# the addr-spec cannot divert the recipient's replacement onto itself. Only the
|
|
59
|
+
# display name — never the recipient — is still located by text (outside any
|
|
60
|
+
# comment), since a mislocated name only misplaces a cosmetic fix. When the
|
|
61
|
+
# field is not a raw string (a Mail::Address handed straight to a detection
|
|
62
|
+
# query) there is nothing to sanitize, so spans are left nil.
|
|
63
|
+
def component_spans(mail_address)
|
|
64
|
+
local, domain, name = mail_address.local, mail_address.domain, mail_address.name
|
|
65
|
+
return {} unless email_address.is_a?(String)
|
|
66
|
+
|
|
67
|
+
raw_local, raw_domain = raw_addr_spec_parts
|
|
68
|
+
addr = "#{raw_local}@#{raw_domain}" if raw_local && raw_domain
|
|
69
|
+
at = addr_offset(addr) if addr
|
|
70
|
+
|
|
71
|
+
if at
|
|
72
|
+
{ name: (name_span(name, avoid: [ at, addr.length ]) if name),
|
|
73
|
+
local: (local_span(at, raw_local) if local),
|
|
74
|
+
domain: ([ at + raw_local.length + 1, raw_domain.length ] if domain) }
|
|
75
|
+
else
|
|
76
|
+
structural_spans(local, domain, name)
|
|
77
|
+
end
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
# The raw local and domain of the addr-spec as they appear in the field, CFWS
|
|
81
|
+
# included, from the address parser — or nils when it cannot supply them.
|
|
82
|
+
def raw_addr_spec_parts
|
|
83
|
+
[ parsed_address&.local, parsed_address&.domain ]
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
def parsed_address
|
|
87
|
+
return @parsed_address if defined?(@parsed_address)
|
|
88
|
+
@parsed_address = Mail::Parsers::AddressListsParser.parse(email_address).addresses.first
|
|
89
|
+
rescue StandardError
|
|
90
|
+
@parsed_address = nil
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
# The mailbox. Mail::Address#local keeps an obsolete route ("@relay:" in
|
|
94
|
+
# "<@relay:local@domain>") in front of it; the route is a list of domains,
|
|
95
|
+
# not part of the mailbox, so it is set aside here and its domains are
|
|
96
|
+
# checked as domains (see #route_domains). Each part is then punycoded on its
|
|
97
|
+
# own, and the route's "@", "," and ":" stay as they are.
|
|
98
|
+
#
|
|
99
|
+
# Comments are set aside too: they are not part of the mailbox, and the
|
|
100
|
+
# parser may take the display name from one ("<local(Name)@host>"), so the
|
|
101
|
+
# name's replacement must not change the text the mailbox is matched by.
|
|
102
|
+
def mailbox(mail_address)
|
|
103
|
+
local, route = mail_address.local, parsed_address&.obs_domain_list
|
|
104
|
+
local = local.delete_prefix(route) if route.present? && local
|
|
105
|
+
without_comments(local)&.strip
|
|
106
|
+
end
|
|
107
|
+
|
|
108
|
+
# The text with its comments removed, honouring nesting, quoted strings and
|
|
109
|
+
# backslash escapes, in one pass.
|
|
110
|
+
def without_comments(text)
|
|
111
|
+
return text unless text&.include?("(")
|
|
112
|
+
|
|
113
|
+
bare, depth, quoted, escaped = +"", 0, false, false
|
|
114
|
+
text.each_char do |char|
|
|
115
|
+
commented = depth > 0
|
|
116
|
+
if escaped
|
|
117
|
+
escaped = false
|
|
118
|
+
elsif char == "\\" && (commented || quoted)
|
|
119
|
+
escaped = true
|
|
120
|
+
elsif commented
|
|
121
|
+
depth += 1 if char == "("
|
|
122
|
+
depth -= 1 if char == ")"
|
|
123
|
+
elsif quoted
|
|
124
|
+
quoted = false if char == '"'
|
|
125
|
+
elsif char == '"'
|
|
126
|
+
quoted = true
|
|
127
|
+
elsif char == "("
|
|
128
|
+
depth, commented = 1, true
|
|
129
|
+
end
|
|
130
|
+
bare << char unless commented
|
|
131
|
+
end
|
|
132
|
+
bare
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
# Each domain of an obsolete route, with its span in the field: the route
|
|
136
|
+
# sits inside the structural angle-address, before the addr-spec.
|
|
137
|
+
def route_domains
|
|
138
|
+
route = parsed_address&.obs_domain_list
|
|
139
|
+
return [] unless email_address.is_a?(String) && route.present?
|
|
140
|
+
|
|
141
|
+
lt, gt = angle_address
|
|
142
|
+
start = structural_index(route, from: lt + 1) if lt
|
|
143
|
+
return [] unless start && start + route.length <= gt
|
|
144
|
+
|
|
145
|
+
route_entries(route).filter_map do |domain, offset, length|
|
|
146
|
+
[ domain, [ start + offset, length ] ] if domain.present? && !domain.start_with?("[")
|
|
147
|
+
end
|
|
148
|
+
end
|
|
149
|
+
|
|
150
|
+
# Split a route ("@a.example, @b.example:") into its domains in one pass, on
|
|
151
|
+
# each "," or ":" outside comments and domain literals ("@[IPv6:2001:db8::1]"
|
|
152
|
+
# is one entry; a literal is no DNS name, so it is never checked as one).
|
|
153
|
+
# Each entry's span starts after its "@";
|
|
154
|
+
# its domain is checked with comments and whitespace set aside (the obsolete
|
|
155
|
+
# syntax allows both between labels), while the sanitizer, which also sets
|
|
156
|
+
# them aside per label, rewrites only the labels within the span.
|
|
157
|
+
def route_entries(route)
|
|
158
|
+
entries = []
|
|
159
|
+
bare, at, entry_start, depth, escaped, literal = +"", nil, 0, 0, false, false
|
|
160
|
+
|
|
161
|
+
route.each_char.with_index do |char, index|
|
|
162
|
+
commented = depth > 0
|
|
163
|
+
if escaped
|
|
164
|
+
escaped = false
|
|
165
|
+
elsif (commented || literal) && char == "\\"
|
|
166
|
+
escaped = true
|
|
167
|
+
elsif commented
|
|
168
|
+
depth += 1 if char == "("
|
|
169
|
+
depth -= 1 if char == ")"
|
|
170
|
+
elsif literal
|
|
171
|
+
literal = false if char == "]"
|
|
172
|
+
elsif char == "("
|
|
173
|
+
depth, commented = 1, true
|
|
174
|
+
elsif char == "["
|
|
175
|
+
literal = true
|
|
176
|
+
elsif char == "," || char == ":"
|
|
177
|
+
entries << [ bare.gsub(/\s/, ""), at || entry_start, index - (at || entry_start) ]
|
|
178
|
+
bare, at, entry_start = +"", nil, index + 1
|
|
179
|
+
next
|
|
180
|
+
elsif char == "@" && at.nil?
|
|
181
|
+
at = index + 1
|
|
182
|
+
next
|
|
183
|
+
end
|
|
184
|
+
bare << char unless commented || at.nil?
|
|
185
|
+
end
|
|
186
|
+
|
|
187
|
+
entries << [ bare.gsub(/\s/, ""), at || entry_start, route.length - (at || entry_start) ]
|
|
188
|
+
end
|
|
189
|
+
|
|
190
|
+
def local_span(at, raw_local)
|
|
191
|
+
[ at, raw_local.length ]
|
|
192
|
+
end
|
|
193
|
+
|
|
194
|
+
# The display name's span. When the name the parser reports is not in the
|
|
195
|
+
# field as written (it unescapes quoted pairs, so "\\x" reads as "x"), there
|
|
196
|
+
# is no text to rewrite safely: its span is empty, so the detection is still
|
|
197
|
+
# reported but the field is left as is rather than rewritten across an
|
|
198
|
+
# escape or onto the mailbox or host.
|
|
199
|
+
def name_span(name, avoid: nil)
|
|
200
|
+
locate(name, avoid:) || [ 0, 0 ]
|
|
201
|
+
end
|
|
202
|
+
|
|
203
|
+
# The addr-spec's offset in the field. When the field has a structural
|
|
204
|
+
# angle-address — its "<" outside every quoted string and comment, since a
|
|
205
|
+
# quoted display name or a comment may itself spell out "<local@domain>" — the
|
|
206
|
+
# addr-spec is searched for inside it rather than matched as "<addr>", because
|
|
207
|
+
# an obsolete route ("<@relay:local@domain>") may precede it; a copy of the
|
|
208
|
+
# addr-spec anywhere else in the field is never the recipient. Without an
|
|
209
|
+
# angle-address, the first bare occurrence outside any comment.
|
|
210
|
+
def addr_offset(addr)
|
|
211
|
+
lt, gt = angle_address
|
|
212
|
+
if lt
|
|
213
|
+
at = structural_index(addr, from: lt + 1)
|
|
214
|
+
at if at && at + addr.length <= gt
|
|
215
|
+
else
|
|
216
|
+
structural_index(addr)
|
|
217
|
+
end
|
|
218
|
+
end
|
|
219
|
+
|
|
220
|
+
# The offsets of the "<" and ">" of the structural angle-address, or nil when
|
|
221
|
+
# the field has none.
|
|
222
|
+
def angle_address
|
|
223
|
+
lt = structural_index("<")
|
|
224
|
+
gt = structural_index(">", from: lt) if lt
|
|
225
|
+
[ lt, gt ] if lt && gt
|
|
226
|
+
end
|
|
227
|
+
|
|
228
|
+
# The first occurrence of `needle` at or after `from` that starts outside
|
|
229
|
+
# every quoted string, domain literal and comment in the field.
|
|
230
|
+
def structural_index(needle, from: 0)
|
|
231
|
+
enclosed, commented, _unsearchable = enclosures
|
|
232
|
+
while (at = index_in_field(needle, from))
|
|
233
|
+
return at unless enclosed[at] || commented[at]
|
|
234
|
+
from = at + 1
|
|
235
|
+
end
|
|
236
|
+
nil
|
|
237
|
+
end
|
|
238
|
+
|
|
239
|
+
# Which character offsets of the field sit inside a quoted string or a domain
|
|
240
|
+
# literal, and which inside a (nestable) comment. Each opening delimiter is
|
|
241
|
+
# outside its own enclosure, so a needle may begin with one ('"john"@host');
|
|
242
|
+
# the closing delimiter is inside. Backslash escapes are honored in all
|
|
243
|
+
# three, and inside a comment neither quotes nor literals have structure —
|
|
244
|
+
# only nesting parentheses do. The third table marks what a replacement must
|
|
245
|
+
# never include: quoted pairs, and every parenthesis inside a comment (a
|
|
246
|
+
# nested comment's delimiters and the closing one).
|
|
247
|
+
def enclosures
|
|
248
|
+
@enclosures ||= begin
|
|
249
|
+
length = email_address.length
|
|
250
|
+
enclosed, commented, unsearchable = Array.new(length, false), Array.new(length, false), Array.new(length, false)
|
|
251
|
+
closer, depth, escaped = nil, 0, false
|
|
252
|
+
email_address.each_char.with_index do |char, i|
|
|
253
|
+
enclosed[i], commented[i] = !closer.nil?, depth > 0
|
|
254
|
+
if escaped
|
|
255
|
+
escaped = false
|
|
256
|
+
unsearchable[i] = true
|
|
257
|
+
elsif char == "\\" && (closer || depth > 0)
|
|
258
|
+
escaped = true
|
|
259
|
+
unsearchable[i] = true
|
|
260
|
+
elsif closer
|
|
261
|
+
closer = nil if char == closer
|
|
262
|
+
elsif depth > 0
|
|
263
|
+
depth += 1 if char == "("
|
|
264
|
+
depth -= 1 if char == ")"
|
|
265
|
+
unsearchable[i] = true if char == "(" || char == ")"
|
|
266
|
+
elsif char == '"'
|
|
267
|
+
closer = '"'
|
|
268
|
+
elsif char == "["
|
|
269
|
+
closer = "]"
|
|
270
|
+
elsif char == "("
|
|
271
|
+
depth = 1
|
|
272
|
+
end
|
|
273
|
+
end
|
|
274
|
+
[ enclosed, commented, unsearchable ]
|
|
275
|
+
end
|
|
276
|
+
end
|
|
277
|
+
|
|
278
|
+
# Fallback when the parser yields no raw addr-spec: bound the components by the
|
|
279
|
+
# structural delimiters instead. The addr-spec sits between the angle brackets
|
|
280
|
+
# when present (split at its last "@", past any obsolete route), otherwise it
|
|
281
|
+
# is the whole field, split at the first "@".
|
|
282
|
+
def structural_spans(local, domain, name)
|
|
283
|
+
lt, gt = angle_address
|
|
284
|
+
at = lt ? last_structural_at(lt, gt) : structural_index("@")
|
|
285
|
+
|
|
286
|
+
if lt && at && at < gt
|
|
287
|
+
{ name: (name_span(name, avoid: [ lt, gt - lt + 1 ]) if name),
|
|
288
|
+
local: ([ lt + 1, at - lt - 1 ] if local),
|
|
289
|
+
domain: ([ at + 1, gt - at - 1 ] if domain) }
|
|
290
|
+
elsif at
|
|
291
|
+
{ name: (name_span(name) if name),
|
|
292
|
+
local: ([ 0, at ] if local),
|
|
293
|
+
domain: (bare_domain_span(at, domain) if domain) }
|
|
294
|
+
else
|
|
295
|
+
{}
|
|
296
|
+
end
|
|
297
|
+
end
|
|
298
|
+
|
|
299
|
+
# Everything after the "@" — unless the domain the parser reports carries an
|
|
300
|
+
# "@" itself, which it does when it folds the mailbox into the domain
|
|
301
|
+
# ("а(comment).b@(note)safe.com"). The boundary between the parts is then
|
|
302
|
+
# unknown, so the domain spans the whole field, as every detection did
|
|
303
|
+
# before spans existed.
|
|
304
|
+
def bare_domain_span(at, domain)
|
|
305
|
+
[ at + 1, char_count - at - 1 ] unless domain.include?("@")
|
|
306
|
+
end
|
|
307
|
+
|
|
308
|
+
# The addr-spec's "@" inside the angle-address: the last structural one, since
|
|
309
|
+
# an obsolete route ("<@relay:local@domain>") puts its own before it.
|
|
310
|
+
def last_structural_at(lt, gt)
|
|
311
|
+
last = nil
|
|
312
|
+
while (at = structural_index("@", from: (last || lt) + 1)) && at < gt
|
|
313
|
+
last = at
|
|
314
|
+
end
|
|
315
|
+
last
|
|
316
|
+
end
|
|
317
|
+
|
|
318
|
+
# The first occurrence of `text` in the field at or after `from` that lies
|
|
319
|
+
# wholly outside every comment and outside `avoid` (the addr-spec, when
|
|
320
|
+
# locating a display name that shares a spelling with the mailbox or host);
|
|
321
|
+
# leading CFWS may repeat a display name. Failing that, the first occurrence
|
|
322
|
+
# wholly inside one comment: the parser takes a display name from a trailing
|
|
323
|
+
# comment ("user@host (Name)"), which its raw domain token also carries, and
|
|
324
|
+
# a comment is never the mailbox or host even when it sits inside `avoid`.
|
|
325
|
+
# An occurrence that crosses a comment's edge is neither, so it never sends
|
|
326
|
+
# a display name's replacement into the mailbox.
|
|
327
|
+
def locate(text, from: 0, avoid: nil)
|
|
328
|
+
outside = comment_runs(commented: false).flat_map { |range| outside_of(range, avoid) }
|
|
329
|
+
search_ranges(text, outside, from:) || search_ranges(text, comment_runs(commented: true), from:)
|
|
330
|
+
end
|
|
331
|
+
|
|
332
|
+
# The maximal [start, finish) runs of characters inside (or outside)
|
|
333
|
+
# comments, without the parentheses that open and close each comment or any
|
|
334
|
+
# comment nested in it, and without quoted pairs ("\\(" and the like), so a
|
|
335
|
+
# display name's replacement can never move a comment's edge or separate an
|
|
336
|
+
# escape from what it escapes.
|
|
337
|
+
# The parser unescapes quoted pairs in the name it reports, so a name that
|
|
338
|
+
# carries one is not in the field as written anyway.
|
|
339
|
+
def comment_runs(commented:)
|
|
340
|
+
@comment_runs ||= begin
|
|
341
|
+
char_to_byte, _byte_to_char = offset_tables
|
|
342
|
+
delimiter = ->(index) { field_bytes.getbyte(char_to_byte[index]) }
|
|
343
|
+
_enclosed, inside_comment, unsearchable = enclosures
|
|
344
|
+
keys = inside_comment.each_with_index.map { |inside, index| unsearchable[index] ? :unsearchable : inside }
|
|
345
|
+
keys.each_with_index.chunk_while { |(a, _), (b, _)| a == b }.filter_map do |run|
|
|
346
|
+
inside, start, finish = run.first.first, run.first.last, run.last.last + 1
|
|
347
|
+
next if inside == :unsearchable
|
|
348
|
+
|
|
349
|
+
finish -= 1 if !inside && finish < char_to_byte.length - 1 && delimiter.(finish - 1) == "(".ord
|
|
350
|
+
[ inside, start, finish ]
|
|
351
|
+
end
|
|
352
|
+
end
|
|
353
|
+
@comment_runs.filter_map { |inside, start, finish| [ start, finish ] if inside == commented && start < finish }
|
|
354
|
+
end
|
|
355
|
+
|
|
356
|
+
def outside_of((start, finish), avoid)
|
|
357
|
+
return [ [ start, finish ] ] unless avoid
|
|
358
|
+
|
|
359
|
+
[ [ start, [ finish, avoid[0] ].min ], [ [ start, avoid[0] + avoid[1] ].max, finish ] ].reject { |a, b| a >= b }
|
|
360
|
+
end
|
|
361
|
+
|
|
362
|
+
# The first occurrence of `text` wholly inside one of `ranges`, searching
|
|
363
|
+
# each range once, so the cost stays linear however often `text` overlaps
|
|
364
|
+
# itself.
|
|
365
|
+
def search_ranges(text, ranges, from:)
|
|
366
|
+
char_to_byte, byte_to_char = offset_tables
|
|
367
|
+
needle, length = text.b, text.length
|
|
368
|
+
ranges.each do |start, finish|
|
|
369
|
+
start = [ start, from ].max
|
|
370
|
+
next if finish - start < length
|
|
371
|
+
|
|
372
|
+
at = field_bytes.byteslice(char_to_byte[start]...char_to_byte[finish]).index(needle)
|
|
373
|
+
return [ byte_to_char[char_to_byte[start] + at], length ] if at
|
|
374
|
+
end
|
|
375
|
+
nil
|
|
376
|
+
end
|
|
377
|
+
|
|
378
|
+
# The first occurrence of `needle` at or after character offset `from`.
|
|
379
|
+
# String#index converts a character offset by walking a multibyte string
|
|
380
|
+
# from its start on every call, so a loop of searches over a long field turns
|
|
381
|
+
# quadratic. Search the field's bytes instead, where an offset costs nothing
|
|
382
|
+
# and a UTF-8 match always starts on a character boundary, and map between
|
|
383
|
+
# bytes and characters through tables built once.
|
|
384
|
+
def index_in_field(needle, from)
|
|
385
|
+
char_to_byte, byte_to_char = offset_tables
|
|
386
|
+
return nil if from >= char_to_byte.length
|
|
387
|
+
|
|
388
|
+
at = field_bytes.index(needle.b, char_to_byte[from])
|
|
389
|
+
byte_to_char[at] if at
|
|
390
|
+
end
|
|
391
|
+
|
|
392
|
+
def char_count
|
|
393
|
+
offset_tables.first.length - 1
|
|
394
|
+
end
|
|
395
|
+
|
|
396
|
+
def field_bytes
|
|
397
|
+
@field_bytes ||= email_address.b
|
|
398
|
+
end
|
|
399
|
+
|
|
400
|
+
def offset_tables
|
|
401
|
+
@offset_tables ||= begin
|
|
402
|
+
char_to_byte, byte_to_char = [], []
|
|
403
|
+
email_address.each_char.with_index do |char, index|
|
|
404
|
+
char_to_byte << byte_to_char.length
|
|
405
|
+
char.bytesize.times { byte_to_char << index }
|
|
406
|
+
end
|
|
407
|
+
char_to_byte << byte_to_char.length
|
|
408
|
+
byte_to_char << email_address.length
|
|
409
|
+
[ char_to_byte, byte_to_char ]
|
|
410
|
+
end
|
|
411
|
+
end
|
|
412
|
+
|
|
413
|
+
|
|
37
414
|
def mail_address_wrap(email_address)
|
|
38
415
|
email_address.is_a?(Mail::Address) ? email_address : Mail::Address.new(email_address)
|
|
39
416
|
end
|
|
@@ -16,6 +16,7 @@ class HomographicSpoofing::Detector::Idn
|
|
|
16
16
|
end
|
|
17
17
|
|
|
18
18
|
def initialize(domain)
|
|
19
|
+
@original_domain = domain
|
|
19
20
|
@domain = domain.downcase
|
|
20
21
|
end
|
|
21
22
|
|
|
@@ -25,15 +26,42 @@ class HomographicSpoofing::Detector::Idn
|
|
|
25
26
|
|
|
26
27
|
def detections
|
|
27
28
|
rules.select(&:attack_detected?).map do |rule|
|
|
28
|
-
HomographicSpoofing::Detector::Detection.new(rule.reason, rule.label)
|
|
29
|
+
HomographicSpoofing::Detector::Detection.new(rule.reason, original_case(rule.label, rule.occurrence))
|
|
29
30
|
end
|
|
30
31
|
rescue PublicSuffix::Error
|
|
31
32
|
# Invalid IDN is a spoof.
|
|
32
|
-
[ HomographicSpoofing::Detector::Detection.new("invalid_domain",
|
|
33
|
+
[ HomographicSpoofing::Detector::Detection.new("invalid_domain", original_domain) ]
|
|
33
34
|
end
|
|
34
35
|
|
|
35
36
|
private
|
|
36
|
-
attr_reader :domain
|
|
37
|
+
attr_reader :domain, :original_domain
|
|
38
|
+
|
|
39
|
+
# Detection runs on the lowercased domain, so labels come back lowercased.
|
|
40
|
+
# Recover the original-cased spelling of the label at its own position, so
|
|
41
|
+
# the sanitizer can substitute by exact match instead of regexp case folding,
|
|
42
|
+
# which both over-matches (folds unrelated ASCII, e.g. ſ/s) and under-matches
|
|
43
|
+
# (misses case pairs folding omits, e.g. Ⱥ/ⱥ). A label repeated in the domain
|
|
44
|
+
# resolves to the casing of its own occurrence, counted left to right.
|
|
45
|
+
def original_case(label, occurrence = 0)
|
|
46
|
+
original_labels.fetch(label, [])[occurrence] || label
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Every whole label of the raw domain, in its original casing, grouped by its
|
|
50
|
+
# lowercased form in left-to-right order. Labels are bounded by ".", never by
|
|
51
|
+
# a match inside a longer sibling ("з" inside "магаЗин"), and keep any
|
|
52
|
+
# whitespace inside them as PublicSuffix does; only the whitespace it strips
|
|
53
|
+
# from the ends of the domain is set aside. Built in one pass, so resolving
|
|
54
|
+
# every detection stays linear in the domain's length.
|
|
55
|
+
def original_labels
|
|
56
|
+
@original_labels ||= begin
|
|
57
|
+
originals = original_domain.split(".", -1)
|
|
58
|
+
originals[0] = originals[0].lstrip if originals.any?
|
|
59
|
+
originals[-1] = originals[-1].rstrip if originals.any?
|
|
60
|
+
originals.each_with_object({}) do |original, labels|
|
|
61
|
+
(labels[original.downcase] ||= []) << original
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
end
|
|
37
65
|
|
|
38
66
|
def rules
|
|
39
67
|
@rules ||= contexts.flat_map { |ctx| rules_for(ctx) }
|
|
@@ -55,8 +83,29 @@ class HomographicSpoofing::Detector::Idn
|
|
|
55
83
|
end
|
|
56
84
|
|
|
57
85
|
def contexts
|
|
58
|
-
|
|
59
|
-
|
|
86
|
+
tld, tld_allowances = public_suffix.tld, {}
|
|
87
|
+
labels.map do |label, occurrence|
|
|
88
|
+
HomographicSpoofing::Detector::Rule::Idn::Context.new(label:, tld:, occurrence:, tld_allowances:)
|
|
89
|
+
end
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
# `trd` is the full subdomain chain ("a.b" in a.b.example.com). Split it on
|
|
93
|
+
# the same dot the renderer draws so each rule sees one real label rather
|
|
94
|
+
# than a chain. The mixed-script, confusable and digit rules are per-label:
|
|
95
|
+
# a combined chain both hides attacks (a benign sibling dilutes an
|
|
96
|
+
# all-look-alike label out of detection) and invents them (two single-script
|
|
97
|
+
# sibling labels look "mixed" together though each is safe on its own).
|
|
98
|
+
#
|
|
99
|
+
# Labels are kept in left-to-right domain order and paired with the index of
|
|
100
|
+
# their occurrence among identical labels, so a repeated label recovers the
|
|
101
|
+
# casing of its own position (see #original_case).
|
|
102
|
+
def labels
|
|
103
|
+
ordered = [ *public_suffix.trd&.split("."), public_suffix.sld ].compact.reject(&:empty?)
|
|
104
|
+
seen = Hash.new(0)
|
|
105
|
+
ordered.map do |label|
|
|
106
|
+
occurrence = seen[label]
|
|
107
|
+
seen[label] += 1
|
|
108
|
+
[ label, occurrence ]
|
|
60
109
|
end
|
|
61
110
|
end
|
|
62
111
|
|
|
@@ -1,8 +1,12 @@
|
|
|
1
1
|
class HomographicSpoofing::Detector::Rule::Context
|
|
2
|
-
attr_reader :label
|
|
2
|
+
attr_reader :label, :occurrence
|
|
3
3
|
|
|
4
|
-
|
|
4
|
+
# `occurrence` disambiguates labels that repeat within a domain: it is the
|
|
5
|
+
# zero-based index of this label among identical labels, in left-to-right
|
|
6
|
+
# order, so the original casing of the right occurrence can be recovered.
|
|
7
|
+
def initialize(label:, occurrence: 0)
|
|
5
8
|
@label = label
|
|
9
|
+
@occurrence = occurrence
|
|
6
10
|
end
|
|
7
11
|
|
|
8
12
|
SCRIPT_COMMON = "Common"
|