ukdah 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: f5f25b1e78bf1060ab0607178933eb83caad1db9aad4090ec052ddacddc43d97
4
+ data.tar.gz: ddbd1df65e46f8c066cf438c5ddd8bef6c9b2484358efd4eb3bfc4bbfebb58bd
5
+ SHA512:
6
+ metadata.gz: '09787badaeca8cb3a6c5c925615139de645166b6a7588f3ac564691824e5ae6018a2e8df3e1d2f7b170321e25a7f8e47284a607dd443c5b10aae26148e7ee2de'
7
+ data.tar.gz: b1e878a5b9a370843603cabb61dda27288fa25e7516e20276f90401148bbd153a452a9cc817cef5a005f64d95ce6ddc3e34ea4336397d2115afa14b2a9b8d9d7
data/CHANGELOG.md ADDED
@@ -0,0 +1,5 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0 — 2026-09-21
4
+
5
+ - Initial release.
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2026 ydah
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,60 @@
1
+ # Ukdah
2
+
3
+ Ukdah (ι Cancri, from Arabic *ʿuqdah*, “knot”) is a dependency-free Ruby
4
+ parser for RFC 5322 messages and MIME bodies. It is deliberately a parser,
5
+ not a mail transport client: it never connects to IMAP, POP, or SMTP.
6
+
7
+ ## Features
8
+
9
+ - Folded headers, RFC 2047 encoded words, comments, quoted names, and groups
10
+ - RFC 2231 parameters, multipart trees, base64 and quoted-printable bodies
11
+ - Attachment and inline-part discovery, including `cid:` identifiers
12
+ - Tolerant parsing with diagnostics for malformed messages and boundaries
13
+ - mbox splitting with envelope and `Content-Length` support
14
+ - Lightweight `Message-ID`/`References` threading and quote segmentation
15
+ - Charset decoding through an injectable decoder; no runtime dependencies
16
+
17
+ ## Installation
18
+
19
+ ```ruby
20
+ gem "ukdah"
21
+ ```
22
+
23
+ ## Quick start
24
+
25
+ ```ruby
26
+ require "ukdah"
27
+
28
+ message = Ukdah::Message.parse(File.binread("message.eml"))
29
+ puts message.subject
30
+ puts message.from.first.email
31
+ puts message.decoded(message.text_part)
32
+ message.attachments.each do |part|
33
+ File.binwrite(part.filename, part.body)
34
+ end
35
+ ```
36
+
37
+ Applications with a charset detector can inject it without adding a runtime
38
+ dependency:
39
+
40
+ ```ruby
41
+ message.decoded(message.text_part, decoder: ->(bytes, charset) {
42
+ Menkar.decode(bytes, Menkar.detect(bytes, hint: charset))
43
+ })
44
+ ```
45
+
46
+ Malformed input is returned as far as it can be read. Inspect
47
+ `message.diagnostics` when a source needs attention. Ukdah does not sanitize
48
+ HTML; pass HTML parts through a sanitizer before displaying them.
49
+
50
+ ## Development
51
+
52
+ ```sh
53
+ bundle install
54
+ bundle exec rake
55
+ gem build --strict ukdah.gemspec
56
+ ```
57
+
58
+ ## License
59
+
60
+ MIT. See [LICENSE.txt](LICENSE.txt).
data/Rakefile ADDED
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "bundler/gem_tasks"
4
+ require "rspec/core/rake_task"
5
+
6
+ RSpec::Core::RakeTask.new(:spec)
7
+
8
+ task test: :spec
9
+ task default: :test
@@ -0,0 +1,18 @@
1
+ # ADR NNN: Implementation decision title
2
+
3
+ - Status: Proposed
4
+ - Date: YYYY-MM-DD
5
+
6
+ ## Context
7
+
8
+ Describe the concrete implementation question and the compatibility,
9
+ data, runtime, or component constraints.
10
+
11
+ ## Decision
12
+
13
+ Describe the durable boundary or architecture choice.
14
+
15
+ ## Consequences
16
+
17
+ Describe the important positive and negative trade-offs, including what would
18
+ make this decision worth revisiting.
@@ -0,0 +1,20 @@
1
+ # ADR 001: Parsing only, no transport
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-09-21
5
+
6
+ ## Context
7
+
8
+ Ukdah is the mail source's parsing boundary. Adding IMAP, POP, or SMTP would
9
+ turn a deterministic library into a network client with credentials, retry,
10
+ and security policy concerns.
11
+
12
+ ## Decision
13
+
14
+ Ukdah parses RFC 5322, MIME, and mbox bytes only. Network transport and message
15
+ construction remain outside the gem.
16
+
17
+ ## Consequences
18
+
19
+ The runtime stays small, dependency-free, and safe to use on untrusted files.
20
+ Applications must provide transport and persistence separately.
@@ -0,0 +1,20 @@
1
+ # ADR 002: Charset decoding is injectable
2
+
3
+ - Status: Accepted
4
+ - Date: 2026-09-21
5
+
6
+ ## Context
7
+
8
+ Mail charset declarations are frequently wrong. A parser should retain the
9
+ decoded transfer bytes without depending on a particular charset detector.
10
+
11
+ ## Decision
12
+
13
+ `Message#decoded` accepts an optional callable receiving bytes and the declared
14
+ charset. The standard-library fallback handles common Ruby encodings, while an
15
+ application may inject a stronger detector such as menkar.
16
+
17
+ ## Consequences
18
+
19
+ Ukdah has no runtime dependency on the reader application or a charset gem.
20
+ Callers that need detection beyond Ruby's encodings must inject that policy.
@@ -0,0 +1,8 @@
1
+ # Architecture decision records
2
+
3
+ These records document durable implementation boundaries. Commands and test
4
+ fixtures belong in the README and test suite.
5
+
6
+ - [ADR template](000-template.md)
7
+ - [ADR 001: Parsing only](001-parsing-only.md)
8
+ - [ADR 002: Charset decoding is injectable](002-decoder-injection.md)
@@ -0,0 +1,125 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ukdah
4
+ Address = Struct.new(:name, :email, :group, keyword_init: true) do
5
+ class << self
6
+ def parse_list(value)
7
+ return [] if value.nil? || value.to_s.strip.empty?
8
+
9
+ result = []
10
+ split_top_level(value.to_s, ",").each do |token|
11
+ if token =~ /\A\s*(.*?)\s*:\s*(.*?)\s*;\s*\z/m
12
+ group = clean_name(Regexp.last_match(1))
13
+ split_top_level(Regexp.last_match(2), ",").each do |member|
14
+ address = parse_one(member, group)
15
+ result << address if address
16
+ end
17
+ else
18
+ address = parse_one(token, nil)
19
+ result << address if address
20
+ end
21
+ end
22
+ result
23
+ end
24
+
25
+ private
26
+
27
+ def parse_one(token, group)
28
+ value = remove_comments(token.to_s).strip
29
+ return nil if value.empty?
30
+
31
+ if value =~ /\A(.*?)\s*<\s*([^<>]+?)\s*>\s*\z/m
32
+ name = clean_name(Regexp.last_match(1))
33
+ email = Regexp.last_match(2).strip
34
+ else
35
+ name = nil
36
+ email = value.gsub(/[<>]/, "").strip
37
+ end
38
+ return nil if email.empty?
39
+
40
+ Address.new(name: name, email: email, group: group)
41
+ end
42
+
43
+ def clean_name(value)
44
+ name = value.to_s.strip
45
+ if name.start_with?('"') && name.end_with?('"') && name.length >= 2
46
+ name = name[1...-1].gsub(/\\(.)/, '\\1')
47
+ end
48
+ Header.decode(name).strip
49
+ end
50
+
51
+ def remove_comments(value)
52
+ result = +""
53
+ depth = 0
54
+ quoted = false
55
+ escaped = false
56
+ value.each_char do |char|
57
+ if escaped
58
+ result << char if depth.zero?
59
+ escaped = false
60
+ elsif char == "\\" && quoted
61
+ result << char if depth.zero?
62
+ escaped = true
63
+ elsif char == '"' && depth.zero?
64
+ quoted = !quoted
65
+ result << char
66
+ elsif char == "(" && !quoted
67
+ depth += 1
68
+ elsif char == ")" && depth.positive?
69
+ depth -= 1
70
+ elsif depth.zero?
71
+ result << char
72
+ end
73
+ end
74
+ result
75
+ end
76
+
77
+ def split_top_level(value, separator)
78
+ pieces = []
79
+ current = +""
80
+ quote = false
81
+ angle = false
82
+ comment = 0
83
+ group = false
84
+ escaped = false
85
+ value.each_char do |char|
86
+ if escaped
87
+ current << char
88
+ escaped = false
89
+ elsif char == "\\" && (quote || comment.positive?)
90
+ current << char
91
+ escaped = true
92
+ elsif char == '"' && comment.zero?
93
+ quote = !quote
94
+ current << char
95
+ elsif char == "<" && !quote && comment.zero?
96
+ angle = true
97
+ current << char
98
+ elsif char == ">" && !quote && comment.zero?
99
+ angle = false
100
+ current << char
101
+ elsif char == "(" && !quote
102
+ comment += 1
103
+ current << char
104
+ elsif char == ")" && comment.positive? && !quote
105
+ comment -= 1
106
+ current << char
107
+ elsif char == ":" && separator == "," && !quote && !angle && comment.zero?
108
+ group = true
109
+ current << char
110
+ elsif char == ";" && separator == "," && !quote && !angle && comment.zero?
111
+ group = false
112
+ current << char
113
+ elsif char == separator && !quote && !angle && comment.zero? && !group
114
+ pieces << current.strip
115
+ current = +""
116
+ else
117
+ current << char
118
+ end
119
+ end
120
+ pieces << current.strip unless current.strip.empty?
121
+ pieces
122
+ end
123
+ end
124
+ end
125
+ end
@@ -0,0 +1,192 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ukdah
4
+ module Header
5
+ ENCODED_WORD = /=\?([^?\s]+)\?([bBqQ])\?([^?]*)\?=/
6
+ BASE64_ALPHABET = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
7
+
8
+ module_function
9
+
10
+ def parse(section, diagnostics = nil)
11
+ result = Hash.new { |hash, key| hash[key] = [] }
12
+ current = nil
13
+ section.to_s.split(/\r?\n/, -1).each do |line|
14
+ if line.match?(/\A[ \t]/)
15
+ if current
16
+ current[:value] << " " << line.strip
17
+ elsif diagnostics
18
+ diagnostics << "header continuation without a header"
19
+ end
20
+ elsif (match = line.match(/\A([^:]+):(.*)\z/))
21
+ current = { name: match[1].strip.downcase, value: match[2].strip }
22
+ result[current[:name]] << current[:value]
23
+ elsif !line.empty? && diagnostics
24
+ diagnostics << "malformed header ignored: #{line.byteslice(0, 80)}"
25
+ end
26
+ end
27
+ result
28
+ end
29
+
30
+ def decode(value)
31
+ original = value.to_s
32
+ return original unless original.include?("=?")
33
+
34
+ text = value.to_s.dup.force_encoding(Encoding::BINARY)
35
+ loop do
36
+ changed = text.gsub(/(=\?[^?\s]+\?[bBqQ]\?[^?]*\?=)[ \t\r\n]+(?==\?)/, "\\1")
37
+ break if changed == text
38
+
39
+ text = changed
40
+ end
41
+ text.gsub(ENCODED_WORD) do
42
+ decode_word(Regexp.last_match(1), Regexp.last_match(2), Regexp.last_match(3))
43
+ end
44
+ rescue StandardError
45
+ value.to_s
46
+ end
47
+
48
+ def decode_word(charset, mode, payload)
49
+ bytes = if mode.casecmp("b").zero?
50
+ decode_base64(payload)
51
+ else
52
+ payload.tr("_", " ").gsub(/=([0-9A-Fa-f]{2})/) { Regexp.last_match(1).to_i(16).chr }
53
+ end
54
+ Ukdah.decode_bytes(bytes, charset)
55
+ rescue StandardError
56
+ payload
57
+ end
58
+
59
+ def decode_base64(payload)
60
+ buffer = 0
61
+ bits = 0
62
+ output = []
63
+ payload.to_s.each_byte do |byte|
64
+ char = byte.chr
65
+ next if char == "=" || char.match?(/[ \t\r\n]/)
66
+
67
+ value = BASE64_ALPHABET.index(char)
68
+ next if value.nil?
69
+
70
+ buffer = (buffer << 6) | value
71
+ bits += 6
72
+ while bits >= 8
73
+ bits -= 8
74
+ output << ((buffer >> bits) & 0xff)
75
+ end
76
+ end
77
+ output.pack("C*")
78
+ end
79
+
80
+ def split_parameters(value)
81
+ pieces = []
82
+ current = +""
83
+ quoted = false
84
+ escaped = false
85
+ value.to_s.each_char do |char|
86
+ if escaped
87
+ current << char
88
+ escaped = false
89
+ elsif char == "\\" && quoted
90
+ current << char
91
+ escaped = true
92
+ elsif char == '"'
93
+ quoted = !quoted
94
+ current << char
95
+ elsif char == ";" && !quoted
96
+ pieces << current.strip
97
+ current = +""
98
+ else
99
+ current << char
100
+ end
101
+ end
102
+ pieces << current.strip unless current.empty?
103
+ pieces
104
+ end
105
+
106
+ def unquote(value)
107
+ text = value.to_s.strip
108
+ text = text[1...-1] if text.start_with?('"') && text.end_with?('"') && text.length >= 2
109
+ text.gsub(/\\(.)/, '\\1')
110
+ end
111
+
112
+ def content_type(value, diagnostics = nil)
113
+ parse_parameterized(value, diagnostics)
114
+ end
115
+
116
+ def disposition(value, diagnostics = nil)
117
+ parse_parameterized(value, diagnostics)
118
+ end
119
+
120
+ def parse_parameterized(value, diagnostics = nil)
121
+ pieces = split_parameters(value)
122
+ main = pieces.shift.to_s.strip.downcase
123
+ raw = {}
124
+ pieces.each do |piece|
125
+ name, parameter = piece.split("=", 2)
126
+ if parameter.nil?
127
+ diagnostics << "malformed parameter ignored: #{piece}" if diagnostics
128
+ next
129
+ end
130
+ raw[name.to_s.strip.downcase] = unquote(parameter)
131
+ end
132
+ [main, decode_parameters(raw, diagnostics)]
133
+ end
134
+
135
+ def decode_parameters(raw, diagnostics = nil)
136
+ parameters = {}
137
+ grouped = Hash.new { |hash, key| hash[key] = [] }
138
+ raw.each do |name, value|
139
+ if (match = name.match(/\A(.+)\*(\d+)(\*)?\z/))
140
+ grouped[match[1]] << [match[2].to_i, !match[3].nil?, value]
141
+ elsif name.end_with?("*")
142
+ parameters[name[0...-1]] = decode_extended(value, diagnostics)
143
+ else
144
+ parameters[name] = value
145
+ end
146
+ end
147
+ grouped.each do |name, segments|
148
+ segments.sort_by!(&:first)
149
+ expected = 0
150
+ charset = nil
151
+ encoded = +""
152
+ segments.each do |index, is_encoded, value|
153
+ if index != expected
154
+ diagnostics << "missing RFC 2231 parameter segment: #{name}*#{expected}" if diagnostics
155
+ expected = index
156
+ end
157
+ expected += 1
158
+ segment = value
159
+ if index.zero? && is_encoded && segment =~ /\A([^']*)'[^']*'(.*)\z/m
160
+ charset = Regexp.last_match(1)
161
+ segment = Regexp.last_match(2)
162
+ end
163
+ encoded << (is_encoded ? percent_decode(segment) : segment)
164
+ end
165
+ parameters[name] = Ukdah.decode_bytes(encoded, charset)
166
+ end
167
+ parameters
168
+ end
169
+
170
+ def decode_extended(value, diagnostics = nil)
171
+ text = value.to_s
172
+ if text =~ /\A([^']*)'[^']*'(.*)\z/m
173
+ Ukdah.decode_bytes(percent_decode(Regexp.last_match(2)), Regexp.last_match(1))
174
+ else
175
+ diagnostics << "malformed RFC 2231 parameter" if diagnostics
176
+ percent_decode(text)
177
+ end
178
+ end
179
+
180
+ def percent_decode(value)
181
+ value.to_s.gsub(/%([0-9A-Fa-f]{2})/) { Regexp.last_match(1).to_i(16).chr }
182
+ end
183
+
184
+ def message_ids(value)
185
+ value.to_s.scan(/<([^>]+)>/).flatten.tap do |ids|
186
+ if ids.empty?
187
+ ids.concat(value.to_s.split(/[ \t,]+/).map { |id| id.gsub(/[<>]/, "") }.reject(&:empty?))
188
+ end
189
+ end
190
+ end
191
+ end
192
+ end
data/lib/ukdah/mbox.rb ADDED
@@ -0,0 +1,72 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ukdah
4
+ module Mbox
5
+ module_function
6
+
7
+ def each(io)
8
+ return enum_for(:each, io) unless block_given?
9
+
10
+ data = io.respond_to?(:read) ? io.read.to_s : io.to_s
11
+ starts = envelope_offsets(data)
12
+ if starts.empty?
13
+ yield Message.parse(data) unless data.empty?
14
+ return
15
+ end
16
+
17
+ index = 0
18
+ while index < starts.length
19
+ start = starts[index]
20
+ message_start = line_end(data, start)
21
+ next_start = starts[index + 1] || data.bytesize
22
+ explicit_end = content_length_end(data, message_start, next_start)
23
+ finish = explicit_end || next_start
24
+ finish = next_start if finish > data.bytesize
25
+ yield Message.parse(data.byteslice(message_start, finish - message_start).to_s)
26
+ if explicit_end
27
+ index += 1
28
+ index += 1 while index < starts.length && starts[index] < finish
29
+ else
30
+ index += 1
31
+ end
32
+ end
33
+ end
34
+
35
+ def envelope_offsets(data)
36
+ result = []
37
+ offset = 0
38
+ data.to_s.each_line do |line|
39
+ if line.match?(/\AFrom \S+ (?:Mon|Tue|Wed|Thu|Fri|Sat|Sun)\s/)
40
+ result << offset
41
+ end
42
+ offset += line.bytesize
43
+ end
44
+ result
45
+ end
46
+
47
+ def line_end(data, offset)
48
+ newline = data.index("\n", offset)
49
+ newline ? newline + 1 : data.bytesize
50
+ end
51
+
52
+ def content_length_end(data, start, limit)
53
+ lf_end = data.index("\n\n", start)
54
+ crlf_end = data.index("\r\n\r\n", start)
55
+ header_end = if lf_end.nil?
56
+ crlf_end
57
+ elsif crlf_end.nil?
58
+ lf_end
59
+ else
60
+ [lf_end, crlf_end].min
61
+ end
62
+ return nil unless header_end && header_end < limit
63
+
64
+ header = data.byteslice(start, header_end - start)
65
+ match = header.match(/(?:\A|\r?\n)Content-Length\s*:\s*(\d+)/i)
66
+ return nil unless match
67
+
68
+ separator_length = data.byteslice(header_end, 4) == "\r\n\r\n" ? 4 : 2
69
+ header_end + separator_length + match[1].to_i
70
+ end
71
+ end
72
+ end
@@ -0,0 +1,234 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "time"
4
+
5
+ module Ukdah
6
+ class Message
7
+ attr_reader :headers, :parts, :raw, :diagnostics
8
+
9
+ def initialize(headers:, parts:, raw:, diagnostics: [])
10
+ @headers = headers
11
+ @parts = parts
12
+ @raw = raw
13
+ @diagnostics = diagnostics
14
+ end
15
+
16
+ def self.parse(bytes)
17
+ Parser.new(bytes).parse
18
+ end
19
+
20
+ def header(name)
21
+ headers_all(name).first
22
+ end
23
+
24
+ def headers_all(name)
25
+ Array(@headers[name.to_s.downcase]).map { |value| Header.decode(value) }
26
+ end
27
+
28
+ def from
29
+ addresses("from")
30
+ end
31
+
32
+ def to
33
+ addresses("to")
34
+ end
35
+
36
+ def cc
37
+ addresses("cc")
38
+ end
39
+
40
+ def bcc
41
+ addresses("bcc")
42
+ end
43
+
44
+ def reply_to
45
+ addresses("reply-to")
46
+ end
47
+
48
+ def subject
49
+ header("subject")
50
+ end
51
+
52
+ def date
53
+ value = header("date")
54
+ return nil if value.nil? || value.empty?
55
+
56
+ Time.rfc2822(value)
57
+ rescue ArgumentError
58
+ begin
59
+ Time.parse(value)
60
+ rescue ArgumentError
61
+ nil
62
+ end
63
+ end
64
+
65
+ def message_id
66
+ Header.message_ids(header("message-id")).first
67
+ end
68
+
69
+ def in_reply_to
70
+ Header.message_ids(header("in-reply-to"))
71
+ end
72
+
73
+ def references
74
+ Header.message_ids(header("references"))
75
+ end
76
+
77
+ def text_part
78
+ find_part { |part| part.content_type == "text/plain" }
79
+ end
80
+
81
+ def html_part
82
+ find_part { |part| part.content_type == "text/html" }
83
+ end
84
+
85
+ def attachments
86
+ leaves.select do |part|
87
+ part.disposition == "attachment" || !part.filename.to_s.empty?
88
+ end
89
+ end
90
+
91
+ def inline_parts
92
+ leaves.each_with_object({}) do |part, result|
93
+ next if part.content_id.to_s.empty?
94
+ next unless part.disposition == "inline" || part.disposition.to_s.empty?
95
+
96
+ id = normalize_content_id(part.content_id)
97
+ result[id] = part
98
+ result["<#{id}>"] = part
99
+ end
100
+ end
101
+
102
+ def decoded(part, decoder: nil)
103
+ return "" unless part
104
+
105
+ bytes = part.body.to_s.dup.force_encoding(Encoding::BINARY)
106
+ charset = part.parameters["charset"]
107
+ if decoder
108
+ result = decoder.arity == 1 ? decoder.call(bytes) : decoder.call(bytes, charset)
109
+ return result.to_s
110
+ end
111
+ Ukdah.decode_bytes(bytes, charset)
112
+ end
113
+
114
+ def leaves
115
+ parts.flat_map { |part| part.each_leaf.to_a }
116
+ end
117
+
118
+ private
119
+
120
+ def addresses(name)
121
+ headers_all(name).flat_map { |value| Address.parse_list(value) }
122
+ end
123
+
124
+ def find_part
125
+ leaves.find { |part| yield part }
126
+ end
127
+
128
+ def normalize_content_id(value)
129
+ value.to_s.strip.gsub(/\A<|>\z/, "").downcase
130
+ end
131
+
132
+ class Parser
133
+ def initialize(bytes)
134
+ @raw = bytes.to_s.dup.force_encoding(Encoding::BINARY)
135
+ @diagnostics = []
136
+ end
137
+
138
+ def parse
139
+ headers, body = entity(@raw)
140
+ root = part_from(headers, body)
141
+ parts = root.multipart? ? root.parts : [root]
142
+ Message.new(headers: headers, parts: parts, raw: @raw, diagnostics: @diagnostics)
143
+ rescue StandardError => error
144
+ @diagnostics << "parser recovered from #{error.class}: #{error.message}"
145
+ Message.new(headers: {}, parts: [], raw: @raw, diagnostics: @diagnostics)
146
+ end
147
+
148
+ private
149
+
150
+ def entity(bytes)
151
+ match = bytes.match(/\r?\n\r?\n/)
152
+ return [Header.parse(bytes, @diagnostics), ""] unless match
153
+
154
+ [Header.parse(bytes.byteslice(0, match.begin(0)), @diagnostics), bytes.byteslice(match.end(0)..-1).to_s]
155
+ end
156
+
157
+ def part_from(headers, body)
158
+ type, parameters = Header.content_type(first(headers, "content-type"), @diagnostics)
159
+ type = "text/plain" if type.nil? || type.empty?
160
+ disposition, disposition_parameters = Header.disposition(first(headers, "content-disposition"), @diagnostics)
161
+ disposition = nil if disposition.nil? || disposition.empty?
162
+ filename = disposition_parameters["filename"] || parameters["name"]
163
+ content_id = Header.decode(first(headers, "content-id").to_s).strip
164
+ encoding = first(headers, "content-transfer-encoding").to_s.downcase
165
+
166
+ if type.start_with?("multipart/")
167
+ children = multipart_children(body, parameters["boundary"])
168
+ return Part.new(content_type: type, parameters: parameters, disposition: disposition,
169
+ filename: filename, content_id: content_id, encoding: encoding,
170
+ body: children.empty? ? transfer_decode(body, encoding) : nil, parts: children)
171
+ end
172
+
173
+ Part.new(content_type: type, parameters: parameters, disposition: disposition,
174
+ filename: filename, content_id: content_id, encoding: encoding,
175
+ body: transfer_decode(body, encoding), parts: [])
176
+ end
177
+
178
+ def multipart_children(body, boundary)
179
+ unless boundary && !boundary.empty?
180
+ @diagnostics << "multipart has no boundary"
181
+ return []
182
+ end
183
+
184
+ marker = "--#{boundary}"
185
+ children = []
186
+ current = nil
187
+ closed = false
188
+ body.to_s.split(/\r?\n/, -1).each do |line|
189
+ if line =~ /\A#{Regexp.escape(marker)}(--)?[ \t]*\z/
190
+ if current
191
+ child = current.join("\n")
192
+ children << part_from(*entity(child)) unless child.strip.empty?
193
+ end
194
+ if Regexp.last_match(1)
195
+ current = nil
196
+ closed = true
197
+ else
198
+ current = []
199
+ end
200
+ elsif current
201
+ current << line
202
+ end
203
+ end
204
+ if current && !current.empty?
205
+ child = current.join("\n")
206
+ children << part_from(*entity(child)) unless child.strip.empty?
207
+ @diagnostics << "multipart boundary terminator missing"
208
+ elsif !closed
209
+ @diagnostics << "multipart boundary not found"
210
+ end
211
+ children
212
+ end
213
+
214
+ def transfer_decode(body, encoding)
215
+ value = body.to_s.dup.force_encoding(Encoding::BINARY)
216
+ case encoding.to_s.downcase
217
+ when "base64"
218
+ Header.decode_base64(value)
219
+ when "quoted-printable"
220
+ value.gsub(/=\r?\n/, "").gsub(/=([0-9A-Fa-f]{2})/) { Regexp.last_match(1).to_i(16).chr }
221
+ else
222
+ value
223
+ end
224
+ rescue StandardError => error
225
+ @diagnostics << "transfer decoding failed for #{encoding}: #{error.message}"
226
+ value
227
+ end
228
+
229
+ def first(headers, name)
230
+ Array(headers[name]).first.to_s
231
+ end
232
+ end
233
+ end
234
+ end
data/lib/ukdah/part.rb ADDED
@@ -0,0 +1,32 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ukdah
4
+ Part = Struct.new(:content_type, :parameters, :disposition, :filename,
5
+ :content_id, :encoding, :body, :parts, keyword_init: true) do
6
+ def multipart?
7
+ content_type.to_s.start_with?("multipart/")
8
+ end
9
+
10
+ def leaf?
11
+ !multipart? && Array(parts).empty?
12
+ end
13
+
14
+ def each_leaf(&block)
15
+ return enum_for(:each_leaf) unless block
16
+
17
+ if Array(parts).empty?
18
+ yield self
19
+ else
20
+ parts.each { |part| part.each_leaf(&block) }
21
+ end
22
+ end
23
+
24
+ def media_type
25
+ content_type.to_s.split("/", 2).first.to_s
26
+ end
27
+
28
+ def subtype
29
+ content_type.to_s.split("/", 2).last.to_s
30
+ end
31
+ end
32
+ end
@@ -0,0 +1,20 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ukdah
4
+ module Quote
5
+ module_function
6
+
7
+ def segments(text)
8
+ groups = []
9
+ text.to_s.split(/\r?\n/, -1).each do |line|
10
+ depth = line[/\A(?:>\s*)+/].to_s.count(">")
11
+ if groups.last && groups.last[0] == depth
12
+ groups.last[1] << line
13
+ else
14
+ groups << [depth, [line]]
15
+ end
16
+ end
17
+ groups
18
+ end
19
+ end
20
+ end
@@ -0,0 +1,73 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ukdah
4
+ module Thread_
5
+ class Node
6
+ attr_accessor :parent
7
+ attr_reader :message, :children
8
+
9
+ def initialize(message = nil)
10
+ @message = message
11
+ @children = []
12
+ @parent = nil
13
+ end
14
+
15
+ def id
16
+ message && message.message_id
17
+ end
18
+
19
+ def placeholder?
20
+ message.nil?
21
+ end
22
+ end
23
+
24
+ module_function
25
+
26
+ def build(messages)
27
+ nodes = {}
28
+ ordered = []
29
+ Array(messages).each do |message|
30
+ id = normalize(message.message_id)
31
+ next if id.nil? || id.empty? || (nodes[id] && !nodes[id].placeholder?)
32
+
33
+ node = nodes[id] || Node.new
34
+ node.instance_variable_set(:@message, message)
35
+ nodes[id] = node
36
+ ordered << node unless ordered.include?(node)
37
+ end
38
+
39
+ Array(messages).each do |message|
40
+ child = nodes[normalize(message.message_id)]
41
+ next unless child
42
+
43
+ parent = nil
44
+ (Array(message.references) + Array(message.in_reply_to)).each do |reference|
45
+ reference_id = normalize(reference)
46
+ next if reference_id.nil? || reference_id.empty?
47
+
48
+ candidate = nodes[reference_id] ||= Node.new
49
+ ordered << candidate unless ordered.include?(candidate)
50
+ parent = candidate
51
+ end
52
+ next unless parent && parent != child
53
+
54
+ detach(child)
55
+ child.parent = parent
56
+ parent.children << child unless parent.children.include?(child)
57
+ end
58
+
59
+ ordered.select { |node| node.parent.nil? }
60
+ end
61
+
62
+ def normalize(id)
63
+ id.to_s.strip.gsub(/\A<|>\z/, "").downcase unless id.nil?
64
+ end
65
+
66
+ def detach(node)
67
+ return unless node.parent
68
+
69
+ node.parent.children.delete(node)
70
+ node.parent = nil
71
+ end
72
+ end
73
+ end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ukdah
4
+ VERSION = "0.1.0"
5
+ end
data/lib/ukdah.rb ADDED
@@ -0,0 +1,26 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative "ukdah/version"
4
+ require_relative "ukdah/header"
5
+ require_relative "ukdah/address"
6
+ require_relative "ukdah/part"
7
+ require_relative "ukdah/message"
8
+ require_relative "ukdah/mbox"
9
+ require_relative "ukdah/quote"
10
+ require_relative "ukdah/thread"
11
+
12
+ module Ukdah
13
+ class Error < StandardError; end
14
+
15
+ class << self
16
+ def decode_bytes(bytes, charset = nil)
17
+ value = bytes.to_s.dup.force_encoding(Encoding::BINARY)
18
+ return value.encode("UTF-8", invalid: :replace, undef: :replace) if charset.nil? || charset.empty?
19
+
20
+ encoding = Encoding.find(charset.to_s.strip.gsub(/\A["']|["']\z/, ""))
21
+ value.force_encoding(encoding).encode("UTF-8", invalid: :replace, undef: :replace)
22
+ rescue ArgumentError, Encoding::UndefinedConversionError, Encoding::InvalidByteSequenceError
23
+ value.force_encoding(Encoding::UTF_8).encode("UTF-8", invalid: :replace, undef: :replace)
24
+ end
25
+ end
26
+ end
data/sig/ukdah.rbs ADDED
@@ -0,0 +1,4 @@
1
+ module Ukdah
2
+ VERSION: String
3
+ # See the writing guide of rbs: https://github.com/ruby/rbs#guides
4
+ end
metadata ADDED
@@ -0,0 +1,64 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: ukdah
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Yudai Takada
8
+ bindir: exe
9
+ cert_chain: []
10
+ date: 1980-01-02 00:00:00.000000000 Z
11
+ dependencies: []
12
+ description: A tolerant pure-Ruby parser for mail headers, MIME trees, attachments,
13
+ mbox files, and threads.
14
+ email:
15
+ - t.yudai92@gmail.com
16
+ executables: []
17
+ extensions: []
18
+ extra_rdoc_files: []
19
+ files:
20
+ - CHANGELOG.md
21
+ - LICENSE.txt
22
+ - README.md
23
+ - Rakefile
24
+ - docs/adr/000-template.md
25
+ - docs/adr/001-parsing-only.md
26
+ - docs/adr/002-decoder-injection.md
27
+ - docs/adr/README.md
28
+ - lib/ukdah.rb
29
+ - lib/ukdah/address.rb
30
+ - lib/ukdah/header.rb
31
+ - lib/ukdah/mbox.rb
32
+ - lib/ukdah/message.rb
33
+ - lib/ukdah/part.rb
34
+ - lib/ukdah/quote.rb
35
+ - lib/ukdah/thread.rb
36
+ - lib/ukdah/version.rb
37
+ - sig/ukdah.rbs
38
+ homepage: https://github.com/noxdea/ukdah
39
+ licenses:
40
+ - MIT
41
+ metadata:
42
+ allowed_push_host: https://rubygems.org
43
+ homepage_uri: https://github.com/noxdea/ukdah
44
+ source_code_uri: https://github.com/noxdea/ukdah/tree/main
45
+ changelog_uri: https://github.com/noxdea/ukdah/blob/main/CHANGELOG.md
46
+ rubygems_mfa_required: 'true'
47
+ rdoc_options: []
48
+ require_paths:
49
+ - lib
50
+ required_ruby_version: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '3.1'
55
+ required_rubygems_version: !ruby/object:Gem::Requirement
56
+ requirements:
57
+ - - ">="
58
+ - !ruby/object:Gem::Version
59
+ version: '0'
60
+ requirements: []
61
+ rubygems_version: 4.0.16
62
+ specification_version: 4
63
+ summary: Dependency-free RFC 5322 and MIME message parser
64
+ test_files: []