sisimai 5.0.0-java → 5.0.2-java

Sign up to get free protection for your applications and to get access to all the features.
Files changed (43) hide show
  1. checksums.yaml +4 -4
  2. data/.github/workflows/rake-test.yml +55 -0
  3. data/ChangeLog.md +40 -1
  4. data/README-JA.md +223 -111
  5. data/README.md +54 -31
  6. data/lib/sisimai/fact.rb +46 -8
  7. data/lib/sisimai/lhost/amazonses.rb +0 -1
  8. data/lib/sisimai/lhost/amazonworkmail.rb +0 -1
  9. data/lib/sisimai/lhost/aol.rb +0 -1
  10. data/lib/sisimai/lhost/bigfoot.rb +0 -1
  11. data/lib/sisimai/lhost/domino.rb +0 -1
  12. data/lib/sisimai/lhost/exchange2007.rb +1 -1
  13. data/lib/sisimai/lhost/exim.rb +7 -16
  14. data/lib/sisimai/lhost/facebook.rb +0 -1
  15. data/lib/sisimai/lhost/googlegroups.rb +2 -1
  16. data/lib/sisimai/lhost/gsuite.rb +0 -1
  17. data/lib/sisimai/lhost/mailru.rb +8 -17
  18. data/lib/sisimai/lhost/messagelabs.rb +0 -1
  19. data/lib/sisimai/lhost/mfilter.rb +1 -1
  20. data/lib/sisimai/lhost/mxlogic.rb +8 -18
  21. data/lib/sisimai/lhost/office365.rb +1 -1
  22. data/lib/sisimai/lhost/outlook.rb +0 -1
  23. data/lib/sisimai/lhost/postfix.rb +0 -1
  24. data/lib/sisimai/lhost/receivingses.rb +0 -1
  25. data/lib/sisimai/lhost/sendgrid.rb +1 -3
  26. data/lib/sisimai/lhost/sendmail.rb +0 -1
  27. data/lib/sisimai/lhost/yandex.rb +0 -1
  28. data/lib/sisimai/message.rb +13 -4
  29. data/lib/sisimai/reason/authfailure.rb +1 -0
  30. data/lib/sisimai/reason/blocked.rb +2 -0
  31. data/lib/sisimai/reason/expired.rb +1 -0
  32. data/lib/sisimai/reason/mailboxfull.rb +1 -0
  33. data/lib/sisimai/reason/securityerror.rb +1 -0
  34. data/lib/sisimai/reason/spamdetected.rb +1 -0
  35. data/lib/sisimai/reason/suspend.rb +1 -0
  36. data/lib/sisimai/rfc5322.rb +120 -64
  37. data/lib/sisimai/rhost/google.rb +320 -59
  38. data/lib/sisimai/rhost/mimecast.rb +9 -2
  39. data/lib/sisimai/smtp/status.rb +3 -0
  40. data/lib/sisimai/version.rb +1 -1
  41. data/lib/sisimai.rb +12 -11
  42. data/set-of-emails/maildir/bsd/lhost-sendmail-60.eml +85 -0
  43. metadata +9 -7
data/README.md CHANGED
@@ -1,7 +1,7 @@
1
1
  ![](https://libsisimai.org/static/images/logo/sisimai-x01.png)
2
2
  [![License](https://img.shields.io/badge/license-BSD%202--Clause-orange.svg)](https://github.com/sisimai/rb-sisimai/blob/master/LICENSE)
3
3
  [![Coverage Status](https://img.shields.io/coveralls/sisimai/rb-sisimai.svg)](https://coveralls.io/r/sisimai/rb-sisimai)
4
- [![Ruby](https://img.shields.io/badge/ruby-v2.4.0--v2.7.0-red.svg)](https://www.ruby-lang.org/)
4
+ [![Ruby](https://img.shields.io/badge/ruby-v2.4.0--v3.3.0-red.svg)](https://www.ruby-lang.org/)
5
5
  [![Gem Version](https://badge.fury.io/rb/sisimai.svg)](https://badge.fury.io/rb/sisimai)
6
6
 
7
7
  > [!IMPORTANT]
@@ -15,10 +15,10 @@
15
15
  > Sisimai 5 requires Ruby 2.4 or later. Check the version of Ruby in your system before installing/upgrading
16
16
  > by `ruby -v` command.
17
17
 
18
- > [!CAUTION]
19
- > [Sisimai 5](https://github.com/sisimai/rb-sisimai/releases/tag/v5.0.0) has not been uploaded to
20
- > [RubyGems.org](https://rubygems.org/gems/sisimai) yet as of February 2nd. It will be available on
21
- > RubyGems.org within a few months, but until then, please clone it from this repository.
18
+ > [!NOTE]
19
+ > Sisimai is a Ruby Gem or Perl module, but it can be used in any environment where JSON can be read,
20
+ > such as PHP, Python, Go, and Rust. By obtaining the analysis results, it is very useful for understanding
21
+ > the bounce occurrence status.
22
22
 
23
23
  - [**README-JA(日本語)**](README-JA.md)
24
24
  - [What is Sisimai](#what-is-sisimai)
@@ -57,7 +57,7 @@ delivery failure, such as the reason for the bounce and the recipient email addr
57
57
  data. It is also possible to output in JSON format. The Ruby version of Sisimai is ported from the
58
58
  Perl version of Sisimai at [github.com/sisimai/p5-sisimai](https://github.com/sisimai/p5-sisimai/).
59
59
 
60
- ![](https://libsisimai.org/static/images/figure/sisimai-overview-1.png)
60
+ ![](https://libsisimai.org/static/images/figure/sisimai-overview-2.png)
61
61
 
62
62
  The key features of Sisimai
63
63
  ---------------------------------------------------------------------------------------------------
@@ -99,26 +99,20 @@ System requirements
99
99
  More details about system requirements are available at
100
100
  [Sisimai | Getting Started](https://libsisimai.org/en/start/) page.
101
101
 
102
-
103
102
  * [Ruby 2.4.0 or later](http://www.ruby-lang.org/)
104
103
  * [__oj | The fastest JSON parser and object serializer__](https://rubygems.org/gems/oj)
105
- * Also works on [JRuby 9.0.4.0 or later](http://jruby.org)
104
+ * Also works on [JRuby 9.2 or later](http://jruby.org)~~
106
105
  * [__jrjackson | A mostly native JRuby wrapper for the java jackson json processor jar__](https://rubygems.org/gems/jrjackson)
107
106
 
108
107
  Install
109
108
  ---------------------------------------------------------------------------------------------------
110
109
  ### From RubyGems
111
- > [!CAUTION]
112
- > [Sisimai 5](https://github.com/sisimai/p5-sisimai/releases/tag/v5.0.0) has not been uploaded to
113
- > [RubyGems.org](https://rubygems.org/gems/sisimai) yet as of February 2nd. It will be available on
114
- > RubyGems.org within a few months, but until then, please clone it from this repository.
115
-
116
110
  ```shell
117
111
  $ sudo gem install sisimai
118
- Fetching: sisimai-4.25.16.gem (100%)
119
- Successfully installed sisimai-4.25.16
120
- Parsing documentation for sisimai-4.25.16
121
- Installing ri documentation for sisimai-4.25.16
112
+ Fetching: sisimai-5.0.2.gem (100%)
113
+ Successfully installed sisimai-5.0.2
114
+ Parsing documentation for sisimai-5.0.2
115
+ Installing ri documentation for sisimai-5.0.2
122
116
  Done installing documentation for sisimai after 6 seconds
123
117
  1 gem installed
124
118
  ```
@@ -146,13 +140,13 @@ if [ -d "/usr/local/jr" ]; then \
146
140
  ...
147
141
  3 gems installed
148
142
  /opt/local/bin/rake install
149
- sisimai 5.0.0 built to pkg/sisimai-5.0.0.gem.
150
- sisimai (5.0.0) installed.
143
+ sisimai 5.0.2 built to pkg/sisimai-5.0.2.gem.
144
+ sisimai (5.0.2) installed.
151
145
  if [ -d "/usr/local/jr" ]; then \
152
146
  PATH="/usr/local/jr/bin:$PATH" /usr/local/jr/bin/rake install; \
153
147
  fi
154
- sisimai 5.0.0 built to pkg/sisimai-5.0.0-java.gem.
155
- sisimai (5.0.0) installed.
148
+ sisimai 5.0.2 built to pkg/sisimai-5.0.2-java.gem.
149
+ sisimai (5.0.2) installed.
156
150
  ```
157
151
 
158
152
  Usage
@@ -220,7 +214,7 @@ puts Sisimai.dump('/path/to/mbox', delivered: true, vacation: true)
220
214
  Callback feature
221
215
  ---------------------------------------------------------------------------------------------------
222
216
  `:c___` (`c` and three `_`s, looks like a fishhook) argument of `Sisimai.rise` and `Sisimai.dump`
223
- is an Array and is a parameter to receive Proc objects for callback feature. The first element of
217
+ is an Array and is a parameter to receive `Proc` objects for callback feature. The first element of
224
218
  `:c___` argument is called at `Sisimai::Message.sift` for dealing email headers and entire message
225
219
  body. The second element of `:c___` argument is called at the end of each email file parsing. The
226
220
  result generated by the callback method is accessible via `Sisimai::Fact.catch`.
@@ -260,9 +254,9 @@ code = lambda do |args|
260
254
  kind = args['kind'] # (String) Sisimai::Mail.kind
261
255
  mail = args['mail'] # (String) Entire email message
262
256
  path = args['path'] # (String) Sisimai::Mail.path
263
- sisi = args['sisi'] # (Array) List of Sisimai::Fact
257
+ fact = args['fact'] # (Array) List of Sisimai::Fact
264
258
 
265
- sisi.each do |e|
259
+ fact.each do |e|
266
260
  # Store custom information in the "catch" accessor
267
261
  e.catch ||= {}
268
262
  e.catch['size'] = mail.size
@@ -275,7 +269,7 @@ code = lambda do |args|
275
269
  e.catch['parsedat'] = Time.new.localtime.to_s
276
270
 
277
271
  # Save the original email with an additional "X-Sisimai-Parsed:" header to a different PATH.
278
- a = sprintf("X-Sisimai-Parsed: %d", sisi.size)
272
+ a = sprintf("X-Sisimai-Parsed: %d", fact.size)
279
273
  p = sprintf("/path/to/another/directory/sisimai-%s.eml", e.token)
280
274
  v = mail.sub(/^(From:.+?)$/, '\1' + "\n" + a)
281
275
  f = File.open(p, 'w:UTF-8')
@@ -310,7 +304,35 @@ Output example
310
304
  ![](https://libsisimai.org/static/images/demo/sisimai-5-cli-dump-p01.gif)
311
305
 
312
306
  ```json
313
- [{"destination":"google.example.com","lhost":"gmail-smtp-in.l.google.com","hardbounce":0,"reason":"authfailure","catch":null,"addresser":"michitsuna@example.jp","alias":"nekochan@example.co.jp","smtpagent":"Postfix","smtpcommand":"DATA","senderdomain":"example.jp","listid":"","action":"failed","feedbacktype":"","messageid":"hwK7pzjzJtz0RF9Y@relay3.example.com","origin":"./gmail-5.7.26.eml","recipient":"kijitora@google.example.com","rhost":"gmail-smtp-in.l.google.com","subject":"Nyaan","timezoneoffset":"+0900","replycode":550,"token":"84656774898baa90660be3e12fe0526e108d4473","diagnostictype":"SMTP","timestamp":1650119685,"diagnosticcode":"host gmail-smtp-in.l.google.com[64.233.187.27] said: This mail has been blocked because the sender is unauthenticated. Gmail requires all senders to authenticate with either SPF or DKIM. Authentication results: DKIM = did not pass SPF [relay3.example.com] with ip: [192.0.2.22] = did not pass For instructions on setting up authentication, go to https://support.google.com/mail/answer/81126#authentication c2-202200202020202020222222cat.127 - gsmtp (in reply to end of DATA command)","deliverystatus":"5.7.26"}]
307
+ [
308
+ {
309
+ "destination": "google.example.com",
310
+ "lhost": "gmail-smtp-in.l.google.com",
311
+ "hardbounce": 0,
312
+ "reason": "authfailure",
313
+ "catch": null,
314
+ "addresser": "michitsuna@example.jp",
315
+ "alias": "nekochan@example.co.jp",
316
+ "smtpagent": "Postfix",
317
+ "smtpcommand": "DATA",
318
+ "senderdomain": "example.jp",
319
+ "listid": "",
320
+ "action": "failed",
321
+ "feedbacktype": "",
322
+ "messageid": "hwK7pzjzJtz0RF9Y@relay3.example.com",
323
+ "origin": "./gmail-5.7.26.eml",
324
+ "recipient": "kijitora@google.example.com",
325
+ "rhost": "gmail-smtp-in.l.google.com",
326
+ "subject": "Nyaan",
327
+ "timezoneoffset": "+0900",
328
+ "replycode": 550,
329
+ "token": "84656774898baa90660be3e12fe0526e108d4473",
330
+ "diagnostictype": "SMTP",
331
+ "timestamp": 1650119685,
332
+ "diagnosticcode": "host gmail-smtp-in.l.google.com[64.233.187.27] said: This mail has been blocked because the sender is unauthenticated. Gmail requires all senders to authenticate with either SPF or DKIM. Authentication results: DKIM = did not pass SPF [relay3.example.com] with ip: [192.0.2.22] = did not pass For instructions on setting up authentication, go to https://support.google.com/mail/answer/81126#authentication c2-202200202020202020222222cat.127 - gsmtp (in reply to end of DATA command)",
333
+ "deliverystatus": "5.7.26"
334
+ }
335
+ ]
314
336
  ```
315
337
 
316
338
  Differences between Sisimai 4 and Sisimai 5
@@ -325,20 +347,21 @@ Beginning with v5.0.0, Sisimai requires **Ruby 2.4.0 or later.**
325
347
 
326
348
  | Features | Sisimai 4 | Sisimai 5 |
327
349
  |------------------------------------------------------|--------------------|---------------------|
328
- | System requirements (CRuby) | 2.1 - | **2.4** - 3.3.0 |
329
- | System requirements (JRuby) | 9.0.4.0 - | 9.0.4.0 - |
350
+ | System requirements (CRuby) | 2.1 - 3.3.0 | **2.4** or later |
351
+ | System requirements (JRuby) | 9.0.4.0 - 9.1.17.0 | **9.2** or later |
330
352
  | Callback feature for the original email file | N/A | Available[^3] |
331
353
  | The number of MTA/ESP modules | 68 | 70 |
332
354
  | The number of detectable bounce reasons | 29 | 34 |
333
355
  | Dependencies (Except Ruby Standard Gems) | 1 gem | 1 gem |
334
- | Source lines of code | 10,300 lines | 11,300 lines |
335
- | The number of tests in spec/,test/ directory | 311,000 tests | 336,000 tests |
356
+ | Source lines of code | 10,300 lines | 11,370 lines |
357
+ | Test frameworks | rspec | minitest |
358
+ | The number of tests in spec/ or test/ directory | 311,000 tests | 338,000 tests |
336
359
  | The number of bounce emails decoded/sec (CRuby)[^4] | 231 emails | 305 emails |
337
360
  | License | 2 Clause BSD | 2 Caluse BSD |
338
361
  | Commercial support | Available | Available |
339
362
 
340
363
  [^3]: The 2nd argument of `:c___` parameter at `Sisimai.rise` method
341
- [^4]: macOS Monterey/1.6GHz Dual-Core Intel Core i5/16GB-RAM/Ruby 2.6.4p104
364
+ [^4]: macOS Monterey/1.6GHz Dual-Core Intel Core i5/16GB-RAM/Ruby 3.3.0
342
365
 
343
366
 
344
367
  Decoding Method
data/lib/sisimai/fact.rb CHANGED
@@ -46,6 +46,18 @@ module Sisimai
46
46
  RFC822Head = Sisimai::RFC5322.HEADERFIELDS(:all)
47
47
  ActionList = { delayed: 1, delivered: 1, expanded: 1, failed: 1, relayed: 1 };
48
48
 
49
+ if RUBY_PLATFORM.start_with?('java')
50
+ # [WORKAROUND] #159 #267 JRuby seems to fail and throws exception at strptime(), but this
51
+ # issue might be fixed in a future version of JRuby.
52
+ # https://gist.github.com/hiroyuki-sato/6ef40245874d4c847a95ef99886e4fa7
53
+ # https://github.com/sisimai/rb-sisimai/issues/267#issuecomment-1976642884
54
+ # https://github.com/jruby/jruby/issues/8139
55
+ # https://github.com/sisimai/rb-sisimai/issues/267
56
+ TimeModule = ::DateTime
57
+ else
58
+ TimeModule = Sisimai::Time
59
+ end
60
+
49
61
  # Constructor of Sisimai::Fact
50
62
  # @param [Hash] argvs Including each parameter
51
63
  # @return [Sisimai::Fact] Structured email data
@@ -188,7 +200,7 @@ module Sisimai
188
200
 
189
201
  begin
190
202
  # Convert from the date string to an object then calculate time zone offset.
191
- t = Sisimai::Time.strptime(datestring, '%a, %d %b %Y %T')
203
+ t = TimeModule.strptime(datestring, '%a, %d %b %Y %T')
192
204
  p['timestamp'] = (t.to_time.to_i - zoneoffset) || nil
193
205
  rescue
194
206
  warn ' ***warning: Failed to strptime ' << datestring.to_s
@@ -196,12 +208,12 @@ module Sisimai
196
208
  next unless p['timestamp']
197
209
 
198
210
  # OTHER_TEXT_HEADERS:
199
- recvheader = mesg1['header']['received'] || []
200
- unless recvheader.empty?
201
- # Get localhost and remote host name from Received header.
202
- %w[lhost rhost].each { |a| e[a] ||= '' }
203
- e['lhost'] = Sisimai::RFC5322.received(recvheader[0]).shift if e['lhost'].empty?
204
- e['rhost'] = Sisimai::RFC5322.received(recvheader[-1]).pop if e['rhost'].empty?
211
+ rr = mesg1['header']['received'] || []
212
+ unless rr.empty?
213
+ # Get a localhost and a remote host name from Received header.
214
+ p['rhost'] = Sisimai::RFC5322.received(rr[-1])[1] if p['rhost'].empty?
215
+ p['lhost'] = '' if p['rhost'] == p['lhost']
216
+ p['lhost'] = Sisimai::RFC5322.received(rr[ 0])[0] if p['lhost'].empty?
205
217
  end
206
218
 
207
219
  # Remove square brackets and curly brackets from the host variable
@@ -339,9 +351,35 @@ module Sisimai
339
351
  o['catch'] = p['catch'] || nil
340
352
  o['hardbounce'] = p['hardbounce']
341
353
  o['replycode'] = Sisimai::SMTP::Reply.find(p['diagnosticcode']).to_s if o['replycode'].empty?
342
- o['timestamp'] = Sisimai::Time.parse(::Time.at(p['timestamp']).to_s)
354
+ o['timestamp'] = TimeModule.parse(::Time.at(p['timestamp']).to_s)
343
355
  o['timezoneoffset'] = p['timezoneoffset'] || '+0000'
344
356
 
357
+ # ALIAS
358
+ while true do
359
+ # Look up the Envelope-To address from the Received: header in the original message
360
+ # when the recipient address is same with the value of o['alias'].
361
+ break if o['alias'].empty?
362
+ break if o['recipient'].address != o['alias']
363
+ break unless rfc822data.has_key?('received')
364
+ break if rfc822data['received'].empty?
365
+
366
+ rfc822data['received'].reverse.each do |er|
367
+ # Search for the string " for " from the Received: header
368
+ next unless er.include?(' for ')
369
+
370
+ af = Sisimai::RFC5322.received(er)
371
+ next if af.empty?
372
+ next if af[5].empty?
373
+ next unless Sisimai::Address.is_emailaddress(af[5])
374
+ next if o['recipient'].address == af[5]
375
+
376
+ o['alias'] = af[5]
377
+ break
378
+ end
379
+ break
380
+ end
381
+ o['alias'] = '' if o['alias'] == o['recipient'].address
382
+
345
383
  # REASON: Decide the reason of email bounce
346
384
  if o['reason'].empty? || RetryIndex[o['reason']]
347
385
  # The value of "reason" is empty or is needed to check with other values again
data/lib/sisimai/lhost/amazonses.rb CHANGED
@@ -305,7 +305,6 @@ module Sisimai::Lhost
305
305
 
306
306
  dscontents.each do |e|
307
307
  # Set default values if each value is empty.
308
- e['lhost'] ||= permessage['rhost']
309
308
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
310
309
 
311
310
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'].to_s.tr("\n", ' '))
data/lib/sisimai/lhost/amazonworkmail.rb CHANGED
@@ -98,7 +98,6 @@ module Sisimai::Lhost
98
98
 
99
99
  dscontents.each do |e|
100
100
  # Set default values if each value is empty.
101
- e['lhost'] ||= permessage['rhost']
102
101
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
103
102
 
104
103
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'])
data/lib/sisimai/lhost/aol.rb CHANGED
@@ -100,7 +100,6 @@ module Sisimai::Lhost
100
100
 
101
101
  dscontents.each do |e|
102
102
  # Set default values if each value is empty.
103
- e['lhost'] ||= permessage['rhost']
104
103
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
105
104
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'].tr("\n", ' '))
106
105
 
data/lib/sisimai/lhost/bigfoot.rb CHANGED
@@ -108,7 +108,6 @@ module Sisimai::Lhost
108
108
 
109
109
  dscontents.each do |e|
110
110
  # Set default values if each value is empty.
111
- e['lhost'] ||= permessage['rhost']
112
111
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
113
112
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'])
114
113
  e['command'] = thecommand || ''
data/lib/sisimai/lhost/domino.rb CHANGED
@@ -123,7 +123,6 @@ module Sisimai::Lhost
123
123
  dscontents.each do |e|
124
124
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'])
125
125
  e['recipient'] = Sisimai::Address.s3s4(e['recipient'])
126
- e['lhost'] ||= permessage['rhost']
127
126
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
128
127
 
129
128
  MessagesOf.each_key do |r|
data/lib/sisimai/lhost/exchange2007.rb CHANGED
@@ -147,7 +147,7 @@ module Sisimai::Lhost
147
147
  next unless p > 0
148
148
 
149
149
  # #550 5.1.1 RESOLVER.ADR.RecipNotFound; not found ##
150
- f = e['diagnosis'][p + 1, e['diagnosis'].index(';') - p -1]
150
+ f = e['diagnosis'][p + 1, e['diagnosis'].index(';') - p - 1]
151
151
  NDRSubject.each_key do |r|
152
152
  # Try to match with error subject strings
153
153
  next unless f == r
data/lib/sisimai/lhost/exim.rb CHANGED
@@ -147,7 +147,6 @@ module Sisimai::Lhost
147
147
  readcursor = 0 # (Integer) Points the current cursor position
148
148
  nextcursor = 0
149
149
  recipients = 0 # (Integer) The number of 'Final-Recipient' header
150
- localhost0 = '' # (String) Local MTA
151
150
  boundary00 = '' # (String) Boundary string
152
151
  v = nil
153
152
 
@@ -342,18 +341,13 @@ module Sisimai::Lhost
342
341
  end
343
342
  return nil unless recipients > 0
344
343
 
345
- unless mhead['received'].empty?
346
- # Get the name of local MTA
347
- # Received: from marutamachi.example.org (c192128.example.net [192.0.2.128])
348
- p1 = mhead['received'][-1].index('from ') || -1
349
- p2 = mhead['received'][-1].index(' ', p1 + 5) || -1
350
- localhost0 = mhead['received'][-1][p1 + 5, p2 - p1 - 5] if p1 > -1 && p2 > -1
351
- end
344
+ # Get the name of the local MTA
345
+ # Received: from marutamachi.example.org (c192128.example.net [192.0.2.128])
346
+ receivedby = mhead['received'] || []
347
+ recvdtoken = Sisimai::RFC5322.received(receivedby[-1])
352
348
 
353
349
  dscontents.each do |e|
354
- # Set default values if each value is empty.
355
- e['lhost'] ||= localhost0
356
-
350
+ # Check the error message, the rhost, the lhost, and the smtp command.
357
351
  unless e['diagnosis']
358
352
  # Empty Diagnostic-Code: or error message
359
353
  unless boundary00.empty?
@@ -405,12 +399,9 @@ module Sisimai::Lhost
405
399
  p1 = e['diagnosis'].index('host ') || -1
406
400
  p2 = e['diagnosis'].index(' ', p1 + 5) || -1
407
401
  e['rhost'] = e['diagnosis'][p1 + 5, p2 - p1 - 5] if p1 > -1
408
-
409
- unless e['rhost']
410
- # Get localhost and remote host name from Received header.
411
- e['rhost'] = Sisimai::RFC5322.received(mhead['received'][-1]).pop unless mhead['received'].empty?
412
- end
402
+ e['rhost'] ||= recvdtoken[1]
413
403
  end
404
+ e['lhost'] ||= recvdtoken[0]
414
405
 
415
406
  unless e['command']
416
407
  # Get the SMTP command name for the session
data/lib/sisimai/lhost/facebook.rb CHANGED
@@ -146,7 +146,6 @@ module Sisimai::Lhost
146
146
  return nil unless recipients > 0
147
147
 
148
148
  dscontents.each do |e|
149
- e['lhost'] ||= permessage['lhost']
150
149
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'])
151
150
 
152
151
  p0 = e['diagnosis'].index('-') || -1
data/lib/sisimai/lhost/googlegroups.rb CHANGED
@@ -48,9 +48,10 @@ module Sisimai::Lhost
48
48
  # * You may need to join the group before receiving permission to post.
49
49
  # * This group may not be open to posting.
50
50
  entiremesg = emailparts[0].split(/\n\n/, 5).slice(0, 4).join(' ').tr("\n", ' ');
51
+ receivedby = mhead['received'] || []
51
52
  recordwide['diagnosis'] = Sisimai::String.sweep(entiremesg)
52
53
  recordwide['reason'] = emailparts[0].scan(/^[ ]?[*][ ]?/).size == 4 ? 'rejected' : 'onhold'
53
- recordwide['rhost'] = Sisimai::RFC5322.received(mhead['received'][0]).shift
54
+ recordwide['rhost'] = Sisimai::RFC5322.received(receivedby[0])[1]
54
55
 
55
56
  mhead['x-failed-recipients'].split(',').each do |e|
56
57
  # X-Failed-Recipients: neko@example.jp, nyaan@example.org, ...
data/lib/sisimai/lhost/gsuite.rb CHANGED
@@ -135,7 +135,6 @@ module Sisimai::Lhost
135
135
 
136
136
  dscontents.each do |e|
137
137
  # Set default values if each value is empty.
138
- e['lhost'] ||= permessage['rhost']
139
138
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
140
139
 
141
140
  if anotherset['diagnosis']
data/lib/sisimai/lhost/mailru.rb CHANGED
@@ -69,7 +69,6 @@ module Sisimai::Lhost
69
69
  bodyslices = emailparts[0].split("\n")
70
70
  readcursor = 0 # (Integer) Points the current cursor position
71
71
  recipients = 0 # (Integer) The number of 'Final-Recipient' header
72
- localhost0 = '' # (String) Local MTA
73
72
  v = nil
74
73
 
75
74
  while e = bodyslices.shift do
@@ -145,18 +144,13 @@ module Sisimai::Lhost
145
144
  end
146
145
  return nil unless recipients > 0
147
146
 
148
- unless mhead['received'].empty?
149
- # Get the name of local MTA
150
- # Received: from marutamachi.example.org (c192128.example.net [192.0.2.128])
151
- p1 = mhead['received'][-1].index('from ') || -1
152
- p2 = mhead['received'][-1].index(' ', p1 + 5) || -1
153
- localhost0 = mhead['received'][-1][p1 + 5, p2 - p1 - 5] if p1 > -1
154
- end
147
+ # Get the name of the local MTA
148
+ # Received: from marutamachi.example.org (c192128.example.net [192.0.2.128])
149
+ receivedby = mhead['received'] || []
150
+ recvdtoken = Sisimai::RFC5322.received(receivedby[-1])
155
151
 
156
152
  dscontents.each do |e|
157
- # Set default values if each value is empty.
158
- e['lhost'] ||= localhost0
159
-
153
+ # Check the error message, the rhost, the lhost, and the smtp command.
160
154
  unless e['alterrors'].to_s.empty?
161
155
  # Copy alternative error message
162
156
  e['diagnosis'] ||= e['alterrors']
@@ -175,13 +169,10 @@ module Sisimai::Lhost
175
169
  # host neko.example.jp [192.0.2.222]: 550 5.1.1 <kijitora@example.jp>... User Unknown
176
170
  p1 = e['diagnosis'].index('host ') || -1
177
171
  p2 = e['diagnosis'].index(' ', p1 + 5) || -1
178
- e['rhost'] = e['diagnosis'][p1 + 5, p2 - p1 - 5] if p1 > -1
179
-
180
- unless e['rhost']
181
- # Get localhost and remote host name from Received header.
182
- e['rhost'] = Sisimai::RFC5322.received(mhead['received'][-1]).pop unless mhead['received'].empty?
183
- end
172
+ e['rhost'] = e['diagnosis'][p1 + 5, p2 - p1 - 5] if p1 > -1
173
+ e['rhost'] ||= recvdtoken[1]
184
174
  end
175
+ e['lhost'] ||= recvdtoken[0]
185
176
 
186
177
  unless e['command']
187
178
  # Get the SMTP command name for the session
data/lib/sisimai/lhost/messagelabs.rb CHANGED
@@ -98,7 +98,6 @@ module Sisimai::Lhost
98
98
 
99
99
  dscontents.each do |e|
100
100
  # Set default values if each value is empty.
101
- e['lhost'] ||= permessage['rhost']
102
101
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
103
102
  e['command'] = commandset.shift || ''
104
103
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'])
data/lib/sisimai/lhost/mfilter.rb CHANGED
@@ -104,7 +104,7 @@ module Sisimai::Lhost
104
104
  rhosts = Sisimai::RFC5322.received(rheads[-1])
105
105
 
106
106
  e['lhost'] ||= Sisimai::RFC5322.received(rheads[0]).shift
107
- while ee = rhosts.shift do
107
+ [rhosts[0], rhosts[1]].each do |ee|
108
108
  # Avoid "... by m-FILTER"
109
109
  next unless ee.include?('.')
110
110
  e['rhost'] = ee
data/lib/sisimai/lhost/mxlogic.rb CHANGED
@@ -89,7 +89,6 @@ module Sisimai::Lhost
89
89
  bodyslices = emailparts[0].split("\n")
90
90
  readcursor = 0 # (Integer) Points the current cursor position
91
91
  recipients = 0 # (Integer) The number of 'Final-Recipient' header
92
- localhost0 = '' # (String) Local MTA
93
92
  v = nil
94
93
 
95
94
  while e = bodyslices.shift do
@@ -135,19 +134,14 @@ module Sisimai::Lhost
135
134
  end
136
135
  return nil unless recipients > 0
137
136
 
138
- unless mhead['received'].empty?
139
- # Get the name of local MTA
140
- p1 = mhead['received'][-1].downcase.index('from ')
141
- p2 = mhead['received'][-1].index(' ', p1 + 5)
142
-
143
- if (p1 + 1) * (p2 + 1) > 0
144
- # Received: from marutamachi.example.org (c192128.example.net [192.0.2.128])
145
- localhost0 = mhead['received'][-1][p1 + 5, p2 - p1 - 5]
146
- end
147
- end
137
+ # Get the name of the local MTA
138
+ # Received: from marutamachi.example.org (c192128.example.net [192.0.2.128])
139
+ receivedby = mhead['received'] || []
140
+ recvdtoken = Sisimai::RFC5322.received(receivedby[-1])
148
141
 
149
142
  dscontents.each do |e|
150
- e['lhost'] = localhost0
143
+ # Check the error message, the rhost, the lhost, and the smtp command.
144
+ e['lhost'] = recvdtoken[0]
151
145
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'].gsub(/[-]{2}.*\z/, ''))
152
146
 
153
147
  unless e['rhost']
@@ -156,12 +150,8 @@ module Sisimai::Lhost
156
150
  p2 = e['diagnosis'].index(' ', p1 + 5)
157
151
 
158
152
  # host neko.example.jp [192.0.2.222]: 550 5.1.1 <kijitora@example.jp>... User Unknown
159
- e['rhost'] = e['diagnosis'][p1 + 5, p2 - p1 - 5] if p1 > -1
160
-
161
- unless e['rhost']
162
- # Get localhost and remote host name from Received header.
163
- e['rhost'] = Sisimai::RFC5322.received(mhead['received'][-1]).pop unless mhead['received'].empty?
164
- end
153
+ e['rhost'] = e['diagnosis'][p1 + 5, p2 - p1 - 5] if p1 > -1
154
+ e['rhost'] ||= recvdtoken[1]
165
155
  end
166
156
 
167
157
  unless e['command']
data/lib/sisimai/lhost/office365.rb CHANGED
@@ -70,7 +70,7 @@ module Sisimai::Lhost
70
70
  %r/\A5[.]7[.]1[23]\z/ => 'rejected',
71
71
  %r/\A5[.]7[.]124\z/ => 'rejected',
72
72
  %r/\A5[.]7[.]13[3-6]\z/ => 'rejected',
73
- %r/\A5[.]7[.]23\z/ => 'blocked',
73
+ %r/\A5[.]7[.]23\z/ => 'authfailure',
74
74
  %r/\A5[.]7[.]25\z/ => 'networkerror',
75
75
  %r/\A5[.]7[.]50[1-3]\z/ => 'spamdetected',
76
76
  %r/\A5[.]7[.]50[4-5]\z/ => 'filtered',
data/lib/sisimai/lhost/outlook.rb CHANGED
@@ -97,7 +97,6 @@ module Sisimai::Lhost
97
97
 
98
98
  dscontents.each do |e|
99
99
  # Set default values if each value is empty.
100
- e['lhost'] ||= permessage['rhost']
101
100
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
102
101
 
103
102
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis']) || ''
data/lib/sisimai/lhost/postfix.rb CHANGED
@@ -227,7 +227,6 @@ module Sisimai::Lhost
227
227
 
228
228
  dscontents.each do |e|
229
229
  # Set default values if each value is empty.
230
- e['lhost'] ||= permessage['rhost']
231
230
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
232
231
 
233
232
  if anotherset['diagnosis']
data/lib/sisimai/lhost/receivingses.rb CHANGED
@@ -96,7 +96,6 @@ module Sisimai::Lhost
96
96
 
97
97
  dscontents.each do |e|
98
98
  # Set default values if each value is empty.
99
- e['lhost'] ||= permessage['rhost']
100
99
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
101
100
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'].tr("\n", ' '))
102
101
 
data/lib/sisimai/lhost/sendgrid.rb CHANGED
@@ -124,6 +124,7 @@ module Sisimai::Lhost
124
124
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'])
125
125
  e['replycode'] = Sisimai::SMTP::Reply.find(e['diagnosis']) || ''
126
126
  e['status'] = e['replycode'][0, 1] + '.0.0' if e['replycode'].size == 3
127
+ e['command'] = thecommand
127
128
 
128
129
  if e['status'] == '5.0.0' || e['status'] == '4.0.0'
129
130
  # Get the value of D.S.N. from the error message or the value of Diagnostic-Code header.
@@ -138,9 +139,6 @@ module Sisimai::Lhost
138
139
  e['status'] = Sisimai::SMTP::Status.code('expired') || e['status']
139
140
  end
140
141
  end
141
-
142
- e['lhost'] ||= permessage['rhost']
143
- e['command'] = thecommand
144
142
  end
145
143
 
146
144
  return { 'ds' => dscontents, 'rfc822' => emailparts[1] }
data/lib/sisimai/lhost/sendmail.rb CHANGED
@@ -168,7 +168,6 @@ module Sisimai::Lhost
168
168
  dscontents.each do |e|
169
169
  # Set default values if each value is empty.
170
170
  e['diagnosis'] ||= ''
171
- e['lhost'] ||= permessage['rhost']
172
171
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
173
172
 
174
173
  if anotherset['diagnosis']
data/lib/sisimai/lhost/yandex.rb CHANGED
@@ -103,7 +103,6 @@ module Sisimai::Lhost
103
103
 
104
104
  dscontents.each do |e|
105
105
  # Set default values if each value is empty.
106
- e['lhost'] ||= permessage['rhost']
107
106
  permessage.each_key { |a| e[a] ||= permessage[a] || '' }
108
107
 
109
108
  e['diagnosis'] = Sisimai::String.sweep(e['diagnosis'].tr("\n", ' '))
data/lib/sisimai/message.rb CHANGED
@@ -198,17 +198,26 @@ module Sisimai
198
198
  # Select and convert all the headers in $argv0. The following regular expression is based on
199
199
  # https://gist.github.com/xtetsuji/b080e1f5551d17242f6415aba8a00239
200
200
  headermaps = { 'subject' => '' }
201
- recvheader = []
201
+ receivedby = []
202
202
  argv0.scan(/^([\w-]+):[ ]*(.*?)\n(?![\s\t])/m) { |e| headermaps[e[0].downcase] = e[1] }
203
203
  headermaps.delete('received')
204
204
  headermaps.each_key { |e| headermaps[e].gsub!(/\n[\s\t]+/, ' ') }
205
205
 
206
206
  if argv0.include?('Received:')
207
207
  # Capture values of each Received: header
208
- recvheader = argv0.scan(/^Received:[ ]*(.*?)\n(?![\s\t])/m).flatten
209
- recvheader.each { |e| e.gsub!(/\n[\s\t]+/, ' ') }
208
+ re = argv0.scan(/^Received:[ ]*(.*?)\n(?![\s\t])/m).flatten
209
+ re.each do |e|
210
+ # 1. Exclude the Received header including "(qmail ** invoked from network)".
211
+ # 2. Convert all consecutive spaces and line breaks into a single space character.
212
+ next if e.include?(' invoked by uid')
213
+ next if e.include?(' invoked from network')
214
+
215
+ e.gsub!(/\n[\s\t]+/, ' ')
216
+ e.squeeze!("\n\t ")
217
+ receivedby << e
218
+ end
210
219
  end
211
- headermaps['received'] = recvheader
220
+ headermaps['received'] = receivedby
212
221
 
213
222
  return headermaps unless argv1
214
223
  return headermaps if headermaps['subject'].empty?
data/lib/sisimai/reason/authfailure.rb CHANGED
@@ -20,6 +20,7 @@ module Sisimai
20
20
  'dmarc policy',
21
21
  'please inspect your spf settings',
22
22
  'sender policy framework (spf) fail',
23
+ 'sender policy framework violation',
23
24
  'spf (sender policy framework) domain authentication fail',
24
25
  'spf check: fail',
25
26
  ].freeze