sisimai 4.25.11 → 4.25.12

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sisimai might be problematic. Click here for more details.

checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b96e6f5eb8587091c2902d4bdf83d63fd5d36606d80120428b509afeae37ab4b
4
- data.tar.gz: 074ee81152220a51456a3a4d2236075715b1aa7045ddcace81aa0fd406676f4f
3
+ metadata.gz: d4e15f3cb5442e9fdc4cf00980aa77ab197ac7ba6eaf75e06f2b49edb4e5a2a9
4
+ data.tar.gz: 7105ac6159bd67aec8020a54517f451c616bb5c0332c7804035e96323d39dc02
5
5
  SHA512:
6
- metadata.gz: d3dbf3a7f80e30ef008b925c3f2105d005663a61e94bff2e361f9c9bf5bd80991cc9ca2e7ab522c657012ebe42a7f717de9d96a241c5b5b4e3d16921f8d6ac77
7
- data.tar.gz: 8903044d48d468b905758a0b7c4abd8851d5609bbefb7f46c4f09379529f06751d6f0bec5214a50f782c7ea5902f2df33ea663a8806418584ff900cc13b5e019
6
+ metadata.gz: 4a51b10b7f1cfc9a0565c55fd1b553adde9ea8c2a6edd1f49aeb9b3e3d7b509cfa0bf6740d8d508ab2e74ea9dca84103b1840bdf2fa63a977add33b229ad7232
7
+ data.tar.gz: 321ef7a5c3515e029990df7ac79bb251bf753e57979bf28ca86ef422e2a57a9233b4c3ee44c098ed6ebbaafed7b76c205b1f1e2ffbdf55d5a0c07cece54cda32
data/ChangeLog.md CHANGED
@@ -3,6 +3,29 @@ RELEASE NOTES for Ruby version of Sisimai
3
3
  - releases: "https://github.com/sisimai/rb-sisimai/releases"
4
4
  - download: "https://rubygems.org/gems/sisimai"
5
5
 
6
+ v4.25.12
7
+ --------------------------------------------------------------------------------
8
+ - release: "Mon, 22 Nov 2021 12:22:22 +0900 (JST)"
9
+ - version: "4.25.12"
10
+ - changes:
11
+ - Fix 2 encoding related errors:
12
+ - Fix an error with message `incompatible character encodings:ASCII-8BIT and
13
+ UTF-8` reported at issue #223. Thanks to @chahn
14
+ - Fix an error with message `invalid byte sequence in UTF-8 (ArgumentError)`
15
+ reported at issue #224. Thanks to @chahn
16
+ - Add `rfc3464-41.eml` and `rfc3464-42.eml` provided by @chahn
17
+ - Add a new error code of La Poste at `Sisimai::Rhost::FrancePTT` module and
18
+ a new sample email `rhost-franceptt-12.eml` into `set-of-emails` directory
19
+ imported from sisimai/p5-sisimai#441.
20
+ - Remove all the HTML elements from the value of `diagnosticcode`.
21
+ - Fix serious bugs:
22
+ - Both of the values of `deliverystatus` and `replycode` detected from the
23
+ message body did not use at `Sisimai::Lhost::Exim`.
24
+ - `true` method strictly checks the value of `smtpcommand` at some classes
25
+ in `Sisimai/Reason`. For example, when a detected reason is `spamdetected`
26
+ and `virusdetected` the value of `smtpcommand` should be `DATA` or an SMTP
27
+ command to be sent after `DATA`.
28
+
6
29
  v4.25.11
7
30
  --------------------------------------------------------------------------------
8
31
  - release: "Mon, 22 Feb 2021 21:15:22 +0900 (JST)"
data/README-JA.md CHANGED
@@ -220,8 +220,8 @@ Differences between Ruby version and Perl version
220
220
  | メール解析速度(1000通のメール) | 2.22秒[2] | 1.35秒 |
221
221
  | インストール方法 | gem install | cpanm, cpm |
222
222
  | 依存モジュール数(コアモジュールを除く) | 1モジュール | 2モジュール |
223
- | LOC:ソースコードの行数 | 10300行 | 10500行 |
224
- | テスト件数(spec/,t/,xt/ディレクトリ) | 236000件 | 265000件 |
223
+ | LOC:ソースコードの行数 | 10400行 | 10500行 |
224
+ | テスト件数(spec/,t/,xt/ディレクトリ) | 241000件 | 270000件 |
225
225
  | ライセンス | 二条項BSD | 二条項BSD |
226
226
  | 開発会社によるサポート契約 | 提供中 | 提供中 |
227
227
 
data/README.md CHANGED
@@ -225,8 +225,8 @@ and bounceHammer are available at
225
225
  | The speed of parsing email(1000 emails) | 2.22s[2] | 1.35s |
226
226
  | How to install | gem install | cpanm, cpm |
227
227
  | Dependencies (Except core modules) | 1 module | 2 modules |
228
- | LOC:Source lines of code | 10300 lines | 10500 lines |
229
- | The number of tests(spec/,t/,xt/) directory | 236000 tests | 265000 tests |
228
+ | LOC:Source lines of code | 10400 lines | 10500 lines |
229
+ | The number of tests(spec/,t/,xt/) directory | 241000 tests | 270000 tests |
230
230
  | License | BSD 2-Clause | BSD 2-Clause |
231
231
  | Support Contract provided by Developer | Available | Available |
232
232
 
@@ -283,7 +283,7 @@ Author
283
283
 
284
284
  Copyright
285
285
  ===============================================================================
286
- Copyright (C) 2015-2020 azumakuniyuki, All Rights Reserved.
286
+ Copyright (C) 2015-2021 azumakuniyuki, All Rights Reserved.
287
287
 
288
288
  License
289
289
  ===============================================================================
data/lib/sisimai/data.rb CHANGED
@@ -262,10 +262,16 @@ module Sisimai
262
262
  # 550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to Gmail,
263
263
  # 550-5.7.1 this message has been blocked. Please visit
264
264
  # 550 5.7.1 https://support.google.com/mail/answer/188131 for more information.
265
- p['diagnosticcode'] = Sisimai::String.sweep(p['diagnosticcode'].gsub(re, ' '))
265
+ p['diagnosticcode'] = p['diagnosticcode'].gsub(re, ' ')
266
+ p['diagnosticcode'] = Sisimai::String.sweep(p['diagnosticcode'].sub(%r|<html>.+</html>|i, ''))
266
267
  end
267
268
  end
268
269
 
270
+ if Sisimai::String.is_8bit(p['diagnosticcode'])
271
+ # To avoid incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError
272
+ p['diagnosticcode'] = p['diagnosticcode'].force_encoding('UTF-8').scrub('?')
273
+ end
274
+
269
275
  p['diagnostictype'] ||= 'X-UNIX' if p['reason'] == 'mailererror'
270
276
  p['diagnostictype'] ||= 'SMTP' unless %w[feedback vacation].include?(p['reason'])
271
277
 
@@ -440,8 +440,8 @@ module Sisimai::Lhost
440
440
  # Diagnostic-Code: smtp; 450 TEMPERROR: retry timeout exceeded
441
441
  # The value of "Status:" indicates permanent error but the value
442
442
  # of SMTP reply code in Diagnostic-Code: field is "TEMPERROR"!!!!
443
- sv = Sisimai::SMTP::Status.find(e['diagnosis'])
444
- rv = Sisimai::SMTP::Reply.find(e['diagnosis'])
443
+ sv = e['status'] || Sisimai::SMTP::Status.find(e['diagnosis'])
444
+ rv = e['replycode'] || Sisimai::SMTP::Reply.find(e['diagnosis'])
445
445
  s1 = 0 # First character of Status as integer
446
446
  r1 = 0 # First character of SMTP reply code as integer
447
447
 
data/lib/sisimai/mime.rb CHANGED
@@ -225,7 +225,7 @@ module Sisimai
225
225
 
226
226
  plain = nil
227
227
  if cv = argv1.match(%r|([+/\=0-9A-Za-z\r\n]+)|) then plain = Base64.decode64(cv[1]) end
228
- return plain.force_encoding('UTF-8')
228
+ return plain.scrub('?')
229
229
  end
230
230
 
231
231
  # Get boundary string
@@ -41,7 +41,7 @@ module Sisimai
41
41
  return false
42
42
  end
43
43
 
44
- # Rejected by domain or address filter ?
44
+ # Rejected by a sender domain or a sender address by a filter ?
45
45
  # @param [Sisimai::Data] argvs Object to be detected the reason
46
46
  # @return [True,False] true: is filtered
47
47
  # false: is not filtered
@@ -53,18 +53,18 @@ module Sisimai
53
53
  tempreason = Sisimai::SMTP::Status.name(argvs.deliverystatus) || ''
54
54
  return false if tempreason == 'suspend'
55
55
 
56
- commandtxt = argvs.smtpcommand || ''
57
56
  diagnostic = argvs.diagnosticcode.downcase || ''
58
- alterclass = Sisimai::Reason::UserUnknown
59
-
60
57
  if tempreason == 'filtered'
61
58
  # Delivery status code points "filtered".
62
- return true if alterclass.match(diagnostic) || match(diagnostic)
63
-
64
- elsif commandtxt != 'RCPT' && commandtxt != 'MAIL'
65
- # Check the value of Diagnostic-Code and the last SMTP command
59
+ return true if Sisimai::Reason::UserUnknown.match(diagnostic)
66
60
  return true if match(diagnostic)
67
- return true if alterclass.match(diagnostic)
61
+ else
62
+ # The value of "reason" isn't "filtered" when the value of "smtpcommand" is an SMTP
63
+ # command to be sent before the SMTP DATA command because all the MTAs read the headers
64
+ # and the entire message body after the DATA command.
65
+ return false if %w[CONN EHLO HELO MAIL RCPT].include?(argvs.smtpcommand)
66
+ return true if match(diagnostic)
67
+ return true if Sisimai::Reason::UserUnknown.match(diagnostic)
68
68
  end
69
69
  return false
70
70
  end
@@ -53,13 +53,9 @@ module Sisimai
53
53
  # @see http://www.ietf.org/rfc/rfc2822.txt
54
54
  def true(argvs)
55
55
  r = argvs.reason || ''
56
- if r.size > 0
57
- # Do not overwrite the reason
58
- return false if r.start_with?('securityerror', 'systemerror', 'undefined')
59
- else
60
- # Check the value of Diagnosic-Code: header with patterns
61
- return true if match(argvs.diagnosticcode.downcase)
62
- end
56
+ return false if r.start_with?('securityerror', 'systemerror', 'undefined')
57
+ return false if %w[CONN EHLO HELO].include?(argvs.smtpcommand)
58
+ return true if match(argvs.diagnosticcode.downcase)
63
59
  return false
64
60
  end
65
61
 
@@ -43,9 +43,9 @@ module Sisimai
43
43
  return true if argvs.reason == 'notaccept'
44
44
 
45
45
  # SMTP Reply Code is 554 or 556
46
- return true if [521, 554, 556].index(argvs.replycode.to_i)
47
- return false if argvs.smtpcommand == 'MAIL'
48
- return true if match(argvs.diagnosticcode.downcase)
46
+ return true if [521, 554, 556].index(argvs.replycode.to_i)
47
+ return false if argvs.smtpcommand != 'MAIL'
48
+ return true if match(argvs.diagnosticcode.downcase)
49
49
  return false
50
50
  end
51
51
 
@@ -56,7 +56,7 @@ module Sisimai
56
56
  return false
57
57
  end
58
58
 
59
- # The bounce reason is security error or not
59
+ # The bounce reason is "policyviolation" or not
60
60
  # @param [Sisimai::Data] argvs Object to be detected the reason
61
61
  # @return [True,False] true: is policy violation
62
62
  # false: is not policy violation
@@ -165,7 +165,12 @@ module Sisimai
165
165
  return nil if argvs.deliverystatus.empty?
166
166
  return true if argvs.reason == 'spamdetected'
167
167
  return true if Sisimai::SMTP::Status.name(argvs.deliverystatus).to_s == 'spamdetected'
168
- return true if match(argvs.diagnosticcode.downcase)
168
+
169
+ # The value of "reason" isn't "spamdetected" when the value of "smtpcommand" is an SMTP
170
+ # command to be sent before the SMTP DATA command because all the MTAs read the headers
171
+ # and the entire message body after the DATA command.
172
+ return false if %w[CONN EHLO HELO MAIL RCPT].include?(argvs.smtpcommand)
173
+ return true if match(argvs.diagnosticcode.downcase)
169
174
  return false
170
175
  end
171
176
 
@@ -38,14 +38,20 @@ module Sisimai
38
38
  return false
39
39
  end
40
40
 
41
- # The bounce reason is security error or not
41
+ # The bounce reason is "virusdetected" or not
42
42
  # @param [Sisimai::Data] argvs Object to be detected the reason
43
43
  # @return [True,False] true: virus detected
44
44
  # false: virus was not detected
45
45
  # @since 4.22.0
46
46
  # @see http://www.ietf.org/rfc/rfc2822.txt
47
- def true(_argvs)
48
- return nil
47
+ def true(argvs)
48
+ # The value of "reason" isn't "visusdetected" when the value of "smtpcommand" is an SMTP
49
+ # command to be sent before the SMTP DATA command because all the MTAs read the headers
50
+ # and the entire message body after the DATA command.
51
+ return true if argvs.reason == 'virusdetected'
52
+ return false if %w[CONN EHLO HELO MAIL RCPT].include?(argvs.smtpcommand)
53
+ return true if match(argvs.diagnosticcode.downcase)
54
+ return false
49
55
  end
50
56
 
51
57
  end
@@ -123,6 +123,10 @@ module Sisimai
123
123
  # Message refuse, verification DMARC en echec LPN007_517
124
124
  '517' => 'policyviolation',
125
125
 
126
+ # 554 5.7.1 Client host rejected
127
+ # LPN000_630
128
+ '630' => 'policyviolation',
129
+
126
130
  # 421 mwinf5c77 ME Service refuse. Veuillez essayer plus tard.
127
131
  # Service refused, please try later. OFR_999 [999]
128
132
  '999' => 'blocked',
@@ -1,4 +1,4 @@
1
1
  # Define the version number of Sisimai
2
2
  module Sisimai
3
- VERSION = '4.25.11'.freeze
3
+ VERSION = '4.25.12'.freeze
4
4
  end
@@ -0,0 +1,19 @@
1
+ From MAILER-DAEMON Mon Sep 20 19:29:27 2021
2
+ Return-Path: <>
3
+ X-Original-To: postmaster@peter.otto.org
4
+ Delivered-To: blackhole@localhost
5
+ From: <foo@bar.net>
6
+ To: postmaster@any.com
7
+ Message-ID: <dummy@ott.pet>
8
+ Date: Mon, 20 Sep 2021 21:29:21 +0200 (CEST)
9
+ X-RZG-CLASS-ID: mi
10
+ X-RZG-TRIGGERING-SMTP-ID: qcZaAFivUJ
11
+ MIME-Version: 1.0
12
+ Content-Type: text/plain; charset="UTF-8"
13
+ Content-Transfer-Encoding: quoted-printable
14
+ Subject: kitununulinitoquekegu
15
+ In-Reply-To: <foobar@dummy.org>
16
+ Auto-Submitted: auto-replied
17
+
18
+ zuk=C3=BCnftig
19
+ karl@hanz.de
@@ -0,0 +1,34 @@
1
+ From MAILER-DAEMON Mon Sep 20 19:33:02 2021
2
+ Return-Path: <>
3
+ X-Original-To: postmaster@blackhole.our-host.net
4
+ Delivered-To: blackhole@localhost
5
+ Received: from mx2.itnetwork.net (mx2.itnetwork.net [123.123.123.202])
6
+ by play.thmthlayer.com (Postfix) with ESMTPS id 94598E117B
7
+ for <postmaster@blackhole.our-host.net>; Mon, 20 Sep 2021 19:33:02 +0000 (UTC)
8
+ Received: from 10.9.37.234 ([10.9.34.13]) by mx2.itnetwork.net with ESMTP id rTR3cr9xLoMiR5vX for <postmaster@blackhole.our-host.net>; Mon, 20 Sep 2021 21:33:01 +0200 (CEST)
9
+ Date: Mon, 20 Sep 2021 21:32:59 +0200 (GMT+02:00)
10
+ From: Postmaster@bit-onbreeeck.org
11
+ To: MyName <message@gelaneeiuet.org>
12
+ Subject: foobar
13
+ Mime-Version: 1.0
14
+ Message-ID: <OFB3228DED.DDDCAC8E-ONC1258756.006B648D-C1258756.006B6493@bit-onbreeeck.org>
15
+ Content-Type: multipart/report; report-type=delivery-status; boundary="==IFJRGLKFGIR7891042UHRUHIHD"
16
+
17
+ --==IFJRGLKFGIR7891042UHRUHIHD
18
+ Content-Type: text/plain; charset=ISO-8859-1
19
+ Content-Transfer-Encoding: base64
20
+
21
+ ICBhdWZnZWb8aHJ0DQoNCg==
22
+
23
+ --==IFJRGLKFGIR7891042UHRUHIHD
24
+ Content-Type: message/delivery-status
25
+
26
+ Reporting-MTA: dns;10.9.37.234
27
+
28
+ Final-Recipient: rfc822;jane.doe@some-domain.net
29
+ Action: failed
30
+ Status: 5.0.0
31
+ Diagnostic-Code: The email account that you tried to reach does not exist.
32
+
33
+ --==IFJRGLKFGIR7891042UHRUHIHD--
34
+
@@ -0,0 +1,82 @@
1
+ Return-Path: <>
2
+ X-Original-To: xxxx@xxxx.net
3
+ Delivered-To: xxxx@xxxx.xxxx.net
4
+ Received: from xxxx.xxxx.net (xxxx.xxxx.net [#.#.#.#])
5
+ by xxxx.xxxx.net (Postfix) with ESMTP id D827520D624B
6
+ for <xxxx@xxxx.net>; Fri, 15 Oct 2021 14:42:03 +0200 (CEST)
7
+ Received: by xxxx.xxxx.net (Postfix)
8
+ id C6602268DDF8; Fri, 15 Oct 2021 14:42:00 +0200 (CEST)
9
+ Date: Fri, 15 Oct 2021 14:42:00 +0200 (CEST)
10
+ From: MAILER-DAEMON@xxxx.xxxx.net (Mail Delivery System)
11
+ Subject: Undelivered Mail Returned to Sender
12
+ To: xxxx.xxxx.net
13
+ Auto-Submitted: auto-replied
14
+ MIME-Version: 1.0
15
+ Content-Type: multipart/report; report-type=delivery-status;
16
+ boundary="64BD1268DDF4.xxxx.xxxx.net"
17
+ Content-Transfer-Encoding: 8bit
18
+ Message-Id: <20211015124200.xxxx.xxxx.net>
19
+
20
+ This is a MIME-encapsulated message.
21
+
22
+ --64BD1268DDF4.xxxx.xxxx.net
23
+ Content-Description: Notification
24
+ Content-Type: text/plain; charset=utf-8
25
+ Content-Transfer-Encoding: 8bit
26
+
27
+ This is the mail system at host xxxx.xxxx.net.
28
+
29
+ I'm sorry to have to inform you that your message could not
30
+ be delivered to one or more recipients. It's attached below.
31
+
32
+ For further assistance, please send mail to postmaster.
33
+
34
+ If you do so, please include this problem report. You can
35
+ delete your own text from the attached returned message.
36
+
37
+ The mail system
38
+
39
+ <xxxx@laposte.net>: host smtpz4.laposte.net said: 554
40
+ 5.7.1 <xxxx.xxxx.net>: Client host rejected:
41
+ #216 lpn LPN000_630 (in reply to RCPT TO command)
42
+
43
+ --64BD1268DDF4.xxxx.xxxx.net
44
+ Content-Description: Delivery report
45
+ Content-Type: message/delivery-status
46
+
47
+ Reporting-MTA: dns; xxxx.xxxx.net
48
+ X-Postfix-Queue-ID: 64BD1268DDF4
49
+ X-Postfix-Sender: rfc822; xxxx.xxxx.net
50
+ Arrival-Date: Fri, 15 Oct 2021 14:41:59 +0200 (CEST)
51
+
52
+ Final-Recipient: rfc822; xxxx@laposte.net
53
+ Original-Recipient: rfc822;xxxx@laposte.net
54
+ Action: failed
55
+ Status: 5.7.1
56
+ Remote-MTA: dns; smtpz4.laposte.net
57
+ Diagnostic-Code: smtp; 554 5.7.1 <xxxx.xxxx.net[#.#.#.#]:49309>:
58
+ Client host rejected: #216 lpn LPN000_630
59
+
60
+ --64BD1268DDF4.xxxx.xxxx.net
61
+ Content-Description: Undelivered Message Headers
62
+ Content-Type: text/rfc822-headers
63
+ Content-Transfer-Encoding: 8bit
64
+
65
+ Return-Path: <xxxx@xxxx.net>
66
+ Received: from [#.#.#.#] (unknown [#.#.#.#])
67
+ by xxxx.xxxx.net (Postfix) with ESMTPS id 64BD1268DDF4
68
+ for <xxxx@laposte.net>; Fri, 15 Oct 2021 14:41:59 +0200 (CEST)
69
+ Message-Id: <xxxx@xxxx.net>
70
+ Feedback-Id: 657:1802
71
+ List-Unsubscribe: <mailto:xxxx@lu.xxxx.net>
72
+ Precedence: bulk
73
+ From: "=?UTF-8?Q?Kipli_par_AM?=" <newsletter@xxxx.net>
74
+ To: xxxx@r-v-b.com
75
+ Reply-To: newsletter@xxxx.net
76
+ Content-Type: multipart/alternative; boundary="_----------=_1634301717326132"
77
+ Subject: =?UTF-8?Q?xxxx?=
78
+ Content-Transfer-Encoding: binary
79
+ MIME-Version: 1.0
80
+ Date: Fri, 15 Oct 2021 14:41:57 +0200
81
+
82
+ --64BD1268DDF4.xxxx.xxxx.net--
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sisimai
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.25.11
4
+ version: 4.25.12
5
5
  platform: ruby
6
6
  authors:
7
7
  - azumakuniyuki
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2021-02-22 00:00:00.000000000 Z
11
+ date: 2021-11-22 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -693,6 +693,8 @@ files:
693
693
  - set-of-emails/maildir/bsd/rfc3464-38.eml
694
694
  - set-of-emails/maildir/bsd/rfc3464-39.eml
695
695
  - set-of-emails/maildir/bsd/rfc3464-40.eml
696
+ - set-of-emails/maildir/bsd/rfc3464-41.eml
697
+ - set-of-emails/maildir/bsd/rfc3464-42.eml
696
698
  - set-of-emails/maildir/bsd/rfc3834-01.eml
697
699
  - set-of-emails/maildir/bsd/rfc3834-02.eml
698
700
  - set-of-emails/maildir/bsd/rfc3834-03.eml
@@ -711,6 +713,7 @@ files:
711
713
  - set-of-emails/maildir/bsd/rhost-franceptt-08.eml
712
714
  - set-of-emails/maildir/bsd/rhost-franceptt-10.eml
713
715
  - set-of-emails/maildir/bsd/rhost-franceptt-11.eml
716
+ - set-of-emails/maildir/bsd/rhost-franceptt-12.eml
714
717
  - set-of-emails/maildir/bsd/rhost-godaddy-02.eml
715
718
  - set-of-emails/maildir/bsd/rhost-godaddy-03.eml
716
719
  - set-of-emails/maildir/bsd/rhost-googleapps-01.eml