sisimai 5.2.0 → 5.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1361ff9bfe8b3766d3ac2a38719c055d91a1ea6ec36574675bfbf46029687470
-  data.tar.gz: 37bb5f43705adcbdedc7ccc2db6a049f405d5769aa8529aa6bdb49f0262b9070
+  metadata.gz: bab63d91aaae7cafcd3eb9fb7d5f10335abdf48ea110aa653e9b1cab0826295b
+  data.tar.gz: a0538d447059d4faf79d3d70de91fc131c1c8718fce4623ea0d1e6d11f4d76a8
 SHA512:
-  metadata.gz: 786482ef8faa20a09f14660bcd294d5ffe504abfe7b1ba1bb1ee1e5dccab023436058c4e1c91345bc5dd0ec721359b6e6278105841b8a049845b59694f4ea7f4
-  data.tar.gz: 78d813269ac427deb8af80dc75f9bf42476bed3ee54cbb1aeda6c132e53d41e6a2fabb98f066b9cb0ae5d77d0a2f42053b8cc1dce893244eb8b9dd9f474af9e7
+  metadata.gz: 8570d1381042de41fd967b5d63731d23b5d5dae98de06d72c8b67e34a47eb8f4d32bc792407eff2a1c336dc90b656999742dddedcb427f8abd3347b9184f7089
+  data.tar.gz: fe629b00d60cfffa2a8f3b37c86f9ae6796fb5baec6bf9383c36be996ffed2d672a5a546b4e279dd53a22b4e91411ba39981e1dea21d231fa07e547e96615456

data/ChangeLog.md

@@ -3,6 +3,16 @@ RELEASE NOTES for Ruby version of Sisimai
 - releases: "https://github.com/sisimai/rb-sisimai/releases"
 - download: "https://rubygems.org/gems/sisimai"
 
+v5.2.1
+---------------------------------------------------------------------------------------------------
+- release: "Wed, 12 Mar 2025 06:50:25 +0900 (JST)"
+- version: "5.2.1"
+- changes:
+  - Implement `Sisimai::Rhost::Cloudflare` #336 #337
+  - #333 NTT DOCOMO (Major Japanese mobile carrier) no longer returns a bounce mail due to domain
+    rejection or similar email settings, but instead were being delivered to the spam folder after
+    March 13th. #334
+
 v5.2.0
 ---------------------------------------------------------------------------------------------------
 - release: "5.2.0"

data/README-JA.md

@@ -65,14 +65,14 @@ Sisimai(シシマイ)は複雑で多種多様なバウンスメールを解析
 The key features of Sisimai
 ---------------------------------------------------------------------------------------------------
 * __バウンスメールを構造化したデータに変換__
-  * 以下24項目の情報を含むデータ構造[^2]
+  * 以下26項目の情報を含むデータ構造[^2]
     * __基本的情報__: `timestamp`, `origin`
     * __発信者情報__: `addresser`, `senderdomain`, 
     * __受信者情報__: `recipient`, `destination`, `alias`
-    * __配信の情報__: `action`, `replycode`,`action`, `replycode`, `deliverystatus`
+    * __配信の情報__: `action`, `replycode`, `deliverystatus`, `command`
     * __エラー情報__: `reason`, `diagnosticcode`, `diagnostictype`, `feedbacktype`, `hardbounce`
     * __メール情報__: `subject`, `messageid`, `listid`,
-    * __その他情報__: `smtpagent`, `timezoneoffset`, `lhost`, `rhost`, `token`, `catch`
+    * __その他情報__: `decodedby`, `timezoneoffset`, `lhost`, `rhost`, `token`, `catch`
   * __出力可能な形式__
     * Ruby (Hash, Array)
     * JSON 
@@ -319,8 +319,8 @@ Output example
     "catch": null,
     "addresser": "michitsuna@example.jp",
     "alias": "nekochan@example.co.jp",
-    "smtpagent": "Postfix",
-    "smtpcommand": "DATA",
+    "decodedby": "Postfix",
+    "command": "DATA",
     "senderdomain": "example.jp",
     "listid": "",
     "action": "failed",
@@ -362,7 +362,7 @@ Sisimai 5.0.0から**Ruby 2.4以上**が必要になります。
 | ソースコードの行数                                   | 10,800 行          | 9,860 行            |
 | テストフレームワーク                                 | rspec              | minitest            |
 | テスト件数(spec/またはtest/ディレクトリ)             | 311,000 件         | 410,000 件          |
-| 1秒間に解析できるバウンスメール数[^4]                | 290 通             | 305 通              |
+| 1秒間に解析できるバウンスメール数[^4]                | 290 通             | 360 通              |
 | ライセンス                                           | 2条項BSD           | 2条項BSD            |
 | 開発会社による商用サポート                           | 提供中             | 提供中              |
 

data/README.md

@@ -67,14 +67,14 @@ Perl version of Sisimai at [github.com/sisimai/p5-sisimai](https://github.com/si
 The key features of Sisimai
 ---------------------------------------------------------------------------------------------------
 * __Decode email bounces to structured data__
-  * Sisimai provides detailed insights into bounce emails by extracting 24 key data points.[^2]
+  * Sisimai provides detailed insights into bounce emails by extracting 26 key data points.[^2]
     * __Essential information__: `timestamp`, `origin`
     * __Sender information__: `addresser`, `senderdomain`, 
     * __Recipient information__: `recipient`, `destination`, `alias`
-    * __Delivery information__: `action`, `replycode`,`action`, `replycode`, `deliverystatus`
-    * __Bounce details__: `reason`, `diagnosticcode`, `diagnostictype`, `feedbacktype`, `hardbounce`
+    * __Delivery information__: `action`, `replycode`, `deliverystatus`, `command`
+    * __Bounce details__: `reason`, `diagnosticcode`, `diagnostictype`, `feedbacktype`, `feedbackid`, `hardbounce`
     * __Message details__: `subject`, `messageid`, `listid`,
-    * __Additional information__: `smtpagent`, `timezoneoffset`, `lhost`, `rhost`, `token`, `catch`
+    * __Additional information__: `decodedby`, `timezoneoffset`, `lhost`, `rhost`, `token`, `catch`
   * Output formats
     * Ruby (Hash, Array)
     * JSON 
@@ -85,9 +85,9 @@ The key features of Sisimai
     * `gem install`
     * `git clone && make`
   * __High Precision of Analysis__
-    * Support [73 MTAs/MDAs/ESPs](https://libsisimai.org/en/engine/)
+    * Support [59 MTAs/MDAs/ESPs](https://libsisimai.org/en/engine/)
     * Support Feedback Loop Message(ARF)
-    * Can detect [34 bounce reasons](https://libsisimai.org/en/reason/)
+    * Can detect [36 bounce reasons](https://libsisimai.org/en/reason/)
 
 [^2]: The callback function allows you to add your own data under the `catch` accessor.
 
@@ -114,10 +114,10 @@ Install
 ### From RubyGems
 ```shell
 $ sudo gem install sisimai
-Fetching: sisimai-5.2.0.gem (100%)
-Successfully installed sisimai-5.2.0
-Parsing documentation for sisimai-5.2.0
-Installing ri documentation for sisimai-5.2.0
+Fetching: sisimai-5.2.1.gem (100%)
+Successfully installed sisimai-5.2.1
+Parsing documentation for sisimai-5.2.1
+Installing ri documentation for sisimai-5.2.1
 Done installing documentation for sisimai after 6 seconds
 1 gem installed
 ```
@@ -145,13 +145,13 @@ if [ -d "/usr/local/jr" ]; then \
 ...
 3 gems installed
 /opt/local/bin/rake install
-sisimai 5.2.0 built to pkg/sisimai-5.2.0.gem.
-sisimai (5.2.0) installed.
+sisimai 5.2.1 built to pkg/sisimai-5.2.1.gem.
+sisimai (5.2.1) installed.
 if [ -d "/usr/local/jr" ]; then \
 		PATH="/usr/local/jr/bin:$PATH" /usr/local/jr/bin/rake install; \
 	fi
-sisimai 5.2.0 built to pkg/sisimai-5.2.0-java.gem.
-sisimai (5.2.0) installed.
+sisimai 5.2.1 built to pkg/sisimai-5.2.1-java.gem.
+sisimai (5.2.1) installed.
 ```
 
 Usage
@@ -318,8 +318,8 @@ Output example
     "catch": null,
     "addresser": "michitsuna@example.jp",
     "alias": "nekochan@example.co.jp",
-    "smtpagent": "Postfix",
-    "smtpcommand": "DATA",
+    "decodedby": "Postfix",
+    "command": "DATA",
     "senderdomain": "example.jp",
     "listid": "",
     "action": "failed",
@@ -355,13 +355,13 @@ Beginning with v5.0.0, Sisimai requires **Ruby 2.4.0 or later.**
 | System requirements (CRuby)                          | 2.1 - 3.3.0        | **2.4** or later    |
 | System requirements (JRuby)                          | 9.0.4.0 - 9.1.17.0 | **9.2** or later    |
 | Callback feature for the original email file         | N/A                | Available[^3]       |
-| The number of MTA/ESP modules                        | 68                 | 58                  |
+| The number of MTA/ESP modules                        | 68                 | 59                  |
 | The number of detectable bounce reasons              | 29                 | 36                  |
 | Dependencies (Except Ruby Standard Gems)             | 1 gem              | 1 gem               |
 | Source lines of code                                 | 10,300 lines       | 9,800 lines         |
 | Test frameworks                                      | rspec              | minitest            |
 | The number of tests in spec/ or test/ directory      | 311,000 tests      | 410,000 tests       | 
-| The number of bounce emails decoded/sec (CRuby)[^4]  | 290 emails         | 305 emails          |
+| The number of bounce emails decoded/sec (CRuby)[^4]  | 290 emails         | 360 emails          |
 | License                                              | 2 Clause BSD       | 2 Caluse BSD        |
 | Commercial support                                   | Available          | Available           |
 

data/lib/sisimai/rhost/cloudflare.rb

@@ -0,0 +1,31 @@
+module Sisimai
+  module Rhost
+    # Sisimai::Rhost detects the bounce reason from the content of Sisimai::Fact object as an argument
+    # of find() method when the value of "rhost" of the object is "*.mx.cloudflare.net".
+    # This class is called only from Sisimai::Fact class.
+    module Cloudflare
+      class << self
+        MessagesOf = {
+          "blocked"     => ["found on one or more DNSBLs"],
+          "systemerror" => ["Upstream error"],
+        }.freeze
+
+        # Detect bounce reason from Cloudflare Email Routing
+        # @param    [Sisimai::Fact] argvs   Decoded email object
+        # @return   [String]                The bounce reason for Cloudflare
+        # @since v5.2.1
+        # @see https://developers.cloudflare.com/email-routing/postmaster/
+        def find(argvs)
+          return "" if argvs["diagnosticcode"].empty?
+          MessagesOf.each_key do |e|
+            # Try to match the error message with message patterns defined in $MessagesOf
+            return e if MessagesOf[e].any? { |a| argvs["diagnosticcode"].include?(a) }
+          end
+          return ""
+        end
+
+      end
+    end
+  end
+end
+

data/lib/sisimai/rhost/nttdocomo.rb

@@ -7,8 +7,9 @@ module Sisimai
       class << self
         MessagesOf = {
           'mailboxfull' => ['552 too much mail data'],
-          'toomanyconn' => ['552 too many recipients'],
           'syntaxerror' => ['503 bad sequence of commands', '504 command parameter not implemented'],
+          'toomanyconn' => ['552 too many recipients'],
+          'userunknown' => ['550 unknown user'],
         }.freeze
 
         # Detect bounce reason from NTT DOCOMO
@@ -51,7 +52,12 @@ module Sisimai
           else
             # The value of "Diagnostic-Code:" field is not empty
             MessagesOf.each_key do |e|
-              # Try to match the error message with message patterns defined in "MessagesOf"
+              # - The key name is a bounce reason name
+              # - https://github.com/sisimai/go-sisimai/issues/64
+              # - After March 12, 2025, if an error message contains "550 Unknown user", the
+              #   bounce reason will be definitively "userunknown". This is because NTT DOCOMO
+              #   no longer rejects emails via SMTP for domain-specific rejection or specified
+              #   reception filters.
               next unless MessagesOf[e].any? { |a| esmtperror.include?(a) }
               reasontext = e
               break

data/lib/sisimai/rhost.rb

@@ -7,6 +7,7 @@ module Sisimai
       RhostClass = {
         "Aol"         => [".mail.aol.com", ".mx.aol.com"],
         "Apple"       => [".mail.icloud.com", ".apple.com", ".me.com"],
+        "Cloudflare"  => [".mx.cloudflare.net"],
         "Cox"         => ["cox.net"],
         "Facebook"    => [".facebook.com"],
         "FrancePTT"   => [".laposte.net", ".orange.fr", ".wanadoo.fr"],

data/lib/sisimai/smtp/status.rb

@@ -735,7 +735,7 @@ module Sisimai
                        else
                          ['5.', '4.', '2.']
                        end
-          esmtperror = ' ' + argv1 + ' '
+          esmtperror = ' ' + argv1 + '   ' # Why 3 space characters? see https://github.com/sisimai/p5-sisimai/issues/574
           lookingfor = []
 
           Sisimai::RFC791.find(esmtperror).each do |e|

data/lib/sisimai/version.rb

@@ -1,4 +1,4 @@
 # Define the version number of Sisimai
 module Sisimai
-  VERSION = '5.2.0'.freeze
+  VERSION = '5.2.1'.freeze
 end

data/set-of-emails/maildir/bsd/rhost-cloudflare-01.eml

@@ -0,0 +1,75 @@
+Return-Path: <>
+Received: from relay.mx.example.jp (unknown [192.0.2.21])
+	by sisimai.mx.example.jp (Postfix) with ESMTPS id m6SrZSzVjhzC3KjV
+	for <neko@mx.example.jp>; Tue,  2 Jul 2024 23:34:45 +0900 (JST)
+Date: Tue,  2 Jul 2024 23:34:45 +0900 (JST)
+From: MAILER-DAEMON@relay.mx.example.jp (Mail Delivery System)
+To: neko@mx.example.jp
+Message-Id: <qQYlj9jhzdzY6WHr@relay.mx.example.jp>
+Subject: Undelivered Mail Returned to Sender
+Auto-Submitted: auto-replied
+MIME-Version: 1.0
+Content-Type: multipart/report; report-type=delivery-status;
+	boundary="yKvJXwrDt6zP1H5n.1316187218/relay.mx.example.jp"
+
+This is a MIME-encapsulated message.
+
+--yKvJXwrDt6zP1H5n.1316187218/relay.mx.example.jp
+Content-Description: Notification
+Content-Type: text/plain; charset=us-ascii
+
+This is the mail system at host relay.mx.example.jp.
+
+I'm sorry to have to inform you that your message could not
+be delivered to one or more recipients. It's attached below.
+
+For further assistance, please send mail to postmaster.
+
+If you do so, please include this problem report. You can
+delete your own text from the attached returned message.
+
+                   The mail system
+
+<kijitora-neko@example.com>: host route3.mx.cloudflare.net[162.159.205.25]
+    said: 421 4.3.0 Upstream error, please check
+    https://developers.cloudflare.com/email-routing/postmaster for possible
+    reasons why. RsrUEMk60942 (in reply to end of DATA command)
+
+--yKvJXwrDt6zP1H5n.1316187218/relay.mx.example.jp
+Content-Description: Delivery report
+Content-Type: message/delivery-status
+
+Reporting-MTA: dns; relay.mx.example.jp
+X-Postfix-Queue-ID: yKvJXwrDt6zP1H5n
+X-Postfix-Sender: rfc822; neko@mx.example.jp
+Arrival-Date: Tue,  2 Jul 2024 23:34:45 +0900 (JST)
+
+Final-Recipient: rfc822; kijitora-neko@example.com
+Original-Recipient: rfc822;kijitora-neko@example.com
+Action: failed
+Status: 4.3.0
+Remote-MTA: dns; route3.mx.cloudflare.net
+Diagnostic-Code: smtp; 421 4.3.0 Upstream error, please check
+    https://developers.cloudflare.com/email-routing/postmaster for possible
+    reasons why. RsrUEMk60942
+
+--yKvJXwrDt6zP1H5n.1316187218/relay.mx.example.jp
+Content-Description: Undelivered Message
+Content-Type: message/rfc822
+
+Return-Path: <neko@mx.example.jp>
+Received: from local-mail-hub-1.example.jp (unknown [192.0.2.253])
+	by relay.mx.example.jp (Postfix) with ESMTP id yKvJXwrDt6zP1H5n
+	for <kijitora-neko@example.com>; Tue,  2 Jul 2024 23:34:45 +0900 (JST)
+From: <neko@mx.example.jp>
+Date: Tue, 02 Jul 2024 23:34:45 +0900
+Message-Id: <QtF9aH-aSDIzG-WP@mx.example.jp>
+To: <kijitora-neko@example.com>
+Subject: Nyaaaaaan
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+
+Nyaan?
+
+--yKvJXwrDt6zP1H5n.1316187218/relay.mx.example.jp--
+

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sisimai
 version: !ruby/object:Gem::Version
-  version: 5.2.0
+  version: 5.2.1
 platform: ruby
 authors:
 - azumakuniyuki
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2025-02-25 00:00:00.000000000 Z
+date: 2025-03-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -194,6 +194,7 @@ files:
 - lib/sisimai/rhost.rb
 - lib/sisimai/rhost/aol.rb
 - lib/sisimai/rhost/apple.rb
+- lib/sisimai/rhost/cloudflare.rb
 - lib/sisimai/rhost/cox.rb
 - lib/sisimai/rhost/facebook.rb
 - lib/sisimai/rhost/franceptt.rb
@@ -761,6 +762,7 @@ files:
 - set-of-emails/maildir/bsd/rhost-apple-02.eml
 - set-of-emails/maildir/bsd/rhost-apple-03.eml
 - set-of-emails/maildir/bsd/rhost-apple-04.eml
+- set-of-emails/maildir/bsd/rhost-cloudflare-01.eml
 - set-of-emails/maildir/bsd/rhost-cox-01.eml
 - set-of-emails/maildir/bsd/rhost-facebook-03.eml
 - set-of-emails/maildir/bsd/rhost-facebook-04.eml