exception_notification 4.2.1 → 4.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 6de21b765af1465df6205f174cbab7e10b403048
-  data.tar.gz: cade5525679b528aded7b1ad6369e97f40b58947
+  metadata.gz: a5eab5531d309930d48027a778d74f8491a5a995
+  data.tar.gz: 67d0b4176fb7decc436754ee60e7f4a9053d48f8
 SHA512:
-  metadata.gz: fe4b7770171a25a7e131d97c922e5c518531f71add293ce42359eb9f2fc84ea7330f2f935657cbf4dc13fa4a8ba2d3d492397469555d665d12d36c52603a82fd
-  data.tar.gz: 92774d00ec26d97246ca9d4e6334f3aac06f04c5242a555eb3a5bd5addfd8a8485bdd72787b0d304146c50e79a7ff4500aaf2a08aca744be963f3a779221b193
+  metadata.gz: 1f92d01eee01a295d84e28bcaff7618edb3a4a626afe56a447974579115d8daf4d3e6d516c1c5bb449b88f53497effafa479c6a46bb614b9cb1eed04f5510211
+  data.tar.gz: ca3718addeb5deb323a4f73c09d0678534381301a4fc85ac648fef406beafccb844500039c1c805d6f20d1ef897696a6f92f94631a6be25ccbc468899ef69ec8

data/CHANGELOG.rdoc

@@ -1,3 +1,10 @@
+== 4.2.2
+
+* enhancements
+  * Error groupiong (by @Martin91)
+  * Additional fields for Slack support (by @schurig)
+  * Enterprise HipChat support (by @seanhuber)
+
 == 4.2.1
 
 * enhancements

data/CONTRIBUTING.md

@@ -30,8 +30,9 @@ should submit a Pull Request!
   bundle
   cd test/dummy
   bundle
-  bundle exec rake db:reset
-  bundle exec rake db:test:prepare
+  bundle exec rake db:reset db:test:prepare
+  cd ../..
+  bundle exec rake test
   ```
 * Create a topic branch from where you want to base your work.
 * Add a test for your change. Only refactoring and documentation changes

data/README.md

@@ -82,7 +82,7 @@ Options -> sections" below.
 
 ## Notifiers
 
-ExceptionNotification relies on notifiers to deliver notifications when errors occur in your applications. By default, six notifiers are available:
+ExceptionNotification relies on notifiers to deliver notifications when errors occur in your applications. By default, 7 notifiers are available:
 
 * [Campfire notifier](#campfire-notifier)
 * [Email notifier](#email-notifier)
@@ -279,6 +279,12 @@ If enabled, include the exception message in the subject. Use `:verbose_subject
 
 If enabled, remove numbers from subject so they thread as a single one. Use `:normalize_subject => true` to enable it.
 
+##### include_controller_and_action_names_in_subject
+
+*Boolean, default: true*
+
+If enabled, include the controller and action names in the subject. Use `:include_controller_and_action_names_in_subject => false` to exclude them.
+
 ##### email_format
 
 *Symbol, default: :text*
@@ -393,6 +399,12 @@ Color of the message. Default : 'red'.
 
 Message will appear from this nickname. Default : 'Exception'.
 
+##### server_url
+
+*String, optional*
+
+Custom Server URL for self-hosted, Enterprise HipChat Server
+
 For all options & possible values see [Hipchat API](https://www.hipchat.com/docs/api/method/rooms/message).
 
 ### IRC notifier
@@ -590,6 +602,12 @@ more information. Default: 'incoming-webhook'
 
 Contains additional payload for a message (e.g avatar, attachments, etc). See [slack-notifier](https://github.com/stevenosloan/slack-notifier#additional-parameters) for more information.. Default: '{}'
 
+##### additional_fields
+
+*Array of Hashes, optional*
+
+Contains additional fields that will be added to the attachement. See [Slack documentation](https://api.slack.com/docs/message-attachments).
+
 ## Mattermost notifier
 
 Post notification in a mattermost channel via [incoming webhook](http://docs.mattermost.com/developer/webhooks-incoming.html)
@@ -600,7 +618,7 @@ Just add the [HTTParty](https://github.com/jnunemaker/httparty) gem to your `Gem
 gem 'httparty'
 ```
 
-To configure it, you **need** to set the `webhook_url` option.  
+To configure it, you **need** to set the `webhook_url` option.
 You can also specify an other channel with `channel` option.
 
 ```ruby
@@ -616,7 +634,7 @@ Rails.application.config.middleware.use ExceptionNotification::Rack,
   }
 ```
 
-If you are using Github or Gitlab for issues tracking, you can specify `git_url` as follow to add a *Create issue* link in you notification.  
+If you are using Github or Gitlab for issues tracking, you can specify `git_url` as follow to add a *Create issue* link in you notification.
 By default this will use your Rails application name to match the git repository. If yours differ you can specify `app_name`.
 
 
@@ -810,6 +828,33 @@ Rails.application.config.middleware.use ExceptionNotification::Rack,
   }
 ```
 
+## Error Grouping
+In general, exception notification will send every notification when an error occured, which may result in a problem: if your site has a high throughput and an same error raised frequently, you will receive too many notifications during a short period time, your mail box may be full of thousands of exception mails or even your mail server will be slow. To prevent this, you can choose to error errors by using `:error_grouping` option and set it to `true`.
+
+Error grouping has a default formula `log2(errors_count)` to determine if it is needed to send the notification based on the accumulated errors count for specified exception, this makes the notifier only send notification when count is: 1, 2, 4, 8, 16, 32, 64, 128, ... (2**n). You can use `:notification_trigger` to override this default formula.
+
+The below shows options used to enable error grouping:
+
+```ruby
+Rails.application.config.middleware.use ExceptionNotification::Rack,
+  :ignore_exceptions => ['ActionView::TemplateError'] + ExceptionNotifier.ignored_exceptions,
+  :email => {
+    :email_prefix         => "[PREFIX] ",
+    :sender_address       => %{"notifier" <notifier@example.com>},
+    :exception_recipients => %w{exceptions@example.com}
+  },
+  :error_grouping => true,
+  # :error_grouping_period => 5.minutes,    # the time before an error is regarded as fixed
+  # :error_grouping_cache => Rails.cache,   # for other applications such as Sinatra, use one instance of ActiveSupport::Cache::Store
+  #
+  # notification_trigger: specify a callback to determine when a notification should be sent,
+  #   the callback will be invoked with two arguments:
+  #     exception: the exception raised
+  #     count: accumulated errors count for this exception
+  #
+  # :notification_trigger => lambda { |exception, count| count % 10 == 0 }
+```
+
 ## Ignore Exceptions
 
 You can choose to ignore certain exceptions, which will make ExceptionNotification avoid sending notifications for those specified. There are three ways of specifying which exceptions to ignore:

data/exception_notification.gemspec

@@ -1,8 +1,8 @@
 Gem::Specification.new do |s|
   s.name = 'exception_notification'
-  s.version = '4.2.1'
+  s.version = '4.2.2'
   s.authors = ["Jamis Buck", "Josh Peek"]
-  s.date = %q{2016-07-17}
+  s.date = %q{2017-08-12}
   s.summary = "Exception notification for Rails apps"
   s.homepage = "https://smartinez87.github.io/exception_notification/"
   s.email = "smartinez87@gmail.com"

data/lib/exception_notification/rack.rb

@@ -6,6 +6,15 @@ module ExceptionNotification
       @app = app
 
       ExceptionNotifier.ignored_exceptions = options.delete(:ignore_exceptions) if options.key?(:ignore_exceptions)
+      ExceptionNotifier.error_grouping = options.delete(:error_grouping) if options.key?(:error_grouping)
+      ExceptionNotifier.error_grouping_period = options.delete(:error_grouping_period) if options.key?(:error_grouping_period)
+      ExceptionNotifier.notification_trigger = options.delete(:notification_trigger) if options.key?(:notification_trigger)
+
+      if options.key?(:error_grouping_cache)
+        ExceptionNotifier.error_grouping_cache = options.delete(:error_grouping_cache)
+      elsif defined?(Rails)
+        ExceptionNotifier.error_grouping_cache = Rails.cache
+      end
 
       if options.key?(:ignore_if)
         rack_ignore = options.delete(:ignore_if)

data/lib/exception_notification/rails.rb

@@ -2,6 +2,7 @@ module ExceptionNotification
   class Engine < ::Rails::Engine
     config.exception_notification = ExceptionNotifier
     config.exception_notification.logger = Rails.logger
+    config.exception_notification.error_grouping_cache = Rails.cache
 
     config.app_middleware.use ExceptionNotification::Rack
   end

data/lib/exception_notifier.rb

@@ -2,8 +2,10 @@ require 'logger'
 require 'active_support/core_ext/string/inflections'
 require 'active_support/core_ext/module/attribute_accessors'
 require 'exception_notifier/base_notifier'
+require 'exception_notifier/modules/error_grouping'
 
 module ExceptionNotifier
+  include ErrorGrouping
 
   autoload :BacktraceCleaner, 'exception_notifier/modules/backtrace_cleaner'
 
@@ -43,6 +45,11 @@ module ExceptionNotifier
     def notify_exception(exception, options={})
       return false if ignored_exception?(options[:ignore_exceptions], exception)
       return false if ignored?(exception, options)
+      if error_grouping
+        errors_count = group_error!(exception, options)
+        return false unless send_notification?(exception, errors_count)
+      end
+
       selected_notifiers = options.delete(:notifiers) || notifiers
       [*selected_notifiers].each do |notifier|
         fire_notification(notifier, exception, options.dup)

data/lib/exception_notifier/campfire_notifier.rb

@@ -19,7 +19,11 @@ module ExceptionNotifier
 
     def call(exception, options={})
       if active?
-        message = "A new exception occurred: '#{exception.message}'"
+        message = if options[:accumulated_errors_count].to_i > 1
+          "The exception occurred #{options[:accumulated_errors_count]} times: '#{exception.message}'"
+        else
+          "A new exception occurred: '#{exception.message}'"
+        end
         message += " on '#{exception.backtrace.first}'" if exception.backtrace
         send_notice(exception, options, message) do |msg, _|
           @room.paste msg

data/lib/exception_notifier/email_notifier.rb

@@ -9,8 +9,8 @@ module ExceptionNotifier
     attr_accessor(:sender_address, :exception_recipients,
     :pre_callback, :post_callback,
     :email_prefix, :email_format, :sections, :background_sections,
-    :verbose_subject, :normalize_subject, :delivery_method, :mailer_settings,
-    :email_headers, :mailer_parent, :template_path, :deliver_with)
+    :verbose_subject, :normalize_subject, :include_controller_and_action_names_in_subject,
+    :delivery_method, :mailer_settings, :email_headers, :mailer_parent, :template_path, :deliver_with)
 
     module Mailer
       class MissingController
@@ -46,7 +46,7 @@ module ExceptionNotifier
             load_custom_views
 
             @exception = exception
-            @options   = options.reverse_merge(default_options)
+            @options   = options.reverse_merge(default_options).symbolize_keys
             @backtrace = exception.backtrace || []
             @timestamp = Time.current
             @sections  = @options[:background_sections]
@@ -60,7 +60,8 @@ module ExceptionNotifier
 
           def compose_subject
             subject = "#{@options[:email_prefix]}"
-            subject << "#{@kontroller.controller_name}##{@kontroller.action_name}" if @kontroller
+            subject << "(#{@options[:accumulated_errors_count]} times) " if @options[:accumulated_errors_count].to_i > 1
+            subject << "#{@kontroller.controller_name}##{@kontroller.action_name}" if @kontroller && @options[:include_controller_and_action_names_in_subject]
             subject << " (#{@exception.class})"
             subject << " #{@exception.message.inspect}" if @options[:verbose_subject]
             subject = EmailNotifier.normalize_digits(subject) if @options[:normalize_subject]
@@ -74,13 +75,17 @@ module ExceptionNotifier
           end
 
           helper_method :inspect_object
-
+          
+          def truncate(string, max)
+            string.length > max ? "#{string[0...max]}..." : string
+          end
+          
           def inspect_object(object)
             case object
               when Hash, Array
-                object.inspect
+                truncate(object.inspect, 300)
               else
-                object.to_s
+                  object.to_s 
             end
           end
 
@@ -138,10 +143,10 @@ module ExceptionNotifier
       options[:mailer_settings] = options.delete(mailer_settings_key)
 
       options.reverse_merge(EmailNotifier.default_options).select{|k,v|[
-        :sender_address, :exception_recipients,
-        :pre_callback, :post_callback,
-        :email_prefix, :email_format, :sections, :background_sections,
-        :verbose_subject, :normalize_subject, :delivery_method, :mailer_settings,
+        :sender_address, :exception_recipients, :pre_callback,
+        :post_callback, :email_prefix, :email_format,
+        :sections, :background_sections, :verbose_subject, :normalize_subject,
+        :include_controller_and_action_names_in_subject, :delivery_method, :mailer_settings,
         :email_headers, :mailer_parent, :template_path, :deliver_with].include?(k)}.each{|k,v| send("#{k}=", v)}
     end
 
@@ -197,6 +202,7 @@ module ExceptionNotifier
         :background_sections => %w(backtrace data),
         :verbose_subject => true,
         :normalize_subject => false,
+        :include_controller_and_action_names_in_subject => true,
         :delivery_method => nil,
         :mailer_settings => nil,
         :email_headers => {},

data/lib/exception_notifier/hipchat_notifier.rb

@@ -13,10 +13,15 @@ module ExceptionNotifier
         opts              = {
                               :api_version => options.delete(:api_version) || 'v1'
                             }
+        opts[:server_url] = options.delete(:server_url) if options[:server_url]
         @from             = options.delete(:from) || 'Exception'
         @room             = HipChat::Client.new(api_token, opts)[room_name]
-        @message_template = options.delete(:message_template) || ->(exception) {
-          msg = "A new exception occurred: '#{Rack::Utils.escape_html(exception.message)}'"
+        @message_template = options.delete(:message_template) || ->(exception, errors_count) {
+          msg = if errors_count > 1
+            "The exception occurred #{errors_count} times: '#{Rack::Utils.escape_html(exception.message)}'"
+          else
+            "A new exception occurred: '#{Rack::Utils.escape_html(exception.message)}'"
+          end
           msg += " on '#{exception.backtrace.first}'" if exception.backtrace
           msg
         }
@@ -30,7 +35,7 @@ module ExceptionNotifier
     def call(exception, options={})
       return if !active?
 
-      message = @message_template.call(exception)
+      message = @message_template.call(exception, options[:accumulated_errors_count].to_i)
       send_notice(exception, options, message, @message_options) do |msg, message_opts|
         @room.send(@from, msg, message_opts)
       end

data/lib/exception_notifier/irc_notifier.rb

@@ -7,7 +7,11 @@ module ExceptionNotifier
     end
 
     def call(exception, options={})
+      errors_count = options[:accumulated_errors_count].to_i
+
       message = "'#{exception.message}'"
+      message.prepend("(#{errors_count} times)") if errors_count > 1
+
       message += " on '#{exception.backtrace.first}'" if exception.backtrace
       if active?
         send_notice(exception, options, message) do |msg, _|

data/lib/exception_notifier/mattermost_notifier.rb

@@ -91,8 +91,9 @@ module ExceptionNotifier
       def message_header
         text = []
 
+        errors_count = @options[:accumulated_errors_count].to_i
         text << "### :warning: Error 500 in #{Rails.env} :warning:"
-        text << "An *#{@exception.class}* occured" + if @controller then " in *#{controller_and_method}*." else "." end
+        text << "#{errors_count > 1 ? errors_count : 'An'} *#{@exception.class}* occured" + if @controller then " in *#{controller_and_method}*." else "." end
         text << "*#{@exception.message}*"
 
         text

data/lib/exception_notifier/modules/error_grouping.rb

@@ -0,0 +1,77 @@
+require 'active_support/core_ext/numeric/time'
+require 'active_support/concern'
+
+module ExceptionNotifier
+  module ErrorGrouping
+    extend ActiveSupport::Concern
+
+    included do
+      mattr_accessor :error_grouping
+      self.error_grouping = false
+
+      mattr_accessor :error_grouping_period
+      self.error_grouping_period = 5.minutes
+
+      mattr_accessor :notification_trigger
+
+      mattr_accessor :error_grouping_cache
+    end
+
+    module ClassMethods
+      # Fallback to the memory store while the specified cache store doesn't work
+      #
+      def fallback_cache_store
+        @fallback_cache_store ||= ActiveSupport::Cache::MemoryStore.new
+      end
+
+      def error_count(error_key)
+        count = begin
+          error_grouping_cache.read(error_key)
+        rescue => e
+          ExceptionNotifier.logger.warn("#{error_grouping_cache.inspect} failed to read, reason: #{e.message}. Falling back to memory cache store.")
+          fallback_cache_store.read(error_key)
+        end
+
+        count.to_i if count
+      end
+
+      def save_error_count(error_key, count)
+        error_grouping_cache.write(error_key, count, expires_in: error_grouping_period)
+      rescue => e
+        ExceptionNotifier.logger.warn("#{error_grouping_cache.inspect} failed to write, reason: #{e.message}. Falling back to memory cache store.")
+        fallback_cache_store.write(error_key, count, expires_in: error_grouping_period)
+      end
+
+      def group_error!(exception, options)
+        message_based_key = "exception:#{Zlib.crc32("#{exception.class.name}\nmessage:#{exception.message}")}"
+        accumulated_errors_count = 1
+
+        if count = error_count(message_based_key)
+          accumulated_errors_count = count + 1
+          save_error_count(message_based_key, accumulated_errors_count)
+        else
+          backtrace_based_key = "exception:#{Zlib.crc32("#{exception.class.name}\npath:#{exception.backtrace.try(:first)}")}"
+
+          if count = Rails.cache.read(backtrace_based_key)
+            accumulated_errors_count = count + 1
+            save_error_count(backtrace_based_key, accumulated_errors_count)
+          else
+            save_error_count(backtrace_based_key, accumulated_errors_count)
+            save_error_count(message_based_key, accumulated_errors_count)
+          end
+        end
+
+        options[:accumulated_errors_count] = accumulated_errors_count
+      end
+
+      def send_notification?(exception, count)
+        if notification_trigger.respond_to?(:call)
+          notification_trigger.call(exception, count)
+        else
+          factor = Math.log2(count)
+          factor.to_i == factor
+        end
+      end
+    end
+  end
+end

data/lib/exception_notifier/slack_notifier.rb

@@ -8,10 +8,12 @@ module ExceptionNotifier
       super
       begin
         @ignore_data_if = options[:ignore_data_if]
-        @backtrace_lines = options[:backtrace_lines]
+        @backtrace_lines = options.fetch(:backtrace_lines, 10)
+        @additional_fields = options[:additional_fields]
 
         webhook_url = options.fetch(:webhook_url)
         @message_opts = options.fetch(:additional_parameters, {})
+        @color = @message_opts.delete(:color) { 'danger' }
         @notifier = Slack::Notifier.new webhook_url, options
       rescue
         @notifier = nil
@@ -19,7 +21,9 @@ module ExceptionNotifier
     end
 
     def call(exception, options={})
-      exception_name = "*#{exception.class.to_s =~ /^[aeiou]/i ? 'An' : 'A'}* `#{exception.class.to_s}`"
+      errors_count = options[:accumulated_errors_count].to_i
+      measure_word = errors_count > 1 ? errors_count : (exception.class.to_s =~ /^[aeiou]/i ? 'An' : 'A')
+      exception_name = "*#{measure_word}* `#{exception.class.to_s}`"
 
       if options[:env].nil?
         data = options[:data] || {}
@@ -30,18 +34,18 @@ module ExceptionNotifier
 
         kontroller = env['action_controller.instance']
         request = "#{env['REQUEST_METHOD']} <#{env['REQUEST_URI']}>"
-        text = "#{exception_name} *occurred while* `#{env['REQUEST_METHOD']} <#{env['REQUEST_URI']}>`"
+        text = "#{exception_name} *occurred while* `#{request}`"
         text += " *was processed by* `#{kontroller.controller_name}##{kontroller.action_name}`" if kontroller
         text += "\n"
       end
 
       clean_message = exception.message.gsub("`", "'")
-      fields = [ { title: 'Exception', value: clean_message} ]
+      fields = [ { title: 'Exception', value: clean_message } ]
 
       fields.push({ title: 'Hostname', value: Socket.gethostname })
 
       if exception.backtrace
-        formatted_backtrace = @backtrace_lines ? "```#{exception.backtrace.first(@backtrace_lines).join("\n")}```" : "```#{exception.backtrace.join("\n")}```"
+        formatted_backtrace = "```#{exception.backtrace.first(@backtrace_lines).join("\n")}```"
         fields.push({ title: 'Backtrace', value: formatted_backtrace })
       end
 
@@ -51,7 +55,9 @@ module ExceptionNotifier
         fields.push({ title: 'Data', value: "```#{data_string}```" })
       end
 
-      attchs = [color: 'danger', text: text, fields: fields, mrkdwn_in: %w(text fields)]
+      fields.concat(@additional_fields) if @additional_fields
+
+      attchs = [color: @color, text: text, fields: fields, mrkdwn_in: %w(text fields)]
 
       if valid?
         send_notice(exception, options, clean_message, @message_opts.merge(attachments: attchs)) do |msg, message_opts|

data/lib/exception_notifier/views/exception_notifier/exception_notification.html.erb

@@ -14,11 +14,9 @@
         title = render("title", :title => section).strip
         [title, summary]
       end
-
     rescue Exception => e
       title = render("title", :title => section).strip
       summary = ["ERROR: Failed to generate exception summary:", [e.class.to_s, e.message].join(": "), e.backtrace && e.backtrace.join("\n")].compact.join("\n\n")
-
       [title, summary]
     end
   end

data/test/dummy/test/functional/posts_controller_test.rb

@@ -35,7 +35,7 @@ class PostsControllerTest < ActionController::TestCase
   test "mail subject should have the proper prefix" do
     assert_includes @mail.subject, "[Dummy ERROR]"
   end
-
+  
   test "mail subject should include descriptive error message" do
     assert_includes @mail.subject, "(NoMethodError) \"undefined method `nw'"
   end
@@ -146,6 +146,25 @@ class PostsControllerTestWithoutVerboseSubject < ActionController::TestCase
   end
 end
 
+class PostsControllerTestWithoutControllerAndActionNames < ActionController::TestCase
+  tests PostsController
+  setup do
+    @email_notifier = ExceptionNotifier::EmailNotifier.new(:include_controller_and_action_names_in_subject => false)
+    begin
+      post :create, method: :post
+    rescue => e
+      @exception = e
+      @mail = @email_notifier.create_email(@exception, {:env => request.env})
+    end
+  end
+
+  test "should include controller and action names in subject" do
+    assert_includes @mail.subject, '[ERROR]'
+    assert_includes @mail.subject, '(NoMethodError)'
+    refute_includes @mail.subject, 'posts#create'
+  end
+end
+
 class PostsControllerTestWithSmtpSettings < ActionController::TestCase
   tests PostsController
   setup do

data/test/exception_notification/rack_test.rb

@@ -5,6 +5,14 @@ class RackTest < ActiveSupport::TestCase
   setup do
     @pass_app = Object.new
     @pass_app.stubs(:call).returns([nil, { 'X-Cascade' => 'pass' }, nil])
+
+    @normal_app = Object.new
+    @normal_app.stubs(:call).returns([nil, { }, nil])
+  end
+
+  teardown do
+    ExceptionNotifier.error_grouping = false
+    ExceptionNotifier.notification_trigger = nil
   end
 
   test "should ignore \"X-Cascade\" header by default" do
@@ -17,4 +25,20 @@ class RackTest < ActiveSupport::TestCase
     ExceptionNotification::Rack.new(@pass_app, :ignore_cascade_pass => false).call({})
   end
 
+  test "should assign error_grouping if error_grouping is specified" do
+    refute ExceptionNotifier.error_grouping
+    ExceptionNotification::Rack.new(@normal_app, error_grouping: true).call({})
+    assert ExceptionNotifier.error_grouping
+  end
+
+  test "should assign notification_trigger if notification_trigger is specified" do
+    assert_nil ExceptionNotifier.notification_trigger
+    ExceptionNotification::Rack.new(@normal_app, notification_trigger: lambda {|i| true}).call({})
+    assert_respond_to ExceptionNotifier.notification_trigger, :call
+  end
+
+  test "should set default cache to Rails cache" do
+    ExceptionNotification::Rack.new(@normal_app, error_grouping: true).call({})
+    assert_equal Rails.cache, ExceptionNotifier.error_grouping_cache
+  end
 end

data/test/exception_notifier/campfire_notifier_test.rb

@@ -52,6 +52,20 @@ class CampfireNotifierTest < ActiveSupport::TestCase
     assert_nil campfire.call(fake_exception)
   end
 
+  test "should send the new exception message if no :accumulated_errors_count option" do
+    campfire = ExceptionNotifier::CampfireNotifier.new({})
+    campfire.stubs(:active?).returns(true)
+    campfire.expects(:send_notice).with{ |_, _, message| message.start_with?("A new exception occurred") }.once
+    campfire.call(fake_exception)
+  end
+
+  test "shoud send the exception message if :accumulated_errors_count option greater than 1" do
+    campfire = ExceptionNotifier::CampfireNotifier.new({})
+    campfire.stubs(:active?).returns(true)
+    campfire.expects(:send_notice).with{ |_, _, message| message.start_with?("The exception occurred 3 times:") }.once
+    campfire.call(fake_exception, accumulated_errors_count: 3)
+  end
+
   test "should call pre/post_callback if specified" do 
     pre_callback_called, post_callback_called = 0,0
     Tinder::Campfire.stubs(:new).returns(Object.new)

data/test/exception_notifier/email_notifier_test.rb

@@ -218,4 +218,18 @@ class EmailNotifierTest < ActiveSupport::TestCase
     mail = email_notifier.call(@exception)
     assert_equal %w{second@example.com}, mail.to
   end
+
+  test "should prepend accumulated_errors_count in email subject if accumulated_errors_count larger than 1" do
+    ActionMailer::Base.deliveries.clear
+
+    email_notifier = ExceptionNotifier::EmailNotifier.new(
+      :email_prefix => '[Dummy ERROR] ',
+      :sender_address => %{"Dummy Notifier" <dummynotifier@example.com>},
+      :exception_recipients => %w{dummyexceptions@example.com},
+      :delivery_method => :test
+    )
+
+    mail = email_notifier.call(@exception, { accumulated_errors_count: 3 })
+    assert mail.subject.start_with?("[Dummy ERROR] (3 times) (ZeroDivisionError)")
+  end
 end

data/test/exception_notifier/hipchat_notifier_test.rb

@@ -101,7 +101,7 @@ class HipchatNotifierTest < ActiveSupport::TestCase
       :api_token => 'good_token',
       :room_name => 'room_name',
       :color     => 'yellow',
-      :message_template => ->(exception) { "This is custom message: '#{exception.message}'" }
+      :message_template => ->(exception, _) { "This is custom message: '#{exception.message}'" }
     }
 
     HipChat::Room.any_instance.expects(:send).with('Exception', "This is custom message: '#{fake_exception.message}'", { :color => 'yellow' })
@@ -110,6 +110,30 @@ class HipchatNotifierTest < ActiveSupport::TestCase
     hipchat.call(fake_exception)
   end
 
+  test "should send hipchat notification exclude accumulated errors count" do
+    options = {
+      :api_token => 'good_token',
+      :room_name => 'room_name',
+      :color     => 'yellow'
+    }
+
+    HipChat::Room.any_instance.expects(:send).with{ |_, msg, _| msg.start_with?("A new exception occurred:") }
+    hipchat = ExceptionNotifier::HipchatNotifier.new(options)
+    hipchat.call(fake_exception)
+  end
+
+  test "should send hipchat notification include accumulated errors count" do
+    options = {
+      :api_token => 'good_token',
+      :room_name => 'room_name',
+      :color     => 'yellow'
+    }
+
+    HipChat::Room.any_instance.expects(:send).with{ |_, msg, _| msg.start_with?("The exception occurred 3 times:") }
+    hipchat = ExceptionNotifier::HipchatNotifier.new(options)
+    hipchat.call(fake_exception, { accumulated_errors_count: 3 })
+  end
+
   test "should send hipchat notification with HTML-escaped meessage if using default message_template" do
     options = {
       :api_token => 'good_token',
@@ -151,6 +175,20 @@ class HipchatNotifierTest < ActiveSupport::TestCase
     hipchat.call(fake_exception)
   end
 
+  test "should allow server_url value (for a self-hosted HipChat Server) if set" do
+    options = {
+      :api_token   => 'good_token',
+      :room_name   => 'room_name',
+      :api_version => 'v2',
+      :server_url  => 'https://domain.com',
+    }
+
+    HipChat::Client.stubs(:new).with('good_token', {:api_version => 'v2', :server_url => 'https://domain.com'}).returns({})
+
+    hipchat = ExceptionNotifier::HipchatNotifier.new(options)
+    hipchat.call(fake_exception)
+  end
+
   private
 
   def fake_body

data/test/exception_notifier/irc_notifier_test.rb

@@ -16,6 +16,22 @@ class IrcNotifierTest < ActiveSupport::TestCase
     irc.call(fake_exception)
   end
 
+  test "should exclude errors count in message if :accumulated_errors_count nil" do
+    irc = ExceptionNotifier::IrcNotifier.new({})
+    irc.stubs(:active?).returns(true)
+
+    irc.expects(:send_message).with{ |message| message.include?("divided by 0") }.once
+    irc.call(fake_exception)
+  end
+
+  test "should include errors count in message if :accumulated_errors_count is 3" do
+    irc = ExceptionNotifier::IrcNotifier.new({})
+    irc.stubs(:active?).returns(true)
+
+    irc.expects(:send_message).with{ |message| message.include?("(3 times)'divided by 0'") }.once
+    irc.call(fake_exception, accumulated_errors_count: 3)
+  end
+
   test "should call pre/post_callback if specified" do
     pre_callback_called, post_callback_called = 0,0
 

data/test/exception_notifier/mattermost_notifier_test.rb

@@ -77,6 +77,23 @@ class MattermostNotifierTest < ActiveSupport::TestCase
     assert 'password', options[:basic_auth][:password]
   end
 
+  test "should use 'An' for exceptions count if :accumulated_errors_count option is nil" do
+    mattermost_notifier = ExceptionNotifier::MattermostNotifier.new
+    exception = ArgumentError.new("foo")
+    mattermost_notifier.instance_variable_set(:@exception, exception)
+    mattermost_notifier.instance_variable_set(:@options, {})
+
+    assert_includes mattermost_notifier.send(:message_header), "An *ArgumentError* occured."
+  end
+
+  test "shoud use direct errors count if :accumulated_errors_count option is 5" do
+    mattermost_notifier = ExceptionNotifier::MattermostNotifier.new
+    exception = ArgumentError.new("foo")
+    mattermost_notifier.instance_variable_set(:@exception, exception)
+    mattermost_notifier.instance_variable_set(:@options, { accumulated_errors_count: 5 })
+
+    assert_includes mattermost_notifier.send(:message_header), "5 *ArgumentError* occured."
+  end
 end
 
 class FakeHTTParty

data/test/exception_notifier/modules/error_grouping_test.rb

@@ -0,0 +1,166 @@
+require 'test_helper'
+
+class ErrorGroupTest < ActiveSupport::TestCase
+
+  setup do
+    module TestModule
+      include ExceptionNotifier::ErrorGrouping
+      @@error_grouping_cache = ActiveSupport::Cache::FileStore.new("test/dummy/tmp/cache")
+    end
+
+    @exception = RuntimeError.new("ERROR")
+    @exception.stubs(:backtrace).returns(["/path/where/error/raised:1"])
+
+    @exception2 = RuntimeError.new("ERROR2")
+    @exception2.stubs(:backtrace).returns(["/path/where/error/found:2"])
+  end
+
+  teardown do
+    TestModule.error_grouping_cache.clear
+    TestModule.fallback_cache_store.clear
+  end
+
+  test "should add additional option: error_grouping" do
+    assert_respond_to TestModule, :error_grouping
+    assert_respond_to TestModule, :error_grouping=
+  end
+
+  test "should set error_grouping to false default" do
+    assert_equal false, TestModule.error_grouping
+  end
+
+  test "should add additional option: error_grouping_cache" do
+    assert_respond_to TestModule, :error_grouping_cache
+    assert_respond_to TestModule, :error_grouping_cache=
+  end
+
+  test "should add additional option: error_grouping_period" do
+    assert_respond_to TestModule, :error_grouping_period
+    assert_respond_to TestModule, :error_grouping_period=
+  end
+
+  test "shoud set error_grouping_period to 5.minutes default" do
+    assert_equal 300, TestModule.error_grouping_period
+  end
+
+  test "should add additional option: notification_trigger" do
+    assert_respond_to TestModule, :notification_trigger
+    assert_respond_to TestModule, :notification_trigger=
+  end
+
+  test "should return errors count nil when not same error for .error_count" do
+    assert_nil TestModule.error_count("something")
+  end
+
+  test "should return errors count when same error for .error_count" do
+    TestModule.error_grouping_cache.write("error_key", 13)
+    assert_equal 13, TestModule.error_count("error_key")
+  end
+
+  test "should fallback to memory store cache if specified cache store failed to read" do
+    TestModule.error_grouping_cache.stubs(:read).raises(RuntimeError.new "Failed to read")
+    original_fallback = TestModule.fallback_cache_store
+    TestModule.expects(:fallback_cache_store).returns(original_fallback).at_least_once
+
+    assert_nil TestModule.error_count("something_to_read")
+  end
+
+  test "should save error with count for .save_error_count" do
+    count = rand(1..10)
+
+    TestModule.save_error_count("error_key", count)
+    assert_equal count, TestModule.error_grouping_cache.read("error_key")
+  end
+
+  test "should fallback to memory store cache if specified cache store failed to write" do
+    TestModule.error_grouping_cache.stubs(:write).raises(RuntimeError.new "Failed to write")
+    original_fallback = TestModule.fallback_cache_store
+    TestModule.expects(:fallback_cache_store).returns(original_fallback).at_least_once
+
+    assert TestModule.save_error_count("something_to_cache", rand(1..10))
+  end
+
+  test "should save accumulated_errors_count into options" do
+    options = {}
+    TestModule.group_error!(@exception, options)
+
+    assert_equal 1, options[:accumulated_errors_count]
+  end
+
+  test "should not group error if different exception in .group_error!" do
+    options1 = {}
+    TestModule.expects(:save_error_count).with{|key, count| key.is_a?(String) && count == 1}.times(4).returns(true)
+    TestModule.group_error!(@exception, options1)
+
+    options2 = {}
+    TestModule.group_error!(NoMethodError.new("method not found"), options2)
+
+    assert_equal 1, options1[:accumulated_errors_count]
+    assert_equal 1, options2[:accumulated_errors_count]
+  end
+
+  test "should not group error is same exception but different message or backtrace" do
+    options1 = {}
+    TestModule.expects(:save_error_count).with{|key, count| key.is_a?(String) && count == 1}.times(4).returns(true)
+    TestModule.group_error!(@exception, options1)
+
+    options2 = {}
+    TestModule.group_error!(@exception2, options2)
+
+    assert_equal 1, options1[:accumulated_errors_count]
+    assert_equal 1, options2[:accumulated_errors_count]
+  end
+
+  test "should group error if same exception and message" do
+    options = {}
+
+    10.times do |i|
+      @exception2.stubs(:backtrace).returns(["/path:#{i}"])
+      TestModule.group_error!(@exception2, options)
+    end
+
+    assert_equal 10, options[:accumulated_errors_count]
+  end
+
+  test "should group error if same exception and backtrace" do
+    options = {}
+
+    10.times do |i|
+      @exception2.stubs(:message).returns("ERRORS#{i}")
+      TestModule.group_error!(@exception2, options)
+    end
+
+    assert_equal 10, options[:accumulated_errors_count]
+  end
+
+  test "should group error by that message have high priority" do
+    message_based_key = "exception:#{Zlib.crc32("RuntimeError\nmessage:ERROR")}"
+    backtrace_based_key = "exception:#{Zlib.crc32("RuntimeError\n/path/where/error/raised:1")}"
+
+    TestModule.save_error_count(message_based_key, 1)
+    TestModule.save_error_count(backtrace_based_key, 1)
+
+    TestModule.expects(:save_error_count).with(message_based_key, 2).once
+    TestModule.expects(:save_error_count).with(backtrace_based_key, 2).never
+
+    TestModule.group_error!(@exception, {})
+  end
+
+  test "use default formula if not specify notification_trigger in .send_notification?" do
+    TestModule.stubs(:notification_trigger).returns(nil)
+
+    count = 16
+    Math.expects(:log2).with(count).returns(4)
+
+    assert TestModule.send_notification?(@exception, count)
+  end
+
+  test "use specified trigger in .send_notification?" do
+    trigger = Proc.new { |exception, count| count % 4 == 0 }
+    TestModule.stubs(:notification_trigger).returns(trigger)
+
+    count = 16
+    trigger.expects(:call).with(@exception, count).returns(true)
+    assert TestModule.send_notification?(@exception, count)
+  end
+end

data/test/exception_notifier/slack_notifier_test.rb

@@ -43,7 +43,8 @@ class SlackNotifierTest < ActiveSupport::TestCase
     slack_notifier = ExceptionNotifier::SlackNotifier.new(options)
     slack_notifier.call(@exception)
 
-    assert_equal slack_notifier.notifier.channel, options[:channel]
+    channel = slack_notifier.notifier.config.defaults[:channel]
+    assert_equal channel, options[:channel]
   end
 
   test "should send the notification to the specified username" do
@@ -57,7 +58,8 @@ class SlackNotifierTest < ActiveSupport::TestCase
     slack_notifier = ExceptionNotifier::SlackNotifier.new(options)
     slack_notifier.call(@exception)
 
-    assert_equal slack_notifier.notifier.username, options[:username]
+    username = slack_notifier.notifier.config.defaults[:username]
+    assert_equal username, options[:username]
   end
 
   test "should send the notification with specific backtrace lines" do
@@ -72,6 +74,22 @@ class SlackNotifierTest < ActiveSupport::TestCase
     slack_notifier.call(@exception)
   end
 
+  test "should send the notification with additional fields" do
+    field = {title: "Branch", value: "master", short: true}
+    options = {
+      webhook_url: "http://slack.webhook.url",
+      additional_fields: [field]
+    }
+
+    Slack::Notifier.any_instance.expects(:ping).with('', fake_notification(@exception, {}, nil, 10, [field]))
+
+    slack_notifier = ExceptionNotifier::SlackNotifier.new(options)
+    slack_notifier.call(@exception)
+
+    additional_fields = slack_notifier.notifier.config.defaults[:additional_fields]
+    assert_equal additional_fields, options[:additional_fields]
+  end
+
   test "should pass the additional parameters to Slack::Notifier.ping" do
     options = {
       webhook_url: "http://slack.webhook.url",
@@ -177,7 +195,7 @@ class SlackNotifierTest < ActiveSupport::TestCase
     ]
   end
 
-  def fake_notification(exception = @exception, notification_options = {}, data_string = nil, expected_backtrace_lines = nil)
+  def fake_notification(exception = @exception, notification_options = {}, data_string = nil, expected_backtrace_lines = 10, additional_fields = [])
     exception_name = "*#{exception.class.to_s =~ /^[aeiou]/i ? 'An' : 'A'}* `#{exception.class.to_s}`"
     if notification_options[:env].nil?
       text = "#{exception_name} *occured in background*"
@@ -196,10 +214,11 @@ class SlackNotifierTest < ActiveSupport::TestCase
     fields = [ { title: 'Exception', value: exception.message} ]
     fields.push({ title: 'Hostname', value: 'example.com' })
     if exception.backtrace
-      formatted_backtrace = expected_backtrace_lines ? "```#{exception.backtrace.first(expected_backtrace_lines).join("\n")}```" : "```#{exception.backtrace.join("\n")}```"
+      formatted_backtrace = "```#{exception.backtrace.first(expected_backtrace_lines).join("\n")}```"
       fields.push({ title: 'Backtrace', value: formatted_backtrace })
     end
     fields.push({ title: 'Data', value: "```#{data_string}```" }) if data_string
+    additional_fields.each { |f| fields.push(f) }
 
     { attachments: [ color: 'danger', text: text, fields: fields, mrkdwn_in: %w(text fields) ] }
   end

data/test/exception_notifier_test.rb

@@ -1,6 +1,21 @@
 require 'test_helper'
 
+class ExceptionOne < StandardError;end
+class ExceptionTwo < StandardError;end
+
 class ExceptionNotifierTest < ActiveSupport::TestCase
+  setup do
+    @notifier_calls = 0
+    @test_notifier = lambda { |exception, options| @notifier_calls += 1 }
+  end
+
+  teardown do
+    ExceptionNotifier.error_grouping = false
+    ExceptionNotifier.notification_trigger = nil
+    ExceptionNotifier.class_eval("@@notifiers.delete_if { |k, _| k.to_s != \"email\"}")  # reset notifiers
+    Rails.cache.clear
+  end
+
   test "should have default ignored exceptions" do
     assert_equal ExceptionNotifier.ignored_exceptions,
       ['ActiveRecord::RecordNotFound', 'Mongoid::Errors::DocumentNotFound', 'AbstractController::ActionNotFound',
@@ -69,37 +84,67 @@ class ExceptionNotifierTest < ActiveSupport::TestCase
       env != "production"
     end
 
-    notifier_calls = 0
-    test_notifier = lambda { |exception, options| notifier_calls += 1 }
-    ExceptionNotifier.register_exception_notifier(:test, test_notifier)
+    ExceptionNotifier.register_exception_notifier(:test, @test_notifier)
 
     exception = StandardError.new
 
     ExceptionNotifier.notify_exception(exception, {:notifiers => :test})
-    assert_equal notifier_calls, 1
+    assert_equal @notifier_calls, 1
 
     env = "development"
     ExceptionNotifier.notify_exception(exception, {:notifiers => :test})
-    assert_equal notifier_calls, 1
+    assert_equal @notifier_calls, 1
 
     ExceptionNotifier.clear_ignore_conditions!
-    ExceptionNotifier.unregister_exception_notifier(:test)
   end
 
   test "should not send notification if one of ignored exceptions" do
-    notifier_calls = 0
-    test_notifier = lambda { |exception, options| notifier_calls += 1 }
-    ExceptionNotifier.register_exception_notifier(:test, test_notifier)
+    ExceptionNotifier.register_exception_notifier(:test, @test_notifier)
 
     exception = StandardError.new
 
     ExceptionNotifier.notify_exception(exception, {:notifiers => :test})
-    assert_equal notifier_calls, 1
+    assert_equal @notifier_calls, 1
 
     ExceptionNotifier.notify_exception(exception, {:notifiers => :test, :ignore_exceptions => 'StandardError' })
-    assert_equal notifier_calls, 1
+    assert_equal @notifier_calls, 1
+  end
+
+  test "should not call group_error! or send_notification? if error_grouping false" do
+    exception = StandardError.new
+    ExceptionNotifier.expects(:group_error!).never
+    ExceptionNotifier.expects(:send_notification?).never
+
+    ExceptionNotifier.notify_exception(exception)
+  end
+
+  test "should call group_error! and send_notification? if error_grouping true" do
+    ExceptionNotifier.error_grouping = true
 
-    ExceptionNotifier.unregister_exception_notifier(:test)
+    exception = StandardError.new
+    ExceptionNotifier.expects(:group_error!).once
+    ExceptionNotifier.expects(:send_notification?).once
+
+    ExceptionNotifier.notify_exception(exception)
+  end
+
+  test "should skip notification if send_notification? is false" do
+    ExceptionNotifier.error_grouping = true
+
+    exception = StandardError.new
+    ExceptionNotifier.expects(:group_error!).once.returns(1)
+    ExceptionNotifier.expects(:send_notification?).with(exception, 1).once.returns(false)
+
+    refute ExceptionNotifier.notify_exception(exception)
   end
 
+  test "should send notification if send_notification? is true" do
+    ExceptionNotifier.error_grouping = true
+
+    exception = StandardError.new
+    ExceptionNotifier.expects(:group_error!).once.returns(1)
+    ExceptionNotifier.expects(:send_notification?).with(exception, 1).once.returns(true)
+
+    assert ExceptionNotifier.notify_exception(exception)
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: exception_notification
 version: !ruby/object:Gem::Version
-  version: 4.2.1
+  version: 4.2.2
 platform: ruby
 authors:
 - Jamis Buck
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-07-17 00:00:00.000000000 Z
+date: 2017-08-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: actionmailer
@@ -269,6 +269,7 @@ files:
 - lib/exception_notifier/irc_notifier.rb
 - lib/exception_notifier/mattermost_notifier.rb
 - lib/exception_notifier/modules/backtrace_cleaner.rb
+- lib/exception_notifier/modules/error_grouping.rb
 - lib/exception_notifier/notifier.rb
 - lib/exception_notifier/slack_notifier.rb
 - lib/exception_notifier/views/exception_notifier/_backtrace.html.erb
@@ -348,6 +349,7 @@ files:
 - test/exception_notifier/hipchat_notifier_test.rb
 - test/exception_notifier/irc_notifier_test.rb
 - test/exception_notifier/mattermost_notifier_test.rb
+- test/exception_notifier/modules/error_grouping_test.rb
 - test/exception_notifier/sidekiq_test.rb
 - test/exception_notifier/slack_notifier_test.rb
 - test/exception_notifier/webhook_notifier_test.rb
@@ -436,6 +438,7 @@ test_files:
 - test/exception_notifier/hipchat_notifier_test.rb
 - test/exception_notifier/irc_notifier_test.rb
 - test/exception_notifier/mattermost_notifier_test.rb
+- test/exception_notifier/modules/error_grouping_test.rb
 - test/exception_notifier/sidekiq_test.rb
 - test/exception_notifier/slack_notifier_test.rb
 - test/exception_notifier/webhook_notifier_test.rb