semantic_logger 4.5.0 → 4.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 34b3b78799e196b270d81aba7deb9b0accd1187a900967a3943dd03301c63199
-  data.tar.gz: 775b6eeca4ca6b9c38e9213e7f32fee9d179ce36148433dfa91a8630f74873b3
+  metadata.gz: b09c37737f1e42edbeee95b036ba0581d5164d756e13de37aca9b22b3093cd45
+  data.tar.gz: 9f66b72480df2371a7d305164e443f9667e1acd5521c1347d54c3841efbfa4c9
 SHA512:
-  metadata.gz: 7cda8a50c62a2dcdb079d5f96cbde61e5cc941c443122a3bbc102a1513bc3686d44a20cca28e7627444bf640a6329035aefc43cb6111a3e6b9cb91411fe33782
-  data.tar.gz: caf355bf16e08b133fe07ddd6b4cab813b6e7224b3e1f152e1ab54cf30f574fc8665f99ae3938078fb35fcfeea3643d565a814cbe2774d263135c93722147b06
+  metadata.gz: fb27aa1a6d5dc1f2e0e6505ba9348a7cec091a153763361e8c9c20c21177bf84ac11ac2f21fc37ec994988ba31763dcd5c3134ac40a169ac987ed617eceec46e
+  data.tar.gz: 5d084bc8cc57b1e836326eae912667ef6b7a284b0125c0b0acee678af78c01017b9667db9a43df7c0fb38af4998b0ab3128ac05d3e2bce266176e9c4d8fadccd

data/README.md

@@ -1,25 +1,13 @@
 # Semantic Logger
-[![Gem Version](https://img.shields.io/gem/v/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![Build Status](https://travis-ci.org/rocketjob/semantic_logger.svg?branch=master)](https://travis-ci.org/rocketjob/semantic_logger) [![Downloads](https://img.shields.io/gem/dt/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](http://opensource.org/licenses/Apache-2.0) ![](https://img.shields.io/badge/status-Production%20Ready-blue.svg) [![Gitter chat](https://img.shields.io/badge/IRC%20(gitter)-Support-brightgreen.svg)](https://gitter.im/rocketjob/support)
+[![Gem Version](https://img.shields.io/gem/v/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![Build Status](https://github.com/reidmorrison/semantic_logger/workflows/build/badge.svg)](https://github.com/reidmorrison/semantic_logger/actions?query=workflow%3Abuild) [![Downloads](https://img.shields.io/gem/dt/semantic_logger.svg)](https://rubygems.org/gems/semantic_logger) [![License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](http://opensource.org/licenses/Apache-2.0) ![](https://img.shields.io/badge/status-Production%20Ready-blue.svg)
 
 Semantic Logger is a feature rich logging framework, and replacement for existing Ruby & Rails loggers.  
 
-* https://rocketjob.github.io/semantic_logger/
+* https://logger.rocketjob.io/
 
 ## Documentation
 
-[Semantic Logger Guide](http://rocketjob.github.io/semantic_logger)
-
-[Reference Documentation](http://www.rubydoc.info/gems/semantic_logger/)
-
-## Upgrading to Semantic Logger v4.4
-
-With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
-workaround for Ruby 2.5 crashes is no longer needed. 
-I.e. Please remove the following line if being called anywhere:
-
-~~~ruby
-SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
-~~~
+[Semantic Logger Guide](https://logger.rocketjob.io/)
 
 ## Logging Destinations
 
@@ -34,7 +22,7 @@ Logging to the following destinations are all supported "out-of-the-box":
 * Splunk
 * MongoDB
 * Honeybadger
-* Sentry
+* Sentry (both with legacy `sentry-raven` and modern `sentry-ruby` gem)
 * HTTP
 * TCP
 * UDP
@@ -50,7 +38,7 @@ handles saving log information to multiple destinations / appenders.
 
 ## Rails
 
-When running Rails, use [rails_semantic_logger](http://github.com/rocketjob/rails_semantic_logger)
+When running Rails, use [rails_semantic_logger](http://github.com/reidmorrison/rails_semantic_logger)
 instead of Semantic Logger directly since it will automatically replace the Rails default logger with Semantic Logger.
 
 ## Rocket Job
@@ -72,11 +60,53 @@ and are therefore not automatically included by this gem:
 - Splunk Appender: gem 'splunk-sdk-ruby'
 - Elasticsearch Appender: gem 'elasticsearch'
 - Kafka Appender: gem 'ruby-kafka'
+- Legacy Sentry Appender: gem 'sentry-raven' (deprecated)
+- Sentry Appender: gem 'sentry-ruby'
+
+## Upgrading to Semantic Logger v4.9
+
+These changes should not be noticeable by the majority of users of Semantic Logger, since
+they are to the internal API. It is possible that advanced users may be using these internal
+API's directly.
+
+This does not affect any calls to the public api `SemanticLogger.add_appender`.
+
+File and IO are now separate appenders. When creating the File appender explicitly, its arguments
+have changed. For example, when requesting an IO stream, it needs to be changed from:
+
+~~~ruby
+SemanticLogger::Appender::File.new(io: $stderr)
+~~~
+to:
+~~~ruby
+SemanticLogger::Appender::IO.new($stderr)
+~~~
+
+Additionally, this needs to be changed from:
+~~~ruby
+SemanticLogger::Appender::File.new(file_name: "file.log")
+~~~
+to:
+~~~ruby
+SemanticLogger::Appender::File.new("file.log")
+~~~
+
+Rails Semantic Logger, if used, needs to be upgraded to v4.9 when upgrading to Semantic Logger v4.9.
+
+## Upgrading to Semantic Logger v4.4
+
+With some forking frameworks it is necessary to call `reopen` after the fork. With v4.4 the
+workaround for Ruby 2.5 crashes is no longer needed.
+I.e. Please remove the following line if being called anywhere:
+
+~~~ruby
+SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
+~~~
 
-## V4 Upgrade notes
+## Upgrading to Semantic Logger v4.0
 
 The following changes need to be made when upgrading to V4:
-- Ruby V2.1 / JRuby V9.1 is now the minimum runtime version.
+- Ruby V2.3 / JRuby V9.1 is now the minimum runtime version.
 - Replace calls to Logger#with_payload with SemanticLogger.named_tagged.
 - Replace calls to Logger#payload with SemanticLogger.named_tags.
 - MongoDB Appender requires Mongo Ruby Client V2 or greater.
@@ -129,13 +159,13 @@ SemanticLogger.default_level = :trace
 SemanticLogger.add_appender(file_name: 'development.log', formatter: :color)
 ~~~
 
-If running rails, see: [Semantic Logger Rails](http://rocketjob.github.io/semantic_logger/rails.html)
+If running rails, see: [Semantic Logger Rails](https://logger.rocketjob.io/rails.html)
 
 ## Author
 
 [Reid Morrison](https://github.com/reidmorrison)
 
-[Contributors](https://github.com/rocketjob/semantic_logger/graphs/contributors)
+[Contributors](https://github.com/reidmorrison/semantic_logger/graphs/contributors)
 
 ## Versioning
 

data/Rakefile

@@ -1,22 +1,22 @@
-require 'rake/clean'
-require 'rake/testtask'
+require "rake/clean"
+require "rake/testtask"
 
-$LOAD_PATH.unshift File.expand_path('lib', __dir__)
-require 'semantic_logger/version'
+$LOAD_PATH.unshift File.expand_path("lib", __dir__)
+require "semantic_logger/version"
 
 task :gem do
-  system 'gem build semantic_logger.gemspec'
+  system "gem build semantic_logger.gemspec"
 end
 
 task publish: :gem do
   system "git tag -a v#{SemanticLogger::VERSION} -m 'Tagging #{SemanticLogger::VERSION}'"
-  system 'git push --tags'
+  system "git push --tags"
   system "gem push semantic_logger-#{SemanticLogger::VERSION}.gem"
   system "rm semantic_logger-#{SemanticLogger::VERSION}.gem"
 end
 
 Rake::TestTask.new(:test) do |t|
-  t.pattern = 'test/**/*_test.rb'
+  t.pattern = "test/**/*_test.rb"
   t.verbose = true
   t.warning = false
 end

data/lib/semantic_logger/ansi_colors.rb

@@ -11,15 +11,5 @@ module SemanticLogger
     MAGENTA = "\e[35m".freeze
     CYAN    = "\e[36m".freeze
     WHITE   = "\e[37m".freeze
-
-    # DEPRECATED - NOT USED
-    LEVEL_MAP = {
-      trace: MAGENTA,
-      debug: GREEN,
-      info:  CYAN,
-      warn:  BOLD,
-      error: RED,
-      fatal: RED
-    }.freeze
   end
 end

data/lib/semantic_logger/appender/async.rb

@@ -1,4 +1,4 @@
-require 'forwardable'
+require "forwardable"
 
 module SemanticLogger
   module Appender
@@ -15,6 +15,7 @@ module SemanticLogger
       def_delegator :@appender, :filter
       def_delegator :@appender, :host
       def_delegator :@appender, :application
+      def_delegator :@appender, :environment
       def_delegator :@appender, :level
       def_delegator :@appender, :level=
       def_delegator :@appender, :logger
@@ -52,13 +53,13 @@ module SemanticLogger
       # Re-open appender after a fork
       def reopen
         # Workaround CRuby crash on fork by recreating queue on reopen
-        #   https://github.com/rocketjob/semantic_logger/issues/103
+        #   https://github.com/reidmorrison/semantic_logger/issues/103
         @queue&.close
         create_queue
 
         appender.reopen if appender.respond_to?(:reopen)
 
-        @thread.kill if @thread&.alive?
+        @thread&.kill if @thread&.alive?
         @thread = Thread.new { process }
       end
 
@@ -72,6 +73,7 @@ module SemanticLogger
       # Starts the worker thread if not running.
       def thread
         return @thread if @thread&.alive?
+
         @thread = Thread.new { process }
       end
 
@@ -114,21 +116,21 @@ module SemanticLogger
         # This thread is designed to never go down unless the main thread terminates
         # or the appender is closed.
         Thread.current.name = logger.name
-        logger.trace 'Async: Appender thread active'
+        logger.trace "Async: Appender thread active"
         begin
           process_messages
-        rescue StandardError => exception
+        rescue StandardError => e
           # This block may be called after the file handles have been released by Ruby
           begin
-            logger.error('Async: Restarting due to exception', exception)
+            logger.error("Async: Restarting due to exception", e)
           rescue StandardError
             nil
           end
           retry
-        rescue Exception => exception
+        rescue Exception => e
           # This block may be called after the file handles have been released by Ruby
           begin
-            logger.error('Async: Stopping due to fatal exception', exception)
+            logger.error("Async: Stopping due to fatal exception", e)
           rescue StandardError
             nil
           end
@@ -136,7 +138,7 @@ module SemanticLogger
           @thread = nil
           # This block may be called after the file handles have been released by Ruby
           begin
-            logger.trace('Async: Thread has stopped')
+            logger.trace("Async: Thread has stopped")
           rescue StandardError
             nil
           end
@@ -158,7 +160,7 @@ module SemanticLogger
             break unless process_message(message)
           end
         end
-        logger.trace 'Async: Queue Closed'
+        logger.trace "Async: Queue Closed"
       end
 
       # Returns false when message processing should be stopped

data/lib/semantic_logger/appender/async_batch.rb

@@ -1,4 +1,4 @@
-require 'concurrent'
+require "concurrent"
 
 module SemanticLogger
   module Appender
@@ -42,7 +42,9 @@ module SemanticLogger
           lag_threshold_s: lag_threshold_s
         )
 
-        raise(ArgumentError, "#{appender.class.name} does not support batching. It must implement #batch") unless appender.respond_to?(:batch)
+        return if appender.respond_to?(:batch)
+
+        raise(ArgumentError, "#{appender.class.name} does not support batching. It must implement #batch")
       end
 
       # Add log message for processing.
@@ -62,6 +64,7 @@ module SemanticLogger
           signal.wait(batch_seconds)
 
           logs          = []
+          messages      = []
           first         = true
           message_count = queue.length
           message_count.times do
@@ -74,10 +77,11 @@ module SemanticLogger
                 first = false
               end
             else
-              process_message(message)
+              messages << message
             end
           end
           appender.batch(logs) if logs.size.positive?
+          messages.each { |message| process_message(message) }
           signal.reset unless queue.size >= batch_size
         end
       end

data/lib/semantic_logger/appender/bugsnag.rb

@@ -1,7 +1,7 @@
 begin
-  require 'bugsnag'
+  require "bugsnag"
 rescue LoadError
-  raise 'Gem bugsnag is required for logging purposes. Please add the gem "bugsnag" to your Gemfile.'
+  raise LoadError, 'Gem bugsnag is required for logging purposes. Please add the gem "bugsnag" to your Gemfile.'
 end
 
 # Send log messages to Bugsnag
@@ -30,7 +30,7 @@ module SemanticLogger
       #     Proc: Only include log messages where the supplied Proc returns true
       #           The Proc must return true or false.
       def initialize(level: :error, **args, &block)
-        raise 'Bugsnag only supports :info, :warn, or :error log levels' unless %i[info warn error fatal].include?(level)
+        raise "Bugsnag only supports :info, :warn, or :error log levels" unless %i[info warn error fatal].include?(level)
 
         # Replace the Bugsnag logger so that we can identify its log messages and not forward them to Bugsnag
         ::Bugsnag.configure { |config| config.logger = SemanticLogger[Bugsnag] }
@@ -40,48 +40,61 @@ module SemanticLogger
 
       # Returns [Hash] of parameters to send to Bugsnag.
       def call(log, logger)
-        h            = SemanticLogger::Formatters::Raw.new.call(log, logger)
-        h[:severity] = log_level(log)
-        h.delete(:message) if h[:exception] && (h[:message] == h[:exception][:message])
-        h.delete(:time)
-        h.delete(:exception)
-        h
+        hash = SemanticLogger::Formatters::Raw.new.call(log, logger)
+        hash.delete(:message) if hash[:exception] && (hash[:message] == hash[:exception][:message])
+        hash.delete(:time)
+        hash.delete(:level_index)
+        hash.delete(:exception)
+        hash[:file] = "#{hash[:file]}:#{hash.delete(:line)}" if hash.key?(:file)
+        hash
       end
 
-      # Send an error notification to Bugsnag
       def log(log)
         # Ignore logs coming from Bugsnag itself
-        return false if log.name == 'Bugsnag'
+        return false if log.name == "Bugsnag"
 
         # Send error messages as Runtime exceptions
-        exception =
-          if log.exception
-            # Manually constructed Exception, without a backtrace.
-            log.exception.set_backtrace(log.backtrace) if !log.exception.backtrace && log.backtrace
-            log.exception
-          else
-            error = RuntimeError.new(log.message)
-            error.set_backtrace(log.backtrace) if log.backtrace
-            error
-          end
-
-        # For more documentation on the Bugsnag.notify method see:
-        # https://bugsnag.com/docs/notifiers/ruby#sending-handled-exceptions
-        ::Bugsnag.notify(exception, formatter.call(log, self))
+        exception = extract_exception(log)
+        hash      = formatter.call(log, self)
+        bugsnag_notify(exception, hash, log_level(log.level))
         true
       end
 
       private
 
+      def bugsnag_notify(exception, hash, level)
+        if ::Bugsnag::VERSION.to_i >= 6
+          ::Bugsnag.notify(exception) do |report|
+            report.severity = level
+            hash.each_pair { |key, value| report.add_tab(key, value) }
+          end
+        else
+          hash[:severity] = level
+          ::Bugsnag.notify(exception, hash)
+        end
+      end
+
+      def extract_exception(log)
+        if log.exception
+          # Manually constructed Exception, without a backtrace.
+          log.exception.set_backtrace(log.backtrace) if !log.exception.backtrace && log.backtrace
+          return log.exception
+        end
+
+        error = RuntimeError.new(log.message)
+        error.set_backtrace(log.backtrace) if log.backtrace
+        error
+      end
+
       # Bugsnag supports: error, warning or info
-      def log_level(log)
-        case log.level
+      def log_level(level)
+        case level
         when :error, :fatal
-          'error'
+          "error"
         when :warn
-          'warning'
+          "warning"
         else
-          'info'
+          "info"
         end
       end
     end

data/lib/semantic_logger/appender/elasticsearch.rb

@@ -1,10 +1,11 @@
 begin
-  require 'elasticsearch'
+  require "elasticsearch"
 rescue LoadError
-  raise 'Gem elasticsearch is required for logging to Elasticsearch. Please add the gem "elasticsearch" to your Gemfile.'
+  raise LoadError,
+        'Gem elasticsearch is required for logging to Elasticsearch. Please add the gem "elasticsearch" to your Gemfile.'
 end
 
-require 'date'
+require "date"
 
 # Forward all log messages to Elasticsearch.
 #
@@ -17,7 +18,8 @@ require 'date'
 module SemanticLogger
   module Appender
     class Elasticsearch < SemanticLogger::Subscriber
-      attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size, :elasticsearch_args
+      attr_accessor :url, :index, :date_pattern, :type, :client, :flush_interval, :timeout_interval, :batch_size,
+                    :elasticsearch_args
 
       # Create Elasticsearch appender over persistent HTTP(S)
       #
@@ -123,16 +125,17 @@ module SemanticLogger
       #   send_get_body_as [String]
       #     Specify the HTTP method to use for GET requests with a body.
       #     Default: 'GET'
-      def initialize(url: 'http://localhost:9200',
-                     index: 'semantic_logger',
-                     date_pattern: '%Y.%m.%d',
-                     type: 'log',
+      def initialize(url: "http://localhost:9200",
+                     index: "semantic_logger",
+                     date_pattern: "%Y.%m.%d",
+                     type: "log",
                      level: nil,
                      formatter: nil,
                      filter: nil,
                      application: nil,
+                     environment: nil,
                      host: nil,
-                     metrics: false,
+                     data_stream: false,
                      **elasticsearch_args,
                      &block)
 
@@ -143,8 +146,9 @@ module SemanticLogger
         @elasticsearch_args          = elasticsearch_args.dup
         @elasticsearch_args[:url]    = url if url && !elasticsearch_args[:hosts]
         @elasticsearch_args[:logger] = logger
+        @data_stream = data_stream
 
-        super(level: level, formatter: formatter, filter: filter, application: application, host: host, metrics: false, &block)
+        super(level: level, formatter: formatter, filter: filter, application: application, environment: environment, host: host, metrics: false, &block)
         reopen
       end
 
@@ -172,20 +176,35 @@ module SemanticLogger
       private
 
       def write_to_elasticsearch(messages)
-        bulk_result = @client.bulk(body: messages)
-        return unless bulk_result['errors']
+        bulk_result = if @data_stream
+                        @client.bulk(index: index, body: messages)
+                      else
+                        @client.bulk(body: messages)
+                      end
 
-        failed = bulk_result['items'].reject { |x| x['status'] == 201 }
+        return unless bulk_result["errors"]
+
+        failed = bulk_result["items"].reject { |x| x["status"] == 201 }
         logger.error("ElasticSearch: Write failed. Messages discarded. : #{failed}")
       end
 
       def bulk_index(log)
         expanded_index_name = log.time.strftime("#{index}-#{date_pattern}")
-        {'index' => {'_index' => expanded_index_name, '_type' => type}}
+        if @data_stream
+          {"create" => {}}
+        else
+          {"index" => {"_index" => expanded_index_name, "_type" => type}}
+        end
       end
 
       def default_formatter
-        SemanticLogger::Formatters::Raw.new(time_format: :iso_8601, time_key: :timestamp)
+        time_key = if @data_stream
+                     "@timestamp"
+                   else
+                     :timestamp
+                   end
+
+        SemanticLogger::Formatters::Raw.new(time_format: :iso_8601, time_key: time_key)
       end
     end
   end

data/lib/semantic_logger/appender/elasticsearch_http.rb

@@ -1,4 +1,4 @@
-require 'date'
+require "date"
 # Forward all log messages to Elasticsearch one at a time via a HTTP post.
 #
 # Note:
@@ -50,9 +50,9 @@ module SemanticLogger
       #   application: [String]
       #     Name of this application to appear in log messages.
       #     Default: SemanticLogger.application
-      def initialize(index: 'semantic_logger',
-                     type: 'log',
-                     url: 'http://localhost:9200',
+      def initialize(index: "semantic_logger",
+                     type: "log",
+                     url: "http://localhost:9200",
                      **http_args,
                      &block)