ezlog 0.7.0 → 0.9.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a3448fa736b7d0c38391f77c299ca38185705c50bfb9a48230166b1bf95d5cea
-  data.tar.gz: 6334e6cfbca5a00b5c10d3c272533711613d556bf4d452176a055ba876dae8f9
+  metadata.gz: 70fb33130f877332e30cbad096c74d1eeed80bba66f8c9ebedf0854f73bf95aa
+  data.tar.gz: 6bcb5959c18eb00a5065a86dd99ac02b193e8fb693cd195b201bdb07bc5e1fe4
 SHA512:
-  metadata.gz: faa9b4f39da472e3e353c8d2d0f95a36f08eae1ca631f0c1611907488da9fbd80dc3a330cde6f7f103056a34e7a0e72469e53580d36180b9119f3c5e96bc03bf
-  data.tar.gz: 24798f5e9e2d2308f443bc2c4a72612aab10e389599fa45ce8eea535bf68fb104f9da8c3c1a8f4ceda225215597ab7f85a1e314985841679376c40e5deebd06c
+  metadata.gz: c9675aea648f31869dfc248199f05cae2dad3b62aed3352c9383d7626d70cf5a7a0c302b58ddb74f04c45c6177a64e94d46ed84dcf2af55f44eba3c0c22b83ae
+  data.tar.gz: '092a73292dd85f7e8bb918b01a2e972f6c828984aaaf1230f580b456134e94f428948e61212231ee9576b11d92b44cb706ce4f54af8fd735c1eb1074f2a90e67'

data/CHANGELOG.md

@@ -1,3 +1,84 @@
+### 0.9.5 (2020-10-1)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.4...v0.9.5)
+
+* Fix
+  * fixing an issue with the Sidekiq job's log context generation:
+    using a namespaced Sidekiq worker (`SomeModule::SomeWorker`) cause the log context generations to fail with: `NameError: wrong constant name SomeModule::SomeWorker`
+
+### 0.9.4 (2020-09-26)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.3...v0.9.4)
+
+* Features & enhancements
+  * added Ruby 2.7 to the list of version CI will test the code with
+  * remove dot-files and Rakefile from the gem
+* Fix
+  * stop using `Hash#merge` with multiple arguments as it's only supported from Ruby 2.6
+
+### 0.9.3 (2020-09-20)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.2...v0.9.3)
+
+* Features & enhancements 
+    * Switching to [Oj](https://github.com/ohler55/oj) for fast JSON serialization
+    * Allow level to be formatted (so it can be logged as a number too)
+    
+        if you want to use Ougai-like numbers you can do something like this: 
+        ```ruby
+        config.ezlog.layout_options = { level_formatter: ->(level_number) { (level_number + 2) * 10 } } 
+        
+        Rails.logger.error('Boom!')
+        #=> {"logger":"Application","timestamp":"2020-09-20T19:29:03+02:00","level":50,"hostname":"BUD010256.local","pid":19872,"message":"Boom!"}
+        ``` 
+    * initial context (a context which will be added to every single line of log) can be configured via `config.ezlog.layout_options` and it defaults to `{environment: ::Rails.env}`
+    
+### 0.9.2 (2020-09-19)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.1...v0.9.2)
+
+* Features & enhancements 
+  * Improvements of the [Sidekiq](https://github.com/mperham/sidekiq) integration
+    * supports additional job information: batch id, tags and thread id (bid, tags, tid)
+    * support logging "death" events (setting up a death_handler) 
+
+### 0.9.1 (2020-05-10)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.0...v0.9.1)
+
+* Bug fixed
+  * Fixed a bug in access log exclusion pattern matching that would exclude partial matches for a path if it was 
+    specified as a string (expecting to be excluded only in case of a full match).
+
+### 0.9.0 (2020-05-10)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.8.0...v0.9.0)
+
+* Features & enhancements
+  * Ezlog now supports [Rails](https://rubyonrails.org/) 6.
+  * Added the ability to exclude certain paths from access logging. Use the `exclude_paths` configuration option to
+    add paths (strings or regexps) to exclude from your access logs.
+
+### 0.8.0 (2020-04-07)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.7.1...v0.8.0)
+
+* Bug fixes
+  * Reverted the change introduced in `v0.5.2` which extended the [Sidekiq](https://github.com/mperham/sidekiq) logger
+    interface because it caused problems with other third-party integrations
+    (e.g. [sidekiq-unique-jobs](https://github.com/mhenrixon/sidekiq-unique-jobs)).
+    [Sidekiq](https://github.com/mperham/sidekiq) itself removed this interface element in `v6.0.1`, so the current
+    change breaks compatibility with [Sidekiq](https://github.com/mperham/sidekiq) `v6.0.0` exclusively. If you're
+    using that version, please upgrade.
+
+### 0.7.1 (2020-03-12)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.7.0...v0.7.1)
+
+* Bug fixes
+  * Fixed a bug in the [Sidekiq](https://github.com/mperham/sidekiq) error handler which caused the error handler
+    to throw an additional error if there was no job hash in the original error context.
+
 ### 0.7.0 (2020-03-11)
 
 [Full Changelog](https://github.com/emartech/ezlog/compare/v0.6.0...v0.7.0)

data/README.md

@@ -6,7 +6,7 @@
 Ezlog is intended to be a zero-configuration structured logging setup for pure Ruby or [Ruby on Rails](https://rubyonrails.org/) 
 projects using any (or all) of the following libraries or frameworks:
 
-* [Ruby on Rails](https://rubyonrails.org/)
+* [Ruby on Rails](https://rubyonrails.org/) (5.x and 6.x supported)
 * [Sidekiq](https://github.com/mperham/sidekiq)
 * [Sequel](https://sequel.jeremyevans.net/) 
 * [Rack::Timeout](https://github.com/heroku/rack-timeout)
@@ -134,6 +134,12 @@ Using this configuration, Ezlog will only log the `action` parameter under the `
 all parameters serialized into a single string under the key `params_serialized`. You won't lose any information,
 but you can make sure that only the relevant parameters of your requests are searchable (and thus protect your logs).
 
+Should you want to exclude certain paths (e.g. a healthcheck URL) from your access logs, you can use the following option:
+
+```ruby
+config.ezlog.exclude_paths = ['/healthcheck', %r(/monitoring/.*)]  # default is empty so everything gets logged
+```
+
 #### The log level
 
 The logger's log level is determined as follows (in order of precedence):

data/ezlog.gemspec

@@ -16,16 +16,17 @@ Gem::Specification.new do |spec|
   }
 
   spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
-    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^((test|spec|features)/|Rakefile|\.)}) }
   end
   spec.bindir        = "exe"
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
   spec.add_dependency "logging", "~> 2.0"
+  spec.add_dependency "oj", "~> 3.0"
 
   spec.add_development_dependency "bundler", "~> 2.0"
-  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rake", ">= 12.3.3"
   spec.add_development_dependency "rspec", "~> 3.0"
   spec.add_development_dependency "sidekiq", ">= 5.0"
   spec.add_development_dependency "sequel", "~> 5.0"

data/lib/ezlog/logging_layout.rb

@@ -1,10 +1,11 @@
 require 'time'
-require 'json'
+require 'oj'
 
 module Ezlog
   class LoggingLayout < ::Logging::Layout
-    def initialize(context = {})
+    def initialize(context = {}, options = {})
       @initial_context = context
+      @level_formatter = options.fetch(:level_formatter, ->(numeric_level) { ::Logging::LNAMES[numeric_level] })
     end
 
     def format(event)
@@ -12,7 +13,7 @@ module Ezlog
       add_initial_context_to log_entry
       add_logging_context_to log_entry
       add_event_information_to log_entry, event
-      ::JSON.dump(log_entry) + "\n"
+      ::Oj.dump(log_entry, mode: :json) + "\n"
     end
 
     private
@@ -20,8 +21,8 @@ module Ezlog
     def basic_information_for(event)
       {
         'logger' => event.logger,
-        'timestamp' => event.time.iso8601,
-        'level' => ::Logging::LNAMES[event.level],
+        'timestamp' => event.time.iso8601(3),
+        'level' => @level_formatter.call(event.level),
         'hostname' => Socket.gethostname,
         'pid' => Process.pid
       }
@@ -46,17 +47,17 @@ module Ezlog
       when Hash
         obj
       else
-        {message: obj}
+        { 'message' => obj }
       end
     end
 
     def exception_message_by(exception)
       {
-        message: exception.message,
-        error: {
-          class: exception.class.name,
-          message: exception.message,
-          backtrace: exception.backtrace&.first(20)
+        'message' => exception.message,
+        'error' => {
+          'class' => exception.class.name,
+          'message' => exception.message,
+          'backtrace' => exception.backtrace&.first(20)
         }
       }
     end

data/lib/ezlog/rails/access_log.rb

@@ -3,10 +3,10 @@ module Ezlog
     class AccessLog
       include LogContextHelper
 
-      def initialize(app, logger, whitelisted_params)
+      def initialize(app, logger, config)
         @app = app
         @logger = logger
-        @whitelisted_params = whitelisted_params&.map &:to_s
+        @config = config
       end
 
       def call(env)
@@ -29,23 +29,36 @@ module Ezlog
       end
 
       def log_request(request, status)
+        return if path_ignored?(request)
+
         message = {
           message: '%s %s - %i (%s)' % [request.method, request.filtered_path, status, Rack::Utils::HTTP_STATUS_CODES[status]],
           remote_ip: request.remote_ip,
           method: request.method,
           path: request.filtered_path,
-          params: params_to_log_in(request),
+          params: params_to_log(request),
           response_status_code: status
         }
-        message.merge! params_serialized: request.filtered_parameters.inspect if @whitelisted_params
+        message.merge! params_serialized: request.filtered_parameters.inspect if @config.log_only_whitelisted_params
         @logger.info message
       end
 
-      def params_to_log_in(request)
-        if @whitelisted_params.nil?
-          request.filtered_parameters
+      def path_ignored?(request)
+        @config.exclude_paths.any? do |pattern|
+          case pattern
+          when Regexp
+            pattern.match? request.path
+          else
+            pattern == request.path
+          end
+        end
+      end
+
+      def params_to_log(request)
+        if @config.log_only_whitelisted_params
+          request.filtered_parameters.slice *@config.whitelisted_params&.map(&:to_s)
         else
-          request.filtered_parameters.slice *@whitelisted_params
+          request.filtered_parameters
         end
       end
     end

data/lib/ezlog/railtie.rb

@@ -4,13 +4,18 @@ module Ezlog
     config.ezlog.enable_sequel_logging = false
     config.ezlog.log_only_whitelisted_params = false
     config.ezlog.whitelisted_params = [:controller, :action]
+    config.ezlog.exclude_paths = []
+    config.ezlog.initial_context = { environment: ::Rails.env }
+    config.ezlog.layout_options = {}
 
     initializer "ezlog.initialize" do
       require "ezlog/rails/extensions"
     end
 
     initializer 'ezlog.configure_logging' do |app|
-      ::Logging.logger.root.appenders = ::Logging.appenders.stdout 'stdout', layout: Ezlog::LoggingLayout.new(environment: ::Rails.env)
+      ::Logging.logger.root.appenders =
+        ::Logging.appenders.stdout 'stdout', layout: Ezlog::LoggingLayout.new(app.config.ezlog.initial_context,
+                                                                              app.config.ezlog.layout_options)
       ::Logging.logger.root.level = app.config.log_level
     end
 
@@ -29,16 +34,26 @@ module Ezlog
     initializer 'ezlog.configure_rails_middlewares' do |app|
       app.config.middleware.insert_after ::ActionDispatch::RequestId, Ezlog::Rails::RequestLogContext
       app.config.middleware.delete ::Rails::Rack::Logger
-      app.config.middleware.insert_before ::ActionDispatch::DebugExceptions, Ezlog::Rails::AccessLog, Ezlog.logger('AccessLog'), whitelisted_params(app)
+      app.config.middleware.insert_before ::ActionDispatch::DebugExceptions, Ezlog::Rails::AccessLog, Ezlog.logger('AccessLog'), config.ezlog
       app.config.middleware.insert_after ::ActionDispatch::DebugExceptions, Ezlog::Rails::LogExceptions, Ezlog.logger('Application')
     end
 
     config.after_initialize do
-      Ezlog::Rails::LogSubscriber.detach ::ActionController::LogSubscriber
-      Ezlog::Rails::LogSubscriber.detach ::ActionView::LogSubscriber
-      if defined? ::ActiveRecord
-        Ezlog::Rails::LogSubscriber.detach ::ActiveRecord::LogSubscriber
-        Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+      case ::Rails::VERSION::MAJOR
+      when 6
+        ::ActionController::LogSubscriber.detach_from :action_controller
+        ::ActionView::LogSubscriber.detach_from :action_view
+        if defined? ::ActiveRecord
+          ::ActiveRecord::LogSubscriber.detach_from :active_record
+          Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+        end
+      else
+        Ezlog::Rails::LogSubscriber.detach ::ActionController::LogSubscriber
+        Ezlog::Rails::LogSubscriber.detach ::ActionView::LogSubscriber
+        if defined? ::ActiveRecord
+          Ezlog::Rails::LogSubscriber.detach ::ActiveRecord::LogSubscriber
+          Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+        end
       end
     end
 
@@ -50,21 +65,18 @@ module Ezlog
     private
 
     def initialize_sidekiq_logging(app)
-      ::Sidekiq.logger = Ezlog.logger('Sidekiq').extend Ezlog::Sidekiq::LoggerExtension
+      ::Sidekiq.logger = Ezlog.logger('Sidekiq')
       ::Sidekiq.logger.level = app.config.log_level
       ::Sidekiq.configure_server do |config|
         config.options[:job_logger] = Ezlog::Sidekiq::JobLogger
         config.error_handlers << Ezlog::Sidekiq::ErrorLogger.new
         config.error_handlers.delete_if { |handler| handler.is_a? ::Sidekiq::ExceptionHandler::Logger }
+        config.death_handlers << Ezlog::Sidekiq::DeathLogger.new
       end
     end
 
     def disable_rack_timeout_logging
       ::Rack::Timeout::Logger.logger = ::Logger.new(nil)
     end
-
-    def whitelisted_params(app)
-      app.config.ezlog.log_only_whitelisted_params ? app.config.ezlog.whitelisted_params : nil
-    end
   end
 end

data/lib/ezlog/sidekiq.rb

@@ -1,8 +1,8 @@
 module Ezlog
   module Sidekiq
+    autoload :DeathLogger, 'ezlog/sidekiq/death_logger'
     autoload :ErrorLogger, 'ezlog/sidekiq/error_logger'
     autoload :JobContext, 'ezlog/sidekiq/job_context'
     autoload :JobLogger, 'ezlog/sidekiq/job_logger'
-    autoload :LoggerExtension, 'ezlog/sidekiq/logger_extension'
   end
 end

data/lib/ezlog/sidekiq/death_logger.rb

@@ -0,0 +1,15 @@
+require 'sidekiq'
+
+module Ezlog
+  module Sidekiq
+    class DeathLogger
+      include LogContextHelper
+
+      def call(job, error)
+        within_log_context(JobContext.from_job_hash(job)) do
+          ::Sidekiq.logger.error error
+        end
+      end
+    end
+  end
+end

data/lib/ezlog/sidekiq/job_context.rb

@@ -3,13 +3,19 @@ module Ezlog
     class JobContext
       class << self
         def from_job_hash(job_hash)
-          basic_info_from(job_hash).merge named_arguments_from(job_hash)
+          return {} if job_hash.nil?
+          thread_info.merge(basic_info_from(job_hash)).merge(named_arguments_from(job_hash))
         end
 
         private
 
+        def thread_info
+          { tid: Thread.current['sidekiq_tid'] || (Thread.current.object_id ^ ::Process.pid).to_s(36) }
+        end
+
+
         def basic_info_from(job)
-          {
+          h = {
             jid: job['jid'],
             queue: job['queue'],
             worker: job_class(job),
@@ -17,6 +23,9 @@ module Ezlog
             enqueued_at: job['enqueued_at'],
             run_count: (job['retry_count'] || -1) + 2
           }
+          h[:bid] = job['bid'] if job['bid']
+          h[:tags] = job['tags'] if job['tags']
+          h
         end
 
         def named_arguments_from(job)
@@ -28,7 +37,7 @@ module Ezlog
         end
 
         def method_parameters_of(job)
-          Kernel.const_get(job_class(job).to_sym).instance_method(:perform).parameters
+          Kernel.const_get(job_class(job)).instance_method(:perform).parameters
         end
 
         def job_class(job)

data/lib/ezlog/version.rb

@@ -1,3 +1,3 @@
 module Ezlog
-  VERSION = '0.7.0'
+  VERSION = '0.9.5'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ezlog
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.9.5
 platform: ruby
 authors:
 - Zoltan Ormandi
-autorequire: 
+autorequire:
 bindir: exe
 cert_chain: []
-date: 2020-03-11 00:00:00.000000000 Z
+date: 2020-10-06 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: logging
@@ -24,6 +24,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: oj
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -42,16 +56,16 @@ dependencies:
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: 12.3.3
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: 12.3.3
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -122,21 +136,17 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '5.0'
-description: 
+description:
 email:
 - zoltan.ormandi@emarsys.com
 executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- ".gitignore"
-- ".rspec"
-- ".travis.yml"
 - CHANGELOG.md
 - Gemfile
 - LICENSE.txt
 - README.md
-- Rakefile
 - bin/console
 - bin/setup
 - ezlog.gemspec
@@ -156,10 +166,10 @@ files:
 - lib/ezlog/rspec/helpers.rb
 - lib/ezlog/rspec/matchers.rb
 - lib/ezlog/sidekiq.rb
+- lib/ezlog/sidekiq/death_logger.rb
 - lib/ezlog/sidekiq/error_logger.rb
 - lib/ezlog/sidekiq/job_context.rb
 - lib/ezlog/sidekiq/job_logger.rb
-- lib/ezlog/sidekiq/logger_extension.rb
 - lib/ezlog/version.rb
 - lib/sequel/extensions/ezlog_logging.rb
 homepage: https://github.com/emartech/ezlog
@@ -167,7 +177,7 @@ licenses:
 - MIT
 metadata:
   changelog_uri: https://github.com/emartech/ezlog/blob/master/CHANGELOG.md
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -182,8 +192,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.3
-signing_key: 
+rubygems_version: 3.1.2
+signing_key:
 specification_version: 4
 summary: A zero-configuration logging solution for projects using Sidekiq, Rails,
   Sequel, etc.

data/.gitignore

@@ -1,13 +0,0 @@
-/.bundle/
-/.yardoc
-/_yardoc/
-/coverage/
-/doc/
-/pkg/
-/spec/reports/
-/tmp/
-
-# rspec failure tracking
-.rspec_status
-.ruby-version
-Gemfile.lock

data/.rspec

@@ -1,3 +0,0 @@
---format progress
---color
---require spec_helper

data/.travis.yml

@@ -1,9 +0,0 @@
----
-sudo: false
-language: ruby
-cache: bundler
-rvm:
-  - 2.4.6
-  - 2.5.5
-  - 2.6.3
-before_install: gem install bundler -v 2.0.1

data/Rakefile

@@ -1,6 +0,0 @@
-require "bundler/gem_tasks"
-require "rspec/core/rake_task"
-
-RSpec::Core::RakeTask.new(:spec)
-
-task :default => :spec

data/lib/ezlog/sidekiq/logger_extension.rb

@@ -1,11 +0,0 @@
-module Ezlog
-  module Sidekiq
-    module LoggerExtension
-      include LogContextHelper
-
-      def with_context(context, &block)
-        within_log_context context, &block
-      end
-    end
-  end
-end