ezlog 0.7.1 → 0.9.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8ca6993d4ad6c1fcd07d3d4895b2be3f2bdfde37eb3463c0c09f6480676eb439
-  data.tar.gz: a802402d7225019622d87d9990b71088f6cbd01d770641d1835dc310d301fd38
+  metadata.gz: 178b2810c13ba87a8a1e389a6679e14f06e4b4f05bcc0a84dea797110cfde3a7
+  data.tar.gz: ae3e3abfbf692e3189a5d6e1bfb1efe4ec3ce064e1e61285a2c13b27922e13fe
 SHA512:
-  metadata.gz: 8eed8c43fa2cdd21cd8f29d49cb86351b33afd054419111a3d98a7564696721f0f67ddbd17ad38be0b421bee6bc55138799014c692a42770061f2a29f707ece7
-  data.tar.gz: 9035a7299dae11147aec96510d49549727fcaed0cb59e43be562dd85710277424a7a213364ff763e3611faca2ebf138c03e623517ce5200b891a691b924bdcdc
+  metadata.gz: 21d6c1fa52dd9b46bbc33fc776d139d3c7c38492cb8f7f47bcfc577c47c53a8014197712993ece0a53807646fff222acca23e803df8af74c32ee820b4b895f7c
+  data.tar.gz: 2ba5df5dcb18e3553f97f4727556626ad386e2e2c4dc16aefe261d3502cbe33d9fc163c439da8aa3a8392359eba43f15ea5a5da7501892da7e6c7e9e149b0553

data/CHANGELOG.md

@@ -1,3 +1,83 @@
+### 0.9.6 (2021-06-30)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.5...v0.9.6)
+
+* Fix
+  * fixing a Rails 6.1 specific issue with ActiveRecord array parameter in a query (`User.where(id: [1,2,3,4])`)
+
+### 0.9.5 (2020-10-01)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.4...v0.9.5)
+
+* Fix
+  * fixing an issue with the Sidekiq job's log context generation:
+    using a namespaced Sidekiq worker (`SomeModule::SomeWorker`) cause the log context generations to fail with: `NameError: wrong constant name SomeModule::SomeWorker`
+
+### 0.9.4 (2020-09-26)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.3...v0.9.4)
+
+* Features & enhancements
+  * added Ruby 2.7 to the list of version CI will test the code with
+  * remove dot-files and Rakefile from the gem
+* Fix
+  * stop using `Hash#merge` with multiple arguments as it's only supported from Ruby 2.6
+
+### 0.9.3 (2020-09-20)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.2...v0.9.3)
+
+* Features & enhancements 
+    * Switching to [Oj](https://github.com/ohler55/oj) for fast JSON serialization
+    * Allow level to be formatted (so it can be logged as a number too)
+    
+        if you want to use Ougai-like numbers you can do something like this: 
+        ```ruby
+        config.ezlog.layout_options = { level_formatter: ->(level_number) { (level_number + 2) * 10 } } 
+        
+        Rails.logger.error('Boom!')
+        #=> {"logger":"Application","timestamp":"2020-09-20T19:29:03+02:00","level":50,"hostname":"BUD010256.local","pid":19872,"message":"Boom!"}
+        ``` 
+    * initial context (a context which will be added to every single line of log) can be configured via `config.ezlog.layout_options` and it defaults to `{environment: ::Rails.env}`
+    
+### 0.9.2 (2020-09-19)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.1...v0.9.2)
+
+* Features & enhancements 
+  * Improvements of the [Sidekiq](https://github.com/mperham/sidekiq) integration
+    * supports additional job information: batch id, tags and thread id (bid, tags, tid)
+    * support logging "death" events (setting up a death_handler) 
+
+### 0.9.1 (2020-05-10)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.0...v0.9.1)
+
+* Bug fixed
+  * Fixed a bug in access log exclusion pattern matching that would exclude partial matches for a path if it was 
+    specified as a string (expecting to be excluded only in case of a full match).
+
+### 0.9.0 (2020-05-10)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.8.0...v0.9.0)
+
+* Features & enhancements
+  * Ezlog now supports [Rails](https://rubyonrails.org/) 6.
+  * Added the ability to exclude certain paths from access logging. Use the `exclude_paths` configuration option to
+    add paths (strings or regexps) to exclude from your access logs.
+
+### 0.8.0 (2020-04-07)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.7.1...v0.8.0)
+
+* Bug fixes
+  * Reverted the change introduced in `v0.5.2` which extended the [Sidekiq](https://github.com/mperham/sidekiq) logger
+    interface because it caused problems with other third-party integrations
+    (e.g. [sidekiq-unique-jobs](https://github.com/mhenrixon/sidekiq-unique-jobs)).
+    [Sidekiq](https://github.com/mperham/sidekiq) itself removed this interface element in `v6.0.1`, so the current
+    change breaks compatibility with [Sidekiq](https://github.com/mperham/sidekiq) `v6.0.0` exclusively. If you're
+    using that version, please upgrade.
+
 ### 0.7.1 (2020-03-12)
 
 [Full Changelog](https://github.com/emartech/ezlog/compare/v0.7.0...v0.7.1)

data/README.md

@@ -6,7 +6,7 @@
 Ezlog is intended to be a zero-configuration structured logging setup for pure Ruby or [Ruby on Rails](https://rubyonrails.org/) 
 projects using any (or all) of the following libraries or frameworks:
 
-* [Ruby on Rails](https://rubyonrails.org/)
+* [Ruby on Rails](https://rubyonrails.org/) (5.x and 6.x supported)
 * [Sidekiq](https://github.com/mperham/sidekiq)
 * [Sequel](https://sequel.jeremyevans.net/) 
 * [Rack::Timeout](https://github.com/heroku/rack-timeout)
@@ -134,6 +134,12 @@ Using this configuration, Ezlog will only log the `action` parameter under the `
 all parameters serialized into a single string under the key `params_serialized`. You won't lose any information,
 but you can make sure that only the relevant parameters of your requests are searchable (and thus protect your logs).
 
+Should you want to exclude certain paths (e.g. a healthcheck URL) from your access logs, you can use the following option:
+
+```ruby
+config.ezlog.exclude_paths = ['/healthcheck', %r(/monitoring/.*)]  # default is empty so everything gets logged
+```
+
 #### The log level
 
 The logger's log level is determined as follows (in order of precedence):

data/ezlog.gemspec

@@ -16,16 +16,17 @@ Gem::Specification.new do |spec|
   }
 
   spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
-    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^((test|spec|features)/|Rakefile|\.)}) }
   end
   spec.bindir        = "exe"
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
   spec.add_dependency "logging", "~> 2.0"
+  spec.add_dependency "oj", "~> 3.0"
 
   spec.add_development_dependency "bundler", "~> 2.0"
-  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rake", ">= 12.3.3"
   spec.add_development_dependency "rspec", "~> 3.0"
   spec.add_development_dependency "sidekiq", ">= 5.0"
   spec.add_development_dependency "sequel", "~> 5.0"

data/lib/ezlog/logging_layout.rb

@@ -1,10 +1,11 @@
 require 'time'
-require 'json'
+require 'oj'
 
 module Ezlog
   class LoggingLayout < ::Logging::Layout
-    def initialize(context = {})
+    def initialize(context = {}, options = {})
       @initial_context = context
+      @level_formatter = options.fetch(:level_formatter, ->(numeric_level) { ::Logging::LNAMES[numeric_level] })
     end
 
     def format(event)
@@ -12,7 +13,7 @@ module Ezlog
       add_initial_context_to log_entry
       add_logging_context_to log_entry
       add_event_information_to log_entry, event
-      ::JSON.dump(log_entry) + "\n"
+      ::Oj.dump(log_entry, mode: :json) + "\n"
     end
 
     private
@@ -20,8 +21,8 @@ module Ezlog
     def basic_information_for(event)
       {
         'logger' => event.logger,
-        'timestamp' => event.time.iso8601,
-        'level' => ::Logging::LNAMES[event.level],
+        'timestamp' => event.time.iso8601(3),
+        'level' => @level_formatter.call(event.level),
         'hostname' => Socket.gethostname,
         'pid' => Process.pid
       }
@@ -46,17 +47,17 @@ module Ezlog
       when Hash
         obj
       else
-        {message: obj}
+        { 'message' => obj }
       end
     end
 
     def exception_message_by(exception)
       {
-        message: exception.message,
-        error: {
-          class: exception.class.name,
-          message: exception.message,
-          backtrace: exception.backtrace&.first(20)
+        'message' => exception.message,
+        'error' => {
+          'class' => exception.class.name,
+          'message' => exception.message,
+          'backtrace' => exception.backtrace&.first(20)
         }
       }
     end

data/lib/ezlog/rails/access_log.rb

@@ -3,10 +3,10 @@ module Ezlog
     class AccessLog
       include LogContextHelper
 
-      def initialize(app, logger, whitelisted_params)
+      def initialize(app, logger, config)
         @app = app
         @logger = logger
-        @whitelisted_params = whitelisted_params&.map &:to_s
+        @config = config
       end
 
       def call(env)
@@ -29,23 +29,36 @@ module Ezlog
       end
 
       def log_request(request, status)
+        return if path_ignored?(request)
+
         message = {
           message: '%s %s - %i (%s)' % [request.method, request.filtered_path, status, Rack::Utils::HTTP_STATUS_CODES[status]],
           remote_ip: request.remote_ip,
           method: request.method,
           path: request.filtered_path,
-          params: params_to_log_in(request),
+          params: params_to_log(request),
           response_status_code: status
         }
-        message.merge! params_serialized: request.filtered_parameters.inspect if @whitelisted_params
+        message.merge! params_serialized: request.filtered_parameters.inspect if @config.log_only_whitelisted_params
         @logger.info message
       end
 
-      def params_to_log_in(request)
-        if @whitelisted_params.nil?
-          request.filtered_parameters
+      def path_ignored?(request)
+        @config.exclude_paths.any? do |pattern|
+          case pattern
+          when Regexp
+            pattern.match? request.path
+          else
+            pattern == request.path
+          end
+        end
+      end
+
+      def params_to_log(request)
+        if @config.log_only_whitelisted_params
+          request.filtered_parameters.slice *@config.whitelisted_params&.map(&:to_s)
         else
-          request.filtered_parameters.slice *@whitelisted_params
+          request.filtered_parameters
         end
       end
     end

data/lib/ezlog/rails/active_record/log_subscriber.rb

@@ -33,6 +33,8 @@ module Ezlog
           end
 
           Hash[param_value_pairs]
+        rescue NoMethodError
+          params
         end
 
         def type_casted_values_from(event)

data/lib/ezlog/railtie.rb

@@ -4,13 +4,18 @@ module Ezlog
     config.ezlog.enable_sequel_logging = false
     config.ezlog.log_only_whitelisted_params = false
     config.ezlog.whitelisted_params = [:controller, :action]
+    config.ezlog.exclude_paths = []
+    config.ezlog.initial_context = { environment: ::Rails.env }
+    config.ezlog.layout_options = {}
 
     initializer "ezlog.initialize" do
       require "ezlog/rails/extensions"
     end
 
     initializer 'ezlog.configure_logging' do |app|
-      ::Logging.logger.root.appenders = ::Logging.appenders.stdout 'stdout', layout: Ezlog::LoggingLayout.new(environment: ::Rails.env)
+      ::Logging.logger.root.appenders =
+        ::Logging.appenders.stdout 'stdout', layout: Ezlog::LoggingLayout.new(app.config.ezlog.initial_context,
+                                                                              app.config.ezlog.layout_options)
       ::Logging.logger.root.level = app.config.log_level
     end
 
@@ -29,16 +34,26 @@ module Ezlog
     initializer 'ezlog.configure_rails_middlewares' do |app|
       app.config.middleware.insert_after ::ActionDispatch::RequestId, Ezlog::Rails::RequestLogContext
       app.config.middleware.delete ::Rails::Rack::Logger
-      app.config.middleware.insert_before ::ActionDispatch::DebugExceptions, Ezlog::Rails::AccessLog, Ezlog.logger('AccessLog'), whitelisted_params(app)
+      app.config.middleware.insert_before ::ActionDispatch::DebugExceptions, Ezlog::Rails::AccessLog, Ezlog.logger('AccessLog'), config.ezlog
       app.config.middleware.insert_after ::ActionDispatch::DebugExceptions, Ezlog::Rails::LogExceptions, Ezlog.logger('Application')
     end
 
     config.after_initialize do
-      Ezlog::Rails::LogSubscriber.detach ::ActionController::LogSubscriber
-      Ezlog::Rails::LogSubscriber.detach ::ActionView::LogSubscriber
-      if defined? ::ActiveRecord
-        Ezlog::Rails::LogSubscriber.detach ::ActiveRecord::LogSubscriber
-        Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+      case ::Rails::VERSION::MAJOR
+      when 6
+        ::ActionController::LogSubscriber.detach_from :action_controller
+        ::ActionView::LogSubscriber.detach_from :action_view
+        if defined? ::ActiveRecord
+          ::ActiveRecord::LogSubscriber.detach_from :active_record
+          Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+        end
+      else
+        Ezlog::Rails::LogSubscriber.detach ::ActionController::LogSubscriber
+        Ezlog::Rails::LogSubscriber.detach ::ActionView::LogSubscriber
+        if defined? ::ActiveRecord
+          Ezlog::Rails::LogSubscriber.detach ::ActiveRecord::LogSubscriber
+          Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+        end
       end
     end
 
@@ -50,21 +65,18 @@ module Ezlog
     private
 
     def initialize_sidekiq_logging(app)
-      ::Sidekiq.logger = Ezlog.logger('Sidekiq').extend Ezlog::Sidekiq::LoggerExtension
+      ::Sidekiq.logger = Ezlog.logger('Sidekiq')
       ::Sidekiq.logger.level = app.config.log_level
       ::Sidekiq.configure_server do |config|
         config.options[:job_logger] = Ezlog::Sidekiq::JobLogger
         config.error_handlers << Ezlog::Sidekiq::ErrorLogger.new
         config.error_handlers.delete_if { |handler| handler.is_a? ::Sidekiq::ExceptionHandler::Logger }
+        config.death_handlers << Ezlog::Sidekiq::DeathLogger.new
       end
     end
 
     def disable_rack_timeout_logging
       ::Rack::Timeout::Logger.logger = ::Logger.new(nil)
     end
-
-    def whitelisted_params(app)
-      app.config.ezlog.log_only_whitelisted_params ? app.config.ezlog.whitelisted_params : nil
-    end
   end
 end

data/lib/ezlog/sidekiq.rb

@@ -1,8 +1,8 @@
 module Ezlog
   module Sidekiq
+    autoload :DeathLogger, 'ezlog/sidekiq/death_logger'
     autoload :ErrorLogger, 'ezlog/sidekiq/error_logger'
     autoload :JobContext, 'ezlog/sidekiq/job_context'
     autoload :JobLogger, 'ezlog/sidekiq/job_logger'
-    autoload :LoggerExtension, 'ezlog/sidekiq/logger_extension'
   end
 end

data/lib/ezlog/sidekiq/death_logger.rb

@@ -0,0 +1,15 @@
+require 'sidekiq'
+
+module Ezlog
+  module Sidekiq
+    class DeathLogger
+      include LogContextHelper
+
+      def call(job, error)
+        within_log_context(JobContext.from_job_hash(job)) do
+          ::Sidekiq.logger.error error
+        end
+      end
+    end
+  end
+end

data/lib/ezlog/sidekiq/job_context.rb

@@ -4,13 +4,18 @@ module Ezlog
       class << self
         def from_job_hash(job_hash)
           return {} if job_hash.nil?
-          basic_info_from(job_hash).merge named_arguments_from(job_hash)
+          thread_info.merge(basic_info_from(job_hash)).merge(named_arguments_from(job_hash))
         end
 
         private
 
+        def thread_info
+          { tid: Thread.current['sidekiq_tid'] || (Thread.current.object_id ^ ::Process.pid).to_s(36) }
+        end
+
+
         def basic_info_from(job)
-          {
+          h = {
             jid: job['jid'],
             queue: job['queue'],
             worker: job_class(job),
@@ -18,6 +23,9 @@ module Ezlog
             enqueued_at: job['enqueued_at'],
             run_count: (job['retry_count'] || -1) + 2
           }
+          h[:bid] = job['bid'] if job['bid']
+          h[:tags] = job['tags'] if job['tags']
+          h
         end
 
         def named_arguments_from(job)
@@ -29,7 +37,7 @@ module Ezlog
         end
 
         def method_parameters_of(job)
-          Kernel.const_get(job_class(job).to_sym).instance_method(:perform).parameters
+          Kernel.const_get(job_class(job)).instance_method(:perform).parameters
         end
 
         def job_class(job)

data/lib/ezlog/version.rb

@@ -1,3 +1,3 @@
 module Ezlog
-  VERSION = '0.7.1'
+  VERSION = '0.9.6'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ezlog
 version: !ruby/object:Gem::Version
-  version: 0.7.1
+  version: 0.9.6
 platform: ruby
 authors:
 - Zoltan Ormandi
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-03-12 00:00:00.000000000 Z
+date: 2021-06-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: logging
@@ -24,6 +24,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: oj
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -42,16 +56,16 @@ dependencies:
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: 12.3.3
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: 12.3.3
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -129,14 +143,10 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- ".gitignore"
-- ".rspec"
-- ".travis.yml"
 - CHANGELOG.md
 - Gemfile
 - LICENSE.txt
 - README.md
-- Rakefile
 - bin/console
 - bin/setup
 - ezlog.gemspec
@@ -156,10 +166,10 @@ files:
 - lib/ezlog/rspec/helpers.rb
 - lib/ezlog/rspec/matchers.rb
 - lib/ezlog/sidekiq.rb
+- lib/ezlog/sidekiq/death_logger.rb
 - lib/ezlog/sidekiq/error_logger.rb
 - lib/ezlog/sidekiq/job_context.rb
 - lib/ezlog/sidekiq/job_logger.rb
-- lib/ezlog/sidekiq/logger_extension.rb
 - lib/ezlog/version.rb
 - lib/sequel/extensions/ezlog_logging.rb
 homepage: https://github.com/emartech/ezlog
@@ -182,7 +192,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.3
+rubygems_version: 3.1.6
 signing_key: 
 specification_version: 4
 summary: A zero-configuration logging solution for projects using Sidekiq, Rails,

data/.gitignore

@@ -1,13 +0,0 @@
-/.bundle/
-/.yardoc
-/_yardoc/
-/coverage/
-/doc/
-/pkg/
-/spec/reports/
-/tmp/
-
-# rspec failure tracking
-.rspec_status
-.ruby-version
-Gemfile.lock

data/.rspec

@@ -1,3 +0,0 @@
---format progress
---color
---require spec_helper

data/.travis.yml

@@ -1,9 +0,0 @@
----
-sudo: false
-language: ruby
-cache: bundler
-rvm:
-  - 2.4.6
-  - 2.5.5
-  - 2.6.3
-before_install: gem install bundler -v 2.0.1

data/Rakefile

@@ -1,6 +0,0 @@
-require "bundler/gem_tasks"
-require "rspec/core/rake_task"
-
-RSpec::Core::RakeTask.new(:spec)
-
-task :default => :spec

data/lib/ezlog/sidekiq/logger_extension.rb

@@ -1,11 +0,0 @@
-module Ezlog
-  module Sidekiq
-    module LoggerExtension
-      include LogContextHelper
-
-      def with_context(context, &block)
-        within_log_context context, &block
-      end
-    end
-  end
-end