ezlog 0.8.0 → 0.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5a2b7013c5c263e56e4531fc3e1fd0d70d5e9ec4f807babd2a95d3c68a66d365
-  data.tar.gz: 3684e63ae988f43c40291383e5c62b36e7907d0c3a377022616c70fe08a8550f
+  metadata.gz: 522f8d12a7a34a7aa83c51c6729dd212909979a3c0db32ad622436056be3ea03
+  data.tar.gz: ebf173e4431cb016de8e6a14b418b71106221982d3ebc082d53b2ed0a31fd3b5
 SHA512:
-  metadata.gz: 9279792e2f20e69fc8594d7034557ba742e708a3140f78ea8439f2c78bf4bd4e303d36215edfd30fc4ca1d9d868f1e61aac1f3fe805145d17e59f6df39d9d913
-  data.tar.gz: a6400bdc976110d23a3aba7df254314950c326846805601778c192d7a5ceaa37c94a12d414e38654f6627954f92dab51e9017ce864c70dbc064b1d4b491540e1
+  metadata.gz: c9efef1e23b6b8b77689d513665676aad77340d04131ca6241b565ac2cf7e8e78f21dbb0adf8ea06ce2a76008b12d7d350cbd7ea35de6d2228367b486bbda7b8
+  data.tar.gz: 14a749943851eadf758f2e71f0a9680f9af86c2b2895e5a946918d50696fe1f19ccacb090b37f40935d972150fb18af1d6602bb93b711917a471b8d8b4ced249

data/CHANGELOG.md

@@ -1,3 +1,78 @@
+### 0.10.0 (2021-07-01)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.6...v0.10.0)
+
+* Features & enhancements
+  * Do not log exceptions which are handled by Rails. Therefore you can't get false alarms from your exception notifier
+
+### 0.9.6 (2021-06-30)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.5...v0.9.6)
+
+* Fix
+  * fixing a Rails 6.1 specific issue with ActiveRecord array parameter in a query (`User.where(id: [1,2,3,4])`)
+
+### 0.9.5 (2020-10-01)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.4...v0.9.5)
+
+* Fix
+  * fixing an issue with the Sidekiq job's log context generation:
+    using a namespaced Sidekiq worker (`SomeModule::SomeWorker`) cause the log context generations to fail with: `NameError: wrong constant name SomeModule::SomeWorker`
+
+### 0.9.4 (2020-09-26)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.3...v0.9.4)
+
+* Features & enhancements
+  * added Ruby 2.7 to the list of version CI will test the code with
+  * remove dot-files and Rakefile from the gem
+* Fix
+  * stop using `Hash#merge` with multiple arguments as it's only supported from Ruby 2.6
+
+### 0.9.3 (2020-09-20)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.2...v0.9.3)
+
+* Features & enhancements 
+    * Switching to [Oj](https://github.com/ohler55/oj) for fast JSON serialization
+    * Allow level to be formatted (so it can be logged as a number too)
+    
+        if you want to use Ougai-like numbers you can do something like this: 
+        ```ruby
+        config.ezlog.layout_options = { level_formatter: ->(level_number) { (level_number + 2) * 10 } } 
+        
+        Rails.logger.error('Boom!')
+        #=> {"logger":"Application","timestamp":"2020-09-20T19:29:03+02:00","level":50,"hostname":"BUD010256.local","pid":19872,"message":"Boom!"}
+        ``` 
+    * initial context (a context which will be added to every single line of log) can be configured via `config.ezlog.layout_options` and it defaults to `{environment: ::Rails.env}`
+    
+### 0.9.2 (2020-09-19)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.1...v0.9.2)
+
+* Features & enhancements 
+  * Improvements of the [Sidekiq](https://github.com/mperham/sidekiq) integration
+    * supports additional job information: batch id, tags and thread id (bid, tags, tid)
+    * support logging "death" events (setting up a death_handler) 
+
+### 0.9.1 (2020-05-10)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.9.0...v0.9.1)
+
+* Bug fixed
+  * Fixed a bug in access log exclusion pattern matching that would exclude partial matches for a path if it was 
+    specified as a string (expecting to be excluded only in case of a full match).
+
+### 0.9.0 (2020-05-10)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.8.0...v0.9.0)
+
+* Features & enhancements
+  * Ezlog now supports [Rails](https://rubyonrails.org/) 6.
+  * Added the ability to exclude certain paths from access logging. Use the `exclude_paths` configuration option to
+    add paths (strings or regexps) to exclude from your access logs.
+
 ### 0.8.0 (2020-04-07)
 
 [Full Changelog](https://github.com/emartech/ezlog/compare/v0.7.1...v0.8.0)

data/README.md

@@ -6,7 +6,7 @@
 Ezlog is intended to be a zero-configuration structured logging setup for pure Ruby or [Ruby on Rails](https://rubyonrails.org/) 
 projects using any (or all) of the following libraries or frameworks:
 
-* [Ruby on Rails](https://rubyonrails.org/)
+* [Ruby on Rails](https://rubyonrails.org/) (5.x and 6.x supported)
 * [Sidekiq](https://github.com/mperham/sidekiq)
 * [Sequel](https://sequel.jeremyevans.net/) 
 * [Rack::Timeout](https://github.com/heroku/rack-timeout)
@@ -93,6 +93,7 @@ In addition to this, Ezlog also does the following:
 * It disables Rails's default logging of uncaught errors and injects its own error logger into the application, which
   * logs 1 line per error, including the error's name and context (stack trace, etc.),
   * logs every error at ERROR level instead of the default FATAL.
+  * does not log exceptions which are handled by Rails (`ActionDispatch::ExceptionWrapper.rescue_responses`)
 * It disables Rails's default request logging, which logs several lines per event during the processing of an action,
   and replaces the default Rack access log with its own access log middleware. The end result is an access log that
   * contains all relevant information (request ID, method, path, params, client IP, duration and response status code), and
@@ -134,6 +135,12 @@ Using this configuration, Ezlog will only log the `action` parameter under the `
 all parameters serialized into a single string under the key `params_serialized`. You won't lose any information,
 but you can make sure that only the relevant parameters of your requests are searchable (and thus protect your logs).
 
+Should you want to exclude certain paths (e.g. a healthcheck URL) from your access logs, you can use the following option:
+
+```ruby
+config.ezlog.exclude_paths = ['/healthcheck', %r(/monitoring/.*)]  # default is empty so everything gets logged
+```
+
 #### The log level
 
 The logger's log level is determined as follows (in order of precedence):

data/ezlog.gemspec

@@ -16,16 +16,17 @@ Gem::Specification.new do |spec|
   }
 
   spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
-    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^((test|spec|features)/|Rakefile|\.)}) }
   end
   spec.bindir        = "exe"
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
   spec.add_dependency "logging", "~> 2.0"
+  spec.add_dependency "oj", "~> 3.0"
 
   spec.add_development_dependency "bundler", "~> 2.0"
-  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rake", ">= 12.3.3"
   spec.add_development_dependency "rspec", "~> 3.0"
   spec.add_development_dependency "sidekiq", ">= 5.0"
   spec.add_development_dependency "sequel", "~> 5.0"

data/lib/ezlog/logging_layout.rb

@@ -1,10 +1,11 @@
 require 'time'
-require 'json'
+require 'oj'
 
 module Ezlog
   class LoggingLayout < ::Logging::Layout
-    def initialize(context = {})
+    def initialize(context = {}, options = {})
       @initial_context = context
+      @level_formatter = options.fetch(:level_formatter, ->(numeric_level) { ::Logging::LNAMES[numeric_level] })
     end
 
     def format(event)
@@ -12,7 +13,7 @@ module Ezlog
       add_initial_context_to log_entry
       add_logging_context_to log_entry
       add_event_information_to log_entry, event
-      ::JSON.dump(log_entry) + "\n"
+      ::Oj.dump(log_entry, mode: :json) + "\n"
     end
 
     private
@@ -20,8 +21,8 @@ module Ezlog
     def basic_information_for(event)
       {
         'logger' => event.logger,
-        'timestamp' => event.time.iso8601,
-        'level' => ::Logging::LNAMES[event.level],
+        'timestamp' => event.time.iso8601(3),
+        'level' => @level_formatter.call(event.level),
         'hostname' => Socket.gethostname,
         'pid' => Process.pid
       }
@@ -46,17 +47,17 @@ module Ezlog
       when Hash
         obj
       else
-        {message: obj}
+        { 'message' => obj }
       end
     end
 
     def exception_message_by(exception)
       {
-        message: exception.message,
-        error: {
-          class: exception.class.name,
-          message: exception.message,
-          backtrace: exception.backtrace&.first(20)
+        'message' => exception.message,
+        'error' => {
+          'class' => exception.class.name,
+          'message' => exception.message,
+          'backtrace' => exception.backtrace&.first(20)
         }
       }
     end

data/lib/ezlog/rails/access_log.rb

@@ -3,10 +3,10 @@ module Ezlog
     class AccessLog
       include LogContextHelper
 
-      def initialize(app, logger, whitelisted_params)
+      def initialize(app, logger, config)
         @app = app
         @logger = logger
-        @whitelisted_params = whitelisted_params&.map &:to_s
+        @config = config
       end
 
       def call(env)
@@ -29,23 +29,36 @@ module Ezlog
       end
 
       def log_request(request, status)
+        return if path_ignored?(request)
+
         message = {
           message: '%s %s - %i (%s)' % [request.method, request.filtered_path, status, Rack::Utils::HTTP_STATUS_CODES[status]],
           remote_ip: request.remote_ip,
           method: request.method,
           path: request.filtered_path,
-          params: params_to_log_in(request),
+          params: params_to_log(request),
           response_status_code: status
         }
-        message.merge! params_serialized: request.filtered_parameters.inspect if @whitelisted_params
+        message.merge! params_serialized: request.filtered_parameters.inspect if @config.log_only_whitelisted_params
         @logger.info message
       end
 
-      def params_to_log_in(request)
-        if @whitelisted_params.nil?
-          request.filtered_parameters
+      def path_ignored?(request)
+        @config.exclude_paths.any? do |pattern|
+          case pattern
+          when Regexp
+            pattern.match? request.path
+          else
+            pattern == request.path
+          end
+        end
+      end
+
+      def params_to_log(request)
+        if @config.log_only_whitelisted_params
+          request.filtered_parameters.slice *@config.whitelisted_params&.map(&:to_s)
         else
-          request.filtered_parameters.slice *@whitelisted_params
+          request.filtered_parameters
         end
       end
     end

data/lib/ezlog/rails/active_record/log_subscriber.rb

@@ -33,6 +33,8 @@ module Ezlog
           end
 
           Hash[param_value_pairs]
+        rescue NoMethodError
+          params
         end
 
         def type_casted_values_from(event)

data/lib/ezlog/rails/log_exceptions.rb

@@ -9,9 +9,15 @@ module Ezlog
       def call(env)
         @app.call(env)
       rescue Exception => exception
-        @logger.error exception
+        @logger.error exception unless handled?(exception)
         raise
       end
+
+      private
+
+      def handled?(exception)
+        ActionDispatch::ExceptionWrapper.rescue_responses.key? exception.class.name
+      end
     end
   end
 end

data/lib/ezlog/railtie.rb

@@ -4,13 +4,18 @@ module Ezlog
     config.ezlog.enable_sequel_logging = false
     config.ezlog.log_only_whitelisted_params = false
     config.ezlog.whitelisted_params = [:controller, :action]
+    config.ezlog.exclude_paths = []
+    config.ezlog.initial_context = { environment: ::Rails.env }
+    config.ezlog.layout_options = {}
 
     initializer "ezlog.initialize" do
       require "ezlog/rails/extensions"
     end
 
     initializer 'ezlog.configure_logging' do |app|
-      ::Logging.logger.root.appenders = ::Logging.appenders.stdout 'stdout', layout: Ezlog::LoggingLayout.new(environment: ::Rails.env)
+      ::Logging.logger.root.appenders =
+        ::Logging.appenders.stdout 'stdout', layout: Ezlog::LoggingLayout.new(app.config.ezlog.initial_context,
+                                                                              app.config.ezlog.layout_options)
       ::Logging.logger.root.level = app.config.log_level
     end
 
@@ -29,16 +34,26 @@ module Ezlog
     initializer 'ezlog.configure_rails_middlewares' do |app|
       app.config.middleware.insert_after ::ActionDispatch::RequestId, Ezlog::Rails::RequestLogContext
       app.config.middleware.delete ::Rails::Rack::Logger
-      app.config.middleware.insert_before ::ActionDispatch::DebugExceptions, Ezlog::Rails::AccessLog, Ezlog.logger('AccessLog'), whitelisted_params(app)
+      app.config.middleware.insert_before ::ActionDispatch::DebugExceptions, Ezlog::Rails::AccessLog, Ezlog.logger('AccessLog'), config.ezlog
       app.config.middleware.insert_after ::ActionDispatch::DebugExceptions, Ezlog::Rails::LogExceptions, Ezlog.logger('Application')
     end
 
     config.after_initialize do
-      Ezlog::Rails::LogSubscriber.detach ::ActionController::LogSubscriber
-      Ezlog::Rails::LogSubscriber.detach ::ActionView::LogSubscriber
-      if defined? ::ActiveRecord
-        Ezlog::Rails::LogSubscriber.detach ::ActiveRecord::LogSubscriber
-        Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+      case ::Rails::VERSION::MAJOR
+      when 6
+        ::ActionController::LogSubscriber.detach_from :action_controller
+        ::ActionView::LogSubscriber.detach_from :action_view
+        if defined? ::ActiveRecord
+          ::ActiveRecord::LogSubscriber.detach_from :active_record
+          Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+        end
+      else
+        Ezlog::Rails::LogSubscriber.detach ::ActionController::LogSubscriber
+        Ezlog::Rails::LogSubscriber.detach ::ActionView::LogSubscriber
+        if defined? ::ActiveRecord
+          Ezlog::Rails::LogSubscriber.detach ::ActiveRecord::LogSubscriber
+          Ezlog::Rails::LogSubscriber.attach Ezlog::Rails::ActiveRecord::LogSubscriber, :active_record
+        end
       end
     end
 
@@ -56,15 +71,12 @@ module Ezlog
         config.options[:job_logger] = Ezlog::Sidekiq::JobLogger
         config.error_handlers << Ezlog::Sidekiq::ErrorLogger.new
         config.error_handlers.delete_if { |handler| handler.is_a? ::Sidekiq::ExceptionHandler::Logger }
+        config.death_handlers << Ezlog::Sidekiq::DeathLogger.new
       end
     end
 
     def disable_rack_timeout_logging
       ::Rack::Timeout::Logger.logger = ::Logger.new(nil)
     end
-
-    def whitelisted_params(app)
-      app.config.ezlog.log_only_whitelisted_params ? app.config.ezlog.whitelisted_params : nil
-    end
   end
 end

data/lib/ezlog/rspec/matchers.rb

@@ -55,3 +55,5 @@ RSpec::Matchers.define :log do
     log_output_is_expected.to include_log_message(expected).at_level(log_level)
   end
 end
+
+RSpec::Matchers.define_negated_matcher :not_log, :log

data/lib/ezlog/sidekiq.rb

@@ -1,5 +1,6 @@
 module Ezlog
   module Sidekiq
+    autoload :DeathLogger, 'ezlog/sidekiq/death_logger'
     autoload :ErrorLogger, 'ezlog/sidekiq/error_logger'
     autoload :JobContext, 'ezlog/sidekiq/job_context'
     autoload :JobLogger, 'ezlog/sidekiq/job_logger'

data/lib/ezlog/sidekiq/death_logger.rb

@@ -0,0 +1,15 @@
+require 'sidekiq'
+
+module Ezlog
+  module Sidekiq
+    class DeathLogger
+      include LogContextHelper
+
+      def call(job, error)
+        within_log_context(JobContext.from_job_hash(job)) do
+          ::Sidekiq.logger.error error
+        end
+      end
+    end
+  end
+end

data/lib/ezlog/sidekiq/job_context.rb

@@ -4,13 +4,18 @@ module Ezlog
       class << self
         def from_job_hash(job_hash)
           return {} if job_hash.nil?
-          basic_info_from(job_hash).merge named_arguments_from(job_hash)
+          thread_info.merge(basic_info_from(job_hash)).merge(named_arguments_from(job_hash))
         end
 
         private
 
+        def thread_info
+          { tid: Thread.current['sidekiq_tid'] || (Thread.current.object_id ^ ::Process.pid).to_s(36) }
+        end
+
+
         def basic_info_from(job)
-          {
+          h = {
             jid: job['jid'],
             queue: job['queue'],
             worker: job_class(job),
@@ -18,6 +23,9 @@ module Ezlog
             enqueued_at: job['enqueued_at'],
             run_count: (job['retry_count'] || -1) + 2
           }
+          h[:bid] = job['bid'] if job['bid']
+          h[:tags] = job['tags'] if job['tags']
+          h
         end
 
         def named_arguments_from(job)
@@ -29,7 +37,7 @@ module Ezlog
         end
 
         def method_parameters_of(job)
-          Kernel.const_get(job_class(job).to_sym).instance_method(:perform).parameters
+          Kernel.const_get(job_class(job)).instance_method(:perform).parameters
         end
 
         def job_class(job)

data/lib/ezlog/version.rb

@@ -1,3 +1,3 @@
 module Ezlog
-  VERSION = '0.8.0'
+  VERSION = '0.10.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ezlog
 version: !ruby/object:Gem::Version
-  version: 0.8.0
+  version: 0.10.0
 platform: ruby
 authors:
 - Zoltan Ormandi
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-04-07 00:00:00.000000000 Z
+date: 2021-07-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: logging
@@ -24,6 +24,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: oj
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -42,16 +56,16 @@ dependencies:
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: 12.3.3
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: 12.3.3
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -129,14 +143,10 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- ".gitignore"
-- ".rspec"
-- ".travis.yml"
 - CHANGELOG.md
 - Gemfile
 - LICENSE.txt
 - README.md
-- Rakefile
 - bin/console
 - bin/setup
 - ezlog.gemspec
@@ -156,6 +166,7 @@ files:
 - lib/ezlog/rspec/helpers.rb
 - lib/ezlog/rspec/matchers.rb
 - lib/ezlog/sidekiq.rb
+- lib/ezlog/sidekiq/death_logger.rb
 - lib/ezlog/sidekiq/error_logger.rb
 - lib/ezlog/sidekiq/job_context.rb
 - lib/ezlog/sidekiq/job_logger.rb
@@ -181,7 +192,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.3
+rubygems_version: 3.1.2
 signing_key: 
 specification_version: 4
 summary: A zero-configuration logging solution for projects using Sidekiq, Rails,

data/.gitignore

@@ -1,13 +0,0 @@
-/.bundle/
-/.yardoc
-/_yardoc/
-/coverage/
-/doc/
-/pkg/
-/spec/reports/
-/tmp/
-
-# rspec failure tracking
-.rspec_status
-.ruby-version
-Gemfile.lock

data/.rspec

@@ -1,3 +0,0 @@
---format progress
---color
---require spec_helper

data/.travis.yml

@@ -1,9 +0,0 @@
----
-sudo: false
-language: ruby
-cache: bundler
-rvm:
-  - 2.4.6
-  - 2.5.5
-  - 2.6.3
-before_install: gem install bundler -v 2.0.1

data/Rakefile

@@ -1,6 +0,0 @@
-require "bundler/gem_tasks"
-require "rspec/core/rake_task"
-
-RSpec::Core::RakeTask.new(:spec)
-
-task :default => :spec