ezlog 0.6.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1c262ec657610e6e59eaa38fca73bfd01ef5c8e9107ad1d45d7fcd652fe202ad
-  data.tar.gz: eb01c4eebdf823979a4184dfb2ef07b63e66f12e3bc65497714f1a94bb348a06
+  metadata.gz: a3448fa736b7d0c38391f77c299ca38185705c50bfb9a48230166b1bf95d5cea
+  data.tar.gz: 6334e6cfbca5a00b5c10d3c272533711613d556bf4d452176a055ba876dae8f9
 SHA512:
-  metadata.gz: fdb2c08f48096b08524b8b40a9f88f868d2b376356d14b5d8bc7e45fa6b6f0cb412f95cc1a4a4be613d041ab518c1128be66871307b71503661e8ff86d7914a4
-  data.tar.gz: 22341fd2d610f7e373dc2fa30eb02a9d2bab28bc140bc5c9f25ccaeff175684a7a93afd9f7b2ea948cf13ac956fbd146f7d272ef45d8e780aae3342b8cffb994
+  metadata.gz: faa9b4f39da472e3e353c8d2d0f95a36f08eae1ca631f0c1611907488da9fbd80dc3a330cde6f7f103056a34e7a0e72469e53580d36180b9119f3c5e96bc03bf
+  data.tar.gz: 24798f5e9e2d2308f443bc2c4a72612aab10e389599fa45ce8eea535bf68fb104f9da8c3c1a8f4ceda225215597ab7f85a1e314985841679376c40e5deebd06c

data/CHANGELOG.md

@@ -1,3 +1,14 @@
+### 0.7.0 (2020-03-11)
+
+[Full Changelog](https://github.com/emartech/ezlog/compare/v0.6.0...v0.7.0)
+
+* Features & enhancements
+  * Added the ability to configure parameter logging of the [Rails](https://rubyonrails.org/) access log.
+    By default, all parameters are logged under the key `params`. By turning on the `log_only_whitelisted_params`
+    config swith, you can make sure that only the parameters whose name is included in the `whitelisted_params`
+    config setting get logged under the `params` key. All parameters will still be logged, but serialized into a
+    single string under the `params_seralized` key, not creating a lot of noise under `params`.
+
 ### 0.6.0 (2019-11-29)
 
 [Full Changelog](https://github.com/emartech/ezlog/compare/v0.5.3...v0.6.0)

data/README.md

@@ -46,6 +46,7 @@ At the moment Ezlog only support [Rails](https://rubyonrails.org/) apps. Non-Rai
 * Initializes the [Logging](https://github.com/TwP/logging) library
 * Configures [Rails](https://rubyonrails.org/)'s logging
 * Configures [Sidekiq](https://github.com/mperham/sidekiq) logging
+* Configures [Sequel](https://sequel.jeremyevans.net/) logging
 * Configures [Rack::Timeout](https://github.com/heroku/rack-timeout) logging
 * Provides support for adding context information to log messages
 * Provides testing support for [RSpec](https://rspec.info/)
@@ -118,6 +119,21 @@ With Ezlog:
 {"logger":"AccessLog","timestamp":"2019-06-08T08:49:31+02:00","level":"INFO","hostname":"MacbookPro.local","pid":75463,"environment":"development","request_id":"9a43631b-284c-4677-9d08-9c1cc5c7d3a7","duration_sec":0.031,"message":"GET /welcome?subsession_id=34ea8596f9764f475f81158667bc2654 - 200 (OK)","remote_ip":"127.0.0.1","method":"GET","path":"/welcome?subsession_id=34ea8596f9764f475f81158667bc2654","params":{"subsession_id":"34ea8596f9764f475f81158667bc2654","controller":"pages","action":"welcome"},"response_status_code":200}
 ```
 
+By default, Ezlog logs all request parameters as a hash (JSON object) under the `params` key. This is very convenient
+in a structured logging system and makes it easy to search for specific request parameter values e.g. in ElasticSearch
+(should you happen to store your logs there). Unfortunately, in some cases - such as when handling large forms - this
+can create quite a bit of noise and impact the searchability of your logs negatively. For this reason, you have the
+option to restrict which parameters get logged by adding something like the following to your application's configuration:
+
+```ruby
+config.ezlog.log_only_whitelisted_params = true  # default is false
+config.ezlog.whitelisted_params = [:action]      # default is [:controller, :action]
+```
+
+Using this configuration, Ezlog will only log the `action` parameter under the `params` key, but will still log
+all parameters serialized into a single string under the key `params_serialized`. You won't lose any information,
+but you can make sure that only the relevant parameters of your requests are searchable (and thus protect your logs).
+
 #### The log level
 
 The logger's log level is determined as follows (in order of precedence):
@@ -157,6 +173,18 @@ TestWorker.perform_async 42
 #=> {"logger":"Sidekiq","timestamp":"2019-05-12T10:38:12+02:00","level":"INFO","hostname":"MacbookPro.local","pid":75538,"jid":"abcdef1234567890","queue":"default","worker":"TestWorker","created_at":"2019-05-12 10:38:10 +0200","enqueued_at":"2019-05-12 10:38:10 +0200","run_count":1,"customer_id":42,"duration_sec":2.667,"message":"TestWorker finished"}
 ```
 
+### Configures Sequel logging
+
+Ezlog adds a logging extension to [Sequel](https://sequel.jeremyevans.net/) that appends a logger to any newly opened
+database connection. This logger is an instance of a [Logging](https://github.com/TwP/logging) logger by the name of
+`Sequel`, behaving as described above. It also sets [Sequel](https://sequel.jeremyevans.net/)'s `sql_log_level` to `debug`. 
+
+This extension isn't enabled by default, because [Sequel](https://sequel.jeremyevans.net/) logging isn't necessarily
+needed in every project. You can enable it by adding the following line to your application's configuration:
+```
+config.ezlog.enable_sequel_logging = true
+```
+
 ### Configures Rack::Timeout logging
 
 [Rack::Timeout](https://github.com/heroku/rack-timeout) is a very useful tool for people running services on Heroku

data/lib/ezlog/rails/access_log.rb

@@ -3,9 +3,10 @@ module Ezlog
     class AccessLog
       include LogContextHelper
 
-      def initialize(app, logger)
+      def initialize(app, logger, whitelisted_params)
         @app = app
         @logger = logger
+        @whitelisted_params = whitelisted_params&.map &:to_s
       end
 
       def call(env)
@@ -28,12 +29,24 @@ module Ezlog
       end
 
       def log_request(request, status)
-        @logger.info message: '%s %s - %i (%s)' % [request.method, request.filtered_path, status, Rack::Utils::HTTP_STATUS_CODES[status]],
-                     remote_ip: request.remote_ip,
-                     method: request.method,
-                     path: request.filtered_path,
-                     params: request.filtered_parameters,
-                     response_status_code: status
+        message = {
+          message: '%s %s - %i (%s)' % [request.method, request.filtered_path, status, Rack::Utils::HTTP_STATUS_CODES[status]],
+          remote_ip: request.remote_ip,
+          method: request.method,
+          path: request.filtered_path,
+          params: params_to_log_in(request),
+          response_status_code: status
+        }
+        message.merge! params_serialized: request.filtered_parameters.inspect if @whitelisted_params
+        @logger.info message
+      end
+
+      def params_to_log_in(request)
+        if @whitelisted_params.nil?
+          request.filtered_parameters
+        else
+          request.filtered_parameters.slice *@whitelisted_params
+        end
       end
     end
   end

data/lib/ezlog/railtie.rb

@@ -2,6 +2,8 @@ module Ezlog
   class Railtie < Rails::Railtie
     config.ezlog = ActiveSupport::OrderedOptions.new
     config.ezlog.enable_sequel_logging = false
+    config.ezlog.log_only_whitelisted_params = false
+    config.ezlog.whitelisted_params = [:controller, :action]
 
     initializer "ezlog.initialize" do
       require "ezlog/rails/extensions"
@@ -27,7 +29,7 @@ module Ezlog
     initializer 'ezlog.configure_rails_middlewares' do |app|
       app.config.middleware.insert_after ::ActionDispatch::RequestId, Ezlog::Rails::RequestLogContext
       app.config.middleware.delete ::Rails::Rack::Logger
-      app.config.middleware.insert_before ::ActionDispatch::DebugExceptions, Ezlog::Rails::AccessLog, Ezlog.logger('AccessLog')
+      app.config.middleware.insert_before ::ActionDispatch::DebugExceptions, Ezlog::Rails::AccessLog, Ezlog.logger('AccessLog'), whitelisted_params(app)
       app.config.middleware.insert_after ::ActionDispatch::DebugExceptions, Ezlog::Rails::LogExceptions, Ezlog.logger('Application')
     end
 
@@ -60,5 +62,9 @@ module Ezlog
     def disable_rack_timeout_logging
       ::Rack::Timeout::Logger.logger = ::Logger.new(nil)
     end
+
+    def whitelisted_params(app)
+      app.config.ezlog.log_only_whitelisted_params ? app.config.ezlog.whitelisted_params : nil
+    end
   end
 end

data/lib/ezlog/rspec/matchers.rb

@@ -23,7 +23,7 @@ RSpec::Matchers.define :include_log_message do |expected|
   def expected_messages_from(object)
     @expected_messages ||= case object
                            when Hash
-                             object.map { |k, v| MultiJson.dump(k => v)[1...-1] }
+                             object.map { |k, v| JSON.dump(k => v)[1...-1] }
                            when String
                              [object]
                            else

data/lib/ezlog/version.rb

@@ -1,3 +1,3 @@
 module Ezlog
-  VERSION = '0.6.0'
+  VERSION = '0.7.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ezlog
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Zoltan Ormandi
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2019-11-29 00:00:00.000000000 Z
+date: 2020-03-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: logging