lumberjack 1.0.13 → 1.2.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: aafe117d91082b575618e962e3d66a0b6d3c81b4
-  data.tar.gz: a4d32d48a82b15e8d67e0e00ab3b3637a63b7e99
+SHA256:
+  metadata.gz: c7912cf1bfe32dcaa911a6a65a0b755fb3051eebc2a7e4a65ff5d422f2f61ad7
+  data.tar.gz: b4077d2ef97be7169ed0e7bca7d107ec3f42372dde45cae1e7cdcedae1fb4e61
 SHA512:
-  metadata.gz: 33d8f2e2e386edad724394264f76c33c17e3df6d9dd394b3444a7abebe12d5e93d3ce8a88726e8b67fcfb2c279615741a9e55b9112d5ec45e50e0c139f481a36
-  data.tar.gz: 129825a3150dad02aecde16130974c2add346fe27d6455e0ea22d6616d6e766306af8c7123ef10ec7e51ca81f06e7a6f1d0cd573abd7566f07b1b15efeed415a
+  metadata.gz: a092775aec0c7670eda8ff4959a7cc928a57b0934ecc8a1190efe395b437bc0a681bdbd2f3455b98f3cf757904f5f39b31c96e24e9c8bcd7a481b806027e92c8
+  data.tar.gz: 885f76da2738f3f6236a363d9a7dea68f83a80a03e123a3c52c578984f963e959de6fd60d0fd222d9a9933c22ed959181538abb4118163c8fd6528292e88673c

data/CHANGELOG.md

@@ -0,0 +1,129 @@
+## 1.2.8
+
+* Add `Logger#untagged` to remove previously set logging tags from a block.
+* Return result of the block when a block is passed to `Logger#tag`.
+
+## 1.2.7
+
+* Allow passing frozen hashes to `Logger#tag`. Tags passed to this method are now duplicated so the logger maintains it's own copy of the hash.
+
+## 1.2.6
+
+* Fix `Logger#tag` so it only ads to the current block's logger tags instead of the global tags if called inside a `Logger#tag` block.
+* Add Logger#remove_tag
+
+## 1.2.5
+
+* Fix logic with recursive reference guard in StructuredFormatter so it only suppresses Enumerable references.
+* Add support for bang methods (error!) for setting the log level.
+
+## 1.2.4
+
+* Enhance `ActiveSupport::TaggedLogging` support so code that Lumberjack loggers can be wrapped with a tagged logger.
+
+## 1.2.3
+
+* Fix structured formatter so no-recursive, duplicate references are allowed.
+
+## 1.2.2
+
+* Prevent infinite loops in the structured formatter where objects have backreferences to each other.
+
+## 1.2.1
+
+* Prevent infinite loops where logging a statement triggers the logger.
+
+## 1.2.0
+
+* Enable compatibility with `ActiveSupport::TaggedLogger` by calling `tagged_logger!` on a logger.
+* Add `tag_formatter` to logger to specify formatting of tags for output.
+* Allow adding and removing classes by name to formatters.
+* Allow adding and removing multiple classes in a single call to a formatter.
+* Allow using symbols and strings as log level for silencing a logger.
+* Ensure flusher thread gets stopped when logger is closed.
+* Add writer for logger device attribute.
+* Handle passing an array of devices to a multi device.
+* Helper method to get a tag with a specified name.
+* Add strip formatter to strip whitespace from strings.
+* Support non-alpha numeric characters in template variables.
+* Add backtrace cleaner to ExceptionFormatter.
+
+## 1.1.1
+
+* Replace Procs in tag values with the value of calling the Proc in log entries.
+
+## 1.1.0
+
+* Change `Lumberjack::Logger` to inherit from ::Logger
+* Add support for tags on log messages
+* Add global tag context for all loggers
+* Add per logger tags and tag contexts
+* Reimplement unit of work id as a tag on log entries
+* Add support for setting datetime format on log devices
+* Performance optimizations
+* Add Multi device to output to multiple devices
+* Add `DateTimeFormatter`, `IdFormatter`, `ObjectFormatter`, and `StructuredFormatter`
+* Add rack `Context` middleware for setting thread global context
+* End support for ruby versions < 2.3
+* Add support for modules in formatters
+
+## 1.0.13
+
+* Reduce amount of code executed inside a mutex lock when writing to the logger stream.
+* Added `:min_roll_check` option to `Lumberjack::Device::RollingLogFile` to reduce file system checks. Default is now to only check if a file needs to be rolled at most once per second.
+* Force immutable strings for Ruby versions that support them.
+
+## 1.0.12
+
+* Add support for `ActionDispatch` request id for better Rails compatibility.
+
+## 1.0.11
+
+* Fix Ruby 2.4 deprecation warning on Fixnum (thanks koic).
+* Fix gemspec files to be flat array (thanks e2).
+
+## 1.0.10
+
+* Expose option to manually roll log files.
+* Minor code cleanup.
+
+## 1.0.9
+
+* Add method so Formatter is compatible with `ActiveSupport` logging extensions.
+
+## 1.0.8
+
+* Fix another internal variable name conflict with `ActiveSupport` logging extensions.
+
+## 1.0.7
+
+* Fix broken formatter attribute method.
+
+## 1.0.6
+
+* Fix internal variable name conflict with `ActiveSupport` logging extensions.
+
+## 1.0.5
+
+* Update docs.
+* Remove autoload calls to make thread safe.
+* Make compatible with Ruby 2.1.1 Pathname.
+* Make compatible with standard library Logger's use of progname as default message.
+
+## 1.0.4
+
+* Add ability to supply a unit of work id for a block instead of having one generated every time.
+
+## 1.0.3
+
+* Change log file output format to binary to avoid encoding warnings.
+* Fixed bug in log file rolling that left the file locked.
+
+## 1.0.2
+
+* Remove deprecation warnings under ruby 1.9.3.
+* Add more error checking around file rolling.
+
+## 1.0.1
+
+* Writes are no longer buffered by default.

data/README.md

@@ -1,3 +1,7 @@
+![Continuous Integration](https://github.com/bdurand/lumberjack/workflows/Continuous%20Integration/badge.svg)
+[![Maintainability](https://api.codeclimate.com/v1/badges/a0abc03721fff9b0cde1/maintainability)](https://codeclimate.com/github/bdurand/lumberjack/maintainability)
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
+
 # Lumberjack
 
 Lumberjack is a simple, powerful, and fast logging implementation in Ruby. It uses nearly the same API as the Logger class in the Ruby standard library and as ActiveSupport::BufferedLogger in Rails.
@@ -33,21 +37,77 @@ The following information is recorded for each message:
 * time - The time at which the message was recorded.
 * program name - The name of the program logging the message. This can be either set for all messages or customized with each message.
 * process id - The process id (pid) of the process that logged the message.
-* unit of work id - The unique 12 byte hexadecimal number generated for a unit of work.
+* tags - An map of name value pairs for addition information about the log context.
+
+### Tags
 
-### Units Of Work
+You can use tags to provide additional meta data about a log message or the context that the log message is being made in. Using tags can keep you log messages clean. You can avoid string interoplation to add additional meta data.
 
-A unit of work can be used to tie together all log messages within a block. This can be very useful to isolate a group of messages that represent one path through the system. For instance, in a web application, a single request represents a natural unit of work,  and when you are looking through a log file, it is useful to see the entire set of log entries as a unit instead of interspersed with messages from other concurrent requests.
+Each of the logger methods includes an additional argument that can be used to specify tags on a messsage:
 
 ```ruby
-  # All log entries in this block will get a common unit of work id.
-  Lumberjack.unit_of_work do
-    logger.info("Begin request")
-    yield
-    logger.info("End request")
-  end
+logger.info("request completed", duration: elapsed_time, status: response.status)
+```
+
+You can also specify tags on a logger that will be included with every log message.
+
+```ruby
+logger.tag(host: Socket.gethostname)
+```
+
+You can specify tags that will only be applied to the logger in a block as well.
+
+```ruby
+logger.tag(thread_id: Thread.current.object_id) do
+  logger.info("here") # Will include the `thread_id` tag
+  logger.tag(count: 15)
+  logger.info("with count") # Will include the `count` tag
+end
+logger.info("there") # Will not include the `thread_id` or `count` tag
+```
+
+You can also set tags to `Proc` objects that will be evaluated when creating a log entry.
+
+```ruby
+logger.tag(thread_id: lambda { Thread.current.object_id })
+Thread.new do
+  logger.info("inside thread") # Will include the `thread_id` tag with id of the spawned thread
+end
+logger.info("outside thread") # Will include the `thread_id` tag with id of the main thread
+```
+
+Finally, you can specify a logging context with tags that apply within a block to all loggers.
+
+```ruby
+Lumberjack.context do
+  Lumberjack.tag(request_id: SecureRandom.hex)
+  logger.info("begin request") # Will include the `request_id` tag
+end
+logger.info("no requests") # Will not include the `request_id` tag
+```
+
+Tag keys are always converted to strings. Tags are inherited so that message tags take precedence over block tags which take precedence over global tags.
+
+#### Compatibility with ActiveSupport::TaggedLogging
+
+`Lumberjack::Logger` version 1.1.2 or greater is compatible with `ActiveSupport::TaggedLogging`. This is so that other code that expect to have a logger that responds to the `tagged` method will work. Any tags added with the `tagged` method will be appended to an array in the the "tagged" tag.
+
+```ruby
+logger.tagged("foo", "bar=1", "other") do
+  logger.info("here") # will include tags: {"tagged" => ["foo", "bar=1", "other"]}
+end
 ```
 
+#### Templates
+
+The built in `Lumberjack::Device::Writer` class has built in support for including tags in the output using the `Lumberjack::Template` class.
+
+You can specify any tag name you want in a template as well as the `:tags` macro for all tags. If a tag name has been used as it's own macro, it will not be included in the `:tags` macro.
+
+#### Unit Of Work
+
+Lumberjack 1.0 had a concept of a unit of work id that could be used to tie log messages together. This has been replaced by tags. There is still an implementation of `Lumberjack.unit_of_work`, but it is just a wrapper on the tag implementation.
+
 ### Pluggable Devices
 
 When a Logger logs a LogEntry, it sends it to a Lumberjack::Device. Lumberjack comes with a variety of devices for logging to IO streams or files.
@@ -56,27 +116,74 @@ When a Logger logs a LogEntry, it sends it to a Lumberjack::Device. Lumberjack c
 * Lumberjack::Device::LogFile - Writes log entries to a file.
 * Lumberjack::Device::DateRollingLogFile - Writes log entries to a file that will automatically roll itself based on date.
 * Lumberjack::Device::SizeRollingLogFile - Writes log entries to a file that will automatically roll itself based on size.
+* Lumberjack::Device::Multi - This device wraps mulitiple other devices and will write log entries to each of them.
 * Lumberjack::Device::Null - This device produces no output and is intended for testing environments.
 
 If you'd like to send you log to a different kind of output, you just need to extend the Device class and implement the +write+ method. Or check out these plugins:
 
 * [lumberjack_syslog_device](https://github.com/bdurand/lumberjack_syslog_device) - send your log messages to the system wide syslog service
-* [lumberjack_multi-device](https://github.com/astevens/lumberjack_multi-device) - send log messages to multiple devices
 * [lumberjack_mongo_device](https://github.com/bdurand/lumberjack_mongo_device) - store your log messages to a [MongoDB](http://www.mongodb.org/) NoSQL data store
 * [lumberjack-couchdb-driver](https://github.com/narkisr/lumberjack-couchdb-driver) - store your log messages to a [CouchDB](http://couchdb.apache.org/) NoSQL data store
 * [lumberjack_heroku_device](https://github.com/tonycoco/lumberjack_heroku_device) - log to Heroku's logging system
 
 ### Customize Formatting
 
-When a message is logged, it is first converted into a string. You can customize how it is converted by adding mappings to a Formatter.
+#### Formatters
+
+The message you send to the logger can be any object type and does not need to be a string. You can specify a `Lumberjack::Formatter` to instruct the logger how to format objects before outputting them to the device. You do this by mapping classes or modules to formatter code. This code can be either a block or an object that responds to the `call` method. The formatter will be called with the object logged as the message and the returned value will be what is sent to the device.
+
+```ruby
+  # Format all floating point number with three significant digits.
+  logger.formatter.add(Float) { |value| value.round(3) }
+
+  # Format all enumerable objects as a comma delimited string.
+  logger.formatter.add(Enumerable) { |value| value.join(", ") }
+```
+
+There are several built in classes you can add as formatters. You can use a symbol to reference built in formatters.
 
 ```ruby
   logger.formatter.add(Hash, :pretty_print)  # use the Formatter::PrettyPrintFormatter for all Hashes
-  logger.formatter.add(MyClass){|obj| "#{obj.class}@#{obj.id}"}  # use a block to provide a custom format
+  logger.formatter.add(Hash, Lumberjack::Formatter::PrettyPrintFormatter.new)  # alternative using a formatter instance
+```
+
+* `:object` - `Lumberjack::Formatter::ObjectFormatter` - no op conversion that returns the object itself.
+* `:string` - `Lumberjack::Formatter::StringFormatter` - calls `to_s` on the object.
+* `:strip` - `Lumberjack::Formatter::StripFormatter` - calls `to_s.strip` on the object.
+* `:inspect` - `Lumberjack::Formatter::InspectFormatter` - calls `inspect` on the object.
+* `:exception` - `Lumberjack::Formatter::ExceptionFormatter` - special formatter for exceptions which logs them as multi line statements with the message and backtrace.
+* `:date_time` - `Lumberjack::Formatter::DateTimeFormatter` - special formatter for dates and times to format them using `strftime`.
+* `:pretty_print` - `Lumberjack::Formatter::PrettyPrintFormatter` - returns the pretty print format of the object.
+* `:id` - `Lumberjack::Formatter::IdFormatter` - returns a hash of the object with keys for the id attribute and class.
+* `:structured` - `Lumberjack::Formatter::StructuredFormatter` - crawls the object and applies the formatter recursively to Enumerable objects found in it (arrays, hashes, etc.).
+
+To define your own formatter, either provide a block or an object that responds to `call` with a single argument.
+
+The default formatter will pass through values for strings, numbers, and booleans, and use the `:inspect` formatter for all objects except for exceptions which will be formatted with the `:exception` formatter.
+
+#### Tag Formatters
+
+The `logger.formatter` will only apply to log messages. You can use `logger.tag_formatter` to register formatters for tags. You can register both default formatters that will apply to all tag values, as well as tag specifice formatters that will apply only to objects with a specific tag name.
+
+The fomatter values can be either a `Lumberjack::Formatter` or a block or an object that responds to `call`. If you supply a `Lumberjack::Formatter`, the tag value will be passed through the rules for that formatter. If you supply a block or other object, it will be called with the tag value.
+
+```ruby
+# These will all do the same thing formatting all tag values with `inspect`
+logger.tag_formatter.default(Lumberjack::Formatter.new.clear.add(Object, :inspect))
+logger.tag_formatter.default(Lumberjack::Formatter::InspectFormatter.new)
+logger.tag_formatter.default { |value| value.inspect }
+
+# This will register formatters only on specific tag names
+logger.tag_formatter.add(:thread) { |thread| "Thread(#{thread.name})" }
+logger.tag_formatter.add(:current_user, Lumberjack::Formatter::IdFormatter.new)
 ```
 
-If you use the built in devices, you can also customize the Template used to format the LogEntry.
-  
+#### Templates
+
+If you use the built in `Lumberjack::Writer` derived devices, you can also customize the Template used to format the LogEntry.
+
+See `Lumberjack::Template` for a complete list of macros you can use in the template. You can also use a block that receives a `Lumberjack::LogEntry` as a template.
+
 ```ruby
   # Change the format of the time in the log
   Lumberjack::Logger.new("application.log", :time_format => "%m/%d/%Y %H:%M:%S")
@@ -84,23 +191,27 @@ If you use the built in devices, you can also customize the Template used to for
   # Use a simple template that only includes the time and the message
   Lumberjack::Logger.new("application.log", :template => ":time - :message")
 
+  # Use a simple template that includes tags, but handles the `duration` tag separately.
+  # All tags will appear at the end of the message except for `duration` which will be at the beginning.
+  Lumberjack::Logger.new("application.log", :template => ":time (:duration) - :message - :tags")
+
   # Use a custom template as a block that only includes the first character of the severity
   template = lambda{|e| "#{e.severity_label[0, 1]} #{e.time} - #{e.message}"}
   Lumberjack::Logger.new("application.log", :template => template)
 ```
 
-### Buffered Performance
+### Buffered Logging
 
-The logger has hooks for devices that support buffering to increase performance by batching physical writes. Log entries are not guaranteed to be written until the Lumberjack::Logger#flush method is called.
+The logger has hooks for devices that support buffering to potentially increase performance by batching physical writes. Log entries are not guaranteed to be written until the Lumberjack::Logger#flush method is called. Buffering can improve performance if I/O is slow or there high overhead writing to the log device.
 
-You can use the <tt>:flush_seconds</tt> option on the logger to periodically flush the log. This is usually a good idea so you can more easily debug hung processes. Without periodic flushing, a process that hangs may never write anything to the log because the messages are sitting in a buffer. By turning on periodic flushing, the logged messages will be written which can greatly aid in debugging the problem.
+You can use the `:flush_seconds` option on the logger to periodically flush the log. This is usually a good idea so you can more easily debug hung processes. Without periodic flushing, a process that hangs may never write anything to the log because the messages are sitting in a buffer. By turning on periodic flushing, the logged messages will be written which can greatly aid in debugging the problem.
 
-The built in stream based logging devices use an internal buffer. The size of the buffer (in bytes) can be set with the <tt>:buffer_size</tt> options when initializing a logger. The default behavior is to not to buffer.
+The built in stream based logging devices use an internal buffer. The size of the buffer (in bytes) can be set with the `:buffer_size` options when initializing a logger. The default behavior is to not to buffer.
 
 ```ruby
   # Set buffer to flush after 8K has been written to the log.
   logger = Lumberjack::Logger.new("application.log", :buffer_size => 8192)
-  
+
   # Turn off buffering so entries are immediately written to disk.
   logger = Lumberjack::Logger.new("application.log", :buffer_size => 0)
 ```
@@ -111,6 +222,14 @@ The built in devices include two that can automatically roll log files based eit
 
 There is a similar feature in the standard library Logger class, but the implementation here is safe to use with multiple processes writing to the same log file.
 
+## Difference Standard Library Logger
+
+`Lumberjack::Logger` does not extend from the `Logger` class in the standard library, but it does implement a compantible API. The main difference is in the flow of how messages are ultimately sent to devices for output.
+
+The standard library Logger logic converts the log entries to strings and then sends the string to the device to be written to a stream. Lumberjack, on the other hand, sends structured data in the form of a `Lumberjack::LogEntry` to the device and lets the device worry about how to format it. The reason for this flip is to better support structured data logging. Devices (even ones that write to streams) can format the entire payload including non-string objects and tags however they need to.
+
+The logging methods (`debug`, 'info', 'warn', 'error', 'fatal') are overloaded with an additional argument for setting tags on the log entry.
+
 ## Examples
 
 These example are for Rails applications, but there is no dependency on Rails for using this gem. Most of the examples are applicable to any Ruby application.
@@ -139,17 +258,17 @@ To set up a logger to roll log files when they get to 100Mb, you could use this:
 ```ruby
   config.logger = Lumberjack::Logger.new(log_file_path, :max_size => 100.megabytes)
 ```
-  
+
 To change the log message format, you could use this code:
 
 ```ruby
   config.logger = Lumberjack::Logger.new(log_file_path, :template => ":time - :message")
 ```
 
-To change the log message format to output JSON, you could use this code (this example requires the multi-json gem):
-  
+To change the log message format to output JSON, you could use this code:
+
 ```ruby
-  config.logger = Lumberjack::Logger.new(log_file_path, :template => lambda{|e| MultiJson.dump(e)})
+  config.logger = Lumberjack::Logger.new(log_file_path, :template => lambda{|e| JSON.dump(time: e.time, level: e.severity_label, message: e.message)})
 ```
 
 To send log messages to syslog instead of to a file, you could use this (require the lumberjack_syslog_device gem):

data/VERSION

@@ -1 +1 @@
-1.0.13
+1.2.8

data/lib/lumberjack.rb

@@ -1,21 +1,27 @@
 # frozen_string_literals: true
 
-require 'rbconfig'
-require 'time'
-require 'thread'
-require 'securerandom'
+require "rbconfig"
+require "time"
+require "securerandom"
+require "logger"
 
 module Lumberjack
-  LINE_SEPARATOR = (RbConfig::CONFIG['host_os'].match(/mswin/i) ? "\r\n" : "\n")
-
-  require File.expand_path("../lumberjack/severity.rb", __FILE__)
-  require File.expand_path("../lumberjack/log_entry.rb", __FILE__)
-  require File.expand_path("../lumberjack/formatter.rb", __FILE__)
-  require File.expand_path("../lumberjack/device.rb", __FILE__)
-  require File.expand_path("../lumberjack/logger.rb", __FILE__)
-  require File.expand_path("../lumberjack/template.rb", __FILE__)
-  require File.expand_path("../lumberjack/rack.rb", __FILE__)
-  
+  LINE_SEPARATOR = (RbConfig::CONFIG["host_os"] =~ /mswin/i ? "\r\n" : "\n")
+
+  require_relative "lumberjack/severity"
+  require_relative "lumberjack/formatter"
+
+  require_relative "lumberjack/context"
+  require_relative "lumberjack/log_entry"
+  require_relative "lumberjack/device"
+  require_relative "lumberjack/logger"
+  require_relative "lumberjack/tags"
+  require_relative "lumberjack/tag_formatter"
+  require_relative "lumberjack/tagged_logger_support"
+  require_relative "lumberjack/tagged_logging"
+  require_relative "lumberjack/template"
+  require_relative "lumberjack/rack"
+
   class << self
     # Define a unit of work within a block. Within the block supplied to this
     # method, calling +unit_of_work_id+ will return the same value that can
@@ -27,19 +33,61 @@ module Lumberjack
     # For the common use case of treating a single web request as a unit of work, see the
     # Lumberjack::Rack::UnitOfWork class.
     def unit_of_work(id = nil)
-      save_val = Thread.current[:lumberjack_logger_unit_of_work_id]
       id ||= SecureRandom.hex(6)
-      Thread.current[:lumberjack_logger_unit_of_work_id] = id
-      begin
-        return yield
-      ensure
-        Thread.current[:lumberjack_logger_unit_of_work_id] = save_val
+      context do
+        context[:unit_of_work_id] = id
+        yield
       end
     end
-    
+
     # Get the UniqueIdentifier for the current unit of work.
     def unit_of_work_id
-      Thread.current[:lumberjack_logger_unit_of_work_id] 
+      context[:unit_of_work_id]
+    end
+
+    # Contexts can be used to store tags that will be attached to all log entries in the block.
+    #
+    # If this method is called with a block, it will set a logging context for the scope of a block.
+    # If there is already a context in scope, a new one will be created that inherits
+    # all the tags of the parent context.
+    #
+    # Otherwise, it will return the current context. If one doesn't exist, it will return a new one
+    # but that context will not be in any scope.
+    def context(&block)
+      current_context = Thread.current[:lumberjack_context]
+      if block
+        use_context(Context.new(current_context), &block)
+      else
+        current_context || Context.new
+      end
+    end
+
+    # Set the context to use within a block.
+    def use_context(context, &block)
+      current_context = Thread.current[:lumberjack_context]
+      begin
+        Thread.current[:lumberjack_context] = (context || Context.new)
+        yield
+      ensure
+        Thread.current[:lumberjack_context] = current_context
+      end
+    end
+
+    # Return true if inside a context block.
+    def context?
+      !!Thread.current[:lumberjack_context]
+    end
+
+    # Return the tags from the current context or nil if there are no tags.
+    def context_tags
+      context = Thread.current[:lumberjack_context]
+      context&.tags
+    end
+
+    # Set tags on the current context
+    def tag(tags)
+      context = Thread.current[:lumberjack_context]
+      context&.tag(tags)
     end
   end
 end