lumberjack 1.0.13 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: aafe117d91082b575618e962e3d66a0b6d3c81b4
-  data.tar.gz: a4d32d48a82b15e8d67e0e00ab3b3637a63b7e99
+SHA256:
+  metadata.gz: 937acfbe9675acf0eecd3fa034e0818952967b00471e365d2ea3f0ab7fafa4d1
+  data.tar.gz: 0b82a010a7c4265c92cec3e7d5b0ffcb5f4ac5e151c7e1e8ee28ac8d0b3adf9b
 SHA512:
-  metadata.gz: 33d8f2e2e386edad724394264f76c33c17e3df6d9dd394b3444a7abebe12d5e93d3ce8a88726e8b67fcfb2c279615741a9e55b9112d5ec45e50e0c139f481a36
-  data.tar.gz: 129825a3150dad02aecde16130974c2add346fe27d6455e0ea22d6616d6e766306af8c7123ef10ec7e51ca81f06e7a6f1d0cd573abd7566f07b1b15efeed415a
+  metadata.gz: 42d94b2e4c35ded1291262acf02e3d886c9b5acc2d26bd40d5d6884ebced0658d8b7baeb83aa402f700a294cbdba05491aaabff4d9f7df9bb03c5edd3954f2a4
+  data.tar.gz: 2ddcf64ed40c77e57cc8152aba8241c6f92c2668d3a6f6aa58c674e46e35cb0e1a78491d24588d67fbdc13d1ce9904c5a200426948367fe8d9ef4b50bd00b23e

data/CHANGELOG.md

@@ -0,0 +1,74 @@
+## 1.1.0
+
+* Change Lumberjack::Logger to inherit from ::Logger
+* Add support for tags on log messages
+* Add global tag context for all loggers
+* Add per logger tags and tag contexts
+* Reimplement unit of work id as a tag on log entries
+* Add support for setting datetime format on log devices
+* Performance optimizations
+* Add Multi device to output to multiple devices
+* Add DateTimeFormatter, IdFormatter, ObjectFormatter, and StructuredFormatter
+* Add rack Context middleware for setting thread global context
+* End support for ruby versions < 2.3
+
+## 1.0.13
+
+* Reduce amount of code executed inside a mutex lock when writing to the logger stream.
+* Added `:min_roll_check` option to `Lumberjack::Device::RollingLogFile` to reduce file system checks. Default is now to only check if a file needs to be rolled at most once per second.
+* Force immutable strings for Ruby versions that support them.
+
+## 1.0.12
+
+* Add support for ActionDispatch request id for better Rails compatibility.
+
+## 1.0.11
+
+* Fix Ruby 2.4 deprecation warning on Fixnum (thanks koic).
+* Fix gemspec files to be flat array (thanks e2).
+
+## 1.0.10
+
+* Expose option to manually roll log files.
+* Minor code cleanup.
+
+## 1.0.9
+
+* Add method so Formatter is compatible with ActiveSupport logging extensions.
+
+## 1.0.8
+
+* Fix another internal variable name conflict with ActiveSupport logging extensions.
+
+## 1.0.7
+
+* Fix broken formatter attribute method.
+
+## 1.0.6
+
+* Fix internal variable name conflict with ActiveSupport logging extensions.
+
+## 1.0.5
+
+* Update docs.
+* Remove autoload calls to make thread safe.
+* Make compatible with Ruby 2.1.1 Pathname.
+* Make compatible with standard library Logger's use of progname as default message.
+
+## 1.0.4
+
+* Add ability to supply a unit of work id for a block instead of having one generated every time.
+
+## 1.0.3
+
+* Change log file output format to binary to avoid encoding warnings.
+* Fixed bug in log file rolling that left the file locked.
+
+## 1.0.2
+
+* Remove deprecation warnings under ruby 1.9.3.
+* Add more error checking around file rolling.
+
+## 1.0.1
+
+* Writes are no longer buffered by default.

data/README.md

@@ -1,5 +1,8 @@
 # Lumberjack
 
+[![Build Status](https://travis-ci.org/bdurand/lumberjack.svg?branch=master)](https://travis-ci.org/bdurand/lumberjack)
+[![Maintainability](https://api.codeclimate.com/v1/badges/a0abc03721fff9b0cde1/maintainability)](https://codeclimate.com/github/bdurand/lumberjack/maintainability)
+
 Lumberjack is a simple, powerful, and fast logging implementation in Ruby. It uses nearly the same API as the Logger class in the Ruby standard library and as ActiveSupport::BufferedLogger in Rails.
 
 ## Usage
@@ -33,21 +36,55 @@ The following information is recorded for each message:
 * time - The time at which the message was recorded.
 * program name - The name of the program logging the message. This can be either set for all messages or customized with each message.
 * process id - The process id (pid) of the process that logged the message.
-* unit of work id - The unique 12 byte hexadecimal number generated for a unit of work.
+* tags - An map of name value pairs for addition information about the log context.
+
+### Tags
 
-### Units Of Work
+You can use tags to provide additional meta data about a log message or the context that the log message is being made in. Using tags can keep you log messages clean. You can avoid string interoplation to add additional meta data.
 
-A unit of work can be used to tie together all log messages within a block. This can be very useful to isolate a group of messages that represent one path through the system. For instance, in a web application, a single request represents a natural unit of work,  and when you are looking through a log file, it is useful to see the entire set of log entries as a unit instead of interspersed with messages from other concurrent requests.
+Each of the logger methods includes an additional argument that can be used to specify tags on a messsage:
 
 ```ruby
-  # All log entries in this block will get a common unit of work id.
-  Lumberjack.unit_of_work do
-    logger.info("Begin request")
-    yield
-    logger.info("End request")
-  end
+logger.info("request completed", duration: elapsed_time, status: response.status)
+```
+
+You can also specify tags on a logger that will be included with every log message.
+
+```ruby
+logger.tag(host: Socket.gethostname)
+```
+
+You can specify tags that will only be applied to the logger in a block as well.
+
+```ruby
+logger.tag(thread_id: Thread.current.object_id) do
+  logger.info("here") # Will include the `thread_id` tag
+end
+logger.info("there") # Will not include the `thread_id` tag
+```
+
+Finally, you can specify a logging context with tags that apply within a block to all loggers.
+
+```ruby
+Lumberjack.context do
+  Lumberjack.tag(request_id: SecureRandom.hex)
+  logger.info("begin request") # Will include the `request_id` tag
+end
+logger.info("no requests") # Will not include the `request_id` tag
 ```
 
+Tag keys are always converted to strings. Tags are inherited so that message tags take precedence over block tags which take precedence over global tags.
+
+#### Templates
+
+The built in `Lumberjack::Device::Writer` class has built in support for including tags in the output using the `Lumberjack::Template` class.
+
+You can specify any tag name you want in a template as well as the `:tags` macro for all tags. If a tag name has been used as it's own macro, it will not be included in the `:tags` macro.
+
+#### Unit Of Work
+
+Lumberjack 1.0 had a concept of a unit of work id that could be used to tie log messages together. This has been replaced by tags. There is still an implementation of `Lumberjack.unit_of_work`, but it is just a wrapper on the tag implementation.
+
 ### Pluggable Devices
 
 When a Logger logs a LogEntry, it sends it to a Lumberjack::Device. Lumberjack comes with a variety of devices for logging to IO streams or files.
@@ -56,27 +93,50 @@ When a Logger logs a LogEntry, it sends it to a Lumberjack::Device. Lumberjack c
 * Lumberjack::Device::LogFile - Writes log entries to a file.
 * Lumberjack::Device::DateRollingLogFile - Writes log entries to a file that will automatically roll itself based on date.
 * Lumberjack::Device::SizeRollingLogFile - Writes log entries to a file that will automatically roll itself based on size.
+* Lumberjack::Device::Multi - This device wraps mulitiple other devices and will write log entries to each of them.
 * Lumberjack::Device::Null - This device produces no output and is intended for testing environments.
 
 If you'd like to send you log to a different kind of output, you just need to extend the Device class and implement the +write+ method. Or check out these plugins:
 
 * [lumberjack_syslog_device](https://github.com/bdurand/lumberjack_syslog_device) - send your log messages to the system wide syslog service
-* [lumberjack_multi-device](https://github.com/astevens/lumberjack_multi-device) - send log messages to multiple devices
 * [lumberjack_mongo_device](https://github.com/bdurand/lumberjack_mongo_device) - store your log messages to a [MongoDB](http://www.mongodb.org/) NoSQL data store
 * [lumberjack-couchdb-driver](https://github.com/narkisr/lumberjack-couchdb-driver) - store your log messages to a [CouchDB](http://couchdb.apache.org/) NoSQL data store
 * [lumberjack_heroku_device](https://github.com/tonycoco/lumberjack_heroku_device) - log to Heroku's logging system
 
 ### Customize Formatting
 
+#### Formatters
+
 When a message is logged, it is first converted into a string. You can customize how it is converted by adding mappings to a Formatter.
 
+There are several built in classes you can add as formatters
+
 ```ruby
   logger.formatter.add(Hash, :pretty_print)  # use the Formatter::PrettyPrintFormatter for all Hashes
+  logger.formatter.add(Hash, Lumberjack::Formatter::PrettyPrintFormatter.new)  # alternative using a formatter instance
+```
+
+* `Lumberjack::Formatter::ObjectFormatter` - no op conversion that returns the object itself.
+* `Lumberjack::Formatter::StringFormatter` - calls `to_s` on the object.
+* `Lumberjack::Formatter::InspectFormatter` - calls `inspect` on the object.
+* `Lumberjack::Formatter::ExceptionFormatter` - special formatter for exceptions which logs them as multi line statements with the message and backtrace.
+* `Lumberjack::Formatter::DateTimeFormatter` - special formatter for dates and times to format them using `strftime`.
+* `Lumberjack::Formatter::PrettyPrintFormatter` - returns the pretty print format of the object.
+* `Lumberjack::Formatter::IdFormatter` - returns a hash of the object with keys for the id attribute and class.
+* `Lumberjack::Formatter::StructuredFormatter` - crawls the object and applies the formatter recursively to Enumerable objects found in it (arrays, hashes, etc.).
+
+You can also specify a block to use as a formatter:
+
+```ruby
   logger.formatter.add(MyClass){|obj| "#{obj.class}@#{obj.id}"}  # use a block to provide a custom format
 ```
 
+#### Templates
+
 If you use the built in devices, you can also customize the Template used to format the LogEntry.
-  
+
+See `Lumberjack::Template` for a complete list of macros you can use in the template. You can also use a block that receives a `Lumberjack::LogEntry` as a template.
+
 ```ruby
   # Change the format of the time in the log
   Lumberjack::Logger.new("application.log", :time_format => "%m/%d/%Y %H:%M:%S")
@@ -84,23 +144,27 @@ If you use the built in devices, you can also customize the Template used to for
   # Use a simple template that only includes the time and the message
   Lumberjack::Logger.new("application.log", :template => ":time - :message")
 
+  # Use a simple template that includes tags, but handles the `duration` tag separately.
+  # All tags will appear at the end of the message except for `duration` which will be at the beginning.
+  Lumberjack::Logger.new("application.log", :template => ":time (:duration) - :message - :tags")
+
   # Use a custom template as a block that only includes the first character of the severity
   template = lambda{|e| "#{e.severity_label[0, 1]} #{e.time} - #{e.message}"}
   Lumberjack::Logger.new("application.log", :template => template)
 ```
 
-### Buffered Performance
+### Buffered Logging
 
-The logger has hooks for devices that support buffering to increase performance by batching physical writes. Log entries are not guaranteed to be written until the Lumberjack::Logger#flush method is called.
+The logger has hooks for devices that support buffering to potentially increase performance by batching physical writes. Log entries are not guaranteed to be written until the Lumberjack::Logger#flush method is called. Buffering can improve performance if I/O is slow or there high overhead writing to the log device.
 
-You can use the <tt>:flush_seconds</tt> option on the logger to periodically flush the log. This is usually a good idea so you can more easily debug hung processes. Without periodic flushing, a process that hangs may never write anything to the log because the messages are sitting in a buffer. By turning on periodic flushing, the logged messages will be written which can greatly aid in debugging the problem.
+You can use the `:flush_seconds` option on the logger to periodically flush the log. This is usually a good idea so you can more easily debug hung processes. Without periodic flushing, a process that hangs may never write anything to the log because the messages are sitting in a buffer. By turning on periodic flushing, the logged messages will be written which can greatly aid in debugging the problem.
 
-The built in stream based logging devices use an internal buffer. The size of the buffer (in bytes) can be set with the <tt>:buffer_size</tt> options when initializing a logger. The default behavior is to not to buffer.
+The built in stream based logging devices use an internal buffer. The size of the buffer (in bytes) can be set with the `:buffer_size` options when initializing a logger. The default behavior is to not to buffer.
 
 ```ruby
   # Set buffer to flush after 8K has been written to the log.
   logger = Lumberjack::Logger.new("application.log", :buffer_size => 8192)
-  
+
   # Turn off buffering so entries are immediately written to disk.
   logger = Lumberjack::Logger.new("application.log", :buffer_size => 0)
 ```
@@ -111,6 +175,14 @@ The built in devices include two that can automatically roll log files based eit
 
 There is a similar feature in the standard library Logger class, but the implementation here is safe to use with multiple processes writing to the same log file.
 
+## Difference Standard Library Logger
+
+`Lumberjack::Logger` does not extend from the `Logger` class in the standard library, but it does implement a compantible API. The main difference is in the flow of how messages are ultimately sent to devices for output.
+
+The standard library Logger logic converts the log entries to strings and then sends the string to the device to be written to a stream. Lumberjack, on the other hand, sends structured data in the form of a `Lumberjack::LogEntry` to the device and lets the device worry about how to format it. The reason for this flip is to better support structured data logging. Devices (even ones that write to streams) can format the entire payload including non-string objects and tags however they need to.
+
+The logging methods (`debug`, 'info', 'warn', 'error', 'fatal') are overloaded with an additional argument for setting tags on the log entry.
+
 ## Examples
 
 These example are for Rails applications, but there is no dependency on Rails for using this gem. Most of the examples are applicable to any Ruby application.
@@ -139,17 +211,17 @@ To set up a logger to roll log files when they get to 100Mb, you could use this:
 ```ruby
   config.logger = Lumberjack::Logger.new(log_file_path, :max_size => 100.megabytes)
 ```
-  
+
 To change the log message format, you could use this code:
 
 ```ruby
   config.logger = Lumberjack::Logger.new(log_file_path, :template => ":time - :message")
 ```
 
-To change the log message format to output JSON, you could use this code (this example requires the multi-json gem):
-  
+To change the log message format to output JSON, you could use this code:
+
 ```ruby
-  config.logger = Lumberjack::Logger.new(log_file_path, :template => lambda{|e| MultiJson.dump(e)})
+  config.logger = Lumberjack::Logger.new(log_file_path, :template => lambda{|e| JSON.dump(time: e.time, level: e.severity_label, message: e.message)})
 ```
 
 To send log messages to syslog instead of to a file, you could use this (require the lumberjack_syslog_device gem):

data/VERSION

@@ -1 +1 @@
-1.0.13
+1.1.0

data/lib/lumberjack.rb

@@ -4,18 +4,21 @@ require 'rbconfig'
 require 'time'
 require 'thread'
 require 'securerandom'
+require 'logger'
 
 module Lumberjack
   LINE_SEPARATOR = (RbConfig::CONFIG['host_os'].match(/mswin/i) ? "\r\n" : "\n")
 
-  require File.expand_path("../lumberjack/severity.rb", __FILE__)
-  require File.expand_path("../lumberjack/log_entry.rb", __FILE__)
-  require File.expand_path("../lumberjack/formatter.rb", __FILE__)
-  require File.expand_path("../lumberjack/device.rb", __FILE__)
-  require File.expand_path("../lumberjack/logger.rb", __FILE__)
-  require File.expand_path("../lumberjack/template.rb", __FILE__)
-  require File.expand_path("../lumberjack/rack.rb", __FILE__)
-  
+  require_relative "lumberjack/severity.rb"
+  require_relative "lumberjack/context.rb"
+  require_relative "lumberjack/log_entry.rb"
+  require_relative "lumberjack/formatter.rb"
+  require_relative "lumberjack/device.rb"
+  require_relative "lumberjack/logger.rb"
+  require_relative "lumberjack/tags.rb"
+  require_relative "lumberjack/template.rb"
+  require_relative "lumberjack/rack.rb"
+
   class << self
     # Define a unit of work within a block. Within the block supplied to this
     # method, calling +unit_of_work_id+ will return the same value that can
@@ -27,19 +30,51 @@ module Lumberjack
     # For the common use case of treating a single web request as a unit of work, see the
     # Lumberjack::Rack::UnitOfWork class.
     def unit_of_work(id = nil)
-      save_val = Thread.current[:lumberjack_logger_unit_of_work_id]
       id ||= SecureRandom.hex(6)
-      Thread.current[:lumberjack_logger_unit_of_work_id] = id
-      begin
-        return yield
-      ensure
-        Thread.current[:lumberjack_logger_unit_of_work_id] = save_val
+      context do
+        context[:unit_of_work_id] = id
+        yield
       end
     end
-    
+
     # Get the UniqueIdentifier for the current unit of work.
     def unit_of_work_id
-      Thread.current[:lumberjack_logger_unit_of_work_id] 
+      context[:unit_of_work_id]
+    end
+
+    # Contexts can be used to store tags that will be attached to all log entries in the block.
+    #
+    # If this method is called with a block, it will set a logging context for the scope of a block.
+    # If there is already a context in scope, a new one will be created that inherits
+    # all the tags of the parent context.
+    #
+    # Otherwise, it will return the current context. If one doesn't exist, it will return a new one
+    # but that context will not be in any scope.
+    def context
+      current_context = Thread.current[:lumberjack_context]
+      if block_given?
+        Thread.current[:lumberjack_context] = Context.new(current_context)
+        begin
+          yield
+        ensure
+          Thread.current[:lumberjack_context] = current_context
+        end
+      else
+        current_context || Context.new
+      end
+    end
+
+    # Return the tags from the current context or nil if there are no tags.
+    def context_tags
+      context = Thread.current[:lumberjack_context]
+      context.tags if context
     end
+
+    # Set tags on the current context
+    def tag(tags)
+      context = Thread.current[:lumberjack_context]
+      context.tag(tags) if context
+    end
+
   end
 end

data/lib/lumberjack/context.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+module Lumberjack
+  # A context is used to store tags that are then added to all log entries within a block.
+  class Context
+    attr_reader :tags
+
+    def initialize(parent_context = nil)
+      @tags = {}
+      @tags.merge!(parent_context.tags) if parent_context
+    end
+
+    # Set tags on the context.
+    def tag(tags)
+      tags.each do |key, value|
+        @tags[key.to_s] = value
+      end
+    end
+
+    # Get a context tag.
+    def [](key)
+      @tags[key.to_s]
+    end
+
+    # Set a context tag.
+    def []=(key, value)
+      @tags[key.to_s] = value
+    end
+
+    # Clear all the context data.
+    def reset
+      @tags.clear
+    end
+  end
+end

data/lib/lumberjack/device.rb

@@ -4,12 +4,13 @@ module Lumberjack
   # This is an abstract class for logging devices. Subclasses must implement the +write+ method and
   # may implement the +close+ and +flush+ methods if applicable.
   class Device
-    require File.expand_path("../device/writer.rb", __FILE__)
-    require File.expand_path("../device/log_file.rb", __FILE__)
-    require File.expand_path("../device/rolling_log_file.rb", __FILE__)
-    require File.expand_path("../device/date_rolling_log_file.rb", __FILE__)
-    require File.expand_path("../device/size_rolling_log_file.rb", __FILE__)
-    require File.expand_path("../device/null.rb", __FILE__)
+    require_relative "device/writer.rb"
+    require_relative "device/log_file.rb"
+    require_relative "device/rolling_log_file.rb"
+    require_relative "device/date_rolling_log_file.rb"
+    require_relative "device/size_rolling_log_file.rb"
+    require_relative "device/multi.rb"
+    require_relative "device/null.rb"
 
     # Subclasses must implement this method to write a LogEntry.
     def write(entry)
@@ -21,8 +22,21 @@ module Lumberjack
       flush
     end
     
+    # Subclasses may implement this method to reopen the device.
+    def reopen(logdev = nil)
+      flush
+    end
+    
     # Subclasses may implement this method to flush any buffers used by the device.
     def flush
     end
+    
+    # Subclasses may implement this method to get the format for log timestamps.
+    def datetime_format
+    end
+    
+    # Subclasses may implement this method to set a format for log timestamps.
+    def datetime_format=(format)
+    end
   end
 end