yarder 0.0.1

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2012 YOURNAME
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,87 @@
+# Yarder
+
+[![Build Status](https://secure.travis-ci.org/rurounijones/yarder.png)](http://travis-ci.org/rurounijones/yarder)
+[![Dependency Status](https://gemnasium.com/rurounijones/yarder.png)](https://gemnasium.com/rurounijones/yarder)
+[![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/rurounijones/yarder)
+
+JSON Based Replacement logging system for Ruby on Rails.
+
+This is an experimental gem to see how easy / difficult it is to completely replace the default Ruby 
+on Rails logging system with one based on outputting JSON messages.
+
+This gem will create JSON based log entries designed for consumption by Logstash (although being 
+JSON they can be read by other software). The JSON will contain the same information as can be found 
+in the default rails logging output.
+
+## Current Status
+
+This gem is not production ready however it is probably ready for people interested in seeing the 
+results. All logging in a Rails3 app should be JSON formatted, including ad-hoc logging.
+
+Yarder has only been tested against Rails 3.2.8 on Ruby 1.9.3 and JRuby running in 1.9 mode. Test
+coverage is reasonable and most of the original Rails3 logging tests are passing. Additional tests
+unique to this gem still need to be created.
+
+There may be issues regarding outputting UTF-8 characters in logs on JRuby 1.6 in --1.9 mode. JRuby
+1.7 is recommended (These same issues exist in the man ruby loggers so use that as a guide).
+
+Any help, feedback or pull-requests would be much appreciated, especially related to refactoring and 
+test improvement
+
+## Installation
+
+Add this line to your Rails application's Gemfile:
+
+```ruby
+gem 'yarder'
+```
+
+## Configuration
+
+Yarder uses the Rails logger (set using config.logger in application.rb)to log output.
+
+By default Rails uses the TaggedLogging class to provide this however because Yarder
+replaces it you will need to change the default to something else.
+
+You will need to specify a Ruby Logger compatible logger. Yarder provides its own
+logger which is a copy of the ActiveSupport::Logger (Formerly known as
+ActiveSupport::BufferedLogger)
+
+If you are not sure what you want yet then set the Yarder::Logger as in the example
+below in your application.rb file.
+
+```ruby
+module MyApp
+  class Application < Rails::Application
+
+    # Set a logger compatible with the standard ruby logger to be used by Yarder
+    config.logger = Yarder::Logger.new(Rails.root.join('log',"#{Rails.env}.log").to_s)
+
+  end
+end
+```
+
+## Logstash Configuration
+
+Yarder currently creates log entries with a hard-coded logtype of "rails_json_log" (This may change 
+in future and may become configurable) therefore your Logstash configuration file should be as 
+follows:
+
+```
+input {
+  file {
+    type => "rails_json_log"
+    path => "/var/www/rails/application-1/log/production.log" # Path to your log file
+    format => "json_event"
+  }
+}
+```
+
+You will need to edit the path to point to your application's log file. Because Yarder creates json 
+serialized Logstash::Event entries there is no need to setup any filters
+
+### Known issues
+
+Yarder currently creates nested JSON. While this is supported in Logstash and Elastic Search the web 
+interfaces do not as yet support it. Depending on whether support is possible or not Yarder may 
+change to a non-nested format.

data/Rakefile

@@ -0,0 +1,38 @@
+#!/usr/bin/env rake
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+begin
+  require 'rdoc/task'
+rescue LoadError
+  require 'rdoc/rdoc'
+  require 'rake/rdoctask'
+  RDoc::Task = Rake::RDocTask
+end
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'Yarder'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.rdoc')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+
+
+
+Bundler::GemHelper.install_tasks
+
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+
+task :default => :test

data/lib/tasks/yarder_tasks.rake

@@ -0,0 +1,4 @@
+# desc "Explaining what the task does"
+# task :yarder do
+#   # Task goes here
+# end

data/lib/yarder.rb

@@ -0,0 +1,15 @@
+require 'yarder/railtie' if defined?(Rails)
+require 'yarder/rack/logger'
+require 'logstash-event'
+require 'yarder/logger'
+require 'yarder/tagged_logging'
+
+module Yarder
+
+  class IncompatibleLogger < StandardError; end
+
+  def self.log_entries
+    @@events ||= {}
+  end
+
+end

data/lib/yarder/action_controller/log_subscriber.rb

@@ -0,0 +1,74 @@
+require 'yarder/core_ext/object/blank'
+
+module Yarder
+  module ActionController
+    class LogSubscriber < ::ActiveSupport::LogSubscriber
+      INTERNAL_PARAMS = %w(controller action format _method only_path)
+
+      def start_processing(event)
+        payload = event.payload
+
+        entry.fields['controller'] = payload[:controller]
+        entry.fields['action'] = payload[:action]
+
+        format  = payload[:format]
+        entry.fields['format']  = format.to_s.downcase if format.is_a?(Symbol)
+
+      end
+
+      def process_action(event)
+
+        payload   = event.payload
+        additions = ::ActionController::Base.log_process_action(payload)
+
+        params = payload[:params].except(*INTERNAL_PARAMS)
+        entry.fields['parameters'] = params unless params.empty?
+
+        entry.fields['controller_duration'] = event.duration
+
+        #TODO  What on earth are additions and how should we handle them?
+        # message << " (#{additions.join(" | ")})" unless additions.blank?
+
+      end
+
+      def halted_callback(event)
+        entry.fields['halted_callback'] = event.payload[:filter]
+      end
+
+      def send_file(event)
+        entry.fields['send_file'] = event.payload[:path]
+        entry.fields['send_file_duration'] = event.duration
+      end
+
+      def redirect_to(event)
+        entry.fields['redirect_to'] = event.payload[:location]
+      end
+
+      def send_data(event)
+        entry.fields['send_data'] = event.payload[:filename]
+        entry.fields['send_data_duration'] = event.duration
+      end
+
+      %w(write_fragment read_fragment exist_fragment?
+         expire_fragment expire_page write_page).each do |method|
+        class_eval <<-METHOD, __FILE__, __LINE__ + 1
+          def #{method}(event)
+            entry.fields['cache'] ||= []
+            cache_event = {}
+            cache_event['key_or_path'] = event.payload[:key] || event.payload[:path]
+            cache_event['type'] = #{method.to_s.humanize.inspect}
+            cache_event['duration'] = event.duration
+            entry.fields['cache'] << cache_event
+          end
+        METHOD
+      end
+
+    private
+
+      def entry
+        Yarder.log_entries[Thread.current]
+      end
+
+    end
+  end
+end

data/lib/yarder/action_view/log_subscriber.rb

@@ -0,0 +1,31 @@
+module Yarder
+  module ActionView
+
+    class LogSubscriber < ::ActiveSupport::LogSubscriber
+
+      def render_template(event)
+        entry.fields['rendering'] ||= []
+        render_entry = {}
+        render_entry['identifier'] = from_rails_root(event.payload[:identifier])
+        render_entry['layout'] = from_rails_root(event.payload[:layout]) if event.payload[:layout]
+        render_entry['duration'] = event.duration
+
+        entry.fields['rendering'] << render_entry
+
+      end
+      alias :render_partial :render_template
+      alias :render_collection :render_template
+
+    private
+
+      def from_rails_root(string)
+        string.sub("#{Rails.root}/", "").sub(/^app\/views\//, "")
+      end
+
+      def entry
+        Yarder.log_entries[Thread.current]
+      end
+
+    end
+  end
+end

data/lib/yarder/active_record/log_subscriber.rb

@@ -0,0 +1,75 @@
+module Yarder
+  module ActiveRecord
+    class LogSubscriber < ::ActiveSupport::LogSubscriber
+
+      def self.runtime=(value)
+        Thread.current["active_record_sql_runtime"] = value
+      end
+
+      def self.runtime
+        Thread.current["active_record_sql_runtime"] ||= 0
+      end
+
+      def self.reset_runtime
+        rt, self.runtime = runtime, 0
+        rt
+      end
+
+      def initialize
+        super
+      end
+
+      def sql(event)
+        self.class.runtime += event.duration
+        return unless logger.debug?
+
+        payload = event.payload
+
+        return if 'SCHEMA' == payload[:name]
+
+        entry.fields['sql'] ||= []
+        sql_entry = {}
+        sql_entry['name'] = payload[:name]
+        sql_entry['duration'] = event.duration
+        sql_entry['sql']= payload[:sql].squeeze(' ')
+
+        binds = nil
+
+        unless (payload[:binds] || []).empty?
+          binds = "  " + payload[:binds].map { |col,v|
+            [col.name, v]
+          }.inspect
+        end
+
+        sql_entry['binds'] = binds unless binds.nil?
+
+        entry.fields['sql'] << sql_entry
+      end
+
+      def identity(event)
+        return unless logger.debug?
+
+        payload = event.payload
+
+        sql_entry = {}
+        sql_entry['name'] = payload[:name]
+        sql_entry['line'] = payload[:line]
+        sql_entry['duration'] = payload[:duration]
+
+        entry.fields['sql'] << sql_entry
+      end
+
+    private
+
+      def logger
+        ::ActiveRecord::Base.logger
+      end
+
+
+      def entry
+        Yarder.log_entries[Thread.current]
+      end
+
+    end
+  end
+end

data/lib/yarder/active_resource/log_subscriber.rb

@@ -0,0 +1,34 @@
+module Yarder
+  module ActiveResource
+    class LogSubscriber < ::ActiveSupport::LogSubscriber
+
+      def request(event)
+
+        #TODO Think of a better name for this!
+        entry.fields['active_resource'] ||= []
+
+        request_entry = {}
+
+        request_entry['method'] = event.payload[:method].to_s.upcase
+        request_entry['uri'] = event.payload[:request_uri]
+
+        result = event.payload[:result]
+
+        request_entry['code'] = result.code
+        request_entry['message'] = result.message
+        request_entry['length'] = result.body.to_s.length
+        request_entry['duration'] = event.duration
+
+        entry.fields['active_resource'] << request_entry
+      end
+
+    private
+
+      def entry
+        Yarder.log_entries[Thread.current]
+      end
+
+    end
+  end
+end
+

data/lib/yarder/configuration.rb

@@ -0,0 +1,8 @@
+module Yarder
+  class Configuration
+    attr_accessor :logger
+
+    def initialize
+    end
+  end
+end

data/lib/yarder/core_ext/object/blank.rb

@@ -0,0 +1,105 @@
+# encoding: utf-8
+
+class Object
+  # An object is blank if it's false, empty, or a whitespace string.
+  # For example, '', ' ', +nil+, [], and {} are all blank.
+  #
+  # This simplifies:
+  #
+  # if address.nil? || address.empty?
+  #
+  # ...to:
+  #
+  # if address.blank?
+  def blank?
+    respond_to?(:empty?) ? empty? : !self
+  end
+
+  # An object is present if it's not <tt>blank?</tt>.
+  def present?
+    !blank?
+  end
+
+  # Returns object if it's <tt>present?</tt> otherwise returns +nil+.
+  # <tt>object.presence</tt> is equivalent to <tt>object.present? ? object : nil</tt>.
+  #
+  # This is handy for any representation of objects where blank is the same
+  # as not present at all. For example, this simplifies a common check for
+  # HTTP POST/query parameters:
+  #
+  # state = params[:state] if params[:state].present?
+  # country = params[:country] if params[:country].present?
+  # region = state || country || 'US'
+  #
+  # ...becomes:
+  #
+  # region = params[:state].presence || params[:country].presence || 'US'
+  def presence
+    self if present?
+  end
+end
+
+class NilClass
+  # +nil+ is blank:
+  #
+  # nil.blank? # => true
+  def blank?
+    true
+  end
+end
+
+class FalseClass
+  # +false+ is blank:
+  #
+  # false.blank? # => true
+  def blank?
+    true
+  end
+end
+
+class TrueClass
+  # +true+ is not blank:
+  #
+  # true.blank? # => false
+  def blank?
+    false
+  end
+end
+
+class Array
+  # An array is blank if it's empty:
+  #
+  # [].blank? # => true
+  # [1,2,3].blank? # => false
+  alias_method :blank?, :empty?
+end
+
+class Hash
+  # A hash is blank if it's empty:
+  #
+  # {}.blank? # => true
+  # { key: 'value' }.blank? # => false
+  alias_method :blank?, :empty?
+end
+
+class String
+  # A string is blank if it's empty or contains whitespaces only:
+  #
+  # ''.blank? # => true
+  # ' '.blank? # => true
+  # '　'.blank? # => true
+  # ' something here '.blank? # => false
+  def blank?
+    self !~ /[^[:space:]]/
+  end
+end
+
+class Numeric #:nodoc:
+  # No number is blank:
+  #
+  # 1.blank? # => false
+  # 0.blank? # => false
+  def blank?
+    false
+  end
+end