md-logstasher 1.0.0.rc1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 08fdc83eeea6e130441ff8d5f6a66a25d0a88565
4
+ data.tar.gz: 9d19f05f78e5b099647e5102d17e3a9f456c580b
5
+ SHA512:
6
+ metadata.gz: a0cfeefec296db67e6f1ca2b66d5eadb59a68ca272d3858afb03db7c95a9e57a0e69e4411efffc0044b90fd2e3e9b43c71e7d71e2307511e849a9f55bd7d9e8d
7
+ data.tar.gz: 458feac68319ae244968c182c66b15811556d6368d9d8b0c82a07c1dff4c557bdd0b458e63e276fc709f0afa3fe049ac88fc2bf4fe6a6b85fafbde5a56de5b1a
data/.gitignore ADDED
@@ -0,0 +1,5 @@
1
+ *.gem
2
+ .bundle
3
+ Gemfile.lock
4
+ pkg/*
5
+ .idea
data/.rspec ADDED
@@ -0,0 +1,2 @@
1
+ --color
2
+ --format progress
data/.travis.yml ADDED
@@ -0,0 +1,12 @@
1
+ language: ruby
2
+ rvm:
3
+ - 1.9.3
4
+ - 2.0.0
5
+
6
+ env:
7
+ - "RAILS_VERSION=4.0"
8
+ - "RAILS_VERSION=3.2"
9
+ - "RAILS_VERSION=3.1"
10
+ - "RAILS_VERSION=3.0"
11
+
12
+ script: bundle exec rspec
data/Gemfile ADDED
@@ -0,0 +1,10 @@
1
+ source "https://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in logstasher.gemspec
4
+ gemspec
5
+
6
+ group :test do
7
+ gem 'pry-nav'
8
+ gem 'rails'
9
+ gem 'timecop'
10
+ end
data/LICENSE.md ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2014 Shadab Ahmed
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,90 @@
1
+ # Logstasher
2
+
3
+ **Awesome Logging for Rails**
4
+
5
+ ## History
6
+
7
+ This is a fork of
8
+ [shadabahmed/logstasher](https://github.com/shadabahmed/logstasher). It has
9
+ been updated to use the [latest event
10
+ schema](https://logstash.jira.com/browse/LOGSTASH-675) and customized to better
11
+ fit the needs of MoneyDesktop--though it is still generally useful. It is not
12
+ backward compatible with its progenitor.
13
+
14
+ ## Purpose
15
+
16
+ This gem makes it easy to generate logstash compatible logs for your rails app.
17
+
18
+ A request that looks like this in your `production.log`:
19
+ ```text
20
+ Started GET "/login" for 10.109.10.135 at 2013-04-30 08:59:01 -0400
21
+ Processing by SessionsController#new as HTML
22
+ Rendered sessions/new.html.haml within layouts/application (4.3ms)
23
+ Rendered shared/_javascript.html.haml (0.6ms)
24
+ Rendered shared/_flashes.html.haml (0.2ms)
25
+ Rendered shared/_header.html.haml (52.9ms)
26
+ Rendered shared/_title.html.haml (0.2ms)
27
+ Rendered shared/_footer.html.haml (0.2ms)
28
+ Completed 200 OK in 532ms (Views: 62.4ms | ActiveRecord: 0.0ms | ND API: 0.0ms)
29
+ ```
30
+
31
+ Will look like this in your `logstash_production.log`:
32
+ ```json
33
+ {"tags":["request"],"method":"GET","path":"/","format":"html","controller":"file_servers"
34
+ ,"action":"index","status":200,"duration":28.34,"view":25.96,"db":0.88,"ip":"127.0.0.1","route":"file_servers#index",
35
+ "parameters":"","ndapi_time":null,"uuid":"e81ecd178ed3b591099f4d489760dfb6","user":"shadab_ahmed@abc.com",
36
+ "site":"internal","@timestamp":"2013-04-30T13:00:46.354500+00:00","@version":"1"}
37
+ ```
38
+
39
+ From there, it's trivial to forward them to your logstash indexer. You can even
40
+ use the included redis log device to send the logs directly to a redis broker
41
+ instead.
42
+
43
+ ## Installation
44
+
45
+ In your Gemfile:
46
+
47
+ gem 'md-logstasher'
48
+
49
+ ### Configure your `<environment>.rb` e.g. `development.rb`
50
+
51
+ # Enable the logstasher logs for the current environment
52
+ config.logstasher.enabled = true
53
+
54
+ # Optionally silience the standard logging to <environment>.log
55
+ config.logstasher.silence_standard_logging = true
56
+
57
+ ## Logging params
58
+
59
+ By default, Logstasher will add params as a JSON encoded string. To disable,
60
+ add the following to your `<environment>.rb`
61
+
62
+ # Disable logging of request parameters
63
+ config.logstasher.include_parameters = false
64
+
65
+ ## Adding custom fields to the log
66
+
67
+ Since some fields are very specific to your application, e.g., *user_name*,
68
+ it is left upto you to add them. Here's how to do it:
69
+
70
+ # Create a file - config/initializers/logstasher.rb
71
+
72
+ if LogStasher.enabled
73
+ LogStasher.append_fields do |fields|
74
+ # This block is run in application_controller context,
75
+ # so you have access to all controller methods
76
+ fields[:user] = current_user && current_user.mail
77
+ fields[:site] = request.path =~ /^\/api/ ? 'api' : 'user'
78
+ end
79
+ end
80
+
81
+ ## Versions
82
+ All versions require Rails 3.0.x and higher and Ruby 1.9.2+. Tested on Rails 4
83
+ and Ruby 2.0
84
+
85
+ ## Development
86
+ - Run tests - `rake`
87
+
88
+ ## Copyright
89
+
90
+ Copyright (c) 2014 Shadab Ahmed, released under the MIT license
data/Rakefile ADDED
@@ -0,0 +1,21 @@
1
+ require 'bundler/gem_tasks'
2
+ require 'rspec/core/rake_task'
3
+
4
+ desc 'Default: run specs.'
5
+ task :default => :spec
6
+
7
+ desc "Run specs"
8
+ RSpec::Core::RakeTask.new('spec') do |spec|
9
+ spec.pattern = "./spec/**/*_spec.rb"
10
+ end
11
+
12
+ require 'rdoc/task'
13
+ RDoc::Task.new do |rdoc|
14
+ require 'logstasher/version'
15
+ version = LogStasher::VERSION
16
+
17
+ rdoc.rdoc_dir = 'rdoc'
18
+ rdoc.title = "logstasher #{version}"
19
+ rdoc.rdoc_files.include('README*')
20
+ rdoc.rdoc_files.include('lib/**/*.rb')
21
+ end
@@ -0,0 +1,14 @@
1
+ module LogStasher
2
+ module ContextWrapper
3
+ def process_action(*)
4
+ Thread.current[:logstasher_context] = {
5
+ :controller => self,
6
+ :request => request
7
+ }
8
+
9
+ super
10
+ ensure
11
+ Thread.current[:logstasher_context] = nil
12
+ end
13
+ end
14
+ end
@@ -0,0 +1,64 @@
1
+ require 'redis'
2
+
3
+ module LogStasher
4
+ module Device
5
+ class Redis
6
+
7
+ attr_reader :options, :redis
8
+
9
+ def initialize(options = {})
10
+ @options = default_options.merge(options)
11
+ validate_options
12
+ configure_redis
13
+ end
14
+
15
+ def data_type
16
+ options[:data_type]
17
+ end
18
+
19
+ def key
20
+ options[:key]
21
+ end
22
+
23
+ def redis_options
24
+ unless @redis_options
25
+ default_keys = default_options.keys
26
+ @redis_options = options.select { |k, v| !default_keys.include?(k) }
27
+ end
28
+
29
+ @redis_options
30
+ end
31
+
32
+ def write(log)
33
+ case data_type
34
+ when 'list'
35
+ redis.rpush(key, log)
36
+ when 'channel'
37
+ redis.publish(key, log)
38
+ else
39
+ fail "Unknown data type #{data_type}"
40
+ end
41
+ end
42
+
43
+ def close
44
+ redis.quit
45
+ end
46
+
47
+ private
48
+
49
+ def configure_redis
50
+ @redis = ::Redis.new(redis_options)
51
+ end
52
+
53
+ def default_options
54
+ { key: 'logstash', data_type: 'list' }
55
+ end
56
+
57
+ def validate_options
58
+ unless ['list', 'channel'].include?(options[:data_type])
59
+ fail 'Expected :data_type to be either "list" or "channel"'
60
+ end
61
+ end
62
+ end
63
+ end
64
+ end
@@ -0,0 +1,101 @@
1
+ # inspired by [lumberjack](https://github.com/bdurand/lumberjack_syslog_device)
2
+
3
+ require 'syslog'
4
+ require 'thread'
5
+
6
+ module LogStasher
7
+ module Device
8
+ class Syslog
9
+
10
+ SEMAPHORE = Mutex.new
11
+
12
+ attr_reader :options
13
+
14
+ def initialize(options = {})
15
+ @options = parse_options(default_options.merge(options))
16
+ @closed = false
17
+ end
18
+
19
+ def close
20
+ SEMAPHORE.synchronize do
21
+ ::Syslog.close if ::Syslog.opened?
22
+ end
23
+
24
+ @closed = true
25
+ end
26
+
27
+ def closed?
28
+ @closed
29
+ end
30
+
31
+ def facility
32
+ options[:facility]
33
+ end
34
+
35
+ def flags
36
+ options[:flags]
37
+ end
38
+
39
+ def identity
40
+ options[:identity]
41
+ end
42
+
43
+ def priority
44
+ options[:priority]
45
+ end
46
+
47
+ def write(log)
48
+ fail ::RuntimeError, 'Cannot write. The device has been closed.' if closed?
49
+
50
+ with_syslog_open do
51
+ ::Syslog.log(facility, log)
52
+ end
53
+ end
54
+
55
+ private
56
+
57
+ def default_options
58
+ {
59
+ :identity => 'logstasher',
60
+ :facility => ::Syslog::LOG_LOCAL0,
61
+ :priority => ::Syslog::LOG_INFO,
62
+ :flags => ::Syslog::LOG_PID | ::Syslog::LOG_CONS
63
+ }
64
+ end
65
+
66
+ def parse_option(value)
67
+ case value
68
+ when ::String
69
+ ::Syslog.const_get(value.to_s)
70
+ when ::Array
71
+ value.reduce(0) { |all, current| all |= parse_option(current) }
72
+ else
73
+ value
74
+ end
75
+ end
76
+
77
+ def parse_options(options)
78
+ options[:facility] = parse_option(options[:facility])
79
+ options[:priority] = parse_option(options[:priority])
80
+ options[:flags] = parse_option(options[:flags])
81
+ options
82
+ end
83
+
84
+ def syslog_configured?
85
+ ::Syslog.ident == identity && ::Syslog.options == flags && ::Syslog.facility == facility
86
+ end
87
+
88
+ def with_syslog_open
89
+ SEMAPHORE.synchronize do
90
+ if ::Syslog.opened?
91
+ ::Syslog.reopen(identity, flags, facility) unless syslog_configured?
92
+ else
93
+ ::Syslog.open(identity, flags, facility)
94
+ end
95
+
96
+ yield
97
+ end
98
+ end
99
+ end
100
+ end
101
+ end
@@ -0,0 +1,138 @@
1
+ require 'action_controller/log_subscriber'
2
+ require 'active_support/log_subscriber'
3
+ require 'json'
4
+ require 'logstash/event'
5
+
6
+ # active_support monkeypatches Hash#to_json with a version that does not encode
7
+ # time correctly for LogStash. Using JSON.generate bypasses the monkeypatch.
8
+ class LogStash::Event
9
+ def to_json
10
+ return JSON.generate(@data)
11
+ end
12
+ end
13
+
14
+ module LogStasher
15
+ class LogSubscriber < ::ActiveSupport::Subscriber
16
+
17
+ INTERNAL_PARAMS = ::ActionController::LogSubscriber::INTERNAL_PARAMS
18
+
19
+ def process_action(event)
20
+ payload = event.payload
21
+ tags = extract_tags(payload)
22
+ fields = extract_request(payload)
23
+
24
+ fields.merge! extract_status(payload)
25
+ fields.merge! runtimes(event)
26
+ fields.merge! location
27
+ fields.merge! extract_exception(payload)
28
+ fields.merge! extract_parameters(payload)
29
+ fields.merge! appended_fields
30
+
31
+ event = LogStash::Event.new(fields.merge('tags' => tags))
32
+
33
+ LogStasher.logger << event.to_json + "\n"
34
+ end
35
+
36
+ def redirect_to(event)
37
+ Thread.current[:logstasher_context][:location] = event.payload[:location]
38
+ end
39
+
40
+ private
41
+
42
+ def appended_fields
43
+ callback = ::LogStasher.append_fields_callback
44
+ {}.tap do |fields|
45
+ controller.instance_exec(fields, &callback) if callback
46
+ end
47
+ end
48
+
49
+ def controller
50
+ Thread.current[:logstasher_context][:controller]
51
+ end
52
+
53
+ def extract_request(payload)
54
+ {
55
+ :action => payload[:action],
56
+ :controller => payload[:controller],
57
+ :format => extract_format(payload),
58
+ :ip => request.remote_ip,
59
+ :method => payload[:method],
60
+ :path => extract_path(payload),
61
+ :route => "#{payload[:controller]}##{payload[:action]}"
62
+ }
63
+ end
64
+
65
+ # Monkey patching to enable exception logging
66
+ def extract_exception(payload)
67
+ if payload[:exception]
68
+ exception, message = payload[:exception]
69
+ status = ActionDispatch::ExceptionWrapper.status_code_for_exception(exception)
70
+ message = "#{exception}\n#{message}\n#{($!.backtrace.join("\n"))}"
71
+ { :status => status, :error => message }
72
+ else
73
+ {}
74
+ end
75
+ end
76
+
77
+ def extract_format(payload)
78
+ if ::ActionPack::VERSION::MAJOR == 3 && ::ActionPack::VERSION::MINOR == 0
79
+ payload[:formats].first
80
+ else
81
+ payload[:format]
82
+ end
83
+ end
84
+
85
+ def extract_parameters(payload)
86
+ if LogStasher.include_parameters?
87
+ { :params => JSON.generate(payload[:params].except(*INTERNAL_PARAMS)) }
88
+ else
89
+ {}
90
+ end
91
+ end
92
+
93
+ def extract_path(payload)
94
+ payload[:path].split("?").first
95
+ end
96
+
97
+ def extract_status(payload)
98
+ if payload[:status]
99
+ { :status => payload[:status].to_i }
100
+ else
101
+ { :status => 0 }
102
+ end
103
+ end
104
+
105
+ def extract_tags(payload)
106
+ tags = ['request']
107
+ tags.push('exception') if payload[:exception]
108
+ tags
109
+ end
110
+
111
+ def location
112
+ location = Thread.current[:logstasher_context][:location]
113
+
114
+ if location
115
+ { :location => location }
116
+ else
117
+ {}
118
+ end
119
+ end
120
+
121
+ def request
122
+ Thread.current[:logstasher_context][:request]
123
+ end
124
+
125
+ def runtimes(event)
126
+ {
127
+ :total => event.duration,
128
+ :view => event.payload[:view_runtime],
129
+ :db => event.payload[:db_runtime]
130
+ }.inject({:runtime => {}}) do |runtimes, (name, runtime)|
131
+ runtimes[:runtime][name] = runtime.to_f.round(2) if runtime
132
+ runtimes
133
+ end
134
+ end
135
+ end
136
+ end
137
+
138
+ ::LogStasher::LogSubscriber.attach_to :action_controller
@@ -0,0 +1,56 @@
1
+ module LogStasher
2
+ class Railtie < ::Rails::Railtie
3
+ config.logstasher = ::ActiveSupport::OrderedOptions.new
4
+ config.logstasher.enabled = false
5
+ config.logstasher.include_parameters = true
6
+ config.logstasher.silence_standard_logging = false
7
+ config.logstasher.logger = nil
8
+ config.logstasher.log_level = ::Logger::INFO
9
+
10
+ initializer 'logstasher.configure' do
11
+ options = config.logstasher
12
+
13
+ ::LogStasher.enabled = options.enabled
14
+ ::LogStasher.include_parameters = options.include_parameters
15
+ ::LogStasher.silence_standard_logging = options.silence_standard_logging
16
+ ::LogStasher.logger = options.logger || default_logger
17
+ ::LogStasher.logger.level = options.log_level
18
+ end
19
+
20
+ initializer 'logstasher.load' do
21
+ if ::LogStasher.enabled?
22
+ ::ActiveSupport.on_load(:action_controller) do
23
+ require 'logstasher/log_subscriber'
24
+ require 'logstasher/context_wrapper'
25
+
26
+ include ::LogStasher::ContextWrapper
27
+ end
28
+ end
29
+ end
30
+
31
+ config.after_initialize do
32
+ if ::LogStasher.enabled? && ::LogStasher.silence_standard_logging?
33
+ require 'logstasher/silent_logger'
34
+
35
+ ::Rails::Rack::Logger.send(:include, ::LogStasher::SilentLogger)
36
+
37
+ ::ActiveSupport::Subscriber.subscribers.each do |subscriber|
38
+ if subscriber.is_a?(::ActiveSupport::LogSubscriber)
39
+ subscriber.class.send(:include, ::LogStasher::SilentLogger)
40
+ end
41
+ end
42
+ end
43
+ end
44
+
45
+ def default_logger
46
+ unless @default_logger
47
+ path = ::Rails.root.join('log', "logstash_#{::Rails.env}.log")
48
+ ::FileUtils.touch(path) # prevent autocreate messages in log
49
+
50
+ @default_logger = ::Logger.new(path)
51
+ end
52
+
53
+ @default_logger
54
+ end
55
+ end
56
+ end
@@ -0,0 +1,16 @@
1
+ module LogStasher
2
+ module SilentLogger
3
+ def self.included(klass)
4
+ klass.class_eval do
5
+ def logger
6
+ unless @logstasher_silent_logger
7
+ @logstasher_silent_logger = ::Logger.new('/dev/null')
8
+ @logstasher_silent_logger.level = ::Logger::UNKNOWN
9
+ end
10
+
11
+ @logstasher_silent_logger
12
+ end
13
+ end
14
+ end
15
+ end
16
+ end
@@ -0,0 +1,3 @@
1
+ module LogStasher
2
+ VERSION = "1.0.0.rc1"
3
+ end
data/lib/logstasher.rb ADDED
@@ -0,0 +1,50 @@
1
+ require 'logger'
2
+
3
+ module LogStasher
4
+ class << self
5
+ attr_reader :append_fields_callback
6
+ attr_writer :enabled
7
+ attr_writer :include_parameters
8
+ attr_writer :silence_standard_logging
9
+
10
+ def append_fields(&block)
11
+ @append_fields_callback = block
12
+ end
13
+
14
+ def enabled?
15
+ @enabled ||= false
16
+ end
17
+
18
+ def include_parameters?
19
+ if @include_parameters.nil?
20
+ @include_parameters = true
21
+ end
22
+
23
+ @include_parameters
24
+ end
25
+
26
+ def initialize_logger(device = $stdout, level = ::Logger::INFO)
27
+ ::Logger.new(device).tap do |new_logger|
28
+ new_logger.level = level
29
+ end
30
+ end
31
+
32
+ def logger
33
+ @logger ||= initialize_logger
34
+ end
35
+
36
+ def logger=(log)
37
+ @logger = log
38
+ end
39
+
40
+ def silence_standard_logging?
41
+ if @silence_standard_logging.nil?
42
+ @silence_standard_logging = false
43
+ end
44
+
45
+ @silence_standard_logging
46
+ end
47
+ end
48
+ end
49
+
50
+ require 'logstasher/railtie' if defined?(Rails)
@@ -0,0 +1 @@
1
+ require 'logstasher'
File without changes
File without changes
data/log/test.log ADDED
File without changes
@@ -0,0 +1,25 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "logstasher/version"
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "md-logstasher"
7
+ s.version = LogStasher::VERSION
8
+ s.authors = ["Devin Christensen"]
9
+ s.email = ["devin.christensen@moneydesktop.com"]
10
+ s.homepage = "https://github.com/moneydesktop/logstasher"
11
+ s.summary = %q{Awesome rails logs}
12
+ s.description = %q{Awesome rails logs}
13
+ s.license = 'MIT'
14
+
15
+ s.files = `git ls-files`.split("\n")
16
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
17
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
18
+ s.require_paths = ["lib"]
19
+
20
+ s.add_runtime_dependency "logstash-event", "~> 1.2"
21
+
22
+ s.add_development_dependency "redis"
23
+ s.add_development_dependency "rspec"
24
+ s.add_development_dependency "bundler", ">= 1.0.0"
25
+ end
@@ -0,0 +1,79 @@
1
+ require 'spec_helper'
2
+
3
+ require 'logstasher/device/redis'
4
+
5
+ describe LogStasher::Device::Redis do
6
+
7
+ let(:redis_mock) { double('Redis') }
8
+
9
+ let(:default_options) {{
10
+ key: 'logstash',
11
+ data_type: 'list'
12
+ }}
13
+
14
+ it 'has default options' do
15
+ device = LogStasher::Device::Redis.new
16
+ device.options.should eq(default_options)
17
+ end
18
+
19
+ it 'creates a redis instance' do
20
+ ::Redis.should_receive(:new).with({})
21
+ LogStasher::Device::Redis.new()
22
+ end
23
+
24
+ it 'assumes unknown options are for redis' do
25
+ ::Redis.should_receive(:new).with(hash_including(db: '0'))
26
+ device = LogStasher::Device::Redis.new(db: '0')
27
+ device.redis_options.should eq(db: '0')
28
+ end
29
+
30
+ it 'has a key' do
31
+ device = LogStasher::Device::Redis.new(key: 'the_key')
32
+ device.key.should eq 'the_key'
33
+ end
34
+
35
+ it 'has a data_type' do
36
+ device = LogStasher::Device::Redis.new(data_type: 'channel')
37
+ device.data_type.should eq 'channel'
38
+ end
39
+
40
+ it 'does not allow unsupported data types' do
41
+ expect {
42
+ device = LogStasher::Device::Redis.new(data_type: 'blargh')
43
+ }.to raise_error()
44
+ end
45
+
46
+ it 'quits the redis connection on #close' do
47
+ device = LogStasher::Device::Redis.new
48
+ device.redis.should_receive(:quit)
49
+ device.close
50
+ end
51
+
52
+ it 'works as a logger device' do
53
+ device = LogStasher::Device::Redis.new
54
+ device.should_receive(:write).with('blargh')
55
+ logger = Logger.new(device)
56
+ logger << 'blargh'
57
+ end
58
+
59
+ describe '#write' do
60
+ it "rpushes logs onto a list" do
61
+ device = LogStasher::Device::Redis.new(data_type: 'list')
62
+ device.redis.should_receive(:rpush).with('logstash', 'the log')
63
+ device.write('the log')
64
+ end
65
+
66
+ it "rpushes logs onto a custom key" do
67
+ device = LogStasher::Device::Redis.new(data_type: 'list', key: 'custom')
68
+ device.redis.should_receive(:rpush).with('custom', 'the log')
69
+ device.write('the log')
70
+ end
71
+
72
+ it "publishes logs onto a channel" do
73
+ device = LogStasher::Device::Redis.new(data_type: 'channel', key: 'custom')
74
+ device.redis.should_receive(:publish).with('custom', 'the log')
75
+ device.write('the log')
76
+ end
77
+ end
78
+
79
+ end
@@ -0,0 +1,120 @@
1
+ require 'spec_helper'
2
+
3
+ require 'logstasher/device/syslog'
4
+
5
+ describe LogStasher::Device::Syslog do
6
+
7
+ let(:default_options) {{
8
+ :identity => 'logstasher',
9
+ :facility => ::Syslog::LOG_LOCAL0,
10
+ :priority => ::Syslog::LOG_INFO,
11
+ :flags => ::Syslog::LOG_PID | ::Syslog::LOG_CONS
12
+ }}
13
+
14
+ before { ::Syslog.stub(:log) }
15
+
16
+ around do |example|
17
+ ::Syslog.close if ::Syslog.opened?
18
+ example.run
19
+ ::Syslog.close if ::Syslog.opened?
20
+ end
21
+
22
+ it 'has default options' do
23
+ device = LogStasher::Device::Syslog.new
24
+ device.options.should eq(default_options)
25
+ end
26
+
27
+ it 'has an identity' do
28
+ device = LogStasher::Device::Syslog.new(:identity => 'rspec')
29
+ device.identity.should eq 'rspec'
30
+ end
31
+
32
+ it 'has a facility' do
33
+ device = LogStasher::Device::Syslog.new(:facility => ::Syslog::LOG_USER)
34
+ device.facility.should eq ::Syslog::LOG_USER
35
+ end
36
+
37
+ it 'accepts facility as a string' do
38
+ device = LogStasher::Device::Syslog.new(:facility => 'LOG_LOCAL7')
39
+ device.facility.should eq ::Syslog::LOG_LOCAL7
40
+ end
41
+
42
+ it 'has a priority' do
43
+ device = LogStasher::Device::Syslog.new(:priority => ::Syslog::LOG_CRIT)
44
+ device.priority.should eq ::Syslog::LOG_CRIT
45
+ end
46
+
47
+ it 'accepts priority as a string' do
48
+ device = LogStasher::Device::Syslog.new(:priority => 'LOG_AUTH')
49
+ device.priority.should eq ::Syslog::LOG_AUTH
50
+ end
51
+
52
+ it 'has flags' do
53
+ device = LogStasher::Device::Syslog.new(:flags => ::Syslog::LOG_NOWAIT)
54
+ device.flags.should eq ::Syslog::LOG_NOWAIT
55
+ end
56
+
57
+ it 'accepts flags as a string' do
58
+ device = LogStasher::Device::Syslog.new(:flags => 'LOG_NDELAY')
59
+ device.flags.should eq ::Syslog::LOG_NDELAY
60
+ end
61
+
62
+ it 'accepts flags as an array of strings' do
63
+ device = LogStasher::Device::Syslog.new(:flags => ['LOG_NOWAIT', 'LOG_ODELAY'])
64
+ device.flags.should eq(::Syslog::LOG_NOWAIT | ::Syslog::LOG_ODELAY)
65
+ end
66
+
67
+ describe '#write' do
68
+ subject { LogStasher::Device::Syslog.new }
69
+
70
+ it 'opens syslog when syslog is closed' do
71
+ ::Syslog.should_receive(:open).with(subject.identity, subject.flags, subject.facility)
72
+ subject.write('a log')
73
+ end
74
+
75
+ it 'does not re-open syslog when its config is in sync' do
76
+ ::Syslog.open(subject.identity, subject.flags, subject.facility)
77
+ ::Syslog.should_not_receive(:open)
78
+ ::Syslog.should_not_receive(:reopen)
79
+ subject.write('a log')
80
+ end
81
+
82
+ it 're-opens syslog when its config is out of sync' do
83
+ ::Syslog.open('temp', ::Syslog::LOG_NDELAY, ::Syslog::LOG_AUTH)
84
+ ::Syslog.should_receive(:reopen).with(subject.identity, subject.flags, subject.facility)
85
+ subject.write('a log')
86
+ end
87
+
88
+ it 'writes the log to syslog' do
89
+ ::Syslog.should_receive(:log).with(subject.facility, 'a log')
90
+ subject.write('a log')
91
+ end
92
+
93
+ it 'fails when the device is closed' do
94
+ subject.close
95
+ expect {
96
+ subject.write('a log')
97
+ }.to raise_error(::RuntimeError, 'Cannot write. The device has been closed.')
98
+ end
99
+ end
100
+
101
+ describe '#close' do
102
+ subject { LogStasher::Device::Syslog.new }
103
+
104
+ it 'closes the device' do
105
+ subject.close
106
+ subject.should be_closed
107
+ end
108
+
109
+ it 'closes syslog when syslog is open' do
110
+ ::Syslog.open(subject.identity, subject.flags, subject.facility)
111
+ ::Syslog.should_receive(:close)
112
+ subject.close
113
+ end
114
+
115
+ it 'does not close syslog if it is already closed' do
116
+ ::Syslog.should_not_receive(:close)
117
+ subject.close
118
+ end
119
+ end
120
+ end
@@ -0,0 +1,155 @@
1
+ require 'spec_helper'
2
+ require 'securerandom'
3
+
4
+ require 'logstasher/log_subscriber'
5
+
6
+ class MockController
7
+ def user_id
8
+ @user_id ||= SecureRandom.hex(16)
9
+ end
10
+ end
11
+
12
+ class MockRequest
13
+ def remote_ip
14
+ '127.0.0.1'
15
+ end
16
+ end
17
+
18
+ describe LogStasher::LogSubscriber do
19
+ subject { described_class.new }
20
+
21
+ let(:logger) { ::Logger.new('/dev/null') }
22
+ let(:mock_controller) { MockController.new }
23
+ let(:mock_request) { MockRequest.new }
24
+ let(:context) {{ :controller => mock_controller, :request => mock_request }}
25
+
26
+ around do |example|
27
+ backup_logger = LogStasher.logger
28
+ LogStasher.logger = logger
29
+ Thread.current[:logstasher_context] = context
30
+ Timecop.freeze { example.run }
31
+ Thread.current[:logstasher_context] = nil
32
+ LogStasher.logger = backup_logger
33
+ end
34
+
35
+ describe '#process_action' do
36
+ let(:timestamp) { ::Time.new.utc.iso8601(3) }
37
+ let(:duration) { 12.4 }
38
+ let(:json_params) { JSON.dump(payload[:params]) }
39
+ let(:payload) {{
40
+ :controller => 'users',
41
+ :action => 'show',
42
+ :params => { 'foo' => 'bar' },
43
+ :format => 'text/plain',
44
+ :method => 'method',
45
+ :path => '/users/1',
46
+ :status => 200
47
+ }}
48
+
49
+ let(:event) { double(:payload => payload, :duration => duration) }
50
+
51
+ it 'logs the event in logstash format' do
52
+ logger.should_receive(:<<) do |json|
53
+ JSON.parse(json).should eq({
54
+ '@timestamp' => timestamp,
55
+ '@version' => '1',
56
+ 'tags' => ['request'],
57
+ 'action' => payload[:action],
58
+ 'controller' => payload[:controller],
59
+ 'format' => payload[:format],
60
+ 'params' => json_params,
61
+ 'ip' => mock_request.remote_ip,
62
+ 'method' => payload[:method],
63
+ 'path' => payload[:path],
64
+ 'route' => "#{payload[:controller]}##{payload[:action]}",
65
+ 'status' => payload[:status],
66
+ 'runtime' => { 'total' => duration }
67
+ })
68
+ end
69
+
70
+ subject.process_action(event)
71
+ end
72
+
73
+ it 'appends fields to the log' do
74
+ ::LogStasher.append_fields do |fields|
75
+ fields['user_id'] = user_id
76
+ fields['other'] = 'stuff'
77
+ end
78
+
79
+ logger.should_receive(:<<) do |json|
80
+ fields = JSON.parse(json)
81
+ fields['user_id'].should eq mock_controller.user_id
82
+ fields['other'].should eq 'stuff'
83
+ end
84
+
85
+ subject.process_action(event)
86
+ end
87
+
88
+ it 'removes parameters from the log' do
89
+ ::LogStasher.stub(:include_parameters? => false)
90
+
91
+ logger.should_receive(:<<) do |json|
92
+ JSON.parse(json)['params'].should be_nil
93
+ end
94
+
95
+ subject.process_action(event)
96
+ end
97
+
98
+ it 'includes redirect location in the log' do
99
+ redirect_event = double(:payload => {:location => 'new/location'})
100
+ subject.redirect_to(redirect_event)
101
+
102
+ logger.should_receive(:<<) do |json|
103
+ JSON.parse(json)['location'].should eq 'new/location'
104
+ end
105
+
106
+ subject.process_action(event)
107
+ end
108
+
109
+ it 'includes runtimes in the log' do
110
+ payload.merge!({
111
+ :view_runtime => 3.3,
112
+ :db_runtime => 2.1
113
+ })
114
+
115
+ logger.should_receive(:<<) do |json|
116
+ runtime = JSON.parse(json)['runtime']
117
+ runtime['view'].should eq 3.3
118
+ runtime['db'].should eq 2.1
119
+ end
120
+
121
+ subject.process_action(event)
122
+ end
123
+
124
+ it 'includes exception info in the log' do
125
+ begin
126
+ fail RuntimeError, 'it no work'
127
+ rescue
128
+ # Test inside the rescue block so $! is properly set
129
+ payload.merge!({
130
+ :exception => ['RuntimeError', 'it no work']
131
+ })
132
+
133
+ logger.should_receive(:<<) do |json|
134
+ log = JSON.parse(json)
135
+ log['error'].should match /^RuntimeError\nit no work\n.+/m
136
+ log['status'].should eq 500
137
+ log['tags'].should include('exception')
138
+ end
139
+
140
+ subject.process_action(event)
141
+ end
142
+ end
143
+ end
144
+
145
+ describe '#redirect_to' do
146
+ let(:location) { "users/#{SecureRandom.hex(16)}" }
147
+ let(:payload) {{ :location => location }}
148
+ let(:event) { double(:payload => payload) }
149
+
150
+ it 'copies the location into the thread local logstasher context' do
151
+ subject.redirect_to(event)
152
+ Thread.current[:logstasher_context][:location].should eq location
153
+ end
154
+ end
155
+ end
@@ -0,0 +1,127 @@
1
+ require 'spec_helper'
2
+
3
+ require 'action_controller/railtie'
4
+ require 'action_controller/log_subscriber'
5
+ require 'action_view/railtie'
6
+ require 'action_view/log_subscriber'
7
+
8
+ require 'logstasher/context_wrapper'
9
+ require 'logstasher/log_subscriber'
10
+ require 'logstasher/railtie'
11
+ require 'logstasher/silent_logger'
12
+
13
+ ENV['RAILS_ENV'] = 'test'
14
+
15
+ class ::LogStasher::RailtieApp < ::Rails::Application
16
+ end
17
+
18
+ describe ::LogStasher::Railtie do
19
+ let(:config) { described_class.config.logstasher }
20
+
21
+ describe 'logstasher.configure' do
22
+ subject do
23
+ described_class.instance.initializers.find do |initializer|
24
+ initializer.name == 'logstasher.configure'
25
+ end
26
+ end
27
+
28
+ it 'should configure LogStasher' do
29
+ config.logger = ::Logger.new('/dev/null')
30
+ config.log_level = "log_level"
31
+ config.enabled = "enabled"
32
+ config.include_parameters = "include_parameters"
33
+ config.silence_standard_logging = "silence_standard_logging"
34
+
35
+ ::LogStasher.should_receive(:enabled=).with("enabled")
36
+ ::LogStasher.should_receive(:include_parameters=).with("include_parameters")
37
+ ::LogStasher.should_receive(:silence_standard_logging=).with("silence_standard_logging")
38
+ ::LogStasher.should_receive(:logger=).with(config.logger).and_call_original
39
+ config.logger.should_receive(:level=).with("log_level")
40
+
41
+ subject.run
42
+ end
43
+ end
44
+
45
+ describe 'logstasher.load' do
46
+ subject do
47
+ described_class.instance.initializers.find do |initializer|
48
+ initializer.name == 'logstasher.load'
49
+ end
50
+ end
51
+
52
+ context 'when logstasher is disabled' do
53
+ it 'does nothing' do
54
+ ::ActiveSupport.should_not_receive(:on_load)
55
+
56
+ subject.run
57
+ end
58
+ end
59
+
60
+ context 'when logstasher is enabled' do
61
+ before { ::LogStasher.stub(:enabled?) { true } }
62
+
63
+ it 'should load LogStasher into ActionController' do
64
+ ::ActionController.should_receive(:require).with('logstasher/log_subscriber')
65
+ ::ActionController.should_receive(:require).with('logstasher/context_wrapper')
66
+ ::ActionController.should_receive(:include).with(::LogStasher::ContextWrapper)
67
+
68
+ subject.run
69
+ ::ActiveSupport.run_load_hooks(:action_controller, ::ActionController)
70
+ end
71
+ end
72
+ end
73
+
74
+ describe 'config.after_initialize' do
75
+ context 'when logstasher is enabled' do
76
+ before { ::LogStasher.stub(:enabled?) { true } }
77
+
78
+ context 'and silence_standard_logging is enabled' do
79
+ before { ::LogStasher.stub(:silence_standard_logging?) { true } }
80
+
81
+ it 'should not silence standard logging' do
82
+ ::ActionController::LogSubscriber.should_receive(:include).with(::LogStasher::SilentLogger)
83
+ ::ActionView::LogSubscriber.should_receive(:include).with(::LogStasher::SilentLogger)
84
+ ::Rails::Rack::Logger.should_receive(:include).with(::LogStasher::SilentLogger)
85
+ ::ActiveSupport.run_load_hooks(:after_initialize, ::LogStasher::RailtieApp)
86
+ end
87
+ end
88
+
89
+ context 'and silence_standard_logging is disabled' do
90
+ before { ::LogStasher.stub(:silence_standard_logging?) { false } }
91
+
92
+ it 'should not silence standard logging' do
93
+ ::ActionController.should_not_receive(:include).with(::LogStasher::SilentLogger)
94
+ ::ActionView.should_not_receive(:include).with(::LogStasher::SilentLogger)
95
+ ::Rails::Rack::Logger.should_not_receive(:include).with(::LogStasher::SilentLogger)
96
+ ::ActiveSupport.run_load_hooks(:after_initialize, ::LogStasher::RailtieApp)
97
+ end
98
+ end
99
+ end
100
+
101
+ context 'when logstasher is disabled' do
102
+ before { ::LogStasher.stub(:enabled?) { false } }
103
+
104
+ context 'and silence_standard_logging is enabled' do
105
+ before { ::LogStasher.stub(:silence_standard_logging?) { true } }
106
+
107
+ it 'should not silence standard logging' do
108
+ ::ActionController::LogSubscriber.should_not_receive(:include).with(::LogStasher::SilentLogger)
109
+ ::ActionView::LogSubscriber.should_not_receive(:include).with(::LogStasher::SilentLogger)
110
+ ::Rails::Rack::Logger.should_not_receive(:include).with(::LogStasher::SilentLogger)
111
+ ::ActiveSupport.run_load_hooks(:after_initialize, ::LogStasher::RailtieApp)
112
+ end
113
+ end
114
+
115
+ context 'and silence_standard_logging is disabled' do
116
+ before { ::LogStasher.stub(:silence_standard_logging?) { false } }
117
+
118
+ it 'should not silence standard logging' do
119
+ ::ActionController.should_not_receive(:include).with(::LogStasher::SilentLogger)
120
+ ::ActionView.should_not_receive(:include).with(::LogStasher::SilentLogger)
121
+ ::Rails::Rack::Logger.should_not_receive(:include).with(::LogStasher::SilentLogger)
122
+ ::ActiveSupport.run_load_hooks(:after_initialize, ::LogStasher::RailtieApp)
123
+ end
124
+ end
125
+ end
126
+ end
127
+ end
@@ -0,0 +1,18 @@
1
+ require 'spec_helper'
2
+
3
+ describe LogStasher do
4
+ it 'has a version' do
5
+ ::LogStasher::VERSION.should_not be_nil
6
+ end
7
+
8
+ it 'has a logger' do
9
+ ::LogStasher.logger.should be_a_kind_of(::Logger)
10
+ end
11
+
12
+ it 'stores a callback for appending fields' do
13
+ callback = proc { |fields| fail 'Did not expect this to run' }
14
+
15
+ ::LogStasher.append_fields(&callback)
16
+ ::LogStasher.append_fields_callback.should be callback
17
+ end
18
+ end
@@ -0,0 +1,17 @@
1
+ # Notice there is a .rspec file in the root folder. It defines rspec arguments
2
+
3
+ # Modify load path so you can require logstasher directly.
4
+ $LOAD_PATH.unshift(File.expand_path('../../lib', __FILE__))
5
+
6
+ require 'rubygems'
7
+ require 'bundler/setup'
8
+
9
+ Bundler.require(:default, :development, :test)
10
+
11
+ require 'logstasher'
12
+
13
+ RSpec.configure do |config|
14
+ config.treat_symbols_as_metadata_keys_with_true_values = true
15
+ config.run_all_when_everything_filtered = true
16
+ config.filter_run :focus
17
+ end
metadata ADDED
@@ -0,0 +1,132 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: md-logstasher
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0.rc1
5
+ platform: ruby
6
+ authors:
7
+ - Devin Christensen
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2014-03-21 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: logstash-event
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ~>
18
+ - !ruby/object:Gem::Version
19
+ version: '1.2'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ~>
25
+ - !ruby/object:Gem::Version
26
+ version: '1.2'
27
+ - !ruby/object:Gem::Dependency
28
+ name: redis
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - '>='
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - '>='
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: rspec
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - '>='
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - '>='
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: bundler
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - '>='
60
+ - !ruby/object:Gem::Version
61
+ version: 1.0.0
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - '>='
67
+ - !ruby/object:Gem::Version
68
+ version: 1.0.0
69
+ description: Awesome rails logs
70
+ email:
71
+ - devin.christensen@moneydesktop.com
72
+ executables: []
73
+ extensions: []
74
+ extra_rdoc_files: []
75
+ files:
76
+ - .gitignore
77
+ - .rspec
78
+ - .travis.yml
79
+ - Gemfile
80
+ - LICENSE.md
81
+ - README.md
82
+ - Rakefile
83
+ - lib/logstasher.rb
84
+ - lib/logstasher/context_wrapper.rb
85
+ - lib/logstasher/device/redis.rb
86
+ - lib/logstasher/device/syslog.rb
87
+ - lib/logstasher/log_subscriber.rb
88
+ - lib/logstasher/railtie.rb
89
+ - lib/logstasher/silent_logger.rb
90
+ - lib/logstasher/version.rb
91
+ - lib/md-logstasher.rb
92
+ - log/logstash_development.log
93
+ - log/logstash_test.log
94
+ - log/test.log
95
+ - logstasher.gemspec
96
+ - spec/lib/logstasher/device/redis_spec.rb
97
+ - spec/lib/logstasher/device/syslog_spec.rb
98
+ - spec/lib/logstasher/log_subscriber_spec.rb
99
+ - spec/lib/logstasher/railtie_spec.rb
100
+ - spec/lib/logstasher_spec.rb
101
+ - spec/spec_helper.rb
102
+ homepage: https://github.com/moneydesktop/logstasher
103
+ licenses:
104
+ - MIT
105
+ metadata: {}
106
+ post_install_message:
107
+ rdoc_options: []
108
+ require_paths:
109
+ - lib
110
+ required_ruby_version: !ruby/object:Gem::Requirement
111
+ requirements:
112
+ - - '>='
113
+ - !ruby/object:Gem::Version
114
+ version: '0'
115
+ required_rubygems_version: !ruby/object:Gem::Requirement
116
+ requirements:
117
+ - - '>'
118
+ - !ruby/object:Gem::Version
119
+ version: 1.3.1
120
+ requirements: []
121
+ rubyforge_project:
122
+ rubygems_version: 2.0.14
123
+ signing_key:
124
+ specification_version: 4
125
+ summary: Awesome rails logs
126
+ test_files:
127
+ - spec/lib/logstasher/device/redis_spec.rb
128
+ - spec/lib/logstasher/device/syslog_spec.rb
129
+ - spec/lib/logstasher/log_subscriber_spec.rb
130
+ - spec/lib/logstasher/railtie_spec.rb
131
+ - spec/lib/logstasher_spec.rb
132
+ - spec/spec_helper.rb