logstasher 0.5.0 → 0.5.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +8 -8
- data/README.md +51 -2
- data/lib/logstasher.rb +25 -2
- data/lib/logstasher/rails_ext/action_controller/metal/instrumentation.rb +5 -2
- data/lib/logstasher/version.rb +1 -1
- data/logstasher.gemspec +1 -0
- data/sample_logstash_configurations/quickstart.conf +23 -0
- data/spec/lib/logstasher_spec.rb +36 -0
- metadata +17 -2
checksums.yaml
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
---
|
2
2
|
!binary "U0hBMQ==":
|
3
3
|
metadata.gz: !binary |-
|
4
|
-
|
4
|
+
NDFjMDY1Nzc2ODM1NzA2NjlmNzZlMjIyZmVhNmVjNjZkMWZjZmUzZQ==
|
5
5
|
data.tar.gz: !binary |-
|
6
|
-
|
6
|
+
OWJmOWU3ODQzMjU2OWM5ZTk4ZmMxYjkxZjM0ZDJhZjkzNTg0MmE5Mw==
|
7
7
|
SHA512:
|
8
8
|
metadata.gz: !binary |-
|
9
|
-
|
10
|
-
|
11
|
-
|
9
|
+
ODQ0NWM2NmMxMTRmNGIzOTdlMzlhOTZjYzE0ZjVhZGZiYjZhZDc5NmE1NWJm
|
10
|
+
OGFiYWRhNWI5ZjJjOWEzZjhjMmIxMGQ2N2ZlYWY0MTE1NjI0YmQzN2MxMTM1
|
11
|
+
ZGJiMDY5ZDg5Nzg0YmFkZWUxNzIxMTRhYTY3ZjUyMTQzZDI2N2Q=
|
12
12
|
data.tar.gz: !binary |-
|
13
|
-
|
14
|
-
|
15
|
-
|
13
|
+
YjhiZDMzM2Q4NGE0MmZhODM2YWU3NjNmNjU5OWU1NDQyNTdlNTEwNjUzNmFk
|
14
|
+
YzUzZTc5YjIxMTFlNTYwNjQyYmI2YWMzYjI2NDY3OTZlOGEzOTQ2MDE4OTVh
|
15
|
+
MDQyZjk0Mzk0YmM5YzRkZGU4MGRiNjg5MDBmZmJiODQzMzllYzc=
|
data/README.md
CHANGED
@@ -3,7 +3,7 @@
|
|
3
3
|
|
4
4
|
This gem is heavily inspired from [lograge](https://github.com/roidrage/lograge), but it's focused on one thing and one thing only. That's making your logs awesome like this:
|
5
5
|
|
6
|
-
[![Awesome Logs](
|
6
|
+
[![Awesome Logs](https://f.cloud.github.com/assets/830679/2407078/dcde03e8-aa82-11e3-85ac-8c5b3a86676e.png)](https://f.cloud.github.com/assets/830679/2407078/dcde03e8-aa82-11e3-85ac-8c5b3a86676e.png)
|
7
7
|
|
8
8
|
How it's done ?
|
9
9
|
|
@@ -11,7 +11,9 @@ By, using these awesome tools:
|
|
11
11
|
* [Logstash](http://logstash.net) - Store and index your logs
|
12
12
|
* [Kibana](http://kibana.org/) - for awesome visualization. This is optional though, and you can use any other visualizer
|
13
13
|
|
14
|
-
|
14
|
+
Update: Logstash now includes Kibana build in, so no need to separately install. Logstasher has been test with **logstash version 1.3.3**
|
15
|
+
|
16
|
+
See [quickstart](#quick-setup-for-logstash) for quickly setting up logstash
|
15
17
|
|
16
18
|
## About logstasher
|
17
19
|
|
@@ -92,6 +94,49 @@ Since some fields are very specific to your application for e.g. *user_name*, so
|
|
92
94
|
end
|
93
95
|
end
|
94
96
|
|
97
|
+
## Listening to `ActiveSupport::Notifications` events
|
98
|
+
|
99
|
+
It is possible to listen to any `ActiveSupport::Notifications` events and store arbitrary data to be included in the final JSON log entry:
|
100
|
+
|
101
|
+
# In config/initializers/logstasher.rb
|
102
|
+
|
103
|
+
# Watch calls the block with the same arguments than any ActiveSupport::Notification, plus a store
|
104
|
+
LogStasher.watch('some.activesupport.notification') do |name, start, finish, id, payload, store|
|
105
|
+
# Do something
|
106
|
+
store[:count] = 42
|
107
|
+
end
|
108
|
+
|
109
|
+
Would change the log entry to:
|
110
|
+
|
111
|
+
```
|
112
|
+
{"@source":"unknown","@tags":["request"],"@fields":{"method":"GET","path":"/","format":"html","controller":"file_servers","action":"index","status":200,"duration":28.34,"view":25.96,"db":0.88,"ip":"127.0.0.1","route":"file_servers#index", "parameters":"","ndapi_time":null,"uuid":"e81ecd178ed3b591099f4d489760dfb6","user":"shadab_ahmed@abc.com", "site":"internal","some.activesupport.notification":{"count":42}},"@timestamp":"2013-04-30T13:00:46.354500+00:00"}
|
113
|
+
```
|
114
|
+
|
115
|
+
The store exposed to the blocked passed to `watch` is thread-safe, and reset after each request.
|
116
|
+
By default, the store is only shared between occurences of the same event.
|
117
|
+
You can easily share the same store between different types of notifications, by assigning them to the same event group:
|
118
|
+
|
119
|
+
# In config/initializers/logstasher.rb
|
120
|
+
|
121
|
+
LogStasher.watch('foo.notification', event_group: 'notification') do |*args, store|
|
122
|
+
# Shared store with 'bar.notification'
|
123
|
+
end
|
124
|
+
|
125
|
+
LogStasher.watch('bar.notification', event_group: 'notification') do |*args, store|
|
126
|
+
# Shared store with 'foo.notification'
|
127
|
+
end
|
128
|
+
|
129
|
+
## Quick Setup for Logstash
|
130
|
+
|
131
|
+
* Download logstash from [logstash.net](http://www.logstash.net/)
|
132
|
+
* Use this sample config file: [quickstart.conf](https://github.com/shadabahmed/logstasher/raw/master/sample_logstash_configurations/quickstart.conf)
|
133
|
+
* Start logstash with the following command:
|
134
|
+
```
|
135
|
+
java -jar logstash-1.3.3-flatjar.jar agent -f quickstart.conf -- web
|
136
|
+
```
|
137
|
+
* Visit http://localhost:9292/ to see the Kibana interface and your parsed logs
|
138
|
+
* For advanced options see the latest logstash documentation at [logstash.net](http://www.logstash.net/) or visit my blog at (shadabahmed.com)[http://shadabahmed.com/blog/2013/04/30/logstasher-for-awesome-rails-logging] (slightly outdated but will sure give you ideas for distributed setup etc.)
|
139
|
+
|
95
140
|
## Versions
|
96
141
|
All versions require Rails 3.0.x and higher and Ruby 1.9.2+. Tested on Rails 4 and Ruby 2.0
|
97
142
|
|
@@ -102,3 +147,7 @@ All versions require Rails 3.0.x and higher and Ruby 1.9.2+. Tested on Rails 4 a
|
|
102
147
|
## Copyright
|
103
148
|
|
104
149
|
Copyright (c) 2013 Shadab Ahmed, released under the MIT license
|
150
|
+
|
151
|
+
|
152
|
+
[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/shadabahmed/logstasher/trend.png)](https://bitdeli.com/free "Bitdeli Badge")
|
153
|
+
|
data/lib/logstasher.rb
CHANGED
@@ -1,11 +1,14 @@
|
|
1
1
|
require 'logstasher/version'
|
2
2
|
require 'logstasher/log_subscriber'
|
3
|
+
require 'request_store'
|
3
4
|
require 'active_support/core_ext/module/attribute_accessors'
|
4
5
|
require 'active_support/core_ext/string/inflections'
|
5
6
|
require 'active_support/ordered_options'
|
6
7
|
|
7
8
|
module LogStasher
|
8
9
|
extend self
|
10
|
+
STORE_KEY = :logstasher_data
|
11
|
+
|
9
12
|
attr_accessor :logger, :enabled, :log_controller_parameters, :source
|
10
13
|
# Setting the default to 'unknown' to define the default behaviour
|
11
14
|
@source = 'unknown'
|
@@ -43,8 +46,12 @@ module LogStasher
|
|
43
46
|
end
|
44
47
|
|
45
48
|
def add_custom_fields(&block)
|
46
|
-
|
47
|
-
|
49
|
+
wrapped_block = lambda do |fields|
|
50
|
+
LogStasher.custom_fields.concat(LogStasher.store.keys)
|
51
|
+
block.call(fields)
|
52
|
+
end
|
53
|
+
ActionController::Metal.send(:define_method, :logtasher_add_custom_fields_to_payload, &wrapped_block)
|
54
|
+
ActionController::Base.send(:define_method, :logtasher_add_custom_fields_to_payload, &wrapped_block)
|
48
55
|
end
|
49
56
|
|
50
57
|
def setup(app)
|
@@ -88,6 +95,22 @@ module LogStasher
|
|
88
95
|
end
|
89
96
|
end
|
90
97
|
|
98
|
+
def store
|
99
|
+
if RequestStore.store[STORE_KEY].nil?
|
100
|
+
# Get each store it's own private Hash instance.
|
101
|
+
RequestStore.store[STORE_KEY] = Hash.new { |hash, key| hash[key] = {} }
|
102
|
+
end
|
103
|
+
RequestStore.store[STORE_KEY]
|
104
|
+
end
|
105
|
+
|
106
|
+
def watch(event, opts = {}, &block)
|
107
|
+
event_group = opts[:event_group] || event
|
108
|
+
ActiveSupport::Notifications.subscribe(event) do |*args|
|
109
|
+
# Calling the processing block with the Notification args and the store
|
110
|
+
block.call(*args, store[event_group])
|
111
|
+
end
|
112
|
+
end
|
113
|
+
|
91
114
|
%w( fatal error warn info debug unknown ).each do |severity|
|
92
115
|
eval <<-EOM, nil, __FILE__, __LINE__ + 1
|
93
116
|
def #{severity}(msg)
|
@@ -16,7 +16,7 @@ module ActionController
|
|
16
16
|
|
17
17
|
ActiveSupport::Notifications.instrument("process_action.action_controller", raw_payload) do |payload|
|
18
18
|
result = super
|
19
|
-
|
19
|
+
|
20
20
|
if self.respond_to?(:logtasher_add_custom_fields_to_payload)
|
21
21
|
before_keys = raw_payload.keys.clone
|
22
22
|
logtasher_add_custom_fields_to_payload(raw_payload)
|
@@ -24,9 +24,12 @@ module ActionController
|
|
24
24
|
# Store all extra keys added to payload hash in payload itself. This is a thread safe way
|
25
25
|
LogStasher.custom_fields += after_keys - before_keys
|
26
26
|
end
|
27
|
-
|
27
|
+
|
28
28
|
payload[:status] = response.status
|
29
29
|
append_info_to_payload(payload)
|
30
|
+
LogStasher.store.each do |key, value|
|
31
|
+
payload[key] = value
|
32
|
+
end
|
30
33
|
result
|
31
34
|
end
|
32
35
|
end
|
data/lib/logstasher/version.rb
CHANGED
data/logstasher.gemspec
CHANGED
@@ -18,6 +18,7 @@ Gem::Specification.new do |s|
|
|
18
18
|
s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
|
19
19
|
s.require_paths = ["lib"]
|
20
20
|
s.add_runtime_dependency "logstash-event", ["~> 1.1.0"]
|
21
|
+
s.add_runtime_dependency "request_store"
|
21
22
|
|
22
23
|
# specify any dependencies here; for example:
|
23
24
|
s.add_development_dependency "rspec"
|
@@ -0,0 +1,23 @@
|
|
1
|
+
input {
|
2
|
+
file {
|
3
|
+
type => "rails logs"
|
4
|
+
path => "/Users/shadab/test_project/logstash_development.log"
|
5
|
+
codec => json {
|
6
|
+
charset => "UTF-8"
|
7
|
+
}
|
8
|
+
}
|
9
|
+
}
|
10
|
+
|
11
|
+
output {
|
12
|
+
# Print each event to stdout.
|
13
|
+
stdout {
|
14
|
+
codec => rubydebug
|
15
|
+
}
|
16
|
+
|
17
|
+
elasticsearch {
|
18
|
+
# Setting 'embedded' will run a real elasticsearch server inside logstash.
|
19
|
+
# This option below saves you from having to run a separate process just
|
20
|
+
# for ElasticSearch, so you can get started quicker!
|
21
|
+
embedded => true
|
22
|
+
}
|
23
|
+
}
|
data/spec/lib/logstasher_spec.rb
CHANGED
@@ -176,4 +176,40 @@ describe LogStasher do
|
|
176
176
|
end
|
177
177
|
end
|
178
178
|
end
|
179
|
+
|
180
|
+
describe '.store' do
|
181
|
+
it "returns a new Hash for each key" do
|
182
|
+
expect(LogStasher.store['a'].object_id).to_not be(LogStasher.store['b'].object_id)
|
183
|
+
end
|
184
|
+
|
185
|
+
it "returns the same store if called several time with the same key" do
|
186
|
+
expect(LogStasher.store['a'].object_id).to be(LogStasher.store['a'].object_id)
|
187
|
+
end
|
188
|
+
|
189
|
+
end
|
190
|
+
|
191
|
+
describe ".watch" do
|
192
|
+
before(:each) { LogStasher.custom_fields = [] }
|
193
|
+
|
194
|
+
it "subscribes to the required event" do
|
195
|
+
ActiveSupport::Notifications.should_receive(:subscribe).with('event_name')
|
196
|
+
LogStasher.watch('event_name')
|
197
|
+
end
|
198
|
+
|
199
|
+
it 'executes the block when receiving an event' do
|
200
|
+
probe = lambda {}
|
201
|
+
LogStasher.watch('custom.event.foo', &probe)
|
202
|
+
expect(probe).to receive(:call)
|
203
|
+
ActiveSupport::Notifications.instrument('custom.event.foo', {})
|
204
|
+
end
|
205
|
+
|
206
|
+
describe "store" do
|
207
|
+
it 'stores the events in a store with the event\'s name' do
|
208
|
+
probe = lambda { |*args, store| store[:foo] = :bar }
|
209
|
+
LogStasher.watch('custom.event.bar', &probe)
|
210
|
+
ActiveSupport::Notifications.instrument('custom.event.bar', {})
|
211
|
+
LogStasher.store['custom.event.bar'].should == {:foo => :bar}
|
212
|
+
end
|
213
|
+
end
|
214
|
+
end
|
179
215
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: logstasher
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.5.
|
4
|
+
version: 0.5.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Shadab Ahmed
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-
|
11
|
+
date: 2014-05-05 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: logstash-event
|
@@ -24,6 +24,20 @@ dependencies:
|
|
24
24
|
- - ~>
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: 1.1.0
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: request_store
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - ! '>='
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - ! '>='
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
27
41
|
- !ruby/object:Gem::Dependency
|
28
42
|
name: rspec
|
29
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -88,6 +102,7 @@ files:
|
|
88
102
|
- lib/logstasher/railtie.rb
|
89
103
|
- lib/logstasher/version.rb
|
90
104
|
- logstasher.gemspec
|
105
|
+
- sample_logstash_configurations/quickstart.conf
|
91
106
|
- spec/lib/logstasher/device/redis_spec.rb
|
92
107
|
- spec/lib/logstasher/log_subscriber_spec.rb
|
93
108
|
- spec/lib/logstasher_spec.rb
|