logster 1.0.1 → 1.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 56302ee5e8c4578baa96303208225a8802e9413a
4
- data.tar.gz: 41aed485e25ed9efd4e57aac6b57ee83a921ef2f
3
+ metadata.gz: 10338f67f3b0de4bc895fab64a21addb21987b02
4
+ data.tar.gz: 3d4b900cba51c32ccbf7aad9b15843c3f2cb5a97
5
5
  SHA512:
6
- metadata.gz: 9a8ea2cf4353b00fb3f60697f094f5f2b9a25d08032fbb160d6639a7462c6dd77624215b20964b4cfcb961e0eb2173815d2595a2db62ee01f5bb56af4f2f9a9b
7
- data.tar.gz: 90535bd3068bc8124e01f5ceb0f771080acb1a353b8a6a047227e00ba267caf624e7f1355ab2865bbc0cb2414c3205e5bcf4fa647dd2583f8abc666cefa2aab9
6
+ metadata.gz: 6ff41f127c57daf3404518e1311d980887b25366f20b7330d73b8b0814e8d5ca9ceed7ad5df4d33cc270806d9eb7a1c3f81787ba805566102dfd06eda42a9186
7
+ data.tar.gz: 9f62de9005f735c84c121ac9a7dc09259afc42dc8c032a162b7d0770bb96b62f8045a0e679febbc9d8a74af8fb2488fa27e08cb077a3aea1bce185a252929075
@@ -0,0 +1,18 @@
1
+ language: ruby
2
+
3
+ matrix:
4
+ fast_finish: true
5
+
6
+ rvm:
7
+ - 2.0.0
8
+ - 2.1
9
+ - 2.2
10
+ - 2.3.0
11
+
12
+ services:
13
+ - redis-server
14
+
15
+ before_install:
16
+ - gem install bundler
17
+
18
+ sudo: false
data/README.md CHANGED
@@ -1,10 +1,10 @@
1
- ![logster logo](https://raw.githubusercontent.com/discourse/logster/master/assets/images/logo-logster-cropped-small.png)
1
+ ![logster logo](https://raw.githubusercontent.com/discourse/logster/master/website/images/logo-logster-cropped-small.png)
2
2
 
3
3
  Logster is an embedded Ruby "exception reporting service" admins can view on live websites, at `http://example.com/logs`
4
4
 
5
5
  ## Interface
6
6
 
7
- ![Screenshot](https://raw.githubusercontent.com/discourse/logster/master/assets/images/logster-screenshot.png)
7
+ ![Screenshot](https://raw.githubusercontent.com/discourse/logster/master/website/images/logster-screenshot.png)
8
8
 
9
9
  Play with a live demo at [logster.info/logs](http://logster.info/logs).
10
10
 
@@ -35,6 +35,23 @@ To run logster in other environments, in `config/application.rb`
35
35
  Logster.set_environments([:development, :staging, :production])
36
36
  ```
37
37
 
38
+ ### Tracking Error Rate
39
+ Logster allows you to register a callback when the rate of errors has exceeded
40
+ a given limit.
41
+
42
+ Tracking buckets available are one minute and an hour.
43
+
44
+ Example:
45
+ ```
46
+ Logster.register_rate_limit_per_minute(Logger::WARN, 60) do |rate|
47
+ puts "O no! The error rate is now #{rate} errors/min"
48
+ end
49
+
50
+ Logster.register_rate_limit_per_hour([Logger::WARN, Logger::ERROR, Logger::FATAL], 60) do |rate|
51
+ puts "O no! The error rate is now #{rate} errors/hour"
52
+ end
53
+ ```
54
+
38
55
  ### Note
39
56
  If you are seeing the error `No such middleware to insert before: ActionDispatch::DebugExceptions` after installing logster,
40
57
  then you are using a conflicting gem like `better_errors`.
@@ -59,7 +76,7 @@ Logster.store = Logster::RedisStore.new(redis_connection)
59
76
  ```
60
77
 
61
78
  ### Heroku Deployment
62
- In case you may be using the `rails_12factor` gem in a production deployment on Heroku, the standard `Rails.logger` will not cooperate properly with Logster. Extend Rails.logger in your `config/application.rb` or `config/initializers/logster.rb` with:
79
+ In case you may be using the `rails_12factor` gem in a production deployment on Heroku, the standard `Rails.logger` will not cooperate properly with Logster. Extend Rails.logger in your `config/application.rb` or `config/initializers/logster.rb` with:
63
80
  ```
64
81
  if Rails.env.production?
65
82
  Rails.logger.extend(ActiveSupport::Logger.broadcast(Logster.logger))
@@ -80,6 +97,9 @@ Logster UI is built using [Ember.js](http://emberjs.com/)
80
97
 
81
98
  # CHANGELOG
82
99
 
100
+ - 2015-02-11: Version 1.1.1
101
+ - Feature: Error rate can now be tracked in one minute and one hour buckets.
102
+
83
103
  - 2015-11-27: Version 1.0.1
84
104
  - New assets and logster logo
85
105
  - Added favicon
@@ -65,10 +65,22 @@ module Logster
65
65
  not_implemented
66
66
  end
67
67
 
68
+ # Registers a rate limit on the given severities of logs
69
+ def register_rate_limit(severities, limit, duration, &block)
70
+ not_implemented
71
+ end
72
+
73
+ # Checks all the existing rate limiters to check if any has been exceeded
74
+ def check_rate_limits(severity)
75
+ not_implemented
76
+ end
77
+
68
78
  def report(severity, progname, msg, opts = {})
69
79
  return if (!msg || (String === msg && msg.empty?)) && skip_empty
70
80
  return if level && severity < level
71
81
 
82
+ check_rate_limits(severity)
83
+
72
84
  message = Logster::Message.new(severity, progname, msg, opts[:timestamp])
73
85
 
74
86
  env = opts[:env] || {}
@@ -1,6 +1,8 @@
1
1
  module Logster
2
2
  class Configuration
3
- attr_accessor :current_context, :allow_grouping, :environments, :application_version, :web_title
3
+ attr_accessor :current_context, :allow_grouping, :environments,
4
+ :application_version, :web_title, :redis_prefix, :redis_raw_connection
5
+
4
6
  attr_writer :subdirectory
5
7
 
6
8
  def initialize
@@ -2,6 +2,88 @@ require 'json'
2
2
  require 'logster/base_store'
3
3
 
4
4
  module Logster
5
+ class RedisRateLimiter
6
+ BUCKETS = 6
7
+
8
+ attr_reader :key, :callback_key
9
+
10
+ def initialize(redis, severities, limit, duration, callback = nil)
11
+ @redis = use_raw_connection? ? Logster.config.redis_raw_connection : redis
12
+ @severities = severities
13
+ @limit = limit
14
+ @duration = duration
15
+ @callback = callback
16
+
17
+ # "_LOGSTER_RATE_LIMIT:012:20:30"
18
+ # Triggers callback when log levels of :debug, :info and :warn occurs 20 times within 30 secs
19
+ @key = "#{key_prefix}#{@severities.join("")}:#{@limit}:#{@duration}"
20
+ @callback_key = "#{@key}:callback_triggered"
21
+ @bucket_range = @duration / BUCKETS
22
+ end
23
+
24
+ def check(severity)
25
+ return unless @severities.include?(severity)
26
+ time = Time.now.to_i
27
+ num = bucket_number(time)
28
+ redis_key = "#{@key}:#{num}"
29
+
30
+ current_rate = @redis.eval <<-LUA
31
+ local bucket_number = #{num}
32
+ local bucket_count = redis.call("INCR", "#{redis_key}")
33
+
34
+ if bucket_count == 1 then
35
+ redis.call("EXPIRE", "#{redis_key}", "#{bucket_expiry(time)}")
36
+ redis.call("DEL", "#{callback_key}")
37
+ end
38
+
39
+ local function retrieve_rate ()
40
+ local sum = 0
41
+ local values = redis.call("MGET", #{mget_keys(num)})
42
+ for index, value in ipairs(values) do
43
+ if value ~= false then sum = sum + value end
44
+ end
45
+ return sum
46
+ end
47
+
48
+ return (retrieve_rate() + bucket_count)
49
+ LUA
50
+
51
+ if !@redis.get(@callback_key) && (current_rate >= @limit)
52
+ @callback.call(current_rate) if @callback
53
+ @redis.set(@callback_key, 1)
54
+ end
55
+
56
+ current_rate
57
+ end
58
+
59
+ private
60
+
61
+ def key_prefix
62
+ prefix = "__LOGSTER__RATE_LIMIT:".freeze
63
+ prefix = "#{Logster.config.redis_prefix}:#{prefix}" if use_raw_connection?
64
+ prefix
65
+ end
66
+
67
+ def mget_keys(bucket_num)
68
+ @mget_keys ||= (0..(BUCKETS - 1)).map { |i| "\"#{key}:#{i}\"" }
69
+ keys = @mget_keys.dup
70
+ keys.delete_at(bucket_num)
71
+ keys.join(", ")
72
+ end
73
+
74
+ def bucket_number(time)
75
+ (time % @duration) / @bucket_range
76
+ end
77
+
78
+ def bucket_expiry(time)
79
+ @duration - ((time % @duration) % @bucket_range)
80
+ end
81
+
82
+ def use_raw_connection?
83
+ Logster.config.redis_prefix && Logster.config.redis_raw_connection
84
+ end
85
+ end
86
+
5
87
  class RedisStore < BaseStore
6
88
 
7
89
  attr_accessor :redis, :max_backlog
@@ -179,8 +261,20 @@ module Logster
179
261
  @redis.hkeys(solved_key) || []
180
262
  end
181
263
 
264
+ def register_rate_limit_per_minute(severities, limit, &block)
265
+ register_rate_limit(severities, limit, 60, block)
266
+ end
267
+
268
+ def register_rate_limit_per_hour(severities, limit, &block)
269
+ register_rate_limit(severities, limit, 3600, block)
270
+ end
271
+
182
272
  protected
183
273
 
274
+ def rate_limits
275
+ @rate_limits ||= []
276
+ end
277
+
184
278
  def clear_solved(count = nil)
185
279
 
186
280
  ignores = Set.new(@redis.hkeys(solved_key) || [])
@@ -300,6 +394,10 @@ module Logster
300
394
 
301
395
  end
302
396
 
397
+ def check_rate_limits(severity)
398
+ rate_limits.each { |rate_limit| rate_limit.check(severity) }
399
+ end
400
+
303
401
  def solved_key
304
402
  @solved_key ||= "__LOGSTER__SOLVED_MAP"
305
403
  end
@@ -319,5 +417,14 @@ module Logster
319
417
  def grouping_key
320
418
  @grouping_key ||= "__LOGSTER__GMAP"
321
419
  end
420
+
421
+ private
422
+
423
+ def register_rate_limit(severities, limit, duration, callback)
424
+ severities = [severities] unless severities.is_a?(Array)
425
+ rate_limiter = RedisRateLimiter.new(@redis, severities, limit, duration, callback)
426
+ rate_limits << rate_limiter
427
+ rate_limiter
428
+ end
322
429
  end
323
430
  end
@@ -1,3 +1,3 @@
1
1
  module Logster
2
- VERSION = "1.0.1"
2
+ VERSION = "1.1.1"
3
3
  end
@@ -29,4 +29,5 @@ Gem::Specification.new do |spec|
29
29
  spec.add_development_dependency "redis"
30
30
  spec.add_development_dependency "guard"
31
31
  spec.add_development_dependency "guard-minitest"
32
+ spec.add_development_dependency "timecop"
32
33
  end
@@ -0,0 +1,178 @@
1
+ require_relative '../test_helper'
2
+ require 'logster/redis_store'
3
+ require 'rack'
4
+
5
+ class TestRedisRateLimiter < Minitest::Test
6
+ def setup
7
+ @redis = Redis.new
8
+ end
9
+
10
+ def teardown
11
+ @redis.flushall
12
+ end
13
+
14
+ def test_check
15
+ time = Time.new(2015, 1, 1, 1, 1)
16
+ Timecop.freeze(time)
17
+ called = 0
18
+
19
+ @rate_limiter = Logster::RedisRateLimiter.new(
20
+ @redis, [Logger::WARN], 8, 60, Proc.new { called += 1 }
21
+ )
22
+
23
+ assert_equal(1, @rate_limiter.check(Logger::WARN))
24
+ assert_redis_key(60, 0)
25
+ assert_equal(1, number_of_buckets)
26
+
27
+ Timecop.freeze(time + 10) do
28
+ assert_equal(2, @rate_limiter.check(Logger::WARN))
29
+ assert_redis_key(60, 1)
30
+ assert_equal(3, @rate_limiter.check(Logger::WARN))
31
+ assert_equal(2, number_of_buckets)
32
+ end
33
+
34
+ Timecop.freeze(time + 20) do
35
+ assert_equal(4, @rate_limiter.check(Logger::WARN))
36
+ assert_redis_key(60, 2)
37
+ assert_equal(3, number_of_buckets)
38
+ end
39
+
40
+ Timecop.freeze(time + 30) do
41
+ assert_equal(5, @rate_limiter.check(Logger::WARN))
42
+ assert_redis_key(60, 3)
43
+ assert_equal(4, number_of_buckets)
44
+ end
45
+
46
+ Timecop.freeze(time + 40) do
47
+ assert_equal(6, @rate_limiter.check(Logger::WARN))
48
+ assert_redis_key(60, 4)
49
+ assert_equal(5, number_of_buckets)
50
+ end
51
+
52
+ Timecop.freeze(time + 50) do
53
+ assert_equal(7, @rate_limiter.check(Logger::WARN))
54
+ assert_redis_key(60, 5)
55
+ assert_equal(6, number_of_buckets)
56
+ end
57
+
58
+ Timecop.freeze(time + 60) do
59
+ @redis.del("#{key}:0")
60
+ assert_equal(5, number_of_buckets)
61
+
62
+ assert_equal(7, @rate_limiter.check(Logger::WARN))
63
+ assert_redis_key(60, 0)
64
+ assert_equal(6, number_of_buckets)
65
+
66
+ assert_equal(8, @rate_limiter.check(Logger::WARN))
67
+ assert_equal(1, called)
68
+ assert_equal(6, number_of_buckets)
69
+ assert_equal("1", @redis.get(@rate_limiter.callback_key))
70
+ end
71
+
72
+ Timecop.freeze(time + 70) do
73
+ @redis.del("#{key}:1")
74
+ assert_equal(7, @rate_limiter.check(Logger::WARN))
75
+ assert_equal(nil, @redis.get(@rate_limiter.callback_key))
76
+ end
77
+ end
78
+
79
+ def test_check_with_multiple_severities
80
+ time = Time.new(2015, 1, 1, 1, 1)
81
+ Timecop.freeze(time)
82
+ called = 0
83
+
84
+ @rate_limiter = Logster::RedisRateLimiter.new(
85
+ @redis, [Logger::WARN, Logger::ERROR], 4, 60, Proc.new { called += 1 }
86
+ )
87
+
88
+ assert_equal(1, @rate_limiter.check(Logger::WARN))
89
+ assert_equal(2, @rate_limiter.check(Logger::ERROR))
90
+
91
+ Timecop.freeze(time + 50) do
92
+ assert_equal(3, @rate_limiter.check(Logger::WARN))
93
+ assert_equal(4, @rate_limiter.check(Logger::ERROR))
94
+ assert_equal(2, number_of_buckets)
95
+ end
96
+
97
+ assert_equal(5, @rate_limiter.check(Logger::ERROR))
98
+ assert_equal(1, called)
99
+ end
100
+
101
+ def test_bucket_number_per_minute
102
+ time = Time.new(2015, 1, 1, 1, 1)
103
+ Timecop.freeze(time)
104
+ @rate_limiter = Logster::RedisRateLimiter.new(@redis, [Logger::WARN], 1, 60)
105
+
106
+ assert_bucket_number(0, time)
107
+ assert_bucket_number(0, time + 9)
108
+ assert_bucket_number(1, time + 11)
109
+ assert_bucket_number(5, time + 59)
110
+ end
111
+
112
+ def test_bucket_number_per_hour
113
+ time = Time.new(2015, 1, 1, 1, 0)
114
+ Timecop.freeze(time)
115
+ @rate_limiter = Logster::RedisRateLimiter.new(@redis, [Logger::WARN], 1, 3600)
116
+
117
+ assert_bucket_number(0, time)
118
+ assert_bucket_number(1, time + 1199)
119
+ assert_bucket_number(2, time + 1200)
120
+ assert_bucket_number(5, time + 3599)
121
+ end
122
+
123
+ def test_bucket_expiry
124
+ time = Time.new(2015, 1, 1, 1, 1)
125
+ Timecop.freeze(time)
126
+ @rate_limiter = Logster::RedisRateLimiter.new(@redis, [Logger::WARN], 1, 60)
127
+
128
+ assert_bucket_expiry(60, time)
129
+ assert_bucket_expiry(55, time + 5)
130
+ assert_bucket_expiry(60, time + 10)
131
+ assert_bucket_expiry(58, time + 12)
132
+ assert_bucket_expiry(55, time + 15)
133
+ assert_bucket_expiry(51, time + 19)
134
+ assert_bucket_expiry(60, time + 20)
135
+ assert_bucket_expiry(55, time + 35)
136
+ end
137
+
138
+ def test_raw_connection
139
+ time = Time.new(2015, 1, 1, 1, 1)
140
+ Timecop.freeze(time)
141
+ Logster.config.redis_prefix = "lobster"
142
+ Logster.config.redis_raw_connection = @redis
143
+
144
+ @rate_limiter = Logster::RedisRateLimiter.new(nil, [Logger::WARN], 1, 60)
145
+
146
+ assert_equal(1, @rate_limiter.check(Logger::WARN))
147
+ assert_includes(key, "lobster")
148
+ assert_redis_key(60, 0)
149
+ end
150
+
151
+ private
152
+
153
+ def key
154
+ @rate_limiter.key
155
+ end
156
+
157
+ def number_of_buckets
158
+ @redis.keys("#{key}:[0-#{Logster::RedisRateLimiter::BUCKETS}]").size
159
+ end
160
+
161
+ def assert_bucket_number(expected, time)
162
+ Timecop.freeze(time) do
163
+ assert_equal(expected, @rate_limiter.send(:bucket_number, Time.now.to_i))
164
+ end
165
+ end
166
+
167
+ def assert_bucket_expiry(expected, time)
168
+ Timecop.freeze(time) do
169
+ assert_equal(expected, @rate_limiter.send(:bucket_expiry, Time.now.to_i))
170
+ end
171
+ end
172
+
173
+ def assert_redis_key(expected_ttl, expected_bucket_number)
174
+ redis_key = "#{key}:#{expected_bucket_number}"
175
+ assert(@redis.get(redis_key))
176
+ assert_equal(expected_ttl, @redis.ttl(redis_key))
177
+ end
178
+ end
@@ -326,4 +326,19 @@ class TestRedisStore < Minitest::Test
326
326
  assert_equal(orig, env)
327
327
  end
328
328
 
329
+ %w{minute hour}.each do |duration|
330
+ define_method "test_register_rate_limit_per_#{duration}" do
331
+ called = false
332
+
333
+ assert_instance_of(
334
+ Logster::RedisRateLimiter,
335
+ @store.public_send("register_rate_limit_per_#{duration}", Logger::WARN, 0) do
336
+ called = true
337
+ end
338
+ )
339
+
340
+ @store.report(Logger::WARN, "test", "test")
341
+ assert called
342
+ end
343
+ end
329
344
  end
@@ -5,6 +5,7 @@ require 'minitest/pride'
5
5
  require 'redis'
6
6
  require 'logster'
7
7
  require 'logster/base_store'
8
+ require 'timecop'
8
9
 
9
10
  class Logster::TestStore < Logster::BaseStore
10
11
  attr_accessor :reported
@@ -28,5 +29,9 @@ class Logster::TestStore < Logster::BaseStore
28
29
  @reported = []
29
30
  end
30
31
 
32
+ def check_rate_limits(severity)
33
+ # Do nothing
34
+ end
35
+
31
36
  # get, protect, unprotect: unimplemented
32
37
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logster
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.1
4
+ version: 1.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - UI for viewing logs in Rack
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2015-11-27 00:00:00.000000000 Z
11
+ date: 2016-02-22 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -94,6 +94,20 @@ dependencies:
94
94
  - - ">="
95
95
  - !ruby/object:Gem::Version
96
96
  version: '0'
97
+ - !ruby/object:Gem::Dependency
98
+ name: timecop
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
97
111
  description: UI for viewing logs in Rack
98
112
  email:
99
113
  - sam.saffron@gmail.com
@@ -102,6 +116,7 @@ extensions: []
102
116
  extra_rdoc_files: []
103
117
  files:
104
118
  - ".gitignore"
119
+ - ".travis.yml"
105
120
  - Gemfile
106
121
  - Guardfile
107
122
  - LICENSE.txt
@@ -160,6 +175,7 @@ files:
160
175
  - test/logster/test_ignore_pattern.rb
161
176
  - test/logster/test_logger.rb
162
177
  - test/logster/test_message.rb
178
+ - test/logster/test_redis_rate_limiter.rb
163
179
  - test/logster/test_redis_store.rb
164
180
  - test/test_helper.rb
165
181
  - vendor/assets/javascripts/logster.js.erb
@@ -183,7 +199,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
183
199
  version: '0'
184
200
  requirements: []
185
201
  rubyforge_project:
186
- rubygems_version: 2.4.5.1
202
+ rubygems_version: 2.5.1
187
203
  signing_key:
188
204
  specification_version: 4
189
205
  summary: UI for viewing logs in Rack
@@ -197,5 +213,6 @@ test_files:
197
213
  - test/logster/test_ignore_pattern.rb
198
214
  - test/logster/test_logger.rb
199
215
  - test/logster/test_message.rb
216
+ - test/logster/test_redis_rate_limiter.rb
200
217
  - test/logster/test_redis_store.rb
201
218
  - test/test_helper.rb