fluent-plugin-stats-notifier 0.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.coveralls.yml +1 -0
- data/.gitignore +14 -0
- data/.rdebugrc +4 -0
- data/.rspec +2 -0
- data/.travis.yml +6 -0
- data/CHANGELOG.md +3 -0
- data/Gemfile +3 -0
- data/README.md +87 -0
- data/Rakefile +15 -0
- data/fluent-plugin-stats-notifier.gemspec +26 -0
- data/lib/fluent/plugin/out_stats_notifier.rb +209 -0
- data/spec/out_stats_notifier_spec.rb +294 -0
- data/spec/spec_helper.rb +16 -0
- metadata +129 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 6a10e45fed39a4a597b00a14d2cf9d9889351042
|
4
|
+
data.tar.gz: 87eb183f4e82343ccfcc0b30a23c668b962a9d4e
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: b8f08b98979b725ba89c26577814572d0fed6ea7d89765435b38da82e853974746e9faf7843d198c0307767f83ad92eada13fc87b42b8e20aa093abd50bda331
|
7
|
+
data.tar.gz: c83b295784a032e281a1ac39cf58cb9bbb8c34670d625f4dd80c14bc7c37ae8f29f4ab0f4296783e9e63c8fde4fa9e1fa83013b1f8c60f6454037338308de260
|
data/.coveralls.yml
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
repo_token: i4DJCtdksuIwhBck1tukIjzKoMCxWIIvQ
|
data/.gitignore
ADDED
data/.rdebugrc
ADDED
data/.rspec
ADDED
data/.travis.yml
ADDED
data/CHANGELOG.md
ADDED
data/Gemfile
ADDED
data/README.md
ADDED
@@ -0,0 +1,87 @@
|
|
1
|
+
# fluent-plugin-stats-notifier [![Build Status](https://secure.travis-ci.org/sonots/fluent-plugin-stats-notifier.png?branch=master)](http://travis-ci.org/sonots/fluent-plugin-stats-notifier)
|
2
|
+
|
3
|
+
Fluentd plugin to calculate statistics and then thresholding
|
4
|
+
|
5
|
+
## Configuration
|
6
|
+
|
7
|
+
<store>
|
8
|
+
type stats_notifier
|
9
|
+
tag notifier
|
10
|
+
interval 5
|
11
|
+
target_key 4xx_count
|
12
|
+
greater_equal 4
|
13
|
+
compare_with max
|
14
|
+
store_file /path/to/store_file.dat
|
15
|
+
</store>
|
16
|
+
|
17
|
+
Assuming following inputs are coming:
|
18
|
+
|
19
|
+
foo.bar1: {"4xx_count":1,"foobar":2"}
|
20
|
+
foo.bar2: {"4xx_count":6,"foobar":2"}
|
21
|
+
|
22
|
+
then this plugin emits an message because the max of `4xx_count` is greater than or equal to the specified value `4`. Output will be as following:
|
23
|
+
|
24
|
+
notifier: {"4xx_count":6.0}
|
25
|
+
|
26
|
+
## Parameters
|
27
|
+
|
28
|
+
- target\_key (required)
|
29
|
+
|
30
|
+
The target key in the event record.
|
31
|
+
|
32
|
+
- interval
|
33
|
+
|
34
|
+
The interval time of calculation and bounding. Default is 60.
|
35
|
+
|
36
|
+
- less\_than
|
37
|
+
|
38
|
+
A `less than` threshold value, that is, emit if `target_key` value < specified value.
|
39
|
+
|
40
|
+
- less\_equal
|
41
|
+
|
42
|
+
A `less than or eqaul` threshold value, that is, emit if `target_key` value <= specified value.
|
43
|
+
|
44
|
+
- greater\_than
|
45
|
+
|
46
|
+
A `greater than` threshold value, that is, emit if `target_key` value > specified value.
|
47
|
+
|
48
|
+
- greater\_equal
|
49
|
+
|
50
|
+
A `greater than or eqaul` threshold value, that is, emit if `target_key` value >= specified value.
|
51
|
+
|
52
|
+
- compare\_with
|
53
|
+
|
54
|
+
`max`, `avg`, `min`, `sum` can be specified. Default is `max`.
|
55
|
+
|
56
|
+
- tag
|
57
|
+
|
58
|
+
The output tag name.
|
59
|
+
|
60
|
+
- add_tag_prefix
|
61
|
+
|
62
|
+
(not available yet) Add tag prefix for output message.
|
63
|
+
|
64
|
+
- aggragate
|
65
|
+
|
66
|
+
(not available yet) Do calculation by each `tag` or `all`. The default value is `tag`.
|
67
|
+
|
68
|
+
- store_file
|
69
|
+
|
70
|
+
Store internal data into a file of the given path on shutdown, and load on starting.
|
71
|
+
|
72
|
+
## ChangeLog
|
73
|
+
|
74
|
+
See [CHANGELOG.md](CHANGELOG.md) for details.
|
75
|
+
|
76
|
+
## Contributing
|
77
|
+
|
78
|
+
1. Fork it
|
79
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
80
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
81
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
82
|
+
5. Create new [Pull Request](../../pull/new/master)
|
83
|
+
|
84
|
+
## Copyright
|
85
|
+
|
86
|
+
Copyright (c) 2013 Naotoshi Seo. See [LICENSE](LICENSE) for details.
|
87
|
+
|
data/Rakefile
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
require "bundler/gem_tasks"
|
3
|
+
|
4
|
+
require 'rspec/core'
|
5
|
+
require 'rspec/core/rake_task'
|
6
|
+
RSpec::Core::RakeTask.new(:spec) do |spec|
|
7
|
+
spec.pattern = FileList['spec/**/*_spec.rb']
|
8
|
+
end
|
9
|
+
task :default => :spec
|
10
|
+
|
11
|
+
desc 'Open an irb session preloaded with the gem library'
|
12
|
+
task :console do
|
13
|
+
sh 'irb -rubygems -I lib'
|
14
|
+
end
|
15
|
+
task :c => :console
|
@@ -0,0 +1,26 @@
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
2
|
+
$:.push File.expand_path("../lib", __FILE__)
|
3
|
+
|
4
|
+
Gem::Specification.new do |s|
|
5
|
+
s.name = "fluent-plugin-stats-notifier"
|
6
|
+
s.version = "0.0.1"
|
7
|
+
s.authors = ["Naotoshi Seo"]
|
8
|
+
s.email = ["sonots@gmail.com"]
|
9
|
+
s.homepage = "https://github.com/sonots/fluent-plugin-stats-notifier"
|
10
|
+
s.summary = "Fluentd plugin to calculate statistics and then thresholding"
|
11
|
+
s.description = s.summary
|
12
|
+
s.licenses = ["MIT"]
|
13
|
+
|
14
|
+
s.rubyforge_project = "fluent-plugin-stats-notifier"
|
15
|
+
|
16
|
+
s.files = `git ls-files`.split("\n")
|
17
|
+
s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
|
18
|
+
s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
|
19
|
+
s.require_paths = ["lib"]
|
20
|
+
|
21
|
+
s.add_runtime_dependency "fluentd"
|
22
|
+
s.add_development_dependency "rake"
|
23
|
+
s.add_development_dependency "rspec"
|
24
|
+
s.add_development_dependency "pry"
|
25
|
+
s.add_development_dependency "pry-nav"
|
26
|
+
end
|
@@ -0,0 +1,209 @@
|
|
1
|
+
# encoding: UTF-8
|
2
|
+
class Fluent::StatsNotifierOutput < Fluent::Output
|
3
|
+
Fluent::Plugin.register_output('stats_notifier', self)
|
4
|
+
|
5
|
+
def initialize
|
6
|
+
super
|
7
|
+
require 'pathname'
|
8
|
+
end
|
9
|
+
|
10
|
+
config_param :target_key, :string
|
11
|
+
config_param :interval, :time, :default => 5
|
12
|
+
config_param :less_than, :float, :default => nil
|
13
|
+
config_param :less_equal, :float, :default => nil
|
14
|
+
config_param :greater_than, :float, :default => nil
|
15
|
+
config_param :greater_equal, :float, :default => nil
|
16
|
+
config_param :compare_with, :string, :default => "max"
|
17
|
+
config_param :tag, :string
|
18
|
+
config_param :store_file, :string, :default => nil
|
19
|
+
|
20
|
+
attr_accessor :counts
|
21
|
+
attr_accessor :matches
|
22
|
+
attr_accessor :saved_duration
|
23
|
+
attr_accessor :saved_at
|
24
|
+
attr_accessor :last_checked
|
25
|
+
|
26
|
+
def configure(conf)
|
27
|
+
super
|
28
|
+
|
29
|
+
@interval = @interval.to_i
|
30
|
+
|
31
|
+
if @less_than and @less_equal
|
32
|
+
raise Fluent::ConfigError, "out_stats_notiifer: Only either of `less_than` or `less_equal` can be specified."
|
33
|
+
end
|
34
|
+
if @greater_than and @greater_equal
|
35
|
+
raise Fluent::ConfigError, "out_stats_notiifer: Only either of `greater_than` or `greater_equal` can be specified."
|
36
|
+
end
|
37
|
+
|
38
|
+
case @compare_with
|
39
|
+
when "sum"
|
40
|
+
@compare_with = :sum
|
41
|
+
when "max"
|
42
|
+
@compare_with = :max
|
43
|
+
when "min"
|
44
|
+
@compare_with = :min
|
45
|
+
when "avg"
|
46
|
+
@compare_with = :avg
|
47
|
+
else
|
48
|
+
raise Fluent::ConfigError, "out_stats_notiifer: `compare_with` must be one of `sum`, `max`, `min`, `avg`"
|
49
|
+
end
|
50
|
+
|
51
|
+
@counts = {}
|
52
|
+
@matches = {}
|
53
|
+
@mutex = Mutex.new
|
54
|
+
end
|
55
|
+
|
56
|
+
def start
|
57
|
+
super
|
58
|
+
load_status(@store_file, @interval) if @store_file
|
59
|
+
@watcher = Thread.new(&method(:watcher))
|
60
|
+
end
|
61
|
+
|
62
|
+
def shutdown
|
63
|
+
super
|
64
|
+
@watcher.terminate
|
65
|
+
@watcher.join
|
66
|
+
save_status(@store_file) if @store_file
|
67
|
+
end
|
68
|
+
|
69
|
+
# Called when new line comes. This method actually does not emit
|
70
|
+
def emit(tag, es, chain)
|
71
|
+
key = @target_key
|
72
|
+
|
73
|
+
# stats
|
74
|
+
count = 0; matches = {}
|
75
|
+
es.each do |time,record|
|
76
|
+
if record[key]
|
77
|
+
# @todo: make an option for statsuation in the same tag. now only sum is supported
|
78
|
+
matches[key] = (matches[key] ? matches[key] + record[key] : record[key])
|
79
|
+
end
|
80
|
+
count += 1
|
81
|
+
end
|
82
|
+
|
83
|
+
# thread safe merge
|
84
|
+
@counts[tag] ||= 0
|
85
|
+
@matches[tag] ||= {}
|
86
|
+
@mutex.synchronize do
|
87
|
+
if matches[key]
|
88
|
+
# @todo: make an option for statsuation in the same tag. now only sum is supported
|
89
|
+
@matches[tag][key] = (@matches[tag][key] ? @matches[tag][key] + matches[key] : matches[key])
|
90
|
+
end
|
91
|
+
@counts[tag] += count
|
92
|
+
end
|
93
|
+
|
94
|
+
chain.next
|
95
|
+
rescue => e
|
96
|
+
$log.warn "#{e.class} #{e.message} #{e.backtrace.first}"
|
97
|
+
end
|
98
|
+
|
99
|
+
# thread callback
|
100
|
+
def watcher
|
101
|
+
# instance variable, and public accessable, for test
|
102
|
+
@last_checked = Fluent::Engine.now
|
103
|
+
# skip the passed time when loading @counts form file
|
104
|
+
@last_checked -= @passed_time if @passed_time
|
105
|
+
while true
|
106
|
+
sleep 0.5
|
107
|
+
begin
|
108
|
+
if Fluent::Engine.now - @last_checked >= @interval
|
109
|
+
now = Fluent::Engine.now
|
110
|
+
flush_emit(now - @last_checked)
|
111
|
+
@last_checked = now
|
112
|
+
end
|
113
|
+
rescue => e
|
114
|
+
$log.warn "#{e.class} #{e.message} #{e.backtrace.first}"
|
115
|
+
end
|
116
|
+
end
|
117
|
+
end
|
118
|
+
|
119
|
+
# This method is the real one to emit
|
120
|
+
def flush_emit(step)
|
121
|
+
time = Fluent::Engine.now
|
122
|
+
flushed_counts, flushed_matches, @counts, @matches = @counts, @matches, {}, {}
|
123
|
+
|
124
|
+
output = generate_output(flushed_counts, flushed_matches)
|
125
|
+
Fluent::Engine.emit(@tag, time, output) if output
|
126
|
+
end
|
127
|
+
|
128
|
+
def generate_output(counts, matches)
|
129
|
+
values = matches.values.map {|match| match[@target_key] }.compact
|
130
|
+
|
131
|
+
case @compare_with
|
132
|
+
when :sum
|
133
|
+
target_value = values.inject(:+)
|
134
|
+
when :max
|
135
|
+
target_value = values.max
|
136
|
+
when :min
|
137
|
+
target_value = values.min
|
138
|
+
when :avg
|
139
|
+
target_value = values.inject(:+) / values.count unless values.empty?
|
140
|
+
end
|
141
|
+
|
142
|
+
return nil if target_value.nil?
|
143
|
+
return nil if target_value == 0 # ignore 0 because standby nodes receive 0 message usually
|
144
|
+
return nil if @less_than and @less_than <= target_value
|
145
|
+
return nil if @less_equal and @less_equal < target_value
|
146
|
+
return nil if @greater_than and target_value <= @greater_than
|
147
|
+
return nil if @greater_equal and target_value < @greater_equal
|
148
|
+
|
149
|
+
output = {}
|
150
|
+
output[@target_key] = target_value
|
151
|
+
output
|
152
|
+
end
|
153
|
+
|
154
|
+
# Store internal status into a file
|
155
|
+
#
|
156
|
+
# @param [String] file_path
|
157
|
+
def save_status(file_path)
|
158
|
+
return unless file_path
|
159
|
+
|
160
|
+
begin
|
161
|
+
Pathname.new(file_path).open('wb') do |f|
|
162
|
+
@saved_at = Fluent::Engine.now
|
163
|
+
@saved_duration = @saved_at - @last_checked
|
164
|
+
Marshal.dump({
|
165
|
+
:counts => @counts,
|
166
|
+
:matches => @matches,
|
167
|
+
:saved_at => @saved_at,
|
168
|
+
:saved_duration => @saved_duration,
|
169
|
+
:target_key => @target_key,
|
170
|
+
}, f)
|
171
|
+
end
|
172
|
+
rescue => e
|
173
|
+
$log.warn "out_stats_notifier: Can't write store_file #{e.class} #{e.message}"
|
174
|
+
end
|
175
|
+
end
|
176
|
+
|
177
|
+
# Load internal status from a file
|
178
|
+
#
|
179
|
+
# @param [String] file_path
|
180
|
+
# @param [Interger] interval
|
181
|
+
def load_status(file_path, interval)
|
182
|
+
return unless (f = Pathname.new(file_path)).exist?
|
183
|
+
|
184
|
+
begin
|
185
|
+
f.open('rb') do |f|
|
186
|
+
stored = Marshal.load(f)
|
187
|
+
if stored[:target_key] == @target_key
|
188
|
+
|
189
|
+
if Fluent::Engine.now <= stored[:saved_at] + interval
|
190
|
+
@counts = stored[:counts]
|
191
|
+
@matches = stored[:matches]
|
192
|
+
@saved_at = stored[:saved_at]
|
193
|
+
@saved_duration = stored[:saved_duration]
|
194
|
+
|
195
|
+
# skip the saved duration to continue counting
|
196
|
+
@last_checked = Fluent::Engine.now - @saved_duration
|
197
|
+
else
|
198
|
+
$log.warn "out_stats_notifier: stored data is outdated. ignore stored data"
|
199
|
+
end
|
200
|
+
else
|
201
|
+
$log.warn "out_stats_notiifer: configuration param was changed. ignore stored data"
|
202
|
+
end
|
203
|
+
end
|
204
|
+
rescue => e
|
205
|
+
$log.warn "out_stats_notifier: Can't load store_file #{e.class} #{e.message}"
|
206
|
+
end
|
207
|
+
end
|
208
|
+
|
209
|
+
end
|
@@ -0,0 +1,294 @@
|
|
1
|
+
# encoding: UTF-8
|
2
|
+
require_relative 'spec_helper'
|
3
|
+
|
4
|
+
class Fluent::Test::OutputTestDriver
|
5
|
+
def emit_with_tag(record, time=Time.now, tag = nil)
|
6
|
+
@tag = tag if tag
|
7
|
+
emit(record, time)
|
8
|
+
end
|
9
|
+
end
|
10
|
+
|
11
|
+
describe Fluent::StatsNotifierOutput do
|
12
|
+
before { Fluent::Test.setup }
|
13
|
+
CONFIG = %[
|
14
|
+
target_key 5xx_count
|
15
|
+
tag foo
|
16
|
+
]
|
17
|
+
let(:tag) { 'foo.bar' }
|
18
|
+
let(:driver) { Fluent::Test::OutputTestDriver.new(Fluent::StatsNotifierOutput, tag).configure(config) }
|
19
|
+
|
20
|
+
describe 'test configure' do
|
21
|
+
describe 'bad configuration' do
|
22
|
+
context "nothing" do
|
23
|
+
let(:config) { '' }
|
24
|
+
it { expect { driver }.to raise_error(Fluent::ConfigError) }
|
25
|
+
end
|
26
|
+
|
27
|
+
context "less_than and less_equal" do
|
28
|
+
let(:config) { CONFIG + %[less_than 2 \n less_equal 3] }
|
29
|
+
it { expect { driver }.to raise_error(Fluent::ConfigError) }
|
30
|
+
end
|
31
|
+
|
32
|
+
context "greater_than and greater_equal" do
|
33
|
+
let(:config) { CONFIG + %[greater_than 2 \n greater_equal 3] }
|
34
|
+
it { expect { driver }.to raise_error(Fluent::ConfigError) }
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
describe 'good configuration' do
|
39
|
+
context 'required' do
|
40
|
+
let(:config) { CONFIG }
|
41
|
+
it { expect { driver }.to_not raise_error }
|
42
|
+
end
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
describe 'test emit' do
|
47
|
+
let(:time) { Time.now.to_i }
|
48
|
+
let(:messages) do
|
49
|
+
[
|
50
|
+
{"4xx_count"=>1,"5xx_count"=>2,"reqtime_max"=>6,"reqtime_min"=>1,"reqtime_avg"=>3},
|
51
|
+
{"4xx_count"=>2,"5xx_count"=>2,"reqtime_max"=>5,"reqtime_min"=>2,"reqtime_avg"=>2},
|
52
|
+
{"4xx_count"=>3,"5xx_count"=>2,"reqtime_max"=>1,"reqtime_min"=>3,"reqtime_avg"=>4},
|
53
|
+
]
|
54
|
+
end
|
55
|
+
let(:emit) do
|
56
|
+
driver.run { messages.each {|message| driver.emit(message, time) } }
|
57
|
+
driver.instance.flush_emit(0)
|
58
|
+
end
|
59
|
+
let(:config) { CONFIG } # 5xx_count, sum
|
60
|
+
let(:expected) do
|
61
|
+
{
|
62
|
+
"5xx_count"=>6,
|
63
|
+
}
|
64
|
+
end
|
65
|
+
|
66
|
+
context "threshold" do
|
67
|
+
context 'no threshold' do # should emit
|
68
|
+
before do
|
69
|
+
Fluent::Engine.stub(:now).and_return(time)
|
70
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
71
|
+
end
|
72
|
+
it { emit }
|
73
|
+
end
|
74
|
+
|
75
|
+
context 'greather than' do
|
76
|
+
let(:config) { CONFIG + %[greater_than 5] }
|
77
|
+
before do
|
78
|
+
Fluent::Engine.stub(:now).and_return(time)
|
79
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
80
|
+
end
|
81
|
+
it { emit }
|
82
|
+
end
|
83
|
+
|
84
|
+
context 'not greather than' do
|
85
|
+
let(:config) { CONFIG + %[greater_than 6] }
|
86
|
+
before do
|
87
|
+
Fluent::Engine.stub(:now).and_return(time)
|
88
|
+
Fluent::Engine.should_not_receive(:emit)
|
89
|
+
end
|
90
|
+
it { emit }
|
91
|
+
end
|
92
|
+
|
93
|
+
context 'greather than or equal to' do
|
94
|
+
let(:config) { CONFIG + %[greater_equal 6] }
|
95
|
+
before do
|
96
|
+
Fluent::Engine.stub(:now).and_return(time)
|
97
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
98
|
+
end
|
99
|
+
it { emit }
|
100
|
+
end
|
101
|
+
|
102
|
+
context 'not greather than or equal to' do
|
103
|
+
let(:config) { CONFIG + %[greater_equal 7] }
|
104
|
+
before do
|
105
|
+
Fluent::Engine.stub(:now).and_return(time)
|
106
|
+
Fluent::Engine.should_not_receive(:emit)
|
107
|
+
end
|
108
|
+
it { emit }
|
109
|
+
end
|
110
|
+
|
111
|
+
context 'less than or equal to' do
|
112
|
+
let(:config) { CONFIG + %[less_equal 6] }
|
113
|
+
before do
|
114
|
+
Fluent::Engine.stub(:now).and_return(time)
|
115
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
116
|
+
end
|
117
|
+
it { emit }
|
118
|
+
end
|
119
|
+
|
120
|
+
context 'not less than or equal to' do
|
121
|
+
let(:config) { CONFIG + %[less_equal 5] }
|
122
|
+
before do
|
123
|
+
Fluent::Engine.stub(:now).and_return(time)
|
124
|
+
Fluent::Engine.should_not_receive(:emit)
|
125
|
+
end
|
126
|
+
it { emit }
|
127
|
+
end
|
128
|
+
|
129
|
+
context 'between' do
|
130
|
+
let(:config) { CONFIG + %[greater_equal 1 \n less_equal 9] }
|
131
|
+
before do
|
132
|
+
Fluent::Engine.stub(:now).and_return(time)
|
133
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
134
|
+
end
|
135
|
+
it { emit }
|
136
|
+
end
|
137
|
+
|
138
|
+
context 'not between' do
|
139
|
+
let(:config) { CONFIG + %[greater_equal 1 \n less_equal 4] }
|
140
|
+
before do
|
141
|
+
Fluent::Engine.stub(:now).and_return(time)
|
142
|
+
Fluent::Engine.should_not_receive(:emit)
|
143
|
+
end
|
144
|
+
it { emit }
|
145
|
+
end
|
146
|
+
end
|
147
|
+
|
148
|
+
context 'compare_with' do
|
149
|
+
let(:emit) do
|
150
|
+
driver.run do
|
151
|
+
driver.emit_with_tag({"5xx_count"=>2}, time, 'foo.bar1')
|
152
|
+
driver.emit_with_tag({"5xx_count"=>6}, time, 'foo.bar2')
|
153
|
+
end
|
154
|
+
driver.instance.flush_emit(0)
|
155
|
+
end
|
156
|
+
|
157
|
+
context 'avg' do
|
158
|
+
let(:config) { CONFIG + %[less_equal 4 \n compare_with avg] }
|
159
|
+
let(:expected) do
|
160
|
+
{
|
161
|
+
"5xx_count" => 4.0
|
162
|
+
}
|
163
|
+
end
|
164
|
+
before do
|
165
|
+
Fluent::Engine.stub(:now).and_return(time)
|
166
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
167
|
+
end
|
168
|
+
it { emit }
|
169
|
+
end
|
170
|
+
|
171
|
+
context 'sum' do
|
172
|
+
let(:config) { CONFIG + %[less_equal 8 \n compare_with sum] }
|
173
|
+
let(:expected) do
|
174
|
+
{
|
175
|
+
"5xx_count" => 8.0
|
176
|
+
}
|
177
|
+
end
|
178
|
+
before do
|
179
|
+
Fluent::Engine.stub(:now).and_return(time)
|
180
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
181
|
+
end
|
182
|
+
it { emit }
|
183
|
+
end
|
184
|
+
|
185
|
+
context 'min' do
|
186
|
+
let(:config) { CONFIG + %[less_equal 2 \n compare_with min] }
|
187
|
+
let(:expected) do
|
188
|
+
{
|
189
|
+
"5xx_count" => 2.0
|
190
|
+
}
|
191
|
+
end
|
192
|
+
before do
|
193
|
+
Fluent::Engine.stub(:now).and_return(time)
|
194
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
195
|
+
end
|
196
|
+
it { emit }
|
197
|
+
end
|
198
|
+
|
199
|
+
context 'max' do
|
200
|
+
let(:config) { CONFIG + %[less_equal 6 \n compare_with max] }
|
201
|
+
let(:expected) do
|
202
|
+
{
|
203
|
+
"5xx_count" => 6.0
|
204
|
+
}
|
205
|
+
end
|
206
|
+
before do
|
207
|
+
Fluent::Engine.stub(:now).and_return(time)
|
208
|
+
Fluent::Engine.should_receive(:emit).with("foo", time, expected)
|
209
|
+
end
|
210
|
+
it { emit }
|
211
|
+
end
|
212
|
+
end
|
213
|
+
|
214
|
+
context 'abnormal case (no data)' do
|
215
|
+
let(:emit) do
|
216
|
+
driver.run do
|
217
|
+
end
|
218
|
+
driver.instance.flush_emit(0)
|
219
|
+
end
|
220
|
+
|
221
|
+
context 'avg' do
|
222
|
+
let(:config) { CONFIG + %[less_equal 4 \n compare_with avg] }
|
223
|
+
before do
|
224
|
+
Fluent::Engine.stub(:now).and_return(time)
|
225
|
+
Fluent::Engine.should_not_receive(:emit)
|
226
|
+
end
|
227
|
+
it { emit }
|
228
|
+
end
|
229
|
+
|
230
|
+
context 'sum' do
|
231
|
+
let(:config) { CONFIG + %[less_equal 8 \n compare_with sum] }
|
232
|
+
before do
|
233
|
+
Fluent::Engine.stub(:now).and_return(time)
|
234
|
+
Fluent::Engine.should_not_receive(:emit)
|
235
|
+
end
|
236
|
+
it { emit }
|
237
|
+
end
|
238
|
+
|
239
|
+
context 'min' do
|
240
|
+
let(:config) { CONFIG + %[less_equal 2 \n compare_with min] }
|
241
|
+
before do
|
242
|
+
Fluent::Engine.stub(:now).and_return(time)
|
243
|
+
Fluent::Engine.should_not_receive(:emit)
|
244
|
+
end
|
245
|
+
it { emit }
|
246
|
+
end
|
247
|
+
|
248
|
+
context 'max' do
|
249
|
+
let(:config) { CONFIG + %[less_equal 6 \n compare_with max] }
|
250
|
+
before do
|
251
|
+
Fluent::Engine.stub(:now).and_return(time)
|
252
|
+
Fluent::Engine.should_not_receive(:emit)
|
253
|
+
end
|
254
|
+
it { emit }
|
255
|
+
end
|
256
|
+
end
|
257
|
+
|
258
|
+
describe "store_file" do
|
259
|
+
let(:store_file) do
|
260
|
+
dirname = "tmp"
|
261
|
+
Dir.mkdir dirname unless Dir.exist? dirname
|
262
|
+
filename = "#{dirname}/test.dat"
|
263
|
+
File.unlink filename if File.exist? filename
|
264
|
+
filename
|
265
|
+
end
|
266
|
+
let(:config) { CONFIG + %[greater_equal 0 \n store_file #{store_file}] }
|
267
|
+
|
268
|
+
it 'stored_data and loaded_data should equal' do
|
269
|
+
driver.run { messages.each {|message| driver.emit(message, time) } }
|
270
|
+
driver.instance.shutdown
|
271
|
+
stored_counts = driver.instance.counts
|
272
|
+
stored_matches = driver.instance.matches
|
273
|
+
stored_saved_at = driver.instance.saved_at
|
274
|
+
stored_saved_duration = driver.instance.saved_duration
|
275
|
+
driver.instance.counts = {}
|
276
|
+
driver.instance.matches = {}
|
277
|
+
driver.instance.saved_at = nil
|
278
|
+
driver.instance.saved_duration = nil
|
279
|
+
|
280
|
+
driver.instance.start
|
281
|
+
loaded_counts = driver.instance.counts
|
282
|
+
loaded_matches = driver.instance.matches
|
283
|
+
loaded_saved_at = driver.instance.saved_at
|
284
|
+
loaded_saved_duration = driver.instance.saved_duration
|
285
|
+
|
286
|
+
loaded_counts.should == stored_counts
|
287
|
+
loaded_matches.should == stored_matches
|
288
|
+
loaded_saved_at.should == stored_saved_at
|
289
|
+
loaded_saved_duration.should == stored_saved_duration
|
290
|
+
end
|
291
|
+
end
|
292
|
+
end
|
293
|
+
end
|
294
|
+
|
data/spec/spec_helper.rb
ADDED
@@ -0,0 +1,16 @@
|
|
1
|
+
# encoding: UTF-8
|
2
|
+
require 'rubygems'
|
3
|
+
require 'bundler'
|
4
|
+
Bundler.setup(:default, :test)
|
5
|
+
Bundler.require(:default, :test)
|
6
|
+
|
7
|
+
#require 'coveralls'
|
8
|
+
#Coveralls.wear!
|
9
|
+
|
10
|
+
require 'fluent/test'
|
11
|
+
require 'rspec'
|
12
|
+
require 'pry'
|
13
|
+
|
14
|
+
$TESTING=true
|
15
|
+
$:.unshift File.join(File.dirname(__FILE__), '..', 'lib')
|
16
|
+
require 'fluent/plugin/out_stats_notifier'
|
metadata
ADDED
@@ -0,0 +1,129 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: fluent-plugin-stats-notifier
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.0.1
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Naotoshi Seo
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2013-12-27 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: fluentd
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - '>='
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '0'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - '>='
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: rake
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - '>='
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
34
|
+
type: :development
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - '>='
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: rspec
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - '>='
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - '>='
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: pry
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - '>='
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - '>='
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '0'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: pry-nav
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - '>='
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '0'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - '>='
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '0'
|
83
|
+
description: Fluentd plugin to calculate statistics and then thresholding
|
84
|
+
email:
|
85
|
+
- sonots@gmail.com
|
86
|
+
executables: []
|
87
|
+
extensions: []
|
88
|
+
extra_rdoc_files: []
|
89
|
+
files:
|
90
|
+
- .coveralls.yml
|
91
|
+
- .gitignore
|
92
|
+
- .rdebugrc
|
93
|
+
- .rspec
|
94
|
+
- .travis.yml
|
95
|
+
- CHANGELOG.md
|
96
|
+
- Gemfile
|
97
|
+
- README.md
|
98
|
+
- Rakefile
|
99
|
+
- fluent-plugin-stats-notifier.gemspec
|
100
|
+
- lib/fluent/plugin/out_stats_notifier.rb
|
101
|
+
- spec/out_stats_notifier_spec.rb
|
102
|
+
- spec/spec_helper.rb
|
103
|
+
homepage: https://github.com/sonots/fluent-plugin-stats-notifier
|
104
|
+
licenses:
|
105
|
+
- MIT
|
106
|
+
metadata: {}
|
107
|
+
post_install_message:
|
108
|
+
rdoc_options: []
|
109
|
+
require_paths:
|
110
|
+
- lib
|
111
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
112
|
+
requirements:
|
113
|
+
- - '>='
|
114
|
+
- !ruby/object:Gem::Version
|
115
|
+
version: '0'
|
116
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
117
|
+
requirements:
|
118
|
+
- - '>='
|
119
|
+
- !ruby/object:Gem::Version
|
120
|
+
version: '0'
|
121
|
+
requirements: []
|
122
|
+
rubyforge_project: fluent-plugin-stats-notifier
|
123
|
+
rubygems_version: 2.0.3
|
124
|
+
signing_key:
|
125
|
+
specification_version: 4
|
126
|
+
summary: Fluentd plugin to calculate statistics and then thresholding
|
127
|
+
test_files:
|
128
|
+
- spec/out_stats_notifier_spec.rb
|
129
|
+
- spec/spec_helper.rb
|