fluent-plugin-measure_time 0.1.1 → 0.1.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (7) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +6 -0
  3. data/README.md +84 -30
  4. data/fluent-plugin-measure_time.gemspec +3 -2
  5. data/spec/in_measure_time_spec.rb +2 -1
  6. data/spec/spec_helper.rb +1 -0
  7. metadata +34 -20
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: f2f3162b94cdaf1e70abdef53ae1cdca1b60f3c8
4
- data.tar.gz: 41debb162108e360783ea14bbb9296a79b3eaf91
3
+ metadata.gz: e19251ed1f95f372fe500788206264184ceaf759
4
+ data.tar.gz: d7c8c6cf65cd9799d1f35ddfd1598669d1b4254e
5
5
  SHA512:
6
- metadata.gz: 116da97ef9fb6cea05e203ca22f58cad55e3317e52a3ee0c22b5a410a3e381f62338a889084f7844e0b41add505297e3a00e3a2c3b3d1f8c54db476c4e88a860
7
- data.tar.gz: 29c5f059fe191a338bc05069aaa3e8d9b4503e59c72a5e6ee8cba1554df96bfd6ad99e7eef315e00852cc64aa26151252abf307466701b04bad4fb2b6f79d5e3
6
+ metadata.gz: f5fdb0a564b5129be907d4a17ed6f0764c88fdd51224f62664d729e5ea6ac2065b72f1b2207a34d51abf4e757b6150e9b795a97c5263e627ce6eb7b52f88c415
7
+ data.tar.gz: 2fce2f754b0023f3fbe81c9368392166eac41233d0a2e6e895dbfceb884b1f2aaa6d73e45a33a0b4e6f78ae4e9a88a277c289d5dfd5a39ccc73eb47f9f18aa6f
data/CHANGELOG.md CHANGED
@@ -1,3 +1,9 @@
1
+ ## 0.1.2 (2014/09/22)
2
+
3
+ Changes:
4
+
5
+ * Remove fluentd version constraint
6
+
1
7
  ## 0.1.1 (2014/04/12)
2
8
 
3
9
  Fixes:
data/README.md CHANGED
@@ -5,17 +5,31 @@
5
5
 
6
6
  Fluentd plugin to measure elapsed time to process messages
7
7
 
8
+
8
9
  ## Installation
9
10
 
10
11
  Use RubyGems:
11
12
 
12
13
  gem install fluent-plugin-measure_time
13
14
 
14
- ## Configuration
15
+ ## Parameters
16
+
17
+ * tag
18
+
19
+ The output tag name. Default is `measure_time`
20
+
21
+ * hook (required)
22
+
23
+ Specify the method to measure time.
24
+
25
+ * interval
26
+
27
+ The time interval to emit measurement results. Default is nil which do not compute statistics and emit the time in each measurement.
15
28
 
16
- This plugin is doing something tricky, which extends arbitrary plugins so that it can use `<measure_time></measure_time>` directive to measure elapsed times.
29
+ ## Configuration Example 1 - Profile an Output Plugin
17
30
 
18
- **Example 1:**
31
+ As an example, let's profile how long the [emit](https://github.com/sonots/fluent-plugin-grep/blob/master/lib/fluent/plugin/out_grep.rb#L56) method of [fluent-plugin-grep](https://github.com/sonots/fluent-plugin-grep) is taking.
32
+ Configure fluentd.conf as below:
19
33
 
20
34
  ```apache
21
35
  <source>
@@ -26,32 +40,84 @@ This plugin is doing something tricky, which extends arbitrary plugins so that i
26
40
  <source>
27
41
  type forward
28
42
  port 24224
29
- <measure_time>
30
- tag measure_time
31
- hook on_message
32
- </measure_time>
33
43
  </source>
34
44
 
45
+ # measure_time plugin output comes here
35
46
  <match measure_time>
36
47
  type stdout
37
48
  </match>
49
+
50
+ # Whatever you want to do
51
+ <match greped.**>
52
+ type stdout
53
+ </match>
54
+
55
+ <match **>
56
+ type grep
57
+ add_tag_prefix greped
58
+ <measure_time>
59
+ tag measure_time
60
+ hook emit
61
+ </measure_time>
62
+ </source>
38
63
  ```
39
64
 
40
- This example hooks the [on_message](https://github.com/fluent/fluentd/blob/e5a9a4ca03d18b45fdb89061d8251592a044e9fc/lib/fluent/plugin/in_forward.rb#L112) method of in_forward plugin, and measures how long it takes for processing. Output becomes as below:
65
+ The output of fluent-plugin-measure_time will be as below:
41
66
 
42
67
  ```
43
- measure_time: {"time":0.000849735,"class":"Fluent::ForwardInput","hook":"on_message","object_id":83935080}
68
+ measure_time: {"time":0.000849735,"class":"Fluent::GrepOutput","hook":"emit","object_id":83935080}
69
+ ```
70
+
71
+ where `time` denotes the measured elapsed time, and `class`, `hook`, and `object_id` denotes the hooked class, the hooked method, and the object id of the plugin instance.
72
+
73
+ ### interval option
74
+
75
+ fluent-plugin-measure_time outputs the elapsed time for each calling, but you can use the `interval` option when you want to get statistics in each interval.
76
+
77
+ ```
78
+ measure_time: {"max":1.011,"avg":0.002","num":10,"class":"Fluent::GrepOutput","hook":"emit","object_id":83935080}
79
+ ```
80
+
81
+ where `max` and `avg` are the maximum and average elapsed times, and `num` is the number of being called in each interval.
82
+
83
+ ## Configuration Example (2) - Profile the in_forward plugin
84
+
85
+ I introduce an interesting example here.
86
+
87
+ Following illustration draws the sequence of that `in_forward` plugin receives a data, processes, and passes the data to output plugins.
88
+
89
+ *Sequence Diagram*
90
+
91
+ ```
92
+ +–––––––––––––+ +––––––––––––––+ +––––––––––––––+
93
+ | in_forwrd | | Output | | Output |
94
+ +––––––+––––––+ +––––––+–––––––+ +––––––+–––––––+
95
+ #on_message | start = Time.now | |
96
+ +––––––––––––––––––> |
97
+ | #emit | |
98
+ | +––––––––––––––––––>
99
+ | | #emit |
100
+ | | |
101
+ | | |
102
+ | <– – – – – – – – – +
103
+ | elapsed = Time.now - start |
104
+ <– – – – – - – – – + |
105
+ | | |
106
+ + + +
44
107
  ```
45
108
 
46
- where `time` denotes the measured elapsed time, and `class`, `hook`, and `object_id` denotes the hooked class, the hooked method, and the object id of the plugin instance.
109
+ As the illustration, by hooking `on_message` method of `in_forward` plugin,
110
+ we can measure the blocking time taking to process the received data,
111
+ which also means that the time taking until `in_forward` will be ready for receiving a next data.
47
112
 
48
- **Example 2: interval**
113
+ This profiling is very useful to investigate when you have a suspicion that throughputs of Fluentd fell down extremely.
49
114
 
50
- With `interval` option, this plugin compute statistics of measured elapsed times in each interval
115
+ The configuration will be as follows:
51
116
 
52
117
  ```apache
53
118
  <source>
54
119
  type measure_time
120
+ # This makes available the `masure_time` directive for all plugins
55
121
  </source>
56
122
 
57
123
  <source>
@@ -59,7 +125,6 @@ With `interval` option, this plugin compute statistics of measured elapsed times
59
125
  port 24224
60
126
  <measure_time>
61
127
  tag measure_time
62
- interval 60
63
128
  hook on_message
64
129
  </measure_time>
65
130
  </source>
@@ -67,30 +132,19 @@ With `interval` option, this plugin compute statistics of measured elapsed times
67
132
  <match measure_time>
68
133
  type stdout
69
134
  </match>
135
+
136
+ # whatever you want
137
+ <match **>
138
+ type stdout
139
+ </match>
70
140
  ```
71
141
 
72
142
  Output becomes as below:
73
143
 
74
144
  ```
75
- measure_time: {"max":1.011,"avg":0.002","num":10,"class":"Fluent::ForwardInput","hook":"on_message","object_id":83935080}
145
+ measure_time: {"time":0.000849735,"class":"Fluent::ForwardInput","hook":"on_message","object_id":83935080}
76
146
  ```
77
147
 
78
- where `max` and `avg` are the maximum and average elapsed times, and `num` is the number of being called in each interval.
79
-
80
- ## Parameters
81
-
82
- * tag
83
-
84
- The output tag name. Default is `measure_time`
85
-
86
- * hook (required)
87
-
88
- Specify the method to measure time.
89
-
90
- * interval
91
-
92
- The time interval to emit measurement results. Default is nil which do not compute statistics and emit the time in each measurement.
93
-
94
148
  ## ChangeLog
95
149
 
96
150
  See [CHANGELOG.md](CHANGELOG.md) for details.
data/fluent-plugin-measure_time.gemspec CHANGED
@@ -3,7 +3,7 @@ $:.push File.expand_path('../lib', __FILE__)
3
3
 
4
4
  Gem::Specification.new do |gem|
5
5
  gem.name = "fluent-plugin-measure_time"
6
- gem.version = "0.1.1"
6
+ gem.version = "0.1.2"
7
7
  gem.authors = ["Naotoshi Seo"]
8
8
  gem.email = "sonots@gmail.com"
9
9
  gem.homepage = "https://github.com/sonots/fluent-plugin-measure_time"
@@ -17,9 +17,10 @@ Gem::Specification.new do |gem|
17
17
  gem.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
18
18
  gem.require_paths = ['lib']
19
19
 
20
- gem.add_dependency "fluentd", "~> 0.10.17"
20
+ gem.add_dependency "fluentd"
21
21
  gem.add_development_dependency "rake"
22
22
  gem.add_development_dependency "rspec"
23
+ gem.add_development_dependency "rspec-its"
23
24
  gem.add_development_dependency "pry"
24
25
  gem.add_development_dependency "pry-nav"
25
26
  end
data/spec/in_measure_time_spec.rb CHANGED
@@ -75,7 +75,8 @@ describe "extends Fluent::ForwardInput" do
75
75
  ]}
76
76
  it 'should flush' do
77
77
  d = driver.instance
78
- d.__send__(:on_message, ['tag1', 0, {'a'=>1}].to_msgpack)
78
+ data = ['tag1', 0, {'a'=>1}].to_msgpack
79
+ d.__send__(:on_message, data, data.bytesize, "hi, yay!")
79
80
  triple = d.measure_time.flush(0)
80
81
  triple[0].should == 'measure_time'
81
82
  triple[2].keys.should =~ [:num, :max, :avg, :class, :hook, :object_id]
data/spec/spec_helper.rb CHANGED
@@ -7,6 +7,7 @@ Bundler.require(:default, :test)
7
7
 
8
8
  require 'fluent/test'
9
9
  require 'rspec'
10
+ require 'rspec/its'
10
11
  require 'pry'
11
12
 
12
13
  $TESTING=true
metadata CHANGED
@@ -1,83 +1,97 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-measure_time
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Naotoshi Seo
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2014-04-12 00:00:00.000000000 Z
11
+ date: 2014-09-21 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: fluentd
15
15
  requirement: !ruby/object:Gem::Requirement
16
16
  requirements:
17
- - - ~>
17
+ - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 0.10.17
19
+ version: '0'
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
- - - ~>
24
+ - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 0.10.17
26
+ version: '0'
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: rake
29
29
  requirement: !ruby/object:Gem::Requirement
30
30
  requirements:
31
- - - '>='
31
+ - - ">="
32
32
  - !ruby/object:Gem::Version
33
33
  version: '0'
34
34
  type: :development
35
35
  prerelease: false
36
36
  version_requirements: !ruby/object:Gem::Requirement
37
37
  requirements:
38
- - - '>='
38
+ - - ">="
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0'
41
41
  - !ruby/object:Gem::Dependency
42
42
  name: rspec
43
43
  requirement: !ruby/object:Gem::Requirement
44
44
  requirements:
45
- - - '>='
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rspec-its
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
46
60
  - !ruby/object:Gem::Version
47
61
  version: '0'
48
62
  type: :development
49
63
  prerelease: false
50
64
  version_requirements: !ruby/object:Gem::Requirement
51
65
  requirements:
52
- - - '>='
66
+ - - ">="
53
67
  - !ruby/object:Gem::Version
54
68
  version: '0'
55
69
  - !ruby/object:Gem::Dependency
56
70
  name: pry
57
71
  requirement: !ruby/object:Gem::Requirement
58
72
  requirements:
59
- - - '>='
73
+ - - ">="
60
74
  - !ruby/object:Gem::Version
61
75
  version: '0'
62
76
  type: :development
63
77
  prerelease: false
64
78
  version_requirements: !ruby/object:Gem::Requirement
65
79
  requirements:
66
- - - '>='
80
+ - - ">="
67
81
  - !ruby/object:Gem::Version
68
82
  version: '0'
69
83
  - !ruby/object:Gem::Dependency
70
84
  name: pry-nav
71
85
  requirement: !ruby/object:Gem::Requirement
72
86
  requirements:
73
- - - '>='
87
+ - - ">="
74
88
  - !ruby/object:Gem::Version
75
89
  version: '0'
76
90
  type: :development
77
91
  prerelease: false
78
92
  version_requirements: !ruby/object:Gem::Requirement
79
93
  requirements:
80
- - - '>='
94
+ - - ">="
81
95
  - !ruby/object:Gem::Version
82
96
  version: '0'
83
97
  description: Fluentd plugin to measure elapsed time to process messages
@@ -86,9 +100,9 @@ executables: []
86
100
  extensions: []
87
101
  extra_rdoc_files: []
88
102
  files:
89
- - .gitignore
90
- - .rspec
91
- - .travis.yml
103
+ - ".gitignore"
104
+ - ".rspec"
105
+ - ".travis.yml"
92
106
  - CHANGELOG.md
93
107
  - Gemfile
94
108
  - LICENSE
@@ -116,17 +130,17 @@ require_paths:
116
130
  - lib
117
131
  required_ruby_version: !ruby/object:Gem::Requirement
118
132
  requirements:
119
- - - '>='
133
+ - - ">="
120
134
  - !ruby/object:Gem::Version
121
135
  version: '0'
122
136
  required_rubygems_version: !ruby/object:Gem::Requirement
123
137
  requirements:
124
- - - '>='
138
+ - - ">="
125
139
  - !ruby/object:Gem::Version
126
140
  version: '0'
127
141
  requirements: []
128
142
  rubyforge_project:
129
- rubygems_version: 2.0.3
143
+ rubygems_version: 2.2.2
130
144
  signing_key:
131
145
  specification_version: 4
132
146
  summary: Fluentd plugin to measure elapsed time to process messages