fluent-plugin-measure_time 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +6 -0
  3. data/README.md +84 -30
  4. data/fluent-plugin-measure_time.gemspec +3 -2
  5. data/spec/in_measure_time_spec.rb +2 -1
  6. data/spec/spec_helper.rb +1 -0
  7. metadata +34 -20
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: f2f3162b94cdaf1e70abdef53ae1cdca1b60f3c8
4
- data.tar.gz: 41debb162108e360783ea14bbb9296a79b3eaf91
3
+ metadata.gz: e19251ed1f95f372fe500788206264184ceaf759
4
+ data.tar.gz: d7c8c6cf65cd9799d1f35ddfd1598669d1b4254e
5
5
  SHA512:
6
- metadata.gz: 116da97ef9fb6cea05e203ca22f58cad55e3317e52a3ee0c22b5a410a3e381f62338a889084f7844e0b41add505297e3a00e3a2c3b3d1f8c54db476c4e88a860
7
- data.tar.gz: 29c5f059fe191a338bc05069aaa3e8d9b4503e59c72a5e6ee8cba1554df96bfd6ad99e7eef315e00852cc64aa26151252abf307466701b04bad4fb2b6f79d5e3
6
+ metadata.gz: f5fdb0a564b5129be907d4a17ed6f0764c88fdd51224f62664d729e5ea6ac2065b72f1b2207a34d51abf4e757b6150e9b795a97c5263e627ce6eb7b52f88c415
7
+ data.tar.gz: 2fce2f754b0023f3fbe81c9368392166eac41233d0a2e6e895dbfceb884b1f2aaa6d73e45a33a0b4e6f78ae4e9a88a277c289d5dfd5a39ccc73eb47f9f18aa6f
data/CHANGELOG.md CHANGED
@@ -1,3 +1,9 @@
1
+ ## 0.1.2 (2014/09/22)
2
+
3
+ Changes:
4
+
5
+ * Remove fluentd version constraint
6
+
1
7
  ## 0.1.1 (2014/04/12)
2
8
 
3
9
  Fixes:
data/README.md CHANGED
@@ -5,17 +5,31 @@
5
5
 
6
6
  Fluentd plugin to measure elapsed time to process messages
7
7
 
8
+
8
9
  ## Installation
9
10
 
10
11
  Use RubyGems:
11
12
 
12
13
  gem install fluent-plugin-measure_time
13
14
 
14
- ## Configuration
15
+ ## Parameters
16
+
17
+ * tag
18
+
19
+ The output tag name. Default is `measure_time`
20
+
21
+ * hook (required)
22
+
23
+ Specify the method to measure time.
24
+
25
+ * interval
26
+
27
+ The time interval to emit measurement results. Default is nil which do not compute statistics and emit the time in each measurement.
15
28
 
16
- This plugin is doing something tricky, which extends arbitrary plugins so that it can use `<measure_time></measure_time>` directive to measure elapsed times.
29
+ ## Configuration Example 1 - Profile an Output Plugin
17
30
 
18
- **Example 1:**
31
+ As an example, let's profile how long the [emit](https://github.com/sonots/fluent-plugin-grep/blob/master/lib/fluent/plugin/out_grep.rb#L56) method of [fluent-plugin-grep](https://github.com/sonots/fluent-plugin-grep) is taking.
32
+ Configure fluentd.conf as below:
19
33
 
20
34
  ```apache
21
35
  <source>
@@ -26,32 +40,84 @@ This plugin is doing something tricky, which extends arbitrary plugins so that i
26
40
  <source>
27
41
  type forward
28
42
  port 24224
29
- <measure_time>
30
- tag measure_time
31
- hook on_message
32
- </measure_time>
33
43
  </source>
34
44
 
45
+ # measure_time plugin output comes here
35
46
  <match measure_time>
36
47
  type stdout
37
48
  </match>
49
+
50
+ # Whatever you want to do
51
+ <match greped.**>
52
+ type stdout
53
+ </match>
54
+
55
+ <match **>
56
+ type grep
57
+ add_tag_prefix greped
58
+ <measure_time>
59
+ tag measure_time
60
+ hook emit
61
+ </measure_time>
62
+ </source>
38
63
  ```
39
64
 
40
- This example hooks the [on_message](https://github.com/fluent/fluentd/blob/e5a9a4ca03d18b45fdb89061d8251592a044e9fc/lib/fluent/plugin/in_forward.rb#L112) method of in_forward plugin, and measures how long it takes for processing. Output becomes as below:
65
+ The output of fluent-plugin-measure_time will be as below:
41
66
 
42
67
  ```
43
- measure_time: {"time":0.000849735,"class":"Fluent::ForwardInput","hook":"on_message","object_id":83935080}
68
+ measure_time: {"time":0.000849735,"class":"Fluent::GrepOutput","hook":"emit","object_id":83935080}
69
+ ```
70
+
71
+ where `time` denotes the measured elapsed time, and `class`, `hook`, and `object_id` denotes the hooked class, the hooked method, and the object id of the plugin instance.
72
+
73
+ ### interval option
74
+
75
+ fluent-plugin-measure_time outputs the elapsed time for each calling, but you can use the `interval` option when you want to get statistics in each interval.
76
+
77
+ ```
78
+ measure_time: {"max":1.011,"avg":0.002","num":10,"class":"Fluent::GrepOutput","hook":"emit","object_id":83935080}
79
+ ```
80
+
81
+ where `max` and `avg` are the maximum and average elapsed times, and `num` is the number of being called in each interval.
82
+
83
+ ## Configuration Example (2) - Profile the in_forward plugin
84
+
85
+ I introduce an interesting example here.
86
+
87
+ Following illustration draws the sequence of that `in_forward` plugin receives a data, processes, and passes the data to output plugins.
88
+
89
+ *Sequence Diagram*
90
+
91
+ ```
92
+ +–––––––––––––+ +––––––––––––––+ +––––––––––––––+
93
+ | in_forwrd | | Output | | Output |
94
+ +––––––+––––––+ +––––––+–––––––+ +––––––+–––––––+
95
+ #on_message | start = Time.now | |
96
+ +––––––––––––––––––> |
97
+ | #emit | |
98
+ | +––––––––––––––––––>
99
+ | | #emit |
100
+ | | |
101
+ | | |
102
+ | <– – – – – – – – – +
103
+ | elapsed = Time.now - start |
104
+ <– – – – – - – – – + |
105
+ | | |
106
+ + + +
44
107
  ```
45
108
 
46
- where `time` denotes the measured elapsed time, and `class`, `hook`, and `object_id` denotes the hooked class, the hooked method, and the object id of the plugin instance.
109
+ As the illustration, by hooking `on_message` method of `in_forward` plugin,
110
+ we can measure the blocking time taking to process the received data,
111
+ which also means that the time taking until `in_forward` will be ready for receiving a next data.
47
112
 
48
- **Example 2: interval**
113
+ This profiling is very useful to investigate when you have a suspicion that throughputs of Fluentd fell down extremely.
49
114
 
50
- With `interval` option, this plugin compute statistics of measured elapsed times in each interval
115
+ The configuration will be as follows:
51
116
 
52
117
  ```apache
53
118
  <source>
54
119
  type measure_time
120
+ # This makes available the `masure_time` directive for all plugins
55
121
  </source>
56
122
 
57
123
  <source>
@@ -59,7 +125,6 @@ With `interval` option, this plugin compute statistics of measured elapsed times
59
125
  port 24224
60
126
  <measure_time>
61
127
  tag measure_time
62
- interval 60
63
128
  hook on_message
64
129
  </measure_time>
65
130
  </source>
@@ -67,30 +132,19 @@ With `interval` option, this plugin compute statistics of measured elapsed times
67
132
  <match measure_time>
68
133
  type stdout
69
134
  </match>
135
+
136
+ # whatever you want
137
+ <match **>
138
+ type stdout
139
+ </match>
70
140
  ```
71
141
 
72
142
  Output becomes as below:
73
143
 
74
144
  ```
75
- measure_time: {"max":1.011,"avg":0.002","num":10,"class":"Fluent::ForwardInput","hook":"on_message","object_id":83935080}
145
+ measure_time: {"time":0.000849735,"class":"Fluent::ForwardInput","hook":"on_message","object_id":83935080}
76
146
  ```
77
147
 
78
- where `max` and `avg` are the maximum and average elapsed times, and `num` is the number of being called in each interval.
79
-
80
- ## Parameters
81
-
82
- * tag
83
-
84
- The output tag name. Default is `measure_time`
85
-
86
- * hook (required)
87
-
88
- Specify the method to measure time.
89
-
90
- * interval
91
-
92
- The time interval to emit measurement results. Default is nil which do not compute statistics and emit the time in each measurement.
93
-
94
148
  ## ChangeLog
95
149
 
96
150
  See [CHANGELOG.md](CHANGELOG.md) for details.
data/fluent-plugin-measure_time.gemspec CHANGED
@@ -3,7 +3,7 @@ $:.push File.expand_path('../lib', __FILE__)
3
3
 
4
4
  Gem::Specification.new do |gem|
5
5
  gem.name = "fluent-plugin-measure_time"
6
- gem.version = "0.1.1"
6
+ gem.version = "0.1.2"
7
7
  gem.authors = ["Naotoshi Seo"]
8
8
  gem.email = "sonots@gmail.com"
9
9
  gem.homepage = "https://github.com/sonots/fluent-plugin-measure_time"
@@ -17,9 +17,10 @@ Gem::Specification.new do |gem|
17
17
  gem.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
18
18
  gem.require_paths = ['lib']
19
19
 
20
- gem.add_dependency "fluentd", "~> 0.10.17"
20
+ gem.add_dependency "fluentd"
21
21
  gem.add_development_dependency "rake"
22
22
  gem.add_development_dependency "rspec"
23
+ gem.add_development_dependency "rspec-its"
23
24
  gem.add_development_dependency "pry"
24
25
  gem.add_development_dependency "pry-nav"
25
26
  end
data/spec/in_measure_time_spec.rb CHANGED
@@ -75,7 +75,8 @@ describe "extends Fluent::ForwardInput" do
75
75
  ]}
76
76
  it 'should flush' do
77
77
  d = driver.instance
78
- d.__send__(:on_message, ['tag1', 0, {'a'=>1}].to_msgpack)
78
+ data = ['tag1', 0, {'a'=>1}].to_msgpack
79
+ d.__send__(:on_message, data, data.bytesize, "hi, yay!")
79
80
  triple = d.measure_time.flush(0)
80
81
  triple[0].should == 'measure_time'
81
82
  triple[2].keys.should =~ [:num, :max, :avg, :class, :hook, :object_id]
data/spec/spec_helper.rb CHANGED
@@ -7,6 +7,7 @@ Bundler.require(:default, :test)
7
7
 
8
8
  require 'fluent/test'
9
9
  require 'rspec'
10
+ require 'rspec/its'
10
11
  require 'pry'
11
12
 
12
13
  $TESTING=true
metadata CHANGED
@@ -1,83 +1,97 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-measure_time
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Naotoshi Seo
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2014-04-12 00:00:00.000000000 Z
11
+ date: 2014-09-21 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: fluentd
15
15
  requirement: !ruby/object:Gem::Requirement
16
16
  requirements:
17
- - - ~>
17
+ - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 0.10.17
19
+ version: '0'
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
- - - ~>
24
+ - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 0.10.17
26
+ version: '0'
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: rake
29
29
  requirement: !ruby/object:Gem::Requirement
30
30
  requirements:
31
- - - '>='
31
+ - - ">="
32
32
  - !ruby/object:Gem::Version
33
33
  version: '0'
34
34
  type: :development
35
35
  prerelease: false
36
36
  version_requirements: !ruby/object:Gem::Requirement
37
37
  requirements:
38
- - - '>='
38
+ - - ">="
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0'
41
41
  - !ruby/object:Gem::Dependency
42
42
  name: rspec
43
43
  requirement: !ruby/object:Gem::Requirement
44
44
  requirements:
45
- - - '>='
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rspec-its
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
46
60
  - !ruby/object:Gem::Version
47
61
  version: '0'
48
62
  type: :development
49
63
  prerelease: false
50
64
  version_requirements: !ruby/object:Gem::Requirement
51
65
  requirements:
52
- - - '>='
66
+ - - ">="
53
67
  - !ruby/object:Gem::Version
54
68
  version: '0'
55
69
  - !ruby/object:Gem::Dependency
56
70
  name: pry
57
71
  requirement: !ruby/object:Gem::Requirement
58
72
  requirements:
59
- - - '>='
73
+ - - ">="
60
74
  - !ruby/object:Gem::Version
61
75
  version: '0'
62
76
  type: :development
63
77
  prerelease: false
64
78
  version_requirements: !ruby/object:Gem::Requirement
65
79
  requirements:
66
- - - '>='
80
+ - - ">="
67
81
  - !ruby/object:Gem::Version
68
82
  version: '0'
69
83
  - !ruby/object:Gem::Dependency
70
84
  name: pry-nav
71
85
  requirement: !ruby/object:Gem::Requirement
72
86
  requirements:
73
- - - '>='
87
+ - - ">="
74
88
  - !ruby/object:Gem::Version
75
89
  version: '0'
76
90
  type: :development
77
91
  prerelease: false
78
92
  version_requirements: !ruby/object:Gem::Requirement
79
93
  requirements:
80
- - - '>='
94
+ - - ">="
81
95
  - !ruby/object:Gem::Version
82
96
  version: '0'
83
97
  description: Fluentd plugin to measure elapsed time to process messages
@@ -86,9 +100,9 @@ executables: []
86
100
  extensions: []
87
101
  extra_rdoc_files: []
88
102
  files:
89
- - .gitignore
90
- - .rspec
91
- - .travis.yml
103
+ - ".gitignore"
104
+ - ".rspec"
105
+ - ".travis.yml"
92
106
  - CHANGELOG.md
93
107
  - Gemfile
94
108
  - LICENSE
@@ -116,17 +130,17 @@ require_paths:
116
130
  - lib
117
131
  required_ruby_version: !ruby/object:Gem::Requirement
118
132
  requirements:
119
- - - '>='
133
+ - - ">="
120
134
  - !ruby/object:Gem::Version
121
135
  version: '0'
122
136
  required_rubygems_version: !ruby/object:Gem::Requirement
123
137
  requirements:
124
- - - '>='
138
+ - - ">="
125
139
  - !ruby/object:Gem::Version
126
140
  version: '0'
127
141
  requirements: []
128
142
  rubyforge_project:
129
- rubygems_version: 2.0.3
143
+ rubygems_version: 2.2.2
130
144
  signing_key:
131
145
  specification_version: 4
132
146
  summary: Fluentd plugin to measure elapsed time to process messages