fluent-plugin-map 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: a4ad3a04fd6a20ad899c5bbcdc8677c4a168d99e
4
- data.tar.gz: e71017da3b0fc6996bd61cda02a65250115e0c0d
3
+ metadata.gz: 4ef5c9646ede101aef6159afc29fe2c23f71ae13
4
+ data.tar.gz: 4b85b98b5bc707817699449ae65b7a9fef64f670
5
5
  SHA512:
6
- metadata.gz: b31eac601f6eaca70e7a13248ffd0d9933577e457358a41ad52cda3652d79fa4d13e2295d126d86a8a543bd77f363e60a52042fc3af76f2e5cc2a74e9d564649
7
- data.tar.gz: cae9c24f03345592f87087e3ac1ab1f5557bbc57cd3825fea2c94a94956e4201f0771f4d5dcde31e53c54e761eb8f3f239c1e4f3025c279dc1b3001a6334e7bc
6
+ metadata.gz: d022df78ec9b3e3b9eaf60b7c24f48f2b21e455f4e551ad147807e76bd01a78a7f7de89ea7ab1ed6e5edf5d0232789c6bdf7b9b47f0462539c4c46e910e14afa
7
+ data.tar.gz: 5927afac4efd90c7b3a5a85901db2e93328f4b2144ba6e87d30312f9988e4612108411cd726656c3120eaa6a9a21d489603e18ed9749cd28ceed3ae14279033c
@@ -4,27 +4,93 @@
4
4
 
5
5
  fluent-plugin-map(out\_map) is the non-buffered plugin that can convert an event log to different event log(s)
6
6
 
7
- ## Example
7
+ ## MapFilter
8
+
9
+ ### Example
10
+
11
+ This sample config filter code file and time file.
12
+
13
+ <source>
14
+ @type tail
15
+ format apache
16
+ path /var/log/httpd-access.log
17
+ tag tag
18
+ @label @raw
19
+ </source>
20
+ <label @raw>
21
+ <match **>
22
+ @type copy
23
+ <store>
24
+ @type relabel
25
+ @label @code
26
+ </store>
27
+ <store>
28
+ @type relabel
29
+ @label @time
30
+ </store>
31
+ </match>
32
+ </label>
33
+ <label @code>
34
+ <filter **>
35
+ @type map
36
+ map ([time, {"code" => record["code"].to_i}])
37
+ </filter>
38
+ <match **>
39
+ @type file
40
+ path code.log
41
+ </match>
42
+ </label>
43
+ <label @time>
44
+ <filter **>
45
+ @type map
46
+ map ([time, {"time" => record["time"].to_i}])
47
+ </filter>
48
+ <match **>
49
+ @type file
50
+ path time.log
51
+ </match>
52
+ </label>
53
+
54
+
55
+ The parameter "map" can use 2 variables in event log; time, record. The format of time is an integer number of seconds since the Epoch. The format of record is hash.
56
+ The config file parses # as the begin of comment. So the "map" value cannot use #{tag} operation.
57
+ This plugin can output multi logs by seting multi to true.
58
+
59
+ If you don't use multi option, you can use time, record parameter. The 2 following filter directive is same:
60
+
61
+ <filter tag>
62
+ @type map
63
+ map ([time, {"code" => record["code"].to_i}])
64
+ </filter>
65
+ <filter tag>
66
+ @type map
67
+ time time
68
+ record ({"code" => record["code"].to_i})
69
+ </filter>
70
+
71
+ ## MapOutput
72
+
73
+ ### Example
8
74
 
9
75
  This sample config output code file and time file.
10
76
 
11
77
  <source>
12
- type tail
78
+ @type tail
13
79
  format apache
14
80
  path /var/log/httpd-access.log
15
81
  tag tag
16
82
  </source>
17
83
  <match tag>
18
- type map
84
+ @type map
19
85
  map ([["code." + tag, time, {"code" => record["code"].to_i}], ["time." + tag, time, {"time" => record["time"].to_i}]])
20
86
  multi true
21
87
  </match>
22
88
  <match code.tag>
23
- type file
89
+ @type file
24
90
  path code.log
25
91
  </match>
26
92
  <match time.tag>
27
- type file
93
+ @type file
28
94
  path time.log
29
95
  </match>
30
96
 
@@ -36,17 +102,20 @@ This plugin can output multi logs by seting multi to true.
36
102
  If you don't use multi option, you can use key, time, record parameter. The 2 following match directive is same:
37
103
 
38
104
  <match tag>
39
- type map
105
+ @type map
40
106
  map (["code." + tag, time, {"code" => record["code"].to_i}])
41
107
  </match>
42
108
  <match tag>
43
- type map
109
+ @type map
44
110
  tag ("code." + tag)
45
111
  time time
46
112
  record ({"code" => record["code"].to_i})
47
113
  </match>
48
114
 
49
- Note: you have to wrap some configuration values with parenthesis like `("code." + tag)`, to avoid parsing by Fluentd itself.
115
+
116
+ ## Note
117
+
118
+ you have to wrap some configuration values with parenthesis like `("code." + tag)`, to avoid parsing by Fluentd itself.
50
119
  See also: [Fluentd | Configuration File | Format tips](http://docs.fluentd.org/articles/config-file#format-tips)
51
120
 
52
121
  ## Copyright
@@ -3,7 +3,7 @@ $:.push File.expand_path("../lib", __FILE__)
3
3
 
4
4
  Gem::Specification.new do |s|
5
5
  s.name = "fluent-plugin-map"
6
- s.version = "0.1.0"
6
+ s.version = "0.1.1"
7
7
  s.authors = ["Kohei Tomita", "Hiroshi Hatake", "Kenji Okomoto"]
8
8
  s.email = ["tommy.fmale@gmail.com", "cosmo0920.oucc@gmail.com", "okkez000@gmail.com"]
9
9
  s.homepage = "https://github.com/fluent-plugins-nursery/fluent-plugin-map"
@@ -0,0 +1,89 @@
1
+ #
2
+ # fluent-plugin-map
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+ #
16
+
17
+ require 'fluent/plugin/map_support'
18
+ require 'fluent/plugin/map_config_param'
19
+ require 'fluent/plugin/parse_map_mixin'
20
+
21
+ module Fluent
22
+ class MapFilter < Fluent::Filter
23
+ Fluent::Plugin.register_filter('map', self)
24
+
25
+ include Fluent::MapConfigParam
26
+ include Fluent::ParseMap::Mixin
27
+
28
+ def configure(conf)
29
+ super
30
+ @format = determine_format()
31
+ configure_format()
32
+ @map = create_map(conf)
33
+ @map_support = Fluent::MapSupport.new(@map, self)
34
+ end
35
+
36
+ def determine_format()
37
+ if @format
38
+ @format
39
+ elsif @map
40
+ "map"
41
+ elsif @time && @record
42
+ "record"
43
+ else
44
+ raise Fluent::ConfigError, "Any of map, 2 parameters(time, and record) or format is required "
45
+ end
46
+ end
47
+
48
+ def configure_format()
49
+ case @format
50
+ when "map"
51
+ # pass
52
+ when "record"
53
+ raise Fluent::ConfigError, "multi and 2 parameters(time, and record) are not compatible" if @multi
54
+ when "multimap"
55
+ # pass.
56
+ else
57
+ raise Fluent::ConfigError, "format #{@format} is invalid."
58
+ end
59
+ end
60
+
61
+ def create_map(conf)
62
+ # return string like double array.
63
+ case @format
64
+ when "map"
65
+ parse_map()
66
+ when "record"
67
+ "[[#{@time}, #{@record}]]"
68
+ when "multimap"
69
+ parse_multimap(conf)
70
+ end
71
+ end
72
+
73
+ def filter_stream(tag, es)
74
+ begin
75
+ new_es = Fluent::MultiEventStream.new
76
+ tag_output_es = @map_support.do_map(tag, es)
77
+ tag_output_es.each_pair do |tag, output_es|
78
+ output_es.each{|time, record|
79
+ new_es.add(time, record)
80
+ }
81
+ end
82
+ new_es
83
+ rescue SyntaxError => e
84
+ log.error "map command is syntax error: #{@map}"
85
+ e #for test
86
+ end
87
+ end
88
+ end
89
+ end
@@ -0,0 +1,30 @@
1
+ #
2
+ # fluent-plugin-map
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+ #
16
+
17
+ module Fluent
18
+ module MapConfigParam
19
+ def self.included(klass)
20
+ klass.instance_eval {
21
+ config_param :map, :string, :default => nil
22
+ config_param :time, :string, :default => nil
23
+ config_param :record, :string, :default => nil
24
+ config_param :multi, :bool, :default => false
25
+ config_param :timeout, :time, :default => 1
26
+ config_param :format, :string, :default => nil
27
+ }
28
+ end
29
+ end
30
+ end
@@ -0,0 +1,113 @@
1
+ #
2
+ # fluent-plugin-map
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+ #
16
+
17
+ module Fluent
18
+ class MapSupport
19
+ def initialize(map, plugin)
20
+ @map = map
21
+ @plugin = plugin
22
+ if defined?(Fluent::Filter) and plugin.is_a?(Fluent::Filter)
23
+ singleton_class.module_eval(<<-CODE)
24
+ def map_func(time, record)
25
+ #{@map}
26
+ end
27
+ CODE
28
+ class << self
29
+ alias_method :generate_tuples, :generate_tuples_filter
30
+ alias_method :do_map, :do_map_filter
31
+ end
32
+ elsif plugin.is_a?(Fluent::Output)
33
+ singleton_class.module_eval(<<-CODE)
34
+ def map_func(tag, time, record)
35
+ #{@map}
36
+ end
37
+ CODE
38
+ class << self
39
+ alias_method :generate_tuples, :generate_tuples_output
40
+ alias_method :do_map, :do_map_output
41
+ end
42
+ end
43
+ end
44
+
45
+ def do_map(tag, es)
46
+ # This method will be overwritten in #initailize.
47
+ end
48
+
49
+ def do_map_filter(tag, es)
50
+ tuples = generate_tuples(tag, es)
51
+
52
+ tag_output_es = Hash.new{|h, key| h[key] = Fluent::MultiEventStream.new}
53
+ tuples.each do |time, record|
54
+ if time == nil || record == nil
55
+ raise SyntaxError.new
56
+ end
57
+ tag_output_es[tag].add(time, record)
58
+ @plugin.log.trace { [tag, time, record].inspect }
59
+ end
60
+ tag_output_es
61
+ end
62
+
63
+ def do_map_output(tag, es)
64
+ tuples = generate_tuples(tag, es)
65
+
66
+ tag_output_es = Hash.new{|h, key| h[key] = Fluent::MultiEventStream.new}
67
+ tuples.each do |tag, time, record|
68
+ if time == nil || record == nil
69
+ raise SyntaxError.new
70
+ end
71
+ tag_output_es[tag].add(time, record)
72
+ @plugin.log.trace { [tag, time, record].inspect }
73
+ end
74
+ tag_output_es
75
+ end
76
+
77
+ def generate_tuples
78
+ # This method will be overwritten in #initailize.
79
+ end
80
+
81
+ def generate_tuples_filter(tag, es)
82
+ tuples = []
83
+ es.each {|time, record|
84
+ timeout_block do
85
+ new_tuple = map_func(time, record)
86
+ tuples.concat new_tuple
87
+ end
88
+ }
89
+ tuples
90
+ end
91
+
92
+ def generate_tuples_output(tag, es)
93
+ tuples = []
94
+ es.each {|time, record|
95
+ timeout_block do
96
+ new_tuple = map_func(tag, time, record)
97
+ tuples.concat new_tuple
98
+ end
99
+ }
100
+ tuples
101
+ end
102
+
103
+ def timeout_block
104
+ begin
105
+ Timeout.timeout(@plugin.timeout){
106
+ yield
107
+ }
108
+ rescue Timeout::Error
109
+ @plugin.log.error {"Timeout: #{Time.at(time)} #{tag} #{record.inspect}"}
110
+ end
111
+ end
112
+ end
113
+ end
@@ -14,10 +14,17 @@
14
14
  # limitations under the License.
15
15
  #
16
16
 
17
+ require 'fluent/plugin/map_support'
18
+ require 'fluent/plugin/map_config_param'
19
+ require 'fluent/plugin/parse_map_mixin'
20
+
17
21
  module Fluent
18
22
  class MapOutput < Fluent::Output
19
23
  Fluent::Plugin.register_output('map', self)
20
24
 
25
+ include Fluent::MapConfigParam
26
+ include Fluent::ParseMap::Mixin
27
+
21
28
  # Define `router` method of v0.12 to support v0.10 or earlier
22
29
  unless method_defined?(:router)
23
30
  define_method("router") { Fluent::Engine }
@@ -27,27 +34,15 @@ module Fluent
27
34
  define_method("log") { $log }
28
35
  end
29
36
 
30
- config_param :map, :string, :default => nil
31
- config_param :tag, :string, :default => nil
32
37
  config_param :key, :string, :default => nil #deprecated
33
- config_param :time, :string, :default => nil
34
- config_param :record, :string, :default => nil
35
- config_param :multi, :bool, :default => false
36
- config_param :timeout, :time, :default => 1
37
- config_param :format, :string, :default => nil
38
-
39
- MMAP_MAX_NUM = 50
38
+ config_param :tag, :string, :default => nil
40
39
 
41
40
  def configure(conf)
42
41
  super
43
42
  @format = determine_format()
44
43
  configure_format()
45
44
  @map = create_map(conf)
46
- singleton_class.module_eval(<<-CODE)
47
- def map_func(tag, time, record)
48
- #{@map}
49
- end
50
- CODE
45
+ @map_support = Fluent::MapSupport.new(@map, self)
51
46
  end
52
47
 
53
48
  def determine_format()
@@ -58,7 +53,7 @@ module Fluent
58
53
  elsif (@tag || @key) && @time && @record
59
54
  "record"
60
55
  else
61
- raise ConfigError, "Any of map, 3 parameters(tag, time, and record) or format is required "
56
+ raise Fluent::ConfigError, "Any of map, 3 parameters(tag, time, and record) or format is required "
62
57
  end
63
58
  end
64
59
 
@@ -68,11 +63,11 @@ module Fluent
68
63
  # pass
69
64
  when "record"
70
65
  @tag ||= @key
71
- raise ConfigError, "multi and 3 parameters(tag, time, and record) are not compatible" if @multi
66
+ raise Fluent::ConfigError, "multi and 3 parameters(tag, time, and record) are not compatible" if @multi
72
67
  when "multimap"
73
68
  # pass.
74
69
  else
75
- raise ConfigError, "format #{@format} is invalid."
70
+ raise Fluent::ConfigError, "format #{@format} is invalid."
76
71
  end
77
72
  end
78
73
 
@@ -88,45 +83,9 @@ module Fluent
88
83
  end
89
84
  end
90
85
 
91
- def parse_map()
92
- if @multi
93
- @map
94
- else
95
- "[#{@map}]"
96
- end
97
- end
98
-
99
- def parse_multimap(conf)
100
- check_mmap_range(conf)
101
-
102
- prev_mmap = nil
103
- result_mmaps = (1..MMAP_MAX_NUM).map { |i|
104
- mmap = conf["mmap#{i}"]
105
- if (i > 1) && prev_mmap.nil? && !mmap.nil?
106
- raise ConfigError, "Jump of mmap index found. mmap#{i - 1} is missing."
107
- end
108
- prev_mmap = mmap
109
- next if mmap.nil?
110
-
111
- mmap
112
- }.compact.join(',')
113
- "[#{result_mmaps}]"
114
- end
115
-
116
- def check_mmap_range(conf)
117
- invalid_mmap = conf.keys.select { |k|
118
- m = k.match(/^mmap(\d+)$/)
119
- m ? !((1..MMAP_MAX_NUM).include?(m[1].to_i)) : false
120
- }
121
- unless invalid_mmap.empty?
122
- raise ConfigError, "Invalid mmapN found. N should be 1 - #{MMAP_MAX_NUM}: " + invalid_mmap.join(",")
123
- end
124
- end
125
-
126
-
127
86
  def emit(tag, es, chain)
128
87
  begin
129
- tag_output_es = do_map(tag, es)
88
+ tag_output_es = @map_support.do_map(tag, es)
130
89
  tag_output_es.each_pair do |tag, output_es|
131
90
  router.emit_stream(tag, output_es)
132
91
  end
@@ -138,38 +97,5 @@ module Fluent
138
97
  e #for test
139
98
  end
140
99
  end
141
-
142
- def do_map(tag, es)
143
- tuples = generate_tuples(tag, es)
144
-
145
- tag_output_es = Hash.new{|h, key| h[key] = MultiEventStream::new}
146
- tuples.each do |tag, time, record|
147
- if time == nil || record == nil
148
- raise SyntaxError.new
149
- end
150
- tag_output_es[tag].add(time, record)
151
- log.trace { [tag, time, record].inspect }
152
- end
153
- tag_output_es
154
- end
155
-
156
- def generate_tuples(tag, es)
157
- tuples = []
158
- es.each {|time, record|
159
- new_tuple = map_func(tag, time, record)
160
- tuples.concat new_tuple
161
- }
162
- tuples
163
- end
164
-
165
- def timeout_block(tag, time, record)
166
- begin
167
- Timeout.timeout(@timeout){
168
- yield
169
- }
170
- rescue Timeout::Error
171
- log.error {"Timeout: #{Time.at(time)} #{tag} #{record.inspect}"}
172
- end
173
- end
174
100
  end
175
101
  end
@@ -0,0 +1,59 @@
1
+ #
2
+ # fluent-plugin-map
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+ #
16
+
17
+ module Fluent
18
+ module ParseMap
19
+ module Mixin
20
+
21
+ MMAP_MAX_NUM = 50
22
+
23
+ def parse_map()
24
+ if @multi
25
+ @map
26
+ else
27
+ "[#{@map}]"
28
+ end
29
+ end
30
+
31
+ def parse_multimap(conf)
32
+ check_mmap_range(conf)
33
+
34
+ prev_mmap = nil
35
+ result_mmaps = (1..MMAP_MAX_NUM).map { |i|
36
+ mmap = conf["mmap#{i}"]
37
+ if (i > 1) && prev_mmap.nil? && !mmap.nil?
38
+ raise Fluent::ConfigError, "Jump of mmap index found. mmap#{i - 1} is missing."
39
+ end
40
+ prev_mmap = mmap
41
+ next if mmap.nil?
42
+
43
+ mmap
44
+ }.compact.join(',')
45
+ "[#{result_mmaps}]"
46
+ end
47
+
48
+ def check_mmap_range(conf)
49
+ invalid_mmap = conf.keys.select { |k|
50
+ m = k.match(/^mmap(\d+)$/)
51
+ m ? !((1..MMAP_MAX_NUM).include?(m[1].to_i)) : false
52
+ }
53
+ unless invalid_mmap.empty?
54
+ raise Fluent::ConfigError, "Invalid mmapN found. N should be 1 - #{MMAP_MAX_NUM}: " + invalid_mmap.join(",")
55
+ end
56
+ end
57
+ end
58
+ end
59
+ end
@@ -23,3 +23,4 @@ unless ENV.has_key?('VERBOSE')
23
23
  end
24
24
 
25
25
  require "fluent/plugin/out_map"
26
+ require "fluent/plugin/filter_map" if defined?(Fluent::Filter)
@@ -0,0 +1,182 @@
1
+ require 'helper'
2
+
3
+ class MapFilterTest < Test::Unit::TestCase
4
+ def setup
5
+ omit "Use Fluentd v0.12 or later" unless defined?(Fluent::Filter)
6
+ Fluent::Test.setup
7
+ end
8
+
9
+ CONFIG = %[
10
+ map [time, record]
11
+ multi false
12
+ ]
13
+
14
+ def create_driver(conf = CONFIG, tag='test.input')
15
+ Fluent::Test::FilterTestDriver.new(Fluent::MapFilter, tag).configure(conf)
16
+ end
17
+
18
+ def test_syntax_error
19
+ tag = "tag"
20
+ time = Time.local(2012, 10, 10, 10, 10, 0).to_i
21
+ record = {'code' => '300'}
22
+
23
+ #map is syntax error
24
+ syntax_error_config = %[
25
+ map time.
26
+ ]
27
+ assert_raise SyntaxError do
28
+ create_driver(syntax_error_config, tag)
29
+ end
30
+ end
31
+
32
+ def test_syntax_error2
33
+ tag = "tag"
34
+ time = Time.local(2012, 10, 10, 10, 10, 0).to_i
35
+ record = {'code' => '300'}
36
+
37
+ #map output lligal format
38
+ syntax_error_config = %[
39
+ map time
40
+ ]
41
+ d1 = create_driver(syntax_error_config, tag)
42
+ es = Fluent::OneEventStream.new(time, record)
43
+ e = d1.instance.filter_stream(tag, es)
44
+ assert e.kind_of?(SyntaxError)
45
+ end
46
+
47
+ def test_tag_convert_using_time_record
48
+ tag = 'tag.raw'
49
+ time = Time.local(2012, 10, 10, 10, 10, 10).to_i
50
+ record = {'code' => '300'}
51
+
52
+ d1 = create_driver %[
53
+ time time
54
+ record record
55
+ ], tag
56
+ d1.run do
57
+ d1.emit(record, time)
58
+ end
59
+ filtered = d1.filtered_as_array
60
+ assert_equal 1, filtered.length
61
+ assert_equal ["tag.raw", time, record], filtered[0]
62
+ end
63
+
64
+ #deprected specification test
65
+ def test_config_error_time
66
+ tag = "tag"
67
+ record = {'code' => '300'}
68
+
69
+ #require time
70
+ assert_raise(Fluent::ConfigError){
71
+ create_driver %[
72
+ record record
73
+ ], tag
74
+ }
75
+ end
76
+
77
+ def test_config_error_record
78
+ tag = "tag"
79
+ time = Time.local(2012, 10, 10, 10, 10, 0).to_i
80
+
81
+ #require record
82
+ assert_raise(Fluent::ConfigError){
83
+ create_driver %[
84
+ time time
85
+ ], tag
86
+ }
87
+ end
88
+
89
+ def test_config_error_multi
90
+ tag = "tag"
91
+ time = Time.local(2012, 10, 10, 10, 10, 0).to_i
92
+ record = {'code' => '300'}
93
+
94
+ #require time
95
+ assert_raise(Fluent::ConfigError){
96
+ create_driver %[
97
+ time time
98
+ record record
99
+ multi true
100
+ ], tag
101
+ }
102
+ end
103
+
104
+ def test_config_error_sleep
105
+ tag = 'tag'
106
+ time = Time.local(2012, 10, 10, 10, 10, 10).to_i
107
+ record = {'code' => '300'}
108
+
109
+ assert_raise(SyntaxError) {
110
+ create_driver %[
111
+ tag "newtag"
112
+ time sleep 10
113
+ record record
114
+ timeout 1s
115
+ ], tag
116
+ }
117
+ end
118
+
119
+ # Add format test
120
+ ## test format type (map, record, maps)
121
+ ## test Backward compatibility without format
122
+ ##
123
+
124
+ def test_convert_format_map
125
+ tag = 'tag.raw'
126
+ time = Time.local(2012, 10, 10, 10, 10, 10).to_i
127
+ record = {'code1' => '300', 'code2' => '400'}
128
+
129
+ d1 = create_driver %[
130
+ format map
131
+ map [[time, record["code1"]], [time, record["code2"]]]
132
+ multi true
133
+ ], tag
134
+ d1.run do
135
+ d1.emit(record, time)
136
+ end
137
+ filtered = d1.filtered_as_array
138
+ assert_equal 2, filtered.length
139
+ assert_equal tag, d1.tag # tag shouldn't be changed
140
+ assert_equal ["tag.raw", time, record["code1"]], filtered[0]
141
+ assert_equal ["tag.raw", time, record["code2"]], filtered[1]
142
+ end
143
+
144
+ def test_tag_convert_format_record
145
+ tag = 'tag.raw'
146
+ time = Time.local(2012, 10, 10, 10, 10, 10).to_i
147
+ record = {'code' => '300'}
148
+
149
+ d1 = create_driver %[
150
+ format record
151
+ time time
152
+ record record
153
+ ], tag
154
+ d1.run do
155
+ d1.emit(record, time)
156
+ end
157
+ filtered = d1.filtered_as_array
158
+ assert_equal 1, filtered.length
159
+ assert_equal tag, d1.tag # tag shouldn't be changed
160
+ assert_equal ["tag.raw", time, record], filtered[0]
161
+ end
162
+
163
+ def test_convert_format_multimap
164
+ tag = 'tag.raw'
165
+ time = Time.local(2012, 10, 10, 10, 10, 10).to_i
166
+ record = {'code1' => '300', 'code2' => '400'}
167
+
168
+ d1 = create_driver %[
169
+ format multimap
170
+ mmap1 [time, record["code1"]]
171
+ mmap2 [time, record["code2"]]
172
+ ], tag
173
+ d1.run do
174
+ d1.emit(record, time)
175
+ end
176
+ filtered = d1.filtered_as_array
177
+ assert_equal 2, filtered.length
178
+ assert_equal tag, d1.tag # tag shouldn't be changed
179
+ assert_equal ["tag.raw", time, record["code1"]], filtered[0]
180
+ assert_equal ["tag.raw", time, record["code2"]], filtered[1]
181
+ end
182
+ end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-map
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kohei Tomita
@@ -10,7 +10,7 @@ authors:
10
10
  autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
- date: 2017-03-10 00:00:00.000000000 Z
13
+ date: 2017-03-14 00:00:00.000000000 Z
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency
16
16
  name: rake
@@ -95,8 +95,13 @@ files:
95
95
  - gemfiles/fluentd_0_10.gemfile
96
96
  - gemfiles/fluentd_0_12.gemfile
97
97
  - gemfiles/fluentd_0_14.gemfile
98
+ - lib/fluent/plugin/filter_map.rb
99
+ - lib/fluent/plugin/map_config_param.rb
100
+ - lib/fluent/plugin/map_support.rb
98
101
  - lib/fluent/plugin/out_map.rb
102
+ - lib/fluent/plugin/parse_map_mixin.rb
99
103
  - test/helper.rb
104
+ - test/plugin/test_filter_map.rb
100
105
  - test/plugin/test_out_map.rb
101
106
  homepage: https://github.com/fluent-plugins-nursery/fluent-plugin-map
102
107
  licenses:
@@ -125,4 +130,5 @@ summary: fluent-plugin-map is the non-buffered plugin that can convert an event
125
130
  to different event log(s).
126
131
  test_files:
127
132
  - test/helper.rb
133
+ - test/plugin/test_filter_map.rb
128
134
  - test/plugin/test_out_map.rb