logstash-codec-protobuf 0.1.1 → 0.1.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 92231ace59a2a8218764c99dab51abb5636db440
4
- data.tar.gz: d8129bf3a5bf757f42bb3c2e95ae8d74455531b2
3
+ metadata.gz: 32ac341961786a035ed34e5e221de5e6944c2226
4
+ data.tar.gz: 8bb727065267603b4d0d771e2ab1937e7eb83660
5
5
  SHA512:
6
- metadata.gz: 6a7379eeb8a33c4dd722a01e55bd67f5daeb37ba7ad323ebd41707b359f27a585ca5dcaaf720ecb74d6746c50230fda17249ef849ff3f01489714dbd9c286769
7
- data.tar.gz: d2175a9b628f60128295e9e4871d895ba8cb65d33202af0a3c66ac63c0aa4540a173b3b7c508f28de8db3d950f4b72604e308d5138530a79393144244c5188a1
6
+ metadata.gz: ba83ef5a43fa82e25d6cb248fa8076982b43d833df1d2ee62ee69c045bedd2664ff04a5b112d880344ebaf470116a176ab008bbccad9f6aab368c30b513ce5d1
7
+ data.tar.gz: 9699d09931ca8bac5d0242c4952b8496467d8c586032decdce6213ae956e43053b08039115a3deee7db7eaa74fd2d8a9700c32a12d794dcbaf338cbb01419c82
data/README.md CHANGED
@@ -10,7 +10,13 @@ This is a codec plugin for [Logstash](https://github.com/elastic/logstash) to pa
10
10
  bin/plugin install /path/to/logstash-codec-protobuf-$VERSION.gem
11
11
  * use the codec in your logstash config file. See details below.
12
12
 
13
- ## Usage
13
+ ## Configuration
14
+
15
+ include_path (required): an array of strings with filenames or directory names where logstash can find your protobuf definitions. Please provide absolute paths. For directories it will only try to import files ending on .rb
16
+
17
+ class_name (required): the name of the protobuf class that is to be decoded or encoded.
18
+
19
+ ## Usage example: decoder
14
20
 
15
21
  Use this as a codec in any logstash input. Just provide the name of the class that your incoming objects will be encoded in, and specify the path to the compiled definition.
16
22
  Here's an example for a kafka input:
@@ -46,11 +52,11 @@ Set the class name to the parent class:
46
52
 
47
53
  class_name => "Foods::Dairy::Cheese"
48
54
 
49
- ## Configuration
55
+ ## Usage example: encoder
50
56
 
51
- include_path (required): an array of strings with filenames or directory names where logstash can find your protobuf definitions. Please provide absolute paths. For directories it will only try to import files ending on .rb
52
-
53
- class_name (required): the name of the protobuf class that is to be decoded.
57
+ The configuration of the codec for encoding logstash events for a protobuf output is pretty much the same as for the decoder input usage as demonstrated above. There are some constraints though that you need to be aware of:
58
+ * the protobuf definition needs to contain all the fields that logstash typically adds to an event, in the corrent data type. Examples for this are @timestamp (string), @version (string), host, path, all of which depend on your input sources and filters aswell. If you do not want to add those fields to your protobuf definition then please use a [modify filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html) to [remove](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-remove_field) the undesired fields.
59
+ * object members starting with @ are somewhat problematic in protobuf definitions. Therefore those fields will automatically be renamed to remove the at character. This also effects the important @timestamp field. Please name it just "timestamp" in your definition.
54
60
 
55
61
 
56
62
  ## Troubleshooting
@@ -59,7 +65,12 @@ class_name (required): the name of the protobuf class that is to be decoded.
59
65
 
60
66
  If you include more than one definition class, consider the order of inclusion. This is especially relevant if you include whole directories. A definition might refer to another definition that is not loaded yet. In this case, please specify the files in the include_path variable in reverse order of reference. See 'Example with referenced definitions' above.
61
67
 
68
+ ### no protobuf output
69
+
70
+ Maybe your protobuf definition does not fullfill the requirements and needs additional fields. Run logstash with the --debug flag and grep for "error 2".
71
+
72
+
73
+ ## Limitations and roadmap
62
74
 
63
- # #Roadmap
75
+ * maybe add support for setting undefined fields from default values in the decoder
64
76
 
65
- Currently the plugin supports the decode functionality only. Maybe we'll add the encoding part also.
@@ -12,72 +12,178 @@ class LogStash::Codecs::Protobuf < LogStash::Codecs::Base
12
12
  # Name of the class to decode
13
13
  config :class_name, :validate => :string, :required => true
14
14
 
15
-
15
+ # For benchmarking only, not intended for public use: change encoder strategy.
16
+ # valid method names are: encoder_strategy_1 (the others are not implemented yet)
17
+ config :encoder_method, :validate => :string, :default => "encoder_strategy_1"
18
+
16
19
  def register
20
+ @pb_metainfo = {}
17
21
  include_path.each { |path| require_pb_path(path) }
18
22
  @obj = create_object_from_name(class_name)
19
23
  @logger.debug("Protobuf files successfully loaded.")
24
+
20
25
  end
21
26
 
22
27
  def decode(data)
23
28
  decoded = @obj.parse(data.to_s)
24
- results = extract_vars(decoded)
29
+ results = keys2strings(decoded.to_hash)
25
30
  yield LogStash::Event.new(results) if block_given?
26
31
  end # def decode
27
32
 
33
+ def keys2strings(data)
34
+ if data.is_a?(::Hash)
35
+ new_hash = Hash.new
36
+ data.each{|k,v| new_hash[k.to_s] = keys2strings(v)}
37
+ new_hash
38
+ else
39
+ data
40
+ end
41
+ end
42
+
43
+
28
44
  def encode(event)
29
- raise 'Encodeør function not implemented yet for protobuf codec. Sorry!'
30
- # @on_event.call(event, event.to_s)
31
- # TODO integrate
45
+ protobytes = generate_protobuf(event)
46
+ @on_event.call(event, protobytes)
32
47
  end # def encode
33
48
 
49
+ private
50
+ def generate_protobuf(event)
51
+ meth = self.method(encoder_method)
52
+ data = meth.call(event, @class_name)
53
+ begin
54
+ msg = @obj.new(data)
55
+ msg.serialize_to_string
56
+ rescue NoMethodError
57
+ @logger.debug("error 2: NoMethodError. Maybe mismatching protobuf definition. Required fields are: " + event.to_hash.keys.join(", "))
58
+ end
59
+ end
34
60
 
61
+ def encoder_strategy_1(event, class_name)
62
+ _encoder_strategy_1(event.to_hash, class_name)
63
+
64
+ end
65
+
66
+ def _encoder_strategy_1(datahash, class_name)
67
+ fields = clean_hash_keys(datahash)
68
+ fields = flatten_hash_values(fields) # TODO we could merge this and the above method back into one to save one iteration, but how are we going to name it?
69
+ meta = get_complex_types(class_name) # returns a hash with member names and their protobuf class names
70
+ meta.map do | (k,typeinfo) |
71
+ if fields.include?(k)
72
+ original_value = fields[k]
73
+ proto_obj = create_object_from_name(typeinfo)
74
+ fields[k] =
75
+ if original_value.is_a?(::Array)
76
+ ecs1_list_helper(original_value, proto_obj, typeinfo)
77
+
78
+ else
79
+ recursive_fix = _encoder_strategy_1(original_value, class_name)
80
+ proto_obj.new(recursive_fix)
81
+ end # if is array
82
+ end
83
+
84
+ end
85
+
86
+ fields
87
+ end
88
+
89
+ def ecs1_list_helper(value, proto_obj, class_name)
90
+ # make this field an array/list of protobuf objects
91
+ # value is a list of hashed complex objects, each of which needs to be protobuffed and
92
+ # put back into the list.
93
+ next unless value.is_a?(::Array)
94
+ value.map { |x| _encoder_strategy_1(x, class_name) }
95
+ value
96
+ end
97
+
98
+ def flatten_hash_values(datahash)
99
+ # 2) convert timestamps and other objects to strings
100
+ next unless datahash.is_a?(::Hash)
101
+
102
+ ::Hash[datahash.map{|(k,v)| [k, (convert_to_string?(v) ? v.to_s : v)] }]
103
+ end
104
+
105
+ def clean_hash_keys(datahash)
106
+ # 1) remove @ signs from keys
107
+ next unless datahash.is_a?(::Hash)
108
+
109
+ ::Hash[datahash.map{|(k,v)| [remove_atchar(k.to_s), v] }]
110
+ end #clean_hash_keys
111
+
112
+ def convert_to_string?(v)
113
+ !(v.is_a?(Fixnum) || v.is_a?(::Hash) || v.is_a?(::Array) || [true, false].include?(v))
114
+ end
115
+
116
+
117
+ def remove_atchar(key) # necessary for @timestamp fields and the likes. Protobuf definition doesn't handle @ in field names well.
118
+ key.dup.gsub(/@/,'')
119
+ end
35
120
 
36
121
  private
37
122
  def create_object_from_name(name)
38
123
  begin
39
124
  @logger.debug("Creating instance of " + name)
40
- return name.split('::').inject(Object) { |n,c| n.const_get c }
125
+ name.split('::').inject(Object) { |n,c| n.const_get c }
41
126
  end
42
127
  end
43
128
 
129
+ def get_complex_types(class_name)
130
+ @pb_metainfo[class_name]
131
+ end
132
+
133
+ def require_with_metadata_analysis(filename)
134
+ require filename
135
+ regex_class_name = /\s*class\s*(?<name>.+?)\s+/
136
+ regex_module_name = /\s*module\s*(?<name>.+?)\s+/
137
+ regex_pbdefs = /\s*(optional|repeated)(\s*):(?<type>.+),(\s*):(?<name>\w+),(\s*)(?<position>\d+)/
138
+ # now we also need to find out which class it contains and the protobuf definitions in it.
139
+ # We'll unfortunately need that later so that we can create nested objects.
140
+ begin
141
+ class_name = ""
142
+ type = ""
143
+ field_name = ""
144
+ classname_found = false
145
+ File.readlines(filename).each do |line|
146
+ if ! (line =~ regex_module_name).nil? && !classname_found # because it might be declared twice in the file
147
+ class_name << $1
148
+ class_name << "::"
149
+
150
+ end
151
+ if ! (line =~ regex_class_name).nil? && !classname_found # because it might be declared twice in the file
152
+ class_name << $1
153
+ @pb_metainfo[class_name] = {}
154
+ classname_found = true
155
+ end
156
+ if ! (line =~ regex_pbdefs).nil?
157
+ type = $1
158
+ field_name = $2
159
+ if type =~ /::/
160
+ @pb_metainfo[class_name][field_name] = type.gsub!(/^:/,"")
161
+
162
+ end
163
+ end
164
+ end
165
+ rescue Exception => e
166
+ @logger.warn("error 3: unable to read pb definition from file " + filename+ ". Reason: #{e.inspect}. Last settings were: class #{class_name} field #{field_name} type #{type}. Backtrace: " + e.backtrace.inspect.to_s)
167
+ end
168
+ if class_name.nil?
169
+ @logger.warn("error 4: class name not found in file " + filename)
170
+ end
171
+ end
44
172
 
45
173
  def require_pb_path(dir_or_file)
46
174
  f = dir_or_file.end_with? ('.rb')
47
175
  begin
48
176
  if f
49
177
  @logger.debug("Including protobuf file: " + dir_or_file)
50
- require dir_or_file
178
+ require_with_metadata_analysis dir_or_file
51
179
  else
52
180
  Dir[ dir_or_file + '/*.rb'].each { |file|
53
181
  @logger.debug("Including protobuf path: " + dir_or_file + "/" + file)
54
- require file
182
+ require_with_metadata_analysis file
55
183
  }
56
184
  end
57
185
  end
58
186
  end
59
187
 
60
188
 
61
- def extract_vars(decoded_object)
62
- return {} if decoded_object.nil?
63
- results = {}
64
- decoded_object.instance_variables.each do |key|
65
- formatted_key = key.to_s.gsub('@', '')
66
- next if (formatted_key == :set_fields || formatted_key == "set_fields")
67
- instance_var = decoded_object.instance_variable_get(key)
68
-
69
- results[formatted_key] =
70
- if instance_var.is_a?(::ProtocolBuffers::Message)
71
- extract_vars(instance_var)
72
- elsif instance_var.is_a?(::Hash)
73
- instance_var.inject([]) { |h, (k, v)| h[k.to_s] = extract_vars(v); h }
74
- elsif instance_var.is_a?(Enumerable) # is a list
75
- instance_var.inject([]) { |h, v| h.push(extract_vars(v)); h }
76
- else
77
- instance_var
78
- end
79
- end
80
- results
81
- end
82
-
83
189
  end # class LogStash::Codecs::Protobuf
@@ -1,7 +1,7 @@
1
1
  Gem::Specification.new do |s|
2
2
 
3
3
  s.name = 'logstash-codec-protobuf'
4
- s.version = '0.1.1'
4
+ s.version = '0.1.2'
5
5
  s.licenses = ['Apache License (2.0)']
6
6
  s.summary = "This codec may be used to decode (via inputs) and encode (via outputs) protobuf messages"
7
7
  s.description = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program"
@@ -1,3 +1,4 @@
1
+ # encoding: utf-8
1
2
  require "logstash/devutils/rspec/spec_helper"
2
3
  require "logstash/codecs/protobuf"
3
4
  require "logstash/event"
@@ -6,23 +7,199 @@ require "insist"
6
7
 
7
8
  describe LogStash::Codecs::Protobuf do
8
9
 
10
+
9
11
  context "#decode" do
10
12
 
11
- let(:plugin) { LogStash::Codecs::Protobuf.new("class_name" => "Animal::Unicorn", "include_path" => ['spec/helpers/unicorn.pb.rb']) }
13
+
14
+ #### Test case 1: Decode simple protobuf bytes for unicorn ####################################################################################################################
15
+ let(:plugin_unicorn) { LogStash::Codecs::Protobuf.new("class_name" => "Animal::Unicorn", "include_path" => ['spec/helpers/unicorn.pb.rb']) }
12
16
  before do
13
- plugin.register
17
+ plugin_unicorn.register
14
18
  end
15
19
 
16
20
  it "should return an event from protobuf encoded data" do
17
21
 
18
- data = {:colour => 'rainbow', :horn_length => 18, :last_seen => 1420081471}
22
+ data = {:colour => 'rainbow', :horn_length => 18, :last_seen => 1420081471, :has_wings => true}
19
23
  unicorn = Animal::Unicorn.new(data)
20
24
 
21
- plugin.decode(unicorn.serialize_to_string) do |event|
22
-
25
+ plugin_unicorn.decode(unicorn.serialize_to_string) do |event|
23
26
  expect(event["colour"] ).to eq(data[:colour] )
24
27
  expect(event["horn_length"] ).to eq(data[:horn_length] )
28
+ expect(event["last_seen"] ).to eq(data[:last_seen] )
29
+ expect(event["has_wings"] ).to eq(data[:has_wings] )
30
+ end
31
+ end # it
32
+
33
+
34
+
35
+ #### Test case 2: Decode complex protobuf bytes for human #####################################################################################################################
36
+
37
+
38
+
39
+
40
+ let(:plugin_human) { LogStash::Codecs::Protobuf.new("class_name" => "Animal::Human", "include_path" => ['spec/helpers/human.pb.rb']) }
41
+ before do
42
+ plugin_human.register
43
+ end
44
+
45
+ it "should return an event from complex nested protobuf encoded data" do
46
+
47
+ data_gm = {:first_name => 'Elisabeth', :last_name => "Oliveoil", :middle_names => ["Maria","Johanna"], :vegetarian=>true}
48
+ grandmother = Animal::Human.new(data_gm)
49
+ data_m = {:first_name => 'Annemarie', :last_name => "Smørebrød", :mother => grandmother}
50
+ mother = Animal::Human.new(data_m)
51
+ data_f = {:first_name => 'Karl', :middle_names => ["Theodor-Augustin"], :last_name => "Falkenstein"}
52
+ father = Animal::Human.new(data_f)
53
+ data = {:first_name => 'Hugo', :middle_names => ["Heinz", "Peter"], :last_name => "Smørebrød",:father => father, :mother => mother}
54
+ hugo = Animal::Human.new(data)
55
+
56
+ plugin_human.decode(hugo.serialize_to_string) do |event|
57
+ expect(event["first_name"] ).to eq(data[:first_name] )
58
+ expect(event["middle_names"] ).to eq(data[:middle_names] )
59
+ expect(event["last_name"] ).to eq(data[:last_name] )
60
+ expect(event["mother"]["first_name"] ).to eq(data_m[:first_name] )
61
+ expect(event["father"]["first_name"] ).to eq(data_f[:first_name] )
62
+ expect(event["mother"]["last_name"] ).to eq(data_m[:last_name] )
63
+ expect(event["mother"]["mother"]["last_name"] ).to eq(data_gm[:last_name] )
64
+ expect(event["mother"]["mother"]["first_name"] ).to eq(data_gm[:first_name] )
65
+ expect(event["mother"]["mother"]["middle_names"] ).to eq(data_gm[:middle_names] )
66
+ expect(event["mother"]["mother"]["vegetarian"] ).to eq(data_gm[:vegetarian] )
67
+ expect(event["father"]["last_name"] ).to eq(data_f[:last_name] )
68
+ expect(event["father"]["middle_names"] ).to eq(data_f[:middle_names] )
25
69
  end
70
+ end # it
71
+
72
+
73
+
74
+
75
+
76
+
77
+ #### Test case 3: Decoder test for enums #####################################################################################################################
78
+
79
+
80
+
81
+
82
+ let(:plugin_col) { LogStash::Codecs::Protobuf.new("class_name" => "ColourProtoTest", "include_path" => ['spec/helpers/ColourTestcase.pb.rb']) }
83
+ before do
84
+ plugin_col.register
85
+ end
86
+
87
+ it "should return an event from protobuf encoded data with enums" do
88
+
89
+ data = {:least_liked => ColourProtoTest::Colour::YELLOW, :favourite_colours => \
90
+ [ColourProtoTest::Colour::BLACK, ColourProtoTest::Colour::BLUE], :booleantest => [true, false, true]}
91
+ pb = ColourProtoTest.new(data)
92
+
93
+ plugin_col.decode(pb.serialize_to_string) do |event|
94
+ expect(event["least_liked"] ).to eq(data[:least_liked] )
95
+ expect(event["favourite_colours"] ).to eq(data[:favourite_colours] )
96
+ expect(event["booleantest"] ).to eq(data[:booleantest] )
97
+ end
98
+ end # it
99
+
100
+
101
+ end # context
102
+
103
+
104
+
105
+
106
+
107
+ #### Test case 4: Encode simple protobuf bytes for unicorn ####################################################################################################################
108
+
109
+ context "#encode" do
110
+ subject do
111
+ next LogStash::Codecs::Protobuf.new("class_name" => "Animal::UnicornEvent", "include_path" => ['spec/helpers/unicorn_event.pb.rb'])
112
+ end
113
+
114
+ event = LogStash::Event.new("colour" => "pink", "horn_length" => 12, "last_seen" => 1410081999, "has_wings" => true)
115
+
116
+ it "should return protobuf encoded data from a simple event" do
117
+ subject.on_event do |event, data|
118
+ insist { data.is_a? String }
119
+ unicorn = Animal::UnicornEvent.parse(data)
120
+
121
+ expect(unicorn.colour ).to eq(event["colour"] )
122
+ expect(unicorn.horn_length ).to eq(event["horn_length"] )
123
+ expect(unicorn.last_seen ).to eq(event["last_seen"] )
124
+ expect(unicorn.has_wings ).to eq(event["has_wings"] )
125
+
126
+ end # subject.on_event
127
+ subject.encode(event)
128
+ end # it
129
+ end # context
130
+
131
+
132
+
133
+
134
+ #### Test case 5: encode complex protobuf bytes for human #####################################################################################################################
135
+
136
+
137
+ context "#encode2" do
138
+ subject do
139
+ next LogStash::Codecs::Protobuf.new("class_name" => "Animal::Human", "include_path" => ['spec/helpers/human.pb.rb'])
26
140
  end
27
- end
141
+
142
+ event = LogStash::Event.new("first_name" => "Jimmy", "middle_names" => ["Bob", "James"], "last_name" => "Doe" \
143
+ , "mother" => {"first_name" => "Jane", "middle_names" => ["Elizabeth"], "last_name" => "Doe" , "age" => 83, "vegetarian"=> false} \
144
+ , "father" => {"first_name" => "John", "last_name" => "Doe", "@email" => "character_replacement_test@nothing" })
145
+
146
+ it "should return protobuf encoded data from a complex event" do
147
+
148
+ subject.on_event do |event, data|
149
+ insist { data.is_a? String }
150
+ jimmy = Animal::Human.parse(data)
151
+
152
+ expect(jimmy.first_name ).to eq(event["first_name"] )
153
+ expect(jimmy.middle_names ).to eq(event["middle_names"] )
154
+ expect(jimmy.last_name ).to eq(event["last_name"] )
155
+ expect(jimmy.mother.first_name ).to eq(event["mother"]["first_name"] )
156
+ expect(jimmy.father.first_name ).to eq(event["father"]["first_name"] )
157
+ expect(jimmy.mother.middle_names ).to eq(event["mother"]["middle_names"] )
158
+ expect(jimmy.mother.age ).to eq(event["mother"]["age"] ) # recursion test for values
159
+ expect(jimmy.mother.vegetarian ).to eq(event["mother"]["vegetarian"] ) # recursion test for values
160
+ expect(jimmy.father.last_name ).to eq(event["father"]["last_name"] )
161
+ expect(jimmy.father.email ).to eq(event["father"]["@email"] ) # recursion test for keys
162
+ expect(jimmy.mother.last_name ).to eq(event["mother"]["last_name"] )
163
+
164
+ end # subject.on_event
165
+ subject.encode(event)
166
+ end # it
167
+ end # context
168
+
169
+
170
+
171
+
172
+
173
+ #### Test case 6: encode enums #########################################################################################################################
174
+
175
+
176
+
177
+ context "#encode3" do
178
+ subject do
179
+ next LogStash::Codecs::Protobuf.new("class_name" => "ColourProtoTest", "include_path" => ['spec/helpers/ColourTestcase.pb.rb'])
180
+ end
181
+
182
+ require 'spec/helpers/ColourTestcase.pb.rb' # otherwise we cant use the colour enums in the next line
183
+ event = LogStash::Event.new("booleantest" => [false, false, true], "least_liked" => ColourProtoTest::Colour::YELLOW, "favourite_colours" => \
184
+ [ColourProtoTest::Colour::BLACK, ColourProtoTest::Colour::BLUE] )
185
+
186
+ it "should return protobuf encoded data from a complex event with enums" do
187
+
188
+ subject.on_event do |event, data|
189
+ insist { data.is_a? String }
190
+
191
+ colpref = ColourProtoTest.parse(data)
192
+
193
+ expect(colpref.booleantest ).to eq(event["booleantest"] )
194
+ expect(colpref.least_liked ).to eq(event["least_liked"] )
195
+ expect(colpref.favourite_colours ).to eq(event["favourite_colours"] )
196
+
197
+
198
+ end # subject.on_event
199
+ subject.encode(event)
200
+ end # it
201
+ end # context
202
+
203
+
204
+
28
205
  end
@@ -0,0 +1,35 @@
1
+ #!/usr/bin/env ruby
2
+ # Generated by the protocol buffer compiler. DO NOT EDIT!
3
+
4
+ require 'protocol_buffers'
5
+
6
+ # forward declarations
7
+ class ColourProtoTest < ::ProtocolBuffers::Message; end
8
+
9
+ class ColourProtoTest < ::ProtocolBuffers::Message
10
+ # forward declarations
11
+
12
+ # enums
13
+ module Colour
14
+ include ::ProtocolBuffers::Enum
15
+
16
+ set_fully_qualified_name "ColourProtoTest.Colour"
17
+
18
+ BLACK = 0
19
+ BLUE = 1
20
+ WHITE = 2
21
+ GREEN = 3
22
+ RED = 4
23
+ YELLOW = 5
24
+ AQUA = 6
25
+ end
26
+
27
+ set_fully_qualified_name "ColourProtoTest"
28
+
29
+ repeated ::ColourProtoTest::Colour, :favourite_colours, 1
30
+ repeated :bool, :booleantest, 2
31
+ optional ::ColourProtoTest::Colour, :least_liked, 3
32
+ optional :string, :timestamp, 4
33
+ optional :string, :version, 5
34
+ end
35
+
@@ -0,0 +1,24 @@
1
+
2
+
3
+ message ColourProtoTest {
4
+
5
+ enum Colour {
6
+ BLACK = 0;
7
+ BLUE = 1;
8
+ WHITE = 2;
9
+ GREEN = 3;
10
+ RED = 4;
11
+ YELLOW = 5;
12
+ AQUA = 6;
13
+
14
+ }
15
+
16
+ // most liked colours; test enums in arrays.
17
+ repeated Colour favourite_colours = 1;
18
+
19
+ // why not also test booleans in arrays while we're at it.
20
+ repeated bool booleantest = 2;
21
+
22
+ // least liked colour
23
+ optional Colour least_liked = 3;
24
+ }
@@ -0,0 +1,26 @@
1
+ #!/usr/bin/env ruby
2
+ # Generated by the protocol buffer compiler. DO NOT EDIT!
3
+
4
+ require 'protocol_buffers'
5
+
6
+ module Animal
7
+ # forward declarations
8
+ class Human < ::ProtocolBuffers::Message; end
9
+
10
+ class Human < ::ProtocolBuffers::Message
11
+ set_fully_qualified_name "animal.Human"
12
+
13
+ optional :string, :first_name, 1
14
+ repeated :string, :middle_names, 2
15
+ optional :string, :last_name, 3
16
+ optional ::Animal::Human, :mother, 4
17
+ optional ::Animal::Human, :father, 5
18
+ optional :string, :path, 6
19
+ optional :string, :version, 7
20
+ optional :string, :timestamp, 8
21
+ optional :string, :email, 9
22
+ optional :bool, :vegetarian, 10
23
+ optional :int32, :age, 11
24
+ end
25
+
26
+ end
@@ -13,6 +13,7 @@ module Animal
13
13
  optional :string, :colour, 1
14
14
  optional :int32, :horn_length, 2
15
15
  optional :int32, :last_seen, 3
16
+ optional :bool, :has_wings, 4
16
17
  end
17
18
 
18
19
  end
@@ -0,0 +1,24 @@
1
+ #!/usr/bin/env ruby
2
+ # Generated by the protocol buffer compiler. DO NOT EDIT!
3
+
4
+ require 'protocol_buffers'
5
+
6
+ module Animal
7
+ # forward declarations
8
+ class UnicornEvent < ::ProtocolBuffers::Message; end
9
+
10
+ class UnicornEvent < ::ProtocolBuffers::Message
11
+ set_fully_qualified_name "animal.UnicornEvent"
12
+
13
+ optional :string, :colour, 1
14
+ optional :int32, :horn_length, 2
15
+ optional :int32, :last_seen, 3
16
+ optional :string, :timestamp, 4
17
+ optional :string, :host, 5
18
+ optional :string, :path, 6
19
+ optional :string, :version, 7
20
+ optional :bool, :has_wings, 8
21
+
22
+ end
23
+
24
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-codec-protobuf
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Inga Feick
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-02-15 00:00:00.000000000 Z
11
+ date: 2016-04-22 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: logstash-core
@@ -74,7 +74,11 @@ files:
74
74
  - lib/logstash/codecs/protobuf.rb
75
75
  - logstash-codec-protobuf.gemspec
76
76
  - spec/codecs/protobuf_spec.rb
77
+ - spec/helpers/ColourTestcase.pb.rb
78
+ - spec/helpers/ColourTestcase.proto
79
+ - spec/helpers/human.pb.rb
77
80
  - spec/helpers/unicorn.pb.rb
81
+ - spec/helpers/unicorn_event.pb.rb
78
82
  homepage:
79
83
  licenses:
80
84
  - Apache License (2.0)
@@ -103,4 +107,8 @@ specification_version: 4
103
107
  summary: This codec may be used to decode (via inputs) and encode (via outputs) protobuf messages
104
108
  test_files:
105
109
  - spec/codecs/protobuf_spec.rb
110
+ - spec/helpers/ColourTestcase.pb.rb
111
+ - spec/helpers/ColourTestcase.proto
112
+ - spec/helpers/human.pb.rb
106
113
  - spec/helpers/unicorn.pb.rb
114
+ - spec/helpers/unicorn_event.pb.rb