fluent-plugin-grok-parser 0.0.2 → 0.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.travis.yml +7 -0
- data/README.md +82 -0
- data/fluent-plugin-grok-parser.gemspec +4 -3
- data/lib/fluent/plugin/grok.rb +84 -0
- data/lib/fluent/plugin/parser_grok.rb +15 -63
- data/lib/fluent/plugin/parser_multiline_grok.rb +48 -0
- data/test/helper.rb +18 -0
- data/test/test_grok_parser.rb +19 -17
- data/test/test_multiline_grok_parser.rb +60 -0
- metadata +26 -6
- data/Gemfile.lock +0 -40
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 74360bb48589acb735b21ef7fc33da8e1e8ea1d5
|
4
|
+
data.tar.gz: 6f0b32cff96d5f45fbea166e819132f262c204da
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: af2ee3177353159e23af8d67561294c30b3ac97f7c99ad92e197670689a836c7b029568bf48450782536bc3f59c0a5ab59300a9e4fba11acfb2c29528a669c0c
|
7
|
+
data.tar.gz: 812f3f366210ceda6deec4af94c4da8cc4e401d664cacd6a6e3927b43f55ec30ce6b60be46ef7e8a035f2890c43a59f1b7e54b6492a0d8135a8eab77576368b1
|
data/.travis.yml
ADDED
data/README.md
CHANGED
@@ -44,6 +44,40 @@ extracts the first IP address that matches in the log.
|
|
44
44
|
</source>
|
45
45
|
```
|
46
46
|
|
47
|
+
### Multiline support
|
48
|
+
|
49
|
+
You can parse multiple line text.
|
50
|
+
|
51
|
+
```aconf
|
52
|
+
<source>
|
53
|
+
type tail
|
54
|
+
path /path/to/log
|
55
|
+
format multiline_grok
|
56
|
+
grok_pattern %{IP:ip_address}\n%{GREEDYDATA:message}
|
57
|
+
multiline_start_regex /^\s/
|
58
|
+
tag grokked_log
|
59
|
+
</source>
|
60
|
+
```
|
61
|
+
|
62
|
+
You can use multiple grok patterns to parse your data.
|
63
|
+
|
64
|
+
```aconf
|
65
|
+
<source>
|
66
|
+
type tail
|
67
|
+
path /path/to/log
|
68
|
+
format multiline_grok
|
69
|
+
<grok>
|
70
|
+
pattern Started %{WORD:verb} "%{URIPATH:pathinfo}" for %{IP:ip} at %{TIMESTAMP_ISO8601:timestamp}\nProcessing by %{WORD:controller}#%{WORD:action} as %{WORD:format}%{DATA:message}Completed %{NUMBER:response} %{WORD} in %{NUMBER:elapsed} (%{DATA:elapsed_details})
|
71
|
+
</grok>
|
72
|
+
tag grokked_log
|
73
|
+
</source>
|
74
|
+
```
|
75
|
+
|
76
|
+
Fluentd accumulates data in the buffer forever to parse complete data when no pattern matches.
|
77
|
+
|
78
|
+
You can use this parser without `multiline_start_regex` when you know your data structure perfectly.
|
79
|
+
|
80
|
+
|
47
81
|
## How to write Grok patterns
|
48
82
|
|
49
83
|
Grok patterns look like `%{PATTERN_NAME:name}` where ":name" is optional. If "name" is provided, then it
|
@@ -80,6 +114,54 @@ This is what the `custom_pattern_path` parameter is for.
|
|
80
114
|
|
81
115
|
`custom_pattern_path` can be either a directory or file. If it's a directory, it reads all the files in it.
|
82
116
|
|
117
|
+
## FAQs
|
118
|
+
|
119
|
+
### 1. How can I convert types of the matched patterns like Logstash's Grok?
|
120
|
+
|
121
|
+
Although every parsed field has type `string` by default, you can specify other types. This is useful when filtering particular fields numerically or storing data with sensible type information.
|
122
|
+
|
123
|
+
The syntax is
|
124
|
+
|
125
|
+
```
|
126
|
+
types <field_name_1>:<type_name_1>,<field_name_2>:<type_name_2>,...
|
127
|
+
```
|
128
|
+
|
129
|
+
e.g.,
|
130
|
+
|
131
|
+
```
|
132
|
+
types user_id:integer,paid:bool,paid_usd_amount:float
|
133
|
+
```
|
134
|
+
|
135
|
+
As demonstrated above, "," is used to delimit field-type pairs while ":" is used to separate a field name with its intended type.
|
136
|
+
|
137
|
+
Unspecified fields are parsed at the default string type.
|
138
|
+
|
139
|
+
The list of supported types are shown below:
|
140
|
+
|
141
|
+
* `string`
|
142
|
+
* `bool`
|
143
|
+
* `integer` ("int" would NOT work!)
|
144
|
+
* `float`
|
145
|
+
* `time`
|
146
|
+
* `array`
|
147
|
+
|
148
|
+
For the `time` and `array` types, there is an optional third field after the type name. For the "time" type, you can specify a time format like you would in `time_format`.
|
149
|
+
|
150
|
+
For the "array" type, the third field specifies the delimiter (the default is ","). For example, if a field called "item\_ids" contains the value "3,4,5", `types item_ids:array` parses it as ["3", "4", "5"]. Alternatively, if the value is "Adam|Alice|Bob", `types item_ids:array:|` parses it as ["Adam", "Alice", "Bob"].
|
151
|
+
|
152
|
+
Here is a sample config using the Grok parser with `in_tail` and the `types` parameter:
|
153
|
+
|
154
|
+
```aconf
|
155
|
+
<source>
|
156
|
+
type tail
|
157
|
+
path /path/to/log
|
158
|
+
format grok
|
159
|
+
grok_pattern %{INT:user_id} paid %{NUMBER:paid_amount}
|
160
|
+
types user_id:integer,paid_amount:float
|
161
|
+
tag payment
|
162
|
+
</source>
|
163
|
+
```
|
164
|
+
|
83
165
|
## License
|
84
166
|
|
85
167
|
Apache 2.0 License
|
@@ -4,12 +4,12 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
4
4
|
|
5
5
|
Gem::Specification.new do |spec|
|
6
6
|
spec.name = "fluent-plugin-grok-parser"
|
7
|
-
spec.version = "0.0.
|
7
|
+
spec.version = "0.0.3"
|
8
8
|
spec.authors = ["kiyoto"]
|
9
9
|
spec.email = ["kiyoto@treasure-data.com"]
|
10
|
-
spec.summary = %q{Fluentd plugin to
|
10
|
+
spec.summary = %q{Fluentd plugin to support Logstash-inspired Grok format for parsing logs}
|
11
11
|
spec.homepage = "https://github.com/kiyoto/fluent-plugin-grok-parser"
|
12
|
-
spec.license = "Apache
|
12
|
+
spec.license = "Apache-2.0"
|
13
13
|
|
14
14
|
spec.files = `git ls-files`.split($/)
|
15
15
|
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
@@ -18,5 +18,6 @@ Gem::Specification.new do |spec|
|
|
18
18
|
|
19
19
|
spec.add_development_dependency "bundler"
|
20
20
|
spec.add_development_dependency "rake"
|
21
|
+
spec.add_development_dependency "test-unit", ">=3.1.5"
|
21
22
|
spec.add_runtime_dependency "fluentd", ">=0.10.58"
|
22
23
|
end
|
@@ -0,0 +1,84 @@
|
|
1
|
+
module Fluent
|
2
|
+
class Grok
|
3
|
+
class GrokPatternNotFoundError < StandardError
|
4
|
+
end
|
5
|
+
|
6
|
+
# Much of the Grok implementation is based on Jordan Sissel's jls-grok
|
7
|
+
# See https://github.com/jordansissel/ruby-grok/blob/master/lib/grok-pure.rb
|
8
|
+
PATTERN_RE = \
|
9
|
+
/%\{ # match '%{' not prefixed with '\'
|
10
|
+
(?<name> # match the pattern name
|
11
|
+
(?<pattern>[A-z0-9]+)
|
12
|
+
(?::(?<subname>[@\[\]A-z0-9_:.-]+))?
|
13
|
+
)
|
14
|
+
\}/x
|
15
|
+
|
16
|
+
attr_reader :parsers
|
17
|
+
|
18
|
+
def initialize(plugin, conf)
|
19
|
+
@pattern_map = {}
|
20
|
+
@parsers = []
|
21
|
+
@multiline_mode = false
|
22
|
+
@conf = conf
|
23
|
+
if plugin.instance_of?(Fluent::TextParser::MultilineGrokParser)
|
24
|
+
@multiline_mode = true
|
25
|
+
end
|
26
|
+
if @conf['multiline_start_regexp']
|
27
|
+
@multiline_start_regexp = Regexp.compile(@conf['multiline_start_regexp'][1..-2])
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
def add_patterns_from_file(path)
|
32
|
+
File.new(path).each_line do |line|
|
33
|
+
next if line[0] == '#' || /^$/ =~ line
|
34
|
+
name, pat = line.chomp.split(/\s+/, 2)
|
35
|
+
@pattern_map[name] = pat
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
def setup
|
40
|
+
if @conf['grok_pattern']
|
41
|
+
@parsers << expand_pattern_expression(@conf['grok_pattern'], @conf)
|
42
|
+
else
|
43
|
+
grok_confs = @conf.elements.select {|e| e.name == 'grok'}
|
44
|
+
grok_confs.each do |grok_conf|
|
45
|
+
@parsers << expand_pattern_expression(grok_conf['pattern'], grok_conf)
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
private
|
51
|
+
|
52
|
+
def expand_pattern_expression(grok_pattern, conf)
|
53
|
+
regexp = expand_pattern(grok_pattern)
|
54
|
+
$log.info "Expanded the pattern #{conf['grok_pattern']} into #{regexp}"
|
55
|
+
options = nil
|
56
|
+
if @multiline_mode
|
57
|
+
options = Regexp::MULTILINE
|
58
|
+
end
|
59
|
+
TextParser::RegexpParser.new(Regexp.new(regexp, options), conf)
|
60
|
+
rescue GrokPatternNotFoundError => e
|
61
|
+
raise e
|
62
|
+
rescue => e
|
63
|
+
$log.error e.backtrace.join("\n")
|
64
|
+
end
|
65
|
+
|
66
|
+
def expand_pattern(pattern)
|
67
|
+
# It's okay to modify in place. no need to expand it more than once.
|
68
|
+
while true
|
69
|
+
m = PATTERN_RE.match(pattern)
|
70
|
+
break unless m
|
71
|
+
curr_pattern = @pattern_map[m["pattern"]]
|
72
|
+
raise GrokPatternNotFoundError unless curr_pattern
|
73
|
+
replacement_pattern = if m["subname"]
|
74
|
+
"(?<#{m["subname"]}>#{curr_pattern})"
|
75
|
+
else
|
76
|
+
curr_pattern
|
77
|
+
end
|
78
|
+
pattern.sub!(m[0]) do |s| replacement_pattern end
|
79
|
+
end
|
80
|
+
|
81
|
+
pattern
|
82
|
+
end
|
83
|
+
end
|
84
|
+
end
|
@@ -1,3 +1,5 @@
|
|
1
|
+
require "fluent/plugin/grok"
|
2
|
+
|
1
3
|
module Fluent
|
2
4
|
class TextParser
|
3
5
|
class GrokPatternNotFoundError < Exception; end
|
@@ -8,87 +10,37 @@ module Fluent
|
|
8
10
|
config_param :grok_pattern, :string, :default => nil
|
9
11
|
config_param :custom_pattern_path, :string, :default => nil
|
10
12
|
|
11
|
-
# Much of the Grok implementation is based on Jordan Sissel's jls-grok
|
12
|
-
# See https://github.com/jordansissel/ruby-grok/blob/master/lib/grok-pure.rb
|
13
|
-
PATTERN_RE = \
|
14
|
-
/%\{ # match '%{' not prefixed with '\'
|
15
|
-
(?<name> # match the pattern name
|
16
|
-
(?<pattern>[A-z0-9]+)
|
17
|
-
(?::(?<subname>[@\[\]A-z0-9_:.-]+))?
|
18
|
-
)
|
19
|
-
\}/x
|
20
|
-
|
21
13
|
def initialize
|
22
14
|
super
|
23
|
-
@pattern_map = {}
|
24
|
-
default_pattern_dir = File.expand_path('../../../../patterns/*', __FILE__)
|
25
|
-
Dir.glob(default_pattern_dir) do |pattern_file_path|
|
26
|
-
add_patterns_from_file(pattern_file_path)
|
27
|
-
end
|
28
15
|
@default_parser = NoneParser.new
|
29
|
-
@parsers = []
|
30
16
|
end
|
31
17
|
|
32
18
|
def configure(conf={})
|
33
19
|
super
|
34
20
|
|
21
|
+
@grok = Grok.new(self, conf)
|
22
|
+
|
23
|
+
default_pattern_dir = File.expand_path('../../../../patterns/*', __FILE__)
|
24
|
+
Dir.glob(default_pattern_dir) do |pattern_file_path|
|
25
|
+
@grok.add_patterns_from_file(pattern_file_path)
|
26
|
+
end
|
27
|
+
|
35
28
|
if @custom_pattern_path
|
36
29
|
if Dir.exists? @custom_pattern_path
|
37
30
|
Dir.glob(@custom_pattern_path + '/*') do |pattern_file_path|
|
38
|
-
add_patterns_from_file(pattern_file_path)
|
31
|
+
@grok.add_patterns_from_file(pattern_file_path)
|
39
32
|
end
|
40
33
|
elsif File.exists? @custom_pattern_path
|
41
|
-
add_patterns_from_file(@custom_pattern_path)
|
34
|
+
@grok.add_patterns_from_file(@custom_pattern_path)
|
42
35
|
end
|
43
36
|
end
|
44
37
|
|
45
|
-
|
46
|
-
@parsers = [expand_pattern_exn(@grok_pattern, conf)]
|
47
|
-
else
|
48
|
-
grok_confs = conf.elements.select {|e| e.name == 'grok'}
|
49
|
-
grok_confs.each do |grok_conf|
|
50
|
-
@parsers << expand_pattern_exn(grok_conf['pattern'], grok_conf)
|
51
|
-
end
|
52
|
-
end
|
53
|
-
end
|
54
|
-
|
55
|
-
def add_patterns_from_file(path)
|
56
|
-
File.new(path).each_line do |line|
|
57
|
-
next if line[0] == '#' || /^$/ =~ line
|
58
|
-
name, pat = line.chomp.split(/\s+/, 2)
|
59
|
-
@pattern_map[name] = pat
|
60
|
-
end
|
61
|
-
end
|
62
|
-
|
63
|
-
def expand_pattern_exn(pattern, conf)
|
64
|
-
regexp = expand_pattern(pattern)
|
65
|
-
$log.info "Expanded the pattern #{conf['grok_pattern']} into #{regexp}"
|
66
|
-
RegexpParser.new(Regexp.new(regexp), conf)
|
67
|
-
rescue => e
|
68
|
-
$log.error e.backtrace.join("\n")
|
69
|
-
end
|
70
|
-
|
71
|
-
def expand_pattern(pattern)
|
72
|
-
# It's okay to modify in place. no need to expand it more than once.
|
73
|
-
while true
|
74
|
-
m = PATTERN_RE.match(pattern)
|
75
|
-
break if not m
|
76
|
-
curr_pattern = @pattern_map[m["pattern"]]
|
77
|
-
raise GrokPatternNotFoundError if not curr_pattern
|
78
|
-
replacement_pattern = if m["subname"]
|
79
|
-
"(?<#{m["subname"]}>#{curr_pattern})"
|
80
|
-
else
|
81
|
-
curr_pattern
|
82
|
-
end
|
83
|
-
pattern.sub!(m[0]) do |s| replacement_pattern end
|
84
|
-
end
|
85
|
-
|
86
|
-
pattern
|
38
|
+
@grok.setup
|
87
39
|
end
|
88
40
|
|
89
|
-
def parse(text)
|
41
|
+
def parse(text, &block)
|
90
42
|
if block_given?
|
91
|
-
@parsers.each do |parser|
|
43
|
+
@grok.parsers.each do |parser|
|
92
44
|
parser.parse(text) do |time, record|
|
93
45
|
if time and record
|
94
46
|
yield time, record
|
@@ -98,7 +50,7 @@ module Fluent
|
|
98
50
|
end
|
99
51
|
yield @default_parser.parse(text)
|
100
52
|
else
|
101
|
-
@parsers.each do |parser|
|
53
|
+
@grok.parsers.each do |parser|
|
102
54
|
parser.parse(text) do |time, record|
|
103
55
|
if time and record
|
104
56
|
return time, record
|
@@ -0,0 +1,48 @@
|
|
1
|
+
require 'fluent/plugin/parser_grok'
|
2
|
+
|
3
|
+
module Fluent
|
4
|
+
class TextParser
|
5
|
+
class MultilineGrokParser < GrokParser
|
6
|
+
Plugin.register_parser('multiline_grok', self)
|
7
|
+
config_param :multiline_start_regexp, :string, :default => nil
|
8
|
+
|
9
|
+
|
10
|
+
def initialize
|
11
|
+
super
|
12
|
+
end
|
13
|
+
|
14
|
+
def configure(conf={})
|
15
|
+
super
|
16
|
+
end
|
17
|
+
|
18
|
+
def has_firstline?
|
19
|
+
!!@multiline_start_regexp
|
20
|
+
end
|
21
|
+
|
22
|
+
def firstline?(text)
|
23
|
+
@multiline_start_regexp && !@multiline_start_regexp.match(text)
|
24
|
+
end
|
25
|
+
|
26
|
+
def parse(text, &block)
|
27
|
+
if block_given?
|
28
|
+
@grok.parsers.each do |parser|
|
29
|
+
parser.parse(text) do |time, record|
|
30
|
+
if time and record
|
31
|
+
yield time, record
|
32
|
+
return
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
36
|
+
else
|
37
|
+
@grok.parsers.each do |parser|
|
38
|
+
parser.parse(text) do |time, record|
|
39
|
+
if time and record
|
40
|
+
return time, record
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
data/test/helper.rb
ADDED
data/test/test_grok_parser.rb
CHANGED
@@ -15,22 +15,6 @@ def str2time(str_time, format = nil)
|
|
15
15
|
end
|
16
16
|
|
17
17
|
class GrokParserTest < ::Test::Unit::TestCase
|
18
|
-
def internal_test_grok_pattern(grok_pattern, text, expected_time, expected_record, options = {})
|
19
|
-
parser = TextParser::GrokParser.new
|
20
|
-
parser.configure({"grok_pattern" => grok_pattern}.merge(options))
|
21
|
-
|
22
|
-
# for the old, return based API
|
23
|
-
time, record = parser.parse(text)
|
24
|
-
assert_equal(expected_time, time) if expected_time
|
25
|
-
assert_equal(expected_record, record)
|
26
|
-
|
27
|
-
# for the new API
|
28
|
-
parser.parse(text) {|time, record|
|
29
|
-
assert_equal(expected_time, time) if expected_time
|
30
|
-
assert_equal(expected_record, record)
|
31
|
-
}
|
32
|
-
end
|
33
|
-
|
34
18
|
def test_call_for_timestamp
|
35
19
|
internal_test_grok_pattern('%{TIMESTAMP_ISO8601:time}', 'Some stuff at 2014-01-01T00:00:00+0900',
|
36
20
|
str2time('2014-01-01T00:00:00+0900'), {})
|
@@ -43,7 +27,7 @@ class GrokParserTest < ::Test::Unit::TestCase
|
|
43
27
|
end
|
44
28
|
|
45
29
|
def test_call_for_grok_pattern_not_found
|
46
|
-
assert_raise
|
30
|
+
assert_raise Grok::GrokPatternNotFoundError do
|
47
31
|
internal_test_grok_pattern('%{THIS_PATTERN_DOESNT_EXIST}', 'Some stuff at somewhere', nil, {})
|
48
32
|
end
|
49
33
|
end
|
@@ -85,4 +69,22 @@ class GrokParserTest < ::Test::Unit::TestCase
|
|
85
69
|
File.delete(pattern_file.path)
|
86
70
|
end
|
87
71
|
end
|
72
|
+
|
73
|
+
private
|
74
|
+
|
75
|
+
def internal_test_grok_pattern(grok_pattern, text, expected_time, expected_record, options = {})
|
76
|
+
parser = TextParser::GrokParser.new
|
77
|
+
parser.configure(Config::Element.new('ROOT', '', {"grok_pattern" => grok_pattern}.merge(options), []))
|
78
|
+
|
79
|
+
# for the old, return based API
|
80
|
+
time, record = parser.parse(text)
|
81
|
+
assert_equal(expected_time, time) if expected_time
|
82
|
+
assert_equal(expected_record, record)
|
83
|
+
|
84
|
+
# for the new API
|
85
|
+
parser.parse(text) {|time, record|
|
86
|
+
assert_equal(expected_time, time) if expected_time
|
87
|
+
assert_equal(expected_record, record)
|
88
|
+
}
|
89
|
+
end
|
88
90
|
end
|
@@ -0,0 +1,60 @@
|
|
1
|
+
require 'fluent/plugin/parser_multiline_grok'
|
2
|
+
require 'fluent/config/parser'
|
3
|
+
|
4
|
+
require 'stringio'
|
5
|
+
|
6
|
+
class MultilineGrokParserTest < Test::Unit::TestCase
|
7
|
+
def test_multiline
|
8
|
+
text=<<TEXT.chomp
|
9
|
+
host1 message1
|
10
|
+
message2
|
11
|
+
message3
|
12
|
+
TEXT
|
13
|
+
message =<<MESSAGE.chomp
|
14
|
+
message1
|
15
|
+
message2
|
16
|
+
message3
|
17
|
+
MESSAGE
|
18
|
+
conf = %[
|
19
|
+
grok_pattern %{HOSTNAME:hostname} %{GREEDYDATA:message}
|
20
|
+
multiline_start_regexp /^\s/
|
21
|
+
]
|
22
|
+
parser = create_parser(conf)
|
23
|
+
|
24
|
+
parser.parse(text) do |time, record|
|
25
|
+
assert_equal({ "hostname" => "host1", "message" => message }, record)
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
def test_without_multiline_start_regexp
|
30
|
+
text = <<TEXT.chomp
|
31
|
+
host1 message1
|
32
|
+
message2
|
33
|
+
message3
|
34
|
+
end
|
35
|
+
TEXT
|
36
|
+
conf = %[
|
37
|
+
grok_pattern %{HOSTNAME:hostname} %{DATA:message1}\\n %{DATA:message2}\\n %{DATA:message3}\\nend
|
38
|
+
]
|
39
|
+
parser = create_parser(conf)
|
40
|
+
|
41
|
+
expected = {
|
42
|
+
"hostname" => "host1",
|
43
|
+
"message1" => "message1",
|
44
|
+
"message2" => "message2",
|
45
|
+
"message3" => "message3"
|
46
|
+
}
|
47
|
+
parser.parse(text) do |time, record|
|
48
|
+
assert_equal(expected, record)
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
private
|
53
|
+
|
54
|
+
def create_parser(conf)
|
55
|
+
parser = TextParser::MultilineGrokParser.new
|
56
|
+
io = StringIO.new(conf)
|
57
|
+
parser.configure(Config::Parser.parse(io, "fluent.conf"))
|
58
|
+
parser
|
59
|
+
end
|
60
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: fluent-plugin-grok-parser
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- kiyoto
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-
|
11
|
+
date: 2015-11-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -38,6 +38,20 @@ dependencies:
|
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
40
|
version: '0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: test-unit
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: 3.1.5
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: 3.1.5
|
41
55
|
- !ruby/object:Gem::Dependency
|
42
56
|
name: fluentd
|
43
57
|
requirement: !ruby/object:Gem::Requirement
|
@@ -59,20 +73,24 @@ executables: []
|
|
59
73
|
extensions: []
|
60
74
|
extra_rdoc_files: []
|
61
75
|
files:
|
76
|
+
- ".travis.yml"
|
62
77
|
- Gemfile
|
63
|
-
- Gemfile.lock
|
64
78
|
- LICENSE
|
65
79
|
- README.md
|
66
80
|
- Rakefile
|
67
81
|
- fluent-plugin-grok-parser.gemspec
|
82
|
+
- lib/fluent/plugin/grok.rb
|
68
83
|
- lib/fluent/plugin/parser_grok.rb
|
84
|
+
- lib/fluent/plugin/parser_multiline_grok.rb
|
69
85
|
- patterns/grok-patterns
|
70
86
|
- patterns/nagios
|
87
|
+
- test/helper.rb
|
71
88
|
- test/test_grok_parser.rb
|
72
89
|
- test/test_grok_parser_in_tcp.rb
|
90
|
+
- test/test_multiline_grok_parser.rb
|
73
91
|
homepage: https://github.com/kiyoto/fluent-plugin-grok-parser
|
74
92
|
licenses:
|
75
|
-
- Apache
|
93
|
+
- Apache-2.0
|
76
94
|
metadata: {}
|
77
95
|
post_install_message:
|
78
96
|
rdoc_options: []
|
@@ -90,10 +108,12 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
90
108
|
version: '0'
|
91
109
|
requirements: []
|
92
110
|
rubyforge_project:
|
93
|
-
rubygems_version: 2.
|
111
|
+
rubygems_version: 2.4.5.1
|
94
112
|
signing_key:
|
95
113
|
specification_version: 4
|
96
|
-
summary: Fluentd plugin to
|
114
|
+
summary: Fluentd plugin to support Logstash-inspired Grok format for parsing logs
|
97
115
|
test_files:
|
116
|
+
- test/helper.rb
|
98
117
|
- test/test_grok_parser.rb
|
99
118
|
- test/test_grok_parser_in_tcp.rb
|
119
|
+
- test/test_multiline_grok_parser.rb
|
data/Gemfile.lock
DELETED
@@ -1,40 +0,0 @@
|
|
1
|
-
PATH
|
2
|
-
remote: .
|
3
|
-
specs:
|
4
|
-
fluent-plugin-grok-parser (0.0.2)
|
5
|
-
fluentd (>= 0.10.58)
|
6
|
-
|
7
|
-
GEM
|
8
|
-
remote: https://rubygems.org/
|
9
|
-
specs:
|
10
|
-
cool.io (1.2.4)
|
11
|
-
fluentd (0.12.1)
|
12
|
-
cool.io (>= 1.2.2, < 2.0.0)
|
13
|
-
http_parser.rb (>= 0.5.1, < 0.7.0)
|
14
|
-
json (>= 1.4.3)
|
15
|
-
msgpack (>= 0.5.4, < 0.6.0)
|
16
|
-
sigdump (~> 0.2.2)
|
17
|
-
string-scrub (>= 0.0.3)
|
18
|
-
tzinfo (>= 1.0.0)
|
19
|
-
tzinfo-data (>= 1.0.0)
|
20
|
-
yajl-ruby (~> 1.0)
|
21
|
-
http_parser.rb (0.6.0)
|
22
|
-
json (1.8.1)
|
23
|
-
msgpack (0.5.9)
|
24
|
-
rake (10.1.1)
|
25
|
-
sigdump (0.2.2)
|
26
|
-
string-scrub (0.0.5)
|
27
|
-
thread_safe (0.3.4)
|
28
|
-
tzinfo (1.2.2)
|
29
|
-
thread_safe (~> 0.1)
|
30
|
-
tzinfo-data (1.2014.10)
|
31
|
-
tzinfo (>= 1.0.0)
|
32
|
-
yajl-ruby (1.2.1)
|
33
|
-
|
34
|
-
PLATFORMS
|
35
|
-
ruby
|
36
|
-
|
37
|
-
DEPENDENCIES
|
38
|
-
bundler
|
39
|
-
fluent-plugin-grok-parser!
|
40
|
-
rake
|