logstash-filter-dateparts 2.0.1 → 2.1.1

This diff has not been reviewed by any users.
Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: a8b480afd89318ea8bb3176bdec0b1cd9fbf3cd3
4
- data.tar.gz: cd5302e4a35e5f4bd1b33b25759605bf4f288d77
3
+ metadata.gz: 5e19d793f2e58398b43d3a02e3cb752627117530
4
+ data.tar.gz: 010f207477326df140e9805693d13cd81f3a3b84
5
5
  SHA512:
6
- metadata.gz: 5c3296b74768a007fc973faa9a428e213ecb627958e82342105e6a7a6fdd107c2edc5099b0e3f0c70133ed4819f14585496fb50fadec8234c9432105e5a5e650
7
- data.tar.gz: a20d130798618bb3c5c0758e93615e1d60c4545edfdf54ce23a9c303625f127d019d1f5ad5d02f07cbf74628a319b360d7ae9a8c95c3b91a1f2d6fd9133ceac6
6
+ metadata.gz: 02946b25be55a6dafcc09037a747376b663360c77fb660473d0f08749285a5571b8e3a46852c0df106375e9562defc1acd75f146c7552a90582bd2b59fcb929d
7
+ data.tar.gz: 4595e1fac4bf60184c1abe2b6971df9d83b4c9358f91f84e8b8b76a3c43fb048864172887301e52a3713c9de08b495767c1c11dfacc71c769629f762a963fd5e
@@ -1,5 +1,14 @@
1
- # 2.0.0
2
- - Breaking: Updated plugin to use new Java Event APIs
3
-
4
- # 1.0.1
5
- - Updated dependencies to avoid being tied to Logstash major release.
1
+ # 2.1.1
2
+ - Added support for caluclating a duration - https://github.com/mikebski/logstash-filter-datepart/issues/4
3
+ - Added support for DateTime and other objects with a to_date method
4
+ - Added TravisCI build hook
5
+
6
+ # 2.0.1
7
+ - Removed extraneous output statement from
8
+ https://github.com/mikebski/logstash-filter-datepart/issues/3
9
+
10
+ # 2.0.0
11
+ - Breaking: Updated plugin to use new Java Event APIs
12
+
13
+ # 1.0.1
14
+ - Updated dependencies to avoid being tied to Logstash major release.
data/Gemfile CHANGED
@@ -1,2 +1,2 @@
1
- source 'https://rubygems.org'
2
- gemspec
1
+ source 'https://rubygems.org'
2
+ gemspec
data/LICENSE CHANGED
@@ -1,13 +1,13 @@
1
- Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
2
-
3
- Licensed under the Apache License, Version 2.0 (the "License");
4
- you may not use this file except in compliance with the License.
5
- You may obtain a copy of the License at
6
-
7
- http://www.apache.org/licenses/LICENSE-2.0
8
-
9
- Unless required by applicable law or agreed to in writing, software
10
- distributed under the License is distributed on an "AS IS" BASIS,
11
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
- See the License for the specific language governing permissions and
13
- limitations under the License.
1
+ Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
2
+
3
+ Licensed under the Apache License, Version 2.0 (the "License");
4
+ you may not use this file except in compliance with the License.
5
+ You may obtain a copy of the License at
6
+
7
+ http://www.apache.org/licenses/LICENSE-2.0
8
+
9
+ Unless required by applicable law or agreed to in writing, software
10
+ distributed under the License is distributed on an "AS IS" BASIS,
11
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
+ See the License for the specific language governing permissions and
13
+ limitations under the License.
data/README.md CHANGED
@@ -1,119 +1,137 @@
1
- # Logstash Plugin
2
-
3
- This is a plugin for [Logstash](https://github.com/elastic/logstash).
4
-
5
- The source for this plugin can be [found here on github](https://github.com/mikebski/logstash-datepart-plugin.git)
6
-
7
- Author: Mike Baranski (mike.baranski@gmail.com). Contributions are welcome.
8
-
9
- [![Gem Version](https://badge.fury.io/rb/logstash-filter-dateparts.svg)](https://badge.fury.io/rb/logstash-filter-dateparts)
10
-
11
- ## License ##
12
-
13
- Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
14
-
15
- Licensed under the Apache License, Version 2.0 (the "License");
16
- you may not use this file except in compliance with the License.
17
- You may obtain a copy of the License at
18
-
19
- http://www.apache.org/licenses/LICENSE-2.0
20
-
21
- Unless required by applicable law or agreed to in writing, software
22
- distributed under the License is distributed on an "AS IS" BASIS,
23
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
24
- See the License for the specific language governing permissions and
25
- limitations under the License.
26
-
27
- ## About
28
-
29
- This plugin is useful if you want to easily query Logstash data on *day of week*, *hour of day*, or other parts of a date. See the usage below for details on the output of the plugin. The date parts that can be generated are:
30
-
31
- * day
32
- * wday
33
- * yday
34
- * month
35
- * year
36
- * hour
37
- * min
38
- * sec
39
-
40
- ## Documentation
41
-
42
- ### Installation
43
-
44
- To manually install the plugin, download the gem and run:
45
-
46
- `bin/plugin install --no-verify logstash-filter-dateparts-1.0.0.gem`
47
-
48
- ### Usage
49
-
50
- To see the most basic usage, you can run the following (on Linux):
51
-
52
- `echo "HI" | bin/logstash -e 'input { stdin {} } filter {dateparts { }} output { stdout { codec=> rubydebug}}'`
53
-
54
- You could also use the logstash generator:
55
-
56
- `bin/logstash -e 'input { generator { lines => ["HI"] count => 1 } } filter {dateparts { }} output { stdout { codec=> rubydebug}}'`
57
-
58
- Here is the sample output:
59
-
60
- {
61
- "message" => "HI",
62
- "@version" => "1",
63
- "@timestamp" => "2015-11-20T12:24:40.217Z",
64
- "host" => "mike-VirtualBox",
65
- "day" => 20,
66
- "wday" => 5,
67
- "yday" => 324,
68
- "month" => 11,
69
- "year" => 2015,
70
- "hour" => 12,
71
- "min" => 24,
72
- "sec" => 40
73
- }
74
-
75
-
76
- This uses the default configuration, which generates the following fields from the `@timestamp` field of the event:
77
-
78
- * day
79
- * wday
80
- * yday
81
- * month
82
- * year
83
- * hour
84
- * min
85
- * sec
86
-
87
- ### Configuration
88
-
89
- #### Fields
90
-
91
- The generated fields are based on the date functions available in the [Ruby time class](http://ruby-doc.org/core-2.2.0/Time.html). You can specify any valid function and it will be added to the event.
92
-
93
- For example, this will add 2 fields, *sec* corresponding to `time.sec()` and *hour* corresponding to `time.hour()`:
94
-
95
- filter {
96
- dateparts {
97
- "fields" => ["sec", "hour"]
98
- }
99
- }
100
-
101
- #### Time Field
102
-
103
- By default, the plugin will use the *@timestamp* field, but you can specify a different one:
104
-
105
- filter {
106
- dateparts {
107
- "time_field" => "some_other_field"
108
- }
109
- }
110
-
111
- #### Error Tags
112
-
113
- By default, the tag *_dateparts_error* is added on exception. You can specify different tag(s) like so:
114
-
115
- filter {
116
- dateparts {
117
- "error_tags" => ["bad_dates", "xyz"]
118
- }
119
- }
1
+ # Logstash Plugin
2
+
3
+ This is a plugin for [Logstash](https://github.com/elastic/logstash).
4
+
5
+ The source for this plugin can be [found here on github](https://github.com/mikebski/logstash-datepart-plugin.git)
6
+
7
+ Author: Mike Baranski (mike.baranski@gmail.com). Contributions are welcome.
8
+
9
+ [![Gem Version](https://badge.fury.io/rb/logstash-filter-dateparts.svg)](https://badge.fury.io/rb/logstash-filter-dateparts)
10
+ [![Build Status](https://travis-ci.org/mikebski/logstash-filter-datepart.svg?branch=master)](https://travis-ci.org/mikebski/logstash-filter-datepart)
11
+
12
+ ## License ##
13
+
14
+ Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
15
+
16
+ Licensed under the Apache License, Version 2.0 (the "License");
17
+ you may not use this file except in compliance with the License.
18
+ You may obtain a copy of the License at
19
+
20
+ http://www.apache.org/licenses/LICENSE-2.0
21
+
22
+ Unless required by applicable law or agreed to in writing, software
23
+ distributed under the License is distributed on an "AS IS" BASIS,
24
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
25
+ See the License for the specific language governing permissions and
26
+ limitations under the License.
27
+
28
+ ## About
29
+
30
+ This plugin is useful if you want to easily query Logstash data on *day of week*, *hour of day*, or other parts of a date. See the usage below for details on the output of the plugin. The date parts that can be generated are:
31
+
32
+ * day
33
+ * wday
34
+ * mday
35
+ * yday
36
+ * month
37
+ * year
38
+ * hour
39
+ * min
40
+ * sec
41
+
42
+ ## Documentation
43
+
44
+ ### Installation
45
+
46
+ To manually install the plugin, download the gem and run:
47
+
48
+ `bin/plugin install --no-verify logstash-filter-dateparts-1.0.0.gem`
49
+
50
+ ### Usage
51
+
52
+ To see the most basic usage, you can run the following (on Linux):
53
+
54
+ `echo "HI" | bin/logstash -e 'input { stdin {} } filter {dateparts { }} output { stdout { codec=> rubydebug}}'`
55
+
56
+ You could also use the logstash generator:
57
+
58
+ `bin/logstash -e 'input { generator { lines => ["HI"] count => 1 } } filter {dateparts { }} output { stdout { codec=> rubydebug}}'`
59
+
60
+ Here is the sample output:
61
+
62
+ {
63
+ "message" => "HI",
64
+ "@version" => "1",
65
+ "@timestamp" => "2015-11-20T12:24:40.217Z",
66
+ "host" => "mike-VirtualBox",
67
+ "day" => 20,
68
+ "wday" => 5,
69
+ "yday" => 324,
70
+ "month" => 11,
71
+ "year" => 2015,
72
+ "hour" => 12,
73
+ "min" => 24,
74
+ "sec" => 40
75
+ }
76
+
77
+
78
+ This uses the default configuration, which generates the following fields from the `@timestamp` field of the event:
79
+
80
+ * day
81
+ * wday
82
+ * yday
83
+ * month
84
+ * year
85
+ * hour
86
+ * min
87
+ * sec
88
+
89
+ ### Configuration
90
+
91
+ #### Fields
92
+
93
+ The generated fields are based on the date functions available in the [Ruby time class](http://ruby-doc.org/core-2.2.0/Time.html). You can specify any valid function and it will be added to the event.
94
+
95
+ For example, this will add 2 fields, *sec* corresponding to `time.sec()` and *hour* corresponding to `time.hour()`:
96
+
97
+ filter {
98
+ dateparts {
99
+ "fields" => ["sec", "hour"]
100
+ }
101
+ }
102
+
103
+ #### Time Field
104
+
105
+ By default, the plugin will use the *@timestamp* field, but you can specify a different one:
106
+
107
+ filter {
108
+ dateparts {
109
+ "time_field" => "some_other_field"
110
+ }
111
+ }
112
+
113
+ #### Duration Field (new in 2.1)
114
+
115
+ 2.1 provides the ability to calculate a duration (in seconds.milliseconds) based on 2 field.s
116
+ The value of the duration is a float with millisecond precision.
117
+
118
+ The input values must both be time values, and you specify an output field for the result
119
+
120
+ filter {
121
+ 'fields' => %w(mday),
122
+ 'duration' => {
123
+ 'start_field' => 'tstart',
124
+ 'end_field' => 'tend',
125
+ 'result_field' => 'duration'
126
+ }
127
+ }
128
+
129
+ #### Error Tags
130
+
131
+ By default, the tag *_dateparts_error* is added on exception. You can specify different tag(s) like so:
132
+
133
+ filter {
134
+ dateparts {
135
+ "error_tags" => ["bad_dates", "xyz"]
136
+ }
137
+ }
@@ -1,82 +1,118 @@
1
- # coding: utf-8
2
- # Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
3
-
4
- # Licensed under the Apache License, Version 2.0 (the "License");
5
- # you may not use this file except in compliance with the License.
6
- # You may obtain a copy of the License at
7
-
8
- # http://www.apache.org/licenses/LICENSE-2.0
9
-
10
- # Unless required by applicable law or agreed to in writing, software
11
- # distributed under the License is distributed on an "AS IS" BASIS,
12
- # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
- # See the License for the specific language governing permissions and
14
- # limitations under the License.
15
-
16
- # encoding: utf-8
17
- require "logstash/filters/base"
18
- require "logstash/namespace"
19
-
20
- # This filter will add date parts to your record based on
21
- # the timestamp field.
22
- #
23
- class LogStash::Filters::DateParts < LogStash::Filters::Base
24
- # Setting the config_name here is required. This is how you
25
- # configure this filter from your Logstash config.
26
- #
27
- # filter {
28
- # dateparts {
29
- #
30
- # }
31
- # }
32
- #
33
- config_name "dateparts"
34
- config :fields, :validate => :array, :default => ["day", "wday", "yday", "month", "year", "hour", "min", "sec"], :required => true
35
- config :time_field, :validate => :string, :default => "@timestamp", :required => true
36
- config :error_tags, :validate => :array, :default => ["_dateparts_error"], :required => true
37
-
38
- public
39
- def register
40
- logger.debug? and logger.debug("DateParts filter registered")
41
- end
42
-
43
- def plugin_error(message, event)
44
- logger.error("DatePart filter error: " + message)
45
- LogStash::Util::Decorators.add_tags(@error_tags, event, "filters/#{self.class.name}")
46
- end
47
-
48
- def get_time_from_field(f)
49
- if f.class == Time
50
- return f
51
- elsif f.respond_to?("time")
52
- return f.time()
53
- else
54
- return nil
55
- end
56
- end
57
-
58
- public
59
- def filter(event)
60
- if @fields.respond_to?("each") and @fields.respond_to?("join")
61
- logger.debug? and logger.debug("DateParts plugin filtering #{@time_field} time_field and adding fields: " + @fields.join(", "))
62
- t = get_time_from_field(event.get(@time_field))
63
- if t == nil
64
- plugin_error("Invalid time field #{@time_field}; Time field must be an instance of Time or provide a time method that returns one", event)
65
- return
66
- end
67
- @fields.each do |field|
68
- begin
69
- event.set(field, t.send(field))
70
- rescue
71
- plugin_error("No such method: #{field}\n", event)
72
- end
73
- end
74
- else
75
- plugin_error("DateParts plugin fields invalid, should be an array of function names")
76
- return
77
- end
78
-
79
- filter_matched(event)
80
- end # def filter
81
-
82
- end # class LogStash::Filters::DateParts
1
+ # encoding: utf-8
2
+ # Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
3
+
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+ # encoding: utf-8
17
+ require 'logstash/filters/base'
18
+ require 'logstash/namespace'
19
+
20
+ # This filter will add date parts to your record based on
21
+ # the timestamp field.
22
+ #
23
+ class LogStash::Filters::DateParts < LogStash::Filters::Base
24
+ # Setting the config_name here is required. This is how you
25
+ # configure this filter from your Logstash config.
26
+ #
27
+ # filter {
28
+ # dateparts {
29
+ #
30
+ # }
31
+ # }
32
+ #
33
+ config_name 'dateparts'
34
+ config :fields, :validate => :array, :default => %w(day wday yday mday month year hour min sec), :required => true
35
+ config :time_field, :validate => :string, :default => '@timestamp', :required => true
36
+ config :error_tags, :validate => :array, :default => ['_dateparts_error'], :required => true
37
+ config :duration, :validate => :hash, :required => false
38
+ public
39
+
40
+ def register
41
+ logger.debug? and logger.debug('DateParts filter registered')
42
+ end
43
+
44
+ def plugin_error(message, event)
45
+ logger.error("DatePart filter error: #{message}")
46
+ LogStash::Util::Decorators.add_tags(@error_tags, event, "filters/#{self.class.name}")
47
+ end
48
+
49
+ def get_time_from_field(f)
50
+ if f.class == Time
51
+ f
52
+ elsif f.respond_to?('time')
53
+ f.time
54
+ elsif f.respond_to?('to_time')
55
+ f.to_time
56
+ else
57
+ nil
58
+ end
59
+ end
60
+
61
+ public
62
+ def filter(event)
63
+ invalid = true
64
+ event_time = get_time_from_field(event.get(@time_field))
65
+ if event_time == nil
66
+ plugin_error("Invalid time field #{@time_field}; Time field must be an instance of Time or provide a time method that returns one", event)
67
+ return
68
+ end
69
+ if @fields.respond_to?('each') and @fields.respond_to?('join')
70
+ invalid = false
71
+ logger.debug? and logger.debug("DateParts plugin filtering #{@time_field} time_field and adding fields: " + @fields.join(', '))
72
+ @fields.each do |field|
73
+ begin
74
+ event.set(field, event_time.send(field))
75
+ rescue
76
+ plugin_error("No such method: #{field}\n", event)
77
+ end
78
+ end
79
+ end
80
+ if @duration != nil
81
+ start_field = @duration['start_field']
82
+ if(start_field == nil)
83
+ start_field = '@timestamp'
84
+ end
85
+ start_time = get_time_from_field(event.get(start_field))
86
+
87
+ end_field = @duration['end_field']
88
+ if(end_field == nil)
89
+ end_field = '@timestamp'
90
+ end
91
+ end_time = get_time_from_field(event.get(end_field))
92
+
93
+ if start_time == nil or end_time == nil
94
+ plugin_error("Invalid start [#{@duration['start_field']}] or end [#{@duration['end_field']}]. Time fields must be an instance of Time or provide a time method that returns one", event)
95
+ return
96
+ end
97
+ if(start_field.eql?(end_field))
98
+ logger.warn("Start and End fields are the same for dateparts filter [#{start_field}]")
99
+ end
100
+
101
+ result_field = @duration['result_field']
102
+ if result_field == nil
103
+ result_field = 'duration_result'
104
+ end
105
+
106
+ duration = end_time - start_time
107
+ event.set(result_field, duration)
108
+ invalid = false
109
+ end
110
+ if invalid
111
+ plugin_error('DateParts plugin error', event)
112
+ return
113
+ end
114
+
115
+ filter_matched(event)
116
+ end # def filter
117
+
118
+ end # class LogStash::Filters::DateParts
@@ -1,39 +1,39 @@
1
- # encoding: utf-8
2
-
3
- # Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
4
-
5
- # Licensed under the Apache License, Version 2.0 (the "License");
6
- # you may not use this file except in compliance with the License.
7
- # You may obtain a copy of the License at
8
-
9
- # http://www.apache.org/licenses/LICENSE-2.0
10
-
11
- # Unless required by applicable law or agreed to in writing, software
12
- # distributed under the License is distributed on an "AS IS" BASIS,
13
- # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14
- # See the License for the specific language governing permissions and
15
- # limitations under the License.
16
-
17
- Gem::Specification.new do |s|
18
- s.name = 'logstash-filter-dateparts'
19
- s.version = '2.0.1'
20
- s.licenses = ['Apache License (2.0)']
21
- s.summary = 'This dateparts fileter adds date information to your event based on your timestamp'
22
- s.description = 'This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program'
23
- s.authors = ['Mike Baranski']
24
- s.email = 'mike.baranski@gmail.com'
25
- s.homepage = 'http://mikeski.net'
26
- s.require_paths = ['lib']
27
-
28
- # Files
29
- s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','Gemfile','LICENSE']
30
- # Tests
31
- s.test_files = s.files.grep(%r{^(test|spec|features)/})
32
-
33
- # Special flag to let us know this is actually a logstash plugin
34
- s.metadata = { 'logstash_plugin' => 'true', 'logstash_group' => 'filter' }
35
-
36
- # Gem dependencies
37
- s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
38
- s.add_development_dependency 'logstash-devutils'
39
- end
1
+ # encoding: utf-8
2
+
3
+ # Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
4
+
5
+ # Licensed under the Apache License, Version 2.0 (the "License");
6
+ # you may not use this file except in compliance with the License.
7
+ # You may obtain a copy of the License at
8
+
9
+ # http://www.apache.org/licenses/LICENSE-2.0
10
+
11
+ # Unless required by applicable law or agreed to in writing, software
12
+ # distributed under the License is distributed on an "AS IS" BASIS,
13
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14
+ # See the License for the specific language governing permissions and
15
+ # limitations under the License.
16
+
17
+ Gem::Specification.new do |s|
18
+ s.name = 'logstash-filter-dateparts'
19
+ s.version = '2.1.1'
20
+ s.licenses = ['Apache License (2.0)']
21
+ s.summary = 'This dateparts fileter adds date information to your event based on your timestamp'
22
+ s.description = 'This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program'
23
+ s.authors = ['Mike Baranski']
24
+ s.email = 'mike.baranski@gmail.com'
25
+ s.homepage = 'http://mikeski.net'
26
+ s.require_paths = ['lib']
27
+
28
+ # Files
29
+ s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','Gemfile','LICENSE']
30
+ # Tests
31
+ s.test_files = s.files.grep(%r{^(test|spec|features)/})
32
+
33
+ # Special flag to let us know this is actually a logstash plugin
34
+ s.metadata = { 'logstash_plugin' => 'true', 'logstash_group' => 'filter' }
35
+
36
+ # Gem dependencies
37
+ s.add_runtime_dependency 'logstash-core-plugin-api', '>= 1.60', '<= 2.99'
38
+ s.add_development_dependency 'logstash-devutils', '1.1.0'
39
+ end
@@ -1,97 +1,206 @@
1
- # coding: utf-8
2
- # Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
3
-
4
- # Licensed under the Apache License, Version 2.0 (the "License");
5
- # you may not use this file except in compliance with the License.
6
- # You may obtain a copy of the License at
7
-
8
- # http://www.apache.org/licenses/LICENSE-2.0
9
-
10
- # Unless required by applicable law or agreed to in writing, software
11
- # distributed under the License is distributed on an "AS IS" BASIS,
12
- # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
- # See the License for the specific language governing permissions and
14
- # limitations under the License.
15
-
16
- require 'spec_helper'
17
- require "logstash/filters/dateparts"
18
- require "logstash/timestamp"
19
- require "logstash/event"
20
-
21
- def get_event(contents = {})
22
- contents["@timestamp"] = LogStash::Timestamp.new
23
- event = LogStash::Event.new(contents)
24
- return event
25
- end
26
-
27
- describe LogStash::Filters::DateParts do
28
- default_ts = "@timestamp"
29
- alt_ts_field = "zxlk"
30
-
31
- it "Default config should result in filter with 8 functions, one error tag and @timestamp as the time field" do
32
- f = LogStash::Filters::DateParts.new({})
33
-
34
- expect(f.class).to eq(LogStash::Filters::DateParts)
35
- expect(f.fields.length).to eq(8)
36
- expect(f.time_field).to eq(default_ts)
37
- expect(f.error_tags.length).to eq(1)
38
- end
39
-
40
- it "Config should result in filter with 2 functions and the alt timestamp field" do
41
- f = LogStash::Filters::DateParts.new({
42
- "fields" => ["sec", "hour"],
43
- "time_field" => alt_ts_field
44
- })
45
-
46
- expect(f.class).to eq(LogStash::Filters::DateParts)
47
- expect(f.fields.length).to eq(2)
48
- expect(f.fields[0]).to eq("sec")
49
- expect(f.time_field).to eq(alt_ts_field)
50
- end
51
-
52
- it "Should generate the default fields (8 of them)" do
53
- event = get_event()
54
- count = event.to_hash().count
55
- f = LogStash::Filters::DateParts.new({})
56
- f.filter(event)
57
-
58
- expect(event.to_hash().count).to eq(count + 8)
59
- expect(event.get('sec')).to be_truthy
60
- expect(event.get('hour')).to be_truthy
61
- expect(event.get('min')).to be_truthy
62
- expect(event.get('month')).to be_truthy
63
- expect(event.get('year')).to be_truthy
64
- expect(event.get('day')).to be_truthy
65
- expect(event.get('wday')).to be_truthy
66
- expect(event.get('yday')).to be_truthy
67
- expect(event.get('tags')).to be_nil
68
- end
69
-
70
- it "Should generate only the specified fields" do
71
- event = get_event()
72
- count = event.to_hash.count
73
- f = LogStash::Filters::DateParts.new({
74
- "fields" => ["sec", "hour"]
75
- })
76
- f.filter(event)
77
- expect(event.to_hash().count).to eq(count + 2)
78
- expect(event.get('sec')).to be_truthy
79
- expect(event.get('hour')).to be_truthy
80
- expect(event.get('min')).to be_nil
81
- expect(event.get('month')).to be_nil
82
- expect(event.get('year')).to be_nil
83
- expect(event.get('day')).to be_nil
84
- expect(event.get('wday')).to be_nil
85
- expect(event.get('yday')).to be_nil
86
- expect(event.get('tags')).to be_nil
87
- end
88
-
89
- it "Should set the error tag on an invalid time field" do
90
- event = get_event()
91
- count = event.to_hash().count
92
- f = LogStash::Filters::DateParts.new({ "time_field" => alt_ts_field })
93
-
94
- f.filter(event)
95
- expect(event.get('tags').include? '_dateparts_error').to eq(true)
96
- end
97
- end
1
+ # coding: utf-8
2
+ # Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
3
+
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+ require 'spec_helper'
17
+ require 'logstash/filters/dateparts'
18
+ require 'logstash/timestamp'
19
+ require 'logstash/event'
20
+
21
+ def get_event(contents = {})
22
+ contents['@timestamp'] = LogStash::Timestamp.new
23
+ LogStash::Event.new(contents)
24
+ end
25
+
26
+ describe LogStash::Filters::DateParts do
27
+ default_ts = '@timestamp'
28
+ alt_ts_field = 'zxlk'
29
+
30
+ it 'Get time from field should work with Time' do
31
+ f = LogStash::Filters::DateParts.new({})
32
+ field_to_test = Time.new
33
+ val = f.get_time_from_field(field_to_test);
34
+ expect(val.class).to be(Time)
35
+ end
36
+
37
+ it 'Get time from field should work with DateTime' do
38
+ f = LogStash::Filters::DateParts.new({})
39
+ field_to_test = DateTime.new
40
+ val = f.get_time_from_field(field_to_test);
41
+ expect(val.class).to be(Time)
42
+ end
43
+
44
+ it 'Default config should result in filter with 8 functions, one error tag and @timestamp as the time field' do
45
+ f = LogStash::Filters::DateParts.new({})
46
+
47
+ expect(f.class).to eq(LogStash::Filters::DateParts)
48
+ expect(f.fields.length).to eq(9)
49
+ expect(f.time_field).to eq(default_ts)
50
+ expect(f.error_tags.length).to eq(1)
51
+ end
52
+
53
+ it 'Config should result in filter with 2 functions and the alt timestamp field' do
54
+ f = LogStash::Filters::DateParts.new({
55
+ 'fields' => %w(sec hour),
56
+ 'time_field' => alt_ts_field
57
+ })
58
+
59
+ expect(f.class).to eq(LogStash::Filters::DateParts)
60
+ expect(f.fields.length).to eq(2)
61
+ expect(f.fields[0]).to eq('sec')
62
+ expect(f.time_field).to eq(alt_ts_field)
63
+ end
64
+
65
+ it 'Should generate the default fields (8 of them)' do
66
+ event = get_event
67
+ count = event.to_hash.count
68
+ f = LogStash::Filters::DateParts.new({})
69
+ f.filter(event)
70
+
71
+ expect(event.to_hash.count).to eq(count + 9)
72
+ expect(event.get('sec')).to be_truthy
73
+ expect(event.get('hour')).to be_truthy
74
+ expect(event.get('min')).to be_truthy
75
+ expect(event.get('month')).to be_truthy
76
+ expect(event.get('year')).to be_truthy
77
+ expect(event.get('day')).to be_truthy
78
+ expect(event.get('wday')).to be_truthy
79
+ expect(event.get('mday')).to be_truthy
80
+ expect(event.get('yday')).to be_truthy
81
+ expect(event.get('tags')).to be_nil
82
+ end
83
+
84
+ it 'Should generate only the specified fields' do
85
+ event = get_event
86
+ count = event.to_hash.count
87
+ f = LogStash::Filters::DateParts.new({
88
+ 'fields' => %w(sec hour)
89
+ })
90
+ f.filter(event)
91
+ expect(event.to_hash.count).to eq(count + 2)
92
+ expect(event.get('sec')).to be_truthy
93
+ expect(event.get('hour')).to be_truthy
94
+ expect(event.get('min')).to be_nil
95
+ expect(event.get('month')).to be_nil
96
+ expect(event.get('year')).to be_nil
97
+ expect(event.get('day')).to be_nil
98
+ expect(event.get('wday')).to be_nil
99
+ expect(event.get('mday')).to be_nil
100
+ expect(event.get('yday')).to be_nil
101
+ expect(event.get('tags')).to be_nil
102
+ end
103
+
104
+ it 'Should set the error tag on an invalid time field' do
105
+ event = get_event
106
+ f = LogStash::Filters::DateParts.new({ 'time_field' => alt_ts_field })
107
+
108
+ f.filter(event)
109
+ expect(event.get('tags').include? '_dateparts_error').to eq(true)
110
+ end
111
+
112
+ it 'Should bail on an invalid date part' do
113
+ event = get_event
114
+ f = LogStash::Filters::DateParts.new({
115
+ 'fields' => %w(seczzz zzhour)
116
+ })
117
+ f.filter(event)
118
+ expect(event.get('tags').include? '_dateparts_error').to eq(true)
119
+ end
120
+
121
+ it 'Should calculate a duration' do
122
+ event = get_event
123
+ f = LogStash::Filters::DateParts.new({
124
+ 'duration' => {
125
+ 'start_field' => '@timestamp',
126
+ 'end_field' => 'sometime',
127
+ 'result_field' => 'duration'
128
+ }
129
+ })
130
+ event.set('sometime', Time.new)
131
+ f.filter(event)
132
+ expect(event.get('tags')).to be_nil
133
+ expect(event.get('duration')).to be > 0
134
+ end
135
+
136
+ it 'Should calculate a duration using 2 fields' do
137
+ event = get_event
138
+ f = LogStash::Filters::DateParts.new({
139
+ 'duration' => {
140
+ 'start_field' => 'tstart',
141
+ 'end_field' => 'tend',
142
+ 'result_field' => 'duration'
143
+ }
144
+ })
145
+
146
+ event.set('tstart', DateTime.new(2016, 1, 1, 12, 0, 0).to_time)
147
+ event.set('tend', DateTime.new(2016, 1, 1, 12, 0, 0).to_time)
148
+ f.filter(event)
149
+ expect(event.get('tags')).to be_nil
150
+ expect(event.get('duration')).to eq(0.0)
151
+ end
152
+
153
+ it 'Should calculate a duration of 1 second using 2 fields' do
154
+ event = get_event
155
+ f = LogStash::Filters::DateParts.new({
156
+ 'duration' => {
157
+ 'start_field' => 'tstart',
158
+ 'end_field' => 'tend',
159
+ 'result_field' => 'duration'
160
+ }
161
+ })
162
+
163
+ event.set('tstart', DateTime.new(2016, 1, 1, 23, 0, 0).to_time)
164
+ event.set('tend', DateTime.new(2016, 1, 1, 23, 0, 1).to_time)
165
+ f.filter(event)
166
+ expect(event.get('tags')).to be_nil
167
+ expect(event.get('duration')).to eq(1.0)
168
+ end
169
+
170
+ it 'Should calculate a duration of 3600 seconds using 2 fields and calculate datepart' do
171
+ event = get_event
172
+ f = LogStash::Filters::DateParts.new({
173
+ #'fields' => %w(mday),
174
+ 'duration' => {
175
+ 'start_field' => 'tstart',
176
+ 'end_field' => 'tend',
177
+ 'result_field' => 'duration'
178
+ }
179
+ })
180
+
181
+ event.set('tstart', DateTime.new(2016, 1, 1, 20, 0, 0).to_time)
182
+ event.set('tend', DateTime.new(2016, 1, 1, 21, 0, 0).to_time)
183
+ f.filter(event)
184
+ expect(event.get('tags')).to be_nil
185
+ expect(event.get('duration')).to eq(3600.0)
186
+ expect(event.get('mday')).to be > -1
187
+
188
+ end
189
+
190
+ it 'Should warn and return 0.0 if start and end are the same field' do
191
+ event = get_event
192
+ f = LogStash::Filters::DateParts.new({
193
+ 'duration' => {
194
+ 'start_field' => 'tstart',
195
+ 'end_field' => 'tstart',
196
+ 'result_field' => 'duration'
197
+ }
198
+ })
199
+
200
+ event.set('tstart', DateTime.new(2016, 1, 1, 20, 0, 0).to_time)
201
+ f.filter(event)
202
+ expect(event.get('tags')).to be_nil
203
+ expect(event.get('duration')).to eq(0.0)
204
+ expect(event.get('mday')).to be > -1
205
+ end
206
+ end
@@ -1,16 +1,16 @@
1
- # coding: utf-8
2
- # Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
3
-
4
- # Licensed under the Apache License, Version 2.0 (the "License");
5
- # you may not use this file except in compliance with the License.
6
- # You may obtain a copy of the License at
7
-
8
- # http://www.apache.org/licenses/LICENSE-2.0
9
-
10
- # Unless required by applicable law or agreed to in writing, software
11
- # distributed under the License is distributed on an "AS IS" BASIS,
12
- # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
- # See the License for the specific language governing permissions and
14
- # limitations under the License.
15
-
16
- require "logstash/devutils/rspec/spec_helper"
1
+ # coding: utf-8
2
+ # Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
3
+
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+ require "logstash/devutils/rspec/spec_helper"
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-filter-dateparts
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.1
4
+ version: 2.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mike Baranski
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-09-30 00:00:00.000000000 Z
11
+ date: 2016-10-03 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: logstash-core-plugin-api
@@ -34,16 +34,16 @@ dependencies:
34
34
  name: logstash-devutils
35
35
  requirement: !ruby/object:Gem::Requirement
36
36
  requirements:
37
- - - '>='
37
+ - - '='
38
38
  - !ruby/object:Gem::Version
39
- version: '0'
39
+ version: 1.1.0
40
40
  type: :development
41
41
  prerelease: false
42
42
  version_requirements: !ruby/object:Gem::Requirement
43
43
  requirements:
44
- - - '>='
44
+ - - '='
45
45
  - !ruby/object:Gem::Version
46
- version: '0'
46
+ version: 1.1.0
47
47
  description: This gem is a logstash plugin required to be installed on top of the
48
48
  Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not
49
49
  a stand-alone program
@@ -82,7 +82,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
82
82
  version: '0'
83
83
  requirements: []
84
84
  rubyforge_project:
85
- rubygems_version: 2.0.14
85
+ rubygems_version: 2.0.14.1
86
86
  signing_key:
87
87
  specification_version: 4
88
88
  summary: This dateparts fileter adds date information to your event based on your