logstash-filter-dateparts 2.0.0 → 2.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -5
- data/Gemfile +1 -1
- data/LICENSE +13 -13
- data/README.md +119 -119
- data/lib/logstash/filters/dateparts.rb +82 -83
- data/logstash-filter-dateparts.gemspec +39 -39
- data/spec/filters/dateparts_spec.rb +97 -97
- data/spec/spec_helper.rb +16 -16
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a8b480afd89318ea8bb3176bdec0b1cd9fbf3cd3
|
4
|
+
data.tar.gz: cd5302e4a35e5f4bd1b33b25759605bf4f288d77
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5c3296b74768a007fc973faa9a428e213ecb627958e82342105e6a7a6fdd107c2edc5099b0e3f0c70133ed4819f14585496fb50fadec8234c9432105e5a5e650
|
7
|
+
data.tar.gz: a20d130798618bb3c5c0758e93615e1d60c4545edfdf54ce23a9c303625f127d019d1f5ad5d02f07cbf74628a319b360d7ae9a8c95c3b91a1f2d6fd9133ceac6
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1
|
-
# 2.0.0
|
2
|
-
- Breaking: Updated plugin to use new Java Event APIs
|
3
|
-
|
4
|
-
# 1.0.1
|
5
|
-
- Updated dependencies to avoid being tied to Logstash major release.
|
1
|
+
# 2.0.0
|
2
|
+
- Breaking: Updated plugin to use new Java Event APIs
|
3
|
+
|
4
|
+
# 1.0.1
|
5
|
+
- Updated dependencies to avoid being tied to Logstash major release.
|
data/Gemfile
CHANGED
@@ -1,2 +1,2 @@
|
|
1
|
-
source 'https://rubygems.org'
|
1
|
+
source 'https://rubygems.org'
|
2
2
|
gemspec
|
data/LICENSE
CHANGED
@@ -1,13 +1,13 @@
|
|
1
|
-
Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
2
|
-
|
3
|
-
Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
-
you may not use this file except in compliance with the License.
|
5
|
-
You may obtain a copy of the License at
|
6
|
-
|
7
|
-
http://www.apache.org/licenses/LICENSE-2.0
|
8
|
-
|
9
|
-
Unless required by applicable law or agreed to in writing, software
|
10
|
-
distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
-
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
-
See the License for the specific language governing permissions and
|
13
|
-
limitations under the License.
|
1
|
+
Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
2
|
+
|
3
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
+
you may not use this file except in compliance with the License.
|
5
|
+
You may obtain a copy of the License at
|
6
|
+
|
7
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
8
|
+
|
9
|
+
Unless required by applicable law or agreed to in writing, software
|
10
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
+
See the License for the specific language governing permissions and
|
13
|
+
limitations under the License.
|
data/README.md
CHANGED
@@ -1,119 +1,119 @@
|
|
1
|
-
# Logstash Plugin
|
2
|
-
|
3
|
-
This is a plugin for [Logstash](https://github.com/elastic/logstash).
|
4
|
-
|
5
|
-
The source for this plugin can be [found here on github](https://github.com/mikebski/logstash-datepart-plugin.git)
|
6
|
-
|
7
|
-
Author: Mike Baranski (mike.baranski@gmail.com). Contributions are welcome.
|
8
|
-
|
9
|
-
[](https://badge.fury.io/rb/logstash-filter-dateparts)
|
10
|
-
|
11
|
-
## License ##
|
12
|
-
|
13
|
-
Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
14
|
-
|
15
|
-
Licensed under the Apache License, Version 2.0 (the "License");
|
16
|
-
you may not use this file except in compliance with the License.
|
17
|
-
You may obtain a copy of the License at
|
18
|
-
|
19
|
-
http://www.apache.org/licenses/LICENSE-2.0
|
20
|
-
|
21
|
-
Unless required by applicable law or agreed to in writing, software
|
22
|
-
distributed under the License is distributed on an "AS IS" BASIS,
|
23
|
-
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
24
|
-
See the License for the specific language governing permissions and
|
25
|
-
limitations under the License.
|
26
|
-
|
27
|
-
## About
|
28
|
-
|
29
|
-
This plugin is useful if you want to easily query Logstash data on *day of week*, *hour of day*, or other parts of a date. See the usage below for details on the output of the plugin. The date parts that can be generated are:
|
30
|
-
|
31
|
-
* day
|
32
|
-
* wday
|
33
|
-
* yday
|
34
|
-
* month
|
35
|
-
* year
|
36
|
-
* hour
|
37
|
-
* min
|
38
|
-
* sec
|
39
|
-
|
40
|
-
## Documentation
|
41
|
-
|
42
|
-
### Installation
|
43
|
-
|
44
|
-
To manually install the plugin, download the gem and run:
|
45
|
-
|
46
|
-
`bin/plugin install --no-verify logstash-filter-dateparts-1.0.0.gem`
|
47
|
-
|
48
|
-
### Usage
|
49
|
-
|
50
|
-
To see the most basic usage, you can run the following (on Linux):
|
51
|
-
|
52
|
-
`echo "HI" | bin/logstash -e 'input { stdin {} } filter {dateparts { }} output { stdout { codec=> rubydebug}}'`
|
53
|
-
|
54
|
-
You could also use the logstash generator:
|
55
|
-
|
56
|
-
`bin/logstash -e 'input { generator { lines => ["HI"] count => 1 } } filter {dateparts { }} output { stdout { codec=> rubydebug}}'`
|
57
|
-
|
58
|
-
Here is the sample output:
|
59
|
-
|
60
|
-
{
|
61
|
-
"message" => "HI",
|
62
|
-
"@version" => "1",
|
63
|
-
"@timestamp" => "2015-11-20T12:24:40.217Z",
|
64
|
-
"host" => "mike-VirtualBox",
|
65
|
-
"day" => 20,
|
66
|
-
"wday" => 5,
|
67
|
-
"yday" => 324,
|
68
|
-
"month" => 11,
|
69
|
-
"year" => 2015,
|
70
|
-
"hour" => 12,
|
71
|
-
"min" => 24,
|
72
|
-
"sec" => 40
|
73
|
-
}
|
74
|
-
|
75
|
-
|
76
|
-
This uses the default configuration, which generates the following fields from the `@timestamp` field of the event:
|
77
|
-
|
78
|
-
* day
|
79
|
-
* wday
|
80
|
-
* yday
|
81
|
-
* month
|
82
|
-
* year
|
83
|
-
* hour
|
84
|
-
* min
|
85
|
-
* sec
|
86
|
-
|
87
|
-
### Configuration
|
88
|
-
|
89
|
-
#### Fields
|
90
|
-
|
91
|
-
The generated fields are based on the date functions available in the [Ruby time class](http://ruby-doc.org/core-2.2.0/Time.html). You can specify any valid function and it will be added to the event.
|
92
|
-
|
93
|
-
For example, this will add 2 fields, *sec* corresponding to `time.sec()` and *hour* corresponding to `time.hour()`:
|
94
|
-
|
95
|
-
filter {
|
96
|
-
dateparts {
|
97
|
-
"fields" => ["sec", "hour"]
|
98
|
-
}
|
99
|
-
}
|
100
|
-
|
101
|
-
#### Time Field
|
102
|
-
|
103
|
-
By default, the plugin will use the *@timestamp* field, but you can specify a different one:
|
104
|
-
|
105
|
-
filter {
|
106
|
-
dateparts {
|
107
|
-
"time_field" => "some_other_field"
|
108
|
-
}
|
109
|
-
}
|
110
|
-
|
111
|
-
#### Error Tags
|
112
|
-
|
113
|
-
By default, the tag *_dateparts_error* is added on exception. You can specify different tag(s) like so:
|
114
|
-
|
115
|
-
filter {
|
116
|
-
dateparts {
|
117
|
-
"error_tags" => ["bad_dates", "xyz"]
|
118
|
-
}
|
119
|
-
}
|
1
|
+
# Logstash Plugin
|
2
|
+
|
3
|
+
This is a plugin for [Logstash](https://github.com/elastic/logstash).
|
4
|
+
|
5
|
+
The source for this plugin can be [found here on github](https://github.com/mikebski/logstash-datepart-plugin.git)
|
6
|
+
|
7
|
+
Author: Mike Baranski (mike.baranski@gmail.com). Contributions are welcome.
|
8
|
+
|
9
|
+
[](https://badge.fury.io/rb/logstash-filter-dateparts)
|
10
|
+
|
11
|
+
## License ##
|
12
|
+
|
13
|
+
Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
14
|
+
|
15
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
16
|
+
you may not use this file except in compliance with the License.
|
17
|
+
You may obtain a copy of the License at
|
18
|
+
|
19
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
20
|
+
|
21
|
+
Unless required by applicable law or agreed to in writing, software
|
22
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
23
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
24
|
+
See the License for the specific language governing permissions and
|
25
|
+
limitations under the License.
|
26
|
+
|
27
|
+
## About
|
28
|
+
|
29
|
+
This plugin is useful if you want to easily query Logstash data on *day of week*, *hour of day*, or other parts of a date. See the usage below for details on the output of the plugin. The date parts that can be generated are:
|
30
|
+
|
31
|
+
* day
|
32
|
+
* wday
|
33
|
+
* yday
|
34
|
+
* month
|
35
|
+
* year
|
36
|
+
* hour
|
37
|
+
* min
|
38
|
+
* sec
|
39
|
+
|
40
|
+
## Documentation
|
41
|
+
|
42
|
+
### Installation
|
43
|
+
|
44
|
+
To manually install the plugin, download the gem and run:
|
45
|
+
|
46
|
+
`bin/plugin install --no-verify logstash-filter-dateparts-1.0.0.gem`
|
47
|
+
|
48
|
+
### Usage
|
49
|
+
|
50
|
+
To see the most basic usage, you can run the following (on Linux):
|
51
|
+
|
52
|
+
`echo "HI" | bin/logstash -e 'input { stdin {} } filter {dateparts { }} output { stdout { codec=> rubydebug}}'`
|
53
|
+
|
54
|
+
You could also use the logstash generator:
|
55
|
+
|
56
|
+
`bin/logstash -e 'input { generator { lines => ["HI"] count => 1 } } filter {dateparts { }} output { stdout { codec=> rubydebug}}'`
|
57
|
+
|
58
|
+
Here is the sample output:
|
59
|
+
|
60
|
+
{
|
61
|
+
"message" => "HI",
|
62
|
+
"@version" => "1",
|
63
|
+
"@timestamp" => "2015-11-20T12:24:40.217Z",
|
64
|
+
"host" => "mike-VirtualBox",
|
65
|
+
"day" => 20,
|
66
|
+
"wday" => 5,
|
67
|
+
"yday" => 324,
|
68
|
+
"month" => 11,
|
69
|
+
"year" => 2015,
|
70
|
+
"hour" => 12,
|
71
|
+
"min" => 24,
|
72
|
+
"sec" => 40
|
73
|
+
}
|
74
|
+
|
75
|
+
|
76
|
+
This uses the default configuration, which generates the following fields from the `@timestamp` field of the event:
|
77
|
+
|
78
|
+
* day
|
79
|
+
* wday
|
80
|
+
* yday
|
81
|
+
* month
|
82
|
+
* year
|
83
|
+
* hour
|
84
|
+
* min
|
85
|
+
* sec
|
86
|
+
|
87
|
+
### Configuration
|
88
|
+
|
89
|
+
#### Fields
|
90
|
+
|
91
|
+
The generated fields are based on the date functions available in the [Ruby time class](http://ruby-doc.org/core-2.2.0/Time.html). You can specify any valid function and it will be added to the event.
|
92
|
+
|
93
|
+
For example, this will add 2 fields, *sec* corresponding to `time.sec()` and *hour* corresponding to `time.hour()`:
|
94
|
+
|
95
|
+
filter {
|
96
|
+
dateparts {
|
97
|
+
"fields" => ["sec", "hour"]
|
98
|
+
}
|
99
|
+
}
|
100
|
+
|
101
|
+
#### Time Field
|
102
|
+
|
103
|
+
By default, the plugin will use the *@timestamp* field, but you can specify a different one:
|
104
|
+
|
105
|
+
filter {
|
106
|
+
dateparts {
|
107
|
+
"time_field" => "some_other_field"
|
108
|
+
}
|
109
|
+
}
|
110
|
+
|
111
|
+
#### Error Tags
|
112
|
+
|
113
|
+
By default, the tag *_dateparts_error* is added on exception. You can specify different tag(s) like so:
|
114
|
+
|
115
|
+
filter {
|
116
|
+
dateparts {
|
117
|
+
"error_tags" => ["bad_dates", "xyz"]
|
118
|
+
}
|
119
|
+
}
|
@@ -1,83 +1,82 @@
|
|
1
|
-
# coding: utf-8
|
2
|
-
# Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
3
|
-
|
4
|
-
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
-
# you may not use this file except in compliance with the License.
|
6
|
-
# You may obtain a copy of the License at
|
7
|
-
|
8
|
-
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
-
|
10
|
-
# Unless required by applicable law or agreed to in writing, software
|
11
|
-
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
-
# See the License for the specific language governing permissions and
|
14
|
-
# limitations under the License.
|
15
|
-
|
16
|
-
# encoding: utf-8
|
17
|
-
require "logstash/filters/base"
|
18
|
-
require "logstash/namespace"
|
19
|
-
|
20
|
-
# This filter will add date parts to your record based on
|
21
|
-
# the timestamp field.
|
22
|
-
#
|
23
|
-
class LogStash::Filters::DateParts < LogStash::Filters::Base
|
24
|
-
# Setting the config_name here is required. This is how you
|
25
|
-
# configure this filter from your Logstash config.
|
26
|
-
#
|
27
|
-
# filter {
|
28
|
-
# dateparts {
|
29
|
-
#
|
30
|
-
# }
|
31
|
-
# }
|
32
|
-
#
|
33
|
-
config_name "dateparts"
|
34
|
-
config :fields, :validate => :array, :default => ["day", "wday", "yday", "month", "year", "hour", "min", "sec"], :required => true
|
35
|
-
config :time_field, :validate => :string, :default => "@timestamp", :required => true
|
36
|
-
config :error_tags, :validate => :array, :default => ["_dateparts_error"], :required => true
|
37
|
-
|
38
|
-
public
|
39
|
-
def register
|
40
|
-
logger.debug? and logger.debug("DateParts filter registered")
|
41
|
-
end
|
42
|
-
|
43
|
-
def plugin_error(message, event)
|
44
|
-
logger.error("DatePart filter error: " + message)
|
45
|
-
LogStash::Util::Decorators.add_tags(@error_tags, event, "filters/#{self.class.name}")
|
46
|
-
end
|
47
|
-
|
48
|
-
def get_time_from_field(f)
|
49
|
-
if f.class == Time
|
50
|
-
return f
|
51
|
-
elsif f.respond_to?("time")
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
t
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
end # class LogStash::Filters::DateParts
|
1
|
+
# coding: utf-8
|
2
|
+
# Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
3
|
+
|
4
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
+
# you may not use this file except in compliance with the License.
|
6
|
+
# You may obtain a copy of the License at
|
7
|
+
|
8
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
+
|
10
|
+
# Unless required by applicable law or agreed to in writing, software
|
11
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
+
# See the License for the specific language governing permissions and
|
14
|
+
# limitations under the License.
|
15
|
+
|
16
|
+
# encoding: utf-8
|
17
|
+
require "logstash/filters/base"
|
18
|
+
require "logstash/namespace"
|
19
|
+
|
20
|
+
# This filter will add date parts to your record based on
|
21
|
+
# the timestamp field.
|
22
|
+
#
|
23
|
+
class LogStash::Filters::DateParts < LogStash::Filters::Base
|
24
|
+
# Setting the config_name here is required. This is how you
|
25
|
+
# configure this filter from your Logstash config.
|
26
|
+
#
|
27
|
+
# filter {
|
28
|
+
# dateparts {
|
29
|
+
#
|
30
|
+
# }
|
31
|
+
# }
|
32
|
+
#
|
33
|
+
config_name "dateparts"
|
34
|
+
config :fields, :validate => :array, :default => ["day", "wday", "yday", "month", "year", "hour", "min", "sec"], :required => true
|
35
|
+
config :time_field, :validate => :string, :default => "@timestamp", :required => true
|
36
|
+
config :error_tags, :validate => :array, :default => ["_dateparts_error"], :required => true
|
37
|
+
|
38
|
+
public
|
39
|
+
def register
|
40
|
+
logger.debug? and logger.debug("DateParts filter registered")
|
41
|
+
end
|
42
|
+
|
43
|
+
def plugin_error(message, event)
|
44
|
+
logger.error("DatePart filter error: " + message)
|
45
|
+
LogStash::Util::Decorators.add_tags(@error_tags, event, "filters/#{self.class.name}")
|
46
|
+
end
|
47
|
+
|
48
|
+
def get_time_from_field(f)
|
49
|
+
if f.class == Time
|
50
|
+
return f
|
51
|
+
elsif f.respond_to?("time")
|
52
|
+
return f.time()
|
53
|
+
else
|
54
|
+
return nil
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
public
|
59
|
+
def filter(event)
|
60
|
+
if @fields.respond_to?("each") and @fields.respond_to?("join")
|
61
|
+
logger.debug? and logger.debug("DateParts plugin filtering #{@time_field} time_field and adding fields: " + @fields.join(", "))
|
62
|
+
t = get_time_from_field(event.get(@time_field))
|
63
|
+
if t == nil
|
64
|
+
plugin_error("Invalid time field #{@time_field}; Time field must be an instance of Time or provide a time method that returns one", event)
|
65
|
+
return
|
66
|
+
end
|
67
|
+
@fields.each do |field|
|
68
|
+
begin
|
69
|
+
event.set(field, t.send(field))
|
70
|
+
rescue
|
71
|
+
plugin_error("No such method: #{field}\n", event)
|
72
|
+
end
|
73
|
+
end
|
74
|
+
else
|
75
|
+
plugin_error("DateParts plugin fields invalid, should be an array of function names")
|
76
|
+
return
|
77
|
+
end
|
78
|
+
|
79
|
+
filter_matched(event)
|
80
|
+
end # def filter
|
81
|
+
|
82
|
+
end # class LogStash::Filters::DateParts
|
@@ -1,39 +1,39 @@
|
|
1
|
-
# encoding: utf-8
|
2
|
-
|
3
|
-
# Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
4
|
-
|
5
|
-
# Licensed under the Apache License, Version 2.0 (the "License");
|
6
|
-
# you may not use this file except in compliance with the License.
|
7
|
-
# You may obtain a copy of the License at
|
8
|
-
|
9
|
-
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
-
|
11
|
-
# Unless required by applicable law or agreed to in writing, software
|
12
|
-
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
-
# See the License for the specific language governing permissions and
|
15
|
-
# limitations under the License.
|
16
|
-
|
17
|
-
Gem::Specification.new do |s|
|
18
|
-
s.name = 'logstash-filter-dateparts'
|
19
|
-
s.version = '2.0.
|
20
|
-
s.licenses = ['Apache License (2.0)']
|
21
|
-
s.summary = 'This dateparts fileter adds date information to your event based on your timestamp'
|
22
|
-
s.description = 'This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program'
|
23
|
-
s.authors = ['Mike Baranski']
|
24
|
-
s.email = 'mike.baranski@gmail.com'
|
25
|
-
s.homepage = 'http://mikeski.net'
|
26
|
-
s.require_paths = ['lib']
|
27
|
-
|
28
|
-
# Files
|
29
|
-
s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','Gemfile','LICENSE']
|
30
|
-
# Tests
|
31
|
-
s.test_files = s.files.grep(%r{^(test|spec|features)/})
|
32
|
-
|
33
|
-
# Special flag to let us know this is actually a logstash plugin
|
34
|
-
s.metadata = { 'logstash_plugin' => 'true', 'logstash_group' => 'filter' }
|
35
|
-
|
36
|
-
# Gem dependencies
|
37
|
-
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
|
38
|
-
s.add_development_dependency 'logstash-devutils'
|
39
|
-
end
|
1
|
+
# encoding: utf-8
|
2
|
+
|
3
|
+
# Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
4
|
+
|
5
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
6
|
+
# you may not use this file except in compliance with the License.
|
7
|
+
# You may obtain a copy of the License at
|
8
|
+
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
+
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
+
# See the License for the specific language governing permissions and
|
15
|
+
# limitations under the License.
|
16
|
+
|
17
|
+
Gem::Specification.new do |s|
|
18
|
+
s.name = 'logstash-filter-dateparts'
|
19
|
+
s.version = '2.0.1'
|
20
|
+
s.licenses = ['Apache License (2.0)']
|
21
|
+
s.summary = 'This dateparts fileter adds date information to your event based on your timestamp'
|
22
|
+
s.description = 'This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program'
|
23
|
+
s.authors = ['Mike Baranski']
|
24
|
+
s.email = 'mike.baranski@gmail.com'
|
25
|
+
s.homepage = 'http://mikeski.net'
|
26
|
+
s.require_paths = ['lib']
|
27
|
+
|
28
|
+
# Files
|
29
|
+
s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','Gemfile','LICENSE']
|
30
|
+
# Tests
|
31
|
+
s.test_files = s.files.grep(%r{^(test|spec|features)/})
|
32
|
+
|
33
|
+
# Special flag to let us know this is actually a logstash plugin
|
34
|
+
s.metadata = { 'logstash_plugin' => 'true', 'logstash_group' => 'filter' }
|
35
|
+
|
36
|
+
# Gem dependencies
|
37
|
+
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
|
38
|
+
s.add_development_dependency 'logstash-devutils'
|
39
|
+
end
|
@@ -1,97 +1,97 @@
|
|
1
|
-
# coding: utf-8
|
2
|
-
# Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
3
|
-
|
4
|
-
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
-
# you may not use this file except in compliance with the License.
|
6
|
-
# You may obtain a copy of the License at
|
7
|
-
|
8
|
-
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
-
|
10
|
-
# Unless required by applicable law or agreed to in writing, software
|
11
|
-
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
-
# See the License for the specific language governing permissions and
|
14
|
-
# limitations under the License.
|
15
|
-
|
16
|
-
require 'spec_helper'
|
17
|
-
require "logstash/filters/dateparts"
|
18
|
-
require "logstash/timestamp"
|
19
|
-
require "logstash/event"
|
20
|
-
|
21
|
-
def get_event(contents = {})
|
22
|
-
contents["@timestamp"] = LogStash::Timestamp.new
|
23
|
-
event = LogStash::Event.new(contents)
|
24
|
-
return event
|
25
|
-
end
|
26
|
-
|
27
|
-
describe LogStash::Filters::DateParts do
|
28
|
-
default_ts = "@timestamp"
|
29
|
-
alt_ts_field = "zxlk"
|
30
|
-
|
31
|
-
it "Default config should result in filter with 8 functions, one error tag and @timestamp as the time field" do
|
32
|
-
f = LogStash::Filters::DateParts.new({})
|
33
|
-
|
34
|
-
expect(f.class).to eq(LogStash::Filters::DateParts)
|
35
|
-
expect(f.fields.length).to eq(8)
|
36
|
-
expect(f.time_field).to eq(default_ts)
|
37
|
-
expect(f.error_tags.length).to eq(1)
|
38
|
-
end
|
39
|
-
|
40
|
-
it "Config should result in filter with 2 functions and the alt timestamp field" do
|
41
|
-
f = LogStash::Filters::DateParts.new({
|
42
|
-
"fields" => ["sec", "hour"],
|
43
|
-
"time_field" => alt_ts_field
|
44
|
-
})
|
45
|
-
|
46
|
-
expect(f.class).to eq(LogStash::Filters::DateParts)
|
47
|
-
expect(f.fields.length).to eq(2)
|
48
|
-
expect(f.fields[0]).to eq("sec")
|
49
|
-
expect(f.time_field).to eq(alt_ts_field)
|
50
|
-
end
|
51
|
-
|
52
|
-
it "Should generate the default fields (8 of them)" do
|
53
|
-
event = get_event()
|
54
|
-
count = event.to_hash().count
|
55
|
-
f = LogStash::Filters::DateParts.new({})
|
56
|
-
f.filter(event)
|
57
|
-
|
58
|
-
expect(event.to_hash().count).to eq(count + 8)
|
59
|
-
expect(event.get('sec')).to be_truthy
|
60
|
-
expect(event.get('hour')).to be_truthy
|
61
|
-
expect(event.get('min')).to be_truthy
|
62
|
-
expect(event.get('month')).to be_truthy
|
63
|
-
expect(event.get('year')).to be_truthy
|
64
|
-
expect(event.get('day')).to be_truthy
|
65
|
-
expect(event.get('wday')).to be_truthy
|
66
|
-
expect(event.get('yday')).to be_truthy
|
67
|
-
expect(event.get('tags')).to be_nil
|
68
|
-
end
|
69
|
-
|
70
|
-
it "Should generate only the specified fields" do
|
71
|
-
event = get_event()
|
72
|
-
count = event.to_hash.count
|
73
|
-
f = LogStash::Filters::DateParts.new({
|
74
|
-
"fields" => ["sec", "hour"]
|
75
|
-
})
|
76
|
-
f.filter(event)
|
77
|
-
expect(event.to_hash().count).to eq(count + 2)
|
78
|
-
expect(event.get('sec')).to be_truthy
|
79
|
-
expect(event.get('hour')).to be_truthy
|
80
|
-
expect(event.get('min')).to be_nil
|
81
|
-
expect(event.get('month')).to be_nil
|
82
|
-
expect(event.get('year')).to be_nil
|
83
|
-
expect(event.get('day')).to be_nil
|
84
|
-
expect(event.get('wday')).to be_nil
|
85
|
-
expect(event.get('yday')).to be_nil
|
86
|
-
expect(event.get('tags')).to be_nil
|
87
|
-
end
|
88
|
-
|
89
|
-
it "Should set the error tag on an invalid time field" do
|
90
|
-
event = get_event()
|
91
|
-
count = event.to_hash().count
|
92
|
-
f = LogStash::Filters::DateParts.new({ "time_field" => alt_ts_field })
|
93
|
-
|
94
|
-
f.filter(event)
|
95
|
-
expect(event.get('tags').include? '_dateparts_error').to eq(true)
|
96
|
-
end
|
97
|
-
end
|
1
|
+
# coding: utf-8
|
2
|
+
# Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
3
|
+
|
4
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
+
# you may not use this file except in compliance with the License.
|
6
|
+
# You may obtain a copy of the License at
|
7
|
+
|
8
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
+
|
10
|
+
# Unless required by applicable law or agreed to in writing, software
|
11
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
+
# See the License for the specific language governing permissions and
|
14
|
+
# limitations under the License.
|
15
|
+
|
16
|
+
require 'spec_helper'
|
17
|
+
require "logstash/filters/dateparts"
|
18
|
+
require "logstash/timestamp"
|
19
|
+
require "logstash/event"
|
20
|
+
|
21
|
+
def get_event(contents = {})
|
22
|
+
contents["@timestamp"] = LogStash::Timestamp.new
|
23
|
+
event = LogStash::Event.new(contents)
|
24
|
+
return event
|
25
|
+
end
|
26
|
+
|
27
|
+
describe LogStash::Filters::DateParts do
|
28
|
+
default_ts = "@timestamp"
|
29
|
+
alt_ts_field = "zxlk"
|
30
|
+
|
31
|
+
it "Default config should result in filter with 8 functions, one error tag and @timestamp as the time field" do
|
32
|
+
f = LogStash::Filters::DateParts.new({})
|
33
|
+
|
34
|
+
expect(f.class).to eq(LogStash::Filters::DateParts)
|
35
|
+
expect(f.fields.length).to eq(8)
|
36
|
+
expect(f.time_field).to eq(default_ts)
|
37
|
+
expect(f.error_tags.length).to eq(1)
|
38
|
+
end
|
39
|
+
|
40
|
+
it "Config should result in filter with 2 functions and the alt timestamp field" do
|
41
|
+
f = LogStash::Filters::DateParts.new({
|
42
|
+
"fields" => ["sec", "hour"],
|
43
|
+
"time_field" => alt_ts_field
|
44
|
+
})
|
45
|
+
|
46
|
+
expect(f.class).to eq(LogStash::Filters::DateParts)
|
47
|
+
expect(f.fields.length).to eq(2)
|
48
|
+
expect(f.fields[0]).to eq("sec")
|
49
|
+
expect(f.time_field).to eq(alt_ts_field)
|
50
|
+
end
|
51
|
+
|
52
|
+
it "Should generate the default fields (8 of them)" do
|
53
|
+
event = get_event()
|
54
|
+
count = event.to_hash().count
|
55
|
+
f = LogStash::Filters::DateParts.new({})
|
56
|
+
f.filter(event)
|
57
|
+
|
58
|
+
expect(event.to_hash().count).to eq(count + 8)
|
59
|
+
expect(event.get('sec')).to be_truthy
|
60
|
+
expect(event.get('hour')).to be_truthy
|
61
|
+
expect(event.get('min')).to be_truthy
|
62
|
+
expect(event.get('month')).to be_truthy
|
63
|
+
expect(event.get('year')).to be_truthy
|
64
|
+
expect(event.get('day')).to be_truthy
|
65
|
+
expect(event.get('wday')).to be_truthy
|
66
|
+
expect(event.get('yday')).to be_truthy
|
67
|
+
expect(event.get('tags')).to be_nil
|
68
|
+
end
|
69
|
+
|
70
|
+
it "Should generate only the specified fields" do
|
71
|
+
event = get_event()
|
72
|
+
count = event.to_hash.count
|
73
|
+
f = LogStash::Filters::DateParts.new({
|
74
|
+
"fields" => ["sec", "hour"]
|
75
|
+
})
|
76
|
+
f.filter(event)
|
77
|
+
expect(event.to_hash().count).to eq(count + 2)
|
78
|
+
expect(event.get('sec')).to be_truthy
|
79
|
+
expect(event.get('hour')).to be_truthy
|
80
|
+
expect(event.get('min')).to be_nil
|
81
|
+
expect(event.get('month')).to be_nil
|
82
|
+
expect(event.get('year')).to be_nil
|
83
|
+
expect(event.get('day')).to be_nil
|
84
|
+
expect(event.get('wday')).to be_nil
|
85
|
+
expect(event.get('yday')).to be_nil
|
86
|
+
expect(event.get('tags')).to be_nil
|
87
|
+
end
|
88
|
+
|
89
|
+
it "Should set the error tag on an invalid time field" do
|
90
|
+
event = get_event()
|
91
|
+
count = event.to_hash().count
|
92
|
+
f = LogStash::Filters::DateParts.new({ "time_field" => alt_ts_field })
|
93
|
+
|
94
|
+
f.filter(event)
|
95
|
+
expect(event.get('tags').include? '_dateparts_error').to eq(true)
|
96
|
+
end
|
97
|
+
end
|
data/spec/spec_helper.rb
CHANGED
@@ -1,16 +1,16 @@
|
|
1
|
-
# coding: utf-8
|
2
|
-
# Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
3
|
-
|
4
|
-
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
-
# you may not use this file except in compliance with the License.
|
6
|
-
# You may obtain a copy of the License at
|
7
|
-
|
8
|
-
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
-
|
10
|
-
# Unless required by applicable law or agreed to in writing, software
|
11
|
-
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
-
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
-
# See the License for the specific language governing permissions and
|
14
|
-
# limitations under the License.
|
15
|
-
|
16
|
-
require "logstash/devutils/rspec/spec_helper"
|
1
|
+
# coding: utf-8
|
2
|
+
# Copyright (c) 2014–2015 Mike Baranski <http://www.mikeski.net>
|
3
|
+
|
4
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
+
# you may not use this file except in compliance with the License.
|
6
|
+
# You may obtain a copy of the License at
|
7
|
+
|
8
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
+
|
10
|
+
# Unless required by applicable law or agreed to in writing, software
|
11
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
+
# See the License for the specific language governing permissions and
|
14
|
+
# limitations under the License.
|
15
|
+
|
16
|
+
require "logstash/devutils/rspec/spec_helper"
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: logstash-filter-dateparts
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0.
|
4
|
+
version: 2.0.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Baranski
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2016-09-
|
11
|
+
date: 2016-09-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: logstash-core-plugin-api
|
@@ -82,7 +82,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
82
82
|
version: '0'
|
83
83
|
requirements: []
|
84
84
|
rubyforge_project:
|
85
|
-
rubygems_version: 2.0.14
|
85
|
+
rubygems_version: 2.0.14
|
86
86
|
signing_key:
|
87
87
|
specification_version: 4
|
88
88
|
summary: This dateparts fileter adds date information to your event based on your
|