logstash-filter-geoip 7.2.1-java → 7.2.5-java
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -0
- data/docs/index.asciidoc +116 -4
- data/lib/logstash/filters/geoip.rb +1 -1
- data/logstash-filter-geoip.gemspec +2 -2
- data/spec/filters/geoip_ecs_spec.rb +1 -1
- data/vendor/jar-dependencies/org/logstash/filters/logstash-filter-geoip/6.0.0/logstash-filter-geoip-6.0.0.jar +0 -0
- metadata +5 -6
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5343724bbb8e5c095b58bce550a623e52b430be296389ca4fa96d590ae75e833
|
4
|
+
data.tar.gz: 1c4d5fbc791114fe2ef3fb84fd4c69ea727232e3e60bd4ceed3b2fb36fb1dee4
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: a982a61ac68f2e8a53c480e26652970be26a0773e0c4aaa5ad062e476b957eabeef1d6671db62ba9469cfa39010f650ec1b5d797e6cd09115644013e13b6cf02
|
7
|
+
data.tar.gz: c3163f27b7cdd8bdd57765f5214c816c9e9a93093b0de4a4931e132df92d92ef544835ad6f69c337fa0376cab4044f34f94ccf50180714eb0844b6c1be7e896f
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,16 @@
|
|
1
|
+
## 7.2.5
|
2
|
+
- Added preview of ECS-v8 support with existing ECS-v1 implementation [#193](https://github.com/logstash-plugins/logstash-filter-geoip/pull/193)
|
3
|
+
|
4
|
+
## 7.2.4
|
5
|
+
- Fix: update to Gradle 7 [#191](https://github.com/logstash-plugins/logstash-filter-geoip/pull/191)
|
6
|
+
- [DOC] Clarify CC licensed database indefinite use condition and air-gapped environment [#192](https://github.com/logstash-plugins/logstash-filter-geoip/pull/192)
|
7
|
+
|
8
|
+
## 7.2.3
|
9
|
+
- [DOC] Add documentation for bootstrapping air-gapped environment for database auto-update [#189](https://github.com/logstash-plugins/logstash-filter-geoip/pull/189)
|
10
|
+
|
11
|
+
## 7.2.2
|
12
|
+
- [DOC] Add documentation for database auto-update behavior and database metrics [#187](https://github.com/logstash-plugins/logstash-filter-geoip/pull/187)
|
13
|
+
|
1
14
|
## 7.2.1
|
2
15
|
- Republish the gem due to missing jars in 7.2.0 [#186](https://github.com/logstash-plugins/logstash-filter-geoip/pull/186)
|
3
16
|
|
data/docs/index.asciidoc
CHANGED
@@ -43,9 +43,7 @@ If you would like to get Autonomous System Number(ASN) information, you can use
|
|
43
43
|
https://www.maxmind.com[MaxMind] changed from releasing the GeoIP database under
|
44
44
|
a Creative Commons (CC) license to a proprietary end-user license agreement
|
45
45
|
(EULA). The MaxMind EULA requires Logstash to update the MaxMind database
|
46
|
-
within 30 days of a database update.
|
47
|
-
for 30 days, the geoip filter will stop enriching events in order to maintain compliance.
|
48
|
-
Events will be tagged with `_geoip_expired_database` tag to facilitate the handling of this situation.
|
46
|
+
within 30 days of a database update.
|
49
47
|
|
50
48
|
The GeoIP filter plugin can manage the database for users running the Logstash default
|
51
49
|
distribution, or you can manage
|
@@ -57,6 +55,120 @@ Otherwise, you are responsible for maintaining compliance.
|
|
57
55
|
The Logstash open source distribution uses the MaxMind Creative Commons license
|
58
56
|
database by default.
|
59
57
|
|
58
|
+
[id="plugins-{type}s-{plugin}-database_auto"]
|
59
|
+
==== Database Auto-update
|
60
|
+
|
61
|
+
This plugin bundles Creative Commons (CC) license databases.
|
62
|
+
Logstash checks for database updates every day. It downloads the latest and can replace the old database
|
63
|
+
while the plugin is running.
|
64
|
+
After Logstash downloads EULA license databases, it will not fallback to CC license databases.
|
65
|
+
|
66
|
+
NOTE: If the database has never been updated successfully, as in air-gapped environments, Logstash can use CC license databases indefinitely.
|
67
|
+
|
68
|
+
After Logstash has switched to a EULA licensed database, the geoip filter will
|
69
|
+
stop enriching events in order to maintain compliance if Logstash fails to
|
70
|
+
check for database updates for 30 days.
|
71
|
+
Events will be tagged with `_geoip_expired_database` tag to facilitate the handling of this situation.
|
72
|
+
|
73
|
+
TIP: When possible, allow Logstash to access the internet to download databases so that they are always up-to-date.
|
74
|
+
|
75
|
+
[id="plugins-{type}s-{plugin}-manage_update"]
|
76
|
+
==== Manage your own database updates
|
77
|
+
|
78
|
+
**Use a proxy endpoint**
|
79
|
+
|
80
|
+
If you can't connect directly to the Elastic GeoIP endpoint, consider setting up
|
81
|
+
a secure proxy. You can then specify the proxy endpoint URL in the
|
82
|
+
`xpack.geoip.download.endpoint` setting in `logstash.yml` file.
|
83
|
+
|
84
|
+
**Use a custom endpoint (air-gapped environments)**
|
85
|
+
|
86
|
+
If you work in air-gapped environment and can't update your databases from the Elastic endpoint,
|
87
|
+
You can then download databases from MaxMind and bootstrap the service.
|
88
|
+
|
89
|
+
. Download your `.mmdb` database files from the
|
90
|
+
http://dev.maxmind.com/geoip/geoip2/geolite2[MaxMind site].
|
91
|
+
|
92
|
+
. Copy your database files to a single directory.
|
93
|
+
|
94
|
+
. https://www.elastic.co/downloads/elasticsearch[Download {es}].
|
95
|
+
|
96
|
+
. From your {es} directory, run:
|
97
|
+
+
|
98
|
+
[source,sh]
|
99
|
+
----
|
100
|
+
./bin/elasticsearch-geoip -s my/database/dir
|
101
|
+
----
|
102
|
+
|
103
|
+
. Serve the static database files from your directory. For example, you can use
|
104
|
+
Docker to serve the files from nginx server:
|
105
|
+
+
|
106
|
+
[source,sh]
|
107
|
+
----
|
108
|
+
docker run -p 8080:80 -v my/database/dir:/usr/share/nginx/html:ro nginx
|
109
|
+
----
|
110
|
+
|
111
|
+
. Specify the service's endpoint URL using the
|
112
|
+
`xpack.geoip.download.endpoint=http://localhost:8080/overview.json` setting in `logstash.yml`.
|
113
|
+
|
114
|
+
Logstash gets automatic updates from this service.
|
115
|
+
|
116
|
+
[id="plugins-{type}s-{plugin}-metrics"]
|
117
|
+
==== Database Metrics
|
118
|
+
|
119
|
+
You can monitor database status through the {logstash-ref}/node-stats-api.html#node-stats-api[Node Stats API].
|
120
|
+
|
121
|
+
The following request returns a JSON document containing database manager stats,
|
122
|
+
including:
|
123
|
+
|
124
|
+
* database status and freshness
|
125
|
+
** `geoip_download_manager.database.*.status`
|
126
|
+
*** `init` : initial CC database status
|
127
|
+
*** `up_to_date` : using up-to-date EULA database
|
128
|
+
*** `to_be_expired` : 25 days without calling service
|
129
|
+
*** `expired` : 30 days without calling service
|
130
|
+
** `fail_check_in_days` : number of days Logstash fails to call service since the last success
|
131
|
+
* info about download successes and failures
|
132
|
+
** `geoip_download_manager.download_stats.successes` number of successful checks and downloads
|
133
|
+
** `geoip_download_manager.download_stats.failures` number of failed check or download
|
134
|
+
** `geoip_download_manager.download_stats.status`
|
135
|
+
*** `updating` : check and download at the moment
|
136
|
+
*** `succeeded` : last download succeed
|
137
|
+
*** `failed` : last download failed
|
138
|
+
|
139
|
+
[source,js]
|
140
|
+
--------------------------------------------------
|
141
|
+
curl -XGET 'localhost:9600/_node/stats/geoip_download_manager?pretty'
|
142
|
+
--------------------------------------------------
|
143
|
+
|
144
|
+
Example response:
|
145
|
+
|
146
|
+
[source,js]
|
147
|
+
--------------------------------------------------
|
148
|
+
{
|
149
|
+
"geoip_download_manager" : {
|
150
|
+
"database" : {
|
151
|
+
"ASN" : {
|
152
|
+
"status" : "up_to_date",
|
153
|
+
"fail_check_in_days" : 0,
|
154
|
+
"last_updated_at": "2021-06-21T16:06:54+02:00"
|
155
|
+
},
|
156
|
+
"City" : {
|
157
|
+
"status" : "up_to_date",
|
158
|
+
"fail_check_in_days" : 0,
|
159
|
+
"last_updated_at": "2021-06-21T16:06:54+02:00"
|
160
|
+
}
|
161
|
+
},
|
162
|
+
"download_stats" : {
|
163
|
+
"successes" : 15,
|
164
|
+
"failures" : 1,
|
165
|
+
"last_checked_at" : "2021-06-21T16:07:03+02:00",
|
166
|
+
"status" : "succeeded"
|
167
|
+
}
|
168
|
+
}
|
169
|
+
}
|
170
|
+
--------------------------------------------------
|
171
|
+
|
60
172
|
==== Details
|
61
173
|
|
62
174
|
A `[geoip][location]` field is created if
|
@@ -172,7 +284,7 @@ For the built-in GeoLite2 City database, the following are available:
|
|
172
284
|
* Value type is <<string,string>>
|
173
285
|
* Supported values are:
|
174
286
|
** `disabled`: unstructured geo data added at root level
|
175
|
-
** `v1`: uses fields that are compatible with Elastic Common Schema (for example, `[client][geo][country_name]`)
|
287
|
+
** `v1`, `v8`: uses fields that are compatible with Elastic Common Schema (for example, `[client][geo][country_name]`)
|
176
288
|
* Default value depends on which version of Logstash is running:
|
177
289
|
** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
|
178
290
|
** Otherwise, the default value is `disabled`.
|
@@ -32,7 +32,7 @@ require "logstash/plugin_mixins/ecs_compatibility_support"
|
|
32
32
|
# --
|
33
33
|
|
34
34
|
class LogStash::Filters::GeoIP < LogStash::Filters::Base
|
35
|
-
include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled, :v1)
|
35
|
+
include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled, :v1, :v8 => :v1)
|
36
36
|
|
37
37
|
config_name "geoip"
|
38
38
|
|
@@ -1,7 +1,7 @@
|
|
1
1
|
Gem::Specification.new do |s|
|
2
2
|
|
3
3
|
s.name = 'logstash-filter-geoip'
|
4
|
-
s.version = '7.2.
|
4
|
+
s.version = '7.2.5'
|
5
5
|
s.licenses = ['Apache License (2.0)']
|
6
6
|
s.summary = "Adds geographical information about an IP address"
|
7
7
|
s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"
|
@@ -22,7 +22,7 @@ Gem::Specification.new do |s|
|
|
22
22
|
|
23
23
|
# Gem dependencies
|
24
24
|
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
|
25
|
-
s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.
|
25
|
+
s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.2'
|
26
26
|
s.add_development_dependency 'logstash-devutils'
|
27
27
|
s.add_development_dependency 'insist'
|
28
28
|
s.add_development_dependency 'benchmark-ips'
|
@@ -14,7 +14,7 @@ describe LogStash::Filters::GeoIP do
|
|
14
14
|
describe "simple ip filter", :aggregate_failures do
|
15
15
|
|
16
16
|
context "when specifying the target", :ecs_compatibility_support do
|
17
|
-
ecs_compatibility_matrix(:disabled, :v1) do |ecs_select|
|
17
|
+
ecs_compatibility_matrix(:disabled, :v1, :v8 => :v1) do |ecs_select|
|
18
18
|
|
19
19
|
let(:ip) { "8.8.8.8" }
|
20
20
|
let(:event) { LogStash::Event.new("message" => ip) }
|
Binary file
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: logstash-filter-geoip
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 7.2.
|
4
|
+
version: 7.2.5
|
5
5
|
platform: java
|
6
6
|
authors:
|
7
7
|
- Elastic
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-11-10 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
requirement: !ruby/object:Gem::Requirement
|
@@ -35,7 +35,7 @@ dependencies:
|
|
35
35
|
requirements:
|
36
36
|
- - "~>"
|
37
37
|
- !ruby/object:Gem::Version
|
38
|
-
version: '1.
|
38
|
+
version: '1.2'
|
39
39
|
name: logstash-mixin-ecs_compatibility_support
|
40
40
|
prerelease: false
|
41
41
|
type: :runtime
|
@@ -43,7 +43,7 @@ dependencies:
|
|
43
43
|
requirements:
|
44
44
|
- - "~>"
|
45
45
|
- !ruby/object:Gem::Version
|
46
|
-
version: '1.
|
46
|
+
version: '1.2'
|
47
47
|
- !ruby/object:Gem::Dependency
|
48
48
|
requirement: !ruby/object:Gem::Requirement
|
49
49
|
requirements:
|
@@ -152,8 +152,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
152
152
|
- !ruby/object:Gem::Version
|
153
153
|
version: '0'
|
154
154
|
requirements: []
|
155
|
-
|
156
|
-
rubygems_version: 2.6.13
|
155
|
+
rubygems_version: 3.1.6
|
157
156
|
signing_key:
|
158
157
|
specification_version: 4
|
159
158
|
summary: Adds geographical information about an IP address
|