logstash-filter-ip2location 2.5.0 → 2.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6fb5dfb5ad31e4cea95d7279c2ed8c2ef8c8dca01b930aa359bf0f6e12d33350
4
- data.tar.gz: 369974ed9ef54551651e242c9f12c7743dc4830795a2bb51256d20f241af1ada
3
+ metadata.gz: a61e61abf0196170c710073bf5db2d9e504a892427e1b3e69511570b535345f9
4
+ data.tar.gz: 9bd9a605d5bfa025dea55f1c33058f95b2eae477d3f1f4e4738897c396325fc1
5
5
  SHA512:
6
- metadata.gz: 24fb83c8fd03b6eadc27e7d98e70b2c77c986d1761370f8795b8d31c1d913b35f61d79b9f9bbfe68e25b55ba8052615e6dc35989d91ffaf382fe3cc6127aab34
7
- data.tar.gz: 4ec49c950dc0f02d634b93b5750c00f0990b192f9956aecdf3679308b8b122a15e97fb731a17f6c7794b2038d23670f614c713cb7feb270edb6ec0d415bc07a7
6
+ metadata.gz: c2a29c9f4d10c260a0b68ee771427045fd61ed621aff80d4c800d1164d3978e68fa57d5090911e577afb94c5c46e85a0aefa67a4be3d13c778eaf8005fe8e356
7
+ data.tar.gz: bf2b3e8df1f696d2730f37d71ed733ae65a907555dd7ee38e0a7fbc313de23b7ec56baf66462386804a78f34be76114e10a993233037f38f94acf543907c9165
data/LICENSE CHANGED
@@ -1,13 +1,13 @@
1
- Copyright (c) 2024 IP2Location.com
2
-
3
- Licensed under the Apache License, Version 2.0 (the "License");
4
- you may not use this file except in compliance with the License.
5
- You may obtain a copy of the License at
6
-
7
- http://www.apache.org/licenses/LICENSE-2.0
8
-
9
- Unless required by applicable law or agreed to in writing, software
10
- distributed under the License is distributed on an "AS IS" BASIS,
11
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
- See the License for the specific language governing permissions and
1
+ Copyright (c) 2025 IP2Location.com
2
+
3
+ Licensed under the Apache License, Version 2.0 (the "License");
4
+ you may not use this file except in compliance with the License.
5
+ You may obtain a copy of the License at
6
+
7
+ http://www.apache.org/licenses/LICENSE-2.0
8
+
9
+ Unless required by applicable law or agreed to in writing, software
10
+ distributed under the License is distributed on an "AS IS" BASIS,
11
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
+ See the License for the specific language governing permissions and
13
13
  limitations under the License.
data/README.md CHANGED
@@ -1,161 +1,164 @@
1
- # IP2Location Filter Plugin
2
- This is IP2Location filter plugin for Logstash that enables Logstash's users to add geolocation information such as country, region, district, city, latitude, longitude, ZIP code, time zone, Internet Service Provider (ISP), domain name, connection speed, IDD code, area code, weather station code, weather station name, mobile country code (MCC), mobile network code (MNC), mobile brand, elevation, usage type, address type, IAB category and ASN by IP address. The library reads the geolocation information from **IP2Location BIN data** file and **IP2Location.io** data.
3
-
4
- Supported IPv4 and IPv6 address.
5
-
6
- For the methods to use IP2Location filter plugin with Elastic Stack (Elasticsearch, Filebeat, Logstash, and Kibana), please take a look on this [tutorial](https://www.ip2location.com/tutorials/how-to-use-ip2location-filter-plugin-with-elastic-stack).
7
-
8
- *Note: This plugin works in Logstash 7 and Logstash 8.*
9
-
10
-
11
- ## Dependencies (IP2LOCATION BIN DATA FILE)
12
- This plugin requires IP2Location BIN data file to function. You may download the BIN data file at
13
- * IP2Location LITE BIN Data (Free): https://lite.ip2location.com
14
- * IP2Location Commercial BIN Data (Commercial): https://www.ip2location.com
15
-
16
- ## Dependencies (IP2LOCATION.IO DATA)
17
- This plugin requires API key to function. You may sign up for a free API key at https://www.ip2location.io/pricing.
18
-
19
-
20
- ## Installation
21
- Install this plugin by the following code:
22
- ```
23
- bin/logstash-plugin install logstash-filter-ip2location
24
- ```
25
-
26
-
27
- ## Config File Example 1
28
- ```
29
- input {
30
- beats {
31
- port => "5043"
32
- }
33
- }
34
-
35
- filter {
36
- grok {
37
- match => { "message" => "%{COMBINEDAPACHELOG}"}
38
- }
39
- ip2location {
40
- source => "[source][address]"
41
- }
42
- }
43
-
44
- output {
45
- elasticsearch {
46
- hosts => [ "localhost:9200" ]
47
- }
48
- }
49
- ```
50
-
51
- ## Config File Example 2
52
- ```
53
- input {
54
- beats {
55
- port => "5043"
56
- }
57
- }
58
-
59
- filter {
60
- grok {
61
- match => { "message" => "%{COMBINEDAPACHELOG}"}
62
- }
63
- ip2location {
64
- source => "[source][address]"
65
- # Set path to the database located
66
- database => "IP2LOCATION_BIN_DATABASE_FILESYSTEM_PATH"
67
- # Enable memory mapped to be used
68
- use_memory_mapped => true
69
- }
70
- }
71
-
72
- output {
73
- elasticsearch {
74
- hosts => [ "localhost:9200" ]
75
- }
76
- }
77
- ```
78
-
79
- ## Config File Example 3 using IP2Location.io
80
- ```
81
- input {
82
- beats {
83
- port => "5043"
84
- }
85
- }
86
-
87
- filter {
88
- grok {
89
- match => { "message" => "%{COMBINEDAPACHELOG}"}
90
- }
91
- ip2location {
92
- source => "[source][address]"
93
- lookup_type => "ws"
94
- api_key => "YOUR_API_KEY"
95
- }
96
- }
97
-
98
-
99
- output {
100
- elasticsearch {
101
- hosts => [ "localhost:9200" ]
102
- }
103
- }
104
- ```
105
-
106
-
107
- ## IP2Location Filter Configuration
108
- |Setting|Input type|Required|
109
- |---|---|---|
110
- |source|string|Yes|
111
- |database|a valid filesystem path|No|
112
- |use_memory_mapped|boolean|No|
113
- |use_cache|boolean|No|
114
- |lookup_type|string|No|
115
- |api_key|string|No|
116
- |hide_unsupported_fields|boolean|No|
117
-
118
- * **source** field is a required setting that containing the IP address or hostname to get the ip information.
119
- * **database** field is an optional setting that containing the path to the IP2Location BIN database file.
120
- * **use_memory_mapped** field is an optional setting that used to allow user to enable the use of memory mapped file. Default value is false.
121
- * **use_cache** field is an optional setting that used to allow user to enable the use of cache. Default value is true.
122
- * **lookup_type** field is an optional setting that used to allow user to decide the lookup method either using IP2Location BIN database file(db) or IP2Location.io data(ws). Default value is db.
123
- * **api_key** field is an optional setting that used to allow user to set the API Key of the IP2Location.io lookup.
124
- * **hide_unsupported_fields** field is an optional setting that used to allow user to hide unsupported fields. Default value is false.
125
-
126
-
127
- ## Sample Output
128
- |Field|Description|
129
- |---|---|
130
- |ip2location.address_type|the IP address type (A-Anycast, B-Broadcast, M-Multicast & U-Unicast) of IP address or domain name|
131
- |ip2location.area_code|the varying length number assigned to geographic areas for call between cities|
132
- |ip2location.as|Autonomous system (AS) name|
133
- |ip2location.asn|the Autonomous system number (ASN)|
134
- |ip2location.category|the IAB content taxonomy category of IP address or domain name|
135
- |ip2location.city|the city name|
136
- |ip2location.country_long|the country name based on ISO 3166|
137
- |ip2location.country_short|the two-character country code based on ISO 3166|
138
- |ip2location.district|the district or county name|
139
- |ip2location.domain|the Internet domain name associated to IP address range|
140
- |ip2location.elevation|the elevation|
141
- |ip2location.idd_code|the IDD prefix to call the city from another country|
142
- |ip2location.ip_address|the IP address|
143
- |ip2location.isp|the Internet Service Provider (ISP) name|
144
- |ip2location.latitude|the city latitude|
145
- |ip2location.location|the city location|
146
- |ip2location.longitude|the city longitude|
147
- |ip2location.mcc|the mobile country code|
148
- |ip2location.mnc|mobile network code|
149
- |ip2location.mobile_brand|the mobile brand|
150
- |ip2location.net_speed|the Internet Connection Speed (DIAL) DIAL-UP,(DSL) DSL/CABLE or(COMP) COMPANY|
151
- |ip2location.region|the region or state name|
152
- |ip2location.time_zone|the Time zone in UTC (Coordinated Universal Time)|
153
- |ip2location.usage_type|the usage type|
154
- |ip2location.weather_station_code|the special code to identify the nearest weather observation station|
155
- |ip2location.weather_station_name|the name of the nearest weather observation station|
156
- |ip2location.zip_code|the ZIP code|
157
-
158
-
159
- ## Support
160
- Email: support@ip2location.com
161
- URL: [https://www.ip2location.com](https://www.ip2location.com)
1
+ # IP2Location Filter Plugin
2
+ This is IP2Location filter plugin for Logstash that enables Logstash's users to add geolocation information such as country, region, district, city, latitude, longitude, ZIP code, time zone, Internet Service Provider (ISP), domain name, connection speed, IDD code, area code, weather station code, weather station name, mobile country code (MCC), mobile network code (MNC), mobile brand, elevation, usage type, address type, IAB category, ASN, AS domain, AS usage type and AS CIDR by IP address. The library reads the geolocation information from **IP2Location BIN data** file and **IP2Location.io** data.
3
+
4
+ Supported IPv4 and IPv6 address.
5
+
6
+ For the methods to use IP2Location filter plugin with Elastic Stack (Elasticsearch, Filebeat, Logstash, and Kibana), please take a look on this [tutorial](https://www.ip2location.com/tutorials/how-to-use-ip2location-filter-plugin-with-elastic-stack).
7
+
8
+ *Note: This plugin works in Logstash 7, 8 and 9.*
9
+
10
+
11
+ ## Dependencies (IP2LOCATION BIN DATA FILE)
12
+ This plugin requires IP2Location BIN data file to function. You may download the BIN data file at
13
+ * IP2Location LITE BIN Data (Free): https://lite.ip2location.com
14
+ * IP2Location Commercial BIN Data (Commercial): https://www.ip2location.com
15
+
16
+ ## Dependencies (IP2LOCATION.IO DATA)
17
+ This plugin requires API key to function. You may sign up for a free API key at https://www.ip2location.io/pricing.
18
+
19
+
20
+ ## Installation
21
+ Install this plugin by the following code:
22
+ ```
23
+ bin/logstash-plugin install logstash-filter-ip2location
24
+ ```
25
+
26
+
27
+ ## Config File Example 1
28
+ ```
29
+ input {
30
+ beats {
31
+ port => "5043"
32
+ }
33
+ }
34
+
35
+ filter {
36
+ grok {
37
+ match => { "message" => "%{COMBINEDAPACHELOG}"}
38
+ }
39
+ ip2location {
40
+ source => "[source][address]"
41
+ }
42
+ }
43
+
44
+ output {
45
+ elasticsearch {
46
+ hosts => [ "localhost:9200" ]
47
+ }
48
+ }
49
+ ```
50
+
51
+ ## Config File Example 2
52
+ ```
53
+ input {
54
+ beats {
55
+ port => "5043"
56
+ }
57
+ }
58
+
59
+ filter {
60
+ grok {
61
+ match => { "message" => "%{COMBINEDAPACHELOG}"}
62
+ }
63
+ ip2location {
64
+ source => "[source][address]"
65
+ # Set path to the database located
66
+ database => "IP2LOCATION_BIN_DATABASE_FILESYSTEM_PATH"
67
+ # Enable memory mapped to be used
68
+ use_memory_mapped => true
69
+ }
70
+ }
71
+
72
+ output {
73
+ elasticsearch {
74
+ hosts => [ "localhost:9200" ]
75
+ }
76
+ }
77
+ ```
78
+
79
+ ## Config File Example 3 using IP2Location.io
80
+ ```
81
+ input {
82
+ beats {
83
+ port => "5043"
84
+ }
85
+ }
86
+
87
+ filter {
88
+ grok {
89
+ match => { "message" => "%{COMBINEDAPACHELOG}"}
90
+ }
91
+ ip2location {
92
+ source => "[source][address]"
93
+ lookup_type => "ws"
94
+ api_key => "YOUR_API_KEY"
95
+ }
96
+ }
97
+
98
+
99
+ output {
100
+ elasticsearch {
101
+ hosts => [ "localhost:9200" ]
102
+ }
103
+ }
104
+ ```
105
+
106
+
107
+ ## IP2Location Filter Configuration
108
+ |Setting|Input type|Required|
109
+ |---|---|---|
110
+ |source|string|Yes|
111
+ |database|a valid filesystem path|No|
112
+ |use_memory_mapped|boolean|No|
113
+ |use_cache|boolean|No|
114
+ |lookup_type|string|No|
115
+ |api_key|string|No|
116
+ |hide_unsupported_fields|boolean|No|
117
+
118
+ * **source** field is a required setting that containing the IP address or hostname to get the ip information.
119
+ * **database** field is an optional setting that containing the path to the IP2Location BIN database file.
120
+ * **use_memory_mapped** field is an optional setting that used to allow user to enable the use of memory mapped file. Default value is false.
121
+ * **use_cache** field is an optional setting that used to allow user to enable the use of cache. Default value is true.
122
+ * **lookup_type** field is an optional setting that used to allow user to decide the lookup method either using IP2Location BIN database file(db) or IP2Location.io data(ws). Default value is db.
123
+ * **api_key** field is an optional setting that used to allow user to set the API Key of the IP2Location.io lookup.
124
+ * **hide_unsupported_fields** field is an optional setting that used to allow user to hide unsupported fields. Default value is false.
125
+
126
+
127
+ ## Sample Output
128
+ |Field|Description|
129
+ |---|---|
130
+ |ip2location.address_type|the IP address type (A-Anycast, B-Broadcast, M-Multicast & U-Unicast) of IP address or domain name|
131
+ |ip2location.area_code|the varying length number assigned to geographic areas for call between cities|
132
+ |ip2location.as|Autonomous system (AS) name|
133
+ |ip2location.asn|the Autonomous system number (ASN)|
134
+ |ip2location.as_cidr|CIDR range for the whole AS|
135
+ |ip2location.as_domain|Domain name of the AS registrant|
136
+ |ip2location.as_usage_type|Usage type of the AS registrant|
137
+ |ip2location.category|the IAB content taxonomy category of IP address or domain name|
138
+ |ip2location.city|the city name|
139
+ |ip2location.country_long|the country name based on ISO 3166|
140
+ |ip2location.country_short|the two-character country code based on ISO 3166|
141
+ |ip2location.district|the district or county name|
142
+ |ip2location.domain|the Internet domain name associated to IP address range|
143
+ |ip2location.elevation|the elevation|
144
+ |ip2location.idd_code|the IDD prefix to call the city from another country|
145
+ |ip2location.ip_address|the IP address|
146
+ |ip2location.isp|the Internet Service Provider (ISP) name|
147
+ |ip2location.latitude|the city latitude|
148
+ |ip2location.location|the city location|
149
+ |ip2location.longitude|the city longitude|
150
+ |ip2location.mcc|the mobile country code|
151
+ |ip2location.mnc|mobile network code|
152
+ |ip2location.mobile_brand|the mobile brand|
153
+ |ip2location.net_speed|the Internet Connection Speed (DIAL) DIAL-UP,(DSL) DSL/CABLE or(COMP) COMPANY|
154
+ |ip2location.region|the region or state name|
155
+ |ip2location.time_zone|the Time zone in UTC (Coordinated Universal Time)|
156
+ |ip2location.usage_type|the usage type|
157
+ |ip2location.weather_station_code|the special code to identify the nearest weather observation station|
158
+ |ip2location.weather_station_name|the name of the nearest weather observation station|
159
+ |ip2location.zip_code|the ZIP code|
160
+
161
+
162
+ ## Support
163
+ Email: support@ip2location.com
164
+ URL: [https://www.ip2location.com](https://www.ip2location.com)
@@ -1,195 +1,195 @@
1
- # encoding: utf-8
2
- require "logstash/filters/base"
3
- require "logstash/namespace"
4
-
5
- require "logstash-filter-ip2location_jars"
6
-
7
- require 'thread'
8
-
9
-
10
- class LogStash::Filters::IP2Location < LogStash::Filters::Base
11
- config_name "ip2location"
12
-
13
- # The path to the IP2Location.BIN database file which Logstash should use.
14
- # If not specified, this will default to the IP2LOCATION-LITE-DB1.IPV6.BIN database that embedded in the plugin.
15
- config :database, :validate => :path
16
-
17
- # The field containing the IP address.
18
- # If this field is an array, only the first value will be used.
19
- config :source, :validate => :string, :required => true
20
-
21
- # The field used to define ip2location as target.
22
- config :target, :validate => :string, :default => 'ip2location'
23
-
24
- # The field used to allow user to enable the use of cache.
25
- config :use_cache, :validate => :boolean, :default => true
26
-
27
- # The field used to allow user to enable the use of memory mapped file.
28
- config :use_memory_mapped, :validate => :boolean, :default => false
29
-
30
- # The field used to allow user to hide unsupported fields.
31
- config :hide_unsupported_fields, :validate => :boolean, :default => false
32
-
33
- # The field used to define lookup type.
34
- config :lookup_type, :validate => :string, :default => 'db'
35
-
36
- # The field used to define the apikey of IP2location.io.
37
- config :api_key, :validate => :string, :default => ''
38
-
39
- # The field used to define the size of the cache. It is not required and the default value is 10 000
40
- config :cache_size, :validate => :number, :required => false, :default => 10_000
41
-
42
- public
43
- def register
44
- if @lookup_type == "ws"
45
- @logger.info("Using IP2Location.io API")
46
- if @api_key == ""
47
- raise "An IP2Location.io API key is required. You may sign up for a free API key at https://www.ip2location.io/pricing."
48
- end
49
- else
50
- if @database.nil?
51
- @database = ::Dir.glob(::File.join(::File.expand_path("../../../vendor/", ::File.dirname(__FILE__)),"IP2LOCATION-LITE-DB1.IPV6.BIN")).first
52
-
53
- if @database.nil? || !File.exists?(@database)
54
- raise "You must specify 'database => ...' in your ip2location filter (I looked for '#{@database}')"
55
- end
56
- end
57
- @logger.info("Using ip2location database", :path => @database)
58
- end
59
-
60
- @ip2locationfilter = org.logstash.filters.IP2LocationFilter.new(@source, @target, @database, @use_memory_mapped, @hide_unsupported_fields, @lookup_type, @api_key)
61
- end
62
-
63
- public
64
- def filter(event)
65
- ip = event.get(@source)
66
-
67
- return unless filter?(event)
68
- if @lookup_type == "ws"
69
- if @ip2locationfilter.handleEvent(event)
70
- filter_matched(event)
71
- else
72
- tag_iplookup_unsuccessful(event)
73
- end
74
- else
75
- if @use_cache
76
- if value = IP2LocationCache.find(event, ip, @ip2locationfilter, @cache_size).get('ip2location')
77
- event.set('ip2location', value)
78
- filter_matched(event)
79
- else
80
- tag_iplookup_unsuccessful(event)
81
- end
82
- else
83
- if @ip2locationfilter.handleEvent(event)
84
- filter_matched(event)
85
- else
86
- tag_iplookup_unsuccessful(event)
87
- end
88
- end
89
- end
90
- end
91
-
92
- def tag_iplookup_unsuccessful(event)
93
- @logger.debug? && @logger.debug("IP #{event.get(@source)} was not found in the database", :event => event)
94
- end
95
-
96
- end # class LogStash::Filters::IP2Location
97
-
98
- class IP2LocationOrderedHash
99
- ONE = 1
100
-
101
- attr_reader :times_queried # ip -> times queried
102
- attr_reader :hash
103
-
104
- def initialize
105
- @times_queried = Hash.new(0) # ip -> times queried
106
- @hash = {} # number of hits -> array of ips
107
- end
108
-
109
- def add(key)
110
- hash[ONE] ||= []
111
- hash[ONE] << key
112
- times_queried[key] = ONE
113
- end
114
-
115
- def reorder(key)
116
- number_of_queries = times_queried[key]
117
-
118
- hash[number_of_queries].delete(key)
119
- hash.delete(number_of_queries) if hash[number_of_queries].empty?
120
-
121
- hash[number_of_queries + 1] ||= []
122
- hash[number_of_queries + 1] << key
123
- end
124
-
125
- def increment(key)
126
- add(key) unless times_queried.has_key?(key)
127
- reorder(key)
128
- times_queried[key] += 1
129
- end
130
-
131
- def delete_least_used
132
- first_pile_with_someting.shift.tap { |key| times_queried.delete(key) }
133
- end
134
-
135
- def first_pile_with_someting
136
- hash[hash.keys.min]
137
- end
138
- end
139
-
140
- class IP2LocationCache
141
- ONE_DAY_IN_SECONDS = 86_400
142
-
143
- @cache = {} # ip -> event
144
- @timestamps = {} # ip -> time of caching
145
- @times_queried = IP2LocationOrderedHash.new # ip -> times queried
146
- @mutex = Mutex.new
147
-
148
- class << self
149
- attr_reader :cache
150
- attr_reader :timestamps
151
- attr_reader :times_queried
152
-
153
-
154
- def find(event, ip, filter, cache_size)
155
- synchronize do
156
- if cache.has_key?(ip)
157
- refresh_event(event, ip, filter) if too_old?(ip)
158
- else
159
- if cache_full?(cache_size)
160
- make_room
161
- end
162
- cache_event(event, ip, filter)
163
- end
164
- times_queried.increment(ip)
165
- cache[ip]
166
- end
167
- end
168
-
169
- def too_old?(ip)
170
- timestamps[ip] < Time.now - ONE_DAY_IN_SECONDS
171
- end
172
-
173
- def make_room
174
- key = times_queried.delete_least_used
175
- cache.delete(key)
176
- timestamps.delete(key)
177
- end
178
-
179
- def cache_full?(cache_size)
180
- cache.size >= cache_size
181
- end
182
-
183
- def cache_event(event, ip, filter)
184
- filter.handleEvent(event)
185
- cache[ip] = event
186
- timestamps[ip] = Time.now
187
- end
188
-
189
- def synchronize(&block)
190
- @mutex.synchronize(&block)
191
- end
192
-
193
- alias_method :refresh_event, :cache_event
194
- end
195
- end
1
+ # encoding: utf-8
2
+ require "logstash/filters/base"
3
+ require "logstash/namespace"
4
+
5
+ require "logstash-filter-ip2location_jars"
6
+
7
+ require 'thread'
8
+
9
+
10
+ class LogStash::Filters::IP2Location < LogStash::Filters::Base
11
+ config_name "ip2location"
12
+
13
+ # The path to the IP2Location.BIN database file which Logstash should use.
14
+ # If not specified, this will default to the IP2LOCATION-LITE-DB1.IPV6.BIN database that embedded in the plugin.
15
+ config :database, :validate => :path
16
+
17
+ # The field containing the IP address.
18
+ # If this field is an array, only the first value will be used.
19
+ config :source, :validate => :string, :required => true
20
+
21
+ # The field used to define ip2location as target.
22
+ config :target, :validate => :string, :default => 'ip2location'
23
+
24
+ # The field used to allow user to enable the use of cache.
25
+ config :use_cache, :validate => :boolean, :default => true
26
+
27
+ # The field used to allow user to enable the use of memory mapped file.
28
+ config :use_memory_mapped, :validate => :boolean, :default => false
29
+
30
+ # The field used to allow user to hide unsupported fields.
31
+ config :hide_unsupported_fields, :validate => :boolean, :default => false
32
+
33
+ # The field used to define lookup type.
34
+ config :lookup_type, :validate => :string, :default => 'db'
35
+
36
+ # The field used to define the apikey of IP2location.io.
37
+ config :api_key, :validate => :string, :default => ''
38
+
39
+ # The field used to define the size of the cache. It is not required and the default value is 10 000
40
+ config :cache_size, :validate => :number, :required => false, :default => 10_000
41
+
42
+ public
43
+ def register
44
+ if @lookup_type == "ws"
45
+ @logger.info("Using IP2Location.io API")
46
+ if @api_key == ""
47
+ raise "An IP2Location.io API key is required. You may sign up for a free API key at https://www.ip2location.io/pricing."
48
+ end
49
+ else
50
+ if @database.nil?
51
+ @database = ::Dir.glob(::File.join(::File.expand_path("../../../vendor/", ::File.dirname(__FILE__)),"IP2LOCATION-LITE-DB1.IPV6.BIN")).first
52
+
53
+ if @database.nil? || !File.exists?(@database)
54
+ raise "You must specify 'database => ...' in your ip2location filter (I looked for '#{@database}')"
55
+ end
56
+ end
57
+ @logger.info("Using ip2location database", :path => @database)
58
+ end
59
+
60
+ @ip2locationfilter = org.logstash.filters.IP2LocationFilter.new(@source, @target, @database, @use_memory_mapped, @hide_unsupported_fields, @lookup_type, @api_key)
61
+ end
62
+
63
+ public
64
+ def filter(event)
65
+ ip = event.get(@source)
66
+
67
+ return unless filter?(event)
68
+ if @lookup_type == "ws"
69
+ if @ip2locationfilter.handleEvent(event)
70
+ filter_matched(event)
71
+ else
72
+ tag_iplookup_unsuccessful(event)
73
+ end
74
+ else
75
+ if @use_cache
76
+ if value = IP2LocationCache.find(event, ip, @ip2locationfilter, @cache_size).get('ip2location')
77
+ event.set('ip2location', value)
78
+ filter_matched(event)
79
+ else
80
+ tag_iplookup_unsuccessful(event)
81
+ end
82
+ else
83
+ if @ip2locationfilter.handleEvent(event)
84
+ filter_matched(event)
85
+ else
86
+ tag_iplookup_unsuccessful(event)
87
+ end
88
+ end
89
+ end
90
+ end
91
+
92
+ def tag_iplookup_unsuccessful(event)
93
+ @logger.debug? && @logger.debug("IP #{event.get(@source)} was not found in the database", :event => event)
94
+ end
95
+
96
+ end # class LogStash::Filters::IP2Location
97
+
98
+ class IP2LocationOrderedHash
99
+ ONE = 1
100
+
101
+ attr_reader :times_queried # ip -> times queried
102
+ attr_reader :hash
103
+
104
+ def initialize
105
+ @times_queried = Hash.new(0) # ip -> times queried
106
+ @hash = {} # number of hits -> array of ips
107
+ end
108
+
109
+ def add(key)
110
+ hash[ONE] ||= []
111
+ hash[ONE] << key
112
+ times_queried[key] = ONE
113
+ end
114
+
115
+ def reorder(key)
116
+ number_of_queries = times_queried[key]
117
+
118
+ hash[number_of_queries].delete(key)
119
+ hash.delete(number_of_queries) if hash[number_of_queries].empty?
120
+
121
+ hash[number_of_queries + 1] ||= []
122
+ hash[number_of_queries + 1] << key
123
+ end
124
+
125
+ def increment(key)
126
+ add(key) unless times_queried.has_key?(key)
127
+ reorder(key)
128
+ times_queried[key] += 1
129
+ end
130
+
131
+ def delete_least_used
132
+ first_pile_with_someting.shift.tap { |key| times_queried.delete(key) }
133
+ end
134
+
135
+ def first_pile_with_someting
136
+ hash[hash.keys.min]
137
+ end
138
+ end
139
+
140
+ class IP2LocationCache
141
+ ONE_DAY_IN_SECONDS = 86_400
142
+
143
+ @cache = {} # ip -> event
144
+ @timestamps = {} # ip -> time of caching
145
+ @times_queried = IP2LocationOrderedHash.new # ip -> times queried
146
+ @mutex = Mutex.new
147
+
148
+ class << self
149
+ attr_reader :cache
150
+ attr_reader :timestamps
151
+ attr_reader :times_queried
152
+
153
+
154
+ def find(event, ip, filter, cache_size)
155
+ synchronize do
156
+ if cache.has_key?(ip)
157
+ refresh_event(event, ip, filter) if too_old?(ip)
158
+ else
159
+ if cache_full?(cache_size)
160
+ make_room
161
+ end
162
+ cache_event(event, ip, filter)
163
+ end
164
+ times_queried.increment(ip)
165
+ cache[ip]
166
+ end
167
+ end
168
+
169
+ def too_old?(ip)
170
+ timestamps[ip] < Time.now - ONE_DAY_IN_SECONDS
171
+ end
172
+
173
+ def make_room
174
+ key = times_queried.delete_least_used
175
+ cache.delete(key)
176
+ timestamps.delete(key)
177
+ end
178
+
179
+ def cache_full?(cache_size)
180
+ cache.size >= cache_size
181
+ end
182
+
183
+ def cache_event(event, ip, filter)
184
+ filter.handleEvent(event)
185
+ cache[ip] = event
186
+ timestamps[ip] = Time.now
187
+ end
188
+
189
+ def synchronize(&block)
190
+ @mutex.synchronize(&block)
191
+ end
192
+
193
+ alias_method :refresh_event, :cache_event
194
+ end
195
+ end
@@ -1,4 +1,4 @@
1
- require 'jar_dependencies'
2
- require_jar('com.ip2location.ip2location', 'ip2location', '8.11.0')
3
- require_jar('com.google.gson', 'gson', '2.11.0')
4
- require_jar('org.logstash.filters', 'logstash-filter-ip2location', '2.5.0')
1
+ require 'jar_dependencies'
2
+ require_jar('com.ip2location.ip2location', 'ip2location', '8.13.0')
3
+ require_jar('com.google.gson', 'gson', '2.11.0')
4
+ require_jar('org.logstash.filters', 'logstash-filter-ip2location', '2.6.0')
@@ -1,25 +1,25 @@
1
- Gem::Specification.new do |s|
2
-
3
- s.name = 'logstash-filter-ip2location'
4
- s.version = '2.5.0'
5
- s.licenses = ['Apache-2.0']
6
- s.summary = "Logstash filter IP2Location"
7
- s.description = "IP2Location filter plugin for Logstash enables Logstash's users to add geolocation information such as country, state, district, city, latitude, longitude, ZIP code, time zone, ISP, domain name, connection speed, IDD code, area code, weather station code, weather station name, MNC, MCC, mobile brand, elevation, usage type, address type, IAB category and ASN by IP address."
8
- s.authors = ["IP2Location"]
9
- s.email = 'support@ip2location.com'
10
- s.homepage = "https://www.ip2location.com"
11
- s.require_paths = ["lib", "vendor/jar-dependencies"]
12
-
13
- # Files
14
- s.files = Dir["lib/**/*",'spec/**/*',"vendor/**/*","vendor/jar-dependencies/**/*.jar","*.gemspec","*.md","Gemfile","LICENSE"]
15
-
16
- # Tests
17
- s.test_files = s.files.grep(%r{^(test|spec|features)/})
18
-
19
- # Special flag to let us know this is actually a logstash plugin
20
- s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
21
-
22
- # Gem dependencies
23
- s.add_runtime_dependency "logstash-core-plugin-api", "~> 2.0"
24
- s.add_development_dependency "logstash-devutils"
25
- end
1
+ Gem::Specification.new do |s|
2
+
3
+ s.name = 'logstash-filter-ip2location'
4
+ s.version = '2.6.0'
5
+ s.licenses = ['Apache-2.0']
6
+ s.summary = "Logstash filter IP2Location"
7
+ s.description = "IP2Location filter plugin for Logstash enables Logstash's users to add geolocation information such as country, state, district, city, latitude, longitude, ZIP code, time zone, ISP, domain name, connection speed, IDD code, area code, weather station code, weather station name, MNC, MCC, mobile brand, elevation, usage type, address type, IAB category, ASN, AS domain, AS usage type and AS CIDR by IP address."
8
+ s.authors = ["IP2Location"]
9
+ s.email = 'support@ip2location.com'
10
+ s.homepage = "https://www.ip2location.com"
11
+ s.require_paths = ["lib", "vendor/jar-dependencies"]
12
+
13
+ # Files
14
+ s.files = Dir["lib/**/*",'spec/**/*',"vendor/**/*","vendor/jar-dependencies/**/*.jar","*.gemspec","*.md","Gemfile","LICENSE"]
15
+
16
+ # Tests
17
+ s.test_files = s.files.grep(%r{^(test|spec|features)/})
18
+
19
+ # Special flag to let us know this is actually a logstash plugin
20
+ s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
21
+
22
+ # Gem dependencies
23
+ s.add_runtime_dependency "logstash-core-plugin-api", "~> 2.0"
24
+ s.add_development_dependency "logstash-devutils"
25
+ end
metadata CHANGED
@@ -1,14 +1,13 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logstash-filter-ip2location
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.5.0
4
+ version: 2.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - IP2Location
8
- autorequire:
9
8
  bindir: bin
10
9
  cert_chain: []
11
- date: 2024-10-18 00:00:00.000000000 Z
10
+ date: 1980-01-02 00:00:00.000000000 Z
12
11
  dependencies:
13
12
  - !ruby/object:Gem::Dependency
14
13
  name: logstash-core-plugin-api
@@ -42,7 +41,7 @@ description: IP2Location filter plugin for Logstash enables Logstash's users to
42
41
  geolocation information such as country, state, district, city, latitude, longitude,
43
42
  ZIP code, time zone, ISP, domain name, connection speed, IDD code, area code, weather
44
43
  station code, weather station name, MNC, MCC, mobile brand, elevation, usage type,
45
- address type, IAB category and ASN by IP address.
44
+ address type, IAB category, ASN, AS domain, AS usage type and AS CIDR by IP address.
46
45
  email: support@ip2location.com
47
46
  executables: []
48
47
  extensions: []
@@ -58,15 +57,14 @@ files:
58
57
  - spec/spec_helper.rb
59
58
  - vendor/IP2LOCATION-LITE-DB1.IPV6.BIN
60
59
  - vendor/jar-dependencies/com/google/gson/gson/2.11.0/gson-2.11.0.jar
61
- - vendor/jar-dependencies/com/ip2location/ip2location/ip2location/8.11.0/ip2location-8.11.0.jar
62
- - vendor/jar-dependencies/org/logstash/filters/logstash-filter-ip2location/2.5.0/logstash-filter-ip2location-2.5.0.jar
60
+ - vendor/jar-dependencies/com/ip2location/ip2location/ip2location/8.13.0/ip2location-8.13.0.jar
61
+ - vendor/jar-dependencies/org/logstash/filters/logstash-filter-ip2location/2.6.0/logstash-filter-ip2location-2.6.0.jar
63
62
  homepage: https://www.ip2location.com
64
63
  licenses:
65
64
  - Apache-2.0
66
65
  metadata:
67
66
  logstash_plugin: 'true'
68
67
  logstash_group: filter
69
- post_install_message:
70
68
  rdoc_options: []
71
69
  require_paths:
72
70
  - lib
@@ -82,8 +80,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
82
80
  - !ruby/object:Gem::Version
83
81
  version: '0'
84
82
  requirements: []
85
- rubygems_version: 3.5.18
86
- signing_key:
83
+ rubygems_version: 3.7.1
87
84
  specification_version: 4
88
85
  summary: Logstash filter IP2Location
89
86
  test_files: