fluent-plugin-geoip 1.0.0 → 1.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/.github/ISSUE_TEMPLATE.md +19 -0
- data/.travis.yml +1 -1
- data/README.md +203 -239
- data/docker-compose.yml +12 -5
- data/dockerfiles/Dockerfile-ruby2.1 +1 -1
- data/dockerfiles/Dockerfile-ruby2.2 +1 -1
- data/dockerfiles/Dockerfile-ruby2.3 +1 -1
- data/dockerfiles/Dockerfile-ruby2.4 +1 -1
- data/dockerfiles/Dockerfile-ruby2.5 +8 -0
- data/fluent-plugin-geoip.gemspec +2 -1
- data/lib/fluent/plugin/filter_geoip.rb +196 -11
- data/test/helper.rb +1 -0
- data/test/plugin/test_filter_geoip.rb +296 -107
- data/utils/dump.rb +27 -0
- metadata +20 -8
- data/lib/fluent/plugin/geoip.rb +0 -174
- data/lib/fluent/plugin/out_geoip.rb +0 -51
- data/test/plugin/test_out_geoip.rb +0 -1076
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 35bca82d01d71c2eed7db735cf4e2394e6af0ec420baa511cfe74431fd0c88c4
|
4
|
+
data.tar.gz: e15c7f9fec5855d0012d97a65d0aa170338a169229e65eea5baf798c4d2d2763
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c67641b18a5254169fa4c0d0fa7b32e1612c2f65c6f7ebfd5afffeaaa7c1f5a1ffa1e77334e10a7f5cecf476e0dbd6b7d64f080d93894b4b0cf8f7c4092b141d
|
7
|
+
data.tar.gz: c7e7fa0d170d4f451f9666bf471a76b193a7f0e466403992b85cb6051bc72b1a94e6f54ffe378cc5414d8cdf251a02402fa26c01507e58baf2b2db313690e4d3
|
@@ -0,0 +1,19 @@
|
|
1
|
+
#### Problem
|
2
|
+
|
3
|
+
...
|
4
|
+
|
5
|
+
#### Steps to replicate
|
6
|
+
|
7
|
+
Provide example config and message
|
8
|
+
|
9
|
+
#### Expected Behavior
|
10
|
+
|
11
|
+
...
|
12
|
+
|
13
|
+
#### Your environment
|
14
|
+
|
15
|
+
* OS version
|
16
|
+
* paste result of ``fluentd --version`` or ``td-agent --version``
|
17
|
+
* plugin version
|
18
|
+
* paste boot log of fluentd or td-agent
|
19
|
+
* paste result of ``fluent-gem list``, ``td-agent-gem list`` or your Gemfile.lock
|
data/.travis.yml
CHANGED
@@ -4,7 +4,7 @@ services:
|
|
4
4
|
|
5
5
|
env:
|
6
6
|
global:
|
7
|
-
DOCKER_COMPOSE_VERSION: 1.
|
7
|
+
DOCKER_COMPOSE_VERSION: 1.18.0
|
8
8
|
matrix:
|
9
9
|
- TARGET_RUBY_VERSION=2.1 BUNDLE_GEMFILE=/app/gemfiles/fluentd_v1.0_with_backport_dig.gemfile
|
10
10
|
- TARGET_RUBY_VERSION=2.2 BUNDLE_GEMFILE=/app/gemfiles/fluentd_v1.0_with_backport_dig.gemfile
|
data/README.md
CHANGED
@@ -1,8 +1,8 @@
|
|
1
1
|
# fluent-plugin-geoip [](https://travis-ci.org/y-ken/fluent-plugin-geoip)
|
2
2
|
|
3
|
-
Fluentd
|
3
|
+
Fluentd Filter plugin to add information about geographical location of IP addresses with Maxmind GeoIP databases.
|
4
4
|
|
5
|
-
fluent-plugin-geoip has bundled cost-free [GeoLite City database](http://dev.maxmind.com/geoip/legacy/geolite/) by default.<br />
|
5
|
+
fluent-plugin-geoip has bundled cost-free [GeoLite2 Free Downloadable Databases](https://dev.maxmind.com/geoip/geoip2/geolite2/) and [GeoLite City database](http://dev.maxmind.com/geoip/legacy/geolite/) by default.<br />
|
6
6
|
Also you can use purchased [GeoIP City database](http://www.maxmind.com/en/city) ([lang:ja](http://www.maxmind.com/ja/city)) which costs starting from $50.
|
7
7
|
|
8
8
|
The accuracy details for GeoLite City (free) and GeoIP City (purchased) has described at the page below.
|
@@ -21,56 +21,7 @@ If you want to use this plugin with Fluentd v0.12.x or earlier use 0.8.x.
|
|
21
21
|
|
22
22
|
### Compatibility notice
|
23
23
|
|
24
|
-
We've
|
25
|
-
So we have dropped some features.
|
26
|
-
|
27
|
-
See also [official document](http://docs.fluentd.org/v1.0/articles/plugin-update-from-v12)
|
28
|
-
|
29
|
-
#### Fluent::Mixin::RewriteTagName
|
30
|
-
|
31
|
-
* `${tag}`, `__TAG__`
|
32
|
-
|
33
|
-
Alternative: Use `${tag}` placeholder
|
34
|
-
|
35
|
-
* `${tag_parts[n]}`, `__TAG_PARTS[n]__`
|
36
|
-
|
37
|
-
Alternative: Use `${tag[n]}` placeholder
|
38
|
-
|
39
|
-
* `${hostname}`, `__HOSTNAME__`
|
40
|
-
|
41
|
-
Alternative1: Use filter before this plugin and chunk keys:
|
42
|
-
```
|
43
|
-
<filter>
|
44
|
-
@type record_transformer
|
45
|
-
<record>
|
46
|
-
hostname ${hostname}
|
47
|
-
</record>
|
48
|
-
</filter>
|
49
|
-
<match **>
|
50
|
-
@type geoip
|
51
|
-
tag geoip.${tag[1]}.${hostname}
|
52
|
-
<record>
|
53
|
-
city ${city["host"]}
|
54
|
-
</record>
|
55
|
-
<buffer tag, hostname>
|
56
|
-
flush_interval 1s
|
57
|
-
</buffer>
|
58
|
-
</match>
|
59
|
-
```
|
60
|
-
|
61
|
-
Alternative2: Just inject hostname into record you can use `<inject>` section instead:
|
62
|
-
```
|
63
|
-
<match **>
|
64
|
-
@type geoip
|
65
|
-
tag geoip.${tag[1]}.${hostname}
|
66
|
-
<record>
|
67
|
-
city ${city["host"]}
|
68
|
-
</record>
|
69
|
-
<inject>
|
70
|
-
hostname_key hostname
|
71
|
-
</inject>
|
72
|
-
</match>
|
73
|
-
```
|
24
|
+
We've removed GeoipOutput since 1.3.0, because GeoipFilter is enough to add information about geographical location of IP addresse.
|
74
25
|
|
75
26
|
## Dependency
|
76
27
|
|
@@ -78,18 +29,22 @@ before use, install dependent library as:
|
|
78
29
|
|
79
30
|
```bash
|
80
31
|
# for RHEL/CentOS
|
81
|
-
$ sudo yum
|
32
|
+
$ sudo yum groupinstall "Development Tools"
|
82
33
|
$ sudo yum install geoip-devel --enablerepo=epel
|
34
|
+
$ sudo yum install libmaxminddb-devel
|
83
35
|
|
84
36
|
# for Ubuntu/Debian
|
85
37
|
$ sudo apt-get install build-essential
|
86
38
|
$ sudo apt-get install libgeoip-dev
|
39
|
+
$ sudo apt-get install libmaxminddb-dev
|
87
40
|
|
88
41
|
# for OS X
|
89
42
|
$ brew install geoip
|
90
43
|
$ bundle config build.geoip-c --with-geoip-dir=/usr/local/include/
|
91
44
|
```
|
92
45
|
|
46
|
+
See [geoip2_c](https://github.com/okkez/geoip2_c#build-requirements), if you failed to install geoip2_c.
|
47
|
+
|
93
48
|
## Installation
|
94
49
|
|
95
50
|
install with `gem` or td-agent provided command as:
|
@@ -107,201 +62,185 @@ $ sudo td-agent-gem install fluent-plugin-geoip
|
|
107
62
|
|
108
63
|
## Usage
|
109
64
|
|
110
|
-
### For
|
65
|
+
### For GeoipFilter
|
66
|
+
|
67
|
+
Note that filter version of geoip plugin does not have handling tag feature.
|
111
68
|
|
112
69
|
```xml
|
113
|
-
<
|
70
|
+
<filter access.apache>
|
114
71
|
@type geoip
|
115
72
|
|
116
73
|
# Specify one or more geoip lookup field which has ip address (default: host)
|
117
|
-
|
118
|
-
geoip_lookup_key host
|
74
|
+
geoip_lookup_keys host
|
119
75
|
|
120
76
|
# Specify optional geoip database (using bundled GeoLiteCity databse by default)
|
121
|
-
geoip_database "/path/to/your/GeoIPCity.dat"
|
77
|
+
# geoip_database "/path/to/your/GeoIPCity.dat"
|
122
78
|
# Specify optional geoip2 database
|
123
|
-
# geoip2_database "/path/to/your/GeoLite2-City.mmdb"
|
124
|
-
# Specify backend library (geoip, geoip2_compat
|
125
|
-
backend_library
|
79
|
+
# geoip2_database "/path/to/your/GeoLite2-City.mmdb" (using bundled GeoLite2-City.mmdb by default)
|
80
|
+
# Specify backend library (geoip2_c, geoip, geoip2_compat)
|
81
|
+
backend_library geoip2_c
|
126
82
|
|
127
83
|
# Set adding field with placeholder (more than one settings are required.)
|
128
84
|
<record>
|
129
|
-
|
130
|
-
|
131
|
-
|
132
|
-
country ${
|
133
|
-
country_name ${
|
134
|
-
|
135
|
-
area ${area_code["host"]}
|
136
|
-
region ${region["host"]}
|
137
|
-
city ${city["host"]}
|
85
|
+
city ${city.names.en["host"]}
|
86
|
+
latitude ${location.latitude["host"]}
|
87
|
+
longitude ${location.longitude["host"]}
|
88
|
+
country ${country.iso_code["host"]}
|
89
|
+
country_name ${country.names.en["host"]}
|
90
|
+
postal_code ${postal.code["host"]}
|
138
91
|
</record>
|
139
92
|
|
140
|
-
# Settings for tag
|
141
|
-
tag geoip.${tag[1]}
|
142
|
-
|
143
93
|
# To avoid get stacktrace error with `[null, null]` array for elasticsearch.
|
144
94
|
skip_adding_null_record true
|
145
95
|
|
146
|
-
# Set log_level
|
147
|
-
log_level info
|
148
|
-
|
149
|
-
<buffer tag>
|
150
|
-
# Set buffering time (default: 0s)
|
151
|
-
flush_interval 1s
|
152
|
-
</buffer>
|
153
|
-
</match>
|
96
|
+
# Set @log_level (default: warn)
|
97
|
+
@log_level info
|
98
|
+
</filter>
|
154
99
|
```
|
155
100
|
|
156
101
|
#### Tips: how to geolocate multiple key
|
157
102
|
|
158
103
|
```xml
|
159
|
-
<
|
104
|
+
<filter access.apache>
|
160
105
|
@type geoip
|
161
|
-
|
106
|
+
geoip_lookup_keys user1_host, user2_host
|
162
107
|
<record>
|
163
|
-
user1_city ${city["user1_host"]}
|
164
|
-
user2_city ${city["user2_host"]}
|
108
|
+
user1_city ${city.names.en["user1_host"]}
|
109
|
+
user2_city ${city.names.en["user2_host"]}
|
165
110
|
</record>
|
166
|
-
|
167
|
-
|
111
|
+
</filter>
|
112
|
+
```
|
113
|
+
|
114
|
+
#### Tips: Modify records without city information
|
115
|
+
|
116
|
+
```
|
117
|
+
<filter access.apache>
|
118
|
+
@type geoip
|
119
|
+
geoip_lookup_keys remote_addr
|
120
|
+
<record>
|
121
|
+
city ${city.names.en["remote_addr"]} # skip adding fields if this field is null
|
122
|
+
latitude ${location.latitude["remote_addr"]}
|
123
|
+
longitude ${location.longitude["remote_addr"]}
|
124
|
+
country ${country.iso_code["remote_addr"]}
|
125
|
+
country_name ${country.names.en["remote_addr"]}
|
126
|
+
postal_code ${postal.code["remote_addr"]}
|
127
|
+
</record>
|
128
|
+
skip_adding_null_record true
|
129
|
+
</filter>
|
130
|
+
```
|
131
|
+
|
132
|
+
Skip adding fields if incoming `remote_addr`'s GeoIP data is like following:
|
133
|
+
|
134
|
+
```ruby
|
135
|
+
# the record does not have "city" field
|
136
|
+
{"continent"=>
|
137
|
+
{"code"=>"NA",
|
138
|
+
"geoname_id"=>6255149,
|
139
|
+
"names"=>
|
140
|
+
{"de"=>"Nordamerika",
|
141
|
+
"en"=>"North America",
|
142
|
+
"es"=>"Norteamérica",
|
143
|
+
"fr"=>"Amérique du Nord",
|
144
|
+
"ja"=>"北アメリカ",
|
145
|
+
"pt-BR"=>"América do Norte",
|
146
|
+
"ru"=>"Северная Америка",
|
147
|
+
"zh-CN"=>"北美洲"}},
|
148
|
+
"country"=>
|
149
|
+
{"geoname_id"=>6252001,
|
150
|
+
"iso_code"=>"US",
|
151
|
+
"names"=>
|
152
|
+
{"de"=>"USA",
|
153
|
+
"en"=>"United States",
|
154
|
+
"es"=>"Estados Unidos",
|
155
|
+
"fr"=>"États-Unis",
|
156
|
+
"ja"=>"アメリカ合衆国",
|
157
|
+
"pt-BR"=>"Estados Unidos",
|
158
|
+
"ru"=>"США",
|
159
|
+
"zh-CN"=>"美国"}},
|
160
|
+
"location"=>
|
161
|
+
{"accuracy_radius"=>1000, "latitude"=>37.751, "longitude"=>-97.822},
|
162
|
+
"registered_country"=>
|
163
|
+
{"geoname_id"=>6252001,
|
164
|
+
"iso_code"=>"US",
|
165
|
+
"names"=>
|
166
|
+
{"de"=>"USA",
|
167
|
+
"en"=>"United States",
|
168
|
+
"es"=>"Estados Unidos",
|
169
|
+
"fr"=>"États-Unis",
|
170
|
+
"ja"=>"アメリカ合衆国",
|
171
|
+
"pt-BR"=>"Estados Unidos",
|
172
|
+
"ru"=>"США",
|
173
|
+
"zh-CN"=>"美国"}}}
|
174
|
+
```
|
175
|
+
|
176
|
+
We can avoid this behavior changing field order in `<record>` like following:
|
177
|
+
|
178
|
+
```
|
179
|
+
<filter access.apache>
|
180
|
+
@type geoip
|
181
|
+
geoip_lookup_keys remote_addr
|
182
|
+
<record>
|
183
|
+
latitude ${location.latitude["remote_addr"]} # this field must not be null
|
184
|
+
longitude ${location.longitude["remote_addr"]}
|
185
|
+
country ${country.iso_code["remote_addr"]}
|
186
|
+
country_name ${country.names.en["remote_addr"]}
|
187
|
+
postal_code ${postal.code["remote_addr"]}
|
188
|
+
city ${city.names.en["remote_addr"]} # adding fields even if this field is null
|
189
|
+
</record>
|
190
|
+
skip_adding_null_record true
|
191
|
+
</filter>
|
168
192
|
```
|
169
193
|
|
194
|
+
#### Tips: nested attributes for geoip_lookup_keys
|
195
|
+
|
196
|
+
See [Record Accessor Plugin Helper](https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-record_accessor)
|
197
|
+
|
198
|
+
**NOTE** Since v1.3.0 does not interpret `host.ip` as nested attribute.
|
199
|
+
|
170
200
|
#### Advanced config samples
|
171
201
|
|
172
202
|
It is a sample to get friendly geo point recdords for elasticsearch with Yajl (JSON) parser.<br />
|
173
203
|
|
174
|
-
**Notice** v0 config will be deprecated in the future.
|
175
|
-
|
176
204
|
```
|
177
|
-
<
|
205
|
+
<filter access.apache>
|
178
206
|
@type geoip
|
179
|
-
|
207
|
+
geoip_lookup_keys host
|
180
208
|
<record>
|
181
209
|
# lat lon as properties
|
182
210
|
# ex. {"lat" => 37.4192008972168, "lon" => -122.05740356445312 }
|
183
|
-
location_properties '{ "lat" : ${latitude["host"]}, "lon" : ${longitude["host"]} }'
|
211
|
+
location_properties '{ "lat" : ${location.latitude["host"]}, "lon" : ${location.longitude["host"]} }'
|
184
212
|
|
185
213
|
# lat lon as string
|
186
214
|
# ex. "37.4192008972168,-122.05740356445312"
|
187
|
-
location_string ${latitude["host"]},${longitude["host"]}
|
215
|
+
location_string ${location.latitude["host"]},${location.longitude["host"]}
|
188
216
|
|
189
217
|
# GeoJSON (lat lon as array) is useful for Kibana's bettermap.
|
190
218
|
# ex. [-122.05740356445312, 37.4192008972168]
|
191
|
-
location_array '[${longitude["host"]},${latitude["host"]}]'
|
219
|
+
location_array '[${location.longitude["host"]},${location.latitude["host"]}]'
|
192
220
|
</record>
|
193
|
-
tag geoip.${tag[1]}
|
194
221
|
|
195
222
|
# To avoid get stacktrace error with `[null, null]` array for elasticsearch.
|
196
223
|
skip_adding_null_record true
|
197
|
-
</
|
224
|
+
</filter>
|
198
225
|
```
|
199
226
|
|
200
227
|
On the case of using td-agent3 (v1-config), it have to quote `{ ... }` or `[ ... ]` block with quotation like below.
|
201
228
|
|
202
229
|
```
|
203
|
-
<match access.apache>
|
204
|
-
@type geoip
|
205
|
-
geoip_lookup_key host
|
206
|
-
<record>
|
207
|
-
location_properties '{ "lat" : ${latitude["host"]}, "lon" : ${longitude["host"]} }'
|
208
|
-
location_string ${latitude["host"]},${longitude["host"]}
|
209
|
-
location_array '[${longitude["host"]},${latitude["host"]}]'
|
210
|
-
</record>
|
211
|
-
remove_tag_prefix access.
|
212
|
-
tag geoip.${tag}
|
213
|
-
skip_adding_null_record true
|
214
|
-
</match>
|
215
|
-
```
|
216
|
-
|
217
|
-
### For GeoipFilter
|
218
|
-
|
219
|
-
Note that filter version of geoip plugin does not have handling tag feature.
|
220
|
-
|
221
|
-
```xml
|
222
230
|
<filter access.apache>
|
223
|
-
@type
|
224
|
-
|
225
|
-
# Specify one or more geoip lookup field which has ip address (default: host)
|
226
|
-
# in the case of accessing nested value, delimit keys by dot like 'host.ip'.
|
227
|
-
geoip_lookup_key host
|
228
|
-
|
229
|
-
# Specify optional geoip database (using bundled GeoLiteCity databse by default)
|
230
|
-
geoip_database "/path/to/your/GeoIPCity.dat"
|
231
|
-
# Specify optional geoip2 database
|
232
|
-
# geoip2_database "/path/to/your/GeoLite2-City.mmdb"
|
233
|
-
# Specify backend library (geoip, geoip2_compat, geoip2_c)
|
234
|
-
backend_library geoip
|
235
|
-
|
236
|
-
# Set adding field with placeholder (more than one settings are required.)
|
231
|
+
@type geoip
|
232
|
+
geoip_lookup_keys host
|
237
233
|
<record>
|
238
|
-
|
239
|
-
|
240
|
-
|
241
|
-
country_code3 ${country_code3["host"]}
|
242
|
-
country ${country_code["host"]}
|
243
|
-
country_name ${country_name["host"]}
|
244
|
-
dma ${dma_code["host"]}
|
245
|
-
area ${area_code["host"]}
|
246
|
-
region ${region["host"]}
|
234
|
+
location_properties '{ "lat" : ${location.latitude["host"]}, "lon" : ${location.longitude["host"]} }'
|
235
|
+
location_string ${location.latitude["host"]},${location.longitude["host"]}
|
236
|
+
location_array '[${location.longitude["host"]},${location.latitude["host"]}]'
|
247
237
|
</record>
|
248
|
-
|
249
|
-
# To avoid get stacktrace error with `[null, null]` array for elasticsearch.
|
250
238
|
skip_adding_null_record true
|
251
|
-
|
252
|
-
# Set log_level for fluentd-v0.10.43 or earlier (default: warn)
|
253
|
-
log_level info
|
254
239
|
</filter>
|
255
240
|
```
|
256
241
|
|
257
242
|
## Tutorial
|
258
243
|
|
259
|
-
### For GeoipOutput
|
260
|
-
|
261
|
-
#### configuration
|
262
|
-
|
263
|
-
```xml
|
264
|
-
<source>
|
265
|
-
@type forward
|
266
|
-
</source>
|
267
|
-
|
268
|
-
<match test.geoip>
|
269
|
-
@type copy
|
270
|
-
<store>
|
271
|
-
@type stdout
|
272
|
-
</store>
|
273
|
-
<store>
|
274
|
-
@type geoip
|
275
|
-
geoip_lookup_key host
|
276
|
-
<record>
|
277
|
-
lat ${latitude["host"]}
|
278
|
-
lon ${longitude["host"]}
|
279
|
-
country ${country_code["host"]}
|
280
|
-
</record>
|
281
|
-
tag debug.${tag[1]}
|
282
|
-
</store>
|
283
|
-
</match>
|
284
|
-
|
285
|
-
<match debug.**>
|
286
|
-
@type stdout
|
287
|
-
</match>
|
288
|
-
```
|
289
|
-
|
290
|
-
#### result
|
291
|
-
|
292
|
-
```bash
|
293
|
-
# forward record with Google's ip address.
|
294
|
-
$ echo '{"host":"66.102.9.80","message":"test"}' | fluent-cat test.geoip
|
295
|
-
|
296
|
-
# check the result at stdout
|
297
|
-
$ tail /var/log/td-agent/td-agent.log
|
298
|
-
2013-08-04 16:21:32 +0900 test.geoip: {"host":"66.102.9.80","message":"test"}
|
299
|
-
2013-08-04 16:21:32 +0900 debug.geoip: {"host":"66.102.9.80","message":"test","lat":37.4192008972168,"lon":-122.05740356445312,"country":"US"}
|
300
|
-
```
|
301
|
-
|
302
|
-
For more details of geoip data format is described at the page below in section `GeoIP City Edition CSV Database Fields`.<br />
|
303
|
-
http://dev.maxmind.com/geoip/legacy/csv/
|
304
|
-
|
305
244
|
### For GeoipFilter
|
306
245
|
|
307
246
|
#### configuration
|
@@ -313,11 +252,11 @@ http://dev.maxmind.com/geoip/legacy/csv/
|
|
313
252
|
|
314
253
|
<filter test.geoip>
|
315
254
|
@type geoip
|
316
|
-
|
255
|
+
geoip_lookup_keys host
|
317
256
|
<record>
|
318
|
-
city ${city["host"]}
|
319
|
-
lat ${latitude["host"]}
|
320
|
-
lon ${longitude["host"]}
|
257
|
+
city ${city.names.en["host"]}
|
258
|
+
lat ${location.latitude["host"]}
|
259
|
+
lon ${location.longitude["host"]}
|
321
260
|
</record>
|
322
261
|
</filter>
|
323
262
|
|
@@ -337,34 +276,15 @@ $ tail /var/log/td-agent/td-agent.log
|
|
337
276
|
2016-02-01 12:04:37 +0900 test.geoip: {"host":"66.102.9.80","message":"test","city":"Mountain View","lat":37.4192008972168,"lon":-122.05740356445312}
|
338
277
|
```
|
339
278
|
|
340
|
-
|
341
|
-
http://dev.maxmind.com/geoip/legacy/csv/
|
342
|
-
|
343
|
-
## Placeholders
|
344
|
-
|
345
|
-
### GeoIP legacy
|
346
|
-
|
347
|
-
Provides these placeholders for adding field of geolocate results.<br />
|
348
|
-
For more example of geolocating, you can try these websites like [Geo IP Address View](http://www.geoipview.com/) or [View my IP information](http://www.geoiptool.com/en/).
|
349
|
-
|
350
|
-
| placeholder attributes | output example | type | note |
|
351
|
-
|--------------------------------|-------------------|--------------|------|
|
352
|
-
| ${city[lookup_field]} | "Ithaca" | varchar(255) | - |
|
353
|
-
| ${latitude[lookup_field]} | 42.4277992248535 | decimal | - |
|
354
|
-
| ${longitude[lookup_field]} | -76.4981994628906 | decimal | - |
|
355
|
-
| ${country_code3[lookup_field]} | "USA" | varchar(3) | - |
|
356
|
-
| ${country_code[lookup_field]} | "US" | varchar(2) | A two-character ISO 3166-1 country code |
|
357
|
-
| ${country_name[lookup_field]} | "United States" | varchar(50) | - |
|
358
|
-
| ${dma_code[lookup_field]} | 555 | unsigned int | **only for US** |
|
359
|
-
| ${area_code[lookup_field]} | 607 | char(3) | **only for US** |
|
360
|
-
| ${region[lookup_field]} | "NY" | char(2) | A two character ISO-3166-2 or FIPS 10-4 code |
|
361
|
-
|
362
|
-
Further more specification available at http://dev.maxmind.com/geoip/legacy/csv/#GeoIP_City_Edition_CSV_Database_Fields
|
279
|
+
You can check geoip data format using [utils/dump.rb](https://github.com/okkez/fluent-plugin-geoip/utils/dump.rb).
|
363
280
|
|
364
|
-
|
281
|
+
```
|
282
|
+
$ bundle exec ruby urils/dump.rb geoip2 66.102.3.80
|
283
|
+
$ bundle exec ruby urils/dump.rb geoip2_compat 66.102.3.80
|
284
|
+
$ bundle exec ruby urils/dump.rb geoip 66.102.3.80
|
285
|
+
```
|
365
286
|
|
366
|
-
|
367
|
-
* `geoip_database`: path to your GeoLiteCity.dat
|
287
|
+
## Placeholders
|
368
288
|
|
369
289
|
### GeoIP2
|
370
290
|
|
@@ -405,43 +325,89 @@ Related configurations:
|
|
405
325
|
* `backend_library`: `geoip2_compat` or `geoip2_c`
|
406
326
|
* `geoip2_database`: path to your GeoLite2-City.mmdb
|
407
327
|
|
328
|
+
### GeoIP legacy
|
329
|
+
|
330
|
+
Provides these placeholders for adding field of geolocate results.<br />
|
331
|
+
For more example of geolocating, you can try these websites like [Geo IP Address View](http://www.geoipview.com/) or [View my IP information](http://www.geoiptool.com/en/).
|
332
|
+
|
333
|
+
| placeholder attributes | output example | type | note |
|
334
|
+
|--------------------------------|-------------------|--------------|------|
|
335
|
+
| ${city[lookup_field]} | "Ithaca" | varchar(255) | - |
|
336
|
+
| ${latitude[lookup_field]} | 42.4277992248535 | decimal | - |
|
337
|
+
| ${longitude[lookup_field]} | -76.4981994628906 | decimal | - |
|
338
|
+
| ${country_code3[lookup_field]} | "USA" | varchar(3) | - |
|
339
|
+
| ${country_code[lookup_field]} | "US" | varchar(2) | A two-character ISO 3166-1 country code |
|
340
|
+
| ${country_name[lookup_field]} | "United States" | varchar(50) | - |
|
341
|
+
| ${dma_code[lookup_field]} | 555 | unsigned int | **only for US** |
|
342
|
+
| ${area_code[lookup_field]} | 607 | char(3) | **only for US** |
|
343
|
+
| ${region[lookup_field]} | "NY" | char(2) | A two character ISO-3166-2 or FIPS 10-4 code |
|
344
|
+
|
345
|
+
Further more specification available at http://dev.maxmind.com/geoip/legacy/csv/#GeoIP_City_Edition_CSV_Database_Fields
|
346
|
+
|
347
|
+
Related configurations:
|
348
|
+
|
349
|
+
* `backend_library`: `geoip` (default)
|
350
|
+
* `geoip_database`: path to your GeoLiteCity.dat
|
351
|
+
|
408
352
|
## Parameters
|
409
353
|
|
410
|
-
###
|
354
|
+
### GeoipFilter
|
411
355
|
|
412
|
-
|
413
|
-
* `tag_key`
|
356
|
+
Note that filter version of `geoip` plugin does not have handling `tag` feature.
|
414
357
|
|
415
|
-
|
416
|
-
Further details are written at http://docs.fluentd.org/articles/in_exec
|
358
|
+
#### Plugin helpers
|
417
359
|
|
418
|
-
*
|
360
|
+
* [compat_parameters](https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-compat_parameters)
|
361
|
+
* [inject](https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-inject)
|
419
362
|
|
420
|
-
|
421
|
-
On the case of getting nothing of GeoIP info (such as local IP), it will output the original record without changing anything.
|
363
|
+
See also [Filter Plugin Overview](https://docs.fluentd.org/filter)
|
422
364
|
|
423
|
-
|
365
|
+
#### Supported sections
|
424
366
|
|
425
|
-
|
367
|
+
* [Inject section configurations](https://docs.fluentd.org/configuration/inject-section)
|
426
368
|
|
427
|
-
|
369
|
+
#### Parameters
|
428
370
|
|
429
|
-
|
371
|
+
[Plugin Common Paramteters](https://docs.fluentd.org/configuration/plugin-common-parameters)
|
430
372
|
|
431
|
-
|
373
|
+
**geoip_database** (string) (optional)
|
432
374
|
|
433
|
-
|
375
|
+
* Default value: bundled database `GeoLiteCity.dat`
|
376
|
+
|
377
|
+
Path to GeoIP database file.
|
378
|
+
|
379
|
+
**geoip2_database** (string) (optional)
|
380
|
+
|
381
|
+
* Default value: bundled database `GeoLite2-City.mmdb`.
|
382
|
+
|
383
|
+
Path to GeoIP2 database file.
|
434
384
|
|
435
|
-
|
385
|
+
**geoip_lookup_keys** (array) (optional)
|
436
386
|
|
437
|
-
|
438
|
-
Further details are written at http://docs.fluentd.org/articles/in_exec
|
387
|
+
* Default_value: `["host"]`
|
439
388
|
|
440
|
-
|
389
|
+
Specify one or more geoip lookup field which has IP address.
|
390
|
+
|
391
|
+
**geoip_lookup_key** (string) (optional) (deprecated)
|
392
|
+
|
393
|
+
* Default value: `nil`.
|
394
|
+
|
395
|
+
Use geoip_lookup_keys instead.
|
396
|
+
|
397
|
+
**skip_adding_null_record** (bool) (optional)
|
398
|
+
|
399
|
+
* Default value: `nil`
|
441
400
|
|
442
401
|
Skip adding geoip fields when this valaues to `true`.
|
443
402
|
On the case of getting nothing of GeoIP info (such as local IP), it will output the original record without changing anything.
|
444
403
|
|
404
|
+
**backend_library** (enum) (optional)
|
405
|
+
|
406
|
+
* Available values: `geoip`, `geoip2_compat`, `geoip2_c`
|
407
|
+
* Default value: `geoip2_c`.
|
408
|
+
|
409
|
+
Set backend library.
|
410
|
+
|
445
411
|
## Articles
|
446
412
|
|
447
413
|
* [IPアドレスを元に位置情報をリアルタイムに付与する fluent-plugin-geoip v0.0.1をリリースしました #fluentd - Y-Ken Studio](http://y-ken.hatenablog.com/entry/fluent-plugin-geoip-has-released)<br />
|
@@ -463,8 +429,6 @@ http://developer.smartnews.be/blog/2013/10/03/easy-data-analysis-using-fluentd-r
|
|
463
429
|
|
464
430
|
Pull requests are very welcome!!
|
465
431
|
|
466
|
-
* support [GeoIP2](http://dev.maxmind.com/geoip/geoip2/whats-new-in-geoip2/)
|
467
|
-
|
468
432
|
## Contributing
|
469
433
|
|
470
434
|
1. Fork it
|