fluent-plugin-geoip 1.0.0 → 1.3.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.github/ISSUE_TEMPLATE.md +19 -0
- data/.travis.yml +1 -1
- data/README.md +203 -239
- data/docker-compose.yml +12 -5
- data/dockerfiles/Dockerfile-ruby2.1 +1 -1
- data/dockerfiles/Dockerfile-ruby2.2 +1 -1
- data/dockerfiles/Dockerfile-ruby2.3 +1 -1
- data/dockerfiles/Dockerfile-ruby2.4 +1 -1
- data/dockerfiles/Dockerfile-ruby2.5 +8 -0
- data/fluent-plugin-geoip.gemspec +2 -1
- data/lib/fluent/plugin/filter_geoip.rb +196 -11
- data/test/helper.rb +1 -0
- data/test/plugin/test_filter_geoip.rb +296 -107
- data/utils/dump.rb +27 -0
- metadata +20 -8
- data/lib/fluent/plugin/geoip.rb +0 -174
- data/lib/fluent/plugin/out_geoip.rb +0 -51
- data/test/plugin/test_out_geoip.rb +0 -1076
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 35bca82d01d71c2eed7db735cf4e2394e6af0ec420baa511cfe74431fd0c88c4
|
4
|
+
data.tar.gz: e15c7f9fec5855d0012d97a65d0aa170338a169229e65eea5baf798c4d2d2763
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c67641b18a5254169fa4c0d0fa7b32e1612c2f65c6f7ebfd5afffeaaa7c1f5a1ffa1e77334e10a7f5cecf476e0dbd6b7d64f080d93894b4b0cf8f7c4092b141d
|
7
|
+
data.tar.gz: c7e7fa0d170d4f451f9666bf471a76b193a7f0e466403992b85cb6051bc72b1a94e6f54ffe378cc5414d8cdf251a02402fa26c01507e58baf2b2db313690e4d3
|
@@ -0,0 +1,19 @@
|
|
1
|
+
#### Problem
|
2
|
+
|
3
|
+
...
|
4
|
+
|
5
|
+
#### Steps to replicate
|
6
|
+
|
7
|
+
Provide example config and message
|
8
|
+
|
9
|
+
#### Expected Behavior
|
10
|
+
|
11
|
+
...
|
12
|
+
|
13
|
+
#### Your environment
|
14
|
+
|
15
|
+
* OS version
|
16
|
+
* paste result of ``fluentd --version`` or ``td-agent --version``
|
17
|
+
* plugin version
|
18
|
+
* paste boot log of fluentd or td-agent
|
19
|
+
* paste result of ``fluent-gem list``, ``td-agent-gem list`` or your Gemfile.lock
|
data/.travis.yml
CHANGED
@@ -4,7 +4,7 @@ services:
|
|
4
4
|
|
5
5
|
env:
|
6
6
|
global:
|
7
|
-
DOCKER_COMPOSE_VERSION: 1.
|
7
|
+
DOCKER_COMPOSE_VERSION: 1.18.0
|
8
8
|
matrix:
|
9
9
|
- TARGET_RUBY_VERSION=2.1 BUNDLE_GEMFILE=/app/gemfiles/fluentd_v1.0_with_backport_dig.gemfile
|
10
10
|
- TARGET_RUBY_VERSION=2.2 BUNDLE_GEMFILE=/app/gemfiles/fluentd_v1.0_with_backport_dig.gemfile
|
data/README.md
CHANGED
@@ -1,8 +1,8 @@
|
|
1
1
|
# fluent-plugin-geoip [![Build Status](https://travis-ci.org/y-ken/fluent-plugin-geoip.png?branch=master)](https://travis-ci.org/y-ken/fluent-plugin-geoip)
|
2
2
|
|
3
|
-
Fluentd
|
3
|
+
Fluentd Filter plugin to add information about geographical location of IP addresses with Maxmind GeoIP databases.
|
4
4
|
|
5
|
-
fluent-plugin-geoip has bundled cost-free [GeoLite City database](http://dev.maxmind.com/geoip/legacy/geolite/) by default.<br />
|
5
|
+
fluent-plugin-geoip has bundled cost-free [GeoLite2 Free Downloadable Databases](https://dev.maxmind.com/geoip/geoip2/geolite2/) and [GeoLite City database](http://dev.maxmind.com/geoip/legacy/geolite/) by default.<br />
|
6
6
|
Also you can use purchased [GeoIP City database](http://www.maxmind.com/en/city) ([lang:ja](http://www.maxmind.com/ja/city)) which costs starting from $50.
|
7
7
|
|
8
8
|
The accuracy details for GeoLite City (free) and GeoIP City (purchased) has described at the page below.
|
@@ -21,56 +21,7 @@ If you want to use this plugin with Fluentd v0.12.x or earlier use 0.8.x.
|
|
21
21
|
|
22
22
|
### Compatibility notice
|
23
23
|
|
24
|
-
We've
|
25
|
-
So we have dropped some features.
|
26
|
-
|
27
|
-
See also [official document](http://docs.fluentd.org/v1.0/articles/plugin-update-from-v12)
|
28
|
-
|
29
|
-
#### Fluent::Mixin::RewriteTagName
|
30
|
-
|
31
|
-
* `${tag}`, `__TAG__`
|
32
|
-
|
33
|
-
Alternative: Use `${tag}` placeholder
|
34
|
-
|
35
|
-
* `${tag_parts[n]}`, `__TAG_PARTS[n]__`
|
36
|
-
|
37
|
-
Alternative: Use `${tag[n]}` placeholder
|
38
|
-
|
39
|
-
* `${hostname}`, `__HOSTNAME__`
|
40
|
-
|
41
|
-
Alternative1: Use filter before this plugin and chunk keys:
|
42
|
-
```
|
43
|
-
<filter>
|
44
|
-
@type record_transformer
|
45
|
-
<record>
|
46
|
-
hostname ${hostname}
|
47
|
-
</record>
|
48
|
-
</filter>
|
49
|
-
<match **>
|
50
|
-
@type geoip
|
51
|
-
tag geoip.${tag[1]}.${hostname}
|
52
|
-
<record>
|
53
|
-
city ${city["host"]}
|
54
|
-
</record>
|
55
|
-
<buffer tag, hostname>
|
56
|
-
flush_interval 1s
|
57
|
-
</buffer>
|
58
|
-
</match>
|
59
|
-
```
|
60
|
-
|
61
|
-
Alternative2: Just inject hostname into record you can use `<inject>` section instead:
|
62
|
-
```
|
63
|
-
<match **>
|
64
|
-
@type geoip
|
65
|
-
tag geoip.${tag[1]}.${hostname}
|
66
|
-
<record>
|
67
|
-
city ${city["host"]}
|
68
|
-
</record>
|
69
|
-
<inject>
|
70
|
-
hostname_key hostname
|
71
|
-
</inject>
|
72
|
-
</match>
|
73
|
-
```
|
24
|
+
We've removed GeoipOutput since 1.3.0, because GeoipFilter is enough to add information about geographical location of IP addresse.
|
74
25
|
|
75
26
|
## Dependency
|
76
27
|
|
@@ -78,18 +29,22 @@ before use, install dependent library as:
|
|
78
29
|
|
79
30
|
```bash
|
80
31
|
# for RHEL/CentOS
|
81
|
-
$ sudo yum
|
32
|
+
$ sudo yum groupinstall "Development Tools"
|
82
33
|
$ sudo yum install geoip-devel --enablerepo=epel
|
34
|
+
$ sudo yum install libmaxminddb-devel
|
83
35
|
|
84
36
|
# for Ubuntu/Debian
|
85
37
|
$ sudo apt-get install build-essential
|
86
38
|
$ sudo apt-get install libgeoip-dev
|
39
|
+
$ sudo apt-get install libmaxminddb-dev
|
87
40
|
|
88
41
|
# for OS X
|
89
42
|
$ brew install geoip
|
90
43
|
$ bundle config build.geoip-c --with-geoip-dir=/usr/local/include/
|
91
44
|
```
|
92
45
|
|
46
|
+
See [geoip2_c](https://github.com/okkez/geoip2_c#build-requirements), if you failed to install geoip2_c.
|
47
|
+
|
93
48
|
## Installation
|
94
49
|
|
95
50
|
install with `gem` or td-agent provided command as:
|
@@ -107,201 +62,185 @@ $ sudo td-agent-gem install fluent-plugin-geoip
|
|
107
62
|
|
108
63
|
## Usage
|
109
64
|
|
110
|
-
### For
|
65
|
+
### For GeoipFilter
|
66
|
+
|
67
|
+
Note that filter version of geoip plugin does not have handling tag feature.
|
111
68
|
|
112
69
|
```xml
|
113
|
-
<
|
70
|
+
<filter access.apache>
|
114
71
|
@type geoip
|
115
72
|
|
116
73
|
# Specify one or more geoip lookup field which has ip address (default: host)
|
117
|
-
|
118
|
-
geoip_lookup_key host
|
74
|
+
geoip_lookup_keys host
|
119
75
|
|
120
76
|
# Specify optional geoip database (using bundled GeoLiteCity databse by default)
|
121
|
-
geoip_database "/path/to/your/GeoIPCity.dat"
|
77
|
+
# geoip_database "/path/to/your/GeoIPCity.dat"
|
122
78
|
# Specify optional geoip2 database
|
123
|
-
# geoip2_database "/path/to/your/GeoLite2-City.mmdb"
|
124
|
-
# Specify backend library (geoip, geoip2_compat
|
125
|
-
backend_library
|
79
|
+
# geoip2_database "/path/to/your/GeoLite2-City.mmdb" (using bundled GeoLite2-City.mmdb by default)
|
80
|
+
# Specify backend library (geoip2_c, geoip, geoip2_compat)
|
81
|
+
backend_library geoip2_c
|
126
82
|
|
127
83
|
# Set adding field with placeholder (more than one settings are required.)
|
128
84
|
<record>
|
129
|
-
|
130
|
-
|
131
|
-
|
132
|
-
country ${
|
133
|
-
country_name ${
|
134
|
-
|
135
|
-
area ${area_code["host"]}
|
136
|
-
region ${region["host"]}
|
137
|
-
city ${city["host"]}
|
85
|
+
city ${city.names.en["host"]}
|
86
|
+
latitude ${location.latitude["host"]}
|
87
|
+
longitude ${location.longitude["host"]}
|
88
|
+
country ${country.iso_code["host"]}
|
89
|
+
country_name ${country.names.en["host"]}
|
90
|
+
postal_code ${postal.code["host"]}
|
138
91
|
</record>
|
139
92
|
|
140
|
-
# Settings for tag
|
141
|
-
tag geoip.${tag[1]}
|
142
|
-
|
143
93
|
# To avoid get stacktrace error with `[null, null]` array for elasticsearch.
|
144
94
|
skip_adding_null_record true
|
145
95
|
|
146
|
-
# Set log_level
|
147
|
-
log_level info
|
148
|
-
|
149
|
-
<buffer tag>
|
150
|
-
# Set buffering time (default: 0s)
|
151
|
-
flush_interval 1s
|
152
|
-
</buffer>
|
153
|
-
</match>
|
96
|
+
# Set @log_level (default: warn)
|
97
|
+
@log_level info
|
98
|
+
</filter>
|
154
99
|
```
|
155
100
|
|
156
101
|
#### Tips: how to geolocate multiple key
|
157
102
|
|
158
103
|
```xml
|
159
|
-
<
|
104
|
+
<filter access.apache>
|
160
105
|
@type geoip
|
161
|
-
|
106
|
+
geoip_lookup_keys user1_host, user2_host
|
162
107
|
<record>
|
163
|
-
user1_city ${city["user1_host"]}
|
164
|
-
user2_city ${city["user2_host"]}
|
108
|
+
user1_city ${city.names.en["user1_host"]}
|
109
|
+
user2_city ${city.names.en["user2_host"]}
|
165
110
|
</record>
|
166
|
-
|
167
|
-
|
111
|
+
</filter>
|
112
|
+
```
|
113
|
+
|
114
|
+
#### Tips: Modify records without city information
|
115
|
+
|
116
|
+
```
|
117
|
+
<filter access.apache>
|
118
|
+
@type geoip
|
119
|
+
geoip_lookup_keys remote_addr
|
120
|
+
<record>
|
121
|
+
city ${city.names.en["remote_addr"]} # skip adding fields if this field is null
|
122
|
+
latitude ${location.latitude["remote_addr"]}
|
123
|
+
longitude ${location.longitude["remote_addr"]}
|
124
|
+
country ${country.iso_code["remote_addr"]}
|
125
|
+
country_name ${country.names.en["remote_addr"]}
|
126
|
+
postal_code ${postal.code["remote_addr"]}
|
127
|
+
</record>
|
128
|
+
skip_adding_null_record true
|
129
|
+
</filter>
|
130
|
+
```
|
131
|
+
|
132
|
+
Skip adding fields if incoming `remote_addr`'s GeoIP data is like following:
|
133
|
+
|
134
|
+
```ruby
|
135
|
+
# the record does not have "city" field
|
136
|
+
{"continent"=>
|
137
|
+
{"code"=>"NA",
|
138
|
+
"geoname_id"=>6255149,
|
139
|
+
"names"=>
|
140
|
+
{"de"=>"Nordamerika",
|
141
|
+
"en"=>"North America",
|
142
|
+
"es"=>"Norteamérica",
|
143
|
+
"fr"=>"Amérique du Nord",
|
144
|
+
"ja"=>"北アメリカ",
|
145
|
+
"pt-BR"=>"América do Norte",
|
146
|
+
"ru"=>"Северная Америка",
|
147
|
+
"zh-CN"=>"北美洲"}},
|
148
|
+
"country"=>
|
149
|
+
{"geoname_id"=>6252001,
|
150
|
+
"iso_code"=>"US",
|
151
|
+
"names"=>
|
152
|
+
{"de"=>"USA",
|
153
|
+
"en"=>"United States",
|
154
|
+
"es"=>"Estados Unidos",
|
155
|
+
"fr"=>"États-Unis",
|
156
|
+
"ja"=>"アメリカ合衆国",
|
157
|
+
"pt-BR"=>"Estados Unidos",
|
158
|
+
"ru"=>"США",
|
159
|
+
"zh-CN"=>"美国"}},
|
160
|
+
"location"=>
|
161
|
+
{"accuracy_radius"=>1000, "latitude"=>37.751, "longitude"=>-97.822},
|
162
|
+
"registered_country"=>
|
163
|
+
{"geoname_id"=>6252001,
|
164
|
+
"iso_code"=>"US",
|
165
|
+
"names"=>
|
166
|
+
{"de"=>"USA",
|
167
|
+
"en"=>"United States",
|
168
|
+
"es"=>"Estados Unidos",
|
169
|
+
"fr"=>"États-Unis",
|
170
|
+
"ja"=>"アメリカ合衆国",
|
171
|
+
"pt-BR"=>"Estados Unidos",
|
172
|
+
"ru"=>"США",
|
173
|
+
"zh-CN"=>"美国"}}}
|
174
|
+
```
|
175
|
+
|
176
|
+
We can avoid this behavior changing field order in `<record>` like following:
|
177
|
+
|
178
|
+
```
|
179
|
+
<filter access.apache>
|
180
|
+
@type geoip
|
181
|
+
geoip_lookup_keys remote_addr
|
182
|
+
<record>
|
183
|
+
latitude ${location.latitude["remote_addr"]} # this field must not be null
|
184
|
+
longitude ${location.longitude["remote_addr"]}
|
185
|
+
country ${country.iso_code["remote_addr"]}
|
186
|
+
country_name ${country.names.en["remote_addr"]}
|
187
|
+
postal_code ${postal.code["remote_addr"]}
|
188
|
+
city ${city.names.en["remote_addr"]} # adding fields even if this field is null
|
189
|
+
</record>
|
190
|
+
skip_adding_null_record true
|
191
|
+
</filter>
|
168
192
|
```
|
169
193
|
|
194
|
+
#### Tips: nested attributes for geoip_lookup_keys
|
195
|
+
|
196
|
+
See [Record Accessor Plugin Helper](https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-record_accessor)
|
197
|
+
|
198
|
+
**NOTE** Since v1.3.0 does not interpret `host.ip` as nested attribute.
|
199
|
+
|
170
200
|
#### Advanced config samples
|
171
201
|
|
172
202
|
It is a sample to get friendly geo point recdords for elasticsearch with Yajl (JSON) parser.<br />
|
173
203
|
|
174
|
-
**Notice** v0 config will be deprecated in the future.
|
175
|
-
|
176
204
|
```
|
177
|
-
<
|
205
|
+
<filter access.apache>
|
178
206
|
@type geoip
|
179
|
-
|
207
|
+
geoip_lookup_keys host
|
180
208
|
<record>
|
181
209
|
# lat lon as properties
|
182
210
|
# ex. {"lat" => 37.4192008972168, "lon" => -122.05740356445312 }
|
183
|
-
location_properties '{ "lat" : ${latitude["host"]}, "lon" : ${longitude["host"]} }'
|
211
|
+
location_properties '{ "lat" : ${location.latitude["host"]}, "lon" : ${location.longitude["host"]} }'
|
184
212
|
|
185
213
|
# lat lon as string
|
186
214
|
# ex. "37.4192008972168,-122.05740356445312"
|
187
|
-
location_string ${latitude["host"]},${longitude["host"]}
|
215
|
+
location_string ${location.latitude["host"]},${location.longitude["host"]}
|
188
216
|
|
189
217
|
# GeoJSON (lat lon as array) is useful for Kibana's bettermap.
|
190
218
|
# ex. [-122.05740356445312, 37.4192008972168]
|
191
|
-
location_array '[${longitude["host"]},${latitude["host"]}]'
|
219
|
+
location_array '[${location.longitude["host"]},${location.latitude["host"]}]'
|
192
220
|
</record>
|
193
|
-
tag geoip.${tag[1]}
|
194
221
|
|
195
222
|
# To avoid get stacktrace error with `[null, null]` array for elasticsearch.
|
196
223
|
skip_adding_null_record true
|
197
|
-
</
|
224
|
+
</filter>
|
198
225
|
```
|
199
226
|
|
200
227
|
On the case of using td-agent3 (v1-config), it have to quote `{ ... }` or `[ ... ]` block with quotation like below.
|
201
228
|
|
202
229
|
```
|
203
|
-
<match access.apache>
|
204
|
-
@type geoip
|
205
|
-
geoip_lookup_key host
|
206
|
-
<record>
|
207
|
-
location_properties '{ "lat" : ${latitude["host"]}, "lon" : ${longitude["host"]} }'
|
208
|
-
location_string ${latitude["host"]},${longitude["host"]}
|
209
|
-
location_array '[${longitude["host"]},${latitude["host"]}]'
|
210
|
-
</record>
|
211
|
-
remove_tag_prefix access.
|
212
|
-
tag geoip.${tag}
|
213
|
-
skip_adding_null_record true
|
214
|
-
</match>
|
215
|
-
```
|
216
|
-
|
217
|
-
### For GeoipFilter
|
218
|
-
|
219
|
-
Note that filter version of geoip plugin does not have handling tag feature.
|
220
|
-
|
221
|
-
```xml
|
222
230
|
<filter access.apache>
|
223
|
-
@type
|
224
|
-
|
225
|
-
# Specify one or more geoip lookup field which has ip address (default: host)
|
226
|
-
# in the case of accessing nested value, delimit keys by dot like 'host.ip'.
|
227
|
-
geoip_lookup_key host
|
228
|
-
|
229
|
-
# Specify optional geoip database (using bundled GeoLiteCity databse by default)
|
230
|
-
geoip_database "/path/to/your/GeoIPCity.dat"
|
231
|
-
# Specify optional geoip2 database
|
232
|
-
# geoip2_database "/path/to/your/GeoLite2-City.mmdb"
|
233
|
-
# Specify backend library (geoip, geoip2_compat, geoip2_c)
|
234
|
-
backend_library geoip
|
235
|
-
|
236
|
-
# Set adding field with placeholder (more than one settings are required.)
|
231
|
+
@type geoip
|
232
|
+
geoip_lookup_keys host
|
237
233
|
<record>
|
238
|
-
|
239
|
-
|
240
|
-
|
241
|
-
country_code3 ${country_code3["host"]}
|
242
|
-
country ${country_code["host"]}
|
243
|
-
country_name ${country_name["host"]}
|
244
|
-
dma ${dma_code["host"]}
|
245
|
-
area ${area_code["host"]}
|
246
|
-
region ${region["host"]}
|
234
|
+
location_properties '{ "lat" : ${location.latitude["host"]}, "lon" : ${location.longitude["host"]} }'
|
235
|
+
location_string ${location.latitude["host"]},${location.longitude["host"]}
|
236
|
+
location_array '[${location.longitude["host"]},${location.latitude["host"]}]'
|
247
237
|
</record>
|
248
|
-
|
249
|
-
# To avoid get stacktrace error with `[null, null]` array for elasticsearch.
|
250
238
|
skip_adding_null_record true
|
251
|
-
|
252
|
-
# Set log_level for fluentd-v0.10.43 or earlier (default: warn)
|
253
|
-
log_level info
|
254
239
|
</filter>
|
255
240
|
```
|
256
241
|
|
257
242
|
## Tutorial
|
258
243
|
|
259
|
-
### For GeoipOutput
|
260
|
-
|
261
|
-
#### configuration
|
262
|
-
|
263
|
-
```xml
|
264
|
-
<source>
|
265
|
-
@type forward
|
266
|
-
</source>
|
267
|
-
|
268
|
-
<match test.geoip>
|
269
|
-
@type copy
|
270
|
-
<store>
|
271
|
-
@type stdout
|
272
|
-
</store>
|
273
|
-
<store>
|
274
|
-
@type geoip
|
275
|
-
geoip_lookup_key host
|
276
|
-
<record>
|
277
|
-
lat ${latitude["host"]}
|
278
|
-
lon ${longitude["host"]}
|
279
|
-
country ${country_code["host"]}
|
280
|
-
</record>
|
281
|
-
tag debug.${tag[1]}
|
282
|
-
</store>
|
283
|
-
</match>
|
284
|
-
|
285
|
-
<match debug.**>
|
286
|
-
@type stdout
|
287
|
-
</match>
|
288
|
-
```
|
289
|
-
|
290
|
-
#### result
|
291
|
-
|
292
|
-
```bash
|
293
|
-
# forward record with Google's ip address.
|
294
|
-
$ echo '{"host":"66.102.9.80","message":"test"}' | fluent-cat test.geoip
|
295
|
-
|
296
|
-
# check the result at stdout
|
297
|
-
$ tail /var/log/td-agent/td-agent.log
|
298
|
-
2013-08-04 16:21:32 +0900 test.geoip: {"host":"66.102.9.80","message":"test"}
|
299
|
-
2013-08-04 16:21:32 +0900 debug.geoip: {"host":"66.102.9.80","message":"test","lat":37.4192008972168,"lon":-122.05740356445312,"country":"US"}
|
300
|
-
```
|
301
|
-
|
302
|
-
For more details of geoip data format is described at the page below in section `GeoIP City Edition CSV Database Fields`.<br />
|
303
|
-
http://dev.maxmind.com/geoip/legacy/csv/
|
304
|
-
|
305
244
|
### For GeoipFilter
|
306
245
|
|
307
246
|
#### configuration
|
@@ -313,11 +252,11 @@ http://dev.maxmind.com/geoip/legacy/csv/
|
|
313
252
|
|
314
253
|
<filter test.geoip>
|
315
254
|
@type geoip
|
316
|
-
|
255
|
+
geoip_lookup_keys host
|
317
256
|
<record>
|
318
|
-
city ${city["host"]}
|
319
|
-
lat ${latitude["host"]}
|
320
|
-
lon ${longitude["host"]}
|
257
|
+
city ${city.names.en["host"]}
|
258
|
+
lat ${location.latitude["host"]}
|
259
|
+
lon ${location.longitude["host"]}
|
321
260
|
</record>
|
322
261
|
</filter>
|
323
262
|
|
@@ -337,34 +276,15 @@ $ tail /var/log/td-agent/td-agent.log
|
|
337
276
|
2016-02-01 12:04:37 +0900 test.geoip: {"host":"66.102.9.80","message":"test","city":"Mountain View","lat":37.4192008972168,"lon":-122.05740356445312}
|
338
277
|
```
|
339
278
|
|
340
|
-
|
341
|
-
http://dev.maxmind.com/geoip/legacy/csv/
|
342
|
-
|
343
|
-
## Placeholders
|
344
|
-
|
345
|
-
### GeoIP legacy
|
346
|
-
|
347
|
-
Provides these placeholders for adding field of geolocate results.<br />
|
348
|
-
For more example of geolocating, you can try these websites like [Geo IP Address View](http://www.geoipview.com/) or [View my IP information](http://www.geoiptool.com/en/).
|
349
|
-
|
350
|
-
| placeholder attributes | output example | type | note |
|
351
|
-
|--------------------------------|-------------------|--------------|------|
|
352
|
-
| ${city[lookup_field]} | "Ithaca" | varchar(255) | - |
|
353
|
-
| ${latitude[lookup_field]} | 42.4277992248535 | decimal | - |
|
354
|
-
| ${longitude[lookup_field]} | -76.4981994628906 | decimal | - |
|
355
|
-
| ${country_code3[lookup_field]} | "USA" | varchar(3) | - |
|
356
|
-
| ${country_code[lookup_field]} | "US" | varchar(2) | A two-character ISO 3166-1 country code |
|
357
|
-
| ${country_name[lookup_field]} | "United States" | varchar(50) | - |
|
358
|
-
| ${dma_code[lookup_field]} | 555 | unsigned int | **only for US** |
|
359
|
-
| ${area_code[lookup_field]} | 607 | char(3) | **only for US** |
|
360
|
-
| ${region[lookup_field]} | "NY" | char(2) | A two character ISO-3166-2 or FIPS 10-4 code |
|
361
|
-
|
362
|
-
Further more specification available at http://dev.maxmind.com/geoip/legacy/csv/#GeoIP_City_Edition_CSV_Database_Fields
|
279
|
+
You can check geoip data format using [utils/dump.rb](https://github.com/okkez/fluent-plugin-geoip/utils/dump.rb).
|
363
280
|
|
364
|
-
|
281
|
+
```
|
282
|
+
$ bundle exec ruby urils/dump.rb geoip2 66.102.3.80
|
283
|
+
$ bundle exec ruby urils/dump.rb geoip2_compat 66.102.3.80
|
284
|
+
$ bundle exec ruby urils/dump.rb geoip 66.102.3.80
|
285
|
+
```
|
365
286
|
|
366
|
-
|
367
|
-
* `geoip_database`: path to your GeoLiteCity.dat
|
287
|
+
## Placeholders
|
368
288
|
|
369
289
|
### GeoIP2
|
370
290
|
|
@@ -405,43 +325,89 @@ Related configurations:
|
|
405
325
|
* `backend_library`: `geoip2_compat` or `geoip2_c`
|
406
326
|
* `geoip2_database`: path to your GeoLite2-City.mmdb
|
407
327
|
|
328
|
+
### GeoIP legacy
|
329
|
+
|
330
|
+
Provides these placeholders for adding field of geolocate results.<br />
|
331
|
+
For more example of geolocating, you can try these websites like [Geo IP Address View](http://www.geoipview.com/) or [View my IP information](http://www.geoiptool.com/en/).
|
332
|
+
|
333
|
+
| placeholder attributes | output example | type | note |
|
334
|
+
|--------------------------------|-------------------|--------------|------|
|
335
|
+
| ${city[lookup_field]} | "Ithaca" | varchar(255) | - |
|
336
|
+
| ${latitude[lookup_field]} | 42.4277992248535 | decimal | - |
|
337
|
+
| ${longitude[lookup_field]} | -76.4981994628906 | decimal | - |
|
338
|
+
| ${country_code3[lookup_field]} | "USA" | varchar(3) | - |
|
339
|
+
| ${country_code[lookup_field]} | "US" | varchar(2) | A two-character ISO 3166-1 country code |
|
340
|
+
| ${country_name[lookup_field]} | "United States" | varchar(50) | - |
|
341
|
+
| ${dma_code[lookup_field]} | 555 | unsigned int | **only for US** |
|
342
|
+
| ${area_code[lookup_field]} | 607 | char(3) | **only for US** |
|
343
|
+
| ${region[lookup_field]} | "NY" | char(2) | A two character ISO-3166-2 or FIPS 10-4 code |
|
344
|
+
|
345
|
+
Further more specification available at http://dev.maxmind.com/geoip/legacy/csv/#GeoIP_City_Edition_CSV_Database_Fields
|
346
|
+
|
347
|
+
Related configurations:
|
348
|
+
|
349
|
+
* `backend_library`: `geoip` (default)
|
350
|
+
* `geoip_database`: path to your GeoLiteCity.dat
|
351
|
+
|
408
352
|
## Parameters
|
409
353
|
|
410
|
-
###
|
354
|
+
### GeoipFilter
|
411
355
|
|
412
|
-
|
413
|
-
* `tag_key`
|
356
|
+
Note that filter version of `geoip` plugin does not have handling `tag` feature.
|
414
357
|
|
415
|
-
|
416
|
-
Further details are written at http://docs.fluentd.org/articles/in_exec
|
358
|
+
#### Plugin helpers
|
417
359
|
|
418
|
-
*
|
360
|
+
* [compat_parameters](https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-compat_parameters)
|
361
|
+
* [inject](https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-inject)
|
419
362
|
|
420
|
-
|
421
|
-
On the case of getting nothing of GeoIP info (such as local IP), it will output the original record without changing anything.
|
363
|
+
See also [Filter Plugin Overview](https://docs.fluentd.org/filter)
|
422
364
|
|
423
|
-
|
365
|
+
#### Supported sections
|
424
366
|
|
425
|
-
|
367
|
+
* [Inject section configurations](https://docs.fluentd.org/configuration/inject-section)
|
426
368
|
|
427
|
-
|
369
|
+
#### Parameters
|
428
370
|
|
429
|
-
|
371
|
+
[Plugin Common Paramteters](https://docs.fluentd.org/configuration/plugin-common-parameters)
|
430
372
|
|
431
|
-
|
373
|
+
**geoip_database** (string) (optional)
|
432
374
|
|
433
|
-
|
375
|
+
* Default value: bundled database `GeoLiteCity.dat`
|
376
|
+
|
377
|
+
Path to GeoIP database file.
|
378
|
+
|
379
|
+
**geoip2_database** (string) (optional)
|
380
|
+
|
381
|
+
* Default value: bundled database `GeoLite2-City.mmdb`.
|
382
|
+
|
383
|
+
Path to GeoIP2 database file.
|
434
384
|
|
435
|
-
|
385
|
+
**geoip_lookup_keys** (array) (optional)
|
436
386
|
|
437
|
-
|
438
|
-
Further details are written at http://docs.fluentd.org/articles/in_exec
|
387
|
+
* Default_value: `["host"]`
|
439
388
|
|
440
|
-
|
389
|
+
Specify one or more geoip lookup field which has IP address.
|
390
|
+
|
391
|
+
**geoip_lookup_key** (string) (optional) (deprecated)
|
392
|
+
|
393
|
+
* Default value: `nil`.
|
394
|
+
|
395
|
+
Use geoip_lookup_keys instead.
|
396
|
+
|
397
|
+
**skip_adding_null_record** (bool) (optional)
|
398
|
+
|
399
|
+
* Default value: `nil`
|
441
400
|
|
442
401
|
Skip adding geoip fields when this valaues to `true`.
|
443
402
|
On the case of getting nothing of GeoIP info (such as local IP), it will output the original record without changing anything.
|
444
403
|
|
404
|
+
**backend_library** (enum) (optional)
|
405
|
+
|
406
|
+
* Available values: `geoip`, `geoip2_compat`, `geoip2_c`
|
407
|
+
* Default value: `geoip2_c`.
|
408
|
+
|
409
|
+
Set backend library.
|
410
|
+
|
445
411
|
## Articles
|
446
412
|
|
447
413
|
* [IPアドレスを元に位置情報をリアルタイムに付与する fluent-plugin-geoip v0.0.1をリリースしました #fluentd - Y-Ken Studio](http://y-ken.hatenablog.com/entry/fluent-plugin-geoip-has-released)<br />
|
@@ -463,8 +429,6 @@ http://developer.smartnews.be/blog/2013/10/03/easy-data-analysis-using-fluentd-r
|
|
463
429
|
|
464
430
|
Pull requests are very welcome!!
|
465
431
|
|
466
|
-
* support [GeoIP2](http://dev.maxmind.com/geoip/geoip2/whats-new-in-geoip2/)
|
467
|
-
|
468
432
|
## Contributing
|
469
433
|
|
470
434
|
1. Fork it
|