fluent-plugin-groonga 1.1.0 → 1.1.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (14) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +24 -13
  3. data/doc/text/configuration.md +55 -30
  4. data/doc/text/constitution.md +83 -83
  5. data/doc/text/news.md +11 -0
  6. data/fluent-plugin-groonga.gemspec +2 -2
  7. data/lib/fluent/plugin/in_groonga.rb +11 -3
  8. data/sample/command.conf +2 -2
  9. data/sample/gqtp.conf +2 -2
  10. data/sample/http.conf +2 -2
  11. data/sample/store-apache.conf +5 -5
  12. data/sample/store-syslog.conf +6 -5
  13. data/sample/store.conf +2 -2
  14. metadata +3 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 1d63f8c4f3214cc7b8c43876faadcb3be1aae8bf
4
- data.tar.gz: 0ffcbf7f152c8824991d50616cb277ec88396ac7
3
+ metadata.gz: 409393e5f37e320bd7236591e119c552ddcc6490
4
+ data.tar.gz: f7070f11fb9c6126ca5ddd07786e510e6209cf6a
5
5
  SHA512:
6
- metadata.gz: 6e91d9055541ceb9ca4adf8a695f3f6752400860c9bd765647a841c378e1e885037db339356758993604ec7a60b3a40af66252ff0d8eecde7c5c714546c1984e
7
- data.tar.gz: 4d83595cf18cfc73fb60332ba235e5f38814df328cb2c9c624bafc69596c3bdb9d7052db2e364163d1108cf1a2fdddd499d24bac9845018b1ff5f330da377e4a
6
+ metadata.gz: dd76ea51915f8f84d6f2d5cfc2245fb02e05cee1eedcba341142ac020f43d72db488ac87aac1e40a7dc0d98674bb45124bb0c541e7f010a7dd0478ae857ae785
7
+ data.tar.gz: 7c29cbb8d38341cf3d36e0d5ff61626baf3e1a16aea1bef71a0ba5146495c6300e70936c3d62a208028cf9bf06766342d8c09aae2489a3cb7ff05fda18032947
data/README.md CHANGED
@@ -13,6 +13,7 @@ Fluent-plugin-groonga is a Fluentd plugin collection to use
13
13
  supports the following two usages:
14
14
 
15
15
  * Store logs collected by Fluentd to Groonga.
16
+
16
17
  * Implement replication system for Groonga.
17
18
 
18
19
  The first usage is normal usage. You can store logs to Groonga and
@@ -32,7 +33,7 @@ The following configuration stores all data in `/var/log/messages`
32
33
  into Groonga:
33
34
 
34
35
  <source>
35
- type tail
36
+ @type tail
36
37
  format syslog
37
38
  path /var/log/syslog.1
38
39
  pos_file /tmp/messages.pos
@@ -41,7 +42,7 @@ into Groonga:
41
42
  </source>
42
43
 
43
44
  <match log.**>
44
- type groonga
45
+ @type groonga
45
46
  store_table logs
46
47
 
47
48
  protocol http
@@ -75,6 +76,7 @@ You can replicate your data by using `copy` output plugin.
75
76
  There are two usages:
76
77
 
77
78
  * Store logs collected by Fluentd to Groonga.
79
+
78
80
  * Implement replication system for Groonga.
79
81
 
80
82
  They are described in other sections.
@@ -93,7 +95,7 @@ There is one required parameter:
93
95
  Here is a minimum configuration:
94
96
 
95
97
  <match log.**>
96
- type groonga
98
+ @type groonga
97
99
  store_table logs
98
100
  </match>
99
101
 
@@ -103,18 +105,25 @@ on `localhost`.
103
105
  There are optional parameters:
104
106
 
105
107
  * `protocol`: It specifies protocol to communicate Groonga server.
108
+
106
109
  * Available values: `http`, `gqtp`, `command`
110
+
107
111
  * Default: `http`
112
+
108
113
  * `host`: It specifies host name or IP address of Groonga server.
114
+
109
115
  * Default: `127.0.0.1`
116
+
110
117
  * `port`: It specifies port number of Groonga server.
118
+
111
119
  * Default for `http` protocol: `10041`
120
+
112
121
  * Default for `gqtp` protocol: `10043`
113
122
 
114
123
  Here is a configuration that specifies optional parameters explicitly:
115
124
 
116
125
  <match log.**>
117
- type groonga
126
+ @type groonga
118
127
  store_table logs
119
128
 
120
129
  protocol http
@@ -132,7 +141,7 @@ Note that there is special tag name. You can't use
132
141
  use the following configuration:
133
142
 
134
143
  <match groonga.command.*>
135
- type groonga
144
+ @type groonga
136
145
  # ...
137
146
  </match>
138
147
 
@@ -144,14 +153,15 @@ replication system for Groonga.
144
153
  See the following documents how to implement replication system for
145
154
  Groonga:
146
155
 
147
- * [Configuration](doc/text/configuration.md)
148
- ([on the Web](http://groonga.org/fluent-plugin-groonga/en/file.configuration.html))
149
- * [Constitution](doc/text/constitution.md)
150
- ([on the Web](http://groonga.org/fluent-plugin-groonga/en/file.constitution.html))
156
+ * [Configuration](doc/text/configuration.md)
157
+ ([on the Web](http://groonga.org/fluent-plugin-groonga/en/file.configuration.html))
158
+
159
+ * [Constitution](doc/text/constitution.md)
160
+ ([on the Web](http://groonga.org/fluent-plugin-groonga/en/file.constitution.html))
151
161
 
152
162
  ## Authors
153
163
 
154
- * Kouhei Sutou `<kou@clear-code.com>`
164
+ * Kouhei Sutou `<kou@clear-code.com>`
155
165
 
156
166
  ## License
157
167
 
@@ -162,8 +172,9 @@ contributed patches.)
162
172
 
163
173
  ## Mailing list
164
174
 
165
- * English: [groonga-talk](https://lists.sourceforge.net/lists/listinfo/groonga-talk)
166
- * Japanese: [groonga-dev](http://lists.sourceforge.jp/mailman/listinfo/groonga-dev)
175
+ * English: [groonga-talk](https://lists.sourceforge.net/lists/listinfo/groonga-talk)
176
+
177
+ * Japanese: [groonga-dev](http://lists.sourceforge.jp/mailman/listinfo/groonga-dev)
167
178
 
168
179
  ## Source
169
180
 
@@ -172,4 +183,4 @@ The repository for fluent-plugin-groonga is on
172
183
 
173
184
  ## Thanks
174
185
 
175
- * ...
186
+ * ...
data/doc/text/configuration.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  # Configuration
4
4
 
5
- Fluent-plugin-groonga includes two fluentd plugins. They are the
5
+ Fluent-plugin-groonga includes two Fluentd plugins. They are the
6
6
  `groonga` input plugin and the `groonga` output plugin. This documents
7
7
  describes configuration parameters of them.
8
8
 
@@ -10,23 +10,34 @@ describes configuration parameters of them.
10
10
 
11
11
  Here are available parameters:
12
12
 
13
- * `protocol`: It specifies protocol for receiving groonga commands.
14
- * available values: `http` and `gqtp`
15
- * default: `http`
16
- * `bind`: It specifies bind address.
17
- * default: `0.0.0.0`
18
- * `port`: It specifies port number.
19
- * default: `10041`
20
- * `real_host`: It specifies real groonga server's address. It is required.
21
- * default: no default.
22
- * `real_port`: It specifies real groonga server's port number.
23
- * default: `10041`
24
- * `emit_commands`: TODO
13
+ * `protocol`: It specifies protocol for receiving Groonga commands.
14
+
15
+ * available values: `http` and `gqtp`
16
+
17
+ * default: `http`
18
+
19
+ * `bind`: It specifies bind address.
20
+
21
+ * default: `0.0.0.0`
22
+
23
+ * `port`: It specifies port number.
24
+
25
+ * default: `10041`
26
+
27
+ * `real_host`: It specifies real Groonga server's address. It is required.
28
+
29
+ * default: no default.
30
+
31
+ * `real_port`: It specifies real Groonga server's port number.
32
+
33
+ * default: `10041`
34
+
35
+ * `emit_commands`: TODO
25
36
 
26
37
  Here is an example:
27
38
 
28
39
  <source>
29
- type groonga
40
+ @type groonga
30
41
  protocol http
31
42
  bind 127.0.0.1
32
43
  port 10041
@@ -36,26 +47,40 @@ Here is an example:
36
47
 
37
48
  ## The `groonga` output plugin
38
49
 
39
- * `protocol`: It specifies protocol for sending groonga commands to groonga.
40
- * available values: `http`, `gqtp` and `command`
41
- * default: `http`
42
- * For `http` and `gqtp` use:
43
- * `host`: It specifies groonga server's address.
44
- * default: `localhost`
45
- * `port`: It specifies groonga server's port number.
46
- * default: `10041`
47
- * For `command` use:
48
- * `groonga`: It specifies path of groonga command.
49
- * default: `groonga`
50
- * `database`: It specifies path of groonga database. It is required.
51
- * default: no default.
52
- * `arguments`: It specifies additional arguments for groonga command.
53
- * default: no additional arguments.
50
+ * `protocol`: It specifies protocol for sending Groonga commands to Groonga.
51
+
52
+ * available values: `http`, `gqtp` and `command`
53
+
54
+ * default: `http`
55
+
56
+ * For `http` and `gqtp` use:
57
+
58
+ * `host`: It specifies Groonga server's address.
59
+
60
+ * default: `localhost`
61
+
62
+ * `port`: It specifies Groonga server's port number.
63
+
64
+ * default: `10041`
65
+
66
+ * For `command` use:
67
+
68
+ * `groonga`: It specifies path of `groonga` command.
69
+
70
+ * default: `groonga`
71
+
72
+ * `database`: It specifies path of Groonga database. It is required.
73
+
74
+ * default: no default.
75
+
76
+ * `arguments`: It specifies additional arguments for `groonga` command.
77
+
78
+ * default: no additional arguments.
54
79
 
55
80
  Here is an example:
56
81
 
57
82
  <match groonga.command.*>
58
- type groonga
83
+ @type groonga
59
84
  protocol command
60
85
  database /tmp/groonga/db
61
86
  </match>
data/doc/text/constitution.md CHANGED
@@ -3,7 +3,7 @@
3
3
  # Constitution
4
4
 
5
5
  You can chose some system constitutions to implemented replication
6
- ready groonga system. This document describes some patterns.
6
+ ready Groonga system. This document describes some patterns.
7
7
 
8
8
  Here are available patterns:
9
9
 
@@ -25,15 +25,15 @@ configure your system constitution.
25
25
 
26
26
  ### Small system
27
27
 
28
- In small system, you just has two servers. One is the master groonga
29
- server and the other is the slave groonga server. You send all update
28
+ In small system, you just has two servers. One is the master Groonga
29
+ server and the other is the slave Groonga server. You send all update
30
30
  commands (e.g. `table_create`, `column_create`, `load` and so on.) to
31
31
  fluentd. In fluentd, the `groonga` input plugin receives commands from
32
- client, passes through them to master groonga server and passes
33
- through responses from master groonga server to client. The `groonga`
32
+ client, passes through them to master Groonga server and passes
33
+ through responses from master Groonga server to client. The `groonga`
34
34
  input plugin converts update commands to fluent messages when the
35
35
  `groonga` input plugin passes through comamands and responses. The
36
- fluent messages are sent to slave groonga server by the `groonga`
36
+ fluent messages are sent to slave Groonga server by the `groonga`
37
37
  output plugin.
38
38
 
39
39
  Here is a diagram of this constitution.
@@ -42,56 +42,56 @@ Here is a diagram of this constitution.
42
42
  and and
43
43
  search +---------+ search +---------+
44
44
  +--------+ <------> | fluentd | <------> | master |
45
- | | +---------+ | groonga |
45
+ | | +---------+ | Groonga |
46
46
  | client | update | +---------+
47
47
  | | \_/
48
48
  | | search +---------+
49
49
  +--------+ <------> | slave |
50
- | groonga |
50
+ | Groonga |
51
51
  +---------+
52
52
 
53
- Fluentd should be placed at client or master groonga server. If you
53
+ Fluentd should be placed at client or master Groonga server. If you
54
54
  have only one client that updates data, client side is reasonable. If
55
- you have multiple clients that update data, master groonga server side
55
+ you have multiple clients that update data, master Groonga server side
56
56
  is reasonable.
57
57
 
58
58
  You can use replication for high performance by providing search
59
59
  service with multi servers. You can't use replication for high
60
- availability. If master groonga server or fluentd is down, this system
61
- can't update data. (Searching is still available because slabe groonga
60
+ availability. If master Groonga server or fluentd is down, this system
61
+ can't update data. (Searching is still available because slave Groonga
62
62
  server is alive.)
63
63
 
64
64
  Here is an example configuration file:
65
65
 
66
- # For master groonga server
66
+ # For master Groonga server
67
67
  <source>
68
- type groonga
68
+ @type groonga
69
69
  protocol gqtp # Or use the below line
70
70
  # protocol http
71
- bind 127.0.0.1 # For client side fluentd
72
- # bind 192.168.0.1 # For master groonga server side fluentd
71
+ bind 127.0.0.1 # For client side Fluentd
72
+ # bind 192.168.0.1 # For master Groonga server side Fluentd
73
73
  port 10041
74
- real_host 192.168.29.1 # IP address of master groonga server
75
- real_port 10041 # Port number of master groonga server
74
+ real_host 192.168.29.1 # IP address of master Groonga server
75
+ real_port 10041 # Port number of master Groonga server
76
76
  # real_port 20041 # Use different port number
77
- # for master groonga server side fluentd
77
+ # for master Groonga server side Fluentd
78
78
  </source>
79
79
 
80
- # For slave groonga server
80
+ # For slave Groonga server
81
81
  <match groonga.command.*>
82
- type groonga
82
+ @type groonga
83
83
  protocol gqtp # Or use the below line
84
84
  # protocol http # You can use different protocol for
85
- # master groonga server and slave groonga server
86
- host 192.168.29.29 # IP address of slave groonga server
87
- port 10041 # Port number of slave groonga server
85
+ # master Groonga server and slave Groonga server
86
+ host 192.168.29.29 # IP address of slave Groonga server
87
+ port 10041 # Port number of slave Groonga server
88
88
 
89
89
  # Buffer
90
90
  flush_interval 1s # Use small value for less delay replication
91
91
 
92
92
  ## Use the following configurations to support resending data to
93
- ## recovered slave groonga server. If you don't care about slave
94
- ## groonga server is down case, you don't need the following
93
+ ## recovered slave Groonga server. If you don't care about slave
94
+ ## Groonga server is down case, you don't need the following
95
95
  ## configuration.
96
96
 
97
97
  ## For supporting resending data after fluentd is restarted
@@ -101,7 +101,7 @@ Here is an example configuration file:
101
101
  ## A value in load command is a chunk.
102
102
  # buffer_chunk_limit 256m
103
103
  ## Use large value if you want to support resending data after
104
- ## slave groonga server is down long time.
104
+ ## slave Groonga server is down long time.
105
105
  ## 17: about 1.5day =
106
106
  ## ((2 ** 0) + (2 ** 1) + ... + (2 ** 17)) / 60.0 / 60.0 / 24.0
107
107
  ## (default)
@@ -121,53 +121,53 @@ commands.
121
121
 
122
122
  You cannot update data until fluentd is up.
123
123
 
124
- #### How to recover from master groonga server down
124
+ #### How to recover from master Groonga server down
125
125
 
126
- Here are recover steps when master groonga server is down:
126
+ Here are recover steps when master Groonga server is down:
127
127
 
128
128
  1. Stop fluentd.
129
129
  2. Run `grndump /PATH/TO/SLAVE/GROONGA/SERVER/DB >
130
- SLAVE_GROONGA_DUMP.grn` on slave groonga server host.
130
+ SLAVE_GROONGA_DUMP.grn` on slave Groonga server host.
131
131
  3. Run `groonga -n /PATH/TO/MASTER/GROONGA/SERVER/DB <
132
- SLAVE_GROONGA_DUMP.grn` on master groonga server.
133
- 4. Run master groonga server.
132
+ SLAVE_GROONGA_DUMP.grn` on master Groonga server.
133
+ 4. Run master Groonga server.
134
134
  5. Start fluentd.
135
135
 
136
136
  You cannot update data until you finish to recover.
137
137
 
138
- #### How to recover from slave groonga server down
138
+ #### How to recover from slave Groonga server down
139
139
 
140
- Here are recover steps when slave groonga server is down:
140
+ Here are recover steps when slave Groonga server is down:
141
141
 
142
142
  1. Run `grndump /PATH/TO/MASTER/GROONGA/SERVER/DB >
143
- MASTER_GROONGA_DUMP.grn` on master groonga server host.
143
+ MASTER_GROONGA_DUMP.grn` on master Groonga server host.
144
144
  2. Run `groonga -n /PATH/TO/SLAVE/GROONGA/SERVER/DB <
145
- MASTER_GROONGA_DUMP.grn` on slave groonga server.
146
- 3. Run slave groonga server.
145
+ MASTER_GROONGA_DUMP.grn` on slave Groonga server.
146
+ 3. Run slave Groonga server.
147
147
 
148
148
  You can update data while you recover. If your system can't process
149
- all search requests by only master groonga server, your system will be
149
+ all search requests by only master Groonga server, your system will be
150
150
  down.
151
151
 
152
- You need to recover slave groonga server before fluentd's buffer queue
152
+ You need to recover slave Groonga server before fluentd's buffer queue
153
153
  is full (see `buffer_queue_limit`) or fluentd gives up retrying (see
154
154
  `retry_limit`). Here are recover steps when you cannot recover slave
155
- groonga server before those situations:
155
+ Groonga server before those situations:
156
156
 
157
157
  1. Stop fluentd.
158
158
  2. Run `grndump /PATH/TO/MASTER/GROONGA/SERVER/DB >
159
- MASTER_GROONGA_DUMP.grn` on master groonga server host.
159
+ MASTER_GROONGA_DUMP.grn` on master Groonga server host.
160
160
  3. Run `groonga -n /PATH/TO/SLAVE/GROONGA/SERVER/DB <
161
- MASTER_GROONGA_DUMP.grn` on slave groonga server host.
162
- 4. Run slave groonga server.
161
+ MASTER_GROONGA_DUMP.grn` on slave Groonga server host.
162
+ 4. Run slave Groonga server.
163
163
  5. Start fluentd.
164
164
 
165
165
  You cannot update data until you finish to recover.
166
166
 
167
167
  ### Medium system
168
168
 
169
- In medium system, you have three or more slave groonga servers. Fluentd
170
- updates two or more slave groonga servers with the `copy` output
169
+ In medium system, you have three or more slave Groonga servers. Fluentd
170
+ updates two or more slave Groonga servers with the `copy` output
171
171
  plugin and the `groonga` output plugin.
172
172
 
173
173
  Here is a diagram of this constitution.
@@ -176,64 +176,64 @@ Here is a diagram of this constitution.
176
176
  and and
177
177
  search +---------+ search +---------+
178
178
  +--------+ <------> | fluentd | <------> | master |
179
- | | +---------+ | groonga |
179
+ | | +---------+ | Groonga |
180
180
  | client | +--------+ +---------+
181
181
  | | |
182
182
  +--------+ search +---------+ |
183
183
  | | <------> | slave | <-+ update
184
- | client | | groonga | |
184
+ | client | | Groonga | |
185
185
  | | +---------+ |
186
186
  +--------+ search +---------+ |
187
187
  | | <------> | slave | <-+ update
188
- | client | | groonga | |
188
+ | client | | Groonga | |
189
189
  | | +---------+ |
190
190
  +- ... -+ ... ... ...
191
191
 
192
192
  Here is an example configuration file:
193
193
 
194
- # For master groonga server
194
+ # For master Groonga server
195
195
  <source>
196
- type groonga
196
+ @type groonga
197
197
  protocol gqtp # Or use the below line
198
198
  # protocol http
199
- bind 127.0.0.1 # For client side fluentd
200
- # bind 192.168.0.1 # For master groonga server side fluentd
199
+ bind 127.0.0.1 # For client side Fluentd
200
+ # bind 192.168.0.1 # For master Groonga server side Fluentd
201
201
  port 10041
202
- real_host 192.168.29.1 # IP address of master groonga server
203
- real_port 10041 # Port number of master groonga server
202
+ real_host 192.168.29.1 # IP address of master Groonga server
203
+ real_port 10041 # Port number of master Groonga server
204
204
  # real_port 20041 # Use different port number
205
- # for master groonga server side fluentd
205
+ # for master Groonga server side fluentd
206
206
  </source>
207
207
 
208
- # For slave groonga servers
208
+ # For slave Groonga servers
209
209
  <match groonga.command.*>
210
- type copy
210
+ @type copy
211
211
 
212
- # The first slave groonga server
212
+ # The first slave Groonga server
213
213
  <store>
214
- type groonga
214
+ @type groonga
215
215
  protocol gqtp # Or use the below line
216
216
  # protocol http # You can use different protocol for
217
- # master groonga server and slave groonga server
218
- host 192.168.29.2 # IP address of slave groonga server
219
- port 10041 # Port number of slave groonga server
217
+ # master Groonga server and slave Groonga server
218
+ host 192.168.29.2 # IP address of slave Groonga server
219
+ port 10041 # Port number of slave Groonga server
220
220
 
221
221
  # Buffer
222
222
  flush_interval 1s # Use small value for less delay replication
223
223
 
224
224
  ## Use the following configurations to support resending data to
225
- ## recovered slave groonga server. If you don't care about slave
226
- ## groonga server is down case, you don't need the following
225
+ ## recovered slave Groonga server. If you don't care about slave
226
+ ## Groonga server is down case, you don't need the following
227
227
  ## configuration.
228
228
 
229
229
  ## For supporting resending data after fluentd is restarted
230
230
  # buffer_type file
231
- # buffer_path /var/log/fluent/groonga.*.buffer
231
+ # buffer_path /var/log/fluent/groonga1.*.buffer
232
232
  ## Use large value if a record has many data in load command.
233
233
  ## A value in load command is a chunk.
234
234
  # buffer_chunk_limit 256m
235
235
  ## Use large value if you want to support resending data after
236
- ## slave groonga server is down long time.
236
+ ## slave Groonga server is down long time.
237
237
  ## 17: about 1.5day =
238
238
  ## ((2 ** 0) + (2 ** 1) + ... + (2 ** 17)) / 60.0 / 60.0 / 24.0
239
239
  ## (default)
@@ -245,22 +245,22 @@ Here is an example configuration file:
245
245
  # buffer_queue_limit 10000
246
246
  </store>
247
247
 
248
- # The second slave groonga server
248
+ # The second slave Groonga server
249
249
  <store>
250
- type groonga
250
+ @type groonga
251
251
  protocol gqtp # Or use the below line
252
252
  # protocol http # You can use different protocol for
253
- # master groonga server and slave groonga server
254
- host 192.168.29.3 # IP address of slave groonga server
255
- port 10041 # Port number of slave groonga server
253
+ # master Groonga server and slave Groonga server
254
+ host 192.168.29.3 # IP address of slave Groonga server
255
+ port 10041 # Port number of slave Groonga server
256
256
 
257
257
  # Buffer
258
258
  # ...
259
259
  </store>
260
260
 
261
- # More slave groonga servers
261
+ # More slave Groonga servers
262
262
  # <store>
263
- # type groonga
263
+ # @type groonga
264
264
  # ...
265
265
  # </store>
266
266
  </match>
@@ -269,12 +269,12 @@ TODO: ...
269
269
 
270
270
  ### Large system
271
271
 
272
- In large system, you have two or more slave groonga server clusters.
273
- Fluentd that connects with master groonga server updates two or more
274
- fluentds that are in slave groonga server clusters with the `copy`
272
+ In large system, you have two or more slave Groonga server clusters.
273
+ Fluentd that connects with master Groonga server updates two or more
274
+ fluentds that are in slave Groonga server clusters with the `copy`
275
275
  output plugin and the `forward` output plugin. A slave cluster has a
276
- fluentd. Fluentd in slave groonga server clusters updates slave
277
- groonga server in the same slave groonga server cluster by the `copy`
276
+ fluentd. Fluentd in slave Groonga server clusters updates slave
277
+ Groonga server in the same slave Groonga server cluster by the `copy`
278
278
  output plugin and `groonga` output plugin.
279
279
 
280
280
  Here is a diagram of this constitution.
@@ -283,18 +283,18 @@ Here is a diagram of this constitution.
283
283
  and and
284
284
  search +---------+ search +---------+
285
285
  +--------+ <--------> | fluentd | <------> | master |
286
- | | +---------+ | groonga |
286
+ | | +---------+ | Groonga |
287
287
  | client | | +---------+
288
288
  | | +------------------------------+
289
289
  +--------+ +----------------------------------+ |
290
290
  | | | slave cluster | |
291
291
  | client | search | +---------+ update +---------+ | |
292
292
  | | <------> | | slave | <------- | fluentd | <-+ update
293
- +--------| | | groonga | +---------+ | |
293
+ +--------| | | Groonga | +---------+ | |
294
294
  | | | +---------+ +-----------+ | |
295
295
  | client | search | +---------+ | | |
296
296
  | | <------> | | slave | <-+ update | |
297
- +--------| | | groonga | | | |
297
+ +--------| | | Groonga | | | |
298
298
  | | | +---------+ | | |
299
299
  | ... | ... | ... ... | |
300
300
  +----------------------------------+ |
@@ -302,11 +302,11 @@ Here is a diagram of this constitution.
302
302
  | | | slave cluster | |
303
303
  | client | search | +---------+ update +---------+ | |
304
304
  | | <------> | | slave | <------- | fluentd | <-+ update
305
- +--------| | | groonga | +---------+ | |
305
+ +--------| | | Groonga | +---------+ | |
306
306
  | | | +---------+ +-----------+ | |
307
307
  | client | search | +---------+ | | |
308
308
  | | <------> | | slave | <-+ update | |
309
- +--------| | | groonga | | | |
309
+ +--------| | | Groonga | | | |
310
310
  | | | +---------+ | | |
311
311
  | ... | ... | ... ... | |
312
312
  +----------------------------------+ |
data/doc/text/news.md CHANGED
@@ -2,6 +2,17 @@
2
2
 
3
3
  # News
4
4
 
5
+ ## 1.1.1: 2016-05-27
6
+
7
+ ### Improvements
8
+
9
+ * in: Stopped using deprecated API.
10
+ [GitHub#6][Reported by okkez]
11
+
12
+ ### Thanks
13
+
14
+ * okkez
15
+
5
16
  ## 1.1.0: 2016-01-24
6
17
 
7
18
  ### Improvements
data/fluent-plugin-groonga.gemspec CHANGED
@@ -1,6 +1,6 @@
1
1
  # -*- mode: ruby; coding: utf-8 -*-
2
2
  #
3
- # Copyright (C) 2012-2014 Kouhei Sutou <kou@clear-code.com>
3
+ # Copyright (C) 2012-2016 Kouhei Sutou <kou@clear-code.com>
4
4
  #
5
5
  # This library is free software; you can redistribute it and/or
6
6
  # modify it under the terms of the GNU Lesser General Public
@@ -17,7 +17,7 @@
17
17
 
18
18
  Gem::Specification.new do |spec|
19
19
  spec.name = "fluent-plugin-groonga"
20
- spec.version = "1.1.0"
20
+ spec.version = "1.1.1"
21
21
  spec.authors = ["Kouhei Sutou"]
22
22
  spec.email = ["kou@clear-code.com"]
23
23
  spec.summary = "Fluentd plugin to store data into Groonga and implement Groonga replication system."
data/lib/fluent/plugin/in_groonga.rb CHANGED
@@ -24,6 +24,8 @@ require "http_parser"
24
24
  require "gqtp"
25
25
  require "groonga/command/parser"
26
26
 
27
+ require "fluent/process"
28
+
27
29
  module Fluent
28
30
  class GroongaInput < Input
29
31
  Plugin.register_input("groonga", self)
@@ -45,9 +47,9 @@ module Fluent
45
47
  super
46
48
  case @protocol
47
49
  when :http
48
- @input = HTTPInput.new
50
+ @input = HTTPInput.new(self)
49
51
  when :gqtp
50
- @input = GQTPInput.new
52
+ @input = GQTPInput.new(self)
51
53
  end
52
54
  @input.configure(conf)
53
55
  end
@@ -107,6 +109,10 @@ module Fluent
107
109
  end
108
110
  end
109
111
 
112
+ def initialize(input_plugin)
113
+ @input_plugin = input_plugin
114
+ end
115
+
110
116
  def configure(conf)
111
117
  super
112
118
 
@@ -147,7 +153,9 @@ module Fluent
147
153
 
148
154
  def emit(command, params)
149
155
  return unless emit_command?(command)
150
- Engine.emit("groonga.command.#{command}", Engine.now, params)
156
+ @input_plugin.router.emit("groonga.command.#{command}",
157
+ Engine.now,
158
+ params)
151
159
  end
152
160
 
153
161
  private
data/sample/command.conf CHANGED
@@ -1,5 +1,5 @@
1
1
  <source>
2
- type groonga
2
+ @type groonga
3
3
  protocol gqtp
4
4
 
5
5
  real_host 127.0.0.1
@@ -7,7 +7,7 @@
7
7
  </source>
8
8
 
9
9
  <match groonga.command.*>
10
- type groonga
10
+ @type groonga
11
11
  protocol command
12
12
  database /tmp/groonga/db
13
13
  flush_interval 1
data/sample/gqtp.conf CHANGED
@@ -1,5 +1,5 @@
1
1
  <source>
2
- type groonga
2
+ @type groonga
3
3
  protocol gqtp
4
4
 
5
5
  real_host 127.0.0.1
@@ -7,7 +7,7 @@
7
7
  </source>
8
8
 
9
9
  <match groonga.command.*>
10
- type groonga
10
+ @type groonga
11
11
  protocol gqtp
12
12
  host 127.0.0.1
13
13
  port 30043
data/sample/http.conf CHANGED
@@ -1,5 +1,5 @@
1
1
  <source>
2
- type groonga
2
+ @type groonga
3
3
  protocol http
4
4
 
5
5
  real_host 127.0.0.1
@@ -7,7 +7,7 @@
7
7
  </source>
8
8
 
9
9
  <match groonga.command.*>
10
- type groonga
10
+ @type groonga
11
11
  protocol http
12
12
  host 127.0.0.1
13
13
  port 30041
data/sample/store-apache.conf CHANGED
@@ -1,9 +1,9 @@
1
1
  <source>
2
- type forward
2
+ @type forward
3
3
  </source>
4
4
 
5
5
  <source>
6
- type tail
6
+ @type tail
7
7
  path /var/log/apache2/access.log
8
8
  pos_file /tmp/apache_access.pos
9
9
  tag apache.raw.log.apache.access
@@ -12,7 +12,7 @@
12
12
  </source>
13
13
 
14
14
  <match apache.**>
15
- type record_reformer
15
+ @type record_reformer
16
16
  enable_ruby false
17
17
 
18
18
  tag ${tag_suffix[1]}
@@ -23,7 +23,7 @@
23
23
  </match>
24
24
 
25
25
  <match raw.log.**>
26
- type record_reformer
26
+ @type record_reformer
27
27
  enable_ruby false
28
28
 
29
29
  tag ${tag_suffix[1]}
@@ -36,7 +36,7 @@
36
36
  </match>
37
37
 
38
38
  <match log.**>
39
- type groonga
39
+ @type groonga
40
40
  store_table Logs
41
41
 
42
42
  protocol http
data/sample/store-syslog.conf CHANGED
@@ -5,19 +5,20 @@
5
5
  # * fluent-plugin-groonga
6
6
 
7
7
  <source>
8
- type forward
8
+ @type forward
9
9
  </source>
10
10
 
11
11
  <source>
12
- type tail
12
+ @type tail
13
13
  path /var/log/messages
14
14
  pos_file /var/log/td-agent/messages.pos
15
15
  tag "raw.messages.log.#{Socket.gethostname}"
16
+ read_from_head true
16
17
  format none
17
18
  </source>
18
19
 
19
20
  <match raw.*.log.**>
20
- type forest
21
+ @type forest
21
22
  subtype parser
22
23
 
23
24
  <template>
@@ -31,7 +32,7 @@
31
32
  </match>
32
33
 
33
34
  <match *.log.*.**>
34
- type record_reformer
35
+ @type record_reformer
35
36
  enable_ruby false
36
37
 
37
38
  tag ${tag_parts[1]}
@@ -44,7 +45,7 @@
44
45
  </match>
45
46
 
46
47
  <match log>
47
- type groonga
48
+ @type groonga
48
49
  store_table Logs
49
50
 
50
51
  protocol http
data/sample/store.conf CHANGED
@@ -1,9 +1,9 @@
1
1
  <source>
2
- type forward
2
+ @type forward
3
3
  </source>
4
4
 
5
5
  <match log.*>
6
- type groonga
6
+ @type groonga
7
7
  store_table Logs
8
8
 
9
9
  protocol http
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-groonga
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 1.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kouhei Sutou
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-01-24 00:00:00.000000000 Z
11
+ date: 2016-05-27 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: fluentd
@@ -186,7 +186,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
186
186
  version: '0'
187
187
  requirements: []
188
188
  rubyforge_project:
189
- rubygems_version: 2.4.5.1
189
+ rubygems_version: 2.5.1
190
190
  signing_key:
191
191
  specification_version: 4
192
192
  summary: Fluentd plugin to store data into Groonga and implement Groonga replication