fluent-plugin-http-pull 0.8.1 → 0.8.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 1f1c2d52b882ab2674e98771e6aa245d96b71488
4
- data.tar.gz: 6ba43090e0db0a5e01911aef12f2c7c4dc60611d
2
+ SHA256:
3
+ metadata.gz: 001a669d9fa3eccbeaa6f6db2e532e27fe85bfba518463a8dceeac3246abeca6
4
+ data.tar.gz: 19effaa1d73dd98100104ce3aaf3966d23bbf6a63ec76ee6338a33d789d237fb
5
5
  SHA512:
6
- metadata.gz: 886dba56bbd60d2cb78c08cee67a99add6bdaf7b22546787b0acfec0e822320d26bedb3bb2c42945941d0b7e689926b5e9923c8b08845238ad6ebdcafe82757a
7
- data.tar.gz: fd52c8a748cd210ea18f9b1969c68ad3ac3bd72ee0b351bc9b2d03710dabf64a1d6437db63e53eba3c530daa4d50ffcb01ec8294b79a9719936d9ecd84251a36
6
+ metadata.gz: 9d3647df586360e9df56a3aae6379c9cdb9df729a55af9440f441c1d6c1da47527ec576de3038de635a8ca262f3fd052b88990a60971ca00aab11a61fa1d11c0
7
+ data.tar.gz: 6198fc5e08f7832a52524f8ef043e8cf10baba91ec2c42ec5312d2f47bbe79409e08eb296e0997912406ffaa9564d7cb766faf416f756fe4126da0342d4337f2
data/README.md CHANGED
@@ -8,7 +8,37 @@
8
8
 
9
9
  [Fluentd](http://fluentd.org/) input plugin to pull log from rest api.
10
10
 
11
- Many of modern server application offer status reporting API via http (even 'fluentd' too). This plugin will help to gathering status log from these status api.
11
+ Many of modern server application offer status reporting API via http (even 'fluentd' too).
12
+ This plugin will help to gathering status log from these status api.
13
+
14
+ * [Installation](#installation)
15
+ * [Usage](#usage)
16
+ * [Basic Usage](#basic-usage)
17
+ * [Monitor Status Code](#monitoring-http-status-code-only)
18
+ * [Override User Agent](#override-user-agent)
19
+ * [HTTP Basic Auth](#http-basic-auth)
20
+ * [HTTP Proxy](#http-proxy)
21
+ * [Logging Response Header](#logging-http-response-header)
22
+ * [Custom Request Header](#custom-request-header)
23
+ * [HTTPS Support](#https-support)
24
+ * [Configuration](#configuration)
25
+ * [tag](#tag-string-required)
26
+ * [url](#url-string-required)
27
+ * [agent](#agent-string-optional-default-fluent-plugin-http-pull)
28
+ * [interval](#interval-time-required)
29
+ * [format](#format-required)
30
+ * [status_only](#status_only-bool-optional-default-false)
31
+ * [http_method](#http_method-enum-optional-default-get)
32
+ * [timeout](#timeout-time-optional-default-10s)
33
+ * [proxy](#proxy-string-optional-default-nil)
34
+ * [user](#user-string-optional-default-nil)
35
+ * [password](#password-string-optional-default-nil)
36
+ * [response_header](#response_header-section-optional-default-nil)
37
+ * [request_header](#response_header-section-optional-default-nil)
38
+ * [verify_ssl](#verify_ssl-bool-optional-default-true)
39
+ * [ca_path](#ca_path-string-optional-defualt-nil)
40
+ * [ca_file](#ca_file-string-optional-defualt-nil)
41
+ * [Copyright](#copyright)
12
42
 
13
43
  ## Installation
14
44
 
@@ -32,87 +62,227 @@ And then execute:
32
62
  $ bundle
33
63
  ```
34
64
 
35
- ## Example
65
+ ## Usage
36
66
 
37
- You can found more examples in `test/plugin/test_in_http_pull.rb`
67
+ ### basic usage
68
+
69
+ In your Fluentd configuration, use @type http_pull.
70
+ `tag`, `url`, `interval`, `format` is mandatory configuration.
71
+
72
+ ```
73
+ <source>
74
+ @type http_pull
75
+
76
+ tag status
77
+ url http://your-infrastructure/api/status.json
78
+ interval 1s
79
+
80
+ format json
81
+ </source>
82
+
83
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
84
+ ```
85
+
86
+ In this example, a response of your infrastructure parsed as json.
87
+ A result contains `url`, `status`, `message`. `message` is a response of your infrastructure.
88
+
89
+ ```
90
+ {
91
+ "url": "http://your-infrastructure/api/status.json",
92
+ "status": 200,
93
+ "message": {
94
+ // response of your infra structure
95
+ }
96
+ }
97
+ ```
98
+
99
+ You can found more examples in this document.
38
100
 
39
101
  ### Monitoring http status code only
102
+
103
+ If you need only http status code, not response body, You can turn off response
104
+ body parser to set `status_only` is true. Remember that `format` is mandatory.
105
+ In this case, you must set `format` is none.
106
+
107
+
40
108
  ```
41
109
  <source>
42
- @type http_pull
110
+ @type http_pull
43
111
 
44
- tag test
45
- url http://www.google.com
46
- interval 1s
112
+ tag fluentd.status
113
+ url http://your-infrastructure/healthcheck
114
+ interval 1s
47
115
 
48
- format none
49
- status_only true
116
+ status_only true
117
+ format none
50
118
  </source>
51
119
 
52
- <match test>
53
- @type stdout
54
- </match>
120
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/healthcheck","status":200}
121
+ ```
122
+
123
+ ### Override User Agent
124
+
125
+ You can set the User Agent by specifying the `agent` option.
55
126
 
56
- # 2017-05-17 21:40:40.413219000 +0900 test: {"url":"http://www.google.com","status":200}
57
- # 2017-05-17 21:40:41.298215000 +0900 test: {"url":"http://www.google.com","status":200}
58
- # 2017-05-17 21:40:42.310993000 +0900 test: {"url":"http://www.google.com","status":200}
59
- # 2017-05-17 21:40:43.305947000 +0900 test: {"url":"http://www.google.com","status":200}
60
127
  ```
128
+ <source>
129
+ @type http_pull
130
+
131
+ tag status
132
+ url http://your-infrastructure/api/status.json
133
+ interval 1s
134
+
135
+ format json
136
+ agent infrastructure_monitor
137
+ </source>
138
+
139
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
140
+ ```
141
+
142
+ ### HTTP basic auth
143
+
144
+ If your infrastructure is protected by HTTP basic auth,
145
+ You can use `user`, `password` options to provide authentication information.
61
146
 
62
- ### Monitoring fluentd itself
63
147
  ```
64
148
  <source>
65
- @type monitor_agent
149
+ @type http_pull
150
+
151
+ tag status
152
+ url http://your-infrastructure/api/status.json
153
+ interval 1s
66
154
 
67
- port 24220
155
+ format json
156
+ user foo
157
+ passwrd bar
68
158
  </source>
69
159
 
160
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
161
+ ```
162
+
163
+ ### HTTP proxy
164
+
165
+ You can send your requests via proxy server.
166
+
167
+ ```
70
168
  <source>
71
- @type http_pull
169
+ @type http_pull
72
170
 
73
- tag fluentd.status
74
- url http://localhost:24220/api/plugins.json
75
- interval 1s
171
+ tag status
172
+ url http://your-infrastructure/api/status.json
173
+ proxy http://your-proxy-server:3128
174
+ interval 1s
76
175
 
77
- format json
176
+ format json
78
177
  </source>
79
178
 
80
- <match fluentd.status>
81
- @type stdout
82
- </match>
179
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
180
+ ```
181
+
182
+ ### Logging HTTP response header
183
+
184
+ If you wish to monitoring not only response body but also response header,
185
+ provide name of header in `response_header` sections.
83
186
 
84
- # 2017-05-17 21:41:47.872951000 +0900 fluentd.status: {"url":"http://localhost:24220/api/plugins.json","status":200,"message":{"plugins":[{"plugin_id":"object:1e7e3d...
85
- # 2017-05-17 21:41:48.955316000 +0900 fluentd.status: {"url":"http://localhost:24220/api/plugins.json","status":200,"message":{"plugins":[{"plugin_id":"object:1e7e3d...
86
- # 2017-05-17 21:41:50.033628000 +0900 fluentd.status: {"url":"http://localhost:24220/api/plugins.json","status":200,"message":{"plugins":[{"plugin_id":"object:1e7e3d...
87
- # 2017-05-17 21:41:51.107372000 +0900 fluentd.status: {"url":"http://localhost:24220/api/plugins.json","status":200,"message":{"plugins":[{"plugin_id":"object:1e7e3d...
88
187
  ```
188
+ <source>
189
+ @type http_pull
190
+
191
+ tag status
192
+ url http://your-infrastructure/api/status.json
193
+ interval 1s
194
+
195
+ format json
89
196
 
197
+ <response_header>
198
+ header Content-Type
199
+ </response_header>
200
+
201
+ <response_header>
202
+ header Content-Length
203
+ </response_header>
204
+ </source>
205
+
206
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
207
+ ```
208
+
209
+
210
+ ### Custom request header
211
+
212
+ The custom request header also supported.
213
+ This will help you when your infrastructure needs authentication headers or something like that.
90
214
 
91
- ### Monitoring elasticsearch cluster health
92
215
  ```
93
216
  <source>
94
- @type http_pull
217
+ @type http_pull
218
+
219
+ tag status
220
+ url http://your-infrastructure/api/status.json
221
+ interval 1s
95
222
 
96
- tag es.cluster.health
97
- url http://localhost:9200/_cluster/health
98
- interval 1s
223
+ format json
99
224
 
100
- format json
225
+ <request_header>
226
+ header API_ACCESS_KEY
227
+ value hatsune
228
+ </request_header>
229
+
230
+ <response_header>
231
+ header API_ACCESS_KEY_SECRET
232
+ value miku
233
+ </response_header>
101
234
  </source>
102
235
 
103
- <match es.cluster.health>
104
- @type stdout
105
- </match>
236
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
237
+ ```
238
+
239
+
240
+ ### HTTPS support
241
+
242
+ If your infrastructure has https endpoints, just use https url for monitoring them.
243
+
244
+ #### self-signed SSL
245
+
246
+
247
+ If your infrastructure has https endpoints secured by self signed certification,
248
+ you can provide custom certification file via `ca_path`, `ca_file` option.
106
249
 
107
- # 2017-05-17 12:49:09.886298008 +0000 es.cluster.health: {"url":"http://localhost:9200/_cluster/health","status":200,"message":{"cluster_name":"elasticsearch","status":"green",...
108
- # 2017-05-17 12:49:10.669431296 +0000 es.cluster.health: {"url":"http://localhost:9200/_cluster/health","status":200,"message":{"cluster_name":"elasticsearch","status":"green",...
109
- # 2017-05-17 12:49:11.668789668 +0000 es.cluster.health: {"url":"http://localhost:9200/_cluster/health","status":200,"message":{"cluster_name":"elasticsearch","status":"green",...
110
- # 2017-05-17 12:49:12.668789849 +0000 es.cluster.health: {"url":"http://localhost:9200/_cluster/health","status":200,"message":{"cluster_name":"elasticsearch","status":"green",...
250
+ ```
251
+ <source>
252
+ @type http_pull
253
+
254
+ tag status
255
+ url https://your-infrastructure/api/status.json
256
+ interval 1s
257
+ ca_path /somewhere/ca/stored
258
+ ca_file /somewhere/ca/stored/server.crt
259
+
260
+ format json
261
+ </source>
262
+
263
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
264
+ ```
265
+
266
+ And, disable SSL verification also supported. (not recommended)
267
+
268
+ ```
269
+ <source>
270
+ @type http_pull
271
+
272
+ tag status
273
+ url https://your-infrastructure/api/status.json
274
+ interval 1s
275
+ verify_ssl false
276
+
277
+ format json
278
+ </source>
279
+
280
+ # 2017-05-17 21:41:47.872951000 +0900 status: {"url":"http://yourinfrastructure/api/status.json","status":200,"message":{ ... }}
111
281
  ```
112
282
 
113
283
  ## Configuration
114
284
 
115
- ### Basic options
285
+ ### Basic Configuration
116
286
 
117
287
  #### tag (string) (required)
118
288
 
@@ -143,7 +313,7 @@ for more detail.
143
313
 
144
314
  If `status_only` is true, body is not parsed.
145
315
 
146
- ###E http_method (enum) (optional, default: :get)
316
+ #### http_method (enum) (optional, default: :get)
147
317
 
148
318
  The http request method for each requests. Avaliable options are listed below.
149
319
 
@@ -157,13 +327,13 @@ If `status_only` is true, `http_method` was override to `head`
157
327
 
158
328
  The timeout of each request.
159
329
 
160
- ### Proxy options`
330
+ ### Proxy Configuration
161
331
 
162
332
  #### proxy (string) (optional, default: nil)
163
333
 
164
334
  The HTTP proxy URL to use for each requests
165
335
 
166
- ### Basic auth options
336
+ ### Basic Auth Configuration
167
337
 
168
338
  #### user (string) (optional, default: nil)
169
339
 
@@ -173,7 +343,7 @@ The user for basic auth
173
343
 
174
344
  The password for basic auth
175
345
 
176
- ### Req/Resp header options
346
+ ### Req/Resp Header Configuration
177
347
 
178
348
  #### response_header (section) (optional, default: nil)
179
349
 
@@ -183,7 +353,7 @@ The name of response header for capture.
183
353
 
184
354
  The name, value pair of custom reuqest header.
185
355
 
186
- ### SSL options
356
+ ### SSL Configuration
187
357
 
188
358
  #### verify_ssl (bool) (optional, default: true)
189
359
 
@@ -195,33 +365,10 @@ The absolute path of directory where ca_file stored. Should be used with `ca_fil
195
365
 
196
366
  #### ca_file (string) (optional, defualt: nil)
197
367
 
198
- The Absolute path of ca_file. Should be used with `ca_path`.
199
-
200
-
201
- ## In case of remote error
202
-
203
- ### Can receive response from remote
204
-
205
- ```
206
- {
207
- "url": url of remote
208
- "status": status code of response
209
- "error": "RestClient::NotFound: 404 Not Found" or something similar
210
- }
211
- ```
212
-
213
- ### All the other case
214
-
215
- ```
216
- {
217
- "url": url of remote
218
- "status": 0
219
- "error": "Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 12345" or something similar
220
- }
221
- ```
368
+ The absolute path of ca_file. Should be used with `ca_path`.
222
369
 
223
370
  ## Copyright
224
371
 
225
372
  * Copyright(c) 2017- filepang
226
373
  * License
227
- * Apache License, Version 2.0
374
+ * Apache License, Version 2.0
@@ -3,7 +3,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
3
3
 
4
4
  Gem::Specification.new do |spec|
5
5
  spec.name = "fluent-plugin-http-pull"
6
- spec.version = "0.8.1"
6
+ spec.version = "0.8.2"
7
7
  spec.authors = ["filepang"]
8
8
  spec.email = ["filepang@gmail.com"]
9
9
 
@@ -57,7 +57,7 @@ module Fluent
57
57
  config_param :user, :string, default: nil
58
58
 
59
59
  desc 'password of basic auth'
60
- config_param :password, :string, default: nil
60
+ config_param :password, :string, default: nil, secret: true
61
61
 
62
62
  # req/res header options
63
63
  config_section :response_header, param_name: :response_headers, multi: true do
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fluent-plugin-http-pull
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.8.1
4
+ version: 0.8.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - filepang
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2018-05-27 00:00:00.000000000 Z
11
+ date: 2018-08-25 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -178,7 +178,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
178
178
  version: '0'
179
179
  requirements: []
180
180
  rubyforge_project:
181
- rubygems_version: 2.6.13
181
+ rubygems_version: 2.7.6
182
182
  signing_key:
183
183
  specification_version: 4
184
184
  summary: fluent-plugin-http-pull