logstash-output-http 5.4.0 → 5.4.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/docs/index.asciidoc +51 -11
- data/lib/logstash/outputs/http.rb +9 -0
- data/logstash-output-http.gemspec +1 -1
- data/spec/outputs/http_spec.rb +17 -0
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 6f8c174b5f5725b3dff206924edcff21ebc71899e386c1a301860dba438bbec5
|
4
|
+
data.tar.gz: dd6f158c15357a1dee6bfc3ad6e0faa4f6080c3fc2a89b6050b188bd3d016d4a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 2d668381891939636b2462361fe0011d5ced8461b5a1a5e7f2662aa765e4672b6377e2dd37a05073eaa8e0602d60632496fa48ec9c42ab5063839f02214f2e9d
|
7
|
+
data.tar.gz: b3ff1de393aee131f18a1d8d75f05389f6a74642e695626b1e9b4b03eec99b7a26017cea50b97315f35594013650785a8b3e11bfcc7acd4e5d32ed118340f208
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,7 @@
|
|
1
|
+
## 5.4.1
|
2
|
+
- Fix retry indefinitely in termination process. This feature requires Logstash 8.1 [#129](https://github.com/logstash-plugins/logstash-output-http/pull/129)
|
3
|
+
- Docs: Add retry policy description [#130](https://github.com/logstash-plugins/logstash-output-http/pull/130)
|
4
|
+
|
1
5
|
## 5.4.0
|
2
6
|
- Introduce retryable unknown exceptions for "connection reset by peer" and "timeout" [#127](https://github.com/logstash-plugins/logstash-output-http/pull/127)
|
3
7
|
|
data/docs/index.asciidoc
CHANGED
@@ -31,6 +31,40 @@ guaranteed!
|
|
31
31
|
|
32
32
|
Beware, this gem does not yet support codecs. Please use the 'format' option for now.
|
33
33
|
|
34
|
+
[id="plugins-{type}s-{plugin}-retry_policy"]
|
35
|
+
==== Retry policy
|
36
|
+
|
37
|
+
This output has two levels of retry: library and plugin.
|
38
|
+
|
39
|
+
[id="plugins-{type}s-{plugin}-library_retry"]
|
40
|
+
===== Library retry
|
41
|
+
|
42
|
+
The library retry applies to IO related failures.
|
43
|
+
Non retriable errors include SSL related problems, unresolvable hosts,
|
44
|
+
connection issues, and OS/JVM level interruptions happening during a request.
|
45
|
+
|
46
|
+
The options for library retry are:
|
47
|
+
|
48
|
+
* <<plugins-{type}s-{plugin}-automatic_retries,`automatic_retries`>>.
|
49
|
+
Controls the number of times the plugin should retry after failures at the library level.
|
50
|
+
* <<plugins-{type}s-{plugin}-retry_non_idempotent,`retry_non_idempotent`>>.
|
51
|
+
When set to `false`, GET, HEAD, PUT, DELETE, OPTIONS, and TRACE requests will be
|
52
|
+
retried.
|
53
|
+
|
54
|
+
[id="plugins-{type}s-{plugin}-plugin_retry"]
|
55
|
+
===== Plugin retry
|
56
|
+
|
57
|
+
The options for plugin level retry are:
|
58
|
+
|
59
|
+
* <<plugins-{type}s-{plugin}-retry_failed,`retry_failed`>>.
|
60
|
+
When set to `true`, the plugin retries indefinitely for HTTP error response codes defined
|
61
|
+
in the <<plugins-{type}s-{plugin}-retryable_codes,`retryable_codes`>> option
|
62
|
+
(429, 500, 502, 503, 504) and retryable exceptions (socket timeout/ error, DNS resolution failure and client protocol exception).
|
63
|
+
* <<plugins-{type}s-{plugin}-retryable_codes,`retryable_codes`>>.
|
64
|
+
Sets http response codes that trigger a retry.
|
65
|
+
|
66
|
+
NOTE: The `retry_failed` option does not control the library level retry.
|
67
|
+
|
34
68
|
[id="plugins-{type}s-{plugin}-options"]
|
35
69
|
==== Http Output Configuration Options
|
36
70
|
|
@@ -85,12 +119,10 @@ output plugins.
|
|
85
119
|
* Value type is <<number,number>>
|
86
120
|
* Default value is `1`
|
87
121
|
|
88
|
-
How many times should the client retry a failing URL. We
|
89
|
-
to zero if keepalive is enabled.
|
90
|
-
|
91
|
-
|
92
|
-
unless `retry_failed` is set.
|
93
|
-
Note: if `retry_non_idempotent` is NOT set only GET, HEAD, PUT, DELETE, OPTIONS, and TRACE requests will be retried.
|
122
|
+
How many times should the client retry a failing URL. We recommend setting this option
|
123
|
+
to a value other than zero if the <<plugins-{type}s-{plugin}-keepalive,`keepalive` option>> is enabled.
|
124
|
+
Some servers incorrectly end keepalives early, requiring a retry.
|
125
|
+
See <<plugins-{type}s-{plugin}-retry_policy,Retry Policy>> for more information.
|
94
126
|
|
95
127
|
[id="plugins-{type}s-{plugin}-cacert"]
|
96
128
|
===== `cacert`
|
@@ -311,7 +343,12 @@ Timeout (in seconds) for the entire request
|
|
311
343
|
* Value type is <<boolean,boolean>>
|
312
344
|
* Default value is `true`
|
313
345
|
|
314
|
-
|
346
|
+
Note that this option controls plugin-level retries only.
|
347
|
+
It has no affect on library-level retries.
|
348
|
+
|
349
|
+
Set this option to `false` if you want to disable infinite retries for HTTP error response codes defined in the <<plugins-{type}s-{plugin}-retryable_codes,`retryable_codes`>> or
|
350
|
+
retryable exceptions (Timeout, SocketException, ClientProtocolException, ResolutionFailure and SocketTimeout).
|
351
|
+
See <<plugins-{type}s-{plugin}-retry_policy,Retry policy>> for more information.
|
315
352
|
|
316
353
|
[id="plugins-{type}s-{plugin}-retry_non_idempotent"]
|
317
354
|
===== `retry_non_idempotent`
|
@@ -319,8 +356,10 @@ Set this to false if you don't want this output to retry failed requests
|
|
319
356
|
* Value type is <<boolean,boolean>>
|
320
357
|
* Default value is `false`
|
321
358
|
|
322
|
-
|
323
|
-
|
359
|
+
When this option is set to `false` and `automatic_retries` is enabled, GET, HEAD, PUT, DELETE, OPTIONS, and TRACE requests will be retried.
|
360
|
+
|
361
|
+
When set to `true` and `automatic_retries` is enabled, this will cause non-idempotent HTTP verbs (such as POST) to be retried.
|
362
|
+
See <<plugins-{type}s-{plugin}-retry_policy,Retry Policy>> for more information.
|
324
363
|
|
325
364
|
[id="plugins-{type}s-{plugin}-retryable_codes"]
|
326
365
|
===== `retryable_codes`
|
@@ -328,7 +367,8 @@ This only affects connectivity related errors (see related `automatic_retries` s
|
|
328
367
|
* Value type is <<number,number>>
|
329
368
|
* Default value is `[429, 500, 502, 503, 504]`
|
330
369
|
|
331
|
-
If
|
370
|
+
If the plugin encounters these response codes, the plugin will retry indefinitely.
|
371
|
+
See <<plugins-{type}s-{plugin}-retry_policy,Retry Policy>> for more information.
|
332
372
|
|
333
373
|
[id="plugins-{type}s-{plugin}-socket_timeout"]
|
334
374
|
===== `socket_timeout`
|
@@ -405,4 +445,4 @@ See https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache
|
|
405
445
|
[id="plugins-{type}s-{plugin}-common-options"]
|
406
446
|
include::{include_path}/{type}.asciidoc[]
|
407
447
|
|
408
|
-
:default_codec!:
|
448
|
+
:default_codec!:
|
@@ -28,6 +28,7 @@ class LogStash::Outputs::Http < LogStash::Outputs::Base
|
|
28
28
|
/Read Timed out/i
|
29
29
|
]
|
30
30
|
|
31
|
+
class PluginInternalQueueLeftoverError < StandardError; end
|
31
32
|
|
32
33
|
# This output lets you send events to a
|
33
34
|
# generic HTTP(S) endpoint
|
@@ -179,6 +180,9 @@ class LogStash::Outputs::Http < LogStash::Outputs::Base
|
|
179
180
|
|
180
181
|
event, attempt = popped
|
181
182
|
|
183
|
+
raise PluginInternalQueueLeftoverError.new("Received pipeline shutdown request but http output has unfinished events. " \
|
184
|
+
"If persistent queue is enabled, events will be retried.") if attempt > 2 && pipeline_shutdown_requested?
|
185
|
+
|
182
186
|
action, event, attempt = send_event(event, attempt)
|
183
187
|
begin
|
184
188
|
action = :failure if action == :retry && !@retry_failed
|
@@ -223,6 +227,11 @@ class LogStash::Outputs::Http < LogStash::Outputs::Base
|
|
223
227
|
raise e
|
224
228
|
end
|
225
229
|
|
230
|
+
def pipeline_shutdown_requested?
|
231
|
+
return super if defined?(super) # since LS 8.1.0
|
232
|
+
nil
|
233
|
+
end
|
234
|
+
|
226
235
|
def sleep_for_attempt(attempt)
|
227
236
|
sleep_for = attempt**2
|
228
237
|
sleep_for = sleep_for <= 60 ? sleep_for : 60
|
@@ -1,6 +1,6 @@
|
|
1
1
|
Gem::Specification.new do |s|
|
2
2
|
s.name = 'logstash-output-http'
|
3
|
-
s.version = '5.4.
|
3
|
+
s.version = '5.4.1'
|
4
4
|
s.licenses = ['Apache License (2.0)']
|
5
5
|
s.summary = "Sends events to a generic HTTP or HTTPS endpoint"
|
6
6
|
s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"
|
data/spec/outputs/http_spec.rb
CHANGED
@@ -501,6 +501,23 @@ describe LogStash::Outputs::Http do
|
|
501
501
|
let(:base_config) { { "http_compression" => true } }
|
502
502
|
end
|
503
503
|
end
|
504
|
+
|
505
|
+
describe "retryable error in termination" do
|
506
|
+
let(:url) { "http://localhost:#{port-1}/invalid" }
|
507
|
+
let(:events) { [event] }
|
508
|
+
let(:config) { {"url" => url, "http_method" => "get", "pool_max" => 1} }
|
509
|
+
|
510
|
+
subject { LogStash::Outputs::Http.new(config) }
|
511
|
+
|
512
|
+
before do
|
513
|
+
subject.register
|
514
|
+
allow(subject).to receive(:pipeline_shutdown_requested?).and_return(true)
|
515
|
+
end
|
516
|
+
|
517
|
+
it "raise exception to exit indefinitely retry" do
|
518
|
+
expect { subject.multi_receive(events) }.to raise_error(LogStash::Outputs::Http::PluginInternalQueueLeftoverError)
|
519
|
+
end
|
520
|
+
end
|
504
521
|
end
|
505
522
|
|
506
523
|
describe LogStash::Outputs::Http do # different block as we're starting web server with TLS
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: logstash-output-http
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 5.4.
|
4
|
+
version: 5.4.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Elastic
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-
|
11
|
+
date: 2022-03-04 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
requirement: !ruby/object:Gem::Requirement
|