logstash-output-elasticsearch 10.6.0-java → 10.8.0-java
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +24 -1
- data/CONTRIBUTORS +1 -0
- data/docs/index.asciidoc +97 -78
- data/lib/logstash/outputs/elasticsearch.rb +300 -165
- data/lib/logstash/outputs/elasticsearch/http_client.rb +7 -2
- data/lib/logstash/outputs/elasticsearch/http_client/pool.rb +13 -28
- data/lib/logstash/outputs/elasticsearch/http_client_builder.rb +1 -0
- data/lib/logstash/outputs/elasticsearch/ilm.rb +9 -5
- data/lib/logstash/outputs/elasticsearch/license_checker.rb +47 -0
- data/lib/logstash/outputs/elasticsearch/template_manager.rb +8 -3
- data/lib/logstash/outputs/elasticsearch/templates/ecs-disabled/elasticsearch-8x.json +39 -33
- data/lib/logstash/plugin_mixins/elasticsearch/api_configs.rb +163 -0
- data/lib/logstash/{outputs → plugin_mixins}/elasticsearch/common.rb +40 -154
- data/lib/logstash/plugin_mixins/elasticsearch/noop_license_checker.rb +9 -0
- data/logstash-output-elasticsearch.gemspec +1 -1
- data/spec/es_spec_helper.rb +32 -12
- data/spec/fixtures/template-with-policy-es8x.json +50 -0
- data/spec/integration/outputs/ilm_spec.rb +34 -20
- data/spec/integration/outputs/metrics_spec.rb +1 -5
- data/spec/unit/outputs/elasticsearch/http_client/pool_spec.rb +45 -5
- data/spec/unit/outputs/elasticsearch/http_client_spec.rb +22 -0
- data/spec/unit/outputs/elasticsearch/template_manager_spec.rb +31 -0
- data/spec/unit/outputs/elasticsearch_spec.rb +22 -0
- data/spec/unit/outputs/license_check_spec.rb +41 -0
- metadata +10 -4
- data/lib/logstash/outputs/elasticsearch/common_configs.rb +0 -167
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 29d2b5e316c2ba3dd1d67af673733bd90dd0b5b82c436b1f107cc11d681bd8b5
|
4
|
+
data.tar.gz: 102c70fde50016c59245287f09eeed0d51d4fb411a617b7d397ab306688f022d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6bd3b56c08debf7408ac1cd31980235d3224b5705a42ecc78d28e8da94825cd310ee1b8d0f47b16400327bcc6b0c1e777e2a937a09b1a5a483584d6ca523317d
|
7
|
+
data.tar.gz: c9ba096fb1a39064616ef10be681d1d3b64ab344a7b3b51560b68391c525cf61c6c98a289e068958e43806e7042d9a7ced140ec29407f80680e0323ba44ce165
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,28 @@
|
|
1
|
+
## 10.8.0
|
2
|
+
- Refactored configuration options into specific and shared in PluginMixins namespace [#973](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/973)
|
3
|
+
- Refactored common methods into specific and shared in PluginMixins namespace [#976](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/976)
|
4
|
+
|
5
|
+
## 10.7.3
|
6
|
+
- Added composable index template support for elasticsearch version 8 [#980](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/980)
|
7
|
+
|
8
|
+
## 10.7.2
|
9
|
+
- [DOC] Fixed links to restructured Logstash-to-cloud docs [#975](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/975)
|
10
|
+
|
11
|
+
## 10.7.1
|
12
|
+
- [DOC] Document the permissions required in secured clusters [#969](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/969)
|
13
|
+
|
14
|
+
## 10.7.0
|
15
|
+
- Changed: don't set the pipeline parameter if the value resolves to an empty string [#962](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/962)
|
16
|
+
|
17
|
+
## 10.6.2
|
18
|
+
- [DOC] Added clarifying info on http compression settings and behaviors [#943](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/943)
|
19
|
+
- [DOC] Fixed entry for ilm_policy default value[#956](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/956)
|
20
|
+
|
21
|
+
## 10.6.1
|
22
|
+
- Fixed an issue introduced in 10.6.0 that broke Logstash Core's monitoring feature when this plugin is run in Logstash 7.7-7.8. [#953](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/953)
|
23
|
+
|
1
24
|
## 10.6.0
|
2
|
-
- Added `ecs_compatiblity` mode, for managing ECS-compatable templates [#952](https://github.com/logstash-plugins/logstash-output-elasticsearch/
|
25
|
+
- Added `ecs_compatiblity` mode, for managing ECS-compatable templates [#952](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/952)
|
3
26
|
|
4
27
|
## 10.5.1
|
5
28
|
- [DOC] Removed outdated compatibility notices, reworked cloud notice, and fixed formatting for `hosts` examples [#938](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/938)
|
data/CONTRIBUTORS
CHANGED
@@ -26,6 +26,7 @@ Contributors:
|
|
26
26
|
* Tom Hodder (tolland)
|
27
27
|
* jimmyjones2
|
28
28
|
* Gabriel Moskovicz (gmoskovicz)
|
29
|
+
* Luca Belluccini (lucabelluccini)
|
29
30
|
|
30
31
|
Note: If you've sent us patches, bug reports, or otherwise contributed to
|
31
32
|
Logstash, and you aren't on the list above and want to be, please let us know
|
data/docs/index.asciidoc
CHANGED
@@ -23,14 +23,14 @@ include::{include_path}/plugin_header.asciidoc[]
|
|
23
23
|
|
24
24
|
If you plan to use the Kibana web interface to analyze data transformed by
|
25
25
|
Logstash, use the Elasticsearch output plugin to get your data into
|
26
|
-
Elasticsearch.
|
26
|
+
Elasticsearch.
|
27
27
|
|
28
28
|
This output only speaks the HTTP protocol as it is the preferred protocol for
|
29
29
|
interacting with Elasticsearch. In previous versions it was possible to
|
30
30
|
communicate with Elasticsearch through the transport protocol, which is now
|
31
31
|
reserved for internal cluster communication between nodes
|
32
32
|
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-transport.html[communication between nodes].
|
33
|
-
Using the https://www.elastic.co/guide/en/elasticsearch/reference/current/java-clients.html[transport protocol]
|
33
|
+
Using the https://www.elastic.co/guide/en/elasticsearch/reference/current/java-clients.html[transport protocol]
|
34
34
|
to communicate with the cluster has been deprecated in Elasticsearch 7.0.0 and
|
35
35
|
will be removed in 8.0.0
|
36
36
|
|
@@ -53,10 +53,7 @@ connecting to Elasticsearch 7.x.
|
|
53
53
|
|
54
54
|
===== Hosted {es} Service on Elastic Cloud
|
55
55
|
|
56
|
-
|
57
|
-
https://www.elastic.co/cloud/elasticsearch-service[hosted {es} Service] on
|
58
|
-
Elastic Cloud. The Elasticsearch Service is available on AWS, Google Cloud
|
59
|
-
Platform, and Microsoft Azure. {ess-trial}[Try the {es} Service for free].
|
56
|
+
{ess-leadin}
|
60
57
|
|
61
58
|
==== Compatibility with the Elastic Common Schema (ECS)
|
62
59
|
|
@@ -76,7 +73,7 @@ and will correctly reject events with fields that conflict and cannot be coerced
|
|
76
73
|
|
77
74
|
[NOTE]
|
78
75
|
================================================================================
|
79
|
-
You cannot use dynamic variable substitution when `ilm_enabled` is `true` and
|
76
|
+
You cannot use dynamic variable substitution when `ilm_enabled` is `true` and
|
80
77
|
when using `ilm_rollover_alias`.
|
81
78
|
|
82
79
|
================================================================================
|
@@ -91,7 +88,10 @@ Each Elasticsearch output is a new client connected to the cluster:
|
|
91
88
|
* it has to initialize the client and connect to Elasticsearch (restart time is longer if you have more clients)
|
92
89
|
* it has an associated connection pool
|
93
90
|
|
94
|
-
In order to minimize the number of open connections to Elasticsearch, maximize
|
91
|
+
In order to minimize the number of open connections to Elasticsearch, maximize
|
92
|
+
the bulk size and reduce the number of "small" bulk requests (which could easily
|
93
|
+
fill up the queue), it is usually more efficient to have a single Elasticsearch
|
94
|
+
output.
|
95
95
|
|
96
96
|
Example:
|
97
97
|
[source,ruby]
|
@@ -100,11 +100,14 @@ Example:
|
|
100
100
|
index => "%{[some_field][sub_field]}-%{+YYYY.MM.dd}"
|
101
101
|
}
|
102
102
|
}
|
103
|
-
|
103
|
+
|
104
104
|
**What to do in case there is no field in the event containing the destination index prefix?**
|
105
105
|
|
106
|
-
You can use the `mutate` filter and conditionals to add a `[@metadata]` field
|
107
|
-
|
106
|
+
You can use the `mutate` filter and conditionals to add a `[@metadata]` field
|
107
|
+
(see
|
108
|
+
https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#metadata)
|
109
|
+
to set the destination index for each event. The `[@metadata]` fields will not
|
110
|
+
be sent to Elasticsearch.
|
108
111
|
|
109
112
|
Example:
|
110
113
|
[source,ruby]
|
@@ -136,7 +139,7 @@ HTTP requests to the bulk API are expected to return a 200 response code. All ot
|
|
136
139
|
The following document errors are handled as follows:
|
137
140
|
|
138
141
|
* 400 and 404 errors are sent to the dead letter queue (DLQ), if enabled. If a DLQ is not enabled, a log message will be emitted, and the event will be dropped. See <<plugins-{type}s-{plugin}-dlq-policy>> for more info.
|
139
|
-
* 409 errors (conflict) are logged as a warning and dropped.
|
142
|
+
* 409 errors (conflict) are logged as a warning and dropped.
|
140
143
|
|
141
144
|
Note that 409 exceptions are no longer retried. Please set a higher `retry_on_conflict` value if you experience 409 exceptions.
|
142
145
|
It is more performant for Elasticsearch to retry these exceptions than this plugin.
|
@@ -218,18 +221,29 @@ not reevaluate its DNS value while the keepalive is in effect.
|
|
218
221
|
|
219
222
|
==== HTTP Compression
|
220
223
|
|
221
|
-
This plugin supports request and response compression. Response compression is
|
222
|
-
for Elasticsearch versions 5.0 and later
|
223
|
-
|
224
|
-
|
224
|
+
This plugin supports request and response compression. Response compression is
|
225
|
+
enabled by default for HTTP and for Elasticsearch versions 5.0 and later.
|
226
|
+
|
227
|
+
You don't have to set any configs in Elasticsearch for it to send back a
|
228
|
+
compressed response. For versions before 5.0, or if HTTPS is enabled,
|
229
|
+
`http.compression` must be set to `true`
|
230
|
+
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html#modules-http[in
|
231
|
+
Elasticsearch] to take advantage of response compression when using this plugin.
|
225
232
|
|
226
|
-
For requests compression, regardless of the Elasticsearch version,
|
227
|
-
setting in
|
233
|
+
For requests compression, regardless of the Elasticsearch version, enable the
|
234
|
+
`http_compression` setting in the Logstash config file.
|
228
235
|
|
229
236
|
==== Authentication
|
230
237
|
|
231
238
|
Authentication to a secure Elasticsearch cluster is possible using one of the `user`/`password`, `cloud_auth` or `api_key` options.
|
232
239
|
|
240
|
+
[id="plugins-{type}s-{plugin}-autz"]
|
241
|
+
==== Authorization
|
242
|
+
|
243
|
+
Authorization to a secure Elasticsearch cluster requires `read` permission at index level and `monitoring` permissions at cluster level.
|
244
|
+
The `monitoring` permission at cluster level is necessary to perform periodic connectivity checks.
|
245
|
+
|
246
|
+
|
233
247
|
[id="plugins-{type}s-{plugin}-options"]
|
234
248
|
==== Elasticsearch Output Configuration Options
|
235
249
|
|
@@ -303,7 +317,7 @@ output plugins.
|
|
303
317
|
|
304
318
|
|
305
319
|
[id="plugins-{type}s-{plugin}-action"]
|
306
|
-
===== `action`
|
320
|
+
===== `action`
|
307
321
|
|
308
322
|
* Value type is <<string,string>>
|
309
323
|
* Default value is `"index"`
|
@@ -334,7 +348,7 @@ Authenticate using Elasticsearch API key. Note that this option also requires en
|
|
334
348
|
Format is `id:api_key` where `id` and `api_key` are as returned by the Elasticsearch https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html[Create API key API].
|
335
349
|
|
336
350
|
[id="plugins-{type}s-{plugin}-bulk_path"]
|
337
|
-
===== `bulk_path`
|
351
|
+
===== `bulk_path`
|
338
352
|
|
339
353
|
* Value type is <<string,string>>
|
340
354
|
* There is no default value for this setting.
|
@@ -343,7 +357,7 @@ HTTP Path to perform the _bulk requests to
|
|
343
357
|
this defaults to a concatenation of the path parameter and "_bulk"
|
344
358
|
|
345
359
|
[id="plugins-{type}s-{plugin}-cacert"]
|
346
|
-
===== `cacert`
|
360
|
+
===== `cacert`
|
347
361
|
|
348
362
|
* Value type is <<path,path>>
|
349
363
|
* There is no default value for this setting.
|
@@ -358,7 +372,7 @@ The .cer or .pem file to validate the server's certificate
|
|
358
372
|
|
359
373
|
Cloud authentication string ("<username>:<password>" format) is an alternative for the `user`/`password` pair.
|
360
374
|
|
361
|
-
For more details, check out the https://www.elastic.co/guide/en/logstash/current/connecting-to-cloud.html
|
375
|
+
For more details, check out the https://www.elastic.co/guide/en/logstash/current/connecting-to-cloud.html[Logstash-to-Cloud documentation]
|
362
376
|
|
363
377
|
[id="plugins-{type}s-{plugin}-cloud_id"]
|
364
378
|
===== `cloud_id`
|
@@ -368,10 +382,10 @@ For more details, check out the https://www.elastic.co/guide/en/logstash/current
|
|
368
382
|
|
369
383
|
Cloud ID, from the Elastic Cloud web console. If set `hosts` should not be used.
|
370
384
|
|
371
|
-
For more details, check out the https://www.elastic.co/guide/en/logstash/current/connecting-to-cloud.html
|
385
|
+
For more details, check out the https://www.elastic.co/guide/en/logstash/current/connecting-to-cloud.html[Logstash-to-Cloud documentation]
|
372
386
|
|
373
387
|
[id="plugins-{type}s-{plugin}-doc_as_upsert"]
|
374
|
-
===== `doc_as_upsert`
|
388
|
+
===== `doc_as_upsert`
|
375
389
|
|
376
390
|
* Value type is <<boolean,boolean>>
|
377
391
|
* Default value is `false`
|
@@ -380,7 +394,7 @@ Enable `doc_as_upsert` for update mode.
|
|
380
394
|
Create a new document with source if `document_id` doesn't exist in Elasticsearch
|
381
395
|
|
382
396
|
[id="plugins-{type}s-{plugin}-document_id"]
|
383
|
-
===== `document_id`
|
397
|
+
===== `document_id`
|
384
398
|
|
385
399
|
* Value type is <<string,string>>
|
386
400
|
* There is no default value for this setting.
|
@@ -388,7 +402,7 @@ Create a new document with source if `document_id` doesn't exist in Elasticsearc
|
|
388
402
|
The document ID for the index. Useful for overwriting existing entries in Elasticsearch with the same ID.
|
389
403
|
|
390
404
|
[id="plugins-{type}s-{plugin}-document_type"]
|
391
|
-
===== `document_type`
|
405
|
+
===== `document_type`
|
392
406
|
|
393
407
|
* Value type is <<string,string>>
|
394
408
|
* There is no default value for this setting.
|
@@ -419,7 +433,7 @@ If you don't set a value for this option:
|
|
419
433
|
** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
|
420
434
|
** Otherwise, the default value is `disabled`.
|
421
435
|
|
422
|
-
Controls this plugin's compatibility with the
|
436
|
+
Controls this plugin's compatibility with the https://www.elastic.co/guide/en/ecs/current/index.html[Elastic Common Schema (ECS)],
|
423
437
|
including the installation of ECS-compatible index templates.
|
424
438
|
The value of this setting affects the _default_ values of:
|
425
439
|
|
@@ -428,7 +442,7 @@ The value of this setting affects the _default_ values of:
|
|
428
442
|
* <<plugins-{type}s-{plugin}-ilm_rollover_alias>>
|
429
443
|
|
430
444
|
[id="plugins-{type}s-{plugin}-failure_type_logging_whitelist"]
|
431
|
-
===== `failure_type_logging_whitelist`
|
445
|
+
===== `failure_type_logging_whitelist`
|
432
446
|
|
433
447
|
* Value type is <<array,array>>
|
434
448
|
* Default value is `[]`
|
@@ -449,7 +463,7 @@ an elasticsearch node. The headers will be used for any kind of request
|
|
449
463
|
These custom headers will be overidden by settings like `http_compression`.
|
450
464
|
|
451
465
|
[id="plugins-{type}s-{plugin}-healthcheck_path"]
|
452
|
-
===== `healthcheck_path`
|
466
|
+
===== `healthcheck_path`
|
453
467
|
|
454
468
|
* Value type is <<string,string>>
|
455
469
|
* There is no default value for this setting.
|
@@ -460,7 +474,7 @@ before it is once again eligible to service requests.
|
|
460
474
|
If you have custom firewall rules you may need to change this
|
461
475
|
|
462
476
|
[id="plugins-{type}s-{plugin}-hosts"]
|
463
|
-
===== `hosts`
|
477
|
+
===== `hosts`
|
464
478
|
|
465
479
|
* Value type is <<uri,uri>>
|
466
480
|
* Default value is `[//127.0.0.1]`
|
@@ -476,13 +490,17 @@ Examples:
|
|
476
490
|
`["https://127.0.0.1:9200"]`
|
477
491
|
`["https://127.0.0.1:9200/mypath"]` (If using a proxy on a subpath)
|
478
492
|
|
479
|
-
|
480
|
-
|
493
|
+
Exclude
|
494
|
+
http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html[dedicated
|
495
|
+
master nodes] from the `hosts` list to prevent Logstash from sending bulk
|
496
|
+
requests to the master nodes. This parameter should reference only data or
|
497
|
+
client nodes in Elasticsearch.
|
481
498
|
|
482
|
-
Any special characters present in the URLs here MUST be URL escaped! This means
|
499
|
+
Any special characters present in the URLs here MUST be URL escaped! This means
|
500
|
+
`#` should be put in as `%23` for instance.
|
483
501
|
|
484
502
|
[id="plugins-{type}s-{plugin}-http_compression"]
|
485
|
-
===== `http_compression`
|
503
|
+
===== `http_compression`
|
486
504
|
|
487
505
|
* Value type is <<boolean,boolean>>
|
488
506
|
* Default value is `false`
|
@@ -519,11 +537,14 @@ NOTE: Updating the pattern will require the index template to be rewritten
|
|
519
537
|
|
520
538
|
NOTE: The pattern must finish with a dash and a number that will be automatically incremented when indices rollover.
|
521
539
|
|
540
|
+
NOTE: The pattern is a 6-digit string padded by zeros, regardless of prior index name. Example: 000001.
|
541
|
+
See {ref}/indices-rollover-index.html#rollover-index-api-path-params[Rollover path parameters API docs] for details.
|
542
|
+
|
522
543
|
[id="plugins-{type}s-{plugin}-ilm_policy"]
|
523
544
|
===== `ilm_policy`
|
524
545
|
|
525
546
|
* Value type is <<string,string>>
|
526
|
-
* Default value is `logstash`
|
547
|
+
* Default value is `logstash-policy`
|
527
548
|
|
528
549
|
Modify this setting to use a custom Index Lifecycle Management policy, rather than the default. If this value is not set, the default policy will
|
529
550
|
be automatically installed into Elasticsearch
|
@@ -547,7 +568,7 @@ NOTE: Updating the rollover alias will require the index template to be rewritte
|
|
547
568
|
NOTE: `ilm_rollover_alias` does NOT support dynamic variable substitution as `index` does.
|
548
569
|
|
549
570
|
[id="plugins-{type}s-{plugin}-index"]
|
550
|
-
===== `index`
|
571
|
+
===== `index`
|
551
572
|
|
552
573
|
* Value type is <<string,string>>
|
553
574
|
* Default value depends on whether <<plugins-{type}s-{plugin}-ecs_compatibility>> is enabled:
|
@@ -563,7 +584,7 @@ LS uses Joda to format the index pattern from event timestamp.
|
|
563
584
|
Joda formats are defined http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html[here].
|
564
585
|
|
565
586
|
[id="plugins-{type}s-{plugin}-keystore"]
|
566
|
-
===== `keystore`
|
587
|
+
===== `keystore`
|
567
588
|
|
568
589
|
* Value type is <<path,path>>
|
569
590
|
* There is no default value for this setting.
|
@@ -572,7 +593,7 @@ The keystore used to present a certificate to the server.
|
|
572
593
|
It can be either .jks or .p12
|
573
594
|
|
574
595
|
[id="plugins-{type}s-{plugin}-keystore_password"]
|
575
|
-
===== `keystore_password`
|
596
|
+
===== `keystore_password`
|
576
597
|
|
577
598
|
* Value type is <<password,password>>
|
578
599
|
* There is no default value for this setting.
|
@@ -580,7 +601,7 @@ It can be either .jks or .p12
|
|
580
601
|
Set the keystore password
|
581
602
|
|
582
603
|
[id="plugins-{type}s-{plugin}-manage_template"]
|
583
|
-
===== `manage_template`
|
604
|
+
===== `manage_template`
|
584
605
|
|
585
606
|
* Value type is <<boolean,boolean>>
|
586
607
|
* Default value is `true`
|
@@ -600,7 +621,7 @@ field names) you should set `manage_template` to false and use the REST
|
|
600
621
|
API to apply your templates manually.
|
601
622
|
|
602
623
|
[id="plugins-{type}s-{plugin}-parameters"]
|
603
|
-
===== `parameters`
|
624
|
+
===== `parameters`
|
604
625
|
|
605
626
|
* Value type is <<hash,hash>>
|
606
627
|
* There is no default value for this setting.
|
@@ -610,7 +631,7 @@ to every host listed in the 'hosts' configuration. If the 'hosts' list contains
|
|
610
631
|
urls that already have query strings, the one specified here will be appended.
|
611
632
|
|
612
633
|
[id="plugins-{type}s-{plugin}-parent"]
|
613
|
-
===== `parent`
|
634
|
+
===== `parent`
|
614
635
|
|
615
636
|
* Value type is <<string,string>>
|
616
637
|
* Default value is `nil`
|
@@ -619,7 +640,7 @@ For child documents, ID of the associated parent.
|
|
619
640
|
This can be dynamic using the `%{foo}` syntax.
|
620
641
|
|
621
642
|
[id="plugins-{type}s-{plugin}-password"]
|
622
|
-
===== `password`
|
643
|
+
===== `password`
|
623
644
|
|
624
645
|
* Value type is <<password,password>>
|
625
646
|
* There is no default value for this setting.
|
@@ -627,7 +648,7 @@ This can be dynamic using the `%{foo}` syntax.
|
|
627
648
|
Password to authenticate to a secure Elasticsearch cluster
|
628
649
|
|
629
650
|
[id="plugins-{type}s-{plugin}-path"]
|
630
|
-
===== `path`
|
651
|
+
===== `path`
|
631
652
|
|
632
653
|
* Value type is <<string,string>>
|
633
654
|
* There is no default value for this setting.
|
@@ -638,16 +659,16 @@ Note that if you use paths as components of URLs in the 'hosts' field you may
|
|
638
659
|
not also set this field. That will raise an error at startup
|
639
660
|
|
640
661
|
[id="plugins-{type}s-{plugin}-pipeline"]
|
641
|
-
===== `pipeline`
|
662
|
+
===== `pipeline`
|
642
663
|
|
643
664
|
* Value type is <<string,string>>
|
644
665
|
* Default value is `nil`
|
645
666
|
|
646
|
-
Set which ingest pipeline you wish to execute for an event. You can also use event dependent configuration
|
647
|
-
|
667
|
+
Set which ingest pipeline you wish to execute for an event. You can also use event dependent configuration here
|
668
|
+
like `pipeline => "%{[@metadata][pipeline]}"`. The pipeline parameter won't be set if the value resolves to empty string ("").
|
648
669
|
|
649
670
|
[id="plugins-{type}s-{plugin}-pool_max"]
|
650
|
-
===== `pool_max`
|
671
|
+
===== `pool_max`
|
651
672
|
|
652
673
|
* Value type is <<number,number>>
|
653
674
|
* Default value is `1000`
|
@@ -658,7 +679,7 @@ Setting this too low may mean frequently closing / opening connections
|
|
658
679
|
which is bad.
|
659
680
|
|
660
681
|
[id="plugins-{type}s-{plugin}-pool_max_per_route"]
|
661
|
-
===== `pool_max_per_route`
|
682
|
+
===== `pool_max_per_route`
|
662
683
|
|
663
684
|
* Value type is <<number,number>>
|
664
685
|
* Default value is `100`
|
@@ -669,7 +690,7 @@ Setting this too low may mean frequently closing / opening connections
|
|
669
690
|
which is bad.
|
670
691
|
|
671
692
|
[id="plugins-{type}s-{plugin}-proxy"]
|
672
|
-
===== `proxy`
|
693
|
+
===== `proxy`
|
673
694
|
|
674
695
|
* Value type is <<uri,uri>>
|
675
696
|
* There is no default value for this setting.
|
@@ -680,7 +701,7 @@ An empty string is treated as if proxy was not set. This is useful when using
|
|
680
701
|
environment variables e.g. `proxy => '${LS_PROXY:}'`.
|
681
702
|
|
682
703
|
[id="plugins-{type}s-{plugin}-resurrect_delay"]
|
683
|
-
===== `resurrect_delay`
|
704
|
+
===== `resurrect_delay`
|
684
705
|
|
685
706
|
* Value type is <<number,number>>
|
686
707
|
* Default value is `5`
|
@@ -690,7 +711,7 @@ Resurrection is the process by which backend endpoints marked 'down' are checked
|
|
690
711
|
to see if they have come back to life
|
691
712
|
|
692
713
|
[id="plugins-{type}s-{plugin}-retry_initial_interval"]
|
693
|
-
===== `retry_initial_interval`
|
714
|
+
===== `retry_initial_interval`
|
694
715
|
|
695
716
|
* Value type is <<number,number>>
|
696
717
|
* Default value is `2`
|
@@ -698,7 +719,7 @@ to see if they have come back to life
|
|
698
719
|
Set initial interval in seconds between bulk retries. Doubled on each retry up to `retry_max_interval`
|
699
720
|
|
700
721
|
[id="plugins-{type}s-{plugin}-retry_max_interval"]
|
701
|
-
===== `retry_max_interval`
|
722
|
+
===== `retry_max_interval`
|
702
723
|
|
703
724
|
* Value type is <<number,number>>
|
704
725
|
* Default value is `64`
|
@@ -706,17 +727,15 @@ Set initial interval in seconds between bulk retries. Doubled on each retry up t
|
|
706
727
|
Set max interval in seconds between bulk retries.
|
707
728
|
|
708
729
|
[id="plugins-{type}s-{plugin}-retry_on_conflict"]
|
709
|
-
===== `retry_on_conflict`
|
730
|
+
===== `retry_on_conflict`
|
710
731
|
|
711
732
|
* Value type is <<number,number>>
|
712
733
|
* Default value is `1`
|
713
734
|
|
714
|
-
The number of times Elasticsearch should internally retry an update/upserted document
|
715
|
-
See the https://www.elastic.co/guide/en/elasticsearch/guide/current/partial-updates.html[partial updates]
|
716
|
-
for more info
|
735
|
+
The number of times Elasticsearch should internally retry an update/upserted document.
|
717
736
|
|
718
737
|
[id="plugins-{type}s-{plugin}-routing"]
|
719
|
-
===== `routing`
|
738
|
+
===== `routing`
|
720
739
|
|
721
740
|
* Value type is <<string,string>>
|
722
741
|
* There is no default value for this setting.
|
@@ -725,7 +744,7 @@ A routing override to be applied to all processed events.
|
|
725
744
|
This can be dynamic using the `%{foo}` syntax.
|
726
745
|
|
727
746
|
[id="plugins-{type}s-{plugin}-script"]
|
728
|
-
===== `script`
|
747
|
+
===== `script`
|
729
748
|
|
730
749
|
* Value type is <<string,string>>
|
731
750
|
* Default value is `""`
|
@@ -741,7 +760,7 @@ Example:
|
|
741
760
|
}
|
742
761
|
|
743
762
|
[id="plugins-{type}s-{plugin}-script_lang"]
|
744
|
-
===== `script_lang`
|
763
|
+
===== `script_lang`
|
745
764
|
|
746
765
|
* Value type is <<string,string>>
|
747
766
|
* Default value is `"painless"`
|
@@ -750,7 +769,7 @@ Set the language of the used script. If not set, this defaults to painless in ES
|
|
750
769
|
When using indexed (stored) scripts on Elasticsearch 6 and higher, you must set this parameter to `""` (empty string).
|
751
770
|
|
752
771
|
[id="plugins-{type}s-{plugin}-script_type"]
|
753
|
-
===== `script_type`
|
772
|
+
===== `script_type`
|
754
773
|
|
755
774
|
* Value can be any of: `inline`, `indexed`, `file`
|
756
775
|
* Default value is `["inline"]`
|
@@ -761,7 +780,7 @@ Define the type of script referenced by "script" variable
|
|
761
780
|
file : "script" contains the name of script stored in elasticsearch's config directory
|
762
781
|
|
763
782
|
[id="plugins-{type}s-{plugin}-script_var_name"]
|
764
|
-
===== `script_var_name`
|
783
|
+
===== `script_var_name`
|
765
784
|
|
766
785
|
* Value type is <<string,string>>
|
767
786
|
* Default value is `"event"`
|
@@ -769,7 +788,7 @@ Define the type of script referenced by "script" variable
|
|
769
788
|
Set variable name passed to script (scripted update)
|
770
789
|
|
771
790
|
[id="plugins-{type}s-{plugin}-scripted_upsert"]
|
772
|
-
===== `scripted_upsert`
|
791
|
+
===== `scripted_upsert`
|
773
792
|
|
774
793
|
* Value type is <<boolean,boolean>>
|
775
794
|
* Default value is `false`
|
@@ -777,7 +796,7 @@ Set variable name passed to script (scripted update)
|
|
777
796
|
if enabled, script is in charge of creating non-existent document (scripted update)
|
778
797
|
|
779
798
|
[id="plugins-{type}s-{plugin}-sniffing"]
|
780
|
-
===== `sniffing`
|
799
|
+
===== `sniffing`
|
781
800
|
|
782
801
|
* Value type is <<boolean,boolean>>
|
783
802
|
* Default value is `false`
|
@@ -787,7 +806,7 @@ For Elasticsearch 1.x and 2.x any nodes with `http.enabled` (on by default) will
|
|
787
806
|
For Elasticsearch 5.x and 6.x any nodes with `http.enabled` (on by default) will be added to the hosts list, excluding master-only nodes.
|
788
807
|
|
789
808
|
[id="plugins-{type}s-{plugin}-sniffing_delay"]
|
790
|
-
===== `sniffing_delay`
|
809
|
+
===== `sniffing_delay`
|
791
810
|
|
792
811
|
* Value type is <<number,number>>
|
793
812
|
* Default value is `5`
|
@@ -795,7 +814,7 @@ For Elasticsearch 5.x and 6.x any nodes with `http.enabled` (on by default) will
|
|
795
814
|
How long to wait, in seconds, between sniffing attempts
|
796
815
|
|
797
816
|
[id="plugins-{type}s-{plugin}-sniffing_path"]
|
798
|
-
===== `sniffing_path`
|
817
|
+
===== `sniffing_path`
|
799
818
|
|
800
819
|
* Value type is <<string,string>>
|
801
820
|
* There is no default value for this setting.
|
@@ -806,7 +825,7 @@ if sniffing_path is set it will be used as an absolute path
|
|
806
825
|
do not use full URL here, only paths, e.g. "/sniff/_nodes/http"
|
807
826
|
|
808
827
|
[id="plugins-{type}s-{plugin}-ssl"]
|
809
|
-
===== `ssl`
|
828
|
+
===== `ssl`
|
810
829
|
|
811
830
|
* Value type is <<boolean,boolean>>
|
812
831
|
* There is no default value for this setting.
|
@@ -816,7 +835,7 @@ is specified in the URLs listed in 'hosts'. If no explicit protocol is specified
|
|
816
835
|
If SSL is explicitly disabled here the plugin will refuse to start if an HTTPS URL is given in 'hosts'
|
817
836
|
|
818
837
|
[id="plugins-{type}s-{plugin}-ssl_certificate_verification"]
|
819
|
-
===== `ssl_certificate_verification`
|
838
|
+
===== `ssl_certificate_verification`
|
820
839
|
|
821
840
|
* Value type is <<boolean,boolean>>
|
822
841
|
* Default value is `true`
|
@@ -826,7 +845,7 @@ For more information on disabling certificate verification please read
|
|
826
845
|
https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf
|
827
846
|
|
828
847
|
[id="plugins-{type}s-{plugin}-template"]
|
829
|
-
===== `template`
|
848
|
+
===== `template`
|
830
849
|
|
831
850
|
* Value type is <<path,path>>
|
832
851
|
* There is no default value for this setting.
|
@@ -835,7 +854,7 @@ You can set the path to your own template here, if you so desire.
|
|
835
854
|
If not set, the included template will be used.
|
836
855
|
|
837
856
|
[id="plugins-{type}s-{plugin}-template_name"]
|
838
|
-
===== `template_name`
|
857
|
+
===== `template_name`
|
839
858
|
|
840
859
|
* Value type is <<string,string>>
|
841
860
|
* Default value depends on whether <<plugins-{type}s-{plugin}-ecs_compatibility>> is enabled:
|
@@ -852,7 +871,7 @@ change this, you will need to prune the old template manually, e.g.
|
|
852
871
|
where `OldTemplateName` is whatever the former setting was.
|
853
872
|
|
854
873
|
[id="plugins-{type}s-{plugin}-template_overwrite"]
|
855
|
-
===== `template_overwrite`
|
874
|
+
===== `template_overwrite`
|
856
875
|
|
857
876
|
* Value type is <<boolean,boolean>>
|
858
877
|
* Default value is `false`
|
@@ -869,7 +888,7 @@ template (logstash), setting this to true will make Logstash to overwrite
|
|
869
888
|
the "logstash" template (i.e. removing all customized settings)
|
870
889
|
|
871
890
|
[id="plugins-{type}s-{plugin}-timeout"]
|
872
|
-
===== `timeout`
|
891
|
+
===== `timeout`
|
873
892
|
|
874
893
|
* Value type is <<number,number>>
|
875
894
|
* Default value is `60`
|
@@ -878,7 +897,7 @@ Set the timeout, in seconds, for network operations and requests sent Elasticsea
|
|
878
897
|
a timeout occurs, the request will be retried.
|
879
898
|
|
880
899
|
[id="plugins-{type}s-{plugin}-truststore"]
|
881
|
-
===== `truststore`
|
900
|
+
===== `truststore`
|
882
901
|
|
883
902
|
* Value type is <<path,path>>
|
884
903
|
* There is no default value for this setting.
|
@@ -888,7 +907,7 @@ It can be either .jks or .p12.
|
|
888
907
|
Use either `:truststore` or `:cacert`.
|
889
908
|
|
890
909
|
[id="plugins-{type}s-{plugin}-truststore_password"]
|
891
|
-
===== `truststore_password`
|
910
|
+
===== `truststore_password`
|
892
911
|
|
893
912
|
* Value type is <<password,password>>
|
894
913
|
* There is no default value for this setting.
|
@@ -896,7 +915,7 @@ Use either `:truststore` or `:cacert`.
|
|
896
915
|
Set the truststore password
|
897
916
|
|
898
917
|
[id="plugins-{type}s-{plugin}-upsert"]
|
899
|
-
===== `upsert`
|
918
|
+
===== `upsert`
|
900
919
|
|
901
920
|
* Value type is <<string,string>>
|
902
921
|
* Default value is `""`
|
@@ -905,7 +924,7 @@ Set upsert content for update mode.
|
|
905
924
|
Create a new document with this parameter as json string if `document_id` doesn't exists
|
906
925
|
|
907
926
|
[id="plugins-{type}s-{plugin}-user"]
|
908
|
-
===== `user`
|
927
|
+
===== `user`
|
909
928
|
|
910
929
|
* Value type is <<string,string>>
|
911
930
|
* There is no default value for this setting.
|
@@ -913,7 +932,7 @@ Create a new document with this parameter as json string if `document_id` doesn'
|
|
913
932
|
Username to authenticate to a secure Elasticsearch cluster
|
914
933
|
|
915
934
|
[id="plugins-{type}s-{plugin}-validate_after_inactivity"]
|
916
|
-
===== `validate_after_inactivity`
|
935
|
+
===== `validate_after_inactivity`
|
917
936
|
|
918
937
|
* Value type is <<number,number>>
|
919
938
|
* Default value is `10000`
|
@@ -928,7 +947,7 @@ have become stale (half-closed) while kept inactive in the pool.'
|
|
928
947
|
See https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html#setValidateAfterInactivity(int)[these docs for more info]
|
929
948
|
|
930
949
|
[id="plugins-{type}s-{plugin}-version"]
|
931
|
-
===== `version`
|
950
|
+
===== `version`
|
932
951
|
|
933
952
|
* Value type is <<string,string>>
|
934
953
|
* There is no default value for this setting.
|
@@ -937,7 +956,7 @@ The version to use for indexing. Use sprintf syntax like `%{my_version}` to use
|
|
937
956
|
See https://www.elastic.co/blog/elasticsearch-versioning-support.
|
938
957
|
|
939
958
|
[id="plugins-{type}s-{plugin}-version_type"]
|
940
|
-
===== `version_type`
|
959
|
+
===== `version_type`
|
941
960
|
|
942
961
|
* Value can be any of: `internal`, `external`, `external_gt`, `external_gte`, `force`
|
943
962
|
* There is no default value for this setting.
|