logstash-output-elasticsearch 10.6.0-java → 10.8.0-java
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +24 -1
- data/CONTRIBUTORS +1 -0
- data/docs/index.asciidoc +97 -78
- data/lib/logstash/outputs/elasticsearch.rb +300 -165
- data/lib/logstash/outputs/elasticsearch/http_client.rb +7 -2
- data/lib/logstash/outputs/elasticsearch/http_client/pool.rb +13 -28
- data/lib/logstash/outputs/elasticsearch/http_client_builder.rb +1 -0
- data/lib/logstash/outputs/elasticsearch/ilm.rb +9 -5
- data/lib/logstash/outputs/elasticsearch/license_checker.rb +47 -0
- data/lib/logstash/outputs/elasticsearch/template_manager.rb +8 -3
- data/lib/logstash/outputs/elasticsearch/templates/ecs-disabled/elasticsearch-8x.json +39 -33
- data/lib/logstash/plugin_mixins/elasticsearch/api_configs.rb +163 -0
- data/lib/logstash/{outputs → plugin_mixins}/elasticsearch/common.rb +40 -154
- data/lib/logstash/plugin_mixins/elasticsearch/noop_license_checker.rb +9 -0
- data/logstash-output-elasticsearch.gemspec +1 -1
- data/spec/es_spec_helper.rb +32 -12
- data/spec/fixtures/template-with-policy-es8x.json +50 -0
- data/spec/integration/outputs/ilm_spec.rb +34 -20
- data/spec/integration/outputs/metrics_spec.rb +1 -5
- data/spec/unit/outputs/elasticsearch/http_client/pool_spec.rb +45 -5
- data/spec/unit/outputs/elasticsearch/http_client_spec.rb +22 -0
- data/spec/unit/outputs/elasticsearch/template_manager_spec.rb +31 -0
- data/spec/unit/outputs/elasticsearch_spec.rb +22 -0
- data/spec/unit/outputs/license_check_spec.rb +41 -0
- metadata +10 -4
- data/lib/logstash/outputs/elasticsearch/common_configs.rb +0 -167
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 29d2b5e316c2ba3dd1d67af673733bd90dd0b5b82c436b1f107cc11d681bd8b5
|
4
|
+
data.tar.gz: 102c70fde50016c59245287f09eeed0d51d4fb411a617b7d397ab306688f022d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6bd3b56c08debf7408ac1cd31980235d3224b5705a42ecc78d28e8da94825cd310ee1b8d0f47b16400327bcc6b0c1e777e2a937a09b1a5a483584d6ca523317d
|
7
|
+
data.tar.gz: c9ba096fb1a39064616ef10be681d1d3b64ab344a7b3b51560b68391c525cf61c6c98a289e068958e43806e7042d9a7ced140ec29407f80680e0323ba44ce165
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,28 @@
|
|
1
|
+
## 10.8.0
|
2
|
+
- Refactored configuration options into specific and shared in PluginMixins namespace [#973](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/973)
|
3
|
+
- Refactored common methods into specific and shared in PluginMixins namespace [#976](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/976)
|
4
|
+
|
5
|
+
## 10.7.3
|
6
|
+
- Added composable index template support for elasticsearch version 8 [#980](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/980)
|
7
|
+
|
8
|
+
## 10.7.2
|
9
|
+
- [DOC] Fixed links to restructured Logstash-to-cloud docs [#975](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/975)
|
10
|
+
|
11
|
+
## 10.7.1
|
12
|
+
- [DOC] Document the permissions required in secured clusters [#969](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/969)
|
13
|
+
|
14
|
+
## 10.7.0
|
15
|
+
- Changed: don't set the pipeline parameter if the value resolves to an empty string [#962](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/962)
|
16
|
+
|
17
|
+
## 10.6.2
|
18
|
+
- [DOC] Added clarifying info on http compression settings and behaviors [#943](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/943)
|
19
|
+
- [DOC] Fixed entry for ilm_policy default value[#956](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/956)
|
20
|
+
|
21
|
+
## 10.6.1
|
22
|
+
- Fixed an issue introduced in 10.6.0 that broke Logstash Core's monitoring feature when this plugin is run in Logstash 7.7-7.8. [#953](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/953)
|
23
|
+
|
1
24
|
## 10.6.0
|
2
|
-
- Added `ecs_compatiblity` mode, for managing ECS-compatable templates [#952](https://github.com/logstash-plugins/logstash-output-elasticsearch/
|
25
|
+
- Added `ecs_compatiblity` mode, for managing ECS-compatable templates [#952](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/952)
|
3
26
|
|
4
27
|
## 10.5.1
|
5
28
|
- [DOC] Removed outdated compatibility notices, reworked cloud notice, and fixed formatting for `hosts` examples [#938](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/938)
|
data/CONTRIBUTORS
CHANGED
@@ -26,6 +26,7 @@ Contributors:
|
|
26
26
|
* Tom Hodder (tolland)
|
27
27
|
* jimmyjones2
|
28
28
|
* Gabriel Moskovicz (gmoskovicz)
|
29
|
+
* Luca Belluccini (lucabelluccini)
|
29
30
|
|
30
31
|
Note: If you've sent us patches, bug reports, or otherwise contributed to
|
31
32
|
Logstash, and you aren't on the list above and want to be, please let us know
|
data/docs/index.asciidoc
CHANGED
@@ -23,14 +23,14 @@ include::{include_path}/plugin_header.asciidoc[]
|
|
23
23
|
|
24
24
|
If you plan to use the Kibana web interface to analyze data transformed by
|
25
25
|
Logstash, use the Elasticsearch output plugin to get your data into
|
26
|
-
Elasticsearch.
|
26
|
+
Elasticsearch.
|
27
27
|
|
28
28
|
This output only speaks the HTTP protocol as it is the preferred protocol for
|
29
29
|
interacting with Elasticsearch. In previous versions it was possible to
|
30
30
|
communicate with Elasticsearch through the transport protocol, which is now
|
31
31
|
reserved for internal cluster communication between nodes
|
32
32
|
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-transport.html[communication between nodes].
|
33
|
-
Using the https://www.elastic.co/guide/en/elasticsearch/reference/current/java-clients.html[transport protocol]
|
33
|
+
Using the https://www.elastic.co/guide/en/elasticsearch/reference/current/java-clients.html[transport protocol]
|
34
34
|
to communicate with the cluster has been deprecated in Elasticsearch 7.0.0 and
|
35
35
|
will be removed in 8.0.0
|
36
36
|
|
@@ -53,10 +53,7 @@ connecting to Elasticsearch 7.x.
|
|
53
53
|
|
54
54
|
===== Hosted {es} Service on Elastic Cloud
|
55
55
|
|
56
|
-
|
57
|
-
https://www.elastic.co/cloud/elasticsearch-service[hosted {es} Service] on
|
58
|
-
Elastic Cloud. The Elasticsearch Service is available on AWS, Google Cloud
|
59
|
-
Platform, and Microsoft Azure. {ess-trial}[Try the {es} Service for free].
|
56
|
+
{ess-leadin}
|
60
57
|
|
61
58
|
==== Compatibility with the Elastic Common Schema (ECS)
|
62
59
|
|
@@ -76,7 +73,7 @@ and will correctly reject events with fields that conflict and cannot be coerced
|
|
76
73
|
|
77
74
|
[NOTE]
|
78
75
|
================================================================================
|
79
|
-
You cannot use dynamic variable substitution when `ilm_enabled` is `true` and
|
76
|
+
You cannot use dynamic variable substitution when `ilm_enabled` is `true` and
|
80
77
|
when using `ilm_rollover_alias`.
|
81
78
|
|
82
79
|
================================================================================
|
@@ -91,7 +88,10 @@ Each Elasticsearch output is a new client connected to the cluster:
|
|
91
88
|
* it has to initialize the client and connect to Elasticsearch (restart time is longer if you have more clients)
|
92
89
|
* it has an associated connection pool
|
93
90
|
|
94
|
-
In order to minimize the number of open connections to Elasticsearch, maximize
|
91
|
+
In order to minimize the number of open connections to Elasticsearch, maximize
|
92
|
+
the bulk size and reduce the number of "small" bulk requests (which could easily
|
93
|
+
fill up the queue), it is usually more efficient to have a single Elasticsearch
|
94
|
+
output.
|
95
95
|
|
96
96
|
Example:
|
97
97
|
[source,ruby]
|
@@ -100,11 +100,14 @@ Example:
|
|
100
100
|
index => "%{[some_field][sub_field]}-%{+YYYY.MM.dd}"
|
101
101
|
}
|
102
102
|
}
|
103
|
-
|
103
|
+
|
104
104
|
**What to do in case there is no field in the event containing the destination index prefix?**
|
105
105
|
|
106
|
-
You can use the `mutate` filter and conditionals to add a `[@metadata]` field
|
107
|
-
|
106
|
+
You can use the `mutate` filter and conditionals to add a `[@metadata]` field
|
107
|
+
(see
|
108
|
+
https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#metadata)
|
109
|
+
to set the destination index for each event. The `[@metadata]` fields will not
|
110
|
+
be sent to Elasticsearch.
|
108
111
|
|
109
112
|
Example:
|
110
113
|
[source,ruby]
|
@@ -136,7 +139,7 @@ HTTP requests to the bulk API are expected to return a 200 response code. All ot
|
|
136
139
|
The following document errors are handled as follows:
|
137
140
|
|
138
141
|
* 400 and 404 errors are sent to the dead letter queue (DLQ), if enabled. If a DLQ is not enabled, a log message will be emitted, and the event will be dropped. See <<plugins-{type}s-{plugin}-dlq-policy>> for more info.
|
139
|
-
* 409 errors (conflict) are logged as a warning and dropped.
|
142
|
+
* 409 errors (conflict) are logged as a warning and dropped.
|
140
143
|
|
141
144
|
Note that 409 exceptions are no longer retried. Please set a higher `retry_on_conflict` value if you experience 409 exceptions.
|
142
145
|
It is more performant for Elasticsearch to retry these exceptions than this plugin.
|
@@ -218,18 +221,29 @@ not reevaluate its DNS value while the keepalive is in effect.
|
|
218
221
|
|
219
222
|
==== HTTP Compression
|
220
223
|
|
221
|
-
This plugin supports request and response compression. Response compression is
|
222
|
-
for Elasticsearch versions 5.0 and later
|
223
|
-
|
224
|
-
|
224
|
+
This plugin supports request and response compression. Response compression is
|
225
|
+
enabled by default for HTTP and for Elasticsearch versions 5.0 and later.
|
226
|
+
|
227
|
+
You don't have to set any configs in Elasticsearch for it to send back a
|
228
|
+
compressed response. For versions before 5.0, or if HTTPS is enabled,
|
229
|
+
`http.compression` must be set to `true`
|
230
|
+
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html#modules-http[in
|
231
|
+
Elasticsearch] to take advantage of response compression when using this plugin.
|
225
232
|
|
226
|
-
For requests compression, regardless of the Elasticsearch version,
|
227
|
-
setting in
|
233
|
+
For requests compression, regardless of the Elasticsearch version, enable the
|
234
|
+
`http_compression` setting in the Logstash config file.
|
228
235
|
|
229
236
|
==== Authentication
|
230
237
|
|
231
238
|
Authentication to a secure Elasticsearch cluster is possible using one of the `user`/`password`, `cloud_auth` or `api_key` options.
|
232
239
|
|
240
|
+
[id="plugins-{type}s-{plugin}-autz"]
|
241
|
+
==== Authorization
|
242
|
+
|
243
|
+
Authorization to a secure Elasticsearch cluster requires `read` permission at index level and `monitoring` permissions at cluster level.
|
244
|
+
The `monitoring` permission at cluster level is necessary to perform periodic connectivity checks.
|
245
|
+
|
246
|
+
|
233
247
|
[id="plugins-{type}s-{plugin}-options"]
|
234
248
|
==== Elasticsearch Output Configuration Options
|
235
249
|
|
@@ -303,7 +317,7 @@ output plugins.
|
|
303
317
|
|
304
318
|
|
305
319
|
[id="plugins-{type}s-{plugin}-action"]
|
306
|
-
===== `action`
|
320
|
+
===== `action`
|
307
321
|
|
308
322
|
* Value type is <<string,string>>
|
309
323
|
* Default value is `"index"`
|
@@ -334,7 +348,7 @@ Authenticate using Elasticsearch API key. Note that this option also requires en
|
|
334
348
|
Format is `id:api_key` where `id` and `api_key` are as returned by the Elasticsearch https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html[Create API key API].
|
335
349
|
|
336
350
|
[id="plugins-{type}s-{plugin}-bulk_path"]
|
337
|
-
===== `bulk_path`
|
351
|
+
===== `bulk_path`
|
338
352
|
|
339
353
|
* Value type is <<string,string>>
|
340
354
|
* There is no default value for this setting.
|
@@ -343,7 +357,7 @@ HTTP Path to perform the _bulk requests to
|
|
343
357
|
this defaults to a concatenation of the path parameter and "_bulk"
|
344
358
|
|
345
359
|
[id="plugins-{type}s-{plugin}-cacert"]
|
346
|
-
===== `cacert`
|
360
|
+
===== `cacert`
|
347
361
|
|
348
362
|
* Value type is <<path,path>>
|
349
363
|
* There is no default value for this setting.
|
@@ -358,7 +372,7 @@ The .cer or .pem file to validate the server's certificate
|
|
358
372
|
|
359
373
|
Cloud authentication string ("<username>:<password>" format) is an alternative for the `user`/`password` pair.
|
360
374
|
|
361
|
-
For more details, check out the https://www.elastic.co/guide/en/logstash/current/connecting-to-cloud.html
|
375
|
+
For more details, check out the https://www.elastic.co/guide/en/logstash/current/connecting-to-cloud.html[Logstash-to-Cloud documentation]
|
362
376
|
|
363
377
|
[id="plugins-{type}s-{plugin}-cloud_id"]
|
364
378
|
===== `cloud_id`
|
@@ -368,10 +382,10 @@ For more details, check out the https://www.elastic.co/guide/en/logstash/current
|
|
368
382
|
|
369
383
|
Cloud ID, from the Elastic Cloud web console. If set `hosts` should not be used.
|
370
384
|
|
371
|
-
For more details, check out the https://www.elastic.co/guide/en/logstash/current/connecting-to-cloud.html
|
385
|
+
For more details, check out the https://www.elastic.co/guide/en/logstash/current/connecting-to-cloud.html[Logstash-to-Cloud documentation]
|
372
386
|
|
373
387
|
[id="plugins-{type}s-{plugin}-doc_as_upsert"]
|
374
|
-
===== `doc_as_upsert`
|
388
|
+
===== `doc_as_upsert`
|
375
389
|
|
376
390
|
* Value type is <<boolean,boolean>>
|
377
391
|
* Default value is `false`
|
@@ -380,7 +394,7 @@ Enable `doc_as_upsert` for update mode.
|
|
380
394
|
Create a new document with source if `document_id` doesn't exist in Elasticsearch
|
381
395
|
|
382
396
|
[id="plugins-{type}s-{plugin}-document_id"]
|
383
|
-
===== `document_id`
|
397
|
+
===== `document_id`
|
384
398
|
|
385
399
|
* Value type is <<string,string>>
|
386
400
|
* There is no default value for this setting.
|
@@ -388,7 +402,7 @@ Create a new document with source if `document_id` doesn't exist in Elasticsearc
|
|
388
402
|
The document ID for the index. Useful for overwriting existing entries in Elasticsearch with the same ID.
|
389
403
|
|
390
404
|
[id="plugins-{type}s-{plugin}-document_type"]
|
391
|
-
===== `document_type`
|
405
|
+
===== `document_type`
|
392
406
|
|
393
407
|
* Value type is <<string,string>>
|
394
408
|
* There is no default value for this setting.
|
@@ -419,7 +433,7 @@ If you don't set a value for this option:
|
|
419
433
|
** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
|
420
434
|
** Otherwise, the default value is `disabled`.
|
421
435
|
|
422
|
-
Controls this plugin's compatibility with the
|
436
|
+
Controls this plugin's compatibility with the https://www.elastic.co/guide/en/ecs/current/index.html[Elastic Common Schema (ECS)],
|
423
437
|
including the installation of ECS-compatible index templates.
|
424
438
|
The value of this setting affects the _default_ values of:
|
425
439
|
|
@@ -428,7 +442,7 @@ The value of this setting affects the _default_ values of:
|
|
428
442
|
* <<plugins-{type}s-{plugin}-ilm_rollover_alias>>
|
429
443
|
|
430
444
|
[id="plugins-{type}s-{plugin}-failure_type_logging_whitelist"]
|
431
|
-
===== `failure_type_logging_whitelist`
|
445
|
+
===== `failure_type_logging_whitelist`
|
432
446
|
|
433
447
|
* Value type is <<array,array>>
|
434
448
|
* Default value is `[]`
|
@@ -449,7 +463,7 @@ an elasticsearch node. The headers will be used for any kind of request
|
|
449
463
|
These custom headers will be overidden by settings like `http_compression`.
|
450
464
|
|
451
465
|
[id="plugins-{type}s-{plugin}-healthcheck_path"]
|
452
|
-
===== `healthcheck_path`
|
466
|
+
===== `healthcheck_path`
|
453
467
|
|
454
468
|
* Value type is <<string,string>>
|
455
469
|
* There is no default value for this setting.
|
@@ -460,7 +474,7 @@ before it is once again eligible to service requests.
|
|
460
474
|
If you have custom firewall rules you may need to change this
|
461
475
|
|
462
476
|
[id="plugins-{type}s-{plugin}-hosts"]
|
463
|
-
===== `hosts`
|
477
|
+
===== `hosts`
|
464
478
|
|
465
479
|
* Value type is <<uri,uri>>
|
466
480
|
* Default value is `[//127.0.0.1]`
|
@@ -476,13 +490,17 @@ Examples:
|
|
476
490
|
`["https://127.0.0.1:9200"]`
|
477
491
|
`["https://127.0.0.1:9200/mypath"]` (If using a proxy on a subpath)
|
478
492
|
|
479
|
-
|
480
|
-
|
493
|
+
Exclude
|
494
|
+
http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html[dedicated
|
495
|
+
master nodes] from the `hosts` list to prevent Logstash from sending bulk
|
496
|
+
requests to the master nodes. This parameter should reference only data or
|
497
|
+
client nodes in Elasticsearch.
|
481
498
|
|
482
|
-
Any special characters present in the URLs here MUST be URL escaped! This means
|
499
|
+
Any special characters present in the URLs here MUST be URL escaped! This means
|
500
|
+
`#` should be put in as `%23` for instance.
|
483
501
|
|
484
502
|
[id="plugins-{type}s-{plugin}-http_compression"]
|
485
|
-
===== `http_compression`
|
503
|
+
===== `http_compression`
|
486
504
|
|
487
505
|
* Value type is <<boolean,boolean>>
|
488
506
|
* Default value is `false`
|
@@ -519,11 +537,14 @@ NOTE: Updating the pattern will require the index template to be rewritten
|
|
519
537
|
|
520
538
|
NOTE: The pattern must finish with a dash and a number that will be automatically incremented when indices rollover.
|
521
539
|
|
540
|
+
NOTE: The pattern is a 6-digit string padded by zeros, regardless of prior index name. Example: 000001.
|
541
|
+
See {ref}/indices-rollover-index.html#rollover-index-api-path-params[Rollover path parameters API docs] for details.
|
542
|
+
|
522
543
|
[id="plugins-{type}s-{plugin}-ilm_policy"]
|
523
544
|
===== `ilm_policy`
|
524
545
|
|
525
546
|
* Value type is <<string,string>>
|
526
|
-
* Default value is `logstash`
|
547
|
+
* Default value is `logstash-policy`
|
527
548
|
|
528
549
|
Modify this setting to use a custom Index Lifecycle Management policy, rather than the default. If this value is not set, the default policy will
|
529
550
|
be automatically installed into Elasticsearch
|
@@ -547,7 +568,7 @@ NOTE: Updating the rollover alias will require the index template to be rewritte
|
|
547
568
|
NOTE: `ilm_rollover_alias` does NOT support dynamic variable substitution as `index` does.
|
548
569
|
|
549
570
|
[id="plugins-{type}s-{plugin}-index"]
|
550
|
-
===== `index`
|
571
|
+
===== `index`
|
551
572
|
|
552
573
|
* Value type is <<string,string>>
|
553
574
|
* Default value depends on whether <<plugins-{type}s-{plugin}-ecs_compatibility>> is enabled:
|
@@ -563,7 +584,7 @@ LS uses Joda to format the index pattern from event timestamp.
|
|
563
584
|
Joda formats are defined http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html[here].
|
564
585
|
|
565
586
|
[id="plugins-{type}s-{plugin}-keystore"]
|
566
|
-
===== `keystore`
|
587
|
+
===== `keystore`
|
567
588
|
|
568
589
|
* Value type is <<path,path>>
|
569
590
|
* There is no default value for this setting.
|
@@ -572,7 +593,7 @@ The keystore used to present a certificate to the server.
|
|
572
593
|
It can be either .jks or .p12
|
573
594
|
|
574
595
|
[id="plugins-{type}s-{plugin}-keystore_password"]
|
575
|
-
===== `keystore_password`
|
596
|
+
===== `keystore_password`
|
576
597
|
|
577
598
|
* Value type is <<password,password>>
|
578
599
|
* There is no default value for this setting.
|
@@ -580,7 +601,7 @@ It can be either .jks or .p12
|
|
580
601
|
Set the keystore password
|
581
602
|
|
582
603
|
[id="plugins-{type}s-{plugin}-manage_template"]
|
583
|
-
===== `manage_template`
|
604
|
+
===== `manage_template`
|
584
605
|
|
585
606
|
* Value type is <<boolean,boolean>>
|
586
607
|
* Default value is `true`
|
@@ -600,7 +621,7 @@ field names) you should set `manage_template` to false and use the REST
|
|
600
621
|
API to apply your templates manually.
|
601
622
|
|
602
623
|
[id="plugins-{type}s-{plugin}-parameters"]
|
603
|
-
===== `parameters`
|
624
|
+
===== `parameters`
|
604
625
|
|
605
626
|
* Value type is <<hash,hash>>
|
606
627
|
* There is no default value for this setting.
|
@@ -610,7 +631,7 @@ to every host listed in the 'hosts' configuration. If the 'hosts' list contains
|
|
610
631
|
urls that already have query strings, the one specified here will be appended.
|
611
632
|
|
612
633
|
[id="plugins-{type}s-{plugin}-parent"]
|
613
|
-
===== `parent`
|
634
|
+
===== `parent`
|
614
635
|
|
615
636
|
* Value type is <<string,string>>
|
616
637
|
* Default value is `nil`
|
@@ -619,7 +640,7 @@ For child documents, ID of the associated parent.
|
|
619
640
|
This can be dynamic using the `%{foo}` syntax.
|
620
641
|
|
621
642
|
[id="plugins-{type}s-{plugin}-password"]
|
622
|
-
===== `password`
|
643
|
+
===== `password`
|
623
644
|
|
624
645
|
* Value type is <<password,password>>
|
625
646
|
* There is no default value for this setting.
|
@@ -627,7 +648,7 @@ This can be dynamic using the `%{foo}` syntax.
|
|
627
648
|
Password to authenticate to a secure Elasticsearch cluster
|
628
649
|
|
629
650
|
[id="plugins-{type}s-{plugin}-path"]
|
630
|
-
===== `path`
|
651
|
+
===== `path`
|
631
652
|
|
632
653
|
* Value type is <<string,string>>
|
633
654
|
* There is no default value for this setting.
|
@@ -638,16 +659,16 @@ Note that if you use paths as components of URLs in the 'hosts' field you may
|
|
638
659
|
not also set this field. That will raise an error at startup
|
639
660
|
|
640
661
|
[id="plugins-{type}s-{plugin}-pipeline"]
|
641
|
-
===== `pipeline`
|
662
|
+
===== `pipeline`
|
642
663
|
|
643
664
|
* Value type is <<string,string>>
|
644
665
|
* Default value is `nil`
|
645
666
|
|
646
|
-
Set which ingest pipeline you wish to execute for an event. You can also use event dependent configuration
|
647
|
-
|
667
|
+
Set which ingest pipeline you wish to execute for an event. You can also use event dependent configuration here
|
668
|
+
like `pipeline => "%{[@metadata][pipeline]}"`. The pipeline parameter won't be set if the value resolves to empty string ("").
|
648
669
|
|
649
670
|
[id="plugins-{type}s-{plugin}-pool_max"]
|
650
|
-
===== `pool_max`
|
671
|
+
===== `pool_max`
|
651
672
|
|
652
673
|
* Value type is <<number,number>>
|
653
674
|
* Default value is `1000`
|
@@ -658,7 +679,7 @@ Setting this too low may mean frequently closing / opening connections
|
|
658
679
|
which is bad.
|
659
680
|
|
660
681
|
[id="plugins-{type}s-{plugin}-pool_max_per_route"]
|
661
|
-
===== `pool_max_per_route`
|
682
|
+
===== `pool_max_per_route`
|
662
683
|
|
663
684
|
* Value type is <<number,number>>
|
664
685
|
* Default value is `100`
|
@@ -669,7 +690,7 @@ Setting this too low may mean frequently closing / opening connections
|
|
669
690
|
which is bad.
|
670
691
|
|
671
692
|
[id="plugins-{type}s-{plugin}-proxy"]
|
672
|
-
===== `proxy`
|
693
|
+
===== `proxy`
|
673
694
|
|
674
695
|
* Value type is <<uri,uri>>
|
675
696
|
* There is no default value for this setting.
|
@@ -680,7 +701,7 @@ An empty string is treated as if proxy was not set. This is useful when using
|
|
680
701
|
environment variables e.g. `proxy => '${LS_PROXY:}'`.
|
681
702
|
|
682
703
|
[id="plugins-{type}s-{plugin}-resurrect_delay"]
|
683
|
-
===== `resurrect_delay`
|
704
|
+
===== `resurrect_delay`
|
684
705
|
|
685
706
|
* Value type is <<number,number>>
|
686
707
|
* Default value is `5`
|
@@ -690,7 +711,7 @@ Resurrection is the process by which backend endpoints marked 'down' are checked
|
|
690
711
|
to see if they have come back to life
|
691
712
|
|
692
713
|
[id="plugins-{type}s-{plugin}-retry_initial_interval"]
|
693
|
-
===== `retry_initial_interval`
|
714
|
+
===== `retry_initial_interval`
|
694
715
|
|
695
716
|
* Value type is <<number,number>>
|
696
717
|
* Default value is `2`
|
@@ -698,7 +719,7 @@ to see if they have come back to life
|
|
698
719
|
Set initial interval in seconds between bulk retries. Doubled on each retry up to `retry_max_interval`
|
699
720
|
|
700
721
|
[id="plugins-{type}s-{plugin}-retry_max_interval"]
|
701
|
-
===== `retry_max_interval`
|
722
|
+
===== `retry_max_interval`
|
702
723
|
|
703
724
|
* Value type is <<number,number>>
|
704
725
|
* Default value is `64`
|
@@ -706,17 +727,15 @@ Set initial interval in seconds between bulk retries. Doubled on each retry up t
|
|
706
727
|
Set max interval in seconds between bulk retries.
|
707
728
|
|
708
729
|
[id="plugins-{type}s-{plugin}-retry_on_conflict"]
|
709
|
-
===== `retry_on_conflict`
|
730
|
+
===== `retry_on_conflict`
|
710
731
|
|
711
732
|
* Value type is <<number,number>>
|
712
733
|
* Default value is `1`
|
713
734
|
|
714
|
-
The number of times Elasticsearch should internally retry an update/upserted document
|
715
|
-
See the https://www.elastic.co/guide/en/elasticsearch/guide/current/partial-updates.html[partial updates]
|
716
|
-
for more info
|
735
|
+
The number of times Elasticsearch should internally retry an update/upserted document.
|
717
736
|
|
718
737
|
[id="plugins-{type}s-{plugin}-routing"]
|
719
|
-
===== `routing`
|
738
|
+
===== `routing`
|
720
739
|
|
721
740
|
* Value type is <<string,string>>
|
722
741
|
* There is no default value for this setting.
|
@@ -725,7 +744,7 @@ A routing override to be applied to all processed events.
|
|
725
744
|
This can be dynamic using the `%{foo}` syntax.
|
726
745
|
|
727
746
|
[id="plugins-{type}s-{plugin}-script"]
|
728
|
-
===== `script`
|
747
|
+
===== `script`
|
729
748
|
|
730
749
|
* Value type is <<string,string>>
|
731
750
|
* Default value is `""`
|
@@ -741,7 +760,7 @@ Example:
|
|
741
760
|
}
|
742
761
|
|
743
762
|
[id="plugins-{type}s-{plugin}-script_lang"]
|
744
|
-
===== `script_lang`
|
763
|
+
===== `script_lang`
|
745
764
|
|
746
765
|
* Value type is <<string,string>>
|
747
766
|
* Default value is `"painless"`
|
@@ -750,7 +769,7 @@ Set the language of the used script. If not set, this defaults to painless in ES
|
|
750
769
|
When using indexed (stored) scripts on Elasticsearch 6 and higher, you must set this parameter to `""` (empty string).
|
751
770
|
|
752
771
|
[id="plugins-{type}s-{plugin}-script_type"]
|
753
|
-
===== `script_type`
|
772
|
+
===== `script_type`
|
754
773
|
|
755
774
|
* Value can be any of: `inline`, `indexed`, `file`
|
756
775
|
* Default value is `["inline"]`
|
@@ -761,7 +780,7 @@ Define the type of script referenced by "script" variable
|
|
761
780
|
file : "script" contains the name of script stored in elasticsearch's config directory
|
762
781
|
|
763
782
|
[id="plugins-{type}s-{plugin}-script_var_name"]
|
764
|
-
===== `script_var_name`
|
783
|
+
===== `script_var_name`
|
765
784
|
|
766
785
|
* Value type is <<string,string>>
|
767
786
|
* Default value is `"event"`
|
@@ -769,7 +788,7 @@ Define the type of script referenced by "script" variable
|
|
769
788
|
Set variable name passed to script (scripted update)
|
770
789
|
|
771
790
|
[id="plugins-{type}s-{plugin}-scripted_upsert"]
|
772
|
-
===== `scripted_upsert`
|
791
|
+
===== `scripted_upsert`
|
773
792
|
|
774
793
|
* Value type is <<boolean,boolean>>
|
775
794
|
* Default value is `false`
|
@@ -777,7 +796,7 @@ Set variable name passed to script (scripted update)
|
|
777
796
|
if enabled, script is in charge of creating non-existent document (scripted update)
|
778
797
|
|
779
798
|
[id="plugins-{type}s-{plugin}-sniffing"]
|
780
|
-
===== `sniffing`
|
799
|
+
===== `sniffing`
|
781
800
|
|
782
801
|
* Value type is <<boolean,boolean>>
|
783
802
|
* Default value is `false`
|
@@ -787,7 +806,7 @@ For Elasticsearch 1.x and 2.x any nodes with `http.enabled` (on by default) will
|
|
787
806
|
For Elasticsearch 5.x and 6.x any nodes with `http.enabled` (on by default) will be added to the hosts list, excluding master-only nodes.
|
788
807
|
|
789
808
|
[id="plugins-{type}s-{plugin}-sniffing_delay"]
|
790
|
-
===== `sniffing_delay`
|
809
|
+
===== `sniffing_delay`
|
791
810
|
|
792
811
|
* Value type is <<number,number>>
|
793
812
|
* Default value is `5`
|
@@ -795,7 +814,7 @@ For Elasticsearch 5.x and 6.x any nodes with `http.enabled` (on by default) will
|
|
795
814
|
How long to wait, in seconds, between sniffing attempts
|
796
815
|
|
797
816
|
[id="plugins-{type}s-{plugin}-sniffing_path"]
|
798
|
-
===== `sniffing_path`
|
817
|
+
===== `sniffing_path`
|
799
818
|
|
800
819
|
* Value type is <<string,string>>
|
801
820
|
* There is no default value for this setting.
|
@@ -806,7 +825,7 @@ if sniffing_path is set it will be used as an absolute path
|
|
806
825
|
do not use full URL here, only paths, e.g. "/sniff/_nodes/http"
|
807
826
|
|
808
827
|
[id="plugins-{type}s-{plugin}-ssl"]
|
809
|
-
===== `ssl`
|
828
|
+
===== `ssl`
|
810
829
|
|
811
830
|
* Value type is <<boolean,boolean>>
|
812
831
|
* There is no default value for this setting.
|
@@ -816,7 +835,7 @@ is specified in the URLs listed in 'hosts'. If no explicit protocol is specified
|
|
816
835
|
If SSL is explicitly disabled here the plugin will refuse to start if an HTTPS URL is given in 'hosts'
|
817
836
|
|
818
837
|
[id="plugins-{type}s-{plugin}-ssl_certificate_verification"]
|
819
|
-
===== `ssl_certificate_verification`
|
838
|
+
===== `ssl_certificate_verification`
|
820
839
|
|
821
840
|
* Value type is <<boolean,boolean>>
|
822
841
|
* Default value is `true`
|
@@ -826,7 +845,7 @@ For more information on disabling certificate verification please read
|
|
826
845
|
https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf
|
827
846
|
|
828
847
|
[id="plugins-{type}s-{plugin}-template"]
|
829
|
-
===== `template`
|
848
|
+
===== `template`
|
830
849
|
|
831
850
|
* Value type is <<path,path>>
|
832
851
|
* There is no default value for this setting.
|
@@ -835,7 +854,7 @@ You can set the path to your own template here, if you so desire.
|
|
835
854
|
If not set, the included template will be used.
|
836
855
|
|
837
856
|
[id="plugins-{type}s-{plugin}-template_name"]
|
838
|
-
===== `template_name`
|
857
|
+
===== `template_name`
|
839
858
|
|
840
859
|
* Value type is <<string,string>>
|
841
860
|
* Default value depends on whether <<plugins-{type}s-{plugin}-ecs_compatibility>> is enabled:
|
@@ -852,7 +871,7 @@ change this, you will need to prune the old template manually, e.g.
|
|
852
871
|
where `OldTemplateName` is whatever the former setting was.
|
853
872
|
|
854
873
|
[id="plugins-{type}s-{plugin}-template_overwrite"]
|
855
|
-
===== `template_overwrite`
|
874
|
+
===== `template_overwrite`
|
856
875
|
|
857
876
|
* Value type is <<boolean,boolean>>
|
858
877
|
* Default value is `false`
|
@@ -869,7 +888,7 @@ template (logstash), setting this to true will make Logstash to overwrite
|
|
869
888
|
the "logstash" template (i.e. removing all customized settings)
|
870
889
|
|
871
890
|
[id="plugins-{type}s-{plugin}-timeout"]
|
872
|
-
===== `timeout`
|
891
|
+
===== `timeout`
|
873
892
|
|
874
893
|
* Value type is <<number,number>>
|
875
894
|
* Default value is `60`
|
@@ -878,7 +897,7 @@ Set the timeout, in seconds, for network operations and requests sent Elasticsea
|
|
878
897
|
a timeout occurs, the request will be retried.
|
879
898
|
|
880
899
|
[id="plugins-{type}s-{plugin}-truststore"]
|
881
|
-
===== `truststore`
|
900
|
+
===== `truststore`
|
882
901
|
|
883
902
|
* Value type is <<path,path>>
|
884
903
|
* There is no default value for this setting.
|
@@ -888,7 +907,7 @@ It can be either .jks or .p12.
|
|
888
907
|
Use either `:truststore` or `:cacert`.
|
889
908
|
|
890
909
|
[id="plugins-{type}s-{plugin}-truststore_password"]
|
891
|
-
===== `truststore_password`
|
910
|
+
===== `truststore_password`
|
892
911
|
|
893
912
|
* Value type is <<password,password>>
|
894
913
|
* There is no default value for this setting.
|
@@ -896,7 +915,7 @@ Use either `:truststore` or `:cacert`.
|
|
896
915
|
Set the truststore password
|
897
916
|
|
898
917
|
[id="plugins-{type}s-{plugin}-upsert"]
|
899
|
-
===== `upsert`
|
918
|
+
===== `upsert`
|
900
919
|
|
901
920
|
* Value type is <<string,string>>
|
902
921
|
* Default value is `""`
|
@@ -905,7 +924,7 @@ Set upsert content for update mode.
|
|
905
924
|
Create a new document with this parameter as json string if `document_id` doesn't exists
|
906
925
|
|
907
926
|
[id="plugins-{type}s-{plugin}-user"]
|
908
|
-
===== `user`
|
927
|
+
===== `user`
|
909
928
|
|
910
929
|
* Value type is <<string,string>>
|
911
930
|
* There is no default value for this setting.
|
@@ -913,7 +932,7 @@ Create a new document with this parameter as json string if `document_id` doesn'
|
|
913
932
|
Username to authenticate to a secure Elasticsearch cluster
|
914
933
|
|
915
934
|
[id="plugins-{type}s-{plugin}-validate_after_inactivity"]
|
916
|
-
===== `validate_after_inactivity`
|
935
|
+
===== `validate_after_inactivity`
|
917
936
|
|
918
937
|
* Value type is <<number,number>>
|
919
938
|
* Default value is `10000`
|
@@ -928,7 +947,7 @@ have become stale (half-closed) while kept inactive in the pool.'
|
|
928
947
|
See https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html#setValidateAfterInactivity(int)[these docs for more info]
|
929
948
|
|
930
949
|
[id="plugins-{type}s-{plugin}-version"]
|
931
|
-
===== `version`
|
950
|
+
===== `version`
|
932
951
|
|
933
952
|
* Value type is <<string,string>>
|
934
953
|
* There is no default value for this setting.
|
@@ -937,7 +956,7 @@ The version to use for indexing. Use sprintf syntax like `%{my_version}` to use
|
|
937
956
|
See https://www.elastic.co/blog/elasticsearch-versioning-support.
|
938
957
|
|
939
958
|
[id="plugins-{type}s-{plugin}-version_type"]
|
940
|
-
===== `version_type`
|
959
|
+
===== `version_type`
|
941
960
|
|
942
961
|
* Value can be any of: `internal`, `external`, `external_gt`, `external_gte`, `force`
|
943
962
|
* There is no default value for this setting.
|