logstash-output-elasticsearch 11.3.3-java → 11.5.0-java

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +16 -0
  3. data/Gemfile +2 -1
  4. data/docs/index.asciidoc +21 -3
  5. data/lib/logstash/outputs/elasticsearch/http_client.rb +2 -2
  6. data/lib/logstash/outputs/elasticsearch/http_client_builder.rb +8 -6
  7. data/lib/logstash/outputs/elasticsearch/templates/ecs-v1/elasticsearch-7x.json +2196 -288
  8. data/lib/logstash/outputs/elasticsearch/templates/ecs-v1/elasticsearch-8x.json +2196 -288
  9. data/lib/logstash/outputs/elasticsearch/templates/ecs-v8/elasticsearch-7x.json +1 -1
  10. data/lib/logstash/outputs/elasticsearch/templates/ecs-v8/elasticsearch-8x.json +1 -1
  11. data/lib/logstash/plugin_mixins/elasticsearch/api_configs.rb +2 -0
  12. data/logstash-output-elasticsearch.gemspec +2 -3
  13. data/spec/es_spec_helper.rb +5 -1
  14. data/spec/fixtures/test_certs/{ca/ca.crt → ca.crt} +0 -0
  15. data/spec/fixtures/test_certs/{ca/ca.key → ca.key} +0 -0
  16. data/spec/fixtures/test_certs/test.p12 +0 -0
  17. data/spec/fixtures/test_certs/test_invalid.crt +36 -0
  18. data/spec/fixtures/test_certs/test_invalid.key +51 -0
  19. data/spec/fixtures/test_certs/test_invalid.p12 +0 -0
  20. data/spec/fixtures/test_certs/test_self_signed.crt +32 -0
  21. data/spec/fixtures/test_certs/test_self_signed.key +54 -0
  22. data/spec/fixtures/test_certs/test_self_signed.p12 +0 -0
  23. data/spec/integration/outputs/ilm_spec.rb +2 -1
  24. data/spec/integration/outputs/index_spec.rb +143 -48
  25. data/spec/integration/outputs/ingest_pipeline_spec.rb +2 -1
  26. data/spec/integration/outputs/no_es_on_startup_spec.rb +12 -6
  27. data/spec/integration/outputs/retry_spec.rb +2 -1
  28. data/spec/unit/outputs/elasticsearch_ssl_spec.rb +1 -1
  29. metadata +22 -8
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3d61a9b747e103df43da5dbfb1d17b1b569be50c28c309b31b465473b745d25c
4
- data.tar.gz: 730421e68d942ed0deb80afc3c8691dc3d42a7c0942b0184e45898d46829a816
3
+ metadata.gz: 7eab2dc342c636e2d5df4e2f9bbd776b446cf110a0803c11fc919fcbbd2f2d83
4
+ data.tar.gz: 565ed3031b685f8541853043d486bda0cff8aa7da2c6f2733e0e2b43345de099
5
5
  SHA512:
6
- metadata.gz: 6541260b1af413acf1b4728aca0d3590e622dce3096257c67e6a27bd66468d773cc197d6b2055fca17439ec5861aed422e198c8c3ab8475ebd257a63f0e6ca56
7
- data.tar.gz: '00410793caecb073abea82106174bc894f13aeb8d8d20e44d22c223ad80a1fd5adfc8b1d7b80fe6670aa0b9fe7f4ce577f6c60f6505064b6678dcfd1100edd0e'
6
+ metadata.gz: 1fbb452170d276531d9b202538d9cf0a23b834053309c818aaaff6f75dd41ecd3d787394234c295639d95d2fe763d1063424cff56b598d1ba16e2fe92374df44
7
+ data.tar.gz: 6b7f17fa679306e65ec94a048ef2205ec9f884cbd11fe1adf861dbb1eccaea752105c3bf149a4fe08f1cdb3c6189200d612dc4cece52ac937eb78d599b188926
data/CHANGELOG.md CHANGED
@@ -1,3 +1,19 @@
1
+ ## 11.5.0
2
+ - Feat: add ssl_supported_protocols option [#1055](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1055)
3
+
4
+ ## 11.4.2
5
+ - [DOC] Add `v8` to supported values for ecs_compatiblity defaults [#1059](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1059)
6
+
7
+ ## 11.4.1
8
+ - Feat: upgrade manticore (http-client) library [#1063](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1063)
9
+ - the underlying changes include latest HttpClient (4.5.13)
10
+ - resolves an old issue with `ssl_certificate_verification => false` still doing some verification logic
11
+
12
+ ## 11.4.0
13
+ - Updates ECS templates [#1062](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1062)
14
+ - Updates v1 templates to 1.12.1 for use with Elasticsearch 7.x and 8.x
15
+ - Updates BETA preview of ECS v8 templates for Elasticsearch 7.x and 8.x
16
+
1
17
  ## 11.3.3
2
18
  - Feat: add support for 'traces' data stream type [#1057](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1057)
3
19
 
data/Gemfile CHANGED
@@ -12,4 +12,5 @@ end
12
12
 
13
13
  if RUBY_VERSION == "1.9.3"
14
14
  gem 'rake', '12.2.1'
15
- end
15
+ end
16
+
data/docs/index.asciidoc CHANGED
@@ -355,6 +355,7 @@ This plugin supports the following configuration options plus the
355
355
  | <<plugins-{type}s-{plugin}-sniffing_path>> |<<string,string>>|No
356
356
  | <<plugins-{type}s-{plugin}-ssl>> |<<boolean,boolean>>|No
357
357
  | <<plugins-{type}s-{plugin}-ssl_certificate_verification>> |<<boolean,boolean>>|No
358
+ | <<plugins-{type}s-{plugin}-ssl_supported_protocols>> |<<string,string>>|No
358
359
  | <<plugins-{type}s-{plugin}-template>> |a valid filesystem path|No
359
360
  | <<plugins-{type}s-{plugin}-template_name>> |<<string,string>>|No
360
361
  | <<plugins-{type}s-{plugin}-template_overwrite>> |<<boolean,boolean>>|No
@@ -554,7 +555,7 @@ If you don't set a value for this option:
554
555
  * Value type is <<string,string>>
555
556
  * Supported values are:
556
557
  ** `disabled`: does not provide ECS-compatible templates
557
- ** `v1`: provides defaults that are compatible with v1 of the Elastic Common Schema
558
+ ** `v1`,`v8`: Elastic Common Schema-compliant behavior
558
559
  * Default value depends on which version of Logstash is running:
559
560
  ** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
560
561
  ** Otherwise, the default value is `disabled`.
@@ -1004,6 +1005,23 @@ Option to validate the server's certificate. Disabling this severely compromises
1004
1005
  For more information on disabling certificate verification please read
1005
1006
  https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf
1006
1007
 
1008
+ [id="plugins-{type}s-{plugin}-ssl_supported_protocols"]
1009
+ ===== `ssl_supported_protocols`
1010
+
1011
+ * Value type is <<string,string>>
1012
+ * Allowed values are: `'TLSv1.1'`, `'TLSv1.2'`, `'TLSv1.3'`
1013
+ * Default depends on the JDK being used. With up-to-date Logstash, the default is `['TLSv1.2', 'TLSv1.3']`.
1014
+ `'TLSv1.1'` is not considered secure and is only provided for legacy applications.
1015
+
1016
+ List of allowed SSL/TLS versions to use when establishing a connection to the Elasticsearch cluster.
1017
+
1018
+ For Java 8 `'TLSv1.3'` is supported only since **8u262** (AdoptOpenJDK), but requires that you set the
1019
+ `LS_JAVA_OPTS="-Djdk.tls.client.protocols=TLSv1.3"` system property in Logstash.
1020
+
1021
+ NOTE: If you configure the plugin to use `'TLSv1.1'` on any recent JVM, such as the one packaged with Logstash,
1022
+ the protocol is disabled by default and needs to be enabled manually by changing `jdk.tls.disabledAlgorithms` in
1023
+ the *$JDK_HOME/conf/security/java.security* configuration file. That is, `TLSv1.1` needs to be removed from the list.
1024
+
1007
1025
  [id="plugins-{type}s-{plugin}-template"]
1008
1026
  ===== `template`
1009
1027
 
@@ -1018,8 +1036,8 @@ If not set, the included template will be used.
1018
1036
 
1019
1037
  * Value type is <<string,string>>
1020
1038
  * Default value depends on whether <<plugins-{type}s-{plugin}-ecs_compatibility>> is enabled:
1021
- ** ECS Compatibility disabled: `logstash`
1022
- ** ECS Compatibility enabled: `ecs-logstash`
1039
+ ** ECS Compatibility disabled: `logstash`
1040
+ ** ECS Compatibility enabled: `ecs-logstash`
1023
1041
 
1024
1042
 
1025
1043
  This configuration option defines how the template is named inside Elasticsearch.
@@ -283,11 +283,11 @@ module LogStash; module Outputs; class ElasticSearch;
283
283
  end
284
284
 
285
285
  def client_settings
286
- @options[:client_settings] || {}
286
+ @_client_settings ||= @options[:client_settings] || {}
287
287
  end
288
288
 
289
289
  def ssl_options
290
- client_settings.fetch(:ssl, {})
290
+ @_ssl_options ||= client_settings.fetch(:ssl, {})
291
291
  end
292
292
 
293
293
  def http_compression
@@ -132,14 +132,16 @@ module LogStash; module Outputs; class ElasticSearch;
132
132
  ssl_options[:keystore] = keystore
133
133
  ssl_options[:keystore_password] = keystore_password.value if keystore_password
134
134
  end
135
+
135
136
  if !params["ssl_certificate_verification"]
136
- logger.warn [
137
- "** WARNING ** Detected UNSAFE options in elasticsearch output configuration!",
138
- "** WARNING ** You have enabled encryption but DISABLED certificate verification.",
139
- "** WARNING ** To make sure your data is secure change :ssl_certificate_verification to true"
140
- ].join("\n")
141
- ssl_options[:verify] = false
137
+ logger.warn "You have enabled encryption but DISABLED certificate verification, " +
138
+ "to make sure your data is secure remove `ssl_certificate_verification => false`"
139
+ ssl_options[:verify] = :disable # false accepts self-signed but still validates hostname
142
140
  end
141
+
142
+ protocols = params['ssl_supported_protocols']
143
+ ssl_options[:protocols] = protocols if protocols && protocols.any?
144
+
143
145
  { ssl: ssl_options }
144
146
  end
145
147