logstash-input-http 3.3.7-java → 3.4.3-java
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -0
- data/VERSION +1 -1
- data/docs/index.asciidoc +93 -2
- data/lib/logstash/inputs/http.rb +46 -3
- data/lib/logstash-input-http_jars.rb +3 -3
- data/logstash-input-http.gemspec +1 -0
- data/spec/inputs/http_spec.rb +142 -76
- data/vendor/jar-dependencies/io/netty/netty-all/{4.1.49.Final/netty-all-4.1.49.Final.jar → 4.1.65.Final/netty-all-4.1.65.Final.jar} +0 -0
- data/vendor/jar-dependencies/org/apache/logging/log4j/log4j-api/2.15.0/log4j-api-2.15.0.jar +0 -0
- data/vendor/jar-dependencies/org/logstash/plugins/input/http/logstash-input-http/3.4.3/logstash-input-http-3.4.3.jar +0 -0
- metadata +20 -7
- data/vendor/jar-dependencies/org/apache/logging/log4j/log4j-api/2.11.1/log4j-api-2.11.1.jar +0 -0
- data/vendor/jar-dependencies/org/logstash/plugins/input/http/logstash-input-http/3.3.7/logstash-input-http-3.3.7.jar +0 -0
    
        checksums.yaml
    CHANGED
    
    | @@ -1,7 +1,7 @@ | |
| 1 1 | 
             
            ---
         | 
| 2 2 | 
             
            SHA256:
         | 
| 3 | 
            -
              metadata.gz:  | 
| 4 | 
            -
              data.tar.gz:  | 
| 3 | 
            +
              metadata.gz: 2aeb67cd027d03518c62b09b4378ef963d0a7b98962a71502b1cc66ba6917ec1
         | 
| 4 | 
            +
              data.tar.gz: 03ca8aabe72cc7dbdb700c055ebf511d0a83721775815df2df8442c87bbc9b92
         | 
| 5 5 | 
             
            SHA512:
         | 
| 6 | 
            -
              metadata.gz:  | 
| 7 | 
            -
              data.tar.gz:  | 
| 6 | 
            +
              metadata.gz: d48e7aff08306639bef632a0446322407208f06a4412229dfd570d45c6ff946d4e4cd980014cb26cec7716146b506fab32181dd405f41c95bfca6b26210b1ffc
         | 
| 7 | 
            +
              data.tar.gz: dc5359dd0f17a583c37ea8c30746e34f9b956aba83a6317c9fd7459c86a1e59f2f47cf22f3914a5d8be3157b79aaebf0c5137674b19e8403bd3bcc3ae9fce890
         | 
    
        data/CHANGELOG.md
    CHANGED
    
    | @@ -1,3 +1,16 @@ | |
| 1 | 
            +
            ## 3.4.3
         | 
| 2 | 
            +
             - Update log4j dependency to 2.15.0
         | 
| 3 | 
            +
             - Fix: update to Gradle 7 [#145](https://github.com/logstash-plugins/logstash-input-http/pull/145)
         | 
| 4 | 
            +
             | 
| 5 | 
            +
            ## 3.4.2
         | 
| 6 | 
            +
             - Docs: added `v8` as an acceptable value for `ecs_compatibility` [#142](https://github.com/logstash-plugins/logstash-input-http/pull/142)
         | 
| 7 | 
            +
             | 
| 8 | 
            +
            ## 3.4.1
         | 
| 9 | 
            +
             - Changed jar dependencies to reflect newer versions [#140](https://github.com/logstash-plugins/logstash-input-http/pull/140)
         | 
| 10 | 
            +
             | 
| 11 | 
            +
            ## 3.4.0
         | 
| 12 | 
            +
             - Add ECS support, mapping Http header to ECS compatible fields [#137](https://github.com/logstash-plugins/logstash-input-http/pull/137)
         | 
| 13 | 
            +
             | 
| 1 14 | 
             
            ## 3.3.7
         | 
| 2 15 | 
             
             - Feat: improved error handling/logging/unwraping [#133](https://github.com/logstash-plugins/logstash-input-http/pull/133)
         | 
| 3 16 |  | 
    
        data/VERSION
    CHANGED
    
    | @@ -1 +1 @@ | |
| 1 | 
            -
            3.3 | 
| 1 | 
            +
            3.4.3
         | 
    
        data/docs/index.asciidoc
    CHANGED
    
    | @@ -32,6 +32,28 @@ This input can also be used to receive webhook requests to integrate with other | |
| 32 32 | 
             
            and applications. By taking advantage of the vast plugin ecosystem available in Logstash
         | 
| 33 33 | 
             
            you can trigger actionable events right from your application.
         | 
| 34 34 |  | 
| 35 | 
            +
            [id="plugins-{type}s-{plugin}-ecs_metadata"]
         | 
| 36 | 
            +
            ==== Event Metadata and the Elastic Common Schema (ECS)
         | 
| 37 | 
            +
            In addition to decoding the events, this input will add HTTP headers containing connection information to each event.
         | 
| 38 | 
            +
            When ECS compatibility is disabled, the headers are stored in the `headers` field, which has the potential to create confusion and schema conflicts downstream.
         | 
| 39 | 
            +
            When ECS is enabled, we can ensure a pipeline maintains access to this metadata throughout the event's lifecycle without polluting the top-level namespace.
         | 
| 40 | 
            +
             | 
| 41 | 
            +
            Here’s how ECS compatibility mode affects output.
         | 
| 42 | 
            +
            [cols="<l,<l,e,<e"]
         | 
| 43 | 
            +
            |=======================================================================
         | 
| 44 | 
            +
            | ECS `disabled` | ECS `v1`,`v8` | Availability | Description
         | 
| 45 | 
            +
             | 
| 46 | 
            +
            | [host] | [host][ip] | Always | Host IP address
         | 
| 47 | 
            +
            | [headers] | [@metadata][input][http][request][headers] | Always | Complete HTTP headers
         | 
| 48 | 
            +
            | [headers][http_version] | [http][version] | Always | HTTP version
         | 
| 49 | 
            +
            | [headers][http_user_agent] | [user_agent][original] | Always | client user agent
         | 
| 50 | 
            +
            | [headers][http_host] | [url][domain] and [url][port] | Always | host domain and port
         | 
| 51 | 
            +
            | [headers][request_method] | [http][method] | Always | HTTP method
         | 
| 52 | 
            +
            | [headers][request_path] | [url][path] | Always | Query path
         | 
| 53 | 
            +
            | [headers][content_length] | [http][request][body][bytes] | Always | Request content length
         | 
| 54 | 
            +
            | [headers][content_type] | [http][request][mime_type] | Always | Request mime type
         | 
| 55 | 
            +
            |=======================================================================
         | 
| 56 | 
            +
             | 
| 35 57 | 
             
            ==== Blocking Behavior
         | 
| 36 58 |  | 
| 37 59 | 
             
            The HTTP protocol doesn't deal well with long running requests. This plugin will either return 
         | 
| @@ -70,6 +92,7 @@ This plugin supports the following configuration options plus the <<plugins-{typ | |
| 70 92 | 
             
            |Setting |Input type|Required
         | 
| 71 93 | 
             
            | <<plugins-{type}s-{plugin}-additional_codecs>> |<<hash,hash>>|No
         | 
| 72 94 | 
             
            | <<plugins-{type}s-{plugin}-cipher_suites>> |<<array,array>>|No
         | 
| 95 | 
            +
            | <<plugins-{type}s-{plugin}-ecs_compatibility>> | <<string,string>>|No
         | 
| 73 96 | 
             
            | <<plugins-{type}s-{plugin}-host>> |<<string,string>>|No
         | 
| 74 97 | 
             
            | <<plugins-{type}s-{plugin}-keystore>> |<<path,path>>|No
         | 
| 75 98 | 
             
            | <<plugins-{type}s-{plugin}-keystore_password>> |<<password,password>>|No
         | 
| @@ -115,6 +138,72 @@ and no codec for the request's content-type is found | |
| 115 138 |  | 
| 116 139 | 
             
            The list of ciphers suite to use, listed by priorities.
         | 
| 117 140 |  | 
| 141 | 
            +
            [id="plugins-{type}s-{plugin}-ecs_compatibility"]
         | 
| 142 | 
            +
            ===== `ecs_compatibility`
         | 
| 143 | 
            +
             | 
| 144 | 
            +
            * Value type is <<string,string>>
         | 
| 145 | 
            +
            * Supported values are:
         | 
| 146 | 
            +
            ** `disabled`: unstructured connection metadata added at root level
         | 
| 147 | 
            +
            ** `v1`,`v8`: headers added under `[@metadata][http][header]`. Some are copied to structured ECS fields `http`, `url`, `user_agent` and `host`
         | 
| 148 | 
            +
             | 
| 149 | 
            +
            Controls this plugin's compatibility with the
         | 
| 150 | 
            +
            {ecs-ref}[Elastic Common Schema (ECS)].
         | 
| 151 | 
            +
            See <<plugins-{type}s-{plugin}-ecs_metadata>> for detailed information.
         | 
| 152 | 
            +
             | 
| 153 | 
            +
            Example output:
         | 
| 154 | 
            +
             | 
| 155 | 
            +
            **Sample output: ECS disabled**
         | 
| 156 | 
            +
            [source,text]
         | 
| 157 | 
            +
            -----
         | 
| 158 | 
            +
            {
         | 
| 159 | 
            +
                "@version" => "1",
         | 
| 160 | 
            +
                "headers" => {
         | 
| 161 | 
            +
                       "request_path" => "/twitter/tweet/1",
         | 
| 162 | 
            +
                        "http_accept" => "*/*",
         | 
| 163 | 
            +
                       "http_version" => "HTTP/1.1",
         | 
| 164 | 
            +
                     "request_method" => "PUT",
         | 
| 165 | 
            +
                          "http_host" => "localhost:8080",
         | 
| 166 | 
            +
                    "http_user_agent" => "curl/7.64.1",
         | 
| 167 | 
            +
                     "content_length" => "5",
         | 
| 168 | 
            +
                       "content_type" => "application/x-www-form-urlencoded"
         | 
| 169 | 
            +
                },
         | 
| 170 | 
            +
                "@timestamp" => 2021-05-28T19:27:28.609Z,
         | 
| 171 | 
            +
                "host" => "127.0.0.1",
         | 
| 172 | 
            +
                "message" => "hello"
         | 
| 173 | 
            +
            }
         | 
| 174 | 
            +
            -----
         | 
| 175 | 
            +
             | 
| 176 | 
            +
            **Sample output: ECS enabled**
         | 
| 177 | 
            +
            [source,text]
         | 
| 178 | 
            +
            -----
         | 
| 179 | 
            +
            {
         | 
| 180 | 
            +
                "@version" => "1",
         | 
| 181 | 
            +
                "user_agent" => {
         | 
| 182 | 
            +
                    "original" => "curl/7.64.1"
         | 
| 183 | 
            +
                },
         | 
| 184 | 
            +
                "http" => {
         | 
| 185 | 
            +
                    "method" => "PUT",
         | 
| 186 | 
            +
                    "request" => {
         | 
| 187 | 
            +
                        "mime_type" => "application/x-www-form-urlencoded",
         | 
| 188 | 
            +
                        "body" => {
         | 
| 189 | 
            +
                            "bytes" => "5"
         | 
| 190 | 
            +
                        }
         | 
| 191 | 
            +
                    },
         | 
| 192 | 
            +
                    "version" => "HTTP/1.1"
         | 
| 193 | 
            +
                },
         | 
| 194 | 
            +
                "url" => {
         | 
| 195 | 
            +
                      "port" => "8080",
         | 
| 196 | 
            +
                    "domain" => "snmp1",
         | 
| 197 | 
            +
                      "path" => "/twitter/tweet/1"
         | 
| 198 | 
            +
                },
         | 
| 199 | 
            +
                "@timestamp" => 2021-05-28T23:32:38.222Z,
         | 
| 200 | 
            +
                "host" => {
         | 
| 201 | 
            +
                    "ip" => "127.0.0.1"
         | 
| 202 | 
            +
                },
         | 
| 203 | 
            +
                "message" => "hello",
         | 
| 204 | 
            +
            }
         | 
| 205 | 
            +
            -----
         | 
| 206 | 
            +
             | 
| 118 207 | 
             
            [id="plugins-{type}s-{plugin}-host"]
         | 
| 119 208 | 
             
            ===== `host` 
         | 
| 120 209 |  | 
| @@ -209,7 +298,8 @@ specify a custom set of response headers | |
| 209 298 | 
             
            ===== `remote_host_target_field` 
         | 
| 210 299 |  | 
| 211 300 | 
             
              * Value type is <<string,string>>
         | 
| 212 | 
            -
              * Default value is `"host"`
         | 
| 301 | 
            +
              * Default value is `"host"` when ECS is disabled
         | 
| 302 | 
            +
              * Default value is `[host][ip]` when ECS is enabled
         | 
| 213 303 |  | 
| 214 304 | 
             
            specify a target field for the client host of the http request
         | 
| 215 305 |  | 
| @@ -217,7 +307,8 @@ specify a target field for the client host of the http request | |
| 217 307 | 
             
            ===== `request_headers_target_field` 
         | 
| 218 308 |  | 
| 219 309 | 
             
              * Value type is <<string,string>>
         | 
| 220 | 
            -
              * Default value is `"headers"`
         | 
| 310 | 
            +
              * Default value is `"headers"` when ECS is disabled
         | 
| 311 | 
            +
              * Default value is `[@metadata][http][header]` when ECS is enabled
         | 
| 221 312 |  | 
| 222 313 | 
             
            specify target field for the client host of the http request
         | 
| 223 314 |  | 
    
        data/lib/logstash/inputs/http.rb
    CHANGED
    
    | @@ -3,6 +3,7 @@ require "logstash/inputs/base" | |
| 3 3 | 
             
            require "logstash/namespace"
         | 
| 4 4 | 
             
            require "stud/interval"
         | 
| 5 5 | 
             
            require "logstash-input-http_jars"
         | 
| 6 | 
            +
            require "logstash/plugin_mixins/ecs_compatibility_support"
         | 
| 6 7 |  | 
| 7 8 | 
             
            # Using this input you can receive single or multiline events over http(s).
         | 
| 8 9 | 
             
            # Applications can send a HTTP POST request with a body to the endpoint started by this
         | 
| @@ -25,6 +26,7 @@ require "logstash-input-http_jars" | |
| 25 26 | 
             
            # format]
         | 
| 26 27 | 
             
            #
         | 
| 27 28 | 
             
            class LogStash::Inputs::Http < LogStash::Inputs::Base
         | 
| 29 | 
            +
              include LogStash::PluginMixins::ECSCompatibilitySupport(:disabled, :v1, :v8 => :v1)
         | 
| 28 30 | 
             
              require "logstash/inputs/http/tls"
         | 
| 29 31 |  | 
| 30 32 | 
             
              java_import "io.netty.handler.codec.http.HttpUtil"
         | 
| @@ -104,10 +106,10 @@ class LogStash::Inputs::Http < LogStash::Inputs::Base | |
| 104 106 | 
             
              config :response_headers, :validate => :hash, :default => { 'Content-Type' => 'text/plain' }
         | 
| 105 107 |  | 
| 106 108 | 
             
              # target field for the client host of the http request
         | 
| 107 | 
            -
              config :remote_host_target_field, :validate => :string | 
| 109 | 
            +
              config :remote_host_target_field, :validate => :string
         | 
| 108 110 |  | 
| 109 111 | 
             
              # target field for the client host of the http request
         | 
| 110 | 
            -
              config :request_headers_target_field, :validate => :string | 
| 112 | 
            +
              config :request_headers_target_field, :validate => :string
         | 
| 111 113 |  | 
| 112 114 | 
             
              config :threads, :validate => :number, :required => false, :default => ::LogStash::Config::CpuCoreStrategy.maximum
         | 
| 113 115 |  | 
| @@ -130,7 +132,7 @@ class LogStash::Inputs::Http < LogStash::Inputs::Base | |
| 130 132 |  | 
| 131 133 | 
             
                validate_ssl_settings!
         | 
| 132 134 |  | 
| 133 | 
            -
                if @user && @password | 
| 135 | 
            +
                if @user && @password
         | 
| 134 136 | 
             
                  token = Base64.strict_encode64("#{@user}:#{@password.value}")
         | 
| 135 137 | 
             
                  @auth_token = "Basic #{token}"
         | 
| 136 138 | 
             
                end
         | 
| @@ -144,6 +146,9 @@ class LogStash::Inputs::Http < LogStash::Inputs::Base | |
| 144 146 | 
             
                require "logstash/inputs/http/message_handler"
         | 
| 145 147 | 
             
                message_handler = MessageHandler.new(self, @codec, @codecs, @auth_token)
         | 
| 146 148 | 
             
                @http_server = create_http_server(message_handler)
         | 
| 149 | 
            +
             | 
| 150 | 
            +
                @remote_host_target_field ||= ecs_select[disabled: "host", v1: "[host][ip]"]
         | 
| 151 | 
            +
                @request_headers_target_field ||= ecs_select[disabled: "headers", v1: "[@metadata][input][http][request][headers]"]
         | 
| 147 152 | 
             
              end # def register
         | 
| 148 153 |  | 
| 149 154 | 
             
              def run(queue)
         | 
| @@ -177,12 +182,50 @@ class LogStash::Inputs::Http < LogStash::Inputs::Base | |
| 177 182 | 
             
              end
         | 
| 178 183 |  | 
| 179 184 | 
             
              def push_decoded_event(headers, remote_address, event)
         | 
| 185 | 
            +
                add_ecs_fields(headers, event)
         | 
| 180 186 | 
             
                event.set(@request_headers_target_field, headers)
         | 
| 181 187 | 
             
                event.set(@remote_host_target_field, remote_address)
         | 
| 182 188 | 
             
                decorate(event)
         | 
| 183 189 | 
             
                @queue << event
         | 
| 184 190 | 
             
              end
         | 
| 185 191 |  | 
| 192 | 
            +
              def add_ecs_fields(headers, event)
         | 
| 193 | 
            +
                return if ecs_compatibility == :disabled
         | 
| 194 | 
            +
             | 
| 195 | 
            +
                http_version = headers.get("http_version")
         | 
| 196 | 
            +
                event.set("[http][version]", http_version) if http_version
         | 
| 197 | 
            +
             | 
| 198 | 
            +
                http_user_agent = headers.get("http_user_agent")
         | 
| 199 | 
            +
                event.set("[user_agent][original]", http_user_agent) if http_user_agent
         | 
| 200 | 
            +
             | 
| 201 | 
            +
                http_host = headers.get("http_host")
         | 
| 202 | 
            +
                domain, port = self.class.get_domain_port(http_host)
         | 
| 203 | 
            +
                event.set("[url][domain]", domain) if domain
         | 
| 204 | 
            +
                event.set("[url][port]", port) if port
         | 
| 205 | 
            +
             | 
| 206 | 
            +
                request_method = headers.get("request_method")
         | 
| 207 | 
            +
                event.set("[http][method]", request_method) if request_method
         | 
| 208 | 
            +
             | 
| 209 | 
            +
                request_path = headers.get("request_path")
         | 
| 210 | 
            +
                event.set("[url][path]", request_path) if request_path
         | 
| 211 | 
            +
             | 
| 212 | 
            +
                content_length = headers.get("content_length")
         | 
| 213 | 
            +
                event.set("[http][request][body][bytes]", content_length) if content_length
         | 
| 214 | 
            +
             | 
| 215 | 
            +
                content_type = headers.get("content_type")
         | 
| 216 | 
            +
                event.set("[http][request][mime_type]", content_type) if content_type
         | 
| 217 | 
            +
              end
         | 
| 218 | 
            +
             | 
| 219 | 
            +
              # match the domain and port in either IPV4, "127.0.0.1:8080", or IPV6, "[2001:db8::8a2e:370:7334]:8080", style
         | 
| 220 | 
            +
              # return [domain, port]
         | 
| 221 | 
            +
              def self.get_domain_port(http_host)
         | 
| 222 | 
            +
                if /^(([^:]+)|\[(.*)\])\:([\d]+)$/ =~ http_host
         | 
| 223 | 
            +
                  ["#{$2 || $3}", $4.to_i]
         | 
| 224 | 
            +
                else
         | 
| 225 | 
            +
                  [http_host, nil]
         | 
| 226 | 
            +
                end
         | 
| 227 | 
            +
              end
         | 
| 228 | 
            +
             | 
| 186 229 | 
             
              def validate_ssl_settings!
         | 
| 187 230 | 
             
                if !@ssl
         | 
| 188 231 | 
             
                  @logger.warn("SSL Certificate will not be used") if @ssl_certificate
         | 
| @@ -1,6 +1,6 @@ | |
| 1 1 | 
             
            # AUTOGENERATED BY THE GRADLE SCRIPT. DO NOT EDIT.
         | 
| 2 2 |  | 
| 3 3 | 
             
            require 'jar_dependencies'
         | 
| 4 | 
            -
            require_jar('io.netty', 'netty-all', '4.1. | 
| 5 | 
            -
            require_jar('org.apache.logging.log4j', 'log4j-api', '2. | 
| 6 | 
            -
            require_jar('org.logstash.plugins.input.http', 'logstash-input-http', '3.3 | 
| 4 | 
            +
            require_jar('io.netty', 'netty-all', '4.1.65.Final')
         | 
| 5 | 
            +
            require_jar('org.apache.logging.log4j', 'log4j-api', '2.15.0')
         | 
| 6 | 
            +
            require_jar('org.logstash.plugins.input.http', 'logstash-input-http', '3.4.3')
         | 
    
        data/logstash-input-http.gemspec
    CHANGED
    
    | @@ -23,6 +23,7 @@ Gem::Specification.new do |s| | |
| 23 23 | 
             
              s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
         | 
| 24 24 | 
             
              s.add_runtime_dependency 'logstash-codec-plain'
         | 
| 25 25 | 
             
              s.add_runtime_dependency 'jar-dependencies', '~> 0.3', '>= 0.3.4'
         | 
| 26 | 
            +
              s.add_runtime_dependency 'logstash-mixin-ecs_compatibility_support', '~>1.2'
         | 
| 26 27 |  | 
| 27 28 | 
             
              s.add_development_dependency 'logstash-devutils'
         | 
| 28 29 | 
             
              s.add_development_dependency 'logstash-codec-json'
         | 
    
        data/spec/inputs/http_spec.rb
    CHANGED
    
    | @@ -6,6 +6,7 @@ require "manticore" | |
| 6 6 | 
             
            require "stud/temporary"
         | 
| 7 7 | 
             
            require "zlib"
         | 
| 8 8 | 
             
            require "stringio"
         | 
| 9 | 
            +
            require 'logstash/plugin_mixins/ecs_compatibility_support/spec_helper'
         | 
| 9 10 |  | 
| 10 11 | 
             
            java_import "io.netty.handler.ssl.util.SelfSignedCertificate"
         | 
| 11 12 |  | 
| @@ -34,20 +35,7 @@ describe LogStash::Inputs::Http do | |
| 34 35 | 
             
                subject { LogStash::Inputs::Http.new("port" => port) }
         | 
| 35 36 |  | 
| 36 37 | 
             
                before :each do
         | 
| 37 | 
            -
                   | 
| 38 | 
            -
                  t = Thread.new { subject.run(logstash_queue) }
         | 
| 39 | 
            -
                  ok = false
         | 
| 40 | 
            -
                  until ok
         | 
| 41 | 
            -
                    begin
         | 
| 42 | 
            -
                      client.post("http://127.0.0.1:#{port}", :body => '{}').call
         | 
| 43 | 
            -
                    rescue => e
         | 
| 44 | 
            -
                      # retry
         | 
| 45 | 
            -
                    else
         | 
| 46 | 
            -
                      ok = true
         | 
| 47 | 
            -
                    end
         | 
| 48 | 
            -
                    sleep 0.01
         | 
| 49 | 
            -
                  end
         | 
| 50 | 
            -
                  logstash_queue.pop if logstash_queue.size == 1 # pop test event
         | 
| 38 | 
            +
                  setup_server_client
         | 
| 51 39 | 
             
                end
         | 
| 52 40 |  | 
| 53 41 | 
             
                describe "handling overflowing requests with a 429" do
         | 
| @@ -85,65 +73,6 @@ describe LogStash::Inputs::Http do | |
| 85 73 | 
             
                  end
         | 
| 86 74 | 
             
                end
         | 
| 87 75 |  | 
| 88 | 
            -
                describe "remote host" do
         | 
| 89 | 
            -
                  subject { LogStash::Inputs::Http.new(config.merge("port" => port)) }
         | 
| 90 | 
            -
                  context "by default" do
         | 
| 91 | 
            -
                    let(:config) { {} }
         | 
| 92 | 
            -
                    it "is written to the \"host\" field" do
         | 
| 93 | 
            -
                      client.post("http://localhost:#{port}/meh.json",
         | 
| 94 | 
            -
                                  :headers => { "content-type" => "text/plain" },
         | 
| 95 | 
            -
                                  :body => "hello").call
         | 
| 96 | 
            -
                      event = logstash_queue.pop
         | 
| 97 | 
            -
                      expect(event.get("host")).to eq("127.0.0.1")
         | 
| 98 | 
            -
                    end
         | 
| 99 | 
            -
                  end
         | 
| 100 | 
            -
             | 
| 101 | 
            -
                  context "when using remote_host_target_field" do
         | 
| 102 | 
            -
                    let(:config) { { "remote_host_target_field" => "remote_host" } }
         | 
| 103 | 
            -
                    it "is written to the value of \"remote_host_target_field\" property" do
         | 
| 104 | 
            -
                      client.post("http://localhost:#{port}/meh.json",
         | 
| 105 | 
            -
                                  :headers => { "content-type" => "text/plain" },
         | 
| 106 | 
            -
                                  :body => "hello").call
         | 
| 107 | 
            -
                      event = logstash_queue.pop
         | 
| 108 | 
            -
                      expect(event.get("remote_host")).to eq("127.0.0.1")
         | 
| 109 | 
            -
                    end
         | 
| 110 | 
            -
                  end
         | 
| 111 | 
            -
                end
         | 
| 112 | 
            -
             | 
| 113 | 
            -
                describe "request headers" do
         | 
| 114 | 
            -
                  subject { LogStash::Inputs::Http.new(config.merge("port" => port)) }
         | 
| 115 | 
            -
                  context "by default" do
         | 
| 116 | 
            -
                    let(:config) { {} }
         | 
| 117 | 
            -
                    it "are written to the \"headers\" field" do
         | 
| 118 | 
            -
                      client.post("http://localhost:#{port}/meh.json",
         | 
| 119 | 
            -
                                  :headers => { "content-type" => "text/plain" },
         | 
| 120 | 
            -
                                  :body => "hello").call
         | 
| 121 | 
            -
                      event = logstash_queue.pop
         | 
| 122 | 
            -
                      expect(event.get("headers")).to be_a(Hash)
         | 
| 123 | 
            -
                      expect(event.get("headers")).to include("request_method" => "POST")
         | 
| 124 | 
            -
                    end
         | 
| 125 | 
            -
                  end
         | 
| 126 | 
            -
                  context "when using request_headers_target_field" do
         | 
| 127 | 
            -
                    let(:config) { { "request_headers_target_field" => "request_headers" } }
         | 
| 128 | 
            -
                    it "are written to the field set in \"request_headers_target_field\"" do
         | 
| 129 | 
            -
                      client.post("http://localhost:#{port}/meh.json",
         | 
| 130 | 
            -
                                  :headers => { "content-type" => "text/plain" },
         | 
| 131 | 
            -
                                  :body => "hello").call
         | 
| 132 | 
            -
                      event = logstash_queue.pop
         | 
| 133 | 
            -
                      expect(event.get("request_headers")).to be_a(Hash)
         | 
| 134 | 
            -
                      expect(event.get("request_headers")).to include("request_method" => "POST")
         | 
| 135 | 
            -
                    end
         | 
| 136 | 
            -
                  end
         | 
| 137 | 
            -
                end
         | 
| 138 | 
            -
             | 
| 139 | 
            -
                it "should include remote host in \"host\" property" do
         | 
| 140 | 
            -
                  client.post("http://127.0.0.1:#{port}/meh.json",
         | 
| 141 | 
            -
                              :headers => { "content-type" => "text/plain" },
         | 
| 142 | 
            -
                              :body => "hello").call
         | 
| 143 | 
            -
                  event = logstash_queue.pop
         | 
| 144 | 
            -
                  expect(event.get("host")).to eq("127.0.0.1")
         | 
| 145 | 
            -
                end
         | 
| 146 | 
            -
             | 
| 147 76 | 
             
                context "with default codec" do
         | 
| 148 77 | 
             
                  subject { LogStash::Inputs::Http.new("port" => port) }
         | 
| 149 78 | 
             
                  context "when receiving a text/plain request" do
         | 
| @@ -368,6 +297,143 @@ describe LogStash::Inputs::Http do | |
| 368 297 | 
             
                end
         | 
| 369 298 | 
             
              end
         | 
| 370 299 |  | 
| 300 | 
            +
              describe "ECS support", :ecs_compatibility_support, :aggregate_failures do
         | 
| 301 | 
            +
                ecs_compatibility_matrix(:disabled, :v1) do |ecs_select|
         | 
| 302 | 
            +
                  let(:host_field) { ecs_select[disabled: "[host]", v1: "[host][ip]"] }
         | 
| 303 | 
            +
                  let(:header_field) { ecs_select[disabled: "headers", v1: "[@metadata][input][http][request][headers]"] }
         | 
| 304 | 
            +
                  let(:http_version_field) { ecs_select[disabled: "[headers][http_version]", v1: "[http][version]"] }
         | 
| 305 | 
            +
                  let(:user_agent_field) { ecs_select[disabled: "[headers][http_user_agent]", v1: "[user_agent][original]"] }
         | 
| 306 | 
            +
                  let(:http_host_field) { "[headers][http_host]" }
         | 
| 307 | 
            +
                  let(:domain_field) { "[url][domain]" }
         | 
| 308 | 
            +
                  let(:port_field) { "[url][port]" }
         | 
| 309 | 
            +
                  let(:request_method_field) { ecs_select[disabled: "[headers][request_method]", v1: "[http][method]"] }
         | 
| 310 | 
            +
                  let(:request_path_field) { ecs_select[disabled: "[headers][request_path]", v1: "[url][path]"] }
         | 
| 311 | 
            +
                  let(:content_length_field) { ecs_select[disabled: "[headers][content_length]", v1: "[http][request][body][bytes]"] }
         | 
| 312 | 
            +
                  let(:content_type_field) { ecs_select[disabled: "[headers][content_type]", v1: "[http][request][mime_type]"] }
         | 
| 313 | 
            +
             | 
| 314 | 
            +
                  before :each do
         | 
| 315 | 
            +
                    allow_any_instance_of(described_class).to receive(:ecs_compatibility).and_return(ecs_compatibility)
         | 
| 316 | 
            +
                    setup_server_client
         | 
| 317 | 
            +
                  end
         | 
| 318 | 
            +
             | 
| 319 | 
            +
                  describe "remote host" do
         | 
| 320 | 
            +
                    subject { LogStash::Inputs::Http.new(config.merge("port" => port)) }
         | 
| 321 | 
            +
                    context "by default" do
         | 
| 322 | 
            +
                      let(:config) { {} }
         | 
| 323 | 
            +
                      it "is written to the \"host\" field" do
         | 
| 324 | 
            +
                        client.post("http://localhost:#{port}/meh.json",
         | 
| 325 | 
            +
                                    :headers => { "content-type" => "text/plain" },
         | 
| 326 | 
            +
                                    :body => "hello").call
         | 
| 327 | 
            +
                        event = logstash_queue.pop
         | 
| 328 | 
            +
                        expect(event.get(host_field)).to eq("127.0.0.1")
         | 
| 329 | 
            +
                      end
         | 
| 330 | 
            +
                    end
         | 
| 331 | 
            +
             | 
| 332 | 
            +
                    context "when using remote_host_target_field" do
         | 
| 333 | 
            +
                      let(:config) { { "remote_host_target_field" => "remote_host" } }
         | 
| 334 | 
            +
                      it "is written to the value of \"remote_host_target_field\" property" do
         | 
| 335 | 
            +
                        client.post("http://localhost:#{port}/meh.json",
         | 
| 336 | 
            +
                                    :headers => { "content-type" => "text/plain" },
         | 
| 337 | 
            +
                                    :body => "hello").call
         | 
| 338 | 
            +
                        event = logstash_queue.pop
         | 
| 339 | 
            +
                        expect(event.get("remote_host")).to eq("127.0.0.1")
         | 
| 340 | 
            +
                      end
         | 
| 341 | 
            +
                    end
         | 
| 342 | 
            +
                  end
         | 
| 343 | 
            +
             | 
| 344 | 
            +
                  describe "request headers" do
         | 
| 345 | 
            +
                    subject { LogStash::Inputs::Http.new(config.merge("port" => port)) }
         | 
| 346 | 
            +
                    context "by default" do
         | 
| 347 | 
            +
                      let(:config) { {} }
         | 
| 348 | 
            +
                      it "are written to the \"headers\" field" do
         | 
| 349 | 
            +
                        client.post("http://localhost:#{port}/meh.json",
         | 
| 350 | 
            +
                                    :headers => { "content-type" => "text/plain" },
         | 
| 351 | 
            +
                                    :body => "hello").call
         | 
| 352 | 
            +
                        event = logstash_queue.pop
         | 
| 353 | 
            +
                        expect(event.get(header_field)).to be_a(Hash)
         | 
| 354 | 
            +
                        expect(event.get(request_method_field)).to eq("POST")
         | 
| 355 | 
            +
                        expect(event.get(request_path_field)).to eq("/meh.json")
         | 
| 356 | 
            +
                        expect(event.get(http_version_field)).to eq("HTTP/1.1")
         | 
| 357 | 
            +
                        expect(event.get(user_agent_field)).to include("Manticore")
         | 
| 358 | 
            +
                        if ecs_compatibility == :disabled
         | 
| 359 | 
            +
                          expect(event.get(http_host_field)).to eq("localhost:#{port}")
         | 
| 360 | 
            +
                        else
         | 
| 361 | 
            +
                          expect(event.get(domain_field)).to eq("localhost")
         | 
| 362 | 
            +
                          expect(event.get(port_field)).to eq(port)
         | 
| 363 | 
            +
                        end
         | 
| 364 | 
            +
             | 
| 365 | 
            +
                        expect(event.get(content_length_field)).to eq("5")
         | 
| 366 | 
            +
                        expect(event.get(content_type_field)).to eq("text/plain")
         | 
| 367 | 
            +
                      end
         | 
| 368 | 
            +
                    end
         | 
| 369 | 
            +
                    context "when using request_headers_target_field" do
         | 
| 370 | 
            +
                      let(:config) { { "request_headers_target_field" => "request_headers" } }
         | 
| 371 | 
            +
                      it "are written to the field set in \"request_headers_target_field\"" do
         | 
| 372 | 
            +
                        client.post("http://localhost:#{port}/meh.json",
         | 
| 373 | 
            +
                                    :headers => { "content-type" => "text/plain" },
         | 
| 374 | 
            +
                                    :body => "hello").call
         | 
| 375 | 
            +
                        event = logstash_queue.pop
         | 
| 376 | 
            +
                        expect(event.get("request_headers")).to be_a(Hash)
         | 
| 377 | 
            +
                        expect(event.get("request_headers")).to include("request_method" => "POST")
         | 
| 378 | 
            +
                        expect(event.get("request_headers")).to include("request_path" => "/meh.json")
         | 
| 379 | 
            +
                        expect(event.get("request_headers")).to include("http_version" => "HTTP/1.1")
         | 
| 380 | 
            +
                        expect(event.get("request_headers")["http_user_agent"]).to include("Manticore")
         | 
| 381 | 
            +
                        expect(event.get("request_headers")).to include("http_host" => "localhost:#{port}")
         | 
| 382 | 
            +
                        expect(event.get("request_headers")).to include("content_length" => "5")
         | 
| 383 | 
            +
                        expect(event.get("request_headers")).to include("content_type" => "text/plain")
         | 
| 384 | 
            +
                      end
         | 
| 385 | 
            +
                    end
         | 
| 386 | 
            +
                  end
         | 
| 387 | 
            +
                end
         | 
| 388 | 
            +
              end
         | 
| 389 | 
            +
             | 
| 390 | 
            +
              # wait until server is ready
         | 
| 391 | 
            +
              def setup_server_client
         | 
| 392 | 
            +
                subject.register
         | 
| 393 | 
            +
                t = Thread.new { subject.run(logstash_queue) }
         | 
| 394 | 
            +
                ok = false
         | 
| 395 | 
            +
                until ok
         | 
| 396 | 
            +
                  begin
         | 
| 397 | 
            +
                    client.post("http://127.0.0.1:#{port}", :body => '{}').call
         | 
| 398 | 
            +
                  rescue => e
         | 
| 399 | 
            +
                    # retry
         | 
| 400 | 
            +
                  else
         | 
| 401 | 
            +
                    ok = true
         | 
| 402 | 
            +
                  end
         | 
| 403 | 
            +
                  sleep 0.01
         | 
| 404 | 
            +
                end
         | 
| 405 | 
            +
                logstash_queue.pop if logstash_queue.size == 1 # pop test event
         | 
| 406 | 
            +
              end
         | 
| 407 | 
            +
             | 
| 408 | 
            +
              describe "parse domain host" do
         | 
| 409 | 
            +
                let(:localhost) { "localhost" }
         | 
| 410 | 
            +
                let(:ipv6) { "2001:db8::8a2e:370:7334" }
         | 
| 411 | 
            +
             | 
| 412 | 
            +
                it "should parse in IPV4 format with port" do
         | 
| 413 | 
            +
                  domain, port = LogStash::Inputs::Http.get_domain_port("#{localhost}:8080")
         | 
| 414 | 
            +
                  expect(domain).to eq(localhost)
         | 
| 415 | 
            +
                  expect(port).to eq(8080)
         | 
| 416 | 
            +
                end
         | 
| 417 | 
            +
             | 
| 418 | 
            +
                it "should parse in IPV4 format without port" do
         | 
| 419 | 
            +
                  domain, port = LogStash::Inputs::Http.get_domain_port(localhost)
         | 
| 420 | 
            +
                  expect(domain).to eq(localhost)
         | 
| 421 | 
            +
                  expect(port).to be_nil
         | 
| 422 | 
            +
                end
         | 
| 423 | 
            +
             | 
| 424 | 
            +
                it "should parse in IPV6 format with port" do
         | 
| 425 | 
            +
                  domain, port = LogStash::Inputs::Http.get_domain_port("[#{ipv6}]:8080")
         | 
| 426 | 
            +
                  expect(domain).to eq(ipv6)
         | 
| 427 | 
            +
                  expect(port).to eq(8080)
         | 
| 428 | 
            +
                end
         | 
| 429 | 
            +
             | 
| 430 | 
            +
                it "should parse in IPV6 format without port" do
         | 
| 431 | 
            +
                  domain, port = LogStash::Inputs::Http.get_domain_port("#{ipv6}")
         | 
| 432 | 
            +
                  expect(domain).to eq(ipv6)
         | 
| 433 | 
            +
                  expect(port).to be_nil
         | 
| 434 | 
            +
                end
         | 
| 435 | 
            +
              end
         | 
| 436 | 
            +
             | 
| 371 437 | 
             
              context "with :ssl => false" do
         | 
| 372 438 | 
             
                subject { LogStash::Inputs::Http.new("port" => port, "ssl" => false) }
         | 
| 373 439 | 
             
                it "should not raise exception" do
         | 
| @@ -440,7 +506,7 @@ describe LogStash::Inputs::Http do | |
| 440 506 | 
             
                  end
         | 
| 441 507 |  | 
| 442 508 | 
             
                  context "with invalid cipher_suites" do
         | 
| 443 | 
            -
                    let(:config) { super.merge("cipher_suites" => "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA38") }
         | 
| 509 | 
            +
                    let(:config) { super().merge("cipher_suites" => "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA38") }
         | 
| 444 510 |  | 
| 445 511 | 
             
                    it "should raise a configuration error" do
         | 
| 446 512 | 
             
                      expect( subject.logger ).to receive(:error) do |msg, opts|
         | 
| @@ -469,7 +535,7 @@ describe LogStash::Inputs::Http do | |
| 469 535 | 
             
                  end
         | 
| 470 536 |  | 
| 471 537 | 
             
                  context "with invalid ssl key config" do
         | 
| 472 | 
            -
                    let(:config) { super.merge("ssl_key_passphrase" => "1234567890") }
         | 
| 538 | 
            +
                    let(:config) { super().merge("ssl_key_passphrase" => "1234567890") }
         | 
| 473 539 |  | 
| 474 540 | 
             
                    it "should raise a configuration error" do
         | 
| 475 541 | 
             
                      expect( subject.logger ).to receive(:error) do |msg, opts|
         | 
| @@ -482,7 +548,7 @@ describe LogStash::Inputs::Http do | |
| 482 548 |  | 
| 483 549 | 
             
                  context "with invalid ssl certificate_authorities" do
         | 
| 484 550 | 
             
                    let(:config) do
         | 
| 485 | 
            -
                      super.merge("ssl_verify_mode" => "peer",
         | 
| 551 | 
            +
                      super().merge("ssl_verify_mode" => "peer",
         | 
| 486 552 | 
             
                                  "ssl_certificate_authorities" => [ ssc.certificate.path, ssc.private_key.path ])
         | 
| 487 553 | 
             
                    end
         | 
| 488 554 |  | 
| Binary file | 
| Binary file | 
| Binary file | 
    
        metadata
    CHANGED
    
    | @@ -1,14 +1,14 @@ | |
| 1 1 | 
             
            --- !ruby/object:Gem::Specification
         | 
| 2 2 | 
             
            name: logstash-input-http
         | 
| 3 3 | 
             
            version: !ruby/object:Gem::Version
         | 
| 4 | 
            -
              version: 3.3 | 
| 4 | 
            +
              version: 3.4.3
         | 
| 5 5 | 
             
            platform: java
         | 
| 6 6 | 
             
            authors:
         | 
| 7 7 | 
             
            - Elastic
         | 
| 8 8 | 
             
            autorequire:
         | 
| 9 9 | 
             
            bindir: bin
         | 
| 10 10 | 
             
            cert_chain: []
         | 
| 11 | 
            -
            date: 2021- | 
| 11 | 
            +
            date: 2021-12-10 00:00:00.000000000 Z
         | 
| 12 12 | 
             
            dependencies:
         | 
| 13 13 | 
             
            - !ruby/object:Gem::Dependency
         | 
| 14 14 | 
             
              requirement: !ruby/object:Gem::Requirement
         | 
| @@ -64,6 +64,20 @@ dependencies: | |
| 64 64 | 
             
                - - ">="
         | 
| 65 65 | 
             
                  - !ruby/object:Gem::Version
         | 
| 66 66 | 
             
                    version: 0.3.4
         | 
| 67 | 
            +
            - !ruby/object:Gem::Dependency
         | 
| 68 | 
            +
              requirement: !ruby/object:Gem::Requirement
         | 
| 69 | 
            +
                requirements:
         | 
| 70 | 
            +
                - - "~>"
         | 
| 71 | 
            +
                  - !ruby/object:Gem::Version
         | 
| 72 | 
            +
                    version: '1.2'
         | 
| 73 | 
            +
              name: logstash-mixin-ecs_compatibility_support
         | 
| 74 | 
            +
              prerelease: false
         | 
| 75 | 
            +
              type: :runtime
         | 
| 76 | 
            +
              version_requirements: !ruby/object:Gem::Requirement
         | 
| 77 | 
            +
                requirements:
         | 
| 78 | 
            +
                - - "~>"
         | 
| 79 | 
            +
                  - !ruby/object:Gem::Version
         | 
| 80 | 
            +
                    version: '1.2'
         | 
| 67 81 | 
             
            - !ruby/object:Gem::Dependency
         | 
| 68 82 | 
             
              requirement: !ruby/object:Gem::Requirement
         | 
| 69 83 | 
             
                requirements:
         | 
| @@ -144,9 +158,9 @@ files: | |
| 144 158 | 
             
            - lib/tasks/build.rake
         | 
| 145 159 | 
             
            - logstash-input-http.gemspec
         | 
| 146 160 | 
             
            - spec/inputs/http_spec.rb
         | 
| 147 | 
            -
            - vendor/jar-dependencies/io/netty/netty-all/4.1. | 
| 148 | 
            -
            - vendor/jar-dependencies/org/apache/logging/log4j/log4j-api/2. | 
| 149 | 
            -
            - vendor/jar-dependencies/org/logstash/plugins/input/http/logstash-input-http/3.3 | 
| 161 | 
            +
            - vendor/jar-dependencies/io/netty/netty-all/4.1.65.Final/netty-all-4.1.65.Final.jar
         | 
| 162 | 
            +
            - vendor/jar-dependencies/org/apache/logging/log4j/log4j-api/2.15.0/log4j-api-2.15.0.jar
         | 
| 163 | 
            +
            - vendor/jar-dependencies/org/logstash/plugins/input/http/logstash-input-http/3.4.3/logstash-input-http-3.4.3.jar
         | 
| 150 164 | 
             
            homepage: http://www.elastic.co/guide/en/logstash/current/index.html
         | 
| 151 165 | 
             
            licenses:
         | 
| 152 166 | 
             
            - Apache License (2.0)
         | 
| @@ -169,8 +183,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement | |
| 169 183 | 
             
                - !ruby/object:Gem::Version
         | 
| 170 184 | 
             
                  version: '0'
         | 
| 171 185 | 
             
            requirements: []
         | 
| 172 | 
            -
             | 
| 173 | 
            -
            rubygems_version: 2.6.13
         | 
| 186 | 
            +
            rubygems_version: 3.1.6
         | 
| 174 187 | 
             
            signing_key:
         | 
| 175 188 | 
             
            specification_version: 4
         | 
| 176 189 | 
             
            summary: Receives events over HTTP or HTTPS
         | 
| Binary file |