logstash-output-scalyr 0.1.24.beta → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 312d501ddc98f27b8d2be32bb20cb29439e7ce096d5f12870ef1b1bf095c32ee
-  data.tar.gz: c0f6c4a51f42acef8266ee30480fc00a19e59098d7b957deddcc017bade71695
+  metadata.gz: 17884743262e123a7719335dd18559236505f4bfbf50ff8b2e2ca24e9416c35f
+  data.tar.gz: 34db0e1f15cc4f9a207fb5a681a5f4d6a1337f1187e01dc6a0f9467440ab53d0
 SHA512:
-  metadata.gz: 2628a0bcfbe9a13dfc5bb9fd93360ef7629fca158cc325184817b4f15492ff5fcb28c03ca992c08dbbf9dc7953d2715b071b5d94f67912c36f66625f7bcf3234
-  data.tar.gz: 1d009fe06794e8f2d15737da874609600e57569fe3d6be0297fb23a1ab99d375f3ac2e9bdbf52b653baa85222bc3d5da001ac8d68a3848c8099ed7eb1510175f
+  metadata.gz: c55678d99d117c2dd08fe48c4505b2da39a57553cf8cc36a84cab7aa38dbf539bb63b564772cf5a2e575c334b617bdb3b5b61d91af13c5cff0c4ea4c18738834
+  data.tar.gz: d8e819de557a452f4129bd94eb581ff40dbdd902a1eba07ef6e7ec838f72ee5dfdd266827100232f03f9269cd933329426846081dcfaf5b3e66084075b79989c

data/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Beta
 
+## 0.2.0.beta, 0.2.0
+
+- Fix a bug and correctly handle ``serverHost`` event level attribute. Now if an event contains
+``serverHost`` attribute, this attribute will be correctly set on the event level and available for
+ "Sources" filtering in the UI.
+- Plugin doesn't set ``serverHost`` attribute with a fixed value of ``Logstash`` on each event
+level anymore. If you still want this behavior, you can achieve that with logstash mutate filter.
+ - Session level ``serverHost`` value now defaults to logstash aggregator node hostname
+ (``use_hostname_for_serverhost`` config option now defaults to true).
+- ``host`` attribute is not removed by default from all the events. By default, logstash adds
+  ``host`` attribute which contains logstash aggregator host to each event. This is now redundant
+  and unncessary with the fixed and improved serverHost behavior (host and serverHost would contain
+  the same value by default). If you want to change this behavior and and still include ``host``
+  attribute on each event you can do that by setting ``remove_host_attribute_from_events`` config
+  option to false.
+
+## 0.1.26.beta
+- Add support for new ``json_library`` config option. Valid values are ``stdlib`` (default) are ``jrjackson``. The later may offer 2-4x faster JSON serialization.
+
 ## 0.1.23.beta
 - Add testing support for disabling estimation of serialized event size for each event in the batch.
 

data/Gemfile

@@ -12,9 +12,13 @@ end
 
 group :test do
   gem "webmock"
+
+  # Require the specific version of `json` used in logstash while testing
+  gem 'json', '1.8.6'
 end
 
 gem 'pry'
 gem 'pry-nav'
 gem 'quantile', '~> 0.2.1'
 gem 'manticore', '~> 0.7.1', platform: :jruby
+gem 'jrjackson', '~> 0.4.14', platform: :jruby

data/README.md

@@ -1,7 +1,6 @@
-
 [![CircleCI](https://circleci.com/gh/scalyr/logstash-output-scalyr.svg?style=svg)](https://circleci.com/gh/scalyr/logstash-output-scalyr)
 
-# [Scalyr output plugin for Logstash (Beta release)]
+# [Scalyr output plugin for Logstash]
 
 This plugin implements a Logstash output plugin that uploads data to [Scalyr](http://www.scalyr.com).
 
@@ -10,7 +9,9 @@ You can view documentation for this plugin [on the Scalyr website](https://app.s
 # Quick start
 
 1. Build the gem, run `gem build logstash-output-scalyr.gemspec` 
-2. Install the gem into a Logstash installation, run `/usr/share/logstash/bin/logstash-plugin install logstash-output-scalyr-0.1.22.beta.gem` or follow the latest official instructions on working with plugins from Logstash.
+2. Install the gem into a Logstash installation, run `/usr/share/logstash/bin/logstash-plugin install logstash-output-scalyr-0.2.0.gem` 
+   or follow the latest official instructions on working with plugins from Logstash. As an alternative, you can directly install latest
+   stable version from RubyGems - ``/usr/share/logstash/bin/logstash-plugin --version 0.2.0 logstash-output-scalyr``
 3. Configure the output plugin (e.g. add it to a pipeline .conf)
 4. Restart Logstash 
 
@@ -40,6 +41,51 @@ output {
 
 In the above example, the Logstash pipeline defines a file input that reads from `/var/log/messages`.  Log events from this source have the `host` and `path` fields.  The pipeline then outputs to the scalyr plugin, which in this example is configured to remap `host`->`serverHost` and `path`->`logfile`, thus facilitating filtering in the Scalyr UI.
 
+## Notes on serverHost attribute handling
+
+> Some of this functionality has been fixed and changed in the v0.2.0 release. In previous
+  versions, plugin added ``serverHost`` attribute with a value of ``Logstash`` to each event and
+  this attribute was not handled correctly - it was treated as a regular event level attribute
+  and not a special attribute which can be used for Source functionality and filtering.
+
+By default this plugin will set ``serverHost`` for all the events in a batch to match hostname of
+the logstash node where the output plugin is running.
+
+You can change that either by setting ``serverHost`` attribute in the ``server_attributes`` config
+option hash or by setting ``serverHost`` attribute on the event level via logstash record attribute.
+
+In both scenarios, you will be able to utilize this value for "Sources" functionality and filterin
+in the Scalyr UI.
+
+For example:
+
+1. Define static value for all the events handled by specific plugin instance
+
+```
+output {
+ scalyr {
+   api_write_token => 'SCALYR_API_KEY'
+   server_attributes => {'serverHost' => 'my-host-1'}
+ }
+}
+```
+
+2. Define static value on the event level which is set via logstash filter
+
+```
+  mutate {
+    add_field => { "serverHost" => "my hostname" }
+  }
+```
+
+3. Define dynamic value on the event level which is set via logstash filter
+
+```
+  mutate {
+    add_field => { "serverHost" => "%{[host][name]}" }
+  }
+```
+
 ## Options
 
 - The Scalyr API write token, these are available at https://www.scalyr.com/keys.  This is the only compulsory configuration field required for proper upload
@@ -72,7 +118,7 @@ In the above example, the Logstash pipeline defines a file input that reads from
 - Related to the server_attributes dictionary above, if you do not define the 'serverHost' key in server_attributes,
  the plugin will automatically set it, using the aggregator hostname as value, if this value is true.
  
-`config :use_hostname_for_serverhost, :validate => :boolean, :default => false`
+`config :use_hostname_for_serverhost, :validate => :boolean, :default => true`
 
 ---
 
@@ -395,3 +441,65 @@ bundle exec rake publish_gem
 
 `RUBY_USER` and `RUBY_PASSWORD` should be replaced with the username and password to the RubyGems.org account you wish to release to,
  these credentials should be found in Keeper.
+
+# Testing Plugin Changes
+
+This section describes how to test the plugin and changes using docker compose setup from
+lostash-config-tester repo available at https://github.com/Kami/logstash-config-tester.
+
+This repo has been forked and already contains some changes which make testing the plugin
+easier.
+
+The logstash configuration in that repo is set up to receive records encoded as JSON via
+standard input, decode the JSON into the event object and print it to stdout + send it
+to the Scalyr output.
+
+```bash
+# 0. Clone the tester repo
+git clone https://github.com/Kami/logstash-config-tester ~/
+
+# 1. Build the plugin
+gem build logstash-output-scalyr.gemspec
+
+# 2. Copy it to the config test repo
+cp logstash-output-scalyr-0.2.0.gem ~/logstash-config-test/logstash-output-scalyr.gem
+
+# 3. Build docker image with the latest dev version of the plugin (may take a while)
+docker-compose build
+
+# 4. Configure API key in docker-compose.yml and make any changes to plugin config in
+# pipeline/scalyr_output.conf.j2, if necessary
+vim docker-compose.yml
+
+vim pipeline/scalyr_output.conf.j2
+
+# 4. Run logstash with the stdin input and stdout + scalyr output
+docker-compose run logstash
+```
+
+A couple of things to keep in mind:
+
+1. Logstash log level is set to debug which is quite verbose, but it makes troubleshooting and
+  testing easier.
+2. Plugin accepts records (events) as JSON via standard input. This means to inject the mock
+   event you can simply copy the JSON event representation string to stdin and press enter. If you
+   want to submit multiples events to be handled as a single batch, paste each event one at a time
+   and press enter at the end. Logstash pipeline has been configured to wait up to 5 seconds before
+   handling the batch which should give you enough time to test batches with multiple events (this
+   setting can be adjusted in ``config/logstash.yml`` - ``pipeline.batch.delay``, if needed)
+
+Example values you can enter into stdin:
+
+1. Single event batch
+
+```javascript
+{"foo": "bar", "message": "test logstash"}
+```
+
+2. Batch with 3 events
+
+```javascript
+{"serverHost": "host-1", "bar": "baz", "message": "test logstash 1"}
+{"serverHost": "host-2", "bar": "baz", "message": "test logstash 2"}
+{"bar": "baz", "message": "test logstash 3"}
+```

data/lib/logstash/outputs/scalyr.rb

@@ -13,6 +13,7 @@ require 'rbzip2'
 require 'zlib'
 require 'stringio'
 require 'quantile'
+require 'jrjackson'
 
 require 'scalyr/common/client'
 require "scalyr/common/util"
@@ -42,7 +43,7 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
 
   # Related to the server_attributes dictionary above, if you do not define the 'serverHost' key in server_attributes,
   # the plugin will automatically set it, using the aggregator hostname as value, if this value is true.
-  config :use_hostname_for_serverhost, :validate => :boolean, :default => false
+  config :use_hostname_for_serverhost, :validate => :boolean, :default => true
 
   # Field that represents the origin of the log event.
   # (Warning: events with an existing 'serverHost' field, it will be overwritten)
@@ -65,6 +66,23 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
   # value with have fields moved.
   config :log_constants, :validate => :array, :default => nil
 
+  # When this option is true and session level server host is defined (either via
+  # server_attributes config option or via node hostname) and some events in a batch contain
+  # "serverHost" attributes, other nodes in a batch which don't contain it will have serverHost
+  # set to the session level value.
+  # This is needed because session level attribute has priority over event level which means
+  # that in case we specify serverHost on some events, other events won't have any value set
+  # for serverHost.
+  # Since this option adds some overhead and requires additional processing time, it's
+  # disabled by default.
+  config :set_session_level_serverhost_on_events, validate: :boolean, default: false
+
+  # By default, logstash will add "host" attribute which includes logstash aggregator server
+  # host to each event. This is not really needed and desired anymore with the fixed and improved
+  # serverHost attribute handling (serverHost now contains logstash aggregator hostname by
+  # default).
+  config :remove_host_attribute_from_events, validate: :boolean, default: true
+
   # If true, nested values will be flattened (which changes keys to delimiter-separated concatenation of all
   # nested keys).
   config :flatten_nested_values, :validate => :boolean, :default => false
@@ -151,6 +169,10 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
   # Scalyr single request limit won't be reached.
   config :estimate_each_event_size, :validate => :boolean, :default => true
 
+  # Library to use for JSON serialization. Valid values are "stdlib" and "jrjackson". The later may offer 2-4 performance
+  # improvements on serialization.
+  config :json_library, :validate => :string, :default => "stdlib"
+
   # Manticore related options
   config :http_connect_timeout, :validate => :number, :default => 10
   config :http_socket_timeout, :validate => :number, :default => 10
@@ -190,6 +212,24 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
       @logger.warn "Maximum request buffer > 6 MB.  This may result in requests being rejected by Scalyr."
     end
 
+    if not @estimate_each_event_size
+      @logger.warn("estimate_each_event_size config option is false, this means very large batches may be rejected or partially processed by the server")
+    end
+
+    if @json_library != "stdlib" and @json_library != "jrjackson"
+      raise LogStash::ConfigurationError, "json_library config option needs to be either stdlib or jrjackson"
+    end
+
+    if @json_library == "stdlib"
+      define_singleton_method "json_encode" do |data|
+        data.to_json
+      end
+    elsif @json_library == "jrjackson"
+      define_singleton_method "json_encode" do |data|
+        JrJackson::Json.dump(data)
+      end
+    end
+
     @dlq_writer = dlq_enabled? ? execution_context.dlq_writer : nil
 
     @message_encoding = nil
@@ -218,12 +258,13 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
       @server_attributes = new_attributes
     end
 
-    # See if we should use the hostname as the server_attributes.serverHost
-    if @use_hostname_for_serverhost
-      if @server_attributes.nil?
+    # See if we should use the hostname as the server_attributes.serverHost (aka if fixed serverHost is not
+    # defined as part of server_attributes config option)
+    if @server_attributes.nil?
         @server_attributes = {}
-      end
+    end
 
+    if @use_hostname_for_serverhost
       # only set serverHost if it doesn't currently exist in server_attributes
       # Note: Use strings rather than symbols for the key, because keys coming
       # from the config file will be strings
@@ -235,6 +276,15 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
     # Add monitor server attribute to identify this as coming from a plugin
     @server_attributes['monitor'] = 'pluginLogstash'
 
+    # We create a fixed copy without host here so we can reference this later if needed to avoid
+    # some of the copy + manipulate overhead per batch
+    @server_attributes_without_serverhost = @server_attributes.clone
+    if @server_attributes_without_serverhost.key? "serverHost"
+      @server_attributes_without_serverhost.delete "serverHost"
+    end
+
+    @session_server_host = @server_attributes["serverHost"]
+
     @scalyr_server << '/' unless @scalyr_server.end_with?('/')
 
     @add_events_uri = URI(@scalyr_server) + "addEvents"
@@ -267,7 +317,8 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
         @http_connect_timeout, @http_socket_timeout, @http_request_timeout, @http_pool_max, @http_pool_max_per_route
     )
 
-    @logger.info(sprintf("Started Scalyr output plugin (%s)." % [PLUGIN_VERSION]), :class => self.class.name)
+    @logger.info(sprintf("Started Scalyr LogStash output plugin %s (compression_type=%s,compression_level=%s,json_library=%s)." %
+                         [PLUGIN_VERSION, @compression_type, @compression_type, @json_library]), :class => self.class.name)
 
     # Finally, send a status line to Scalyr
     # We use a special separate short lived client session for sending the initial client status.
@@ -482,7 +533,7 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
         @dlq_writer.write(l_event, "#{exc_data[:message]}")
       }
     else
-      @logger.warn("Deal letter queue not configured, dropping #{multi_event_request[:logstash_events].length} events after #{exc_retries} tries.", :sample_events => sample_events)
+      @logger.warn("Dead letter queue not configured, dropping #{multi_event_request[:logstash_events].length} events after #{exc_retries} tries.", :sample_events => sample_events)
     end
   end
 
@@ -526,6 +577,8 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
     logs_ids = Hash.new
     next_log_id = 1
 
+    batch_has_event_level_server_host = false
+
     logstash_events.each {|l_event|
 
       record = l_event.to_hash
@@ -581,6 +634,11 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
       # Rename user-specified logfile field -> 'logfile'
       rename.call(@logfile_field, 'logfile')
 
+      # Remove "host" attribute
+      if @remove_host_attribute_from_events and record.key? "host"
+        record.delete("host")
+      end
+
       # Set a default parser is none is present in the event
       if record['parser'].to_s.empty?
         record['parser'] = "logstashParser"
@@ -591,26 +649,31 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
         record['logfile'] = "/logstash/#{serverHost}"
       end
 
-      # Set a default if no serverHost value is present.
-      if serverHost.nil?
-        record['serverHost'] = "Logstash"
+      # Rename serverHost (if exists) to __origServerHost so sources filtering works correctly
+      # It's important that this happens at the very end of the event processing in this function.
+      record_has_server_host_attribute = record.key? 'serverHost'
+      batch_has_event_level_server_host |= record_has_server_host_attribute
+
+      if record_has_server_host_attribute
+        record[EVENT_LEVEL_SERVER_HOST_ATTRIBUTE_NAME] = record['serverHost']
+        record.delete('serverHost')
       end
 
+      # To reduce duplication of common event-level attributes, we "fold" them into top-level "logs" attribute
+      # and reference log entry inside the event
       log_identifier = nil
       add_log = false
       if serverHost
        log_identifier = serverHost + record['logfile']
       end
+
       if log_identifier and not logs.key? log_identifier
         add_log = true
         logs[log_identifier] = {
           'id' => next_log_id,
           'attrs' => Hash.new
         }
-        if not record['serverHost'].to_s.empty?
-          logs[log_identifier]['attrs']['serverHost'] = record['serverHost']
-          record.delete('serverHost')
-        end
+
         if not record['logfile'].to_s.empty?
           logs[log_identifier]['attrs']['logfile'] = record['logfile']
           record.delete('logfile')
@@ -623,10 +686,21 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
             end
           }
         end
+
         logs_ids[log_identifier] = next_log_id
         next_log_id += 1
       end
 
+      # If we already contain "logs" entry for this record, we remove duplicated serverHost from
+      # the event attributes since it's already part of the log level attributes which are
+      # referenced by the event.
+      if log_identifier and logs.key? log_identifier
+        if not record[EVENT_LEVEL_SERVER_HOST_ATTRIBUTE_NAME].to_s.empty?
+          logs[log_identifier]['attrs'][EVENT_LEVEL_SERVER_HOST_ATTRIBUTE_NAME] = record[EVENT_LEVEL_SERVER_HOST_ATTRIBUTE_NAME]
+          record.delete(EVENT_LEVEL_SERVER_HOST_ATTRIBUTE_NAME)
+        end
+      end
+
       # Delete unwanted fields from record
       record.delete('@version')
       record.delete('@timestamp')
@@ -672,7 +746,7 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
         :attrs => record
       }
 
-      # optionally set thread
+      # optionally set thread and referenced log file
       if serverHost
         scalyr_event[:thread] = thread_id.to_s
         scalyr_event[:log] = logs_ids[log_identifier]
@@ -681,10 +755,10 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
       if @estimate_each_event_size
         # get json string of event to keep track of how many bytes we are sending
         begin
-          event_json = scalyr_event.to_json
+          event_json = self.json_encode(scalyr_event)
           log_json = nil
           if add_log
-            log_json = logs[log_identifier].to_json
+            log_json = self.json_encode(logs[log_identifier])
           end
         rescue JSON::GeneratorError, Encoding::UndefinedConversionError => e
           @logger.warn "#{e.class}: #{e.message}"
@@ -700,7 +774,7 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
                 "UTF-8", :invalid => :replace, :undef => :replace, :replace => "<?>"
             ).force_encoding('UTF-8')
           end
-          event_json = scalyr_event.to_json
+          event_json = self.json_encode(scalyr_event)
         rescue Java::JavaLang::ClassCastException => e
           # Most likely we ran into the issue described here: https://github.com/flori/json/issues/336
           # Because of the version of jruby logstash works with we don't have the option to just update this away,
@@ -711,10 +785,10 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
             @multi_receive_statistics[:total_java_class_cast_errors] += 1
           end
           Scalyr::Common::Util.convert_bignums(scalyr_event)
-          event_json = scalyr_event.to_json
+          event_json = self.json_encode(scalyr_event)
           log_json = nil
           if add_log
-            log_json = logs[log_identifier].to_json
+            log_json = self.json_encode(logs[log_identifier])
           end
         end
 
@@ -733,7 +807,8 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
             append_event = false
           end
 
-          multi_event_request = self.create_multi_event_request(scalyr_events, l_events, current_threads, logs)
+          Scalyr::Common::Util.set_session_level_serverhost_on_events(@session_server_host, scalyr_events, logs, batch_has_event_level_server_host)
+          multi_event_request = self.create_multi_event_request(scalyr_events, l_events, current_threads, logs, batch_has_event_level_server_host)
           multi_event_request_array << multi_event_request
 
           total_bytes = 0
@@ -742,6 +817,7 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
           logs_ids = Hash.new
           scalyr_events = Array.new
           l_events = Array.new
+          batch_has_event_level_server_host = false
         end
       else
         # If size estimation is disabled we simply append the event and handle splitting later on (if needed)
@@ -761,14 +837,14 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
 
     # create a final request with any left over events (and make sure there is at least one event)
     if scalyr_events.size >= 1
-      multi_event_request = self.create_multi_event_request(scalyr_events, l_events, current_threads, logs)
+      Scalyr::Common::Util.set_session_level_serverhost_on_events(@session_server_host, scalyr_events, logs, batch_has_event_level_server_host)
+      multi_event_request = self.create_multi_event_request(scalyr_events, l_events, current_threads, logs, batch_has_event_level_server_host)
       multi_event_request_array << multi_event_request
     end
 
     multi_event_request_array
   end
 
-
   # Helper method that adds a client_timestamp to a batch addEvents request body
   def add_client_timestamp_to_body(body)
     current_time_millis = DateTime.now.strftime('%Q').to_i
@@ -781,7 +857,7 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
   # A request comprises multiple Scalyr Events.  This function creates a request hash for
   # final upload to Scalyr (from an array of events, and an optional hash of current threads)
   # Note: The request body field will be json-encoded.
-  def create_multi_event_request(scalyr_events, logstash_events, current_threads, current_logs)
+  def create_multi_event_request(scalyr_events, logstash_events, current_threads, current_logs, batch_has_event_level_server_host = false)
 
     body = {
       :session => @session_id + Thread.current.object_id.to_s,
@@ -810,26 +886,33 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
     end
 
     # add serverAttributes
-    body[:sessionInfo] = @server_attributes if @server_attributes
+    # If serverHost is defined on any of the events, we don't send it via sessionInfo since
+    # sesionInfo has the higest priority and would always overwritte the event level one
+    if batch_has_event_level_server_host
+      body[:sessionInfo] = @server_attributes_without_serverhost if @server_attributes_without_serverhost
+    else
+      body[:sessionInfo] = @server_attributes if @server_attributes
+    end
 
     # We time serialization to get some insight on how long it takes to serialize the request body
     start_time = Time.now.to_f
     begin
-      serialized_body = body.to_json
+      serialized_body = self.json_encode(body)
     rescue Java::JavaLang::ClassCastException => e
       @logger.warn("Error serializing events to JSON, likely due to the presence of Bignum values. Converting Bignum values to strings.")
       @stats_lock.synchronize do
         @multi_receive_statistics[:total_java_class_cast_errors] += 1
       end
       Scalyr::Common::Util.convert_bignums(body)
-      serialized_body = body.to_json
+      serialized_body = self.json_encode(body)
     end
     end_time = Time.now.to_f
     serialization_duration = end_time - start_time
 
     serialized_request_size = serialized_body.bytesize
 
-    if serialized_request_size >= @max_request_buffer
+    # We give it "buffer" since the splitting code allows for some slack and doesn't take into account top-level non-event attributes
+    if not @estimate_each_event_size and serialized_request_size >= @max_request_buffer + 5000
       # TODO: If we end up here is estimate config opsion is false, split the request here into multiple ones
       @logger.warn("Serialized request size (#{serialized_request_size}) is larger than max_request_buffer (#{max_request_buffer})!")
     end
@@ -902,8 +985,9 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
     }
     @send_stats.synchronize do
       if !@last_status_transmit_time
-        status_event[:attrs]['message'] = sprintf("Started Scalyr LogStash output plugin (%s)." % [PLUGIN_VERSION])
-        status_event[:attrs]['serverHost'] = @node_hostname
+        status_event[:attrs]['message'] = sprintf("Started Scalyr LogStash output plugin %s (compression_type=%s,compression_level=%s,json_library=%s)." %
+                                                  [PLUGIN_VERSION, @compression_type, @compression_type, @json_library])
+        status_event[:attrs][EVENT_LEVEL_SERVER_HOST_ATTRIBUTE_NAME] = @node_hostname
       else
         cur_time = Time.now()
         return if (cur_time.to_i - @last_status_transmit_time.to_i) < @status_report_interval
@@ -925,7 +1009,7 @@ class LogStash::Outputs::Scalyr < LogStash::Outputs::Base
           cnt += 1
         end
         status_event[:attrs]['message'] = msg
-        status_event[:attrs]['serverHost'] = @node_hostname
+        status_event[:attrs][EVENT_LEVEL_SERVER_HOST_ATTRIBUTE_NAME] = @node_hostname
         status_event[:attrs]['parser'] = @status_parser
       end
       multi_event_request = create_multi_event_request([status_event], nil, nil, nil)