logstash-output-seq 0.0.3 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 11cd6474d4c4171afc1c0f9e82db56065a8c4a35
-  data.tar.gz: ffd056dbb0f3a4b43963031537eb42eb816857e1
+  metadata.gz: 77fb12736b1e9f994bb397f7c928d69b05e09722
+  data.tar.gz: 83cd2c244614defb64b18d5eb31e52165cd902b5
 SHA512:
-  metadata.gz: 54b045ff64e0c4a620d0b0452ed6a6ea982c320f5cf2904183c11292f9b31aa7cd05ad78e015e2b8ddaa07e86f8fc8840f1f366a1f39bd18c1ae2424f07dab10
-  data.tar.gz: e1d5933a4cb68302e3301771538a603533b03c446e28d6148bf65746584681fbddf1374373ad5206df9844ab5916865e60cf47ba2736cf55554bd40d47bd91d5
+  metadata.gz: a0749fc14cdfdb6b5727fef69e74d52d718ba08b0aa76a66f75f40bc173ca18adad9772938332486415c3eaea4309aa6f50f2ecfb6f85ff594321b525b711baf
+  data.tar.gz: dab7f60b4dcff4ccd36491140009ffaf8b30c2bf660d1a300171f74133903567608b9a38bca82b932c9d5415412c5fc1d05d2512d5bed0a92d3c1fe9ca9e730e

data/CHANGELOG.md

@@ -1,7 +1,7 @@
-## 2.0.1
- - Add encoding: utf-8 to spec files. This can help prevent issues during testing.
-## 2.0.0
- - Plugins were updated to follow the new shutdown semantic, this mainly allows Logstash to instruct input plugins to terminate gracefully, 
-   instead of using Thread.raise on the plugins' threads. Ref: https://github.com/elastic/logstash/pull/3895
- - Dependency on logstash-core update to 2.0
-
+## 2.0.1
+ - Add encoding: utf-8 to spec files. This can help prevent issues during testing.
+## 2.0.0
+ - Plugins were updated to follow the new shutdown semantic, this mainly allows Logstash to instruct input plugins to terminate gracefully, 
+   instead of using Thread.raise on the plugins' threads. Ref: https://github.com/elastic/logstash/pull/3895
+ - Dependency on logstash-core update to 2.0
+

data/CONTRIBUTORS

@@ -1,10 +1,11 @@
-The following is a list of people who have contributed ideas, code, bug
-reports, or in general have helped the Seq plugin for Logstash along its way.
-
-Contributors:
-* Adam Friedman (tintoy)
-
-Note: If you've sent us patches, bug reports, or otherwise contributed to
-Logstash, and you aren't on the list above and want to be, please let us know
-and we'll make sure you're here. Contributions from folks like you are what make
-open source awesome.
+The following is a list of people who have contributed ideas, code, bug
+reports, or in general have helped the Seq plugin for Logstash along its way.
+
+Contributors:
+* Adam Friedman ([@tintoy](https://github.com/tintoy))
+* [@mjnorman](https://github.com/mjnorman)
+
+Note: If you've sent us patches, bug reports, or otherwise contributed to
+Logstash, and you aren't on the list above and want to be, please let us know
+and we'll make sure you're here. Contributions from folks like you are what make
+open source awesome.

data/DEVELOPER.md

@@ -1,2 +1,2 @@
-# logstash-output-seq
-Seq output plugin.
+# logstash-output-seq
+Seq output plugin.

data/DEVELOPING.md

@@ -1,69 +1,69 @@
-## Developing
-
-### 1. Plugin Developement and Testing
-
-#### Code
-- To get started, you'll need JRuby with the Bundler gem installed.
-
-- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
-
-- Install dependencies
-```sh
-bundle install
-```
-
-#### Test
-
-- Update your dependencies
-
-```sh
-bundle install
-```
-
-- Run tests
-
-```sh
-bundle exec rspec
-```
-
-### 2. Running your unpublished Plugin in Logstash
-
-#### 2.1 Run in a local Logstash clone
-
-- Edit Logstash `Gemfile` and add the local plugin path, for example:
-```ruby
-gem "logstash-output-seq", :path => "/your/local/logstash-output-seq"
-```
-- Install plugin
-```sh
-# Logstash 2.3 and higher
-bin/logstash-plugin install --no-verify
-
-# Prior to Logstash 2.3
-bin/plugin install --no-verify
-
-```
-- Run Logstash with your plugin
-```sh
-bin/logstash -e 'output {seq {}}'
-```
-At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
-
-#### 2.2 Run in an installed Logstash
-
-You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
-
-- Build your plugin gem
-```sh
-gem build logstash-output-seq.gemspec
-```
-- Install the plugin from the Logstash home
-```sh
-# Logstash 2.3 and higher
-bin/logstash-plugin install --no-verify
-
-# Prior to Logstash 2.3
-bin/plugin install --no-verify
-
-```
-- Start Logstash and proceed to test the plugin
+## Developing
+
+### 1. Plugin Developement and Testing
+
+#### Code
+- To get started, you'll need JRuby with the Bundler gem installed.
+
+- Create a new plugin or clone and existing from the GitHub [logstash-plugins](https://github.com/logstash-plugins) organization. We also provide [example plugins](https://github.com/logstash-plugins?query=example).
+
+- Install dependencies
+```sh
+bundle install
+```
+
+#### Test
+
+- Update your dependencies
+
+```sh
+bundle install
+```
+
+- Run tests
+
+```sh
+bundle exec rspec
+```
+
+### 2. Running your unpublished Plugin in Logstash
+
+#### 2.1 Run in a local Logstash clone
+
+- Edit Logstash `Gemfile` and add the local plugin path, for example:
+```ruby
+gem "logstash-output-seq", :path => "/your/local/logstash-output-seq"
+```
+- Install plugin
+```sh
+# Logstash 2.3 and higher
+bin/logstash-plugin install --no-verify
+
+# Prior to Logstash 2.3
+bin/plugin install --no-verify
+
+```
+- Run Logstash with your plugin
+```sh
+bin/logstash -e 'output {seq {}}'
+```
+At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.
+
+#### 2.2 Run in an installed Logstash
+
+You can use the same **2.1** method to run your plugin in an installed Logstash by editing its `Gemfile` and pointing the `:path` to your local plugin development directory or you can build the gem and install it using:
+
+- Build your plugin gem
+```sh
+gem build logstash-output-seq.gemspec
+```
+- Install the plugin from the Logstash home
+```sh
+# Logstash 2.3 and higher
+bin/logstash-plugin install --no-verify
+
+# Prior to Logstash 2.3
+bin/plugin install --no-verify
+
+```
+- Start Logstash and proceed to test the plugin

data/Gemfile

@@ -1,2 +1,11 @@
-source 'http://rubygems.org'
-gemspec
+source 'https://rubygems.org'
+
+gemspec
+
+logstash_path = ENV["LOGSTASH_PATH"] || "../../logstash"
+use_logstash_source = ENV["LOGSTASH_SOURCE"] && ENV["LOGSTASH_SOURCE"].to_s == "1"
+
+if Dir.exist?(logstash_path) && use_logstash_source
+  gem 'logstash-core', :path => "#{logstash_path}/logstash-core"
+  gem 'logstash-core-plugin-api', :path => "#{logstash_path}/logstash-core-plugin-api"
+end

data/LICENSE

@@ -1,22 +1,22 @@
-The MIT License (MIT)
-
-Copyright (c) 2015 tin oscillator
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
+The MIT License (MIT)
+
+Copyright (c) 2015 tin oscillator
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

data/README.md

@@ -1,38 +1,38 @@
-# Seq output plugin for Logstash
-
-[![Travis Build Status](https://travis-ci.org/tintoy/logstash-output-seq.svg)](https://travis-ci.org/tintoy/logstash-output-seq)
-[![Gem Version](https://badge.fury.io/rb/logstash-output-seq.svg)](https://badge.fury.io/rb/logstash-output-seq)
-[![Coverage Status (master)](https://coveralls.io/repos/github/tintoy/logstash-output-seq/badge.svg?branch=master)](https://coveralls.io/github/tintoy/logstash-output-seq?branch=master)
-
-This is an output plugin for [Logstash](https://github.com/elastic/logstash) that publishes events to [Seq](https://getseq.net/).
-
-It is fully free and fully open source. The license is MIT, meaning you are pretty much free to use it however you want in whatever way.
-
-## Usage
-
-To install the Seq output plugin for Logstash:
-
-```sh
-# Logstash 2.3 and higher
-${LOGSTASH_HOME}/bin/logstash-plugin install logstash-output-seq
-```
-
-The plugin has the following configuration options:
-
-* `url` (Required) - The Seq server URL (e.g. `http://localhost:5341/`)
-* `api_key` (Optional) - The Seq API key (if any) to use for authentication.
-* Any parameters from the Logstash HttpClient mix-in (e.g. configuring SSL behaviour, etc).
-
-### Docker
-
-There is also a [docker image](https://hub.docker.com/r/tintoy/logstash-output-seq/) available that has this plugin preinstalled.
-
-## Developing
-
-[Developing](DEVELOPING.md)
-
-## Contributing
-
-All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
-
+# Seq output plugin for Logstash
+
+[![Travis Build Status](https://travis-ci.org/tintoy/logstash-output-seq.svg)](https://travis-ci.org/tintoy/logstash-output-seq)
+[![Gem Version](https://badge.fury.io/rb/logstash-output-seq.svg)](https://badge.fury.io/rb/logstash-output-seq)
+[![Coverage Status (master)](https://coveralls.io/repos/github/tintoy/logstash-output-seq/badge.svg?branch=master)](https://coveralls.io/github/tintoy/logstash-output-seq?branch=master)
+
+This is an output plugin for [Logstash](https://github.com/elastic/logstash) that publishes events to [Seq](https://getseq.net/).
+
+It is fully free and fully open source. The license is MIT, meaning you are pretty much free to use it however you want in whatever way.
+
+## Usage
+
+To install the Seq output plugin for Logstash:
+
+```sh
+# Logstash 2.3 and higher
+${LOGSTASH_HOME}/bin/logstash-plugin install logstash-output-seq
+```
+
+The plugin has the following configuration options:
+
+* `url` (Required) - The Seq server URL (e.g. `http://localhost:5341/`)
+* `api_key` (Optional) - The Seq API key (if any) to use for authentication.
+* Any parameters from the Logstash HttpClient mix-in (e.g. configuring SSL behaviour, etc).
+
+### Docker
+
+There is also a [docker image](https://hub.docker.com/r/tintoy/logstash-output-seq/) available that has this plugin preinstalled.
+
+## Developing
+
+[Developing](DEVELOPING.md)
+
+## Contributing
+
+All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.
+
 For more information about contributing, see the [CONTRIBUTING](https://github.com/tintoy/logstash-output-seq/blob/master/.github/CONTRIBUTING.md) file.

data/lib/logstash/outputs/seq.rb

@@ -1,145 +1,145 @@
-# encoding: utf-8
-require 'logstash/outputs/base'
-require 'logstash/plugin_mixins/http_client'
-require 'logstash/json'
-require 'logstash/namespace'
-
-require 'jrjackson'
-
-# An output that sends events to Seq.
-class LogStash::Outputs::Seq < LogStash::Outputs::Base
-  include LogStash::PluginMixins::HttpClient
-
-  @@ignore_properties = {
-      '@timestamp' => true,
-      '@level' => true,
-      '@version' => true,
-      'message' => true
-  }
-  
-  config_name 'seq'
-
-  # The Seq server URL
-  config :url, :validate => :string, :required => :true
-
-  # The Seq API key
-  config :api_key, :validate => :string, :optional => :true
-
-  public
-  def register
-    url = @url
-
-    url += '/' unless url.end_with? '/'
-    url += 'api/events/raw'
-
-    @url = url
-
-    @default_headers = {
-        'X-Seq-ApiKey' => @api_key,
-        'Content-Type' => 'application/json'
-    }
-
-    # We count outstanding requests with this queue; it tracks outstanding requests to create backpressure
-    # When this queue is empty no new requests may be sent, tokens must be added back by the client on success
-    @request_tokens = SizedQueue.new(@pool_max)
-    @pool_max.times {|t| @request_tokens << true }
-  end # def register
-
-  public
-  def multi_receive(events)
-    payload = {
-        'Events' => events.map {|event| to_seq_payload(event)}
-    }
-    post_to_seq(payload)
-
-    "#{events.length} events received"
-  end # def multi_receive
-
-  public
-  def receive(event)
-    payload = {
-        'Events' => [to_seq_payload(event)]
-    }
-    post_to_seq(payload)
-
-    'Event received'
-  end # def receive
-
-  private
-  def post_to_seq(payload)
-    token = @request_tokens.pop
-
-    request = client.post(@url, {
-      headers: @default_headers,
-      body:    JrJackson::Json.dump(payload),
-      async:   true
-    })
-
-    request.on_complete do
-      @request_tokens << token
-    end
-
-    request.on_success do |response|
-      if response.code < 200 || response.code > 299
-        log_failure("Encountered non-200 HTTP code #{200}",
-          :response_code => response.code,
-          :url => url,
-          :event => event
-        )
-      end
-    end
-
-    request.on_failure do |exception|
-      log_failure("Could not submit POST request.",
-        :url => url,
-        :method => @http_method,
-        :body => body,
-        :headers => headers,
-        :message => exception.message,
-        :class => exception.class.name,
-        :backtrace => exception.backtrace
-      )
-    end
-
-    request_async_background(request)
-  end # def post_to_seq
-
-  # Convert a Logstash event to a Seq event payload.
-  #
-  # Note that we return a hash here, not the JSON, because it's more efficient to convert batched events to JSON all-in-one go.
-  private
-  def to_seq_payload(event)
-    props = {
-        '@Version' => event['@version']
-    }
-    payload = {
-        :Timestamp => event['@timestamp'],
-        :Level => get_level(event),
-        :MessageTemplate => event['message'],
-        :Properties => props
-    }
-
-    event.instance_variable_get(:@data).each do |property, value|
-      props[property] = value unless @@ignore_properties.has_key? property
-    end
-
-    payload
-  end # def to_seq_payload
-
-  private
-  def get_level(event)
-    level = event['@level']
-
-    level ? level : 'Verbose'
-  end # def get_level
-
-  # Manticore doesn't provide a way to attach handlers to background or async requests well
-  # It wants you to use futures. The #async method kinda works but expects single thread batches
-  # and background only returns futures.
-  # Proposed fix to manticore here: https://github.com/cheald/manticore/issues/32
-  private
-  def request_async_background(request)
-    @method ||= client.executor.java_method(:submit, [java.util.concurrent.Callable.java_class])
-    @method.call(request)
-  end
-end # class LogStash::Outputs::Seq
+# encoding: utf-8
+require 'logstash/outputs/base'
+require 'logstash/plugin_mixins/http_client'
+require 'logstash/json'
+require 'logstash/namespace'
+
+require 'jrjackson'
+
+# An output that sends events to Seq.
+class LogStash::Outputs::Seq < LogStash::Outputs::Base
+  include LogStash::PluginMixins::HttpClient
+
+  @@ignore_properties = {
+      '@timestamp' => true,
+      '@level' => true,
+      '@version' => true,
+      'message' => true
+  }
+  
+  config_name 'seq'
+
+  # The Seq server URL
+  config :url, :validate => :string, :required => :true
+
+  # The Seq API key
+  config :api_key, :validate => :string, :optional => :true
+
+  public
+  def register
+    url = @url
+
+    url += '/' unless url.end_with? '/'
+    url += 'api/events/raw'
+
+    @url = url
+
+    @default_headers = {
+        'X-Seq-ApiKey' => @api_key,
+        'Content-Type' => 'application/json'
+    }
+
+    # We count outstanding requests with this queue; it tracks outstanding requests to create backpressure
+    # When this queue is empty no new requests may be sent, tokens must be added back by the client on success
+    @request_tokens = SizedQueue.new(@pool_max)
+    @pool_max.times {|t| @request_tokens << true }
+  end # def register
+
+  public
+  def multi_receive(events)
+    payload = {
+        'Events' => events.map {|event| to_seq_payload(event)}
+    }
+    post_to_seq(payload)
+
+    "#{events.length} events received"
+  end # def multi_receive
+
+  public
+  def receive(event)
+    payload = {
+        'Events' => [to_seq_payload(event)]
+    }
+    post_to_seq(payload)
+
+    'Event received'
+  end # def receive
+
+  private
+  def post_to_seq(payload)
+    token = @request_tokens.pop
+
+    request = client.post(@url, {
+      headers: @default_headers,
+      body:    JrJackson::Json.dump(payload),
+      async:   true
+    })
+
+    request.on_complete do
+      @request_tokens << token
+    end
+
+    request.on_success do |response|
+      if response.code < 200 || response.code > 299
+        log_failure("Encountered non-200 HTTP code #{200}",
+          :response_code => response.code,
+          :url => url,
+          :event => event
+        )
+      end
+    end
+
+    request.on_failure do |exception|
+      log_failure("Could not submit POST request.",
+        :url => url,
+        :method => @http_method,
+        :body => body,
+        :headers => headers,
+        :message => exception.message,
+        :class => exception.class.name,
+        :backtrace => exception.backtrace
+      )
+    end
+
+    request_async_background(request)
+  end # def post_to_seq
+
+  # Convert a Logstash event to a Seq event payload.
+  #
+  # Note that we return a hash here, not the JSON, because it's more efficient to convert batched events to JSON all-in-one go.
+  private
+  def to_seq_payload(event)
+    props = {
+        '@Version' => event.get('@version')
+    }
+    payload = {
+        :Timestamp => event.get('@timestamp'),
+        :Level => get_level(event),
+        :MessageTemplate => event.get('message'),
+        :Properties => props
+    }
+
+    event.to_hash.each do |property, value|
+      props[property] = value unless @@ignore_properties.has_key? property
+    end
+
+    payload
+  end # def to_seq_payload
+
+  private
+  def get_level(event)
+    level = event.get('@level')
+
+    level ? level : 'Verbose'
+  end # def get_level
+
+  # Manticore doesn't provide a way to attach handlers to background or async requests well
+  # It wants you to use futures. The #async method kinda works but expects single thread batches
+  # and background only returns futures.
+  # Proposed fix to manticore here: https://github.com/cheald/manticore/issues/32
+  private
+  def request_async_background(request)
+    @method ||= client.executor.java_method(:submit, [java.util.concurrent.Callable.java_class])
+    @method.call(request)
+  end
+end # class LogStash::Outputs::Seq