logstash-input-beats 2.0.1 → 2.0.2

data/vendor/jruby/1.9/gems/logstash-codec-json-2.0.3/vendor/gems/stud-0.0.22/LICENSE

@@ -0,0 +1,14 @@
+Copyright 2012-2013 Jordan Sissel and contributors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+

data/vendor/jruby/1.9/gems/logstash-codec-json-2.0.3/vendor/gems/stud-0.0.22/README.md

@@ -0,0 +1,27 @@
+# Stud.
+
+Ruby's stdlib is missing many things I use to solve most of my software
+problems. Things like like retrying on a failure, supervising workers, resource
+pools, etc.
+
+In general, I started exploring solutions to these things in code over in my
+[software-patterns](https://github.com/jordansissel/software-patterns) repo.
+This library (stud) aims to be a well-tested, production-quality implementation
+of the patterns described in that repo.
+
+For now, these all exist in a single repo because, so far, implementations of
+each 'pattern' are quite small by code size.
+
+## Features
+
+* {Stud::Try} (and {Stud.try}) - retry on failure, with back-off, where failure is any exception.
+* {Stud::Pool} - generic resource pools
+* {Stud::Task} - tasks (threads that can return values, exceptions, etc)
+* {Stud.interval} - interval execution (do X every N seconds)
+* {Stud::Buffer} - batch & flush behavior.
+
+## TODO:
+
+* Make sure all things are documented. rubydoc.info should be able to clearly
+  show folks how to use features of this library.
+* Add tests to cover all supported features.

data/vendor/jruby/1.9/gems/logstash-codec-json-2.0.3/vendor/gems/stud-0.0.22/lib/stud/buffer.rb

@@ -0,0 +1,260 @@
+module Stud
+
+  # @author {Alex Dean}[http://github.com/alexdean]
+  #
+  # Implements a generic framework for accepting events which are later flushed
+  # in batches. Flushing occurs whenever +:max_items+ or +:max_interval+ (seconds)
+  # has been reached.
+  #
+  # Including class must implement +flush+, which will be called with all
+  # accumulated items either when the output buffer fills (+:max_items+) or
+  # when a fixed amount of time (+:max_interval+) passes.
+  #
+  # == batch_receive and flush
+  # General receive/flush can be implemented in one of two ways.
+  #
+  # === batch_receive(event) / flush(events)
+  # +flush+ will receive an array of events which were passed to +buffer_receive+.
+  #
+  #   batch_receive('one')
+  #   batch_receive('two')
+  #
+  # will cause a flush invocation like
+  #
+  #   flush(['one', 'two'])
+  #
+  # === batch_receive(event, group) / flush(events, group)
+  # flush() will receive an array of events, plus a grouping key.
+  #
+  #   batch_receive('one',   :server => 'a')
+  #   batch_receive('two',   :server => 'b')
+  #   batch_receive('three', :server => 'a')
+  #   batch_receive('four',  :server => 'b')
+  #
+  # will result in the following flush calls
+  #
+  #   flush(['one', 'three'], {:server => 'a'})
+  #   flush(['two', 'four'],  {:server => 'b'})
+  #
+  # Grouping keys can be anything which are valid Hash keys. (They don't have to
+  # be hashes themselves.) Strings or Fixnums work fine. Use anything which you'd
+  # like to receive in your +flush+ method to help enable different handling for
+  # various groups of events.
+  #
+  # == on_flush_error
+  # Including class may implement +on_flush_error+, which will be called with an
+  # Exception instance whenever buffer_flush encounters an error.
+  #
+  # * +buffer_flush+ will automatically re-try failed flushes, so +on_flush_error+
+  #   should not try to implement retry behavior.
+  # * Exceptions occurring within +on_flush_error+ are not handled by
+  #   +buffer_flush+.
+  #
+  # == on_full_buffer_receive
+  # Including class may implement +on_full_buffer_receive+, which will be called
+  # whenever +buffer_receive+ is called while the buffer is full.
+  #
+  # +on_full_buffer_receive+ will receive a Hash like <code>{:pending => 30,
+  # :outgoing => 20}</code> which describes the internal state of the module at
+  # the moment.
+  #
+  # == final flush
+  # Including class should call <code>buffer_flush(:final => true)</code>
+  # during a teardown/shutdown routine (after the last call to buffer_receive)
+  # to ensure that all accumulated messages are flushed.
+  module Buffer
+
+    public
+    # Initialize the buffer.
+    #
+    # Call directly from your constructor if you wish to set some non-default
+    # options. Otherwise buffer_initialize will be called automatically during the
+    # first buffer_receive call.
+    #
+    # Options:
+    # * :max_items, Max number of items to buffer before flushing. Default 50.
+    # * :max_interval, Max number of seconds to wait between flushes. Default 5.
+    # * :logger, A logger to write log messages to. No default. Optional.
+    #
+    # @param [Hash] options
+    def buffer_initialize(options={})
+      if ! self.class.method_defined?(:flush)
+        raise ArgumentError, "Any class including Stud::Buffer must define a flush() method."
+      end
+
+      @buffer_config = {
+        :max_items => options[:max_items] || 50,
+        :max_interval => options[:max_interval] || 5,
+        :logger => options[:logger] || nil,
+        :has_on_flush_error => self.class.method_defined?(:on_flush_error),
+        :has_on_full_buffer_receive => self.class.method_defined?(:on_full_buffer_receive)
+      }
+      @buffer_state = {
+        # items accepted from including class
+        :pending_items => {},
+        :pending_count => 0,
+
+        # guard access to pending_items & pending_count
+        :pending_mutex => Mutex.new,
+
+        # items which are currently being flushed
+        :outgoing_items => {},
+        :outgoing_count => 0,
+
+        # ensure only 1 flush is operating at once
+        :flush_mutex => Mutex.new,
+
+        # data for timed flushes
+        :last_flush => Time.now,
+        :timer => Thread.new do
+          loop do
+            sleep(@buffer_config[:max_interval])
+            buffer_flush(:force => true)
+          end
+        end
+      }
+
+      # events we've accumulated
+      buffer_clear_pending
+    end
+
+    # Determine if +:max_items+ has been reached.
+    #
+    # buffer_receive calls will block while <code>buffer_full? == true</code>.
+    #
+    # @return [bool] Is the buffer full?
+    def buffer_full?
+      @buffer_state[:pending_count] + @buffer_state[:outgoing_count] >= @buffer_config[:max_items]
+    end
+
+    # Save an event for later delivery
+    #
+    # Events are grouped by the (optional) group parameter you provide.
+    # Groups of events, plus the group name, are later passed to +flush+.
+    #
+    # This call will block if +:max_items+ has been reached.
+    #
+    # @see Stud::Buffer The overview has more information on grouping and flushing.
+    #
+    # @param event An item to buffer for flushing later.
+    # @param group Optional grouping key. All events with the same key will be
+    #              passed to +flush+ together, along with the grouping key itself.
+    def buffer_receive(event, group=nil)
+      buffer_initialize if ! @buffer_state
+
+      # block if we've accumulated too many events
+      while buffer_full? do
+        on_full_buffer_receive(
+          :pending => @buffer_state[:pending_count],
+          :outgoing => @buffer_state[:outgoing_count]
+        ) if @buffer_config[:has_on_full_buffer_receive]
+        sleep 0.1
+      end
+
+      @buffer_state[:pending_mutex].synchronize do
+        @buffer_state[:pending_items][group] << event
+        @buffer_state[:pending_count] += 1
+      end
+
+      buffer_flush
+    end
+
+    # Try to flush events.
+    #
+    # Returns immediately if flushing is not necessary/possible at the moment:
+    # * :max_items have not been accumulated
+    # * :max_interval seconds have not elapased since the last flush
+    # * another flush is in progress
+    #
+    # <code>buffer_flush(:force => true)</code> will cause a flush to occur even
+    # if +:max_items+ or +:max_interval+ have not been reached. A forced flush
+    # will still return immediately (without flushing) if another flush is
+    # currently in progress.
+    #
+    # <code>buffer_flush(:final => true)</code> is identical to <code>buffer_flush(:force => true)</code>,
+    # except that if another flush is already in progress, <code>buffer_flush(:final => true)</code>
+    # will block/wait for the other flush to finish before proceeding.
+    #
+    # @param [Hash] options Optional. May be <code>{:force => true}</code> or <code>{:final => true}</code>.
+    # @return [Fixnum] The number of items successfully passed to +flush+.
+    def buffer_flush(options={})
+      force = options[:force] || options[:final]
+      final = options[:final]
+
+      # final flush will wait for lock, so we are sure to flush out all buffered events
+      if options[:final]
+        @buffer_state[:flush_mutex].lock
+      elsif ! @buffer_state[:flush_mutex].try_lock # failed to get lock, another flush already in progress
+        return 0
+      end
+
+      items_flushed = 0
+
+      begin
+        time_since_last_flush = (Time.now - @buffer_state[:last_flush])
+
+        return 0 if @buffer_state[:pending_count] == 0
+        return 0 if (!force) &&
+           (@buffer_state[:pending_count] < @buffer_config[:max_items]) &&
+           (time_since_last_flush < @buffer_config[:max_interval])
+
+        @buffer_state[:pending_mutex].synchronize do
+          @buffer_state[:outgoing_items] = @buffer_state[:pending_items]
+          @buffer_state[:outgoing_count] = @buffer_state[:pending_count]
+          buffer_clear_pending
+        end
+
+        @buffer_config[:logger].debug("Flushing output",
+          :outgoing_count => @buffer_state[:outgoing_count],
+          :time_since_last_flush => time_since_last_flush,
+          :outgoing_events => @buffer_state[:outgoing_items],
+          :batch_timeout => @buffer_config[:max_interval],
+          :force => force,
+          :final => final
+        ) if @buffer_config[:logger]
+
+        @buffer_state[:outgoing_items].each do |group, events|
+          begin
+            if group.nil?
+              flush(events,final)
+            else
+              flush(events, group, final)
+            end
+
+            @buffer_state[:outgoing_items].delete(group)
+            events_size = events.size
+            @buffer_state[:outgoing_count] -= events_size
+            items_flushed += events_size
+
+          rescue => e
+
+            @buffer_config[:logger].warn("Failed to flush outgoing items",
+              :outgoing_count => @buffer_state[:outgoing_count],
+              :exception => e.class.name,
+              :backtrace => e.backtrace
+            ) if @buffer_config[:logger]
+
+            if @buffer_config[:has_on_flush_error]
+              on_flush_error e
+            end
+
+            sleep 1
+            retry
+          end
+          @buffer_state[:last_flush] = Time.now
+        end
+
+      ensure
+        @buffer_state[:flush_mutex].unlock
+      end
+
+      return items_flushed
+    end
+
+    private
+    def buffer_clear_pending
+      @buffer_state[:pending_items] = Hash.new { |h, k| h[k] = [] }
+      @buffer_state[:pending_count] = 0
+    end
+  end
+end

data/vendor/jruby/1.9/gems/logstash-codec-json-2.0.3/vendor/gems/stud-0.0.22/lib/stud/interval.rb

@@ -0,0 +1,110 @@
+require "thread"
+
+module Stud
+  STUD_STOP_REQUESTED = :stud_stop_requested
+
+  # This implementation tries to keep clock more accurately.
+  # Prior implementations still permitted skew, where as this one
+  # will attempt to correct for skew.
+  #
+  # The execution patterns of this method should be that
+  # the start time of 'block.call' should always be at time T*interval
+  def self.interval(time, opts = {}, &block)
+    start = Time.now
+    while true
+      if opts[:sleep_then_run]
+        start = sleep_for_interval(time, start)
+        break if stop?
+        block.call
+      else
+        block.call
+        start = sleep_for_interval(time, start)
+        break if stop?
+      end
+    end # loop forever
+  end # def interval
+
+  def interval(time, opts = {}, &block)
+    Stud.interval(time, opts, &block)
+  end # def interval
+
+  # stop! instructs interval to stop and exit its execution loop before going to
+  # sleep between block executions.
+  # NOW the tricky part is: this is typically an operation that will be called
+  # from another thread than the thread running the interval loop in which case
+  # the target parameter must be set to the Thread object which is running the
+  # interval loop.
+  # Note that the stop logic is compatible with Stud::Task so if interval is run
+  # inside a Stud::Task, calling Stud::Task#stop! will stop the interval the same
+  # way as calling stop! on the interval itself.
+  # @param target [Thread] the target thread to stop, defaut to Thread.current
+  def self.stop!(target = Thread.current)
+    # setting/getting threalocal var is thread safe in JRuby
+    target[STUD_STOP_REQUESTED] = true
+    target.wakeup
+    nil
+  end
+
+  # stop? returns true if stop! has been called
+  # @param target [Thread] the target thread to check for stop, defaut to Thread.current
+  # @return [Boolean] true if the stop! has been called
+  def self.stop?(target = Thread.current)
+    # setting/getting threalocal var is thread safe in JRuby
+    target[STUD_STOP_REQUESTED]
+  end
+
+  class << Stud
+    # also support Stud.interrupted? for backward compatibility.
+    alias_method :interrupted?, :stop?
+  end
+
+  # stoppable_sleep will try to sleep for the given duration seconds (which may be any number,
+  # including a Float with fractional seconds). an optional stop_condition_block can be supplied
+  # to verify for sleep interruption if the block returns a truthy value. if not block is supplied
+  # it will check for the Stud.stop? condition. this check will be performed at 1s interval
+  # by default or you can supply a different stop_condition_interval.
+  #
+  # note that to achieve this, stoppable_sleep will actually perform a series of incremental sleeps
+  # but will try accurately spend the requested duration period in the overall stoppable_sleep method call.
+  # in other words this means that the duration supplied will be accurate for the time spent in
+  # the stoppable_sleep method not the actual total time spent in the underlying multiple sleep calls.
+  #
+  # @param duration [Numeric] sleep time in (fractional) seconds
+  # @param stop_condition_interval [Numeric] optional interval in (fractional) seconds to perform the sleep interruption verification, default is 1s
+  # @param stop_condition_block [Proc] optional sleep interruption code block that must evaluate to a truthy value, default is to use Stud.stop?
+  # @return [Numeric] the actual duration in (fractional) seconds spent in stoppable_sleep
+  def self.stoppable_sleep(duration, stop_condition_interval = 1.0, &stop_condition_block)
+    sleep_start = Time.now
+
+    # default to using Stud.stop? as the condition block
+    stop_condition_block ||= lambda { stop? }
+
+    while (remaining_duration = (duration - (Time.now - sleep_start))) >= stop_condition_interval
+      # sleep X if there is more than X remaining to sleep in relation to the loop start time
+      sleep(stop_condition_interval)
+
+      return(Time.now - sleep_start) if stop_condition_block.call
+    end
+
+    # here we know we have less than 1s reminding to sleep,
+    sleep(remaining_duration) if remaining_duration > 0.0
+
+    Time.now - sleep_start
+  end
+
+  private
+
+  def self.sleep_for_interval(time, start)
+    duration = Time.now - start
+
+    # sleep only if the duration was less than the time interval
+    if duration < time
+      stoppable_sleep(time - duration)
+      start += time
+    else
+      # duration exceeded interval time, reset the clock and do not sleep.
+      start = Time.now
+    end
+  end
+
+end # module Stud

data/vendor/jruby/1.9/gems/logstash-codec-json-2.0.3/vendor/gems/stud-0.0.22/lib/stud/pool.rb

@@ -0,0 +1,215 @@
+require "thread"
+
+module Stud
+  # Public: A thread-safe, generic resource pool.
+  #
+  # This class is agnostic as to the resources in the pool. You can put
+  # database connections, sockets, threads, etc. It's up to you!
+  #
+  # Examples:
+  #
+  #     pool = Pool.new
+  #     pool.add(Sequel.connect("postgres://pg-readonly-1/prod"))
+  #     pool.add(Sequel.connect("postgres://pg-readonly-2/prod"))
+  #     pool.add(Sequel.connect("postgres://pg-readonly-3/prod"))
+  #
+  #     pool.fetch # =>  Returns one of the Sequel::Database values from the pool
+  class Pool
+
+    class Error < StandardError; end
+
+    # An error indicating a given resource is busy.
+    class ResourceBusy < Error; end
+
+    # An error indicating a given resource is not found.
+    class NotFound < Error; end
+
+    # You performed an invalid action.
+    class InvalidAction < Error; end
+
+    # Default all methods to private. See the bottom of the class definition
+    # for public method declarations.
+    private
+
+    # Public: initialize a new pool.
+    #
+    # max_size - if specified, limits the number of resources allowed in the pool.
+    def initialize(max_size=nil)
+      # Available resources
+      @available = Hash.new
+      # Busy resources
+      @busy = Hash.new
+
+      # The pool lock
+      @lock = Mutex.new
+
+      # Locks for blocking {#fetch} calls if the pool is full.
+      @full_lock = Mutex.new
+      @full_cv = ConditionVariable.new
+
+      # Maximum size of this pool.
+      @max_size = max_size
+    end # def initialize
+
+    # Private: Is this pool size-limited?
+    #
+    # Returns true if this pool was created with a max_size. False, otherwise.
+    def sized?
+      return !@max_size.nil?
+    end # def sized?
+
+    # Private: Is this pool full?
+    #
+    # Returns true if the pool is sized and the count of resources is at maximum.
+    def full?
+      return sized? && (count == @max_size)
+    end # def full?
+
+    # Public: the count of resources in the pool
+    #
+    # Returns the count of resources in the pool.
+    def count
+      return (@busy.size + @available.size)
+    end # def count
+
+    # Public: Add a new resource to this pool.
+    #
+    # The resource, once added, is assumed to be available for use.
+    # That means once you add it, you must not use it unless you receive it from
+    # {Pool#fetch}
+    #
+    # resource - the object resource to add to the pool.
+    #
+    # Returns nothing
+    def add(resource)
+      @lock.synchronize do
+        @available[resource.object_id] = resource
+      end
+      return nil
+    end # def add
+
+    # Public: Fetch an available resource.
+    #
+    # If no resource is available, and the pool is not full, the
+    # default_value_block will be called and the return value of it used as the
+    # resource.
+    #
+    # If no resource is availabe, and the pool is full, this call will block
+    # until a resource is available.
+    #
+    # Returns a resource ready to be used.
+    def fetch(&default_value_block)
+      @lock.synchronize do
+        object_id, resource = @available.shift
+        if !resource.nil?
+          @busy[resource.object_id] = resource
+          return resource
+        end
+      end
+
+      @full_lock.synchronize do
+        if full?
+          # This should really use a logger.
+          puts "=> Pool is full and nothing available. Waiting for a release..."
+          @full_cv.wait(@full_lock)
+          return fetch(&default_value_block)
+        end
+      end
+
+      # TODO(sissel): If no block is given, we should block until a resource is
+      # available.
+
+      # If we get here, no resource is available and the pool is not full.
+      resource = default_value_block.call
+      # Only add the resource if the default_value_block returned one.
+      if !resource.nil?
+        add(resource)
+        return fetch
+      end
+    end # def fetch
+
+    # Public: Remove a resource from the pool.
+    #
+    # This is useful if the resource is no longer useful. For example, if it is
+    # a database connection and that connection has failed.
+    #
+    # This resource *MUST* be available and not busy.
+    #
+    # Raises Pool::NotFound if no such resource is found.
+    # Raises Pool::ResourceBusy if the resource is found but in use.
+    def remove(resource)
+      # Find the object by object_id
+      #p [:internal, :busy => @busy, :available => @available]
+      @lock.synchronize do
+        if available?(resource)
+          raise InvalidAction, "This resource must be busy for you to remove " \
+            "it (ie; it must be fetched from the pool)"
+        end
+        @busy.delete(resource.object_id)
+      end
+    end # def remove
+
+    # Private: Verify this resource is in the pool.
+    #
+    # You *MUST* call this method only when you are holding @lock.
+    #
+    # Returns :available if it is available, :busy if busy, false if not in the pool.
+    def include?(resource)
+      if @available.include?(resource.object_id)
+        return :available
+      elsif @busy.include?(resource.object_id)
+        return :busy
+      else
+        return false
+      end
+    end # def include?
+
+    # Private: Is this resource available?
+    # You *MUST* call this method only when you are holding @lock.
+    #
+    # Returns true if this resource is available in the pool.
+    # Raises NotFound if the resource given is not in the pool at all.
+    def available?(resource)
+      case include?(resource)
+        when :available; return true
+        when :busy; return false
+        else; raise NotFound, "No resource, #{resource.inspect}, found in pool"
+      end
+    end # def avilable?
+
+    # Private: Is this resource busy?
+    #
+    # You *MUST* call this method only when you are holding @lock.
+    #
+    # Returns true if this resource is busy.
+    # Raises NotFound if the resource given is not in the pool at all.
+    def busy?(resource)
+      return !available?(resource)
+    end # def busy?
+
+    # Public: Release this resource back to the pool.
+    #
+    # After you finish using a resource you received with {#fetch}, you must
+    # release it back to the pool using this method.
+    #
+    # Alternately, you can {#remove} it if you want to remove it from the pool
+    # instead of releasing it.
+    def release(resource)
+      @lock.synchronize do
+        if !include?(resource)
+          raise NotFound, "No resource, #{resource.inspect}, found in pool"
+        end
+
+        # Release is a no-op if this resource is already available.
+        #return if available?(resource)
+        @busy.delete(resource.object_id)
+        @available[resource.object_id] = resource
+
+        # Notify any threads waiting on a resource from the pool.
+        @full_lock.synchronize { @full_cv.signal }
+      end
+    end # def release
+
+    public(:add, :remove, :fetch, :release, :sized?, :count, :initialize)
+  end # class Pool
+end # module Stud