bean_counter 0.0.4 → 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: e093642cbffceb8f4714bc9f8fece7a8dd7c44c4
+  data.tar.gz: 49f7dc68b35365ebca7d40544936a8b6a8a3a23b
+SHA512:
+  metadata.gz: 28d291b01222cbe8b50158f4ec55a1cdb836bbda6abd3c7a3b65b8222e2fd005af07232982b363fe9a7c405a9a23472f2e62a965a76029575ff4f731582311f8
+  data.tar.gz: 60bc42d02c66e50642faf210b897ec977d0bff9d279655b6286fa6ae24c4bd5f873c3cb3840badb746d5c6c588f1fc6d8ab92c1c2d4daff3948ed0814127e18f

data/.travis.yml

@@ -3,6 +3,7 @@ language: ruby
 rvm:
   - 1.9.3
   - 2.0.0
+  - 2.1.0
   - rbx-19mode
 
 before_install:

data/README.md

@@ -172,16 +172,42 @@ BeanCounter.beanstalkd_url
 ###BeanCounter.strategy
 `BeanCounter.strategy` allows you to choose and configure the strategy
 BeanCounter uses for accessing and interacting with the beanstalkd pool. At
-present, there is only a single strategy available, but at least one other is
-in the works.
+present, there are two strategies available: StalkClimberStrategy and
+GemeraldBeanstalkStrategy.
 
-The strategy currently available, BeanCounter::Strategy::StalkClimber, utilizes
+The default BeanCounter::Strategy::StalkClimber strategy utilizes
 [StalkClimber](https://github.com/gemeraldbeanstalk/stalk_climber.git) to
 navigate the beanstalkd pool. The job traversal method employed by StalkClimber
 suffers from some inefficiencies that come with trying to sequential access a
 queue, but overall it is a powerful strategy that supports multiple servers and
 offers solid performance given the design of beanstalkd.
 
+The BeanCounter::Strategy::GemeraldBeanstalkStrategy strategy utilizes
+[GemeraldBeanstalk](https://github.com/gemeraldbeanstalk/gemerald_beanstalk.git)
+servers as the backend for Beaneater. This means that no beanstalkd server is
+required for testing. Beaneater still connects and communicates with
+GemeraldBeanstalk via a TCPSocket, however BeanCounter is able to talk directly
+to the GemeraldBeanstalk servers allowing for faster and more consistent testing.
+When using the GemeraldBeanstalkStrategy, BeanCounter will automatically start
+a GemeraldBeanstalk server at each of the addresses specified by beanstalkd_url.
+
+```ruby
+# No configuration is required to use the StalkClimberStrategy
+
+# To use the GemeraldBeanstalkStrategy
+BeanCounter.strategy = :'BeanCounter::Strategy::GemeraldBeanstalkStrategy'
+  #=> :"BeanCounter::Strategy::GemeraldBeanstalkStrategy"
+
+# Because the GemeraldBeanstalkStrategy starts a GemeraldBeanstalk server
+# immediately, you may want to configure a different address/port for
+# your GemeraldBeanstalk servers to avoid conflicts when you have a Beanstalkd
+# server running:
+BeanCounter.beanstalkd_url = ['127.0.0.1:11400']
+BeanCounter.strategy = :'BeanCounter::Strategy::GemeraldBeanstalkStrategy'
+  #=> :"BeanCounter::Strategy::GemeraldBeanstalkStrategy"
+```
+
+
 ## Usage
 Whether you are using the TestUnit/MiniTest or RSpec matchers, the usage is the
 same, the only difference is the invocation.
@@ -312,7 +338,7 @@ verify that the exports tube is paused:
 For more detailed explanations and more examples make sure to check out the
 docs, expectations, and respective tests:
 
-- [docs](http://rubydoc.info/gems/bean_counter/0.0.4/frames)
+- [docs](http://rubydoc.info/gems/bean_counter/0.1.0/frames)
 - [enqueued_expectation](https://github.com/gemeraldbeanstalk/bean_counter/tree/master/lib/bean_counter/enqueued_expectation.rb)
 - [tube_expectation](https://github.com/gemeraldbeanstalk/bean_counter/tree/master/lib/bean_counter/tube_expectation.rb)
 - [test_assertions](https://github.com/gemeraldbeanstalk/bean_counter/tree/master/lib/bean_counter/test_assertions.rb)

data/bean_counter.gemspec

@@ -20,6 +20,7 @@ Gem::Specification.new do |spec|
 
   spec.add_dependency 'beaneater'
   spec.add_dependency 'stalk_climber', '>= 0.1.0'
+  spec.add_dependency 'gemerald_beanstalk', '>= 0.0.1'
 
   spec.add_development_dependency 'bundler', '~> 1.3'
   spec.add_development_dependency 'rake'

data/lib/bean_counter.rb

@@ -4,5 +4,6 @@ require 'bean_counter/version'
 require 'bean_counter/core'
 require 'bean_counter/strategy'
 require 'bean_counter/strategies/stalk_climber_strategy'
+require 'bean_counter/strategies/gemerald_beanstalk_strategy'
 require 'bean_counter/enqueued_expectation'
 require 'bean_counter/tube_expectation'

data/lib/bean_counter/strategies/gemerald_beanstalk_strategy.rb

@@ -0,0 +1,3 @@
+require 'bean_counter/strategies/gemerald_beanstalk_strategy/strategy'
+require 'bean_counter/strategies/gemerald_beanstalk_strategy/tube'
+require 'bean_counter/strategies/gemerald_beanstalk_strategy/job'

data/lib/bean_counter/strategies/gemerald_beanstalk_strategy/job.rb

@@ -0,0 +1,73 @@
+require 'forwardable'
+
+class BeanCounter::Strategy::GemeraldBeanstalkStrategy::Job
+
+  extend Forwardable
+
+  def_delegators :@gemerald_job, :body, :stats
+
+  attr_reader :connection
+
+  # Attributes that should be retrieved via the gemerald job's stats. This
+  # list is used to dynamically create the appropriate accessor methods
+  STATS_METHODS = %w[
+    age buries delay id kicks pri releases reserves
+    state time-left timeouts ttr tube
+  ]
+
+  # Simple accessors allowing direct access to job stats attributes
+  STATS_METHODS.each do |stat_method|
+    define_method stat_method.gsub(/-/, '_') do
+      return @gemerald_job.stats[stat_method]
+    end
+  end
+
+
+  # Attempts to delete the job. Returns true if deletion succeeds or if job
+  # does not exist. Returns false if job could not be deleted (typically due
+  # to it being reserved by another connection).
+  #
+  # @return [Boolean] If the given job was successfully deleted or does not
+  #   exist, returns true. Otherwise returns false.
+  def delete
+    response = connection.transmit("delete #{id}")
+    if response == "DELETED\r\n" || !exists?
+      return true
+    else
+      return false
+    end
+  end
+
+
+  # Returns a Boolean indicating whether or not the job still exists.
+  # @return [Boolean] If job state is deleted or the job does not exist
+  #   on the beanstalk server, returns false. If job state is not deleted,
+  #   returns true if the job exists on the beanstalk server.
+  def exists?
+    return false if state == 'deleted'
+    return connection.transmit("stats-job #{id}") != "NOT_FOUND\r\n"
+  end
+
+
+  # Initialize a new GemeraldBeanstalkStrategy::Job wrapping the provided
+  # `gemerald_job` in the context of the provided `connection`.
+  def initialize(gemerald_job, connection)
+    @gemerald_job = gemerald_job
+    @connection = connection
+  end
+
+
+  # Augment job stats to provide a Hash representation of the job.
+  # @return [Hash] Hash representation of the job
+  def to_hash
+    stats_pairs = stats.to_a
+    stats_pairs << ['body', body]
+    stats_pairs << ['connection', connection]
+
+    hash = Hash[stats_pairs.sort_by!(&:first)]
+    hash.delete('file')
+
+    return hash
+  end
+
+end

data/lib/bean_counter/strategies/gemerald_beanstalk_strategy/strategy.rb

@@ -0,0 +1,251 @@
+require 'gemerald_beanstalk'
+require 'gemerald_beanstalk/plugins/introspection'
+require 'gemerald_beanstalk/plugins/direct_connection'
+require 'forwardable'
+
+class BeanCounter::Strategy::GemeraldBeanstalkStrategy < BeanCounter::Strategy
+
+  # Regex for checking for valid V4 IP addresses
+  V4_IP_REGEX = /^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(?::\d+)?$/
+
+  extend Forwardable
+
+  def_delegator :job_enumerator, :each, :jobs
+  def_delegator :tube_enumerator, :each, :tubes
+
+
+  # Collects all jobs enqueued during the execution of the provided `block`.
+  # Returns an Array of GemeraldBeanstalkStrategy::Job.
+  #
+  # Fulfills {BeanCounter::Strategy#collect_new_jobs} contract.
+  #
+  # @see BeanCounter::Strategy#collect_new_jobs
+  # @see BeanCounter::Strategy::GemeraldBeanstalkStrategy::Job
+  # @yield Nothing is yielded to the provided `block`
+  # @raise [ArgumentError] if a block is not provided.
+  # @return [Array<GemeraldBeanstalkStrategy::Job>] all jobs enqueued during the
+  #   execution of the provided `block`
+  def collect_new_jobs
+    raise ArgumentError, 'Block required' unless block_given?
+
+    max_id_pairs = beanstalks.map do |beanstalk|
+      [beanstalk, beanstalk.jobs[-1] ? beanstalk.jobs[-1].id : 0]
+    end
+    max_ids = Hash[max_id_pairs]
+
+    yield
+
+    new_jobs = []
+    # Jobs are collected in reverse, so iterate beanstalks in reverse too
+    beanstalks.reverse.each do |beanstalk|
+      jobs = beanstalk.jobs
+      index = -1
+      while (job = jobs[index]) && job.id > max_ids[beanstalk] do
+        new_jobs << strategy_job(job)
+        index -= 1
+      end
+    end
+    # Flip new jobs to maintain beanstalk, id asc order
+    return new_jobs.reverse
+  end
+
+
+  # Attempts to delete the provided GemeraldBeanstalkStrategy::Job `job`.
+  # Returns true if deletion succeeds or if `job` does not exist. Returns false
+  # if `job` could not be deleted (typically due to it being reserved by
+  # another connection).
+  #
+  # Fulfills {BeanCounter::Strategy#delete_job} contract.
+  #
+  # @see BeanCounter::Strategy#delete_job
+  # @param job [GemeraldBeanstalkStrategy::Job] the job to be deleted
+  # @return [Boolean] If the given job was successfully deleted or does not
+  #   exist, returns true. Otherwise returns false.
+  def delete_job(job)
+    return job.delete
+  end
+
+
+  # Initialize a new GemeraldBeanstalkStrategy. This includes initializing
+  # GemeraldBeanstalk::Servers for each of the beanstalk_urls visible to
+  # BeanCounter and also spawning direct connections to those servers.
+  def initialize
+    @clients = {}
+    @beanstalk_servers = []
+    BeanCounter.beanstalkd_url.each do |url|
+      server = GemeraldBeanstalk::Server.new(*parse_url(url))
+      @clients[server.beanstalk] = server.beanstalk.direct_connection_client
+      @beanstalk_servers << server
+    end
+  end
+
+
+  # Returns a Boolean indicating whether or not the provided
+  # GemeraldBeanstalkStrategy::Job `job` matches the given Hash of `options`.
+  #
+  # See {MATCHABLE_JOB_ATTRIBUTES} for a list of
+  # attributes that can be used when matching.
+  #
+  # Fulfills {BeanCounter::Strategy#job_matches?} contract.
+  #
+  # @see MATCHABLE_JOB_ATTRIBUTES
+  # @see BeanCounter::Strategy#job_matches?
+  # @param job [GemeraldBeanstalkStrategy::Job] the job to evaluate for a matche.
+  # @param options
+  #   [Hash{String, Symbol => Numeric, Proc, Range, Regexp, String, Symbol}]
+  #   Options to be used to evaluate a match.
+  # @return [Boolean] If job exists and matches the provided options, returns
+  #   true. Otherwise, returns false.
+  def job_matches?(job, options = {})
+    return matcher(MATCHABLE_JOB_ATTRIBUTES, job, options)
+  end
+
+
+  # Returns a String representation of the GemeraldBeanstalkStrategy::Job `job`
+  # in a pretty, human readable format.
+  #
+  # Fulfills {BeanCounter::Strategy#pretty_print_job} contract.
+  #
+  # @see BeanCounter::Strategy#pretty_print_job
+  # @param job [GemeraldBeanstalkStrategy::Job] the job to print in a more
+  #   readable format.
+  # @return [String] A more human-readable representation of `job`.
+  def pretty_print_job(job)
+    hash = job.to_hash
+    hash.delete('connection')
+    return hash.to_s
+  end
+
+
+  # Returns a String representation of `tube` in a pretty, human readable format.
+  #
+  # Fulfills {BeanCounter::Strategy#pretty_print_tube} contract.
+  #
+  # @see BeanCounter::Strategy#pretty_print_tube
+  # @param tube [GemeraldBeanstalkStrategy::Tube] the tube to print in a more
+  #   readable format.
+  # @return [String] A more human-readable representation of `tube`.
+  def pretty_print_tube(tube)
+    return tube.to_hash.to_s
+  end
+
+
+  # Returns a boolean indicating whether or not the provided
+  # GemeraldBeanstalkStrategy::Tube, `tube`, matches the given Hash of `options`.
+  #
+  # See {MATCHABLE_TUBE_ATTRIBUTES} for a list of attributes that can be used
+  # when evaluating a match.
+  #
+  # Fulfills {BeanCounter::Strategy#tube_matches?} contract.
+  #
+  # @see BeanCounter::Strategy#tube_matches?
+  # @param tube [GemeraldBeanstalkStrategy::Tube] the tube to evaluate a match
+  #   against.
+  # @param options
+  #   [Hash{String, Symbol => Numeric, Proc, Range, Regexp, String, Symbol}]
+  #   a Hash of options to use when evaluating a match.
+  # @return [Boolean] If `tube` exists and matches against the provided options,
+  #   returns true. Otherwise returns false.
+  def tube_matches?(tube, options = {})
+    return false if tube.nil?
+    return matcher(MATCHABLE_TUBE_ATTRIBUTES, tube, options)
+  end
+
+  private
+
+  # The collection of GemeraldBeanstalk servers the strategy built during
+  # initialization
+  attr_reader :beanstalk_servers
+
+
+  # The collection of beanstalks belonginging to the GemeraldBeanstalk servers
+  # that were created during initialization.
+  #
+  # @return [Array<GemeraldBeanstalk::Beanstalk>] the Beanstalks of the
+  #   GemeraldBeanstalk::Servers
+  def beanstalks
+    return @beanstalks ||= @beanstalk_servers.map(&:beanstalk)
+  end
+
+
+  # Returns an enumerator that enumerates all jobs on all beanstalk servers in
+  # the order the beanstalk servers are defined in ascending order. Jobs are
+  # returned as GemeraldBeanstalkStrategy::Job.
+  def job_enumerator
+    return Enumerator.new do |yielder|
+      beanstalks.each do |beanstalk|
+        beanstalk.jobs.each do |job|
+          yielder << strategy_job(job)
+        end
+      end
+    end
+  end
+
+
+  # Generic match evaluator that compares the Hash of `options` given to the
+  # Strategy representation of the given `matchable`.
+  def matcher(valid_attributes, matchable, options = {})
+    return false unless matchable.exists?
+    return (options.keys & valid_attributes).all? do |key|
+      options[key] === matchable.send(key.to_s.gsub(/-/, '_'))
+    end
+  end
+
+
+  # Parses a variety of forms of beanstalk URL and returns a pair including the
+  # hostname and possibly a port given by the url.
+  def parse_url(url)
+    unless V4_IP_REGEX === url
+      uri = URI.parse(url)
+      if uri.scheme && uri.host
+        raise(ArgumentError, "Invalid beanstalk URI: #{url}") unless uri.scheme == 'beanstalk'
+        host = uri.host
+        port = uri.port
+      end
+    end
+    unless host
+      match = url.split(/:/)
+      host = match[0]
+      port = match[1]
+    end
+    return port ? [host, Integer(port)] : [host]
+  end
+
+
+  # Helper method to transform a GemeraldBeanstalk::Job into a
+  # BeanCounter::Strategy::GemeraldBeanstalkStrategy::Job. The strategy
+  # specific job class is intended to hide native job implementation interface
+  # specifics and prevent meddling with native Job internals.
+  def strategy_job(gemerald_job)
+    return BeanCounter::Strategy::GemeraldBeanstalkStrategy::Job.new(gemerald_job, @clients[gemerald_job.beanstalk])
+  end
+
+
+  # Helper method to transform a `tube_name` into a
+  # BeanCounter::Strategy::GemeraldBeanstalkStrategy::Tube. The strategy
+  # specific tube class is intended to hide native tube implementation interface
+  # specifics and prevent meddling with native Tube internals. Also handles
+  # merging of tube data from multiple servers, as such, requires access to
+  # beanstalk servers.
+  def strategy_tube(tube_name)
+    return BeanCounter::Strategy::GemeraldBeanstalkStrategy::Tube.new(tube_name, beanstalks)
+  end
+
+
+  # Returns an enumerator that enumerates all tubes on all beanstalk servers.
+  # Each tube is included in the enumeration only once, regardless of how many
+  # servers contain an instance of that tube. Tube stats are merged before they
+  # are yielded by the enumerator as a GemeraldBeanstalkdStrategy::Tube.
+  def tube_enumerator
+    tubes_in_pool = beanstalks.inject([]) do |memo, beanstalk|
+      memo.concat(beanstalk.tubes.keys)
+    end
+    tubes_in_pool.uniq!
+    return Enumerator.new do |yielder|
+      tubes_in_pool.each do |tube_name|
+        yielder << strategy_tube(tube_name)
+      end
+    end
+  end
+
+end

data/lib/bean_counter/strategies/gemerald_beanstalk_strategy/tube.rb

@@ -0,0 +1,56 @@
+class BeanCounter::Strategy::GemeraldBeanstalkStrategy::Tube
+
+  # The name of the tube represented by the object
+  attr_reader :name
+
+  # Attributes that should be obtained via the tube's stats. This list is used
+  # to dynamically create the attribute accessor methods.
+  STATS_METHODS = %w[
+    cmd-delete cmd-pause-tube current-jobs-buried current-jobs-delayed
+    current-jobs-ready current-jobs-reserved current-jobs-urgent current-using
+    current-waiting current-watching pause pause-time-left total-jobs
+  ]
+
+  STATS_METHODS.each do |attr_method|
+    define_method attr_method.gsub(/-/, '_') do
+      return to_hash[attr_method]
+    end
+  end
+
+  # Returns a Boolean indicating whether or not the tube exists in the pool.
+  # @return [Boolean] Returns true if tube exists on any of the servers in the
+  #   pool, otherwise returns false.
+  def exists?
+    tubes_in_pool = @beanstalks.inject([]) do |memo, beanstalk|
+      memo.concat(beanstalk.tubes.keys)
+    end
+    tubes_in_pool.uniq!
+    return tubes_in_pool.include?(@name)
+  end
+
+
+  # Initialize a new GemeraldBeanstalkStrategy::Tube representing all tubes
+  # named `tube_name` in the given pool of `beanstalks`.
+  def initialize(tube_name, beanstalks)
+    @name = tube_name
+    @beanstalks = beanstalks
+  end
+
+
+  # Retrieves stats for the given `tube_name` from all known servers and merges
+  # numeric stats. Matches Beaneater's treatment of a tube as a collective
+  # entity and not a per-server entity.
+  def to_hash
+    return @beanstalks.inject({}) do |hash, beanstalk|
+      next hash if (tube = beanstalk.tubes[name]).nil?
+      next tube.stats if hash.empty?
+
+      tube.stats.each do |stat, value|
+        next unless value.is_a?(Numeric)
+        hash[stat] += value
+      end
+      hash
+    end
+  end
+
+end