infopark-politics 0.2.7

data/History.rdoc

@@ -0,0 +1,31 @@
+= Changelog
+
+== 0.2.5 (2009-02-04)
+
+* Gracefully handle MemCache::MemCacheErrors.  Just sleep until memcached comes back.
+
+== 0.2.4 (2009-01-28)
+
+* Reduce leader token expiration time to discourage a get/set race condition. (Brian Dainton)
+
+== 0.2.3 (2009-01-12)
+
+* Fix invalid result check in previous change. (Brian Dainton)
+
+== 0.2.2 (2009-01-07)
+
+* Fix invalid leader? logic in TokenWorker which could allow
+  two workers to become leader at the same time. (Brian Dainton)
+
+== 0.2.1 (2008-11-04)
+
+* Cleanup and prepare for public release for RubyConf 2008.
+* Election Day.  Politics.  Get it?  Hee hee.
+
+== 0.2.0 (2008-10-24)
+
+* Remove BucketWorker based on initial feedback.  Add StaticQueueWorker as a more reliable replacement.
+
+== 0.1.0 (2008-10-07)
+
+* Add BucketWorker and TokenWorker mixins.

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008 Mike Perham
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,49 @@
+= Politics
+
+Politics is a Ruby library providing utilities and algorithms for solving common distributed
+computing problems.  Distributed Computing and Politics have a number of things in common:
+1) they can be beautiful in theory but get really ugly in reality; 2) after working with
+either for a few weeks/months/years (depending on your moral flexibility) you'll find yourself
+intellectually devoid, a hollow shell of a man/woman/cybernetic killing machine.
+
+So the name is to be taken tongue in cheek.  Onto the real details.
+
+== Common Problems in Distributed Computing
+
+Ruby services are often deployed as a cloud of many processes across several machines, 
+for fault tolerance.  This introduces the problem of coordination between those processes.
+Specifically, how do you keep those processes from stepping on each other's electronic
+toes?  There are several answers:
+
+1. Break the processing into N 'buckets'.  Have an individual process fetch a bucket,
+   work on it, and ask for another.  This is a very scalable solution as it allows N workers 
+   to work on different parts of the same task concurrently.  See the +StaticQueueWorker+ mixin.
+1. Elect a leader for a short period of time.  The leader is the process which performs the
+   actual processing.  After a length of time, a new leader is elected from the group.  This
+   is fault tolerant but not as scalable, as only one process is performing the task at a given
+   point in time.  See the +TokenWorker+ mixin.
+
+== Installation
+
+  sudo gem install mperham-politics -s http://gems.github.com
+
+== Dependencies
+
+StaticQueueWorker mixin
+* memcached - the mechanism to elect a leader amongst a set of peers.
+* DRb - the mechanism to communicate between peers.
+* mDNS - the mechanism to discover peers.
+
+TokenWorker mixin
+* memcached - the mechanism to elect a leader amongst a set of peers.
+
+
+= Author
+
+Name::      Mike Perham
+Email::     mailto:mperham@gmail.com
+Twitter::   http://twitter.com/mperham
+Homepage::  http://mikeperham.com/
+
+This software is free for you to use as you'd like.  If you find it useful, please consider giving 
+me a recommendation at {Working with Rails}[http://workingwithrails.com/person/10797-mike-perham].

data/examples/queue_worker_example.rb

@@ -0,0 +1,37 @@
+#gem 'mperham-politics'
+require 'politics'
+require 'politics/static_queue_worker'
+
+# Test this example by starting memcached locally and then in two irb sessions, run this:
+#
+=begin
+ require 'queue_worker_example'
+ p = Politics::QueueWorkerExample.new
+ p.start
+=end
+#
+# You can then watch as one of them is elected leader.  You can kill the leader and verify
+# the backup process is elected after approximately iteration_length seconds.
+#
+module Politics
+  class QueueWorkerExample
+    include Politics::StaticQueueWorker
+    TOTAL_BUCKETS = 20
+
+    def initialize
+      register_worker 'queue-example', TOTAL_BUCKETS, :iteration_length => 60, :servers => memcached_servers
+    end
+    
+    def start
+      process_bucket do |bucket|
+        puts "PID #{$$} processing bucket #{bucket}/#{TOTAL_BUCKETS} at #{Time.now}..."
+        sleep 1.5
+      end
+    end
+    
+    def memcached_servers
+      ['127.0.0.1:11211']
+    end
+
+  end
+end

data/examples/token_worker_example.rb

@@ -0,0 +1,35 @@
+#gem 'mperham-politics'
+require 'politics'
+require 'politics/token_worker'
+
+# Test this example by starting memcached locally and then in two irb sessions, run this:
+#
+=begin
+ require 'token_worker_example'
+ p = Politics::TokenWorkerExample.new
+ p.start
+=end
+#
+# You can then watch as one of them is elected leader.  You can kill the leader and verify
+# the backup process is elected after approximately iteration_length seconds.
+#
+module Politics
+  class TokenWorkerExample
+    include Politics::TokenWorker
+    
+    def initialize
+      register_worker 'token-example', :iteration_length => 10, :servers => memcached_servers
+    end
+    
+    def start
+      process do
+        puts "PID #{$$} processing at #{Time.now}..."
+      end
+    end
+    
+    def memcached_servers
+      ['localhost:11211']
+    end
+
+  end
+end

data/lib/init.rb

@@ -0,0 +1,4 @@
+require 'politics'
+require 'politics/token_worker'
+require 'politics/static_queue_worker'
+require 'politics/discoverable_node'

data/lib/politics/discoverable_node.rb

@@ -0,0 +1,137 @@
+require 'socket'
+require 'ipaddr'
+require 'uri'
+require 'drb'
+
+require 'net/dns/mdns-sd'
+require 'net/dns/resolv-mdns'
+require 'net/dns/resolv-replace'
+
+=begin
+IRB setup:
+require 'lib/politics'
+require 'lib/politics/discoverable_node'
+require 'lib/politics/convention'
+Object.send(:include, Election::Candidate)
+p = Object.new
+p.register
+=end
+
+module Politics
+
+  # A module to solve the Group Membership problem in distributed computing.
+  # The "group" is the cloud of processes which are replicas and need to coordinate.
+  # Handling group membership is the first step in solving distributed computing
+  # problems.  There are two issues:
+  # 1) replica discovery
+  # 2) controlling and maintaining a consistent group of replicas in each replica
+  #
+  # Peer discovery is implemented using Bonjour for local network auto-discovery.
+  # Each process registers itself on the network as a process of a given type.
+  # Each process then queries the network for other replicas of the same type.
+  #
+  # The replicas then run the Multi-Paxos algorithm to provide consensus on a given
+  # replica set.  The algorithm is robust in the face of crash failures, but not
+  # Byzantine failures.
+	module DiscoverableNode
+	  
+	  attr_accessor :group
+	  attr_accessor :coordinator
+	  
+		def register(group='foo')
+		  self.group = group
+		  start_drb
+		  register_with_bonjour(group)
+		  Politics::log.info { "Registered #{self} in group #{group} with RID #{rid}" }
+		  sleep 0.5
+		  find_replicas(0)
+		end
+		
+    def replicas
+      @replicas ||= {}
+    end
+    
+    def find_replicas(count)
+      replicas.clear if count % 5 == 0
+      return if count > 10 # Guaranteed to terminate, but not successfully :-(
+
+      #puts "Finding replicas"
+      peer_set = []
+      bonjour_scan do |replica|
+        (his_rid, his_peers) = replica.hello(rid)
+        unless replicas.has_key?(his_rid)
+          replicas[his_rid] = replica
+        end
+        his_peers.each do |peer|
+          peer_set << peer unless peer_set.include? peer
+        end
+      end
+      #p [peer_set.sort, replicas.keys.sort]
+      if peer_set.sort != replicas.keys.sort
+        # Recursively call ourselves until the network has settled down and all
+        # peers have reached agreement on the peer group membership.
+        sleep 0.2
+        find_replicas(count + 1)
+      end
+      Politics::log.info { "Found #{replicas.size} peers: #{replicas.keys.sort.inspect}" } if count == 0
+      replicas
+    end
+    
+    # Called for one peer to introduce itself to another peer.  The caller
+    # sends his RID, the responder sends his RID and his list of current peer
+    # RIDs.
+    def hello(remote_rid)
+      [rid, replicas.keys]
+    end
+    
+	  # A process's Replica ID is its PID + a random 16-bit value.  We don't want
+	  # weigh solely based on PID or IP as that may unduly load one machine.
+	  def rid
+	    @rid ||= begin
+	      rand(65536) + $$
+      end
+    end
+
+		private
+		
+		def register_with_bonjour(group)
+		  # Register our DRb server with Bonjour.
+      handle = Net::DNS::MDNSSD.register("#{self.group}-#{local_ip}-#{$$}", 
+          "_#{self.group}._tcp", 'local', @port)
+          
+      ['INT', 'TERM'].each { |signal| 
+        trap(signal) { handle.stop }
+      }
+	  end
+		
+		def start_drb
+		  server = DRb.start_service(nil, self)
+		  @port = URI.parse(DRb.uri).port
+      ['INT', 'TERM'].each { |signal| 
+        trap(signal) { server.stop_service }
+      }
+	  end
+		
+		def bonjour_scan
+      Net::DNS::MDNSSD.browse("_#{@group}._tcp") do |b|
+        Net::DNS::MDNSSD.resolve(b.name, b.type) do |r|
+          drburl = "druby://#{r.target}:#{r.port}"
+          replica = DRbObject.new(nil, drburl)
+          yield replica
+        end
+      end
+	  end
+	  
+    # http://coderrr.wordpress.com/2008/05/28/get-your-local-ip-address/
+    def local_ip
+      orig, Socket.do_not_reverse_lookup = Socket.do_not_reverse_lookup, true # turn off reverse DNS resolution temporarily
+ 
+      UDPSocket.open do |s|
+        s.connect '64.233.187.99', 1
+        IPAddr.new(s.addr.last).to_i
+      end
+    ensure
+      Socket.do_not_reverse_lookup = orig
+    end	  
+  end
+end

data/lib/politics/static_queue_worker.rb

@@ -0,0 +1,306 @@
+require 'socket'
+require 'ipaddr'
+require 'uri'
+require 'drb'
+
+begin
+require 'net/dns/mdns-sd'
+require 'net/dns/resolv-mdns'
+require 'net/dns/resolv-replace'
+rescue LoadError => e
+  puts "Unable to load net-mdns, please run `sudo gem install net-mdns`: #{e.message}"
+  exit(1)
+end
+
+begin
+  require 'memcache'
+rescue LoadError => e
+  puts "Unable to load memcache client, please run `sudo gem install memcache-client`: #{e.message}"
+  exit(1)
+end
+
+module Politics
+
+  # The StaticQueueWorker mixin allows a processing daemon to "lease" or checkout
+  # a portion of a problem space to ensure no other process is processing that same
+  # space at the same time.  The processing space is cut into N "buckets", each of
+  # which is placed in a queue.  Processes then fetch entries from the queue
+  # and process them.  It is up to the application to map the bucket number onto its
+  # specific problem space.
+  #
+  # Note that memcached is used for leader election.  The leader owns the queue during
+  # the iteration period and other peers fetch buckets from the current leader during the
+  # iteration.
+  #
+  # The leader hands out buckets in order.  Once all the buckets have been processed, the
+  # leader returns nil to the processors which causes them to sleep until the end of the
+  # iteration.  Then everyone wakes up, a new leader is elected, and the processing starts
+  # all over again.
+  #
+  # DRb and mDNS are used for peer discovery and communication.
+  #
+  # Example usage:
+  #
+  #  class Analyzer
+  #    include Politics::StaticQueueWorker
+  #    TOTAL_BUCKETS = 16
+  #
+  #    def start
+  #      register_worker(self.class.name, TOTAL_BUCKETS)
+  #      process_bucket do |bucket|
+  #        puts "Analyzing bucket #{bucket} of #{TOTAL_BUCKETS}"
+  #        sleep 5
+  #      end
+  #    end
+  #  end
+  #
+  # Note: process_bucket never returns i.e. this should be the main loop of your processing daemon.
+  #
+  module StaticQueueWorker
+
+    def self.included(model) #:nodoc:
+      model.class_eval do
+        attr_accessor :group_name, :iteration_length
+      end
+    end
+
+    # Register this process as able to work on buckets.
+    def register_worker(name, bucket_count, config={})
+      options = {
+            :iteration_length => 60,
+            :servers => ['127.0.0.1:11211']
+          }
+      options.merge!(config)
+
+      self.group_name = name
+      self.iteration_length = options[:iteration_length]
+      @nominated_at = Time.now - self.iteration_length
+      @memcache_client = client_for(Array(options[:servers]))
+      # FIXME: Tests
+      @domain = options[:domain]
+
+      @buckets = []
+      @bucket_count = bucket_count
+
+      register_with_bonjour
+      log.progname = @uri
+      log.info { "Registered in group #{group_name} at port #{@port}" }
+    end
+
+    # Fetch a bucket out of the queue and pass it to the given block to be processed.
+    #
+    # +bucket+::            The bucket number to process, within the range 0...TOTAL_BUCKETS
+    def process_bucket(&block)
+      log.debug "start bucket processing"
+      raise ArgumentError, "process_bucket requires a block!" unless block_given?
+      raise ArgumentError, "You must call register_worker before processing!" unless @memcache_client
+
+      begin
+        begin
+          nominate
+          if leader?
+            # Drb thread handles leader duties
+            log.info { "has been elected leader" }
+            initialize_buckets
+            # keeping leader state as long as buckets are available by renominating before
+            # nomination times out
+            while !@buckets.empty? do
+              log.debug { "relaxes half the time until next iteration" }
+              relax(until_next_iteration / 2)
+              log.debug { "renew nomination too keep the hat and finish the work" }
+              @memcache_client.set(token, @uri, iteration_length)
+              @nominated_at = Time.now
+              log.error { "tried to staying leader but failed" } unless leader?
+            end
+            relax until_next_iteration
+          else
+            # Get a bucket from the leader and process it
+            begin
+              log.debug "getting bucket request from leader (#{leader_uri}) and processing it"
+              bucket_process(*leader.bucket_request, &block)
+            rescue DRb::DRbError => dre
+              log.error { "Error talking to leader: #{dre.message}" }
+              relax until_next_iteration
+            end
+          end
+        rescue MemCache::MemCacheError => e
+          log.error { "Unexpected MemCacheError: #{e.message}" }
+          relax until_next_iteration
+        end
+      end while loop?
+    end
+
+    def bucket_request
+      if leader?
+        log.debug "delivering bucket request"
+        [@buckets.pop, until_next_iteration]
+      else
+        log.debug "received request for bucket but am not leader - delivering :not_leader"
+        [:not_leader, 0]
+      end
+    end
+
+    def until_next_iteration
+      left = iteration_length - (Time.now - @nominated_at)
+      left > 0 ? left : 0
+    end
+
+    private
+
+    def bucket_process(bucket, sleep_time)
+      case bucket
+      when nil
+        # No more buckets to process this iteration
+        log.info { "No more buckets in this iteration, sleeping for #{sleep_time} sec" }
+        sleep sleep_time
+      when :not_leader
+        # Uh oh, race condition?  Invalid any local cache and check again
+        log.warn { "Recv'd NOT_LEADER from peer." }
+        relax 1
+        @leader_uri = nil
+      else
+        log.info { "processing #{bucket}" }
+        yield bucket
+      end
+    end
+
+    def log
+      @logger ||= Logger.new(STDOUT)
+    end
+
+    def initialize_buckets
+      @buckets.clear
+      @bucket_count.times { |idx| @buckets << idx }
+    end
+
+    def replicas
+      @replicas ||= []
+    end
+
+    def leader
+      log.debug "trying to get the leader (#{leader_uri})"
+      name = leader_uri
+      repl = nil
+      log.debug "replicas: #{replicas}"
+      loops = 0
+      while loops < 10 and (replicas.empty? or repl == nil)
+        repl = replicas.detect { |replica| replica.__drburi == name }
+        log.debug "repl: #{repl && repl.__drburi}"
+        unless repl
+          log.debug "scan bonjour for other nodes (replicas)"
+          relax 1
+          bonjour_scan do |replica|
+            log.debug "found replica #{replica.__drburi}"
+            replicas << replica
+          end
+        end
+        loops += 1
+      end
+      repl || raise(DRb::DRbError.new("Could not contact leader #{leader_uri}"))
+    end
+
+    def loop?
+      true
+    end
+
+    def token
+      "#{group_name}_token"
+    end
+
+    def cleanup
+      at_exit do
+        @memcache_client.delete(token) if leader?
+      end
+    end
+
+    def pause_until_expiry(elapsed)
+      pause_time = (iteration_length - elapsed).to_f
+      if pause_time > 0
+        relax(pause_time)
+      else
+        raise ArgumentError, "Negative iteration time left.  Assuming the worst and exiting... #{iteration_length}/#{elapsed}"
+      end
+    end
+
+    def relax(time)
+      sleep time
+    end
+
+    # Nominate ourself as leader by contacting the memcached server
+    # and attempting to add the token with our name attached.
+    def nominate
+      log.debug("try to nominate")
+      @memcache_client.add(token, @uri, iteration_length)
+      @nominated_at = Time.now
+      @leader_uri = nil
+    end
+
+    def leader_uri
+      @leader_uri ||= @memcache_client.get(token)
+    end
+
+    # Check to see if we are leader by looking at the process name
+    # associated with the token.
+    def leader?
+      until_next_iteration > 0 && @uri == leader_uri
+    end
+
+    # Easy to mock or monkey-patch if another MemCache client is preferred.
+    def client_for(servers)
+      MemCache.new(servers)
+    end
+
+    def time_for(&block)
+      a = Time.now
+      yield
+      Time.now - a
+    end
+
+
+    def register_with_bonjour
+      server = DRb.start_service(nil, self)
+      @uri = DRb.uri
+      @port = URI.parse(DRb.uri).port
+
+      # Register our DRb server with Bonjour.
+      name = "#{self.group_name}-#{local_ip}-#{$$}"
+      type = "_#{group_name}._tcp"
+      domain = "local"
+      log.debug "register service #{name} of type #{type} within domain #{domain} at port #{@port}"
+      handle = Net::DNS::MDNSSD.register(name, type, domain, @port) do |reply|
+        log.debug "registered as #{reply.fullname}"
+      end
+
+      # ['INT', 'TERM'].each { |signal|
+      #   trap(signal) do
+      #     handle.stop
+      #     server.stop_service
+      #   end
+      # }
+    end
+
+    def bonjour_scan
+      Net::DNS::MDNSSD.browse("_#{group_name}._tcp") do |b|
+        Net::DNS::MDNSSD.resolve(b.name, b.type, b.domain) do |r|
+          yield DRbObject.new(nil, "druby://#{r.target}:#{r.port}")
+          unless !@domain || r.target =~ /\.#{@domain}$/
+            yield DRbObject.new(nil, "druby://#{r.target}.#{@domain}:#{r.port}")
+          end
+        end
+      end
+    end
+
+    # http://coderrr.wordpress.com/2008/05/28/get-your-local-ip-address/
+    def local_ip
+      orig, Socket.do_not_reverse_lookup = Socket.do_not_reverse_lookup, true # turn off reverse DNS resolution temporarily
+
+      UDPSocket.open do |s|
+        s.connect '64.233.187.99', 1
+        IPAddr.new(s.addr.last).to_i
+      end
+    ensure
+      Socket.do_not_reverse_lookup = orig
+    end
+
+  end
+end

data/lib/politics/token_worker.rb

@@ -0,0 +1,174 @@
+begin
+  require 'memcache'
+rescue LoadError => e
+  puts "Unable to load memcache client, please run `sudo gem install memcache-client`: #{e.message}"
+  exit(1)
+end
+
+module Politics
+
+  # An algorithm to provide leader election between a set of identical processing daemons.
+  #
+  # Each TokenWorker is an instance which needs to perform some processing.
+  # The worker instance must obtain the leader token before performing some task.
+  # We use a memcached server as a central token authority to provide a shared,
+  # network-wide view for all processors.  This reliance on a single resource means
+  # if your memcached server goes down, so do the processors.  Oftentimes, 
+  # this is an acceptable trade-off since many high-traffic web sites would 
+  # not be useable without memcached running anyhow.
+  #
+  # Essentially each TokenWorker attempts to elect itself every +:iteration_length+
+  # seconds by simply setting a key in memcached to its own name.  Memcached tracks
+  # which name got there first.  The key expires after +:iteration_length+ seconds.
+  #
+  # Example usage:
+  #  class Analyzer
+  #    include Politics::TokenWorker
+  #
+  #    def initialize
+  #      register_worker 'analyzer', :iteration_length => 120, :servers => ['localhost:11211']
+  #    end
+  #
+  #    def start
+  #      process do
+  #        # do analysis here, will only be done when this process
+  #        # is actually elected leader, otherwise it will sleep for
+  #        # iteration_length seconds.
+  #      end
+  #    end
+  #  end
+  #
+  # Notes:
+  # * This will not work with multiple instances in the same Ruby process.
+  #   The library is only designed to elect a leader from a set of processes, not instances within
+  #   a single process.
+  # * The algorithm makes no attempt to keep the same leader during the next iteration.
+  #   This can often times be quite beneficial (e.g. leveraging a warm cache from the last iteration)
+  #   for performance but is left to the reader to implement.
+  module TokenWorker
+    
+    def self.included(model) #:nodoc:
+      model.class_eval do
+        attr_accessor :memcache_client, :token, :iteration_length, :worker_name
+        class << self
+          attr_accessor :worker_instance #:nodoc:
+        end
+      end
+    end
+
+    # Register this instance as a worker.
+    #
+    # Options:
+    # +:iteration_length+::    The length of a processing iteration, in seconds.  The 
+    #                          leader's 'reign' lasts for this length of time.
+    # +:servers+::             An array of memcached server strings
+    def register_worker(name, config={})
+      # track the latest instance of this class, there's really only supposed to be
+      # a single TokenWorker instance per process.
+      self.class.worker_instance = self
+
+      options = { :iteration_length => 60, :servers => ['localhost:11211'] }
+      options.merge!(config)
+
+      self.token = "#{name}_token"
+      self.memcache_client = client_for(Array(options[:servers]))
+      self.iteration_length = options[:iteration_length]
+      self.worker_name = "#{Socket.gethostname}:#{$$}"
+
+      cleanup
+    end
+    
+    def process(*args, &block)
+      verify_registration
+
+      begin
+        # Try to add our name as the worker with the master token.
+        # If another process got there first, this is a noop.
+        # We add an expiry so that the master token will constantly
+        # need to be refreshed (in case the current leader dies).
+        time = 0
+        begin
+          nominate
+
+          if leader?
+            Politics::log.info { "#{worker_name} elected leader at #{Time.now}" }
+            # If we are the master worker, do the work.
+            time = time_for do
+              result = block.call(*args)
+            end
+          end
+        rescue MemCache::MemCacheError => me
+          Politics::log.error("Error from memcached, pausing until the next iteration...")
+          Politics::log.error(me.message)
+          Politics::log.error(me.backtrace.join("\n"))
+          self.memcache_client.reset
+        end
+        
+        pause_until_expiry(time)
+        reset_state
+      end while loop?
+    end
+
+    private
+    
+    def reset_state
+      @leader = nil
+    end
+    
+    def verify_registration
+      unless self.class.worker_instance
+        raise ArgumentError, "Cannot call process without first calling register_worker"
+      end
+      unless self.class.worker_instance == self
+        raise SecurityError, "Only one instance of #{self.class} per process.  Another instance was created after this one."
+      end
+    end
+    
+    def loop?
+      true
+    end
+    
+    def cleanup
+      at_exit do
+        memcache_client.delete(token) if leader?
+      end
+    end
+    
+    def pause_until_expiry(elapsed)
+      pause_time = (iteration_length - elapsed).to_f
+      if pause_time > 0
+        relax(pause_time) 
+      else
+        raise ArgumentError, "Negative iteration time left.  Assuming the worst and exiting... #{iteration_length}/#{elapsed}"
+      end
+    end
+    
+    def relax(time)
+      sleep time
+    end
+    
+    # Nominate ourself as leader by contacting the memcached server
+    # and attempting to add the token with our name attached.
+    # The result will tell us if memcached stored our value and therefore
+    # if we are now leader.
+    def nominate
+      result = memcache_client.add(token, worker_name, (iteration_length * 0.9).to_i)
+      @leader = (result =~ /\ASTORED/)
+    end
+
+    def leader?
+      @leader
+    end
+
+    # Easy to mock or monkey-patch if another MemCache client is preferred.
+    def client_for(servers)
+      MemCache.new(servers)
+    end
+
+    def time_for(&block)
+      a = Time.now
+      yield
+      Time.now - a
+    end
+  end
+end

data/lib/politics/version.rb

@@ -0,0 +1,5 @@
+module Politics
+  module Version
+    STRING = "0.2.3"
+  end
+end

data/lib/politics.rb

@@ -0,0 +1,15 @@
+module Politics
+
+  def self.log=(value)
+    @log = log
+  end
+  
+  def self.log
+    @log ||= if defined?(RAILS_DEFAULT_LOGGER)
+      RAILS_DEFAULT_LOGGER
+    else
+      require 'logger'
+      Logger.new(STDOUT)
+    end
+  end
+end

data/spec/static_queue_worker_spec.rb

@@ -0,0 +1,54 @@
+# FIXME oberen Teil in spec_helper.rb auslagern
+require 'rubygems'
+$:.unshift(File.dirname(__FILE__) + '/../lib')
+require File.dirname(__FILE__) + '/../lib/init'
+Politics::log.level = Logger::FATAL
+
+class Worker
+  include Politics::StaticQueueWorker
+  def initialize
+    log.level = Logger::FATAL
+    register_worker 'worker', 10, :iteration_length => 10
+  end
+
+  def start
+    process_bucket do |bucket|
+      sleep 1
+    end
+  end
+end
+
+describe Worker do
+  before do
+    @worker = Worker.new
+  end
+
+  it "it should provide 'until_next_iteration' even if nominate was not completed" do
+    @worker.until_next_iteration
+  end
+
+  it "should return zero for 'until_next_iteration' if nominate was not completed" do
+    @worker.until_next_iteration.should == 0
+  end
+
+  describe Worker, "when processing bucket" do
+    before do
+      @worker.stub!(:until_next_iteration).and_return 666
+      @worker.should_receive(:loop?).and_return true, true, true, false
+    end
+
+    it "should relax until next iteration on MemCache errors during nomination" do
+      @worker.should_receive(:nominate).at_least(1).and_raise MemCache::MemCacheError.new("Buh!")
+      @worker.should_receive(:relax).with(666).exactly(4).times
+
+      @worker.start
+    end
+
+    it "should relax until next iteration on MemCache errors during request for leader" do
+      @worker.should_receive(:leader_uri).at_least(1).and_raise(MemCache::MemCacheError.new("Buh!"))
+      @worker.should_receive(:relax).with(666).exactly(4).times
+
+      @worker.start
+    end
+  end
+end

data/test/static_queue_worker_test.rb

@@ -0,0 +1,42 @@
+require File.dirname(__FILE__) + '/test_helper'
+
+Thread.abort_on_exception = true
+
+class Worker
+  include Politics::StaticQueueWorker
+  def initialize
+    register_worker 'worker', 10, :iteration_length => 10
+  end
+  
+  def start
+    process_bucket do |bucket|
+      sleep 1
+    end
+  end
+end
+
+class StaticQueueWorkerTest < Test::Unit::TestCase
+  
+  context "nodes" do
+    setup do
+      @nodes = []
+      5.times do
+        @nodes << nil
+      end
+    end
+    
+    should "start up" do
+      processes = @nodes.map do
+        fork do
+          ['INT', 'TERM'].each { |signal| 
+            trap(signal) { exit(0) }
+          }
+          Worker.new.start
+        end
+      end
+      sleep 10
+      puts "Terminating"
+      Process.kill('INT', *processes)
+    end
+  end
+end

data/test/test_helper.rb

@@ -0,0 +1,19 @@
+require 'rubygems'
+require 'test/unit'
+
+begin
+  gem 'thoughtbot-shoulda', '>=2.0.2'
+  require 'shoulda'
+rescue LoadError => e
+  puts "Please install shoulda: `sudo gem install thoughtbot-shoulda -s http://gems.github.com`"
+end
+
+begin
+  require 'mocha'
+rescue LoadError => e
+  puts "Please install mocha: `sudo gem install mocha`"
+end
+
+$:.unshift(File.dirname(__FILE__) + '/../lib')
+require File.dirname(__FILE__) + '/../lib/init'
+Politics::log.level = Logger::WARN

data/test/token_worker_test.rb

@@ -0,0 +1,78 @@
+require 'test_helper'
+
+class TokenWorkerTest < Test::Unit::TestCase
+  
+  context "token workers" do
+    setup do
+      @harness = Class.new
+      @harness.send(:include, Politics::TokenWorker)
+      @harness.any_instance.stubs(:cleanup)
+      @harness.any_instance.stubs(:loop?).returns(false)
+      @harness.any_instance.stubs(:pause_until_expiry)
+      @harness.any_instance.stubs(:relax)
+
+      @worker = @harness.new
+    end
+
+    should "test_instance_property_accessors" do
+      assert @worker.iteration_length = 20
+      assert_equal 20, @worker.iteration_length
+    end
+
+    should 'test_tracks_a_registered_singleton' do
+      assert_nil @worker.class.worker_instance
+      @worker.register_worker('testing')
+      assert_equal @worker.class.worker_instance, @worker
+    end
+    
+    should 'not process if they are not leader' do
+      @worker.expects(:nominate)
+      @worker.expects(:leader?).returns(false)
+      @worker.register_worker('testing')
+      @worker.process do
+        assert false
+      end
+    end
+
+    should 'handle unexpected MemCache errors' do
+      @worker.expects(:nominate)
+      @worker.expects(:leader?).raises(MemCache::MemCacheError)
+      Politics::log.expects(:error).times(3)
+
+      @worker.register_worker('testing')
+      @worker.process do
+        assert false
+      end
+    end
+
+    should 'process if they are leader' do
+      @worker.expects(:nominate)
+      @worker.expects(:leader?).returns(true)
+      @worker.register_worker('testing')
+
+      worked = 0
+      @worker.process do
+        worked += 1
+      end
+
+      assert_equal 1, worked
+    end
+    
+    should 'not allow processing without registration' do
+      assert_raises ArgumentError do
+        @worker.process
+      end
+    end
+
+    should 'not allow processing by old instances' do
+      @worker.register_worker('testing')
+
+      foo = @worker.class.new
+      foo.register_worker('testing')
+      
+      assert_raises SecurityError do
+        @worker.process
+      end
+    end
+  end
+end

metadata

@@ -0,0 +1,98 @@
+--- !ruby/object:Gem::Specification 
+name: infopark-politics
+version: !ruby/object:Gem::Version 
+  version: 0.2.7
+platform: ruby
+authors: 
+- Mike Perham
+- "Tilo Pr\xC3\xBCtz"
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2009-05-29 00:00:00 -07:00
+default_executable: 
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: memcache-client
+  type: :runtime
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: 1.5.0
+    version: 
+- !ruby/object:Gem::Dependency 
+  name: starling-starling
+  type: :runtime
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: 0.9.8
+    version: 
+- !ruby/object:Gem::Dependency 
+  name: net-mdns
+  type: :runtime
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: "0.4"
+    version: 
+description: Algorithms and Tools for Distributed Computing in Ruby.
+email: tilo@infopark.de
+executables: []
+
+extensions: []
+
+extra_rdoc_files: 
+- History.rdoc
+- LICENSE
+- README.rdoc
+files: 
+- examples/queue_worker_example.rb
+- examples/token_worker_example.rb
+- lib/init.rb
+- lib/politics.rb
+- lib/politics/discoverable_node.rb
+- lib/politics/static_queue_worker.rb
+- lib/politics/token_worker.rb
+- lib/politics/version.rb
+- History.rdoc
+- LICENSE
+- README.rdoc
+has_rdoc: true
+homepage: http://github.com/infopark/politics
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+  version: 
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+  version: 
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.2.0
+signing_key: 
+specification_version: 2
+summary: Algorithms and Tools for Distributed Computing in Ruby.
+test_files: 
+- spec/static_queue_worker_spec.rb
+- test/test_helper.rb
+- test/static_queue_worker_test.rb
+- test/token_worker_test.rb