racoon 0.5.0 → 0.6.0

data/README.mdown

@@ -8,7 +8,7 @@ has since taken on a different path. How does it differ from apnserver? By a few
 3. Expects certificates as strings instead of paths to files;
 4. Does not assume there is only one certificate (read: supports multiple projects); and
 5. Receives packets containing notifications from beanstalkd instead of a listening socket;
-6. Operates on a distributed architecture (many parallel workers, one firehose.
+6. Operates on a distributed architecture (many parallel workers, one firehose).
 
 The above changes were made because of the need to replace an existing APNs provider with something
 more robust, and better suited to scaling upwards. This APNs provider had a couple requirements:
@@ -20,6 +20,12 @@ more robust, and better suited to scaling upwards. This APNs provider had a coup
 It should be noted that the development of this project is independent of the work bpoweski
 is doing on apnserver. If you're looking for that project, [go here](https://github.com/bpoweski/apnserver).
 
+## BIT FAT DEPENDENCY WARNING
+
+Since version 0.6.0, Racoon uses ZMQMachine. However, the mainline ZMQMachine does not support PUSH/PULL
+sockets as of the time I wrote this (probably still the same way, else I'd have removed this by now).
+As such, I strongly urge you to [fork this repository](https://github.com/jeremytregunna/zmqmachine) and install it before installing racoon.
+
 ## Description
 
 Racoon consists of a firehose, which maintains the connections to Apple's APNs service. It also

data/bin/racoon-firehose

@@ -6,6 +6,8 @@ require 'rubygems'
 require 'daemons'
 require 'racoon'
 require 'csv'
+require 'yaml'
+require 'zmqmachine'
 
 def usage
   puts "Usage: racoon-firehose [switches]"
@@ -54,6 +56,28 @@ Racoon::Config.logger = Logger.new(@log_file)
 if daemon
   daemonize
 else
-  puts "Starting racoon worker."
+  puts "Starting racoon firehose."
 end
-Racoon::Firehose.new.start!
+
+ZM::Reactor.new(:firehose).run do |context|
+  firehose = Racoon::Firehose.new(context) do |feedback_record|
+    Racoon::Config.logger.info "[Feedback] Time: #{feedback_record[:feedback_at]} Device: #{feedback_record[:device_token]} Length: #{feedback_record[:length]}"
+  end
+  context.pull_socket(firehose)
+
+  context.periodical_timer(28800) do
+    firehose.connections.each_pair do |key, data|
+      begin
+        project = data[:project]
+        uri = "gateway.#{project[:sandbox] ? 'sandbox.' : ''}push.apple.com"
+        feedback = Racoon::APNS::FeedbackConnection.new(data[:certificate], uri)
+        feedback.connect!
+        feedback.read.each do |record|
+          @feedback_callback.call(record) if @feedback_callback
+        end
+      rescue Errno::EPIPE, OpenSSL::SSL::SSLError, Errno::ECONNRESET
+        feedback.disconnect!
+      end
+    end
+  end
+end.join

data/bin/racoon-worker

@@ -6,11 +6,15 @@ require 'rubygems'
 require 'daemons'
 require 'racoon'
 require 'csv'
+require 'yaml'
+require 'zmqmachine'
+require 'beanstalk-client'
 
 def usage
   puts "Usage: racoon-worker [switches]"
   puts " --beanstalk <127.0.0.1:11300>        csv list of ip:port for beanstalk servers"
   puts " --tube <racoon>                      the beanstalk tube to use"
+  puts " --firehose <127.0.0.1:11555>         the address of the firehose"
   puts " --pid </var/run/racoon-worker.pid>   the path to store the pid"
   puts " --log </var/log/racoon-worker.log>   the path to store the log"
   puts " --daemon                             to daemonize the server"
@@ -29,14 +33,16 @@ end
 opts = GetoptLong.new(
   ["--beanstalk", "-b", GetoptLong::REQUIRED_ARGUMENT],
   ["--tube", "-t", GetoptLong::REQUIRED_ARGUMENT],
+  ["--firehose", "-f", GetoptLong::REQUIRED_ARGUMENT],
   ["--pid", "-i", GetoptLong::REQUIRED_ARGUMENT],
   ["--log", "-l", GetoptLong::REQUIRED_ARGUMENT],
   ["--help", "-h", GetoptLong::NO_ARGUMENT],
   ["--daemon", "-d", GetoptLong::NO_ARGUMENT]
 )
 
-beanstalks = ["127.0.0.1:11300"]
-tube = 'racoon'
+@beanstalks = ["127.0.0.1:11300"]
+firehose_address = "127.0.0.1:11555".split(":")
+@tube = 'racoon'
 @pid_file = '/var/run/racoon-worker.pid'
 @log_file = '/var/log/racoon-worker.log'
 daemon = false
@@ -47,9 +53,11 @@ opts.each do |opt, arg|
     usage
     exit 1
   when '--beanstalk'
-    beanstalks = CSV.parse(arg)[0]
+    @beanstalks = CSV.parse(arg)[0]
   when '--tube'
-    tube = arg
+    @tube = arg
+  when '--firehose'
+    firehose_address = arg.split(":")
   when '--pid'
     @pid_file = arg
   when '--log'
@@ -61,9 +69,52 @@ end
 
 Racoon::Config.logger = Logger.new(@log_file)
 
+def beanstalk
+  return @beanstalk if @beanstalk
+  @beanstalk = Beanstalk::Pool.new(@beanstalks)
+  %w{use watch}.each { |s| @beanstalk.send(s, @tube) }
+  @beanstalk.ignore('default')
+  @beanstalk
+end
+
+# Expects json ala:
+# json = {
+#   "project":{
+#      "name":"foobar",
+#      "certificate":"...",
+#      "sandbox":false
+#   },
+#   "bytes":"..."
+# }
+def process(job, worker)
+  packet = job.ybody
+  project = packet[:project]
+
+  notification = Racoon::Notification.create_from_packet(packet)
+
+  data = { :project => project, :bytes => notification.to_bytes }
+  worker.send_message(YAML::dump(data))
+end
+
 if daemon
   daemonize
 else
   puts "Starting racoon worker."
 end
-Racoon::Worker.new(beanstalks, tube).start!
+
+ZM::Reactor.new(:worker).run do |context|
+  worker = Racoon::Worker.new(context, ZM::Address.new(firehose_address[0], firehose_address[1], :tcp))
+  context.push_socket(worker)
+
+  context.periodical_timer(0.1) do
+    begin
+      if beanstalk.peek_ready
+        job = beanstalk.reserve(1)
+        process(job, worker)
+        job.delete
+      end
+    rescue Beanstalk::TimedOut
+      Racoon::Config.logger.info "[Beanstalk] Unable to secure job, timed out."
+    end
+  end
+end.join

data/lib/racoon.rb

@@ -8,5 +8,5 @@ require 'racoon/worker'
 require 'racoon/firehose'
 
 module Racoon
-  VERSION = "0.5.0"
+  VERSION = "0.6.0"
 end

data/lib/racoon/firehose.rb

@@ -5,59 +5,42 @@
 # connections to Apple, and sending data over the right ones.
 
 require 'digest/sha1'
-require 'eventmachine'
-require 'ffi-rzmq'
+require 'zmqmachine'
+require 'yaml'
 
 module Racoon
   class Firehose
-    def initialize(address = "tcp://*:11555", context = ZMQ::Context.new(1), &feedback_callback)
+    attr_accessor :connections, :feedback_callback
+
+    def initialize(reactor, address = ZM::Address.new('*', 11555, :tcp), &feedback_callback)
       @connections = {}
-      @context = context
-      @firehose = context.socket(ZMQ::PULL)
+      @reactor = reactor
       @address = address
       @feedback_callback = feedback_callback
     end
 
-    def start!
-      EventMachine::run do
-        @firehose.bind(@address)
-
-        EventMachine::PeriodicTimer.new(28800) do
-          @connections.each_pair do |key, data|
-            begin
-              uri = "gateway.#{project[:sandbox] ? 'sandbox.' : ''}push.apple.com"
-              feedback = Racoon::APNS::FeedbackConnection.new(data[:certificate], uri)
-              feedback.connect!
-              feedback.read.each do |record|
-                @feedback_callback.call(record) if @feedback_callback
-              end
-            rescue Errno::EPIPE, OpenSSL::SSL::SSLError, Errno::ECONNRESET
-              feedback.disconnect!
-            end
-          end
-        end
-
-        EventMachine::PeriodicTimer.new(0.1) do
-          received_message = ZMQ::Message.new
-          @firehose.recv(received_message, ZMQ::NOBLOCK)
-          yaml_string = received_message.copy_out_string
-          
-          if yaml_string and yaml_string != ""
-            packet = YAML::load(yaml_string)
+    def on_attach(socket)
+      socket.bind(@address)
+    end
 
-            apns(packet[:project], packet[:bytes])
-          end
-        end
+    def on_readable(socket, messages)
+      messages.each do |message|
+        packet = YAML::load(message.copy_out_string)
+        apns(packet[:project], packet[:bytes])
       end
     end
 
+    private
+
     def apns(project, bytes, retries=2)
       uri = "gateway.#{project[:sandbox] ? 'sandbox.' : ''}push.apple.com"
       hash = Digest::SHA1.hexdigest("#{project[:name]}-#{project[:certificate]}")
 
       begin
-        connection = Racoon::APNS::Connection.new(project[:certificate], uri)
-        @connections[hash] ||= { :connection => connection, :certificate => project[:certificate], :sandbox => project[:sandbox] }
+        @connections[hash] ||= { :connection => Racoon::APNS::Connection.new(project[:certificate], uri),
+                                 :certificate => project[:certificate],
+                                 :sandbox => project[:sandbox] }
+        connection = @connections[hash][:connection]
 
         connection.connect! unless connection.connected?
         connection.write(bytes)

data/lib/racoon/worker.rb

@@ -4,73 +4,35 @@
 # This module contains the worker which processes notifications before sending them off
 # down to the firehose.
 
-require 'beanstalk-client'
-require 'eventmachine'
 require 'ffi-rzmq'
-require 'yaml'
+require 'zmqmachine'
 
 module Racoon
   class Worker
-    def initialize(beanstalk_uris, tube = "racoon", address = "tcp://*:11555", context = ZMQ::Context.new(1))
-      @beanstalk_uris = beanstalk_uris
-      @context = context
-      @firehose = context.socket(ZMQ::PUSH)
-      @tube = tube
+    def initialize(reactor, address)
+      @reactor = reactor
       @address = address
-      # First packet, send something silly, the firehose ignores it
-      @send_batch = true
+      @send_queue = []
     end
-    
-    def start!
-      EventMachine::run do
-        @firehose.connect(@address)
 
-        if @send_batch
-          @send_batch = false
-          @firehose.send_string("")
-        end
+    def on_attach(socket)
+      @socket = socket
 
-        EventMachine::PeriodicTimer.new(0.5) do
-          begin
-            if beanstalk.peek_ready
-              job = beanstalk.reserve(1)
-              process job
-              job.delete
-            end
-          rescue Beanstalk::TimedOut
-            Config.logger.info "[Beanstalk] Unable to secure job, operation timed out."
-          end
-        end
-      end
+      socket.connect(@address.to_s)
     end
 
-    private
-
-    def beanstalk
-      return @beanstalk if @beanstalk
-      @beanstalk ||= Beanstalk::Pool.new(@beanstalk_uris)
-      %w{use watch}.each { |s| @beanstalk.send(s, @tube) }
-      @beanstalk.ignore('default')
-      @beanstalk
+    def on_writable(socket)
+      unless @send_queue.empty?
+        message = @send_queue.shift
+        socket.send_message_string(message)
+      else
+        @reactor.deregister_writable(socket)
+      end
     end
 
-    # Expects json ala:
-    # json = {
-    #   "project":{
-    #      "name":"foobar",
-    #      "certificate":"...",
-    #      "sandbox":false
-    #   },
-    #   "bytes":"..."
-    # }
-    def process(job)
-      packet = job.ybody
-      project = packet[:project]
-
-      notification = Notification.create_from_packet(packet)
-
-      data = { :project => project, :bytes => notification.to_bytes }
-      @firehose.send_string(YAML::dump(data))
+    def send_message(message)
+      @send_queue.push(message)
+      @reactor.register_writable(@socket)
     end
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: racoon
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.6.0
   prerelease: 
 platform: ruby
 authors:
@@ -14,7 +14,7 @@ default_executable:
 dependencies:
 - !ruby/object:Gem::Dependency
   name: yajl-ruby
-  requirement: &70355758737600 !ruby/object:Gem::Requirement
+  requirement: &70102154456440 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -22,10 +22,10 @@ dependencies:
         version: 0.7.0
   type: :runtime
   prerelease: false
-  version_requirements: *70355758737600
+  version_requirements: *70102154456440
 - !ruby/object:Gem::Dependency
   name: beanstalk-client
-  requirement: &70355758737040 !ruby/object:Gem::Requirement
+  requirement: &70102154455980 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -33,10 +33,10 @@ dependencies:
         version: 1.0.0
   type: :runtime
   prerelease: false
-  version_requirements: *70355758737040
+  version_requirements: *70102154455980
 - !ruby/object:Gem::Dependency
   name: ffi-rzmq
-  requirement: &70355758736520 !ruby/object:Gem::Requirement
+  requirement: &70102154455520 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -44,10 +44,10 @@ dependencies:
         version: 0.8.0
   type: :runtime
   prerelease: false
-  version_requirements: *70355758736520
+  version_requirements: *70102154455520
 - !ruby/object:Gem::Dependency
   name: bundler
-  requirement: &70355758735740 !ruby/object:Gem::Requirement
+  requirement: &70102154455060 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -55,18 +55,7 @@ dependencies:
         version: 1.0.0
   type: :development
   prerelease: false
-  version_requirements: *70355758735740
-- !ruby/object:Gem::Dependency
-  name: eventmachine
-  requirement: &70355758734300 !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: 0.12.8
-  type: :development
-  prerelease: false
-  version_requirements: *70355758734300
+  version_requirements: *70102154455060
 description: A distributed Apple Push Notification Service (APNs) provider developed
   for hosting multiple projects.
 email: jeremy.tregunna@me.com