racoon 0.4.0 → 0.5.0

data/README.mdown

@@ -6,25 +6,28 @@ has since taken on a different path. How does it differ from apnserver? By a few
 1. It implements the APNS feedback service;
 2. Uses Yajl for JSON encoding/decoding rather than ActiveSupport;
 3. Expects certificates as strings instead of paths to files;
-4. Does not assume there is only one certificate; and
-5. Receives packets containing notifications from beanstalkd instead of a listening socket.
+4. Does not assume there is only one certificate (read: supports multiple projects); and
+5. Receives packets containing notifications from beanstalkd instead of a listening socket;
+6. Operates on a distributed architecture (many parallel workers, one firehose.
 
-The above changes were made because of the need for an APNS provider to replace the current
-provider used by [Diligent Street](http://www.diligentstreet.com/) with something more robust. As such, it needs to be
-suitable for a hosted environment, where multiple—unrelated—users of the service will be
-using it.
+The above changes were made because of the need to replace an existing APNs provider with something
+more robust, and better suited to scaling upwards. This APNs provider had a couple requirements:
+
+1. Support fully the APNs protocol (including feedback)
+2. Scale outwards horizontally
+3. Support multiple projects
 
 It should be noted that the development of this project is independent of the work bpoweski
 is doing on apnserver. If you're looking for that project, [go here](https://github.com/bpoweski/apnserver).
 
 ## Description
 
-racoon is a server and a set of command line programs to send push notifications to iOS devices.
-Apple recommends to maintain an open connection to the push notification service, and refrain
-from opening up and tearing down SSL connections repeatedly. As such, a separate daemon is
-introduced that has messages queued up (beanstalkd) for consumption by this daemon. This
-decouples the APNS server from your backend system. Those notifications are sent over a
-persistent connection to Apple.
+Racoon consists of a firehose, which maintains the connections to Apple's APNs service. It also
+consists of a worker, which works on a beanstalk tube to pop notifications off and process them,
+before sending them off to the firehose. You can run many workers, they all run in parallel to
+one another. Additionally, it includes a command line tool to send (test) the system.
+
+At this time, Racoon only supports Apple's APNs service.
 
 ## Remaining Tasks & Issues
 
@@ -46,99 +49,109 @@ using frac.as, this is the file you would upload to the web service.
 If you're not using frac.as, then the contents of this file are what you need to use as
 your certificate, not the path to the file.
 
-## APN Server Daemon
-
-<pre>
-Usage: racoond [options]
-  --beanstalk <csv ip:port mappings>
-	The comma-separated list of ip:port for beanstalk servers
-
-  --pid <pid file path>
-	Path used for the PID file. Defaults to /var/run/racoon.pid
+## Firehose
 
-  --log <log file path>
-	Path used for the log file. Defaults to /var/log/racoon.log
+The firehose sits at the end of the pipeline. All the workers deliver messages to this
+part of racoon. Think of it as the drain in your sink.
 
-  --help
-    usage message
-
-  --daemon
-    Runs process as daemon, not available on Windows
+<pre>
+Usage: racoon-firehose [switches]
+ --pid </var/run/racoon-firehose.pid>   the path to store the pid
+ --log </var/log/racoon-firehosed.log>  the path to store the log
+ --daemon                               to daemonize the server
+ --help                                 this message
 </pre>
 
-## APN Server Client
-
-TODO: Document this
+## Worker
 
-## Sending Notifications from Ruby
+The worker is the part of the system which interacts with a beanstalk cluster. Each worker
+can talk to more than one beanstalk server, thus forming the "cluster". Since beanstalk
+clustering is all done client side, it's not a traditional cluster, but I'm going to call
+it that way anyway.
 
-You need to set up a connection to the beanstalkd service. We can do this simply by defining
-the following method:
+The worker pops items that are ready off of the beanstalk cluster. It grabs those packets,
+constructs a message out of it, and then forms another packet, suitable for sending to
+Apple, before sending that packet to the firehose.
 
-```ruby
-def beanstalk
-  return @beanstalk if @beanstalk
-  @beanstalk = Beanstalk::Pool.new ["127.0.0.1:11300"]
-  @beanstalk.use "racoon-apns"
-  @beanstalk
-end
-```
+You may run as many workers as your little heart desires. Keep in mind however, there will
+come a point when having only one firehose is not suitable to handle the amount of traffic
+you will be passing through from the workers. At this point, a second firehose will need
+to be started. I doubt anyone will ever be processing enough messages a second to warrant
+a second firehose.
 
-In this way, whenever we need access to beanstalk, we'll make the connection and set up to use
-the appropriate tube whether the connection is open yet or not.
-
-We will also need two pieces of information: A project, and a notification.
+<pre>
+Usage: racoon-worker [switches]
+ --beanstalk <127.0.0.1:11300>        csv list of ip:port for beanstalk servers
+ --pid </var/run/racoon-worker.pid>   the path to store the pid
+ --log </var/log/racoon-worker.log>   the path to store the log
+ --daemon                             to daemonize the server
+ --help                               this message
+</pre>
 
-**Note that the tube should always be "racoon-apns". The server won't look on any other tube for
-push notifications.**
+It should be noted that the worker's `--beanstalk` parameter requires comma separated values
+in `IP:port` pairings. Where `IP` is either an IP address, or a hostname to a host running a
+beanstalkd server on the associated port `port`. In the future, I will create a config file
+instead of having to specify this information on the command line each time. My apologies for
+the inconvenience.
 
-### Project
+## Sender
 
-A project os comprised of a few pieces of information at a minimum (you may supply more if you
-wish, but racoond will ignore them):
+The sender is the program used to form a packet, place it on beanstalk for racoon to consume.
+It is useful during testing.
 
-```ruby
-project = { :name => "Awesome project", :certificate => "contents of the generated .pem file" }
-```
+<pre>
+Usage: racoon-send [switches] (--b64-token | --hex-token) <token>
+ --beanstalk <127.0.0.1:11300>        csv of ip:port for beanstalk servers
+ --pem <path>                         the path to the pem file, if a pem is supplied the server defaults to gateway.push.apple.com:2195
+ --alert <message>                    the message to send
+ --sound <default>                    the sound to play, defaults to 'default'
+ --badge <number>                     the badge number
+ --custom <json string>               a custom json string to be added to the main object
+ --b64-token <token>                  a base 64 encoded device token
+ --hex-token <token>                  a hex encoded device token
+ --help                               this message
+</pre>
 
-### Notification
+## Preparing packets to place on beanstalk
 
-A notification is a ruby hash containing the things to be sent along, including the device token.
-An example notification may look like this:
+Until an API can be built that you can tie into with your own applications, please take care
+to construct your notifications as YAML payload with the following format. I will use ruby
+syntax for this example:
 
 ```ruby
-notification = { :aps => { :alert => "Some text",
-                           :sound => "Audio_file",
-                           :badge => 42,
-                           :custom => { :field => "lala",
-                                        :stuff => 42 }
-                         }
-               }
+{
+  :project => { :name => "My awesome app", :certificate => "...", :sandbox => true },
+  :identifier => 12345,
+  :notification => { :aps => { :alert => "text",
+                               :sound => "default",
+                               :badge => 1,
+                               :custom => { ... }
+                             }
+                   },
+  :device_token => "..."
+}
 ```
 
-Finally within we can send a push notification using the following code:
+A few key points need to be raised here. For starters, the `sandbox` key should only be true if
+you desire to work in the sandbox, and your `certificate` contains the text of your development
+certificate.
 
-```ruby
-beanstalk.yput({ :project => project,
-                 :notification => notification,
-                 :send_at => Time.mktime(2012, 12, 21, 4, 15)
-                 :device_token => "binary encoded token",
-                 :sandbox => true
-})
-```
+Secondly, the `identifier` must be a unique 32-bit number identifying your message should you
+choose to want useful error messages. (This feature is not presently written, as such, you can
+supply anything you want, just make sure it falls within the range of `0` to `4294967295`.
 
-This will construct a notification which is to be scheduled to be delivered on December 21st,
-2012 at 4:15 AM localtime to the server. That is, if the server's timezone is UTC, you must
-account for that in your client application.
+The `notification` key represents the payload we'll send to Apple. The `custom` key must be
+present if you intend to send custom data. Note however, that `custom` will be removed, and the
+items you place in its hash will be substituted in with the payload when the message is passed
+to Apple. As such, if you want a custom key -> value pair of: `"foo" => "bar"`, you would ensure
+you have: `:custom => { "foo" => "bar" }` in the notification. Your application should just look
+for "foo" in the payload delivered to the app.
 
-Note that the `sandbox` parameter is used to indicate whether or not we're using a development
-certificate, and as such, should contact Apple's sandbox APN service instead of the production
-certificate. If left out, we assume production.
+Finally, the ```device_token``` is a binary encoded representation of your devices token. Your
+app gets it in hex, please ensure you convert it to binary before sending it to beanstalk.
 
-This will schedule the push on beanstalkd. Racoon is constantly polling beanstalkd looking for
-ready jobs it can pop off and process (send to Apple). Using beanstalkd however allows us to
-queue up items, and during peak times, add another **N** more racoon servers to make up any
-backlog, to ensure our messages are sent fast, and that we can scale.
+TODO: Document new way of handling scheduling notifications in the future. Abstract beanstalk
+away from users of the library, give them a proper client interface to use instead.
 
 ## Installation
 

data/bin/racoon-firehose

@@ -0,0 +1,59 @@
+#!/usr/bin/env ruby
+$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
+
+require 'getoptlong'
+require 'rubygems'
+require 'daemons'
+require 'racoon'
+require 'csv'
+
+def usage
+  puts "Usage: racoon-firehose [switches]"
+  puts " --pid </var/run/racoon-firehose.pid>   the path to store the pid"
+  puts " --log </var/log/racoon-firehosed.log>  the path to store the log"
+  puts " --daemon                               to daemonize the server"
+  puts " --help                                 this message"
+end
+
+def daemonize
+  Daemonize.daemonize(@log_file, 'racoon-firehose')
+  open(@pid_file,"w") { |f| f.write(Process.pid) }
+  open(@pid_file,"w") do |f|
+    f.write(Process.pid)
+    File.chmod(0644, @pid_file)
+  end
+end
+
+opts = GetoptLong.new(
+  ["--pid", "-i", GetoptLong::REQUIRED_ARGUMENT],
+  ["--log", "-l", GetoptLong::REQUIRED_ARGUMENT],
+  ["--help", "-h", GetoptLong::NO_ARGUMENT],
+  ["--daemon", "-d", GetoptLong::NO_ARGUMENT]
+)
+
+@pid_file = '/var/run/racoon-firehose.pid'
+@log_file = '/var/log/racoon-firehose.log'
+daemon = false
+
+opts.each do |opt, arg|
+  case opt
+  when '--help'
+    usage
+    exit 1
+  when '--pid'
+    @pid_file = arg
+  when '--log'
+    @log_file = arg
+  when '--daemon'
+    daemon = true
+  end
+end
+
+Racoon::Config.logger = Logger.new(@log_file)
+
+if daemon
+  daemonize
+else
+  puts "Starting racoon worker."
+end
+Racoon::Firehose.new.start!

data/bin/racoon-send

@@ -14,6 +14,7 @@ require 'beanstalk-client'
 def usage
   puts "Usage: racoon-send [switches] (--b64-token | --hex-token) <token>"
   puts " --beanstalk <127.0.0.1:11300>        csv of ip:port for beanstalk servers"
+  puts " --tube <racoon>                      the beanstalk tube to use"
   puts " --pem <path>                         the path to the pem file, if a pem is supplied the server defaults to gateway.push.apple.com:2195"
   puts " --alert <message>                    the message to send"
   puts " --sound <default>                    the sound to play, defaults to 'default'"
@@ -26,6 +27,7 @@ end
 
 opts = GetoptLong.new(
   ["--beanstalk", "-b", GetoptLong::REQUIRED_ARGUMENT],
+  ["--tube", "-t", GetoptLong::REQUIRED_ARGUMENT],
   ["--pem", "-c", GetoptLong::REQUIRED_ARGUMENT],
   ["--alert", "-a", GetoptLong::REQUIRED_ARGUMENT],
   ["--sound", "-S", GetoptLong::REQUIRED_ARGUMENT],
@@ -37,6 +39,7 @@ opts = GetoptLong.new(
 )
 
 beanstalks = ["127.0.0.1:11300"]
+tube = 'racoon'
 certificate = nil
 notification = Racoon::Notification.new
 
@@ -47,6 +50,8 @@ opts.each do |opt, arg|
     exit
   when '--beanstalk'
     beanstalks = CSV.parse(arg)[0]
+  when '--tube'
+    tube = arg
   when '--pem'
     certificate = File.read(arg)
   when '--alert'
@@ -69,12 +74,13 @@ if notification.device_token.nil?
   exit
 else
   bs = Beanstalk::Pool.new beanstalks
-  %w{use watch}.each { |s| bs.send(s, "racoon-apns") }
+  %w{use watch}.each { |s| bs.send(s, tube) }
   bs.ignore("default")
-  project = { :name => "test", :certificate => certificate }
-  bs.yput({ :project => project,
+  project = { :name => "test", :certificate => certificate, :sandbox => true }
+  notif = { :project => project,
+            :identifier => 1,
             :notification => notification.payload,
             :device_token => notification.device_token,
-            :sandbox => true
-  })
+  }
+  bs.yput(notif)
 end

data/bin/{racoond → racoon-worker}

@@ -4,22 +4,21 @@ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
 require 'getoptlong'
 require 'rubygems'
 require 'daemons'
-require 'eventmachine'
 require 'racoon'
-require 'racoon/server'
 require 'csv'
 
 def usage
-  puts "Usage: racoond [switches] --beanstalk a.b.c.d:11300,...,w.x.y.z:11300"
+  puts "Usage: racoon-worker [switches]"
   puts " --beanstalk <127.0.0.1:11300>        csv list of ip:port for beanstalk servers"
-  puts " --pid </var/run/racoond.pid>         the path to store the pid"
-  puts " --log </var/log/racoond.log>         the path to store the log"
+  puts " --tube <racoon>                      the beanstalk tube to use"
+  puts " --pid </var/run/racoon-worker.pid>   the path to store the pid"
+  puts " --log </var/log/racoon-worker.log>   the path to store the log"
   puts " --daemon                             to daemonize the server"
   puts " --help                               this message"
 end
 
 def daemonize
-  Daemonize.daemonize(@log_file, 'racoond')
+  Daemonize.daemonize(@log_file, 'racoon-worker')
   open(@pid_file,"w") { |f| f.write(Process.pid) }
   open(@pid_file,"w") do |f|
     f.write(Process.pid)
@@ -29,6 +28,7 @@ end
 
 opts = GetoptLong.new(
   ["--beanstalk", "-b", GetoptLong::REQUIRED_ARGUMENT],
+  ["--tube", "-t", GetoptLong::REQUIRED_ARGUMENT],
   ["--pid", "-i", GetoptLong::REQUIRED_ARGUMENT],
   ["--log", "-l", GetoptLong::REQUIRED_ARGUMENT],
   ["--help", "-h", GetoptLong::NO_ARGUMENT],
@@ -36,8 +36,9 @@ opts = GetoptLong.new(
 )
 
 beanstalks = ["127.0.0.1:11300"]
-@pid_file = '/var/run/racoond.pid'
-@log_file = '/var/log/racoond.log'
+tube = 'racoon'
+@pid_file = '/var/run/racoon-worker.pid'
+@log_file = '/var/log/racoon-worker.log'
 daemon = false
 
 opts.each do |opt, arg|
@@ -47,6 +48,8 @@ opts.each do |opt, arg|
     exit 1
   when '--beanstalk'
     beanstalks = CSV.parse(arg)[0]
+  when '--tube'
+    tube = arg
   when '--pid'
     @pid_file = arg
   when '--log'
@@ -58,8 +61,9 @@ end
 
 Racoon::Config.logger = Logger.new(@log_file)
 
-daemonize if daemon
-server = Racoon::Server.new(beanstalks) do |feedback_record|
-  Racoon::Config.logger.info "Received feedback at #{feedback_record[:feedback_at]} (length: #{feedback_record[:length]}): #{feedback_record[:device_token]}"
+if daemon
+  daemonize
+else
+  puts "Starting racoon worker."
 end
-server.start!
+Racoon::Worker.new(beanstalks, tube).start!

data/lib/racoon.rb

@@ -2,9 +2,11 @@ require 'logger'
 require 'racoon/config'
 require 'racoon/payload'
 require 'racoon/notification'
-require 'racoon/client'
-require 'racoon/feedback_client'
+require 'racoon/apns/connection'
+require 'racoon/apns/feedback_connection'
+require 'racoon/worker'
+require 'racoon/firehose'
 
 module Racoon
-  VERSION = "0.4.0"
+  VERSION = "0.5.0"
 end

data/lib/racoon/apns/connection.rb

@@ -0,0 +1,52 @@
+# Racoon - A distributed APNs provider
+# Copyright (c) 2011, Jeremy Tregunna, All Rights Reserved.
+#
+# This module contains the connection to the APNs service.
+
+require 'openssl'
+require 'socket'
+
+module Racoon
+  module APNS
+    class Connection
+      attr_accessor :pem, :host, :port, :password
+
+      def initialize(pem, host = 'gateway.push.apple.com', port = 2195, pass = nil)
+        @pem, @host, @port, @password = pem, host, port, pass
+      end
+
+      def connect!
+        raise "Your certificate is not set." unless self.pem
+
+        @context      = OpenSSL::SSL::SSLContext.new
+        @context.cert = OpenSSL::X509::Certificate.new(self.pem)
+        @context.key  = OpenSSL::PKey::RSA.new(self.pem, self.password)
+
+        @sock         = TCPSocket.new(self.host, self.port.to_i)
+        @ssl          = OpenSSL::SSL::SSLSocket.new(@sock, @context)
+        @ssl.connect
+
+        return @sock, @ssl
+      end
+
+      def disconnect!
+        @ssl.close
+        @sock.close
+        @ssl = nil
+        @sock = nil
+      end
+
+      def write(bytes)
+        if host.include? "sandbox"
+          notification = Notification.parse(bytes)
+          Config.logger.debug "#{Time.now} [#{host}:#{port}] Device: #{notification.device_token.unpack('H*')} sending #{notification.json_payload}"
+        end
+        @ssl.write(notification.to_bytes)
+      end
+
+      def connected?
+        @ssl
+      end
+    end
+  end
+end

data/lib/racoon/apns/feedback_connection.rb

@@ -0,0 +1,29 @@
+# Racoon - A distributed APNs provider
+# Copyright (c) 2011, Jeremy Tregunna, All Rights Reserved.
+#
+# This module contains the code that connects to the feedback service.
+
+module Racoon
+  module APNS
+    class FeedbackConnection < Connection
+      def initialize(pem, host = 'feedback.push.apple.com', port = 2196, pass = nil)
+        @pem, @host, @port, @pass = pem, host, port, pass
+      end
+
+      def read
+        records ||= []
+        while record = @ssl.read(38)
+          records << parse_tuple(record)
+        end
+        records
+      end
+
+      private
+
+      def parse_tuple(data)
+        feedback = data.unpack("N1n1H*")
+        { :feedback_at => Time.at(feedback[0]), :length => feedback[1], :device_token => feedback[2] }
+      end
+    end
+  end
+end

data/lib/racoon/config.rb

@@ -1,3 +1,8 @@
+# Racoon - A distributed APNs provider
+# Copyright (c) 2011, Jeremy Tregunna, All Rights Reserved.
+#
+# Configuration settings.
+
 module Racoon
   class Config
     class << self

data/lib/racoon/firehose.rb

@@ -0,0 +1,70 @@
+# Racoon - A distributed APNs provider
+# Copyright (c) 2011, Jeremy Tregunna, All Rights Reserved.
+#
+# This module contains the firehose which is responsible for maintaining all the open
+# connections to Apple, and sending data over the right ones.
+
+require 'digest/sha1'
+require 'eventmachine'
+require 'ffi-rzmq'
+
+module Racoon
+  class Firehose
+    def initialize(address = "tcp://*:11555", context = ZMQ::Context.new(1), &feedback_callback)
+      @connections = {}
+      @context = context
+      @firehose = context.socket(ZMQ::PULL)
+      @address = address
+      @feedback_callback = feedback_callback
+    end
+
+    def start!
+      EventMachine::run do
+        @firehose.bind(@address)
+
+        EventMachine::PeriodicTimer.new(28800) do
+          @connections.each_pair do |key, data|
+            begin
+              uri = "gateway.#{project[:sandbox] ? 'sandbox.' : ''}push.apple.com"
+              feedback = Racoon::APNS::FeedbackConnection.new(data[:certificate], uri)
+              feedback.connect!
+              feedback.read.each do |record|
+                @feedback_callback.call(record) if @feedback_callback
+              end
+            rescue Errno::EPIPE, OpenSSL::SSL::SSLError, Errno::ECONNRESET
+              feedback.disconnect!
+            end
+          end
+        end
+
+        EventMachine::PeriodicTimer.new(0.1) do
+          received_message = ZMQ::Message.new
+          @firehose.recv(received_message, ZMQ::NOBLOCK)
+          yaml_string = received_message.copy_out_string
+          
+          if yaml_string and yaml_string != ""
+            packet = YAML::load(yaml_string)
+
+            apns(packet[:project], packet[:bytes])
+          end
+        end
+      end
+    end
+
+    def apns(project, bytes, retries=2)
+      uri = "gateway.#{project[:sandbox] ? 'sandbox.' : ''}push.apple.com"
+      hash = Digest::SHA1.hexdigest("#{project[:name]}-#{project[:certificate]}")
+
+      begin
+        connection = Racoon::APNS::Connection.new(project[:certificate], uri)
+        @connections[hash] ||= { :connection => connection, :certificate => project[:certificate], :sandbox => project[:sandbox] }
+
+        connection.connect! unless connection.connected?
+        connection.write(bytes)
+      rescue Errno::EPIPE, OpenSSL::SSL::SSLError, Errno::ECONNRESET
+        connection.disconnect!
+        retry if (retries -= 1) > 0
+      end
+    end
+  end
+end

data/lib/racoon/notification.rb

@@ -1,3 +1,8 @@
+# Racoon - A distributed APNs provider
+# Copyright (c) 2011, Jeremy Tregunna, All Rights Reserved.
+#
+# This module contains the class that represents notifications and all their details.
+
 require 'racoon/payload'
 require 'base64'
 require 'yajl'
@@ -6,11 +11,10 @@ module Racoon
   class Notification
     include Racoon::Payload
 
-    attr_accessor :identifier, :expiry, :device_token, :alert, :badge, :sound, :custom, :send_at, :expiry
+    attr_accessor :identifier, :expiry, :device_token, :alert, :badge, :sound, :custom, :expiry
 
     def initialize
       @expiry = 0
-      @send_at = Time.now
     end
 
     def payload
@@ -75,5 +79,20 @@ module Racoon
 
       notification
     end
+
+    def self.create_from_packet(packet)
+      aps = packet[:notification][:aps]
+
+      notification = Notification.new
+      notification.identifier = packet[:identifier]
+      notification.expiry = packet[:expiry] || 0
+      notification.device_token = packet[:device_token]
+      notification.badge = aps[:badge] if aps.has_key? :badge
+      notification.alert = aps[:alert] if aps.has_key? :alert
+      notification.sound = aps[:sound] if aps.has_key? :sound
+      notification.custom = aps[:custom] if aps.has_key? :custom
+
+      notification
+    end
   end
 end

data/lib/racoon/payload.rb

@@ -1,3 +1,8 @@
+# Racoon - A distributed APNs provider
+# Copyright (c) 2011, Jeremy Tregunna, All Rights Reserved.
+#
+# APNs payload data
+
 module Racoon
   module Payload
     PayloadInvalid = Class.new(RuntimeError)

data/lib/racoon/worker.rb

@@ -0,0 +1,76 @@
+# Racoon - A distributed APNs provider
+# Copyright (c) 2011, Jeremy Tregunna, All Rights Reserved.
+#
+# This module contains the worker which processes notifications before sending them off
+# down to the firehose.
+
+require 'beanstalk-client'
+require 'eventmachine'
+require 'ffi-rzmq'
+require 'yaml'
+
+module Racoon
+  class Worker
+    def initialize(beanstalk_uris, tube = "racoon", address = "tcp://*:11555", context = ZMQ::Context.new(1))
+      @beanstalk_uris = beanstalk_uris
+      @context = context
+      @firehose = context.socket(ZMQ::PUSH)
+      @tube = tube
+      @address = address
+      # First packet, send something silly, the firehose ignores it
+      @send_batch = true
+    end
+    
+    def start!
+      EventMachine::run do
+        @firehose.connect(@address)
+
+        if @send_batch
+          @send_batch = false
+          @firehose.send_string("")
+        end
+
+        EventMachine::PeriodicTimer.new(0.5) do
+          begin
+            if beanstalk.peek_ready
+              job = beanstalk.reserve(1)
+              process job
+              job.delete
+            end
+          rescue Beanstalk::TimedOut
+            Config.logger.info "[Beanstalk] Unable to secure job, operation timed out."
+          end
+        end
+      end
+    end
+
+    private
+
+    def beanstalk
+      return @beanstalk if @beanstalk
+      @beanstalk ||= Beanstalk::Pool.new(@beanstalk_uris)
+      %w{use watch}.each { |s| @beanstalk.send(s, @tube) }
+      @beanstalk.ignore('default')
+      @beanstalk
+    end
+
+    # Expects json ala:
+    # json = {
+    #   "project":{
+    #      "name":"foobar",
+    #      "certificate":"...",
+    #      "sandbox":false
+    #   },
+    #   "bytes":"..."
+    # }
+    def process(job)
+      packet = job.ybody
+      project = packet[:project]
+
+      notification = Notification.create_from_packet(packet)
+
+      data = { :project => project, :bytes => notification.to_bytes }
+      @firehose.send_string(YAML::dump(data))
+    end
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: racoon
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.5.0
   prerelease: 
 platform: ruby
 authors:
@@ -14,7 +14,7 @@ default_executable:
 dependencies:
 - !ruby/object:Gem::Dependency
   name: yajl-ruby
-  requirement: &70324876524860 !ruby/object:Gem::Requirement
+  requirement: &70355758737600 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -22,10 +22,10 @@ dependencies:
         version: 0.7.0
   type: :runtime
   prerelease: false
-  version_requirements: *70324876524860
+  version_requirements: *70355758737600
 - !ruby/object:Gem::Dependency
   name: beanstalk-client
-  requirement: &70324876524360 !ruby/object:Gem::Requirement
+  requirement: &70355758737040 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -33,10 +33,21 @@ dependencies:
         version: 1.0.0
   type: :runtime
   prerelease: false
-  version_requirements: *70324876524360
+  version_requirements: *70355758737040
+- !ruby/object:Gem::Dependency
+  name: ffi-rzmq
+  requirement: &70355758736520 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 0.8.0
+  type: :runtime
+  prerelease: false
+  version_requirements: *70355758736520
 - !ruby/object:Gem::Dependency
   name: bundler
-  requirement: &70324876523860 !ruby/object:Gem::Requirement
+  requirement: &70355758735740 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -44,10 +55,10 @@ dependencies:
         version: 1.0.0
   type: :development
   prerelease: false
-  version_requirements: *70324876523860
+  version_requirements: *70355758735740
 - !ruby/object:Gem::Dependency
   name: eventmachine
-  requirement: &70324876523360 !ruby/object:Gem::Requirement
+  requirement: &70355758734300 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -55,27 +66,30 @@ dependencies:
         version: 0.12.8
   type: :development
   prerelease: false
-  version_requirements: *70324876523360
-description: A toolkit for proxying and sending Apple Push Notifications prepared
-  for a hosted environment
+  version_requirements: *70355758734300
+description: A distributed Apple Push Notification Service (APNs) provider developed
+  for hosting multiple projects.
 email: jeremy.tregunna@me.com
 executables:
 - racoon-send
-- racoond
+- racoon-worker
+- racoon-firehose
 extensions: []
 extra_rdoc_files:
 - README.mdown
 files:
 - bin/apnserverd.fedora.init
 - bin/apnserverd.ubuntu.init
+- bin/racoon-firehose
 - bin/racoon-send
-- bin/racoond
-- lib/racoon/client.rb
+- bin/racoon-worker
+- lib/racoon/apns/connection.rb
+- lib/racoon/apns/feedback_connection.rb
 - lib/racoon/config.rb
-- lib/racoon/feedback_client.rb
+- lib/racoon/firehose.rb
 - lib/racoon/notification.rb
 - lib/racoon/payload.rb
-- lib/racoon/server.rb
+- lib/racoon/worker.rb
 - lib/racoon.rb
 - README.mdown
 - spec/models/client_spec.rb
@@ -108,7 +122,7 @@ rubyforge_project: racoon
 rubygems_version: 1.6.2
 signing_key: 
 specification_version: 3
-summary: Apple Push Notification Toolkit for hosted environments
+summary: Distributed Apple Push Notification provider suitable for multi-project hosting.
 test_files:
 - spec/models/client_spec.rb
 - spec/models/feedback_client_spec.rb

data/lib/racoon/client.rb

@@ -1,44 +0,0 @@
-require 'openssl'
-require 'socket'
-
-module Racoon
-  class Client
-    attr_accessor :pem, :host, :port, :password
-
-    def initialize(pem, host = 'gateway.push.apple.com', port = 2195, pass = nil)
-      @pem, @host, @port, @password = pem, host, port, pass
-    end
-
-    def connect!
-      raise "Your certificate is not set." unless self.pem
-
-      @context      = OpenSSL::SSL::SSLContext.new
-      @context.cert = OpenSSL::X509::Certificate.new(self.pem)
-      @context.key  = OpenSSL::PKey::RSA.new(self.pem, self.password)
-
-      @sock         = TCPSocket.new(self.host, self.port.to_i)
-      @ssl          = OpenSSL::SSL::SSLSocket.new(@sock, @context)
-      @ssl.connect
-
-      return @sock, @ssl
-    end
-
-    def disconnect!
-      @ssl.close
-      @sock.close
-      @ssl = nil
-      @sock = nil
-    end
-
-    def write(notification)
-      if host.include? "sandbox"
-        Config.logger.debug "#{Time.now} [#{host}:#{port}] Device: #{notification.device_token.unpack('H*')} sending #{notification.json_payload}"
-      end
-      @ssl.write(notification.to_bytes)
-    end
-
-    def connected?
-      @ssl
-    end
-  end
-end

data/lib/racoon/feedback_client.rb

@@ -1,24 +0,0 @@
-# Feedback service
-
-module Racoon
-  class FeedbackClient < Client
-    def initialize(pem, host = 'feedback.push.apple.com', port = 2196, pass = nil)
-      @pem, @host, @port, @pass = pem, host, port, pass
-    end
-
-    def read
-      records ||= []
-      while record = @ssl.read(38)
-        records << parse_tuple(record)
-      end
-      records
-    end
-
-    private
-
-    def parse_tuple(data)
-      feedback = data.unpack("N1n1H*")
-      { :feedback_at => Time.at(feedback[0]), :length => feedback[1], :device_token => feedback[2] }
-    end
-  end
-end

data/lib/racoon/server.rb

@@ -1,193 +0,0 @@
-require 'beanstalk-client'
-
-module Racoon
-  class Server
-    attr_accessor :beanstalkd_uris, :feedback_callback
-
-    def initialize(beanstalkd_uris = ["127.0.0.1:11300"], &feedback_blk)
-      @beanstalks = {}
-      @clients = {}
-      @feedback_callback = feedback_blk
-      @beanstalkd_uris = beanstalkd_uris
-    end
-
-    def beanstalk(arg)
-      tube = "racoon-#{arg}"
-      return @beanstalks[tube] if @beanstalks[tube]
-      @beanstalks[tube] = Beanstalk::Pool.new @beanstalkd_uris
-      @beanstalks[tube]
-    end
-
-    def start!
-      EventMachine::run do
-        EventMachine::PeriodicTimer.new(3600) do
-          begin
-            b = beanstalk('feedback')
-            %w{watch use}.each { |s| b.send(s, "racoon-feedback") }
-            b.ignore('default')
-            if b.peek_ready
-              item = b.reserve(1)
-              handle_feedback(item)
-            end
-          rescue Beanstalk::TimedOut
-            Config.logger.info "(Beanstalkd) Unable to secure a job, operation timed out."
-          end
-        end
-
-        # Every minute,poll all the clients, ensuring they've been inactive for 20+ minutes.
-        EventMachine::PeriodicTimer.new(60) do
-          remove_clients = []
-
-          @clients.each_pair do |project_name, packet|
-            if Time.now - packet[:timestamp] >= 1200 # 20 minutes
-              packet[:connection].disconnect!
-              remove_clients << project_name
-            end
-          end
-
-          remove_clients.each do |project_name|
-            @clients[project_name] = nil
-          end
-        end
-
-        EventMachine::PeriodicTimer.new(2) do
-          begin
-            b = beanstalk('apns')
-            %w{watch use}.each { |s| b.send(s, "racoon-apns") }
-            b.ignore('default')
-            jobs = []
-            until b.peek_ready.nil?
-              item = b.reserve(1)
-              jobs << item
-            end
-            handle_jobs jobs if jobs.count > 0
-          rescue Beanstalk::TimedOut
-            Config.logger.info "(Beanstalkd) Unable to secure a job, operation timed out."
-          end
-        end
-      end
-    end
-
-    private
-
-    # Received a notification. job is YAML encoded hash in the following format:
-    # job = {
-    #   :project => {
-    #     :name => "Foo",
-    #     :certificate => "contents of a certificate.pem"
-    #   },
-    #   :device_token => "0f21ab...def",
-    #   :notification => notification.payload,
-    #   :sandbox => true # Development environment?
-    # }
-    def handle_jobs(jobs)
-      connections = {}
-      jobs.each do |job|
-        packet = job.ybody
-        project = packet[:project]
-
-        client = get_client(project[:name], project[:certificate], packet[:sandbox])
-        conn = client[:connection]
-        connections[conn] ||= []
-
-        notification = create_notification_from_packet(packet)
-
-        connections[conn] << { :job => job, :notification => notification }
-      end
-
-      connections.each_pair do |conn, tasks|
-        conn.connect! unless conn.connected?
-        tasks.each do |data|
-          job = data[:job]
-          notif = data[:notification]
-
-          begin
-            conn.write(notif)
-            @clients[project[:name]][:timestamp] = Time.now
-
-            # TODO: Listen for error responses from Apple
-            job.delete
-          rescue Errno::EPIPE, OpenSSL::SSL::SSLError, Errno::ECONNRESET
-            Config.logger.error "Caught error, closing connection and adding notification back to queue"
-
-            connection.disconnect!
-
-            job.release
-          rescue RuntimeError => e
-            Config.logger.error "Unable to handle: #{e}"
-
-            job.delete
-          end
-        end
-      end
-    end
-
-    # Will be a hash with two keys:
-    # :certificate and :sandbox.
-    def handle_feedback(job)
-      begin
-        packet = job.ybody
-        uri = "feedback.#{packet[:sandbox] ? 'sandbox.' : ''}push.apple.com"
-        feedback_client = Racoon::FeedbackClient.new(packet[:certificate], uri)
-        feedback_client.connect!
-        feedback_client.read.each do |record|
-          feedback_callback.call record
-        end
-        feedback_client.disconnect!
-        job.delete
-      rescue Errno::EPIPE, OpenSSL::SSL::SSLError, Errno::ECONNRESET
-        Config.logger.error "(Feedback) Caught Error, closing connection"
-        feedback_client.disconnect!
-        job.release
-      rescue RuntimeError => e
-        Config.logger.error "(Feedback) Unable to handle: #{e}"
-        job.delete
-      end
-    end
-
-    # Returns a hash containing a timestamp referring to when the connection was opened.
-    # This timestamp will be updated to reflect when there was last activity over the socket.
-    def get_client(project_name, certificate, sandbox = false)
-      uri = "gateway.#{sandbox ? 'sandbox.' : ''}push.apple.com"
-      unless @clients[project_name]
-        @clients[project_name] = { :timestamp => Time.now, :connection => Racoon::Client.new(certificate, uri) }
-      end
-      @clients[project_name] ||= { :timestamp => Time.now, :connection => Racoon::Client.new(certificate, uri) }
-      client = @clients[project_name]
-      connection = client[:connection]
-
-      # If the certificate has changed, but we still are connected using the old certificate,
-      # disconnect and reconnect.
-      unless connection.pem.eql?(certificate)
-        connection.disconnect! if connection.connected?
-        @clients[project_name] = { :timestamp => Time.now, :connection => Racoon::Client.new(certificate, uri) }
-        client = @clients[project_name]
-      end
-
-      client
-    end
-
-    def purge_client(job)
-      project_name = job.ybody
-      client = @clients[project_name]
-      client.disconnect! if client
-      @clients[project_name] = nil
-      job.delete
-    end
-
-    def create_notification_from_packet(packet)
-      aps = packet[:notification][:aps]
-
-      notification = Notification.new
-      notification.identifier = packet[:identifier]
-      notification.expiry = packet[:expiry]
-      notification.device_token = packet[:device_token]
-      notification.badge = aps[:badge] if aps.has_key? :badge
-      notification.alert = aps[:alert] if aps.has_key? :alert
-      notification.sound = aps[:sound] if aps.has_key? :sound
-      notification.custom = aps[:custom] if aps.has_key? :custom
-
-      notification
-    end
-  end
-end