haredo 0.0.2 → 0.0.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 85222fb487a3ac64174d10d323211ba37ddf427b
-  data.tar.gz: 758c7117f150c96b882132d14d38e2e0495ea080
+  metadata.gz: 3a240cc2da3eb83e870575dfd20ddeb87f7fe7eb
+  data.tar.gz: b0285cbf74adfa83800c08c3cd06b7d841c52a07
 SHA512:
-  metadata.gz: ac949da33da127bdf7b1b587c346349a5abdb8e5ccf30c4bc18c57613fa1729c50b2776bad73357ae3852f326e906870daecacacc745ed0006f42f4bab6f9141
-  data.tar.gz: 2a5d9416907828acbcdf9d6a4e8259b56b06d82025eb4901e2acecbd39488a41f0ef4a32af4b20258c54e45b8ab17e3503fe56d9da8e0f10681d83e3e3987f83
+  metadata.gz: 192aaeeaa0eb4e7e22f881c29fe60b793b8b1769565b18a5311a13baaab7826f4f4123777c90711315fcb068c94ddc1a35cf57341ec41687cd4daf9c164d8d5d
+  data.tar.gz: 38dbeacc3a6073722226b37c2d98b62c1e9a66a48f30b2c58c07b45805b4d4f68d7dd73f4a3f5447f41731a492150c22ec0a3e9d8a5a085b64bfb2f82d1295b3

data/README.md

@@ -3,36 +3,53 @@
 ## About
 
 This is an easy-to-use framework for creating peer-to-peer applications in Ruby
-via [RabbitMQ](http://www.rabbitmq.com/). While RabbitMQ is a robust, feature
-rich message broker, it (like all AMQP message brokers) uses a somewhat complex
-terminology that can be a little bewildering to to newcomers of AMQP.
-
-This gem -- built atop the [Bunny AMQP client](http://rubybunny.info/) -- uses a
-small subset of RabbitMQ's capabilities to provide an intuitive client/server
-framework for easily and quickly implementing network services and simple peer
-to peer applications. It specifically designed for those who are not familiar
-with the ins and outs of the AMQP protocol.
+via [RabbitMQ](http://www.rabbitmq.com/). Built atop the [Bunny AMQP
+client](http://rubybunny.info/), it uses a small subset of RabbitMQ's
+capabilities to provide an intuitive client/server framework for implementing
+network services and simple peer to peer applications. It is specifically
+designed for those who are not familiar with the ins and outs of the AMQP
+protocol and employs RabbitMQ's default AMQP exchange to implement direct peer
+to peer communication.
+
+This framework's design targets a common case within web applications and other
+miscellaneous contexts (logging, event handling, data retrieval, etc.), when you
+just need to send a request somewhere and get a reply back. In this case
+specifically:
+
+* Clients send messages asynchronously (to one or more peers) and can receive
+  synchronously (blocking with a configurable timeout).
+* Services run indefinitely either within the same process (in separate
+  threads), or separate processes.
+
+In summary, this gem employs a small, practical and useful subset of the vast
+array of features available within RabbitMQ to make it easy to whip up network
+services quickly and easily without having to master all aspects of RabbitMQ or
+the AMQP protocol.
 
 ## How It Works
 
-Here is a complete example -- the following program creates a simple network
-service that takes a number and add one, along with a client that uses it. They
-both run simultaneously in the same script.
+The easiest way to explain how it works is by example. The following program
+illustrates creating a network service and a client to talk to it. The service
+simply takes a number from the message headers, adds one to it and sends the
+result back.
+
+In this example, both client and service run simultaneously in the same
+script. However, services can run in separate scripts just as easily (more on
+this below).
 
 ```ruby
 #!/usr/bin/env ruby
 
-$rabbitmq_host     = 'localhost'
-$rabbitmq_username = 'guest'
-$rabbitmq_password = 'guest'
-$haredo_test_queue = 'HareDo'
+$mq_host     = 'localhost'
+$mq_username = 'guest'
+$mq_password = 'guest'
+$queue       = 'HareDo'
 
-# Simple service
 class Service < HareDo::Service
 
   def initialize(name)
     super
-    connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
+    connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
   end
 
   def serve(msg)
@@ -40,74 +57,85 @@ class Service < HareDo::Service
     send(msg.properties.reply_to, :data => data.to_s)
   end
 
-end # class Server
-
-def dump_message(msg)
-  puts 'Headers:'
-  msg.properties.each do |k,v|
-    puts "    #{k}: #{v}"
-  end
-  
-  puts "Data: #{msg.data}"
 end
 
-client = HareDo::Client.new()
-client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
-service = Service.new($haredo_test_queue)
-
+service = Service.new($queue)
 service.run(:blocking => false)
 
+client = HareDo::Client.new()
+client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
+
 1.upto(10) do |i|
-  client.send($haredo_test_queue, :headers => { :i => i })
+  client.send($queue, :headers => { :i => i })
   msg = @client.receive()
-
-  dump_message msg
       
   puts msg.data.to_i == i + 1
 end
 ```
 
-The service takes a single argument -- the queue name. This is like an email
-address or an IP address. It's an identifier you use to direct messages to the
-service.
+The service takes a single argument -- the a name. This name is the name of the
+AMQP queue used to send messages to the service. A queue is like an email
+address, phone number or an IP address -- it's just a unique identifier. In this
+case, its identifer you use to direct messages to the service. The run method
+then causes the servers to listen on the queue.
 
 The client sends 10 messages to the service, each containing monotonically
 increasing integer values. It waits for the response (using a blocking timeout)
 after each call and checks the results.
 
-### Request/Response Pattern
-
-While it is not required that a service reply, the requests/response pattern is
-the main function of this library. We are often interested in sending a request
-and getting one (or more responses) back. Implementing this is only a slight
-change from the above example. The following implements a service that responds
-with a single message.
-
-The server code is as follows:
+The client is under no obligation to wait for a response. This is just how we've
+coded the service in this example -- to send something back. The interaction
+between the two forms an ad-hoc protocol. Hence the point of this project --
+quick and easy network protocols with minimal code (thanks to Rabbit and Bunny).
+
+## Clients
+
+There are three basic use-cases the client addresses:
+
+* Event Dispatch: Client sends one or messages expecting a response(s). This is
+  the case when you want to log events -- you don't really care about getting a
+  response back. You just want to send out the event and be done with it.
+* Send/Receive: Client sends one or more messages expecting response and
+  processes those responses in no particular order. That is, there is no need to
+  correlate a response with its request, it can be processed in its own
+  right. This could be like loading a set of images or documents from a remote
+  service. When the document arrives, you don't really care about the request
+  used to obtain it. All you care about is the document itself. In this case, a
+  client could send out 10 messages asynchronously and then wait in a loop for
+  each to come back.
+* Remote Procedure Call (RPC): Client sends one or more messages expecting
+  response and needs to processes those responses in a specific order. That is,
+  it needs to correlate the response to its associated request. This is like
+  calling a function. You need to know the specific like a return value for the
+  specific function you called.
+
+Event dispatch is simple -- you don't send back a reply from the service, nor do
+you expect a response on the client-side. Take a simple logging service for
+example. Here is the server code:
 
 ```ruby
 #!/usr/bin/env ruby
 
 require 'haredo/peer'
 
-$rabbitmq_host     = 'localhost'
-$rabbitmq_username = 'guest'
-$rabbitmq_password = 'guest'
-$haredo_test_queue = 'HareDo'
+$mq_host     = 'localhost'
+$mq_username = 'guest'
+$mq_password = 'guest'
 
-class Service < HareDo::Service
+class Logger < HareDo::Service
 
   def initialize(name)
     super
-    connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
+    connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
   end
 
   def serve(msg)
-    reply(msg, :data => 'Reply')
+    # Log the message somewhere
   end
+
 end
 
-Service.new($haredo_test_queue).run()
+Service.new('logger').run()
 ```
 
 The client code is a follows:
@@ -117,34 +145,82 @@ The client code is a follows:
 
 require 'haredo/peer'
 
-$rabbitmq_host     = 'localhost'
-$rabbitmq_username = 'guest'
-$rabbitmq_password = 'guest'
-$haredo_test_queue = 'HareDo'
+$mq_host     = 'localhost'
+$mq_username = 'guest'
+$mq_password = 'guest'
 
 client = HareDo::Client.new()
-client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
-response = @client.call($haredo_test_queue, :data => 'jujifruit')
+client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
+
+client.send('logger', :data => 'Backup script failed')
 ```
 
 You can run both in the same script by setting the service code to non-blocking
-(passing <tt>:blocking => false</tt> to the <tt>Service::run()</tt> method):
+(passing <tt>:blocking => false</tt> to the <tt>Service::run()</tt>
+method). Services in either blocking or non-blocking mode, enabling you to run
+them either in the same script as clients (non-blocking), or in separate scripts
+where they idle and wait for incoming messages (blocking). By default, services
+run in blocking mode, as it is assumed that they will be be run in their own
+independent scripts/processes.
+
+```ruby
+#!/usr/bin/env ruby
+
+require 'haredo/peer'
+
+$mq_host     = 'localhost'
+$mq_username = 'guest'
+$mq_password = 'guest'
+
+class Logger < HareDo::Service
+
+  def initialize(name)
+    super
+    connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
+  end
+
+  def serve(msg)
+    # Log the data somewhere
+  end
+
+end
+
+Service.new('logger').run(:blocking=>false)
+
+client = HareDo::Client.new()
+client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
+client.send('logger', :data => 'Backup script failed')
+```
+
+### Send/Receive
+
+The send/receive pattern is illustrated in the initial example -- the client
+sends out ten integers and wait for the response. While the responses come back
+in the correct order, there is no guarantee that this will happen every time. So
+technically, the client example could break here. What the client needs here is
+to correlate the response with the request. This is the RPC pattern.
+
+### Remote Procedure Call
+
+This is the purpose of the <tt>Client::call()</tt> method. It assigns a specific
+ID to the outgoing message and waits for a response back containing that message
+ID. So <tt>call()</tt> acts like a synchronous function call. It sends out a
+request and blocks waiting for a specific reply. Here is an example:
 
 ```ruby
 #!/usr/bin/env ruby
 
 require 'haredo/peer'
 
-$rabbitmq_host     = 'localhost'
-$rabbitmq_username = 'guest'
-$rabbitmq_password = 'guest'
-$haredo_test_queue = 'HareDo'
+$mq_host     = 'localhost'
+$mq_username = 'guest'
+$mq_password = 'guest'
 
 class Service < HareDo::Service
 
   def initialize(name)
     super
-    connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
+    connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
   end
 
   def serve(msg)
@@ -152,73 +228,91 @@ class Service < HareDo::Service
   end
 end
 
-service = Service.new($haredo_test_queue)
+service = Service.new('rpc')
 service.run(:blocking => false)
 
 client = HareDo::Client.new()
-client.connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
-response = @client.call($haredo_test_queue, :data => 'jujifruit')
+client.connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
+response = @client.call('rpc', :data => 'jujifruit')
 ```
 
-The client uses the <tt>call()</tt> method to send a message and wait for a
-response. This is just a convenience method that wraps <tt>send()</tt> and
-<tt>receive()</tt>. Interally, a unique message ID is assigned to the request
-message, and the service will send back a reply with the message ID set in the
-message headers.
-
-The client will block waiting for the reponse -- up to <tt>client.timeout</tt>
-seconds (floating point value). The default is 1 second. You can change this to
-whatever value you like. If a timeout occurs, the <tt>call()</tt> will return
-<tt>nil</tt>. Subsequent calls will fail to get the response, even if it comes
-in later (it will be discarded).
-
-The service uses the <tt>reply()</tt> method rather than the <tt>send()</tt>
-method to return a response. This method takes the original request message,
-extracts the <tt>to</tt> address along with message ID and addresses a message
-back to the client. It adds a header value of <tt>reply=true</tt>. This lets the
-client know that the message coming in is a reply. It needs this because what if
-another peer sent it a message with the same message id? It might interpret that
-message as the reply from the service. Therefore this flag lets the client know
-that the message is a reply to a previous request.
-
-Using this simple pattern, you can easily create heterogeneous network services
-in just a few lines of code.
-
-### Roud-Robin Services
-
-Say you want to scale your service to three nodes distributed across three
-different machines. The only thing you need to do differently is redefine one
-method -- <tt>Service::createQueue()</tt>. The following is updates the above
-example and makes is available to run in this way:
+The <tt>call()</tt> method will block waiting for the reponse up to
+<tt>client.timeout</tt> seconds (floating point value). The default is 1
+second. You can change this to whatever value you like. If a timeout occurs,
+<tt>call()</tt> will return <tt>nil</tt>.
+
+The <tt>call()</tt> method is just a convenience method that wraps
+<tt>send()</tt> and <tt>receive()</tt>. Interally, <tt>send()</tt> always
+assigns a unique message ID to each outgoing message. It passes this ID back as
+the return value. You can use this ID to fetch specific messages from
+<tt>receive()</tt>.
+
+Normally <tt>receive()</tt> just returns the oldest message to
+arrive. Internally is keeps a queue of messages as they are recieved. When you
+call it with no arguments, it just pops the oldest message off the queue (if
+there are any), otherwise it blocks waiting for a message to come in until the
+timeout interval expires. However, if you pass an ID to <tt>receive()</tt>, it
+will look specifically for a message with that ID. If none is found, again, it
+will block for the timeout interval waiting for it to come it. If none does, it
+returns <tt>nil</tt>.
+
+If <tt>call()</tt> times-out, subsequent calls will fail to get the response,
+even if it comes in later as it will be discarded. This keeps dilatory messages
+from filling up the receive queue. Once you've given up on a message, it makes
+no sense to keep it around should it arrive sometime in the future.
+
+### The Service Side
+
+Notice that the service uses the <tt>Service::reply()</tt> method rather than
+the <tt>send()</tt> method to return a response. This method takes the original
+request message, extracts the <tt>to</tt> address along with message ID and
+addresses a message back to the client. It also adds a header value of
+<tt>reply=true</tt>. This header lets the client know that the message coming in
+is specifically a reply. This is needed this because what if another peer sent
+the client a message which happened to have the same message ID? The client
+would confuse that message as the reply from the service and that would mess
+things up fast. Therefore the <tt>reply<tt> flag lets the client know that the
+message is a reply to a previous request and can then use the message ID to
+correlate it with the original request.
+
+## Services
+
+Most of what needs to be said about services has be covered. However there is
+one important feature RabbitMQ accords you which we need to address.
+
+Say you want to scale your service from just a single thread or process and
+distribute accross multiple machines which are perhaps even on different
+networks around the world. You can enable this by modifying a single field. The
+only thing you need to do differently is redefine one method --
+<tt>Service::createQueue()</tt>. The following is updates the above example and
+makes is available to run in this way:
 
 ```ruby
 #!/usr/bin/env ruby
 
 require 'haredo/peer'
 
-$rabbitmq_host     = 'localhost'
-$rabbitmq_username = 'guest'
-$rabbitmq_password = 'guest'
-$haredo_test_queue = 'HareDo'
+$mq_host     = 'localhost'
+$mq_username = 'guest'
+$mq_password = 'guest'
 
 class Service < HareDo::Service
 
   def initialize(name)
     super
-    connect($rabbitmq_username, $rabbitmq_password, $rabbitmq_host)
+    connect(:user=>$mq_username, :password=>$mq_password, :host=>$mq_host)
   end
 
   def createQueue()
     return @channel.queue(@queue_name, :auto_delete => true)
   end   
 
-
   def serve(msg)
     reply(msg, :data => 'Reply')
   end
 end
 
-Service.new($haredo_test_queue).run()
+Service.new('rpc').run()
 ```
 
 The only thing we did is remove the <tt>exclusive</tt> attribute which is by

data/src/lib/haredo/peer.rb

@@ -42,22 +42,35 @@ class Node
 
   # Connect to RabbitMQ
   #
-  # @param user The RabbitMQ username
-  # @param password The RabbitMQ password
-  # @param host The RabbitMQ host
-  # @param port The RabbitMQ port
+  # @param user The RabbitMQ username (default guest)
+  # @param password The RabbitMQ password (default guest)
+  # @param host The RabbitMQ host (default localhost)
+  # @param port The RabbitMQ port (default 5672)
+  # @param vhost The RabbitMQ vhost (default /)
+  # @param ssl Use SSL (default false)
   # @return Returns true if connection was successful, false othewise.
 
-  def connect(user, password, host='localhost', port='5672')
+  def connect(args)
+
+    user       = args[:user]     || 'guest'
+    password   = args[:password] || 'guest'
+    host       = args[:host]     || 'localhost'
+    port       = args[:post]     || '5672'
+    vhost      = args[:vhost]    || ''
+    ssl        = ''
+    
+    if args[:ssl] == true
+      ssl = 's'
+    end
 
     begin
-      @cnx = Bunny.new("amqp://#{user}:#{password}@#{host}:#{port}")
+      @cnx = Bunny.new("amqp#{ssl}://#{user}:#{password}@#{host}:#{port}#{vhost}")
       @cnx.start()
     rescue Bunny::TCPConnectionFailed => e
       return false
     end
 
-    @channel  = @cnx.create_channel()
+    @channel = @cnx.create_channel()
 
     return true
   end
@@ -116,13 +129,17 @@ class Client < Node
     @mid = 0
   end
 
-  def connect(user, password, host='localhost', port='5672')
-    super
+  def connect(args)
+    ret = super
 
-    @queue = @channel.queue( '', :auto_delete => true, 
-                             :arguments => { "x-message-ttl" => 1000 } )
+    if ret == true
+      @queue = @channel.queue( '', :auto_delete => true, 
+                               :arguments => { "x-message-ttl" => 1000 } )
 
-    @exchange = @channel.default_exchange()
+      @exchange = @channel.default_exchange()
+    end
+
+    return ret
   end
 
   # Disconnect from RabbitMQ
@@ -235,10 +252,14 @@ end # class Client
 
 class Service < Node
 
+  # @param name The name of the queue the service will attempt to bind to.
   def initialize(name)
     super()
 
     @queue_name = name
+
+    # The number of messages to prefecth from Rabbit
+    @prefetch = 10
   end
 
   # Defined the queue this service will listen on. Assumes a single-instance
@@ -260,7 +281,10 @@ class Service < Node
 
     block = args[:blocking] || false
 
-    queue.subscribe(:block => block) do |info, props, data|
+    @channel.prefetch(@prefetch)
+
+    queue.subscribe(:block => block, :ack => true) do |info, props, data|
+      @channel.acknowledge(info.delivery_tag, false)
       serve Message.new(info, props, data)
     end
   end

data/src/lib/haredo/version.rb

@@ -1,5 +1,5 @@
 module HareDo
 
-VERSION = '0.0.2-1'
+VERSION = '0.0.3-1'
 
 end # module HareDo

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: haredo
 version: !ruby/object:Gem::Version
-  version: 0.0.2
+  version: 0.0.3
 platform: ruby
 authors:
 - Mike Owens
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-09-19 00:00:00.000000000 Z
+date: 2013-09-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bunny