bmabey-rosetta_queue 0.1.3 → 0.2.0

data/History.txt

@@ -1,11 +1,32 @@
-== 0.1.x (git)
+== 0.2.x (git)
+
+ === New features
+ === Bufixes
+
+== 0.2.0
+ === New features
+  * Synchronous AMQP adapter
+    * Uses celldee's synchronous AMQP client, 'Bunny'.
+  * Added .delete method to Consumer class
+    * Allows user to purge a queue.
+    * Works for AMQP adapters only.
+ === Bufixes
+  * client :ack now working.
+
+== 0.1.4
 
  === New features
   * Removed dependency on ActiveSupport and brought over needed String methods. (Derek and Matt - Greenview Data, Inc.)
   * Safer shutdown in ThreadedManager (Rob Sanheim - Relevance)
   * Beanstalk Adaper (David Brady - Lead Media Partners)
     * Still needs some work to have it take advantage of beanstalk's subscribe functionality.
- === Bufixes
+  * Synchronous AMQP adapter
+    * Uses celldee's synchronous AMQP client, 'Bunny'.
+  * Added .delete method to Consumer class
+    * Allows user to purge a queue.
+    * Works for AMQP adapters only.
+  * Beanstalk Adaper (David Brady)
+    * Still needs some work to have it take advantage of beanstalk's subscribe funtionality.
 
 == 0.1.0 / 2008-01-28 - Initial Release
 RosettaQueue was realased in the wild! RQ's initial development was primarily sponsored by Alliance Health Networks (thanks!!).  The original authors were Chris Wyckoff and Ben Mabey. The initial release included adapters for stomp and amqp, in addition to a null and fake adapters for testing.

data/README.rdoc

@@ -4,17 +4,28 @@ Rosetta Queue is a messaging gateway API with adapters for many messaging system
 
 The adapters provided currently are for stomp, amqp, and beanstalk.  We would like to add adapters for other messaging gateways.  The stomp adapter has been used in production along side with Apache's ActiveMQ.  The amqp adapter currently works along side RabbitMQ and passes the acceptance tests but as of yet has not been used in production.
 
+
+== Authors
+
+Ben Mabey [http://github.com/bmabey]
+
+Chris Wyckoff [http://github.com/cwyckoff]
+
+
 == Quick Tutorial
+
 Note: The API will most likely change until we reach 1.0.  We will be moving to a more concise API (i.e. queue(:test_queue) << message, etc...)  We will also be changing how exceptions are handled.
 
 When using Rosetta Queue in an application you will need to configure the queues, adapters, and filters (optional).  These configurations should be placed in a file that gets loaded once when your program starts.  If you are using Rails then a good place for this is config/initializers/rosetta_queue.rb.
 
-To set up destinations to produce messages to and consume messages from:
+
+To set up destinations for producing and consuming messages:
 
 RosettaQueue::Destinations.define do |queue|
   queue.map :test_queue, '/queue/my_test_queue'
 end
 
+
 Defining your adapter:
 
   RosettaQueue::Adapter.define do |a|
@@ -25,6 +36,7 @@ Defining your adapter:
     a.type = "stomp"
   end
 
+
 Define a logger for Rosetta Queue to use.  The logger should be a standard ruby logger:
 
   RosettaQueue.logger = Logger.new('/my_project/rosetta_queue.log')
@@ -76,33 +88,72 @@ It is recommended that you set your adapter to the 'null' adapter for your specs
 Please look at the publishing matchers for more information.  For examples on how to write acceptance tests for your Rosetta Queue's code please see RosettaQueue's own Cucumber features and read this {blog post}[http://www.benmabey.com/2009/02/17/using-cucumber-to-integrate-distributed-systems-and-test-messaging/].
 
 
+== Adapters
 
-== How to contribute
-----------------------------------------------------------------
-Gems you will need:
-cucumber, rspec, yaml, stomp, tmm1-amqp, beanstalk-client
+=== AMQP
 
-You should be able to run the rspec code examples (specs) without any brokers running with autospec or 'rake spec'.
+RosettaQueue comes with two AMQP adapters: a synchronous adapter and an evented one.  The former uses the synchronous AMQP client "Bunny" by celldee [http://github.com/celldee/bunny] and the latter, the evented AMQP client by Aman Gupta [http://github.com/tmm1/amqp].
 
-To run the cucumber features you will need the a messaging system setup that can speak stomp and AMQP.  We have been using the following brokers in testing:
+==== Set Up
 
-=== Apache ActiveMQ (for the stomp adapter)
-Go to http://activemq.apache.org/download.html to download the latest version, and see http://activemq.apache.org/getting-started.html for installation and configuration instructions.
-The stomp client and features should work with ApacheMQ out of the box.  If you are running any ApacheMQ servers in production on your network you will want to disable the multicast autodiscovery in conf/activemq.xml.  (Around lines 56 and 98.)
+Install Erlang, unless already installed, and the RabbitMQ messaging broker.  Mac OSX users can install from ports.  Linux users should use their package manager or choice.  One installed, you can launch rabbitmq by running
 
-=== RabbitMQ (for the amqp adapter)
-Download the right tar from here: http://www.rabbitmq.com/download.html and follow the installation directions that comes with it.
-The features rely on a user of 'rosetta' being setup with the password of 'password' and added to the default virtualhost. You can set them up like so:
+  rabbitmq-server
+
+or
+
+  rabbitmqctl start_app
 
-  rabbitmqctl add_user rosetta password
-  rabbitmqctl map_user_vhost rosetta /
 
-=== Beanstalk (for the beanstalk adapter) ===
+==== Configuration
+
+If you are using an AMQP adapter, there are some important rules when defining the adapter and mapping your destinations.  Two of the basic building blocks of AMQP are queues and exchanges.  Queues bind to exchanges in several different ways.  A can queue bind to an exchange and requests messages that match a specific routing key, which is called 'direct exchange' and is analogous to the 'point-to-point' messaging pattern.  Alternately, queues can bind to an exchange to receive all messages sent to that exchange; this is called 'fanout exchange' and is analogous to the 'publish-subscribe' pattern.  So, when you want to define a simple 'direct-exchange' queue, your queue name must begin with the term "queue".
+
+   RosettaQueue::Destinations.define do |queue|
+     queue.map :test_queue, 'queue.my_test_queue'
+   end
+
+And, when you want to define a queue that will bind to a 'fanout-exchange', your queue name must begin with the term "fanout".
+
+   RosettaQueue::Destinations.define do |queue|
+     queue.map :test_queue, 'fanout.my_test_queue'
+   end
+
+Finally, when defining your adapter, the synchronous AMQP adapter should be defined as :*amqp_synch*, and the evented adapter as :*amqp_evented*.
+
+==== Evented AMQP adapter
+
+When publishing and subscribing using the evented AMQP adapter, you need to wrap that code in an event machine run block.  For example, publishing a message might look like this:
+
+  EM.run do
+    Em.add_periodic_timer(1) do
+      RosettaQueue::Producer.publish(:foo, "Hello World!")
+    end
+  end
+
+And a sample consumer might look like this:
+
+  EM.run do
+    RosettaQueue::Consumer.new(MyMessageHandler.new).receive
+  end
+
+Although, if you are using the evented AMQP adapter to consume messages you could simply wrap your consumers in RosettaQueue's EventedManager:
+
+  RosettaQueue::EventedManager.create do |m|
+    m.add(MyFirstMessageHandler.new)
+    m.add(MySecondMessageHandler.new)
+    m.add(MyThirdMessageHandler.new)
+
+    m.start
+  end 
+
+
+=== Beanstalk
 
 You should set up and run a local instance of beanstalk for your
 tests.
 
-==== beanstalkd ====
+==== beanstalkd
 
 Mac OSX users can install beanstalkd from ports. Linux users should
 check their package manager of choice. This is probably the optimal
@@ -120,7 +171,7 @@ which the specs use. You might also want to daemonize beanstalk with
 the -d option but for testing this is not recommended, as the queue
 can get stuck with old messages, which will break your specs.
 
-==== beanstalk-client ====
+==== beanstalk-client
 
 You will also need the Ruby beanstalk-client gem.
 
@@ -128,4 +179,25 @@ You will also need the Ruby beanstalk-client gem.
 
 
 
+== How to contribute
+----------------------------------------------------------------
+Gems you will need:
+cucumber, rspec, yaml, stomp, amqp, bunny, beanstalk-client
+
+You should be able to run the rspec code examples (specs) without any brokers running with autospec or 'rake spec'.
+
+To run the cucumber features you will need the a messaging system setup that can speak stomp and AMQP.  We have been using the following brokers in testing:
+
+=== Apache ActiveMQ (for the stomp adapter)
+Go to http://activemq.apache.org/download.html to download the latest version, and see http://activemq.apache.org/getting-started.html for installation and configuration instructions.
+The stomp client and features should work with ApacheMQ out of the box.  If you are running any ApacheMQ servers in production on your network you will want to disable the multicast autodiscovery in conf/activemq.xml.  (Around lines 56 and 98.)
+
+=== RabbitMQ (for the amqp adapter)
+Download the right tar from here: http://www.rabbitmq.com/download.html and follow the installation directions that comes with it.
+The features rely on a user of 'rosetta' being setup with the password of 'password' and added to the default virtualhost. You can set them up like so:
+
+  rabbitmqctl add_user rosetta password
+  rabbitmqctl map_user_vhost rosetta /
+
+
 

data/Rakefile

@@ -18,8 +18,8 @@ begin
     s.name = "rosetta_queue"
     s.rubyforge_project = "rosetta-queue"
     s.summary = %Q{Messaging gateway API with adapters for many messaging systems available in Ruby.}
-    s.email = "ben@benmabey.com"
-    s.homepage = "http://github.com/bmabey/rosetta_queue"
+    s.email = "cbwyckoff@gmail.com"
+    s.homepage = "http://github.com/cwyckoff/rosetta_queue"
     s.description = %Q{Messaging gateway API with adapters for many messaging systems available in Ruby. Messaging systems can be easily switched out with a small configuration change. Code for testing on the object and application level is also provided.}
     s.extra_rdoc_files = ["README.rdoc", "MIT-LICENSE.txt"]
     s.files = FileList["[A-Z]*.*", "{bin,generators,lib,features,spec}/**/*", "Rakefile", "cucumber.yml"]

data/VERSION.yml

@@ -1,4 +1,4 @@
 --- 
-:minor: 1
-:patch: 4
 :major: 0
+:minor: 2
+:patch: 0

data/examples/sample_amqp_consumer.rb

@@ -0,0 +1,35 @@
+require 'rubygems'
+require File.dirname(__FILE__) + '/../init.rb'
+require File.expand_path(File.dirname(__FILE__) + '/../lib/rosetta_queue/consumer_managers/threaded')
+
+module RosettaQueue
+
+  Adapter.define do |a|
+    a.user      = "rosetta"
+    a.password  = "password"
+    a.host      = "localhost"
+    a.type      = 'amqp_synch'
+  end
+
+  Destinations.define do |dest|
+    dest.map :foo, "queuep.foo"
+  end  
+
+  class MessageHandlerFoo
+    include RosettaQueue::MessageHandler
+    subscribes_to :foo
+    options :ack => true
+    attr_reader :msg
+
+    def on_message(msg)
+      puts "FOO received message: #{msg}"
+    end
+
+  end
+
+  ThreadedManager.create do |m|
+    m.add MessageHandlerFoo.new
+    m.start
+  end 
+
+end

data/examples/sample_amqp_fanout_consumer.rb

@@ -0,0 +1,49 @@
+require 'rubygems'
+require File.dirname(__FILE__) + '/../init.rb'
+require File.expand_path(File.dirname(__FILE__) + '/../lib/rosetta_queue/consumer_managers/threaded.rb')
+
+module RosettaQueue
+
+  Adapter.define do |a|
+    a.user      = "rosetta"
+    a.password  = "password"
+    a.host      = "localhost"
+    a.type      = 'amqp'
+  end
+
+  Destinations.define do |dest|
+    dest.map :foo, "fanout.foo"
+  end  
+
+  class MessageHandlerFoo
+    include RosettaQueue::MessageHandler
+    subscribes_to :foo
+    options :ack => true
+    attr_reader :msg
+
+    def on_message(msg)
+      puts "FOO received message:  #{msg}"
+    end
+
+  end
+
+  class MessageHandlerBar
+    include RosettaQueue::MessageHandler
+    subscribes_to :foo
+    options :ack => true
+    attr_reader :msg
+
+    def on_message(msg)
+      puts "BAR received message:  #{msg}"
+    end
+  end
+
+
+  # threaded version
+  ThreadedManager.create do |m|
+    m.add MessageHandlerFoo.new
+    m.add MessageHandlerBar.new
+    m.start
+  end 
+
+end

data/examples/sample_amqp_fanout_producer.rb

@@ -0,0 +1,17 @@
+require 'rubygems'
+require File.dirname(__FILE__) + '/../init.rb'
+
+RosettaQueue::Adapter.define do |a|
+  a.user      = "rosetta"
+  a.password  = "password"
+  a.host      = "localhost"
+  a.type      = "amqp"
+end
+
+RosettaQueue::Destinations.define do |dest|
+  dest.map :foo, "fanout.foo"
+end  
+
+
+RosettaQueue::Producer.publish(:foo, "hello there")
+

data/examples/sample_amqp_producer.rb

@@ -0,0 +1,15 @@
+require 'rubygems'
+require File.dirname(__FILE__) + '/../init.rb'
+
+RosettaQueue::Adapter.define do |a|
+  a.user      = "rosetta"
+  a.password  = "password"
+  a.host      = "localhost"
+  a.type      = "amqp_synch"
+end
+
+RosettaQueue::Destinations.define do |dest|
+  dest.map :foo, "queue.foo"
+end  
+
+RosettaQueue::Producer.publish(:foo, "hello there")

data/features/filtering.feature

@@ -9,27 +9,23 @@ Feature: Message Filtering
     And a point-to-point destination is set
     And a receiving filter is defined to prepend 'Foo' to all messages
     And the message "Hello World" is published to queue "foo"
-  
     When the message on "foo" is consumed
-  
     Then the consumed message should equal "Foo Hello World"
-    
+
     Examples:
     | Adapter     |
+    | amqp_synch  |
     | stomp       |
-    | amqp        |   
-    
-  Scenario Outline: sending filter
+
+Scenario Outline: sending filter
     Given RosettaQueue is configured for '<Adapter>'
     And a point-to-point destination is set
     And a sending filter is defined to prepend 'Foo' to all messages
     And the message "Hello World" is published to queue "foo"
-
     When the message on "foo" is consumed
-
     Then the consumed message should equal "Foo Hello World"
 
     Examples:
     | Adapter     |
+    | amqp_synch  |
     | stomp       |
-    | amqp        |

data/features/messaging.feature

@@ -0,0 +1,40 @@
+Story: Producing and Consuming
+
+  As a RosettaQueue user
+  I want to publish and consume point-to-point using various messaging protocols
+  So that I can reliably integrate my systems with a message broker
+
+  Scenario Outline: Point-to-Point
+    Given RosettaQueue is configured for '<Adapter>'
+    And a point-to-point destination is set
+    When a message is published to queue 'foo'
+    Then the message should be consumed
+
+    Examples:
+    | Adapter     |
+    | amqp_synch  |
+    | stomp       |
+    | beanstalk   |
+
+  Scenario Outline: Delete queue
+    Given RosettaQueue is configured for '<Adapter>'
+    And a point-to-point destination is set
+    When a message is published to queue '<Queue>'
+    And the queue '<Queue>' is deleted
+    Then the queue '<Queue>' should no longer exist
+
+    Examples:
+    | Adapter   | Queue   |
+    | amqp_synch  | foo   |
+
+
+#   Scenario Outline: Publish-Subscribe
+#     Given RosettaQueue is configured for '<Adapter>'
+#     And a '<PublishSubscribe>' destination is set
+#     When a message is published to 'foobar'
+#     Then multiple messages should be consumed from the topic
+
+#     Examples:
+#    | Adapter    | PublishSubscribe  |
+#    | amqp_synch | fanout    |
+#    | stomp    | topic     |

data/features/step_definitions/common_messaging_steps.rb

@@ -0,0 +1,45 @@
+Given /^RosettaQueue is configured for '(\w+)'$/ do |adapter_type|
+  @adapter_type = adapter_type
+  RosettaQueue::Adapter.define do |a|
+    a.user      = "rosetta"
+    a.password  = "password"
+    a.host      = "localhost"
+    a.type      = @adapter_type
+    a.port      = case @adapter_type
+                    when /stomp/
+                    "61613"
+                    when /beanstalk/
+                    "11300"
+                    else
+                    nil
+                    end
+  end
+end
+
+Given /^a point-to-point destination is set$/ do
+  RosettaQueue::Destinations.define do |dest|
+    dest.map :foo, "/queue/bar"
+  end  
+end
+
+Given /^a '(.*)' destination is set$/ do |pub_sub|
+  case pub_sub
+  when /fanout/
+    RosettaQueue::Destinations.define do |dest|
+      dest.map :foobar, "/fanout/foobar"
+    end  
+  when /topic/
+    RosettaQueue::Destinations.define do |dest|
+      dest.map :foobar, "/topic/foobar"
+    end  
+  end 
+end
+
+When /^the queue '(.*)' is deleted$/ do |queue|
+  system("rabbitmqctl list_queues | grep bar").should be_true
+  RosettaQueue::Consumer.delete(queue.to_sym)
+end
+
+Then /^the queue 'foo' should no longer exist$/ do
+  system("rabbitmqctl list_queues | grep bar").should be_false
+end

data/features/{messaging/step_definitions → step_definitions}/point_to_point_steps.rb

@@ -1,20 +1,12 @@
+Given /^the message "(.+)" is published to queue "(.+)"$/ do |message, queue_name|
+  publish_message(message, {:options => {:ack => "client"}}.merge(:to => queue_name))
+end
+
 When /^a message is published to queue '(\w+)'$/ do |q|
   publish_message("Hello World!", {:options => {:ack => "client"}}.merge(:to => q))
 end
 
 Then /^the message should be consumed$/ do
-  sleep 1
   RosettaQueue::Consumer.receive(:foo).should =~ /Hello World!/
   # consume_once_with(SampleConsumer).should =~ /Hello World!/
-
-  # RosettaQueue::EventedManager.create do |m|
-  # 
-  #   m.add(SampleConsumer.new)
-  #   m.start
-  # 
-  # end
 end
-
-Given /^the message "(.+)" is published to queue "(.+)"$/ do |message, queue_name|
-  publish_message(message, {:options => {:ack => "client"}}.merge(:to => queue_name))
-end

data/features/{messaging/step_definitions → step_definitions}/publish_subscribe_steps.rb

@@ -1,5 +1,5 @@
-When /^a message is published to topic (\w+)$/ do |topic|
-  RosettaQueue::Producer.publish(topic.to_sym, "Hello World 1!", {:persistent => true})
+When /^a message is published to '(\w+)'$/ do |topic|
+  RosettaQueue::Producer.publish(topic.to_sym, "Hello World!", {:durable => true})
   # publish_message("Hello World!", {:options => {:ack => "client"}}.merge(:to => topic))
 end
 
@@ -8,4 +8,4 @@ Then /^multiple messages should be consumed from the topic$/ do
   RosettaQueue::Consumer.receive(:foobar).should =~ /Hello World!/
   # RosettaQueue::Consumer.receive(:bar).should =~ /Hello World!/
   # consume_once_with(SampleConsumer).should == "Hello World!"
-end
+end

data/features/support/env.rb

@@ -4,10 +4,9 @@ require 'rosetta_queue'
 require 'rosetta_queue/spec_helpers'
 require 'spec/expectations'
 require 'rosetta_queue/spec_helpers'
-require 'amqp'
 
 begin
-  RosettaQueue.logger = RosettaQueue::Logger.new(File.join(File.dirname(__FILE__), '../../log', 'rosetta_queue.log'))
+  RosettaQueue.logger = RosettaQueue::Logger.new(File.join(File.dirname(__FILE__), '../../../log', 'rosetta_queue.log'))
 rescue Errno::ENOENT
   Kernel.warn "No log directory setup at the root of rosetta_queue. Using the null logger instead."
   class NullLogger
@@ -22,6 +21,3 @@ rescue Errno::ENOENT
 end
 
 World(RosettaQueue::SpecHelpers)
-
-AMQP.logging = false # true
-# Thread.new {EM.run{}}

data/lib/rosetta_queue/adapter.rb

@@ -4,30 +4,36 @@ module RosettaQueue
   class Adapter
 
     class << self
-      attr_writer :user, :password, :host, :port
+      attr_writer :user, :password, :host, :port, :options
       
       def define
         yield self
       end
-      
+            
       def reset
-        @user, @password, @host, @port, @adapter_class = nil, nil, nil, nil, nil
+        @user, @password, @host, @port, @options, @adapter_class = nil, nil, nil, nil, nil, nil
       end
       
       def type=(adapter_prefix)
-        begin
-          require "rosetta_queue/adapters/#{adapter_prefix}"
+        require "rosetta_queue/adapters/#{adapter_prefix}"
+        @adapter_class = RosettaQueue::Gateway.const_get("#{adapter_prefix.to_s.classify}Adapter")
+
         rescue LoadError
           raise AdapterException, "Adapter type '#{adapter_prefix}' does not match existing adapters!"
-        end
-        @adapter_class = RosettaQueue::Gateway.const_get("#{adapter_prefix.to_s.classify}Adapter")
       end
 
       def instance
         raise AdapterException, "Adapter type was never defined!" unless @adapter_class
-        @adapter_class.new(@user, @password, @host, @port)
+        @adapter_class.new({:user => @user, :password => @password, :host => @host, :port => @port, :opts => opts})
       end
 
+      private
+      
+      def opts
+        raise AdapterException, "Adapter options should be a hash" unless @options.nil? || @options.is_a?(Hash)
+        @options ||= {}
+      end
+      
     end
   end  
 end