amqp 0.5.2 → 0.5.3

data/README

@@ -1,4 +1,11 @@
-Simple AMQP driver for Ruby/EventMachine.
+Simple AMQP driver for Ruby/EventMachine
+  (c) 2008 Aman Gupta (tmm1)
+ 
+  http://github.com/tmm1/amqp
+  http://rubyforge.org/projects/amqp
+  http://hopper.squarespace.com/blog/2008/7/22/simple-amqp-library-for-ruby.html
+  http://groups.google.com/group/ruby-amqp
+  http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2008-July/001417.html
 
 This library was tested primarily with RabbitMQ, although it should be compatible with any
 server implementing the AMQP 0-8 spec.
@@ -12,15 +19,17 @@ To use with RabbitMQ, first run the server:
 
 To get started, refer to the various bundled examples:
 
-  ruby examples/pingpong.rb   # 1-1 communication using amq.direct
-  ruby examples/clock.rb      # 1-N communication using amq.fanout
-  ruby examples/stocks.rb     # 1-subscriber communication using amq.topic
-  ruby examples/hashtable.rb  # simple async rpc layer
-  ruby examples/primes.rb 4   # parallelized prime number generation
+  ruby examples/mq/simple.rb     # low-level Queue/Exchange api
+  ruby examples/mq/pingpong.rb   # 1-1 communication using amq.direct
+  ruby examples/mq/clock.rb      # 1-N communication using amq.fanout
+  ruby examples/mq/stocks.rb     # 1-subscriber communication using amq.topic
+  ruby examples/mq/hashtable.rb  # simple async rpc layer
+  ruby examples/mq/primes.rb 4   # parallelized prime number generation
+  ruby examples/mq/logger.rb     # simple logging api
 
 For more details into the lower level AMQP client API, run the simple client example:
 
-  ruby examples/simple.rb
+  ruby examples/amqp/simple.rb   # low-level AMQP api
 
 Or refer to protocol/doc.txt, which enumerates packets sent between a server and client
 during a typical session, in both binary and decoded formats.
@@ -36,32 +45,47 @@ The lib/amqp/spec.rb file is generated automatically based on the AMQP specifica
 This project was inspired by py-amqplib, rabbitmq, qpid and rubbyt.
 Special thanks to Dmitriy Samovskiy, Ben Hood and Tony Garnock-Jones.
 
-Other AMQP resources:
+AMQP resources:
 
   Servers:
-    RabbitMQ (Rabbit Technologies, Erlang/OTP, MPL)  http://rabbitmq.com
-    OpenAMP  (iMatrix, C, GPL2)                      http://openamq.org
-    ActiveMQ (Apache Foundation, Java, apache2)      http://activemq.apache.org/
+    RabbitMQ (Rabbit Technologies, Erlang/OTP, MPL) - http://rabbitmq.com
+    ZeroMQ   (iMatrix/FastMQ/Intel, C++, GPL3)      - http://www.zeromq.org
+    OpenAMQ  (iMatrix, C, GPL2)                     - http://openamq.org
+    ActiveMQ (Apache Foundation, Java, apache2)     - http://activemq.apache.org/
 
-  John O'Hara's article on the history of AMQP:
+  Steve Vinoski explains AMQP in his column, Towards Integration
+    http://steve.vinoski.net/pdf/IEEE-Advanced_Message_Queuing_Protocol.pdf
+
+  John O'Hara on the history of AMQP
     http://www.acmqueue.org/modules.php?name=Content&pa=showpage&pid=485
 
-  Barry Pederson's py-amqplib:
+  ZeroMQ's analysis of the messaging technology market
+    http://www.zeromq.org/whitepapers:market-analysis
+
+  ZeroMQ's background to AMQP
+    http://www.zeromq.org/whitepapers:amqp-analysis
+
+  Barry Pederson's py-amqplib
     http://barryp.org/software/py-amqplib/
 
-  Ben Hood's article on writing an AMQP client:
+  Ben Hood on writing an AMQP client
     http://hopper.squarespace.com/blog/2008/6/21/build-your-own-amqp-client.html
 
-  Dmitriy Samovskiy's introduction to Ruby+QPid+RabbitMQ:
+  Dmitriy Samovskiy introduces Ruby + QPid + RabbitMQ
     http://somic-org.homelinux.org/blog/2008/06/24/ruby-amqp-rabbitmq-example/
 
-  Ben Hood's AMQP client in AS3:
+  Ben Hood's as3-amqp
     http://github.com/0x6e6562/as3-amqp
+    http://hopper.squarespace.com/blog/2008/7/4/server-side-as3.html
+    http://hopper.squarespace.com/blog/2008/3/24/as3-amqp-client-first-cut.html
 
-  RabbitMQ's protocol code generator:
+  RabbitMQ's protocol code generator
     http://hg.rabbitmq.com/rabbitmq-codegen/
 
-  Jonahthan Conway's series on RabbitMQ and using it with Ruby/Merb:
+  Erlang Exchange presentation on the implementation of RabbitMQ
+    http://www.lshift.net/blog/2008/07/01/slides-from-our-erlang-exchange-talk
+
+  Jonathan Conway's series on RabbitMQ and using it with Ruby/Merb
     http://jaikoo.com/2008/3/20/daemonize-rabbitmq
     http://jaikoo.com/2008/3/14/oh-hai-rabbitmq
     http://jaikoo.com/2008/2/29/friday-round-up-2008-02-29
@@ -77,3 +101,9 @@ Messaging and distributed systems resources:
 
   Convenience Over Correctness
     http://steve.vinoski.net/pdf/IEEE-Convenience_Over_Correctness.pdf
+
+  Metaprotocol Taxonomy and Communications Patterns
+    http://hessian.caucho.com/doc/metaprotocol-taxonomy.xtp
+
+  Joe Armstrong on Erlang messaging vs RPC:
+    http://armstrongonsoftware.blogspot.com/2008/05/road-we-didnt-go-down.html

data/examples/{simple.rb → amqp/simple.rb}

@@ -1,4 +1,4 @@
-$:.unshift File.dirname(__FILE__) + '/../lib'
+$:.unshift File.dirname(__FILE__) + '/../../lib'
 require 'amqp'
 
 module SimpleClient

data/examples/{clock.rb → mq/clock.rb}

@@ -1,4 +1,4 @@
-$:.unshift File.dirname(__FILE__) + '/../lib'
+$:.unshift File.dirname(__FILE__) + '/../../lib'
 require 'mq'
 
 EM.run{

data/examples/{hashtable.rb → mq/hashtable.rb}

@@ -1,4 +1,4 @@
-$:.unshift File.dirname(__FILE__) + '/../lib'
+$:.unshift File.dirname(__FILE__) + '/../../lib'
 require 'mq'
 
 EM.run{

data/examples/mq/logger.rb

@@ -0,0 +1,139 @@
+$:.unshift File.dirname(__FILE__) + '/../../lib'
+require 'mq'
+
+class Logger
+  def initialize *args, &block
+    opts = args.pop if args.last.is_a? Hash
+    opts ||= {}
+    printer(block) if block
+    @log = []
+    @tags = ([:timestamp] + args).uniq
+  end
+
+  def log severity, *args
+    opts = args.pop if args.last.is_a? Hash and args.size != 1
+    opts ||= {}
+    data = args.shift
+
+    data = {:type => :exception,
+            :name => data.class.to_s.intern,
+            :backtrace => data.backtrace,
+            :message => data.message} if data.is_a? Exception
+        
+    (@tags + args).each do |tag|
+      tag = tag.to_sym
+      case tag
+      when :timestamp
+        opts.update :timestamp => Time.now.to_i
+      when :hostname
+        @hostname ||= { :hostname => `hostname`.strip }
+        opts.update @hostname
+      when :process
+        @process_id ||= { :process_id => Process.pid,
+                          :process_name => $0,
+                          :process_parent_id => Process.ppid,
+                          :thread_id => Thread.current.object_id }
+        opts.update :process => @process_id
+      else
+        (opts[:tags] ||= []) << tag
+      end
+    end
+    
+    opts.update(:severity => severity,
+                :data => data)
+    
+    print(opts)
+    MQ.queue('logger', :durable => true).publish(Marshal.dump(opts), :persistent => true)
+    opts
+  end
+  alias :method_missing :log
+
+  def print data = nil, &block
+    if block
+      @printer = block
+    elsif data.is_a? Proc
+      @printer = data
+    elsif @printer and data
+      @printer.call(data)
+    else
+      @printer
+    end
+  end
+  alias :printer :print
+end
+
+EM.run{
+  # AMQP.logging = true
+  # MQ.logging = true
+
+  if ARGV[0] == 'server'
+
+    MQ.queue('logger', :durable => true).subscribe{|msg|
+      msg = Marshal.load(msg)
+      require 'pp'
+      pp(msg)
+      puts
+    }
+    
+  else
+
+    log = Logger.new
+    log.debug 'its working!'
+    
+    log = Logger.new do |msg|
+      require 'pp'
+      pp msg
+      puts
+    end
+
+    log.info '123'
+    log.debug [1,2,3]
+    log.debug :one => 1, :two => 2
+    log.error Exception.new('123')
+
+    log.info '123', :process_id => Process.pid
+    log.info '123', :process
+    log.debug 'login', :session => 'abc', :user => 123
+
+    log = Logger.new(:webserver, :timestamp, :hostname, &log.printer)
+    log.info 'Request for /', :GET, :session => 'abc'
+
+    AMQP.stop
+
+  end
+}
+
+__END__
+
+{:data=>"123", :timestamp=>1216846102, :severity=>:info}
+
+{:data=>[1, 2, 3], :timestamp=>1216846102, :severity=>:debug}
+
+{:data=>
+  {:type=>:exception, :name=>:Exception, :message=>"123", :backtrace=>nil},
+ :timestamp=>1216846102,
+ :severity=>:error}
+
+{:data=>"123", :timestamp=>1216846102, :process_id=>1814, :severity=>:info}
+
+{:process=>
+  {:thread_id=>109440,
+   :process_id=>1814,
+   :process_name=>"/Users/aman/code/amqp/examples/logger.rb",
+   :process_parent_id=>1813},
+ :data=>"123",
+ :timestamp=>1216846102,
+ :severity=>:info}
+
+{:session=>"abc",
+ :data=>"login",
+ :timestamp=>1216846102,
+ :severity=>:debug,
+ :user=>123}
+
+{:session=>"abc",
+ :tags=>[:webserver, :GET],
+ :data=>"Request for /",
+ :timestamp=>1216846102,
+ :severity=>:info,
+ :hostname=>"gc"}

data/examples/{pingpong.rb → mq/pingpong.rb}

@@ -1,4 +1,4 @@
-$:.unshift File.dirname(__FILE__) + '/../lib'
+$:.unshift File.dirname(__FILE__) + '/../../lib'
 require 'mq'
 
 EM.run{

data/examples/{primes.rb → mq/primes.rb}

@@ -1,4 +1,4 @@
-$:.unshift File.dirname(__FILE__) + '/../lib'
+$:.unshift File.dirname(__FILE__) + '/../../lib'
 require 'mq'
 
 # check MAX numbers for prime-ness

data/examples/mq/simple.rb

@@ -0,0 +1,41 @@
+$:.unshift File.dirname(__FILE__) + '/../../lib'
+require 'mq'
+require 'pp'
+
+EM.run do
+  
+  # open a channel on the AMQP connection
+  channel = MQ.new
+
+  # declare a queue on the channel
+  queue = MQ::Queue.new(channel, 'queue name')
+
+  # use the default fanout exchange
+  exchange = MQ::Exchange.new(channel, :fanout, 'all queues')
+
+  # bind the queue to the exchange
+  queue.bind(exchange)
+
+  # publish a message to the exchange
+  exchange.publish('hello world')
+
+  # subscribe to messages from the queue 
+  queue.subscribe do |headers, msg|
+    pp [:got, headers, msg]
+    AMQP.stop
+  end
+  
+end
+
+__END__
+
+[:got,
+ #<AMQP::Protocol::Header:0x118a438
+  @klass=AMQP::Protocol::Basic,
+  @properties=
+   {:priority=>0,
+    :delivery_mode=>1,
+    :content_type=>"application/octet-stream"},
+  @size=11,
+  @weight=0>,
+ "hello world"]

data/examples/{stocks.rb → mq/stocks.rb}

@@ -1,4 +1,4 @@
-$:.unshift File.dirname(__FILE__) + '/../lib'
+$:.unshift File.dirname(__FILE__) + '/../../lib'
 require 'mq'
 
 EM.run{

data/lib/amqp.rb

@@ -1,9 +1,11 @@
 module AMQP
+  VERSION = '0.5.3'
+
   DIR = File.expand_path(File.dirname(File.expand_path(__FILE__)))
-  
   $:.unshift DIR
   
   require 'ext/em'
+  require 'ext/blankslate'
   
   %w[ buffer spec protocol frame client ].each do |file|
     require "amqp/#{file}"
@@ -12,5 +14,42 @@ module AMQP
   class << self
     @logging = false
     attr_accessor :logging
+    attr_reader :stopping
+  end
+
+  def self.settings
+    @settings ||= {
+      :user => 'guest',
+      :pass => 'guest',
+      :vhost => '/'
+    }
+  end
+
+  def self.start *args
+    @conn ||= Client.connect *args
+  end
+  
+  def self.stop stop_reactor = true, &on_stop
+    if @conn
+      @conn.callback{ |c|
+        if c.channels.keys.any?
+          c.channels.each do |_, mq|
+            mq.close
+          end
+        else
+          c.close
+        end
+      }
+      @on_stop = proc{
+        @conn = nil
+        on_stop.call if on_stop
+        EM.stop_event_loop if stop_reactor
+      }
+    end
+  end
+  
+  def self.stopped
+    @on_stop.call if @on_stop
+    @on_stop = nil
   end
 end

data/lib/amqp/client.rb

@@ -1,7 +1,8 @@
 require 'amqp/frame'
-require 'pp'
 
 module AMQP
+  class Error < Exception; end
+
   module BasicClient
     def process_frame frame
       if mq = channels[frame.channel]
@@ -16,10 +17,10 @@ module AMQP
           send Protocol::Connection::StartOk.new({:platform => 'Ruby/EventMachine',
                                                   :product => 'AMQP',
                                                   :information => 'http://github.com/tmm1/amqp',
-                                                  :version => '0.5.0'},
+                                                  :version => VERSION},
                                                  'AMQPLAIN',
-                                                 {:LOGIN => 'guest',
-                                                  :PASSWORD => 'guest'},
+                                                 {:LOGIN => @settings[:user],
+                                                  :PASSWORD => @settings[:pass]},
                                                  'en_US')
 
         when Protocol::Connection::Tune
@@ -27,12 +28,18 @@ module AMQP
                                                 :frame_max => 131072,
                                                 :heartbeat => 0)
 
-          send Protocol::Connection::Open.new(:virtual_host => '/',
+          send Protocol::Connection::Open.new(:virtual_host => @settings[:vhost],
                                               :capabilities => '',
                                               :insist => false)
 
         when Protocol::Connection::OpenOk
           @dfr.succeed(self)
+
+        when Protocol::Connection::Close
+          raise Error, "#{method.reply_text} in #{Protocol.classes[method.class_id].methods[method.method_id]}"
+
+        when Protocol::Connection::CloseOk
+          AMQP.stopped
         end
       end
     end
@@ -48,8 +55,9 @@ module AMQP
   end
 
   module Client
-    def initialize dfr
+    def initialize dfr, opts = {}
       @dfr = dfr
+      @settings = opts
       extend AMQP.client
     end
 
@@ -70,8 +78,8 @@ module AMQP
     end
   
     def receive_data data
+      # log 'receive_data', data
       @buf << data
-      log 'receive_data', data
 
       while frame = Frame.parse(@buf)
         log 'receive', frame
@@ -92,9 +100,16 @@ module AMQP
       send_data data.to_s
     end
 
-    def send_data data
-      log 'send_data', data
-      super
+    # def send_data data
+    #   log 'send_data', data
+    #   super
+    # end
+
+    def close
+      send Protocol::Connection::Close.new(:reply_code => 200,
+                                           :reply_text => 'Goodbye',
+                                           :class_id => 0,
+                                           :method_id => 0)
     end
 
     def unbind
@@ -102,13 +117,14 @@ module AMQP
     end
   
     def self.connect opts = {}
+      opts = AMQP.settings.merge(opts)
       opts[:host] ||= 'localhost'
       opts[:port] ||= PORT
 
       dfr = EM::DefaultDeferrable.new
       
       EM.run{
-        EM.connect opts[:host], opts[:port], self, dfr
+        EM.connect opts[:host], opts[:port], self, dfr, opts
       }
       
       dfr
@@ -118,16 +134,9 @@ module AMQP
   
     def log *args
       return unless AMQP.logging
+      require 'pp'
       pp args
       puts
     end
   end
-
-  def self.start *args
-    @conn ||= Client.connect *args
-  end
-end
-
-if $0 == __FILE__
-  AMQP.start
 end

data/lib/amqp/protocol.rb

@@ -23,6 +23,12 @@ module AMQP
         end
       end
 
+      def arguments
+        self.class.arguments.inject({}) do |hash, (type, name)|
+          hash.update name => instance_variable_get("@#{name}")
+        end
+      end
+
       def to_binary
         buf = Buffer.new
         buf.write :short, self.class.parent.id
@@ -116,8 +122,8 @@ module AMQP
       end
 
       def method_missing meth, *args, &blk
-        @klass.properties.map{|_,name| name }.include?(meth) ? @properties[meth] :
-                                                               super
+        @properties.has_key?(meth) || @klass.properties.find{|_,name| name == meth } ? @properties[meth] :
+                                                                                       super
       end
     end
 

data/lib/ext/blankslate.rb

@@ -0,0 +1,7 @@
+unless defined?(BlankSlate)
+  class BlankSlate < BasicObject; end if defined?(BasicObject)
+
+  class BlankSlate
+    instance_methods.each { |m| undef_method m unless m =~ /^__/ }
+  end
+end

data/lib/mq.rb

@@ -10,6 +10,8 @@ class MQ
     @logging = false
     attr_accessor :logging
   end
+
+  class Error < Exception; end
 end
 
 class MQ
@@ -35,6 +37,7 @@ class MQ
     when Frame::Body
       @body << frame.payload
       if @body.length >= @header.size
+        @header.properties.update(@method.arguments)
         @consumer.receive @header, @body
         @body = ''
       end
@@ -49,12 +52,30 @@ class MQ
 
       when Protocol::Access::RequestOk
         @ticket = method.ticket
+        callback{
+          send Protocol::Channel::Close.new(:reply_code => 200,
+                                            :reply_text => 'bye',
+                                            :method_id => 0,
+                                            :class_id => 0)
+        } if @closing
         succeed
 
       when Protocol::Basic::Deliver
+        @method = method
         @header = nil
         @body = ''
         @consumer = queues[ method.consumer_tag ]
+
+
+      when Protocol::Channel::Close
+        raise Error, "#{method.reply_text} in #{Protocol.classes[method.class_id].methods[method.method_id]}"
+
+      when Protocol::Channel::CloseOk
+        @closing = false
+        conn.callback{ |c|
+          c.channels.delete(@channel)
+          c.close unless c.channels.keys.any?
+        }
       end
     end
   end
@@ -83,12 +104,15 @@ class MQ
     rpcs[name] ||= RPC.new(self, name, obj)
   end
 
-  private
-  
-  def log *args
-    return unless MQ.logging
-    pp args
-    puts
+  def close
+    if @deferred_status == :succeeded
+      send Protocol::Channel::Close.new(:reply_code => 200,
+                                        :reply_text => 'bye',
+                                        :method_id => 0,
+                                        :class_id => 0)
+    else
+      @closing = true
+    end
   end
 
   # keep track of proxy objects
@@ -105,9 +129,18 @@ class MQ
     @rcps ||= {}
   end
 
+  private
+  
+  def log *args
+    return unless MQ.logging
+    pp args
+    puts
+  end
+
   # create a class level connection on demand
 
   def connection
+    raise 'MQ can only be used within EM.run{}' unless EM.reactor_running?
     @@connection ||= AMQP.start
   end
   alias :conn :connection

data/lib/mq/exchange.rb

@@ -5,6 +5,7 @@ class MQ
     def initialize mq, type, name, opts = {}
       @mq = mq
       @type, @name = type, name
+      @mq.exchanges[@name = name] ||= self
       @key = opts[:key]
 
       @mq.callback{
@@ -17,18 +18,16 @@ class MQ
 
     def publish data, opts = {}
       @mq.callback{
-        EM.next_tick do
-          @mq.send Protocol::Basic::Publish.new({ :exchange => name,
-                                                  :routing_key => opts.delete(:key) || @key }.merge(opts))
-        
-          data = data.to_s
+        @mq.send Protocol::Basic::Publish.new({ :exchange => name,
+                                                :routing_key => opts.delete(:key) || @key }.merge(opts))
+      
+        data = data.to_s
 
-          @mq.send Protocol::Header.new(Protocol::Basic,
-                                            data.length, { :content_type => 'application/octet-stream',
-                                                           :delivery_mode => 1,
-                                                           :priority => 0 }.merge(opts))
-          @mq.send Frame::Body.new(data)
-        end
+        @mq.send Protocol::Header.new(Protocol::Basic,
+                                          data.length, { :content_type => 'application/octet-stream',
+                                                         :delivery_mode => (opts.delete(:persistent) ? 2 : 1),
+                                                         :priority => 0 }.merge(opts))
+        @mq.send Frame::Body.new(data)
       }
       self
     end

data/lib/mq/queue.rb

@@ -4,7 +4,7 @@ class MQ
     
     def initialize mq, name, opts = {}
       @mq = mq
-      @name = name
+      @mq.queues[@name = name] ||= self
       @mq.callback{
         @mq.send Protocol::Queue::Declare.new({ :queue => name,
                                                 :nowait => true }.merge(opts))

data/lib/mq/rpc.rb

@@ -1,15 +1,8 @@
-unless defined?(BlankSlate)
-  class BlankSlate < BasicObject; end if defined?(BasicObject)
-
-  class BlankSlate
-    instance_methods.each { |m| undef_method m unless m =~ /^__/ }
-  end
-end
-
 class MQ
   class RPC < BlankSlate
     def initialize mq, queue, obj = nil
       @mq = mq
+      @mq.rpcs[queue] ||= self
 
       if obj
         @obj = case obj

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: amqp
 version: !ruby/object:Gem::Version 
-  version: 0.5.2
+  version: 0.5.3
 platform: ruby
 authors: 
 - Aman Gupta
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2008-07-22 00:00:00 -07:00
+date: 2008-07-29 00:00:00 -07:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -32,19 +32,22 @@ extra_rdoc_files: []
 
 files: 
 - README
-- examples/clock.rb
-- examples/hashtable.rb
-- examples/pingpong.rb
-- examples/primes-simple.rb
-- examples/primes.rb
-- examples/simple.rb
-- examples/stocks.rb
+- examples/amqp/simple.rb
+- examples/mq/clock.rb
+- examples/mq/hashtable.rb
+- examples/mq/logger.rb
+- examples/mq/pingpong.rb
+- examples/mq/primes-simple.rb
+- examples/mq/primes.rb
+- examples/mq/simple.rb
+- examples/mq/stocks.rb
 - lib/amqp/buffer.rb
 - lib/amqp/client.rb
 - lib/amqp/frame.rb
 - lib/amqp/protocol.rb
 - lib/amqp/spec.rb
 - lib/amqp.rb
+- lib/ext/blankslate.rb
 - lib/ext/em.rb
 - lib/ext/emfork.rb
 - lib/mq/exchange.rb