ivanvanderbyl-amqp 0.6.13.1

data/.gitignore

@@ -0,0 +1,4 @@
+*.gem
+*.rbc
+~*
+#*

data/HISTORY

@@ -0,0 +1,27 @@
+== 0.6.7 / 2010-11-03
+
+* Forked tmm1/amqp at Github
+
+== 0.6.8 / 2010-11-03
+
+* Additional improvements from latest forks added
+
+== 0.6.9 / 2010-11-04
+
+* Exchange#publish raises error if no connection to broker
+
+== 0.6.10 / 2010-11-04
+
+* Bacon tests converted to Rspec
+
+== 0.6.11 / 2010-11-04
+
+* AMQP class behavior specified
+
+== 0.6.12 / 2010-11-04
+
+* MQ class methods specified
+
+== 0.6.13 / 2010-11-05
+
+* Secondary MQ instance methods specified

data/README.md

@@ -0,0 +1,169 @@
+What AMQP gem is
+================
+
+Simple asynchronous AMQP driver for Ruby/EventMachine
+This library works with Ruby 1.8, Ruby 1.9, JRuby and Rubinius,
+and is licensed under [the Ruby License](http://www.ruby-lang.org/en/LICENSE.txt).
+
+This library was tested primarily with RabbitMQ, although it should be compatible with any
+server implementing the [AMQP 0-8 spec](http://www.amqp.org/confluence/download/attachments/720900/amqp0-8.pdf).
+
+
+This fork of AMQP
+=================
+
+Intention is to describe AMQP lib behavior using RSpec, with reasonable level of completeness.
+Given a solid spec suite, it will be easier to expand the library going forward.
+
+Some improvements that are currently merged in, but not yet fully specified:
+
+ * Support for setting extended headers in MQ::Exchange#publish. Particularly useful for :reply_to for RPC.
+ * Multibyte character support (Ruby 1.9) in MQ::Exchange#publish.
+ * MQ::Exchange#publish raises error if no connection to broker.
+ * MQ::Queue only wraps headers with new MQ::Headers if they are not nil. This allows pops to tell more easily when they've requested a message from an empty queue. See (https://github.com/tmm1/amqp/issues#issue/22)
+ * Support for receiving Headers with zero-size data packets. Such contents with no body frames are totally legit if indicated header size is zero.
+
+Some improvements that are planned:
+ * Support for AMQP::Protocol::Basic::Return method. See (https://github.com/tmm1/amqp/issues#issue/1).
+
+Getting started
+===============
+
+First things first, start with:
+
+  $ gem install arvicco-amqp
+
+To use examples with RabbitMQ, first [install the broker](http://www.rabbitmq.com/install.html).
+
+Then have a look at the various bundled examples:
+
+    ruby examples/mq/pingpong.rb   # 1-1 communication using amq.direct
+    ruby examples/mq/clock.rb      # 1-N communication using amq.fanout
+    ruby examples/mq/stocks.rb     # 1-subscriber communication using amq.topic
+
+    ruby examples/mq/multiclock.rb # header based routing (new rabbitmq feature)
+    ruby examples/mq/ack.rb        # using ack
+    ruby examples/mq/pop.rb        # pop off messages one at a time
+
+    ruby examples/mq/hashtable.rb  # simple async rpc layer
+    ruby examples/mq/primes.rb 4   # parallelized prime number generation
+    ruby examples/mq/logger.rb     # simple logging api
+
+For high level API documentation see MQ class.
+For more details into the lower level AMQP client API, run the simple client example:
+
+    ruby examples/amqp/simple.rb   # low-level AMQP api
+    ruby examples/mq/internal.rb   # low-level Queue/Exchange api
+
+Or refer to protocol/doc.txt, which enumerates packets sent between a server and client
+during a typical session, in both binary and decoded formats.
+
+How to use AMQP gem with Ruby on Rails, Merb, Sinatra and other web frameworks
+==============================================================================
+
+To use AMQP gem from web applications, you would need to have EventMachine reactor running.
+If you use [Thin](http://code.macournoyer.com/thin/), you are set: Thin uses EventMachine under
+the hook.
+
+With other web servers, you need to start EventMachine reactor in it's own thread like this:
+
+    Thread.new { EM.run }
+
+because otherwise EventMachine will block current thread. Then connect to AMQP broker:
+
+    AMQP.connect(:host => "localhost", :user => "guest", :pass => "guest", :vhost => "/")
+
+In a Ruby on Rails app, probably the best place for this code is initializer
+(like config/initializers/amqp.rb). For Merb apps, it is config/init.rb. For
+Sinatra and pure Rack applications, place it next to other configuration
+code.
+
+Same separate thread technique can be used to make EventMachine play nicely with other
+libraries that would block current thread (like [File::Tail](http://rubygems.org/gems/file-tail)).
+
+AMQP gem resources
+==================
+
+* [AMQP gem mailing list](http://groups.google.com/group/ruby-amqp)
+* [AMQP gem at GitHub](http://github.com/tmm1/amqp)
+* [AMQP gem at Rubygems](http://rubygems.org/gems/amqp)
+* [This fork at GitHub](http://github.com/arvicco/amqp)
+* [This fork as a gem at Rubygems](http://rubygems.org/gems/arvicco-amqp)
+
+Running specifications suite
+============================
+
+To run the test suite make sure you have [Rspec2](http://gemcutter.org/gems/rspec) gem installed and run:
+
+    rake spec
+
+The lib/amqp/spec.rb file is generated automatically based on the [AMQP specification](http://www.amqp.org/confluence/display/AMQP/AMQP+Specification). To generate it:
+
+    rake codegen
+
+Writing your own AMQP/evented specs
+===================================
+
+Writing evented specs properly is a bit difficult due to gory details of event loop management. Libraries such as
+[AMQP-Spec](http://github.com/arvicco/amqp-spec) and [Moqueue](https://github.com/danielsdeleo/moqueue) may be helpful.
+
+
+Credits and more information
+============================
+
+(c) 2008—2010 [Aman Gupta](http://github.com/tmm1) (tmm1)
+
+This project was inspired by [py-amqplib](http://barryp.org/software/py-amqplib/), [rabbitmq](http://rabbitmq.com), [qpid](http://qpid.apache.org/) and [rubbyt](http://github.com/rubbyt/rubbyt).
+Special thanks to Dmitriy Samovskiy, Ben Hood and Tony Garnock-Jones.
+
+AMQP brokers
+------------
+
+ * [RabbitMQ](http://rabbitmq.com) (Rabbit Technologies, Erlang/OTP, MPL)
+ * [ZeroMQ](http://www.zeromq.org) (iMatix/FastMQ/Intel, C++, GPL3)
+ * [OpenAMQ](http://openamq.org) (iMatix, C, GPL2)
+ * [ActiveMQ](http://activemq.apache.org) (Apache Foundation, Java, Apache2)
+
+AMQP resources
+--------------
+
+ * Steve Vinoski [explains AMQP](http://steve.vinoski.net/pdf/IEEE-Advanced_Message_Queuing_Protocol.pdf) in his column, Towards Integration
+
+ * John O'Hara on [the history of AMQP](http://www.acmqueue.org/modules.php?name=Content&pa=showpage&pid=485)
+
+ * Dmitriy's [presentation on RabbitMQ/AMQP](http://somic-org.homelinux.org/blog/2008/07/31/slides-for-my-amqprabbitmq-talk/)
+
+ * ZeroMQ's [analysis of the messaging technology market](http://www.zeromq.org/whitepapers:market-analysis)
+
+ * Pieter Hintjens's [background to AMQP](http://www.openamq.org/doc:amqp-background)
+
+ * Barry Pederson's [py-amqplib](http://barryp.org/software/py-amqplib/)
+
+ * Ben Hood on [writing an AMQP client](http://hopper.squarespace.com/blog/2008/6/21/build-your-own-amqp-client.html)
+
+ * Dmitriy Samovskiy introduces [Ruby + QPid + RabbitMQ](http://somic-org.homelinux.org/blog/2008/06/24/ruby-amqp-rabbitmq-example/)
+
+ * Ben Hood's [as3-amqp](http://github.com/0x6e6562/as3-amqp) ([two](http://hopper.squarespace.com/blog/2008/7/4/server-side-as3.html), [three](http://hopper.squarespace.com/blog/2008/3/24/as3-amqp-client-first-cut.html))
+
+ * RabbitMQ's [AMQP protocol code generator](http://hg.rabbitmq.com/rabbitmq-codegen/)
+
+ * Erlang Exchange [presentation on the implementation of RabbitMQ](http://skillsmatter.com/podcast/erlang/presenting-rabbitmq-an-erlang-based-implementatio-nof-amqp) (and on the [LShift blog](http://www.lshift.net/blog/2008/07/01/slides-from-our-erlang-exchange-talk))
+
+ * Jonathan Conway's series on RabbitMQ and using it with Ruby and Merb: [One](http://jaikoo.com/2007/9/4/didn-t-you-get-the-memo), [Two](http://jaikoo.com/2008/2/29/friday-round-up-2008-02-29), [Three](http://jaikoo.com/2008/3/14/oh-hai-rabbitmq), [Four](http://jaikoo.com/2008/3/20/daemonize-rabbitmq)
+
+ * Open Enterprise's series on messaging middleware and AMQP: [Part 1](http://www1.interopsystems.com/analysis/can-amqp-break-ibms-mom-monopoly-part-1.html), [Part 2](http://www1.interopsystems.com/analysis/can-amqp-break-ibms-mom-monopoly-part-2.html), [Part 3](http://www1.interopsystems.com/analysis/can-amqp-break-ibms-mom-monopoly-part-3.html)
+
+Messaging and distributed systems resources
+-------------------------------------------
+
+ * [A Critique of the Remote Procedure Call Paradigm](http://www.cs.vu.nl/~ast/publications/euteco-1988.pdf)
+
+ * [A Note on Distributed Computing](http://research.sun.com/techrep/1994/smli_tr-94-29.pdf)
+
+ * [Convenience Over Correctness](http://steve.vinoski.net/pdf/IEEE-Convenience_Over_Correctness.pdf)
+
+ * [Metaprotocol Taxonomy and Communications Patterns](http://hessian.caucho.com/doc/metaprotocol-taxonomy.xtp)
+
+ * Joe Armstrong on [Erlang messaging vs RPC](http://armstrongonsoftware.blogspot.com/2008/05/road-we-didnt-go-down.html)
+
+ * [SEDA: scalable internet services using message queues](http://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdf)

data/Rakefile

@@ -0,0 +1,24 @@
+require 'rake'
+require 'pathname'
+
+BASE_PATH = Pathname.new(__FILE__).dirname
+LIB_PATH =  BASE_PATH + 'lib'
+PKG_PATH =  BASE_PATH + 'pkg'
+DOC_PATH =  BASE_PATH + 'rdoc'
+
+$LOAD_PATH.unshift LIB_PATH.to_s unless $LOAD_PATH.include? LIB_PATH.to_s
+
+require 'amqp/version'
+
+NAME = 'ivanvanderbyl-amqp'
+CLASS_NAME = AMQP
+VERSION = CLASS_NAME::VERSION
+
+# Load rakefile tasks
+Dir['tasks/*.rake'].sort.each { |file| load file }
+
+desc "Generate AMQP specification classes"
+task :codegen do
+  sh 'ruby protocol/codegen.rb > lib/amqp/spec.rb'
+  sh 'ruby lib/amqp/spec.rb'
+end

data/TODO

@@ -0,0 +1,32 @@
+- breaks with header values that are nil (deleting nil headers in Exchange#publish for now)
+
+- generate amqp/spec.rb from original xml spec
+- add peek and pop to queues
+- use rabbitmq generated consumer tag from basic.consume-ok reply
+
+- allow temporary queues with amq.queue(nil) syntax (use uuids)
+  - use as temp queue in rpc
+  - use uuids for message ids in rpc
+
+- add ack/completed responses for messages
+- deleting queues/bindings/exchanges
+  + queue.unbind
+  - queue.remove or queue.close or queue.delete
+  - exchange.remove
+  - rpc.remove
+
+- handle errors and exceptions
+    binding to a non-existent (or not yet created in clock.rb) exchange
+      #<AMQP::Protocol::Channel::Close:0x11d35d4
+       @class_id=50,
+       @debug=1,
+       @method_id=20,
+       @reply_code=404,
+       @reply_text="NOT_FOUND - no exchange 'clock' in vhost '/'">>]
+
+- handle connection.redirect during connect (for rabbitmq in distributed mode) [or just set insist to true]
+
+- add amq.queue('name').size{ |num| "#{num} messages in the queue" } (send declare passive, look at declare-ok response)
+- clean up MQ.default on disconnect
+
+- add proper handling of Basic::Return (:immediate and :mandatory flags)

data/VERSION

@@ -0,0 +1 @@
+0.6.13.1

data/doc/EXAMPLE_01_PINGPONG

@@ -0,0 +1,2 @@
+== Ping Pong Example
+  :include: mq/pingpong.rb

data/doc/EXAMPLE_02_CLOCK

@@ -0,0 +1,2 @@
+== Clock Example
+  :include: mq/clock.rb

data/doc/EXAMPLE_03_STOCKS

@@ -0,0 +1,2 @@
+== Stocks Example
+  :include: mq/stocks.rb

data/doc/EXAMPLE_04_MULTICLOCK

@@ -0,0 +1,2 @@
+== Muti-format Clock Example
+  :include: mq/multiclock.rb

data/doc/EXAMPLE_05_ACK

@@ -0,0 +1,2 @@
+== Ack Example
+  :include: mq/ack.rb

data/doc/EXAMPLE_05_POP

@@ -0,0 +1,2 @@
+== Pop Example
+  :include: mq/pop.rb

data/doc/EXAMPLE_06_HASHTABLE

@@ -0,0 +1,2 @@
+== HashTable RPC Example
+  :include: mq/hashtable.rb

data/lib/amqp.rb

@@ -0,0 +1,110 @@
+require File.expand_path('../ext/em', __FILE__)
+require File.expand_path('../ext/blankslate', __FILE__)
+  
+%w[ version buffer spec protocol frame client ].each do |file|
+  require File.expand_path("../amqp/#{file}", __FILE__)
+end
+
+module AMQP
+  class << self
+    @logging = false
+    attr_accessor :logging
+    attr_reader :conn, :closing
+    alias :closing? :closing
+    alias :connection :conn
+  end
+
+  def self.connect *args
+    Client.connect *args
+  end
+
+  def self.settings
+    @settings ||= {
+      # server address
+      :host => '127.0.0.1',
+      :port => PORT,
+
+      # login details
+      :user => 'guest',
+      :pass => 'guest',
+      :vhost => '/',
+
+      # connection timeout
+      :timeout => nil,
+
+      # logging
+      :logging => false,
+
+      # ssl
+      :ssl => false
+    }
+  end
+
+  # Must be called to startup the connection to the AMQP server.
+  #
+  # The method takes several arguments and an optional block.
+  #
+  # This takes any option that is also accepted by EventMachine::connect.
+  # Additionally, there are several AMQP-specific options.
+  #
+  # * :user => String (default 'guest')
+  # The username as defined by the AMQP server.
+  # * :pass => String (default 'guest')
+  # The password for the associated :user as defined by the AMQP server.
+  # * :vhost => String (default '/')
+  # The virtual host as defined by the AMQP server.
+  # * :timeout => Numeric (default nil)
+  # Measured in seconds.
+  # * :logging => true | false (default false)
+  # Toggle the extremely verbose logging of all protocol communications
+  # between the client and the server. Extremely useful for debugging.
+  #
+  #  AMQP.start do
+  #    # default is to connect to localhost:5672
+  #
+  #    # define queues, exchanges and bindings here.
+  #    # also define all subscriptions and/or publishers
+  #    # here.
+  #
+  #    # this block never exits unless EM.stop_event_loop
+  #    # is called.
+  #  end
+  #
+  # Most code will use the MQ api. Any calls to MQ.direct / MQ.fanout /
+  # MQ.topic / MQ.queue will implicitly call #start. In those cases,
+  # it is sufficient to put your code inside of an EventMachine.run
+  # block. See the code examples in MQ for details.
+  #
+  def self.start *args, &blk
+    EM.run{
+      @conn ||= connect *args
+      @conn.callback(&blk) if blk
+      @conn
+    }
+  end
+
+  class << self
+    alias :run :start
+  end
+  
+  def self.stop
+    if @conn and not @closing
+      @closing = true
+      @conn.close{
+        yield if block_given?
+        @conn = nil
+        @closing = false
+      }
+    end
+  end
+
+  def self.fork workers
+    EM.fork(workers) do
+      # clean up globals in the fork
+      Thread.current[:mq] = nil
+      AMQP.instance_variable_set('@conn', nil)
+
+      yield
+    end
+  end
+end

data/lib/amqp/buffer.rb

@@ -0,0 +1,270 @@
+if [].map.respond_to? :with_index
+  class Array #:nodoc:
+    def enum_with_index
+      each.with_index
+    end
+  end
+else
+  require 'enumerator'
+end
+
+module AMQP
+  class Buffer #:nodoc: all
+    class Overflow < StandardError; end
+    class InvalidType < StandardError; end
+    
+    def initialize data = ''
+      @data = data
+      @pos = 0
+    end
+
+    attr_reader :pos
+    
+    def data
+      @data.clone
+    end
+    alias :contents :data
+    alias :to_s :data
+
+    def << data
+      @data << data.to_s
+      self
+    end
+    
+    def length
+      @data.bytesize
+    end
+    
+    def empty?
+      pos == length
+    end
+    
+    def rewind
+      @pos = 0
+    end
+    
+    def read_properties *types
+      types.shift if types.first == :properties
+      
+      i = 0
+      values = []
+
+      while props = read(:short)
+        (0..14).each do |n|
+          # no more property types
+          break unless types[i]
+          
+          # if flag is set
+          if props & (1<<(15-n)) != 0
+            if types[i] == :bit
+              # bit values exist in flags only
+              values << true
+            else
+              # save type name for later reading
+              values << types[i]
+            end
+          else
+            # property not set or is false bit
+            values << (types[i] == :bit ? false : nil)
+          end
+
+          i+=1
+        end
+
+        # bit(0) == 0 means no more property flags
+        break unless props & 1 == 1
+      end
+
+      values.map do |value|
+        value.is_a?(Symbol) ? read(value) : value
+      end
+    end
+
+    def read *types
+      if types.first == :properties
+        return read_properties(*types)
+      end
+
+      values = types.map do |type|
+        case type
+        when :octet
+          _read(1, 'C')
+        when :short
+          _read(2, 'n')
+        when :long
+          _read(4, 'N')
+        when :longlong
+          upper, lower = _read(8, 'NN')
+          upper << 32 | lower
+        when :shortstr
+          _read read(:octet)
+        when :longstr
+          _read read(:long)
+        when :timestamp
+          Time.at read(:longlong)
+        when :table
+          t = Hash.new
+
+          table = Buffer.new(read(:longstr))
+          until table.empty?
+            key, type = table.read(:shortstr, :octet)
+            key = key.intern
+            t[key] ||= case type
+                       when 83 # 'S'
+                         table.read(:longstr)
+                       when 73 # 'I'
+                         table.read(:long)
+                       when 68 # 'D'
+                         exp = table.read(:octet)
+                         num = table.read(:long)
+                         num / 10.0**exp
+                       when 84 # 'T'
+                         table.read(:timestamp)
+                       when 70 # 'F'
+                         table.read(:table)
+                       end
+          end
+
+          t
+        when :bit
+          if (@bits ||= []).empty?
+            val = read(:octet)
+            @bits = (0..7).map{|i| (val & 1<<i) != 0 }
+          end
+
+          @bits.shift
+        else
+          raise InvalidType, "Cannot read data of type #{type}"
+        end
+      end
+      
+      types.size == 1 ? values.first : values
+    end
+    
+    def write type, data
+      case type
+      when :octet
+        _write(data, 'C')
+      when :short
+        _write(data, 'n')
+      when :long
+        _write(data, 'N')
+      when :longlong
+        lower =  data & 0xffffffff
+        upper = (data & ~0xffffffff) >> 32
+        _write([upper, lower], 'NN')
+      when :shortstr
+        data = (data || '').to_s
+        _write([data.bytesize, data], 'Ca*')
+      when :longstr
+        if data.is_a? Hash
+          write(:table, data)
+        else
+          data = (data || '').to_s
+          _write([data.bytesize, data], 'Na*')
+        end
+      when :timestamp
+        write(:longlong, data.to_i)
+      when :table
+        data ||= {}
+        write :longstr, (data.inject(Buffer.new) do |table, (key, value)|
+                          table.write(:shortstr, key.to_s)
+
+                          case value
+                          when String
+                            table.write(:octet, 83) # 'S'
+                            table.write(:longstr, value.to_s)
+                          when Fixnum
+                            table.write(:octet, 73) # 'I'
+                            table.write(:long, value)
+                          when Float
+                            table.write(:octet, 68) # 'D'
+                            # XXX there's gotta be a better way to do this..
+                            exp = value.to_s.split('.').last.bytesize
+                            num = value * 10**exp
+                            table.write(:octet, exp)
+                            table.write(:long, num)
+                          when Time
+                            table.write(:octet, 84) # 'T'
+                            table.write(:timestamp, value)
+                          when Hash
+                            table.write(:octet, 70) # 'F'
+                            table.write(:table, value)
+                          end
+
+                          table
+                        end)
+      when :bit
+        [*data].to_enum(:each_slice, 8).each{|bits|
+          write(:octet, bits.enum_with_index.inject(0){ |byte, (bit, i)|
+            byte |= 1<<i if bit
+            byte
+           })
+         }
+      when :properties
+        values = []
+        data.enum_with_index.inject(0) do |short, ((type, value), i)|
+          n = i % 15
+          last = i+1 == data.size
+
+          if (n == 0 and i != 0) or last
+            if data.size > i+1
+              short |= 1<<0
+            elsif last and value
+              values << [type,value]
+              short |= 1<<(15-n)
+            end
+
+            write(:short, short)
+            short = 0
+          end
+
+          if value and !last
+            values << [type,value] 
+            short |= 1<<(15-n)
+          end
+
+          short
+        end
+        
+        values.each do |type, value|
+          write(type, value) unless type == :bit
+        end
+      else
+        raise InvalidType, "Cannot write data of type #{type}"
+      end
+      
+      self
+    end
+
+    def extract
+      begin
+        cur_data, cur_pos = @data.clone, @pos
+        yield self
+      rescue Overflow
+        @data, @pos = cur_data, cur_pos
+        nil
+      end
+    end
+
+    def _read size, pack = nil
+      if @pos + size > length
+        raise Overflow
+      else
+        data = @data[@pos,size]
+        @data[@pos,size] = ''
+        if pack
+          data = data.unpack(pack)
+          data = data.pop if data.size == 1
+        end
+        data
+      end
+    end
+    
+    def _write data, pack = nil
+      data = [*data].pack(pack) if pack
+      @data[@pos,0] = data
+      @pos += data.bytesize
+    end
+  end
+end