ffi-rzmq 0.6.0 → 0.7.0

data/.bnsignore

@@ -0,0 +1,22 @@
+# The list of files that should be ignored by Mr Bones.
+# Lines that start with '#' are comments.
+#
+# A .gitignore file can be used instead by setting it as the ignore
+# file in your Rakefile:
+#
+#   Bones {
+#     ignore_file  '.gitignore'
+#   }
+#
+# For a project with a C extension, the following would be a good set of
+# exclude patterns (uncomment them if you want to use them):
+# *.[oa]
+# *~
+announcement.txt
+coverage
+doc
+pkg
+*.tmproj
+*.gem
+*.rbc
+*.html

data/History.txt

@@ -1,3 +1,39 @@
+== 0.7.0 / 20101222
+  * Improved performance of calls to Socket#getsockopt. There are usually
+    a lot of calls passing RCVMORE, so we now cache those buffers instead
+    of reallocating them every time.
+    
+  * Updated the docs on Poller#poll to warn about a possible busy-loop
+    condition.
+    
+  * Fixed some more specs to conform with the 0mq 2.1 requirement that
+    all sockets must be closed explicitly otherwise the program may
+    hang on exit.
+
+== 0.6.1 / 20101127
+  * API Change!
+    Moved the #version method from the Util module and made it a class
+    method instead. Invoke as ZMQ::Util.version. Used for conditionally
+    enabling certain features based upon the 0mq version that is loaded.
+
+  * Preliminary support for the Windows platform. Patches supplied
+    by arvicco.
+    
+  * Added support for FD and EVENTS socket options. These were added
+    in 0mq 2.1.0. Patches + specs supplied by andrewvc.
+    
+  * Added support for LINGER, RECONNECT_IVL, BACKLOG  and
+    RECOVERY_IVL_MSEC socket options.
+    
+  * Conditionally re-enable the socket finalizer when we are running
+    with 0mq 2.1.0 or later.
+    
+  * Drop support for MRI 1.8.x since the 'ffi' gem has dropped it as a
+    supported Ruby runtime with its 1.0 release. No action is taken to
+    prevent running with MRI 1.8.x but it won't be supported.
+    
+  * Misc. spec fixes. Need more specs!
+
 == 0.6.0 / 20100911
   * API Change! Modified ZMQ::Message by removing automatic memory
     management. While doing some performance tests I saw that

data/README.rdoc

@@ -7,10 +7,10 @@ ffi-rzmq
 This gem wraps the ZeroMQ networking library using the ruby FFI (foreign
 function interface). It's a pure ruby wrapper so this gem can be loaded
 and run by any ruby runtime that supports FFI. That's all of them:
-MRI 1.8.7+, 1.9.x, Rubinius and JRuby.
+MRI 1.9.x, Rubinius and JRuby.
 
 The impetus behind this library was to provide support for ZeroMQ in
-JRuby which has native threads. Unlike MRI, MacRuby, IronRuby and
+JRuby which has native threads. Unlike MRI, IronRuby and
 Rubinius which all have a GIL, JRuby allows for threaded access to ruby
 code from outside extensions. ZeroMQ is heavily threaded, so until the
 other runtimes remove their GIL, JRuby will likely be the best
@@ -28,18 +28,9 @@ This gem needs more tests. I'm certain there are a
 ton of bugs, so please open issues for them here or fork this project,
 fix them, and send me a pull request.
 
-Running this gem with MRI 1.8.x will likely exhibit a LOT of problems in
-the form of hangs, segmentation faults and bus errors. The threading support
-in MRI 1.8.x is irrevocably broken (according to the brains behind the
-Ruby FFI project) so tread carefully.
-
-Running this gem with MRI 1.9.x may also exhibit some hangs particularly at
-shutdown. This is due to a signaling interaction between the 0mq library
-and MRI. If the 0mq library is blocked on a call and the program receives
-a signal (e.g. SIGINT), the Ruby runtime has no opportunity to run its signal
-handler and process it. The 2.1.x branch of 0mq resolves this problem by
-interrupting the blocking call and returning EINTR. The 2.0.x branch will
-not get this fix because the API change breaks backward compatibility.
+The 'ffi' gem has dropped support for MRI 1.8.x. Since this project relies
+on that gem to load and run this code, then this project also no longer
+supports MRI 1.8.x. 
 
 All features are implemented with the exception of the 0mq devices
 (forwarder, queue, streamer). For implementations of these devices, see
@@ -104,10 +95,18 @@ Server code:
     msg = s.recv_string 0
     raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
   end
+  
+== Better Examples
+
+I highly recommend visiting the Learn Ruby 0mq project for a bunch of good code examples.
+
+  http://github.com/andrewvc/learn-ruby-zeromq
+  
+
 
 == REQUIREMENTS:
 
-  * 0mq 2.0.8 or 2.0.9
+  * 0mq 2.0.10 or 2.1+
   
 The ZeroMQ library must be installed on your system in a well-known location
 like /usr/local/lib. This is the default for new ZeroMQ installs.
@@ -115,15 +114,12 @@ like /usr/local/lib. This is the default for new ZeroMQ installs.
 Future releases may include the library as a C extension built at
 time of installation.
 
-  * ffi (> 0.6.3)
+  * ffi (>= 1.0.0)
   
-Install the current master version of FFI that fixes several threading problems
-under MRI 1.9.x. Unfortunately, MRI 1.8.x is irretrievably broken so it isn't
-recommended if you plan to use threads or callbacks.
-
-Code and installation instructions can be found on github:
+This is a requirement for MRI only. Both Rubinius and JRuby have FFI support built
+in as a standard component. Do *not* run this gem under MRI with an old 'ffi' gem.
+It will crash randomly and you will be sad.
 
-  http://github.com/ffi/ffi
 
 == INSTALL:
 

data/Rakefile

@@ -1,4 +1,3 @@
-
 begin
   require 'bones'
 rescue LoadError
@@ -8,12 +7,30 @@ end
 task :default => 'test:run'
 task 'gem:release' => 'test:run'
 
+namespace :win do
+
+  desc 'Build and install gem under Windows. Mr Bones just has to break things using tar.'
+  task :install do
+    PKG_PATH = File.join(File.dirname(__FILE__), 'pkg')
+    NAME = File.basename(File.dirname(__FILE__))
+    rm_rf PKG_PATH
+    system "gem build #{NAME}.gemspec"
+    mkdir_p PKG_PATH
+    mv "#{NAME}-0.6.0.gem", PKG_PATH
+    system "gem install #{PKG_PATH}/#{NAME}-0.6.0.gem"
+  end
+end
+
 Bones {
-  name  'ffi-rzmq'
-  authors  'Chuck Remes'
-  email    'cremes@mac.com'
-  url      'http://github.com/chuckremes/ffi-rzmq'
+  name 'ffi-rzmq'
+  authors 'Chuck Remes'
+  email 'cremes@mac.com'
+  url 'http://github.com/chuckremes/ffi-rzmq'
   readme_file 'README.rdoc'
   ruby_opts.clear # turn off warnings
+  
+  # necessary for MRI; unnecessary for JRuby and RBX
+  # can't enable this until JRuby & RBX have a way of dealing with it cleanly
+  #depend_on 'ffi', '>= 1.0.0'
 }
 

data/examples/local_lat_poll.rb

@@ -0,0 +1,54 @@
+require 'rubygems'
+require 'ffi-rzmq'
+
+if ARGV.length < 3
+  puts "usage: local_lat <connect-to> <message-size> <roundtrip-count>"
+  exit
+end
+
+link = ARGV[0]
+message_size = ARGV[1].to_i
+roundtrip_count = ARGV[2].to_i
+
+#link = "tcp://127.0.0.1:5555"
+
+ctx = ZMQ::Context.new
+s1 = ctx.socket ZMQ::REQ
+s2 = ctx.socket ZMQ::REP
+
+s1.connect link
+s2.bind link
+
+poller = ZMQ::Poller.new
+poller.register_readable s2
+poller.register_readable s1
+
+
+start_time = Time.now
+
+# kick it off
+message = ZMQ::Message.new("a" * message_size)
+s1.send message, ZMQ::NOBLOCK
+i = roundtrip_count
+
+until i.zero?
+  i -= 1
+  
+  begin
+    poller.poll_nonblock
+  rescue ZMQ::PollError => e
+    puts "efault? [#{e.efault?}]"
+    raise
+  end
+
+  poller.readables.each do |socket|
+    received_message = socket.recv_string ZMQ::NOBLOCK
+    socket.send ZMQ::Message.new(received_message), ZMQ::NOBLOCK
+  end  
+end
+
+elapsed_usecs = (Time.now.to_f - start_time.to_f) * 1_000_000
+latency = elapsed_usecs / roundtrip_count / 2
+
+puts "mean latency: %.3f [us]" % latency
+puts "received all messages in %.3f seconds" % (elapsed_usecs / 1_000_000)

data/ffi-rzmq.gemspec

@@ -2,42 +2,42 @@
 
 Gem::Specification.new do |s|
   s.name = %q{ffi-rzmq}
-  s.version = "0.6.0"
+  s.version = "0.7.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Chuck Remes"]
-  s.date = %q{2010-09-21}
+  s.date = %q{2010-12-22}
   s.description = %q{This gem wraps the ZeroMQ networking library using the ruby FFI (foreign
 function interface). It's a pure ruby wrapper so this gem can be loaded
 and run by any ruby runtime that supports FFI. That's all of them:
-MRI 1.8.7+, 1.9.x, Rubinius and JRuby.
+MRI 1.9.x, Rubinius and JRuby.
 
 The impetus behind this library was to provide support for ZeroMQ in
-JRuby which has native threads. Unlike MRI, MacRuby, IronRuby and
+JRuby which has native threads. Unlike MRI, IronRuby and
 Rubinius which all have a GIL, JRuby allows for threaded access to ruby
 code from outside extensions. ZeroMQ is heavily threaded, so until the
 other runtimes remove their GIL, JRuby will likely be the best
 environment to run this library.}
   s.email = %q{cremes@mac.com}
   s.extra_rdoc_files = ["History.txt", "README.rdoc", "examples/README.rdoc", "version.txt"]
-  s.files = [".gitignore", "History.txt", "README.rdoc", "Rakefile", "examples/README.rdoc", "examples/local_lat.rb", "examples/local_lat_zerocopy.rb", "examples/local_throughput.rb", "examples/publish_subscribe.rb", "examples/remote_lat.rb", "examples/remote_lat_zerocopy.rb", "examples/remote_throughput.rb", "examples/reqrep_poll.rb", "examples/request_response.rb", "examples/xreqxrep_poll.rb", "ffi-rzmq.gemspec", "lib/ffi-rzmq.rb", "lib/ffi-rzmq/context.rb", "lib/ffi-rzmq/exceptions.rb", "lib/ffi-rzmq/message.rb", "lib/ffi-rzmq/poll.rb", "lib/ffi-rzmq/poll_items.rb", "lib/ffi-rzmq/socket.rb", "lib/ffi-rzmq/wrapper.rb", "lib/ffi-rzmq/zmq.rb", "spec/context_spec.rb", "spec/message_spec.rb", "spec/pushpull_spec.rb", "spec/reqrep_spec.rb", "spec/socket_spec.rb", "spec/spec_helper.rb", "version.txt"]
+  s.files = [".bnsignore", "History.txt", "README.rdoc", "Rakefile", "examples/README.rdoc", "examples/local_lat.rb", "examples/local_lat_poll.rb", "examples/local_lat_zerocopy.rb", "examples/local_throughput.rb", "examples/publish_subscribe.rb", "examples/remote_lat.rb", "examples/remote_lat_zerocopy.rb", "examples/remote_throughput.rb", "examples/reqrep_poll.rb", "examples/request_response.rb", "ffi-rzmq.gemspec", "lib/ffi-rzmq.rb", "lib/ffi-rzmq/context.rb", "lib/ffi-rzmq/exceptions.rb", "lib/ffi-rzmq/message.rb", "lib/ffi-rzmq/poll.rb", "lib/ffi-rzmq/poll_items.rb", "lib/ffi-rzmq/socket.rb", "lib/ffi-rzmq/wrapper.rb", "lib/ffi-rzmq/zmq.rb", "spec/context_spec.rb", "spec/message_spec.rb", "spec/pushpull_spec.rb", "spec/reqrep_spec.rb", "spec/socket_spec.rb", "spec/spec_helper.rb", "version.txt"]
   s.homepage = %q{http://github.com/chuckremes/ffi-rzmq}
   s.rdoc_options = ["--main", "README.rdoc"]
   s.require_paths = ["lib"]
   s.rubyforge_project = %q{ffi-rzmq}
   s.rubygems_version = %q{1.3.7}
-  s.summary = %q{This gem wraps the ZeroMQ networking library using the ruby FFI (foreign function interface)}
+  s.summary = %q{This gem wraps the ZeroMQ networking library using the ruby FFI (foreign function interface).}
 
   if s.respond_to? :specification_version then
     current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
     s.specification_version = 3
 
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
-      s.add_development_dependency(%q<bones>, [">= 3.4.7"])
+      s.add_development_dependency(%q<bones>, [">= 3.5.4"])
     else
-      s.add_dependency(%q<bones>, [">= 3.4.7"])
+      s.add_dependency(%q<bones>, [">= 3.5.4"])
     end
   else
-    s.add_dependency(%q<bones>, [">= 3.4.7"])
+    s.add_dependency(%q<bones>, [">= 3.5.4"])
   end
 end

data/lib/ffi-rzmq/context.rb

@@ -38,7 +38,7 @@ module ZMQ
     # be allocated.
     #
     def initialize io_threads = 1
-      @sockets ||= []
+      @sockets = []
       @context = LibZMQ.zmq_init io_threads
       @pointer = @context
       error_check ZMQ_INIT_STR, @context.null? ? 1 : 0

data/lib/ffi-rzmq/poll.rb

@@ -28,6 +28,12 @@ module ZMQ
     # registered sockets belongs to an application thread in another
     # Context.
     #
+    # This method will return *immediately* when there are no registered
+    # sockets. In that case, the +timeout+ parameter is not honored. To
+    # prevent a CPU busy-loop, the caller of this method should detect
+    # this possible condition (via #size) and throttle the call
+    # frequency.
+    #
     def poll timeout = :blocking
       unless @items.empty?
         timeout = adjust timeout
@@ -178,7 +184,7 @@ module ZMQ
       if :blocking == timeout || -1 == timeout
         -1
       else
-        timeout *= 1000
+        (timeout * 1000).to_i
       end
     end
   end

data/lib/ffi-rzmq/socket.rb

@@ -27,7 +27,7 @@ module ZMQ
     #
     #  sock = Socket.new(Context.new, ZMQ::REQ, :receiver_class => ZMQ::ManagedMessage)
     #
-    # Advanced users may want to replace the receiver class with their 
+    # Advanced users may want to replace the receiver class with their
     # own custom class. The custom class must conform to the same public API
     # as ZMQ::Message.
     #
@@ -43,13 +43,19 @@ module ZMQ
 
       unless context_ptr.null?
         @socket = LibZMQ.zmq_socket context_ptr, type
-        error_check ZMQ_SOCKET_STR, @socket.null? ? 1 : 0
-        @name = SocketTypeNameMap[type]
+        if @socket
+          error_check ZMQ_SOCKET_STR, @socket.null? ? 1 : 0
+          @name = SocketTypeNameMap[type]
+        else
+          raise ContextError.new ZMQ_SOCKET_STR, 0, ETERM, "Socket pointer was null"
+        end
       else
         raise ContextError.new ZMQ_SOCKET_STR, 0, ETERM, "Context pointer was null"
       end
 
-      #define_finalizer
+      @sockopt_cache = {}
+
+      define_finalizer
     end
 
     # Set the queue options on this socket.
@@ -61,6 +67,10 @@ module ZMQ
     #  ZMQ::RATE
     #  ZMQ::RECOVERY_IVL
     #  ZMQ::MCAST_LOOP
+    #  ZMQ::LINGER
+    #  ZMQ::RECONNECT_IVL
+    #  ZMQ::BACKLOG
+    #  ZMQ::RECOVER_IVL_MSEC
     #
     # Valid +option_name+ values that take a string +option_value+ are:
     #  ZMQ::IDENTITY
@@ -75,13 +85,18 @@ module ZMQ
     def setsockopt option_name, option_value, option_len = nil
       option_value = sanitize_value option_name, option_value
       option_len ||= option_value.size
-      
+
       begin
         case option_name
-        when HWM, SWAP, AFFINITY, RATE, RECOVERY_IVL, MCAST_LOOP, SNDBUF, RCVBUF
+        when HWM, SWAP, AFFINITY, RATE, RECOVERY_IVL, MCAST_LOOP, SNDBUF, RCVBUF, RECOVERY_IVL_MSEC
           option_value_ptr = LibC.malloc option_len
           option_value_ptr.write_long option_value
 
+        when LINGER, RECONNECT_IVL, BACKLOG
+          option_len = 4 # hard-code "int" length to 4 bytes
+          option_value_ptr = LibC.malloc option_len
+          option_value_ptr.write_int option_value
+
         when IDENTITY, SUBSCRIBE, UNSUBSCRIBE
           # note: not checking errno for failed memory allocations :(
           option_value_ptr = LibC.malloc option_len
@@ -114,6 +129,12 @@ module ZMQ
     #  ZMQ::MCAST_LOOP - boolean
     #  ZMQ::SNDBUF - integer
     #  ZMQ::RCVBUF - integer
+    #  ZMQ::FD     - fd in an integer
+    #  ZMQ::EVENTS - bitmap integer
+    #  ZMQ::LINGER - integer measured in milliseconds
+    #  ZMQ::RECONNECT_IVL - integer measured in milliseconds
+    #  ZMQ::BACKLOG - integer
+    #  ZMQ::RECOVER_IVL_MSEC - integer measured in milliseconds
     #
     # Can raise two kinds of exceptions depending on the error.
     # ContextError:: Raised when a socket operation is attempted on a terminated
@@ -125,8 +146,10 @@ module ZMQ
         option_value = FFI::MemoryPointer.new :pointer
         option_length = FFI::MemoryPointer.new :size_t
 
-        unless [RCVMORE, HWM, SWAP, AFFINITY, RATE, RECOVERY_IVL, MCAST_LOOP,
-        IDENTITY, SNDBUF, RCVBUF].include? option_name
+        unless [
+          RCVMORE, HWM, SWAP, AFFINITY, RATE, RECOVERY_IVL, MCAST_LOOP, IDENTITY,
+          SNDBUF, RCVBUF, FD, EVENTS, LINGER, RECONNECT_IVL, BACKLOG, RECOVERY_IVL_MSEC
+        ].include? option_name
         # we didn't understand the passed option argument
         # will force a raise
         error_check ZMQ_SETSOCKOPT_STR, -1
@@ -142,8 +165,10 @@ module ZMQ
       when RCVMORE, MCAST_LOOP
         # boolean return
         ret = option_value.read_long_long != 0
-      when HWM, SWAP, AFFINITY, RATE, RECOVERY_IVL, SNDBUF, RCVBUF
+      when HWM, SWAP, AFFINITY, RATE, RECOVERY_IVL, SNDBUF, RCVBUF, RECOVERY_IVL_MSEC
         ret = option_value.read_long_long
+      when LINGER, RECONNECT_IVL, BACKLOG, FD, EVENTS
+        ret = option_value.read_int
       when IDENTITY
         ret = option_value.read_string(option_length.read_long_long)
       end
@@ -192,12 +217,16 @@ module ZMQ
     error_check ZMQ_CONNECT_STR, result_code
   end
 
-  # Closes the socket. Any unprocessed messages in queue are dropped.
+  # Closes the socket. Any unprocessed messages in queue are sent or dropped
+  # depending upon the value of the socket option ZMQ::LINGER.
   #
   def close
-    remove_finalizer
-    result_code = LibZMQ.zmq_close @socket
-    error_check ZMQ_CLOSE_STR, result_code
+    if @socket
+      remove_finalizer
+      result_code = LibZMQ.zmq_close @socket
+      error_check ZMQ_CLOSE_STR, result_code
+      @socket = nil
+    end
   end
 
   # Queues the message for transmission. Message is assumed to conform to the
@@ -326,21 +355,37 @@ module ZMQ
     flags != NOBLOCK ? error_check(ZMQ_RECV_STR, result_code) : error_check_nonblock(result_code)
   end
 
+  # Calls to ZMQ.getsockopt require us to pass in some pointers. We can cache and save those buffers
+  # for subsequent calls. This is a big perf win for calling RCVMORE which happens quite often.
+  # Cannot save the buffer for the IDENTITY.
   def alloc_temp_sockopt_buffers option_name
-    length = FFI::MemoryPointer.new :int64
-
     case option_name
-    when RCVMORE, MCAST_LOOP, HWM, SWAP, AFFINITY, RATE, RECOVERY_IVL, SNDBUF, RCVBUF
+    when RCVMORE, MCAST_LOOP, HWM, SWAP, AFFINITY, RATE, RECOVERY_IVL, SNDBUF, RCVBUF, RECOVERY_IVL_MSEC
       # int64_t
-      length.write_long_long 8
-      [FFI::MemoryPointer.new(:int64), length]
+      unless @sockopt_cache[:int64]
+        length = FFI::MemoryPointer.new :int64
+        length.write_long_long 8
+        @sockopt_cache[:int64] = [FFI::MemoryPointer.new(:int64), length]
+      end
+      @sockopt_cache[:int64]
+
+    when LINGER, RECONNECT_IVL, BACKLOG, FD, EVENTS
+      # int, 0mq assumes int is 4-bytes
+      unless @sockopt_cache[:int32]
+        length = FFI::MemoryPointer.new :int32
+        length.write_int 4
+        @sockopt_cache[:int32] = [FFI::MemoryPointer.new(:int32), length]
+      end
+      @sockopt_cache[:int32]
+
     when IDENTITY
+      length = FFI::MemoryPointer.new :int64
       # could be a string of up to 255 bytes
       length.write_long_long 255
       [FFI::MemoryPointer.new(255), length]
     end
   end
-  
+
   def sanitize_value option_name, option_value
     case option_name
     when HWM, AFFINITY, SNDBUF, RCVBUF
@@ -352,8 +397,17 @@ module ZMQ
     end
   end
 
-  def define_finalizer
-    ObjectSpace.define_finalizer(self, self.class.close(@socket))
+  # require a minimum of 0mq 2.1.0 to support socket finalizers; it contains important
+  # fixes for sockets and threads so that a garbage collector thread can successfully
+  # reap this resource without crashing
+  if Util.minimum_api?([2, 1, 0])
+    def define_finalizer
+      ObjectSpace.define_finalizer(self, self.class.close(@socket))
+    end
+  else
+    def define_finalizer
+      # no op
+    end
   end
 
   def remove_finalizer