ffi-rzmq 0.5.0

data/History.txt

@@ -0,0 +1,92 @@
+== 0.5.0 / 20100606
+  * Updated the bindings to conform to the 0mq 2.0.7 release.
+  Several parts of the API changed.
+  
+  * Updated all examples to use the new Context api.
+  
+  * Added Socket#getsockopt.
+  
+  * Added a Socket#identity and Socket#identity= method pair to
+  allow for easy get/put on socket identities. Useful for async
+  request/reply using XREQ/XREP sockets.
+  
+  * Added more specs (slowly but surely).
+  
+  * Support multi-part messages (new as of 2.0.7). I am unsure how
+  to best support multi-part messages so the Message (and related)
+  API may change in the future. Added Socket#more_parts?.
+  
+  * Lots of fixes. Many classes use finalizers to deallocate native
+  memory when they go out of scope; be sure to use JRuby 1.5.1 or
+  later to get important finalizer fixes.
+
+== 0.4.1 / 20100511
+  * I was misusing all of the FFI memory allocator classes. I now
+  wrap libc and use malloc/free directly for creating buffers
+  used by libzmq.
+
+== 0.4.0 / 20100510
+  * Changed the Socket#recv method signature to take an optional
+  message object as its first argument. This allows the library
+  user to allocate and pass in their own message object for the 
+  purposes of zero-copy. Original behavior was for the library to
+  *always* allocate a new message object to receive a message into.
+  Hopefully this is the last change required.
+  
+  * Modified the Socket constructor to take an optional hash as its
+  final argument. It honors two keys; :receiver_klass and
+  :sender_klass. Passing in a new constant for either (or both) keys
+  will override the class used by Socket for allocating new
+  Message objects. 
+
+== 0.3.1 / 20100509
+  * Modified ZMQ::Message so we have both an UnmanagedMessage where
+  memory management is manual via the #close method, and Message where
+  memory management is automated via a finalizer method run during
+  garbage collection.
+  
+  * Updated ZMQ::Message docs to make it clearer how to use a subclass
+  and FFI::Struct to lazily access the message buffer. This gets us as
+  close to zero-copy as possible for performance.
+  
+  * Fixed a memory leak in Message where the FFI::Struct backing the
+  C struct was not being freed.
+  
+  * Tested the FFI code against MRI 1.8.x and 1.9.x. It works!
+  
+  * Patched a potential problem in LibZMQ::MessageDeallocator. It was
+  crashing under MRI because it complained that FFI::Pointer did not
+  have a free method. It now checks for :free before calling it. 
+  Need to investigate this further because it never happened under
+  JRuby.
+  
+  * Modified the Socket constructor slightly to allow for using
+  unmanaged or managed messages.
+  
+  * Changed the /examples to print a throughput (msgs/s) number upon
+  completion.
+
+== 0.3.0 / 20100507
+* ZMQ::Socket#send and ZMQ::Socket#recv semantics changed
+  * The official 0mq ruby bindings utilize strings for #send and #recv.
+    However, to do so requires lots of copying to and from buffers which
+    greatly impacts performance. These methods now return a ZMQ::Message
+    object which can be subclassed to do lazy evaluation of the buffer.
+    
+  * Added ZMQ::Socket#send_string and ZMQ::Socket#recv_string. They
+    automatically convert the messages to strings just like the official
+    0mq ruby bindings.
+
+  * Fixed bug in ZMQ::Util#error_string
+  
+  * Split the ZMQ::Message class into two classes. The base class called
+    UnmanagedMessage requires manual memory management. The Message
+    class (used by default by Socket) has a finalizer defined to
+    automatically release memory when the message object gets garbage
+    collected.
+  
+  
+== 0.2.0 / 20100505
+
+* 1 major enhancement
+  * Birthday!

data/README.rdoc

@@ -0,0 +1,162 @@
+ffi-rzmq
+    by Chuck Remes
+    http://www.zeromq.org/bindings:ruby-ffi
+
+== DESCRIPTION:
+
+This gem wraps the ZeroMQ networking library using the ruby FFI (foreign
+function interface). It's a pure ruby wrapper so this gem can be loaded
+and run by any ruby runtime that supports FFI. Right now that means
+MRI 1.8.7, 1.9.x and JRuby.
+
+The impetus behind this library was to provide support for ZeroMQ in
+JRuby which has native threads. Unlike MRI, MacRuby, IronRuby and
+Rubinius which all have a GIL, JRuby allows for threaded access to ruby
+code from outside extensions. ZeroMQ is heavily threaded, so until the
+other runtimes remove their GIL, JRuby will likely be the best
+environment to run this library.
+
+== PERFORMANCE
+
+Using FFI introduces some minimal overhead. In my latest benchmarks,
+I was unable to detect any measurable performance drop due to FFI
+regardless of which ruby runtime was tested. JRuby had the best overall
+performance (with --server) once it warmed up. MRI behaved quite well 
+too and has a much lower memory footprint than JRuby (use the trunk
+version of the FFI bindings to fix several threading issues affecting
+MRI).
+
+Due to odd interactions between 0mq threads and the GIL (Giant Interpreter
+Lock), I recommend using JRuby 1.5.1 or later. JRuby has no GIL.
+
+The hope is that in a multi-threaded environment that JRuby's native
+threads and lack of GIL will provide the best ZeroMQ performance using
+the ruby language.
+
+Unfortunately, there is really no reasonable way to support zero-copy
+using Ruby. Any time data needs to be accessible by the Ruby runtime,
+it must be copied out of native memory to the Ruby heap. The same is
+true for the reverse. I am investigating ways to "pin" primitive arrays
+in memory for Rubinius and JRuby to achieve zero-copy, but that is
+a ways off.
+
+== FEATURES/PROBLEMS:
+
+This gem is brand new and has minimal tests. I'm certain there are a
+ton of bugs, so please open issues for them here or fork this project,
+fix them, and send me a pull request.
+
+All features are implemented with the exception of the 0mq devices
+(forwarder, queue, streamer).
+
+== SYNOPSIS:
+
+Client code:
+  
+  require 'rubygems'
+  require 'ffi-rzmq'
+  
+  if ARGV.length < 3
+    puts "usage: local_lat <connect-to> <message-size> <roundtrip-count>"
+    exit
+  end
+  
+  bind_to = ARGV[0]
+  message_size = ARGV[1].to_i
+  roundtrip_count = ARGV[2].to_i
+  
+  ctx = ZMQ::Context.new 1
+  s = ctx.socket ZMQ::REP
+  s.setsockopt(ZMQ::HWM, 100)
+  s.bind(bind_to)
+  
+  roundtrip_count.times do
+    msg = s.recv_string 0
+    raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
+    s.send_string msg, 0
+  end
+  
+  # give the lib time to flush any remaining messages
+  sleep 1
+
+Server code:
+
+  require 'rubygems'
+  require 'ffi-rzmq'
+  
+  if ARGV.length < 3
+    puts "usage: remote_lat <connect-to> <message-size> <roundtrip-count>"
+    exit
+  end
+  
+  connect_to = ARGV[0]
+  message_size = ARGV[1].to_i
+  roundtrip_count = ARGV[2].to_i
+  
+  ctx = ZMQ::Context.new 1
+  s = ctx.socket ZMQ::REQ
+  s.connect(connect_to)
+  
+  msg = "#{ '3' * message_size }"
+  
+  start_time = Time.now
+  
+  roundtrip_count.times do
+    s.send_string msg, 0
+    msg = s.recv_string 0
+    raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
+  end
+
+== REQUIREMENTS:
+
+  * 0mq 2.0.7
+  
+The ZeroMQ library must be installed on your system in a well-known location
+like /usr/local/lib. This is the default for new ZeroMQ installs.
+
+Future releases may include the library as a C extension built at
+time of installation.
+
+  * ffi (> 0.6.3)
+  
+Install the current master version of FFI that fixes several threading problems
+under MRI 1.9.x. Unfortunately, MRI 1.8.x is irretrievably broken so it isn't
+recommended if you plan to use threads or callbacks.
+
+Code and installation instructions can be found on github:
+
+  http://github.com/ffi/ffi
+
+== INSTALL:
+
+Make sure the ZeroMQ library is already installed on your system. Secondly,
+make sure the FFI gem is built from its project master (see Requirements).
+
+ % gem build ffi-rzmq.gemspec
+ % gem install ffi-rzmq-*.gem
+ 
+
+== LICENSE:
+
+(The MIT License)
+
+Copyright (c) 2010 Chuck Remes
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,19 @@
+
+begin
+  require 'bones'
+rescue LoadError
+  abort '### Please install the "bones" gem ###'
+end
+
+task :default => 'test:run'
+task 'gem:release' => 'test:run'
+
+Bones {
+  name  'ffi-rzmq'
+  authors  'Chuck Remes'
+  email    'cremes@mac.com'
+  url      'http://github.com/chuckremes/ffi-rzmq'
+  readme_file 'README.rdoc'
+  ruby_opts.clear # turn off warnings
+}
+

data/examples/local_lat.rb

@@ -0,0 +1,43 @@
+#
+#    Copyright (c) 2007-2010 iMatix Corporation
+#
+#    This file is part of 0MQ.
+#
+#    0MQ is free software; you can redistribute it and/or modify it under
+#    the terms of the Lesser GNU General Public License as published by
+#    the Free Software Foundation; either version 3 of the License, or
+#    (at your option) any later version.
+#
+#    0MQ is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    Lesser GNU General Public License for more details.
+#
+#    You should have received a copy of the Lesser GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+require 'rubygems'
+require 'ffi-rzmq'
+
+if ARGV.length < 3
+  puts "usage: local_lat <connect-to> <message-size> <roundtrip-count>"
+  exit
+end
+
+bind_to = ARGV[0]
+message_size = ARGV[1].to_i
+roundtrip_count = ARGV[2].to_i
+
+ctx = ZMQ::Context.new 1
+s = ctx.socket ZMQ::REP
+s.setsockopt(ZMQ::HWM, 100)
+s.bind(bind_to)
+
+roundtrip_count.times do
+  msg = s.recv_string 0
+  raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
+  s.send_string msg, 0
+end
+
+# give the lib time to flush any remaining messages
+sleep 1

data/examples/local_lat_zerocopy.rb

@@ -0,0 +1,24 @@
+require 'rubygems'
+require 'ffi-rzmq'
+
+if ARGV.length < 3
+  puts "usage: local_lat <connect-to> <message-size> <roundtrip-count>"
+  exit
+end
+
+bind_to = ARGV[0]
+message_size = ARGV[1].to_i
+roundtrip_count = ARGV[2].to_i
+
+ctx = ZMQ::Context.new 1
+s = ZMQ::Socket.new ctx.context, ZMQ::REP
+s.setsockopt ZMQ::HWM, 100
+s.bind bind_to
+
+msg = ZMQ::Message.new
+
+roundtrip_count.times do
+  result = s.recv msg, 0
+  raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
+  s.send msg, 0
+end

data/examples/publish_subscribe.rb

@@ -0,0 +1,52 @@
+require 'rubygems'
+require 'ffi-rzmq'
+
+
+link = "tcp://127.0.0.1:5555"
+
+ctx = ZMQ::Context.new 1
+s1 = ctx.socket ZMQ::PUB
+s2 = ctx.socket ZMQ::SUB
+s3 = ctx.socket ZMQ::SUB
+s4 = ctx.socket ZMQ::SUB
+s5 = ctx.socket ZMQ::SUB
+
+s2.setsockopt ZMQ::SUBSCRIBE, '' # receive all
+s3.setsockopt ZMQ::SUBSCRIBE, 'animals' # receive any starting with this string
+s4.setsockopt ZMQ::SUBSCRIBE, 'animals.dog'
+s5.setsockopt ZMQ::SUBSCRIBE, 'animals.cat'
+
+s1.bind link
+s2.connect link
+s3.connect link
+s4.connect link
+s5.connect link
+
+sleep 1
+
+topic = "animals.dog"
+payload = "Animal crackers!"
+
+s1.identity = "publisher-A"
+puts "sending"
+# use the new multi-part messaging support to
+# automatically separate the topic from the body
+s1.send_string topic, ZMQ::SNDMORE
+s1.send_string payload, ZMQ::SNDMORE
+s1.send_string s1.identity
+
+topic = s2.recv_string
+body = s2.recv_string if s2.more_parts?
+identity = s2.recv_string if s2.more_parts?
+puts "s2 received topic [#{topic}], body [#{body}], identity [#{identity}]"
+
+topic = s3.recv_string
+body = s3.recv_string if s3.more_parts?
+puts "s3 received topic [#{topic}], body [#{body}]"
+
+topic = s4.recv_string
+body = s4.recv_string if s4.more_parts?
+puts "s4 received topic [#{topic}], body [#{body}]"
+
+s5_string = s5.recv_string ZMQ::NOBLOCK
+puts(s5_string.nil? ? "s5 received no messages" : "s5 FAILED")

data/examples/remote_lat.rb

@@ -0,0 +1,53 @@
+#
+#    Copyright (c) 2007-2010 iMatix Corporation
+#
+#    This file is part of 0MQ.
+#
+#    0MQ is free software; you can redistribute it and/or modify it under
+#    the terms of the Lesser GNU General Public License as published by
+#    the Free Software Foundation; either version 3 of the License, or
+#    (at your option) any later version.
+#
+#    0MQ is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    Lesser GNU General Public License for more details.
+#
+#    You should have received a copy of the Lesser GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+require 'rubygems'
+require 'ffi-rzmq'
+
+if ARGV.length < 3
+  puts "usage: remote_lat <connect-to> <message-size> <roundtrip-count>"
+  exit
+end
+
+connect_to = ARGV[0]
+message_size = ARGV[1].to_i
+roundtrip_count = ARGV[2].to_i
+
+ctx = ZMQ::Context.new 1
+s = ctx.socket ZMQ::REQ
+s.connect(connect_to)
+
+msg = "#{ '3' * message_size }"
+
+start_time = Time.now
+
+roundtrip_count.times do
+  s.send_string msg, 0
+  msg = s.recv_string 0
+  raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
+end
+
+end_time = Time.now
+elapsed_secs = (end_time.to_f - start_time.to_f)
+elapsed_usecs = elapsed_secs * 1000000
+latency = elapsed_usecs / roundtrip_count / 2
+
+puts "message size: %i [B]" % message_size
+puts "roundtrip count: %i" % roundtrip_count
+puts "throughput (msgs/s): %i" % (roundtrip_count / elapsed_secs)
+puts "mean latency: %.3f [us]" % latency