ffi-rzmq 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,92 @@
1
+ == 0.5.0 / 20100606
2
+ * Updated the bindings to conform to the 0mq 2.0.7 release.
3
+ Several parts of the API changed.
4
+
5
+ * Updated all examples to use the new Context api.
6
+
7
+ * Added Socket#getsockopt.
8
+
9
+ * Added a Socket#identity and Socket#identity= method pair to
10
+ allow for easy get/put on socket identities. Useful for async
11
+ request/reply using XREQ/XREP sockets.
12
+
13
+ * Added more specs (slowly but surely).
14
+
15
+ * Support multi-part messages (new as of 2.0.7). I am unsure how
16
+ to best support multi-part messages so the Message (and related)
17
+ API may change in the future. Added Socket#more_parts?.
18
+
19
+ * Lots of fixes. Many classes use finalizers to deallocate native
20
+ memory when they go out of scope; be sure to use JRuby 1.5.1 or
21
+ later to get important finalizer fixes.
22
+
23
+ == 0.4.1 / 20100511
24
+ * I was misusing all of the FFI memory allocator classes. I now
25
+ wrap libc and use malloc/free directly for creating buffers
26
+ used by libzmq.
27
+
28
+ == 0.4.0 / 20100510
29
+ * Changed the Socket#recv method signature to take an optional
30
+ message object as its first argument. This allows the library
31
+ user to allocate and pass in their own message object for the
32
+ purposes of zero-copy. Original behavior was for the library to
33
+ *always* allocate a new message object to receive a message into.
34
+ Hopefully this is the last change required.
35
+
36
+ * Modified the Socket constructor to take an optional hash as its
37
+ final argument. It honors two keys; :receiver_klass and
38
+ :sender_klass. Passing in a new constant for either (or both) keys
39
+ will override the class used by Socket for allocating new
40
+ Message objects.
41
+
42
+ == 0.3.1 / 20100509
43
+ * Modified ZMQ::Message so we have both an UnmanagedMessage where
44
+ memory management is manual via the #close method, and Message where
45
+ memory management is automated via a finalizer method run during
46
+ garbage collection.
47
+
48
+ * Updated ZMQ::Message docs to make it clearer how to use a subclass
49
+ and FFI::Struct to lazily access the message buffer. This gets us as
50
+ close to zero-copy as possible for performance.
51
+
52
+ * Fixed a memory leak in Message where the FFI::Struct backing the
53
+ C struct was not being freed.
54
+
55
+ * Tested the FFI code against MRI 1.8.x and 1.9.x. It works!
56
+
57
+ * Patched a potential problem in LibZMQ::MessageDeallocator. It was
58
+ crashing under MRI because it complained that FFI::Pointer did not
59
+ have a free method. It now checks for :free before calling it.
60
+ Need to investigate this further because it never happened under
61
+ JRuby.
62
+
63
+ * Modified the Socket constructor slightly to allow for using
64
+ unmanaged or managed messages.
65
+
66
+ * Changed the /examples to print a throughput (msgs/s) number upon
67
+ completion.
68
+
69
+ == 0.3.0 / 20100507
70
+ * ZMQ::Socket#send and ZMQ::Socket#recv semantics changed
71
+ * The official 0mq ruby bindings utilize strings for #send and #recv.
72
+ However, to do so requires lots of copying to and from buffers which
73
+ greatly impacts performance. These methods now return a ZMQ::Message
74
+ object which can be subclassed to do lazy evaluation of the buffer.
75
+
76
+ * Added ZMQ::Socket#send_string and ZMQ::Socket#recv_string. They
77
+ automatically convert the messages to strings just like the official
78
+ 0mq ruby bindings.
79
+
80
+ * Fixed bug in ZMQ::Util#error_string
81
+
82
+ * Split the ZMQ::Message class into two classes. The base class called
83
+ UnmanagedMessage requires manual memory management. The Message
84
+ class (used by default by Socket) has a finalizer defined to
85
+ automatically release memory when the message object gets garbage
86
+ collected.
87
+
88
+
89
+ == 0.2.0 / 20100505
90
+
91
+ * 1 major enhancement
92
+ * Birthday!
@@ -0,0 +1,162 @@
1
+ ffi-rzmq
2
+ by Chuck Remes
3
+ http://www.zeromq.org/bindings:ruby-ffi
4
+
5
+ == DESCRIPTION:
6
+
7
+ This gem wraps the ZeroMQ networking library using the ruby FFI (foreign
8
+ function interface). It's a pure ruby wrapper so this gem can be loaded
9
+ and run by any ruby runtime that supports FFI. Right now that means
10
+ MRI 1.8.7, 1.9.x and JRuby.
11
+
12
+ The impetus behind this library was to provide support for ZeroMQ in
13
+ JRuby which has native threads. Unlike MRI, MacRuby, IronRuby and
14
+ Rubinius which all have a GIL, JRuby allows for threaded access to ruby
15
+ code from outside extensions. ZeroMQ is heavily threaded, so until the
16
+ other runtimes remove their GIL, JRuby will likely be the best
17
+ environment to run this library.
18
+
19
+ == PERFORMANCE
20
+
21
+ Using FFI introduces some minimal overhead. In my latest benchmarks,
22
+ I was unable to detect any measurable performance drop due to FFI
23
+ regardless of which ruby runtime was tested. JRuby had the best overall
24
+ performance (with --server) once it warmed up. MRI behaved quite well
25
+ too and has a much lower memory footprint than JRuby (use the trunk
26
+ version of the FFI bindings to fix several threading issues affecting
27
+ MRI).
28
+
29
+ Due to odd interactions between 0mq threads and the GIL (Giant Interpreter
30
+ Lock), I recommend using JRuby 1.5.1 or later. JRuby has no GIL.
31
+
32
+ The hope is that in a multi-threaded environment that JRuby's native
33
+ threads and lack of GIL will provide the best ZeroMQ performance using
34
+ the ruby language.
35
+
36
+ Unfortunately, there is really no reasonable way to support zero-copy
37
+ using Ruby. Any time data needs to be accessible by the Ruby runtime,
38
+ it must be copied out of native memory to the Ruby heap. The same is
39
+ true for the reverse. I am investigating ways to "pin" primitive arrays
40
+ in memory for Rubinius and JRuby to achieve zero-copy, but that is
41
+ a ways off.
42
+
43
+ == FEATURES/PROBLEMS:
44
+
45
+ This gem is brand new and has minimal tests. I'm certain there are a
46
+ ton of bugs, so please open issues for them here or fork this project,
47
+ fix them, and send me a pull request.
48
+
49
+ All features are implemented with the exception of the 0mq devices
50
+ (forwarder, queue, streamer).
51
+
52
+ == SYNOPSIS:
53
+
54
+ Client code:
55
+
56
+ require 'rubygems'
57
+ require 'ffi-rzmq'
58
+
59
+ if ARGV.length < 3
60
+ puts "usage: local_lat <connect-to> <message-size> <roundtrip-count>"
61
+ exit
62
+ end
63
+
64
+ bind_to = ARGV[0]
65
+ message_size = ARGV[1].to_i
66
+ roundtrip_count = ARGV[2].to_i
67
+
68
+ ctx = ZMQ::Context.new 1
69
+ s = ctx.socket ZMQ::REP
70
+ s.setsockopt(ZMQ::HWM, 100)
71
+ s.bind(bind_to)
72
+
73
+ roundtrip_count.times do
74
+ msg = s.recv_string 0
75
+ raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
76
+ s.send_string msg, 0
77
+ end
78
+
79
+ # give the lib time to flush any remaining messages
80
+ sleep 1
81
+
82
+ Server code:
83
+
84
+ require 'rubygems'
85
+ require 'ffi-rzmq'
86
+
87
+ if ARGV.length < 3
88
+ puts "usage: remote_lat <connect-to> <message-size> <roundtrip-count>"
89
+ exit
90
+ end
91
+
92
+ connect_to = ARGV[0]
93
+ message_size = ARGV[1].to_i
94
+ roundtrip_count = ARGV[2].to_i
95
+
96
+ ctx = ZMQ::Context.new 1
97
+ s = ctx.socket ZMQ::REQ
98
+ s.connect(connect_to)
99
+
100
+ msg = "#{ '3' * message_size }"
101
+
102
+ start_time = Time.now
103
+
104
+ roundtrip_count.times do
105
+ s.send_string msg, 0
106
+ msg = s.recv_string 0
107
+ raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
108
+ end
109
+
110
+ == REQUIREMENTS:
111
+
112
+ * 0mq 2.0.7
113
+
114
+ The ZeroMQ library must be installed on your system in a well-known location
115
+ like /usr/local/lib. This is the default for new ZeroMQ installs.
116
+
117
+ Future releases may include the library as a C extension built at
118
+ time of installation.
119
+
120
+ * ffi (> 0.6.3)
121
+
122
+ Install the current master version of FFI that fixes several threading problems
123
+ under MRI 1.9.x. Unfortunately, MRI 1.8.x is irretrievably broken so it isn't
124
+ recommended if you plan to use threads or callbacks.
125
+
126
+ Code and installation instructions can be found on github:
127
+
128
+ http://github.com/ffi/ffi
129
+
130
+ == INSTALL:
131
+
132
+ Make sure the ZeroMQ library is already installed on your system. Secondly,
133
+ make sure the FFI gem is built from its project master (see Requirements).
134
+
135
+ % gem build ffi-rzmq.gemspec
136
+ % gem install ffi-rzmq-*.gem
137
+
138
+
139
+ == LICENSE:
140
+
141
+ (The MIT License)
142
+
143
+ Copyright (c) 2010 Chuck Remes
144
+
145
+ Permission is hereby granted, free of charge, to any person obtaining
146
+ a copy of this software and associated documentation files (the
147
+ 'Software'), to deal in the Software without restriction, including
148
+ without limitation the rights to use, copy, modify, merge, publish,
149
+ distribute, sublicense, and/or sell copies of the Software, and to
150
+ permit persons to whom the Software is furnished to do so, subject to
151
+ the following conditions:
152
+
153
+ The above copyright notice and this permission notice shall be
154
+ included in all copies or substantial portions of the Software.
155
+
156
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
157
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
158
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
159
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
160
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
161
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
162
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,19 @@
1
+
2
+ begin
3
+ require 'bones'
4
+ rescue LoadError
5
+ abort '### Please install the "bones" gem ###'
6
+ end
7
+
8
+ task :default => 'test:run'
9
+ task 'gem:release' => 'test:run'
10
+
11
+ Bones {
12
+ name 'ffi-rzmq'
13
+ authors 'Chuck Remes'
14
+ email 'cremes@mac.com'
15
+ url 'http://github.com/chuckremes/ffi-rzmq'
16
+ readme_file 'README.rdoc'
17
+ ruby_opts.clear # turn off warnings
18
+ }
19
+
@@ -0,0 +1,43 @@
1
+ #
2
+ # Copyright (c) 2007-2010 iMatix Corporation
3
+ #
4
+ # This file is part of 0MQ.
5
+ #
6
+ # 0MQ is free software; you can redistribute it and/or modify it under
7
+ # the terms of the Lesser GNU General Public License as published by
8
+ # the Free Software Foundation; either version 3 of the License, or
9
+ # (at your option) any later version.
10
+ #
11
+ # 0MQ is distributed in the hope that it will be useful,
12
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
13
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14
+ # Lesser GNU General Public License for more details.
15
+ #
16
+ # You should have received a copy of the Lesser GNU General Public License
17
+ # along with this program. If not, see <http://www.gnu.org/licenses/>.
18
+
19
+ require 'rubygems'
20
+ require 'ffi-rzmq'
21
+
22
+ if ARGV.length < 3
23
+ puts "usage: local_lat <connect-to> <message-size> <roundtrip-count>"
24
+ exit
25
+ end
26
+
27
+ bind_to = ARGV[0]
28
+ message_size = ARGV[1].to_i
29
+ roundtrip_count = ARGV[2].to_i
30
+
31
+ ctx = ZMQ::Context.new 1
32
+ s = ctx.socket ZMQ::REP
33
+ s.setsockopt(ZMQ::HWM, 100)
34
+ s.bind(bind_to)
35
+
36
+ roundtrip_count.times do
37
+ msg = s.recv_string 0
38
+ raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
39
+ s.send_string msg, 0
40
+ end
41
+
42
+ # give the lib time to flush any remaining messages
43
+ sleep 1
@@ -0,0 +1,24 @@
1
+ require 'rubygems'
2
+ require 'ffi-rzmq'
3
+
4
+ if ARGV.length < 3
5
+ puts "usage: local_lat <connect-to> <message-size> <roundtrip-count>"
6
+ exit
7
+ end
8
+
9
+ bind_to = ARGV[0]
10
+ message_size = ARGV[1].to_i
11
+ roundtrip_count = ARGV[2].to_i
12
+
13
+ ctx = ZMQ::Context.new 1
14
+ s = ZMQ::Socket.new ctx.context, ZMQ::REP
15
+ s.setsockopt ZMQ::HWM, 100
16
+ s.bind bind_to
17
+
18
+ msg = ZMQ::Message.new
19
+
20
+ roundtrip_count.times do
21
+ result = s.recv msg, 0
22
+ raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
23
+ s.send msg, 0
24
+ end
@@ -0,0 +1,52 @@
1
+ require 'rubygems'
2
+ require 'ffi-rzmq'
3
+
4
+
5
+ link = "tcp://127.0.0.1:5555"
6
+
7
+ ctx = ZMQ::Context.new 1
8
+ s1 = ctx.socket ZMQ::PUB
9
+ s2 = ctx.socket ZMQ::SUB
10
+ s3 = ctx.socket ZMQ::SUB
11
+ s4 = ctx.socket ZMQ::SUB
12
+ s5 = ctx.socket ZMQ::SUB
13
+
14
+ s2.setsockopt ZMQ::SUBSCRIBE, '' # receive all
15
+ s3.setsockopt ZMQ::SUBSCRIBE, 'animals' # receive any starting with this string
16
+ s4.setsockopt ZMQ::SUBSCRIBE, 'animals.dog'
17
+ s5.setsockopt ZMQ::SUBSCRIBE, 'animals.cat'
18
+
19
+ s1.bind link
20
+ s2.connect link
21
+ s3.connect link
22
+ s4.connect link
23
+ s5.connect link
24
+
25
+ sleep 1
26
+
27
+ topic = "animals.dog"
28
+ payload = "Animal crackers!"
29
+
30
+ s1.identity = "publisher-A"
31
+ puts "sending"
32
+ # use the new multi-part messaging support to
33
+ # automatically separate the topic from the body
34
+ s1.send_string topic, ZMQ::SNDMORE
35
+ s1.send_string payload, ZMQ::SNDMORE
36
+ s1.send_string s1.identity
37
+
38
+ topic = s2.recv_string
39
+ body = s2.recv_string if s2.more_parts?
40
+ identity = s2.recv_string if s2.more_parts?
41
+ puts "s2 received topic [#{topic}], body [#{body}], identity [#{identity}]"
42
+
43
+ topic = s3.recv_string
44
+ body = s3.recv_string if s3.more_parts?
45
+ puts "s3 received topic [#{topic}], body [#{body}]"
46
+
47
+ topic = s4.recv_string
48
+ body = s4.recv_string if s4.more_parts?
49
+ puts "s4 received topic [#{topic}], body [#{body}]"
50
+
51
+ s5_string = s5.recv_string ZMQ::NOBLOCK
52
+ puts(s5_string.nil? ? "s5 received no messages" : "s5 FAILED")
@@ -0,0 +1,53 @@
1
+ #
2
+ # Copyright (c) 2007-2010 iMatix Corporation
3
+ #
4
+ # This file is part of 0MQ.
5
+ #
6
+ # 0MQ is free software; you can redistribute it and/or modify it under
7
+ # the terms of the Lesser GNU General Public License as published by
8
+ # the Free Software Foundation; either version 3 of the License, or
9
+ # (at your option) any later version.
10
+ #
11
+ # 0MQ is distributed in the hope that it will be useful,
12
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
13
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14
+ # Lesser GNU General Public License for more details.
15
+ #
16
+ # You should have received a copy of the Lesser GNU General Public License
17
+ # along with this program. If not, see <http://www.gnu.org/licenses/>.
18
+
19
+ require 'rubygems'
20
+ require 'ffi-rzmq'
21
+
22
+ if ARGV.length < 3
23
+ puts "usage: remote_lat <connect-to> <message-size> <roundtrip-count>"
24
+ exit
25
+ end
26
+
27
+ connect_to = ARGV[0]
28
+ message_size = ARGV[1].to_i
29
+ roundtrip_count = ARGV[2].to_i
30
+
31
+ ctx = ZMQ::Context.new 1
32
+ s = ctx.socket ZMQ::REQ
33
+ s.connect(connect_to)
34
+
35
+ msg = "#{ '3' * message_size }"
36
+
37
+ start_time = Time.now
38
+
39
+ roundtrip_count.times do
40
+ s.send_string msg, 0
41
+ msg = s.recv_string 0
42
+ raise "Message size doesn't match, expected [#{message_size}] but received [#{msg.size}]" if message_size != msg.size
43
+ end
44
+
45
+ end_time = Time.now
46
+ elapsed_secs = (end_time.to_f - start_time.to_f)
47
+ elapsed_usecs = elapsed_secs * 1000000
48
+ latency = elapsed_usecs / roundtrip_count / 2
49
+
50
+ puts "message size: %i [B]" % message_size
51
+ puts "roundtrip count: %i" % roundtrip_count
52
+ puts "throughput (msgs/s): %i" % (roundtrip_count / elapsed_secs)
53
+ puts "mean latency: %.3f [us]" % latency