emdrb 0.2.0 → 0.3.0

data/History.txt

@@ -1,3 +1,12 @@
+== 0.3.0 / 2009-01-27
+
+* Added provisions for DRb servers to have methods which are
+  aware of EventMachine.
+* Made internal changes so that other transports can be more easily supported.
+* Unix domain socket transport added.
+* Connection closing and stop_service are made to work.
+* Added support for standard DRb ACLs.
+
 == 0.2.0 / 2009-01-23
 
 * DRb client implementation, including asynchronous calls.

data/Manifest.txt

@@ -6,6 +6,7 @@ README.txt
 Rakefile
 lib/emdrb.rb
 lib/emdrb/emdrb.rb
+lib/emdrb/unix.rb
 lib/emdrb/version.rb
 spec/client_spec.rb
 spec/drbserver.rb

data/README.txt

@@ -16,25 +16,35 @@ that uses traditional Ruby sockets.  This should at the very least
 play better with other programs that have an EventMachine event loop,
 and hopefully provide somewhat better scalability.
 
-Obviously, this is a quick and dirty release, just to get something
-out there, and of course it has a number of limitations:
+EMDRb already supports the following features of the standard DRb:
 
-* No SSL support.
-* No support for ACLs.
-* No support for DRb over Unix domain sockets.
+* Clients and servers over TCP/IP
+* Clients and servers over Unix domain sockets
+* Standard distributed Ruby ACLs
+
+It also has the following extensions, which derive from its basis in
+EventMachine:
+
+* Client-side asynchronous method calls
+* EventMachine-aware server methods capable of asynchrony.
+
+It is, however, not yet a complete replacement for the standard DRb:
+
+* No SSL support.  To support this fully, EventMachine needs to have
+  more comprehensive SSL/TLS support.
 * RSpec tests are very basic, and need a lot more comprehensive work.
 * Many standard configuration options for DRb still unsupported
 
 These and many other problems are scheduled to be addressed in the
-next release.
+following releases.
 
 == SYNOPSIS:
 
-EMDRb basically reopens several classes and adds methods and overrides
-some methods in the basic distributed Ruby implementation to make it
-use EventMachine's infrastructure instead of the normal networking
-layer.  One could do the following, which is practically identical to
-one of the examples for distributed Ruby:
+EMDRb basically reopens several classes, adds methods, and overrides other
+methods in the basic distributed Ruby implementation to make it use
+EventMachine's infrastructure instead of the traditional Ruby networking
+code.  One could do the following, which is practically identical to one
+of the examples for distributed Ruby:
 
   require 'emdrb'
 
@@ -96,5 +106,5 @@ Copyright © 2008, 2009 Rafael R. Sevilla.  You can redistribute it
 and/or modify it under the same terms as Ruby.  Please see the file
 COPYING for more details.
 
-$Id: README.txt 63 2009-01-23 09:15:55Z dido $
+$Id: README.txt 81 2009-01-29 04:46:14Z dido $
 

data/lib/emdrb/emdrb.rb

@@ -4,7 +4,7 @@
 # Homepage:: http://emdrb.rubyforge.org/
 # License:: GNU General Public License / Ruby License
 #
-# $Id: emdrb.rb 49 2009-01-23 08:51:47Z dido $
+# $Id: emdrb.rb 78 2009-01-29 04:21:25Z dido $
 #
 #----------------------------------------------------------------------------
 #
@@ -28,6 +28,44 @@ module DRb
   DEFAULT_LOAD_LIMIT = 256 * 102400
   DEFAULT_SAFE_LEVEL = 0
 
+  ##
+  # The DRbEMSafe module allows the designation of certain methods as
+  # 'deferrable methods' which are run from within the event loop rather
+  # than within a thread.  It gives one the ability to circumvent the
+  # sugaring and black magic that EMDRb does in order to provide the
+  # complete illusion that remote objects are the same as local ones,
+  # most especially with respect to blocks.  This may also provide more
+  # performance if one is using a method that itself makes use of
+  # Deferrables to accomplish the job.
+  # 
+  module DRbEMSafe
+    module ClassMethods
+      ##
+      # Mark the method as a deferrable method.  Such a method must
+      # accept two arguments, an array of parameters and a block
+      # (usually nil).  The method must always return a Deferrable
+      # which should be set to success with the result of the method
+      # when the method is done, and failed with the exception object
+      # if the method failed.  The block, if any, is a DRbObject that
+      # should be invoked by send_async(:call).  Using a direct call
+      # on the DRbObject will most likely result in a threading
+      # deadlock since deferrable methods are invoked from the main
+      # thread!
+      def deferrable_method(method_name)
+        @deferrable_methods ||= {}
+        @deferrable_methods[method_name] = true
+      end
+
+      def deferrable_method?(method_name)
+        return(@deferrable_methods.has_key?(method_name))
+      end
+    end
+
+    def self.included(base)
+      base.extend(ClassMethods)
+    end
+  end
+
   ##
   # Common protocol elements for distributed Ruby, used by both the
   # client and server.
@@ -132,6 +170,11 @@ module DRb
     # The safe level for this connection.
     attr_accessor :safe_level
 
+    ##
+    # The peer address for this connection.
+    attr_accessor :peer_addr
+
+
     ##
     # The post initialization process sets up the default load
     # and argument length limits, the idconv object, the initial
@@ -145,7 +188,7 @@ module DRb
       @state = :ref
       @msgbuffer = ""
       @request = {}
-      @server = @argv = @argc = nil
+      @peer_addr = @server = @argv = @argc = nil
     end
 
     private
@@ -188,7 +231,6 @@ module DRb
             end
           end
         rescue
-          p $!
           EventMachine::next_tick do
             df.set_deferred_status(:failed, $!)
           end
@@ -265,6 +307,14 @@ module DRb
     #
     def perform
       @server.check_insecure_method(@request[:ro], @request[:msg])
+      if @request[:ro].kind_of?(DRbEMSafe) &&
+          @request[:ro].class.deferrable_method?(@request[:msg])
+        # A deferrable method will return an actual Deferrable that we
+        # can use instead.
+        return(@request[:ro].__send__(@request[:msg],
+                                      @request[:argv],
+                                      @request[:block]))
+      end
       return((@request[:block]) ? perform_with_block : perform_without_block)
     end
 
@@ -346,6 +396,210 @@ module DRb
 
   end
 
+  ##
+  # The default DRb transport, which performs communication over a TCP
+  # socket.  These protocol objects are used in a different way from the
+  # way they are used internally by the standard DRb.  In the standard
+  # DRb, open_server is a class method.  Here, it is an instance method
+  # that is called when the server object is started.
+  class DRbTCPSocket
+    private
+
+    def self.host_inaddr_any(host)
+      infos = Socket::getaddrinfo(host, nil,
+                                  Socket::AF_UNSPEC,
+                                  Socket::SOCK_STREAM,
+                                  0,
+                                  Socket::AI_PASSIVE)
+      family = infos.collect { |af, *_| af }.uniq
+      case family
+      when ['AF_INET']
+        return("0.0.0.0")
+      when ['AF_INET6']
+        return("::")
+      else
+        raise "Unknown network class"
+      end
+    end
+
+    public
+
+    attr_reader :uri
+    attr_reader :option
+
+    def initialize(uri, config={})
+      @uri = (uri.nil?) ? 'druby://:0' : uri
+      @host, @port, @option = self.class.parse_uri(@uri)
+      @host.untaint
+      @port.untaint
+      @config = config
+      @acl = config[:tcp_acl]
+      if @host.size == 0
+        @host = self.class.getservername
+        @connhost = self.class.host_inaddr_any(@host)
+      end
+    end
+
+    ##
+    # This method starts a TCP server using EventMachine.
+    def start_server(prot)
+      # If start_server is called several times, only the one started
+      # server will be made.
+      if @conndesc
+        return(@conndesc)
+      end
+      @conndesc = EventMachine::start_server(@host, @port, prot) do |conn|
+        # Obtain the address of the peer, and check it against
+        # any ACLs that might have been specified.
+        peeraddr = Socket.unpack_sockaddr_in(conn.get_peername)[1]
+        @config[:reverse_dns] = true
+        if @config[:reverse_dns]
+          a = Socket.gethostbyname(peeraddr)
+          peeraddr = begin
+                       [Socket.gethostbyaddr(a[3], a[2])[0], peeraddr]
+                     rescue
+                       [nil, peeraddr]
+                     end
+        else
+          peeraddr = [nil, peeraddr]
+        end
+        conn.peer_addr = [nil, nil] + peeraddr
+
+        if block_given?
+          yield conn
+        end
+
+        if @acl && !@acl.allow_addr?(conn.peer_addr)
+          conn.close_connection
+        end
+
+      end
+      # NOTE: This is an undocumented method in EventMachine.  Revise
+      # as necessary when we receive feedback from the EventMachine
+      # developers on the canonical way to determine the real port number
+      # if port 0 was specified in start_server.
+      addr = Socket.unpack_sockaddr_in(EventMachine.get_sockname(@conndesc))
+      @port = addr[0] if @port == 0
+      @uri = "druby://#{@host}:#{@port}"
+      return(@conndesc)
+    end
+
+    ##
+    # This method makes a client connection using EventMachine
+    def client_connect(prot)
+      EventMachine.connect(@host, @port, prot) do |c|
+        if block_given?
+          yield c
+        end
+      end
+    end
+
+
+    ##
+    # This method closes a server that has been started using this
+    # protocol object instance.
+    def stop_server
+      if @conndesc
+        @conndesc = nil
+        if EventMachine::reactor_running?
+          EventMachine::stop_server(@conndesc)
+        end
+      end
+    end
+  end
+
+  ##
+  # Module managing the underlying network transport(s) used by EMDRb.
+  # This is analogous to the DRbProtocol module used by the original
+  # DRb.  The network transport classes must define the following
+  # class method:
+  #
+  #   [uri_option(uri, config)] Take a URI, possibly containing an option
+  #                             component (e.g. a trailing '?param=val'), 
+  #                             and return a [uri, option] tuple.
+  #
+  # A network transport class is instantiated whenever an EMDRb client
+  # or server starts operation, provided with the URI and any configuration
+  # options.  The transport initialize method should raise a DRbBadScheme
+  # exception if the URI is invalid.  This is how the DRbTransport module
+  # determines which transport implementation serves a particular URI.
+  #
+  # The transport class instance must provide the following methods:
+  #
+  #   [start_server(prot)] Open a server listening at the URI specified when
+  #                        the transport instance was created using the
+  #                        protocol handler module or class +prot+.  A block
+  #                        passed here will be called whenever a connection is
+  #                        made to the server, just after the post_init method
+  #                        of the handler is called.
+  #   [client_connect(prot)] Open a client connection to the URI specified when
+  #                          the when the transport instance was created, using
+  #                          the protocol handler module or class +prot+.  A
+  #                          block passed here will be called when the
+  #                          connection has been initiated, just after the
+  #                          post_init method of the handler is called.
+  #   [stop_server] Stop a server that was started with start_server.
+  # 
+  module DRbTransport
+    @transports = [DRbTCPSocket]
+
+    def add_transport(tr)
+      @transports.push(tr)
+    end
+    module_function :add_transport
+
+    ##
+    # Rather than open_server, EMDRb uses this protocol factory method
+    # to create a protocol object instance that DRbServer and DRbClient
+    # can use later on when they want to connect.
+    def factory(uri, config, first=true)
+      @transports.each do |t|
+        begin
+          return(t.new(uri, config))
+	rescue DRbBadScheme
+	end
+      end
+      if first && (config[:auto_load] != false)
+	auto_load(uri, config)
+	return factory(uri, config, false)
+      end
+      raise DRbBadURI, 'can\'t parse uri:' + uri
+    end
+    module_function :factory
+
+    ##
+    # Parse +uri+ into a [uri, option] pair.
+    #
+    # The DRbTransport module asks each registered transport class in turn to
+    # try to parse the URI.  Each transport signals that it does not handle
+    # that URI by raising a DRbBadScheme error.  If no transport recognizes
+    # the URI, then a DRbBadURI error is raised.  
+    def uri_option(uri, config, first=true)
+      @transports.each do |t|
+	begin
+	  uri, opt = t.uri_option(uri, config)
+	  # opt = nil if opt == ''
+	  return uri, opt
+	rescue DRbBadScheme
+	end
+      end
+      if first && (config[:auto_load] != false)
+	auto_load(uri, config)
+        return uri_option(uri, config, false)
+      end
+      raise DRbBadURI, 'can\'t parse uri:' + uri
+    end
+    module_function :uri_option
+
+    def auto_load(uri, config)  # :nodoc:
+      if uri =~ /^drb([a-z0-9]+):/
+	require("emdrb/#{$1}") rescue nil
+      end
+    end
+    module_function :auto_load
+
+  end
+
   ##
   # Class representing a drb server instance.  This subclasses DRb::DRbServer
   # for brevity.  DRbServer instances are normally created indirectly using
@@ -362,8 +616,8 @@ module DRb
 	}
       end
 
-      @uri = (uri.nil?) ? "druby://:0" : uri
       @config = self.class.make_config(config)
+      @protocol = DRbTransport.factory(uri, config)
       @front = front
       @idconv = @config[:idconv]
       @safe_level = @config[:safe_level]
@@ -372,43 +626,12 @@ module DRb
       end
     end
 
-    private
-
-    def self.host_inaddr_any
-      host = Socket::gethostname
-      begin
-        host = Socket::gethostbyname(host)[0]
-      rescue
-        host = "localhost"
-      end
-      infos = Socket::getaddrinfo(host, nil,
-                                  Socket::AF_UNSPEC,
-                                  Socket::SOCK_STREAM,
-                                  0,
-                                  Socket::AI_PASSIVE)
-      family = infos.collect { |af, *_| af }.uniq
-      case family
-      when ['AF_INET']
-        return("0.0.0.0")
-      when ['AF_INET6']
-        return("::")
-      else
-        raise "Unknown network class"
-      end
-    end
-
-    public
-
     ##
     # Start a DRb server from within an event loop.
     #
     def start_drb_server
       @thread = Thread.current
-      host, port, opt = DRb::parse_uri_drb(@uri)
-      if host.size == 0
-        host = self.class.host_inaddr_any
-      end
-      r = EventMachine::start_server(host, port, DRbServerProtocol) do |conn|
+      @protocol.start_server(DRbServerProtocol) do |conn|
         Thread.current['DRb'] = { 'client' => conn, 'server' => self }
         conn.front = @front
         conn.load_limit = @config[:load_limit]
@@ -417,31 +640,22 @@ module DRb
         conn.server = self
         conn.safe_level = self.safe_level
       end
-      # NOTE: This is an undocumented method in EventMachine.  Revise
-      # as necessary when we receive feedback from the EventMachine
-      # developers on the canonical way to determine the real port number
-      # if port 0 was specified in start_server.
-      addr = Socket.unpack_sockaddr_in(EventMachine.get_sockname(r))
-      port = addr[0] if port == 0
-      @uri = "druby://#{host}:#{port}"
+      @uri = @protocol.uri
     end
 
-  end
-
-  def parse_uri_drb(uri)
-    if uri =~ /^druby:\/\/(.*?):(\d+)(\?(.*))?$/
-      host = $1
-      port = $2.to_i
-      option = $4
-      [host, port, option]
-    else
-      unless uri =~ /^druby:/
-        raise DRb::DRbBadScheme.new(uri)
+    ##
+    # Stop this server.
+    def stop_service
+      DRb.remove_server(self)
+      if Thread.current['DRb'] && Thread.current['DRb']['server'] == self
+        Thread.current['DRb']['stop_service'] = true
+      else
+        @thread.kill
       end
-      raise DRb::DRbBadURI.new('can\'t parse uri:' + uri)
+      @protocol.stop_server
     end
+
   end
-  module_function :parse_uri_drb
 
   @eventloop = nil
 
@@ -451,29 +665,41 @@ module DRb
   def start_drbserver(uri=nil, front=nil, config=nil)
     serv = DRbServer.new(uri, front, config)
     serv.start_drb_server
+    DRb.regist_server(serv)
     return(serv)
   end
   module_function :start_drbserver
 
-  ##
-  # This start_service emulates DRb#start_service.
-  #
-  def start_service(uri=nil, front=nil, config=nil)
+  def start_evloop
     unless EventMachine::reactor_running?
       @eventloop = Thread.new do
-        EventMachine::run do
-          # Start an empty event loop.  The DRb server(s) will be started
-          # by EM#next_tick calls.
+        begin
+          EventMachine::run do
+            # Start an empty event loop.  The DRb server(s) will be started
+            # by EM#next_tick calls.
+          end
+        ensure
+          # close all servers if the event loop ends for whatever reason
+          @server.each do |uri,srv|
+            srv.stop_service
+          end
         end
       end
     end
+  end
+  module_function :start_evloop
+
+  ##
+  # This start_service emulates DRb#start_service.
+  #
+  def start_service(uri=nil, front=nil, config=nil)
+    DRb.start_evloop
     queue = Queue.new
     EventMachine::next_tick do
       queue << self.start_drbserver(uri, front, config)
     end
     serv = queue.shift
     @primary_server = serv
-    DRb.regist_server(serv)
     return(serv)
   end
   module_function :start_service
@@ -544,9 +770,8 @@ module DRb
       @ref = nil
       if obj.nil?
 	return if uri.nil?
-        @uri = uri
-        ref = nil
-        @host, @port, @opt = DRb::parse_uri_drb(@uri)
+        @uri, option = DRbTransport.uri_option(uri, DRb.config)
+        @ref = DRbURIOption.new(option) unless option.nil?
       else
 	@uri = uri ? uri : (DRb.uri rescue nil)
         @ref = obj ? DRb.to_id(obj) : nil
@@ -559,10 +784,9 @@ module DRb
     # callbacks can be attached.
     def send_async(msg_id, *a, &b)
       df = EventMachine::DefaultDeferrable.new
-      if @host.nil? || @port.nil?
-        @host, @port, @opt = DRb::parse_uri_drb(@uri)
-      end
-      EventMachine.connect(@host, @port, DRbClientProtocol) do |c|
+      @protocol ||= DRbTransport.factory(@uri, DRb.config)
+
+      @protocol.client_connect(DRbClientProtocol) do |c|
         c.ref = self
         c.msg_id = msg_id
         c.args = a
@@ -575,7 +799,7 @@ module DRb
     ##
     # Route method calls to the referenced object.  This synchronizes
     # an asynchronous call by using a Queue to synchronize the DRb
-    # event thread with the calling thread, so use of this mechanism,
+    # event thread with the calling thread, so use of this mechanism
     # to make method calls within an event loop will thus result in a
     # threading deadlock!  Use the send_async method if you want to
     # use EMDRb from within an event loop.
@@ -604,5 +828,4 @@ module DRb
     end
 
   end
-
 end

data/lib/emdrb/unix.rb

@@ -0,0 +1,93 @@
+#
+# Author:: Rafael R. Sevilla (mailto:dido@imperium.ph)
+# Copyright:: Copyright © 2008, 2009 Rafael R. Sevilla
+# Homepage:: http://emdrb.rubyforge.org/
+# License:: GNU General Public License / Ruby License
+#
+# $Id: emdrb.rb 72 2009-01-28 09:53:08Z dido $
+#
+#----------------------------------------------------------------------------
+#
+# Copyright © 2008, 2009 Rafael Sevilla
+# This file is part of EMDRb
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of either: 1) the GNU General Public License
+# as published by the Free Software Foundation; either version 2 of the
+# License, or (at your option) any later version; or 2) Ruby's License.
+# 
+# See the file COPYING for complete licensing information.
+#----------------------------------------------------------------------------
+#
+require 'drb/unix'
+
+module DRb
+
+  ##
+  # Unix domain socket transport
+  class DRbUNIXSocket < DRbTCPSocket
+    public
+
+    def initialize(uri, config={})
+      @uri = (uri.nil?) ? 'drbunix:' : uri
+      @filename, @option = self.class.parse_uri(@uri)
+      @filename.untaint
+      @port.untaint
+      @config = config
+      @acl = nil
+    end
+
+    def start_server(prot)
+      if @conndesc
+        return(@conndesc)
+      end
+
+      lock = nil
+      if @filename.nil?
+        tmpdir = Dir::tmpdir
+        n = 0
+        while true
+          begin
+            tmpname = sprintf('%s/druby%d.%d', tmpdir, $$, n)
+            lock = tmpname + '.lock'
+            unless File.exist?(tmpname) or File.exist?(lock)
+              Dir.mkdir(lock)
+              @filename = tmpname
+              break
+            end
+          rescue
+            raise "cannot generate tempfile `%s'" % tmpname if n >= Max_try
+            #sleep(1)
+          end
+          n += 1
+        end
+      end
+
+      @conndesc = EventMachine::start_unix_domain_server(@filename,
+                                                         prot) do |conn|
+        if block_given?
+          yield conn
+        end
+      end
+
+      if lock
+        Dir.rmdir(lock)
+      end
+    end
+
+    def client_connect(prot)
+      EventMachine.connect_unix_domain(@filename, prot) do |c|
+        if block_given?
+          yield c
+        end
+      end
+    end
+
+    def stop_server
+      super
+      File.unlink(@filename)
+    end
+  end
+
+  DRbTransport.add_transport(DRbUNIXSocket)
+end

data/lib/emdrb/version.rb

@@ -4,7 +4,7 @@
 # Homepage:: http://emdrb.rubyforge.org/
 # License:: GNU Lesser General Public License / Ruby License
 #
-# $Id: version.rb 57 2009-01-23 09:07:54Z dido $
+# $Id: version.rb 68 2009-01-27 07:26:59Z dido $
 #
 #----------------------------------------------------------------------------
 #
@@ -25,7 +25,7 @@ module EMDRb
   module Version
 
     MAJOR = 0
-    MINOR = 2
+    MINOR = 3
     TINY = 0
 
     # The version of EMDRb in use.

data/spec/drbserver.rb

@@ -4,7 +4,7 @@
 # Homepage:: http://emdrb.rubyforge.org/
 # License:: GNU General Public License / Ruby License
 #
-# $Id: drbserver.rb 46 2009-01-23 08:10:42Z dido $
+# $Id: drbserver.rb 66 2009-01-27 07:25:23Z dido $
 #
 #----------------------------------------------------------------------------
 #
@@ -72,6 +72,40 @@ class TestServer
   end
 end
 
+if ARGV[0] == "emdrb"
+  class TestServer
+    include DRb::DRbEMSafe
+    include EventMachine::Deferrable
+
+    deferrable_method :block_df
+
+    ##
+    # Simple example of a deferrable method structured as a state
+    # machine.
+    def block_df(args, block, state={:index => 0, :retval => 0 })
+      if state[:index] >= args.length
+        self.set_deferred_status(:succeeded, state[:retval])
+        return(self)
+      end
+
+      df = block.send_async(:call, args[state[:index]])
+      df.callback do |succ,result|
+        if succ
+          state[:retval] += result
+          state[:index] += 1
+          EventMachine::next_tick do
+            self.block_df(args, block, state)
+          end
+        else
+          self.set_deferred_status(:failed,  res)
+        end
+      end
+      df.errback do |res|
+      end
+      return(self)
+    end
+  end
+end
 
 
 DRb.start_service("druby://:12345", TestServer.new)

data/spec/server_spec.rb

@@ -4,7 +4,7 @@
 # Homepage:: http://emdrb.rubyforge.org/
 # License:: GNU General Public License / Ruby License
 #
-# $Id: server_spec.rb 48 2009-01-23 08:48:43Z dido $
+# $Id: server_spec.rb 65 2009-01-27 07:24:53Z dido $
 #
 #----------------------------------------------------------------------------
 #
@@ -48,4 +48,9 @@ describe "EMDRb Server" do
     @obj.sum(1,2,3,4,5).should == 15
   end
 
+  it "should use deferrable methods correctly" do
+    res = @obj.block_df(1,2,3,4,5) { |x| x }
+    res.should == 15
+  end
+
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: emdrb
 version: !ruby/object:Gem::Version 
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors: 
 - dido@imperium.ph
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-01-23 00:00:00 +08:00
+date: 2009-01-29 00:00:00 +08:00
 default_executable: 
 dependencies: []
 
@@ -31,6 +31,7 @@ files:
 - Rakefile
 - lib/emdrb.rb
 - lib/emdrb/emdrb.rb
+- lib/emdrb/unix.rb
 - lib/emdrb/version.rb
 - spec/client_spec.rb
 - spec/drbserver.rb