virtual_module 0.1.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 4bfab3d164f02ef899d40766d55420467ece2e23
-  data.tar.gz: 8f125f80cc967b5e40fa59879a53dcb97bbca308
+  metadata.gz: bd5f0e87cb8b078374f942887d00b5d32e3f2245
+  data.tar.gz: f350813eb2781d0158ac0d1a931c8a9bd0643ccf
 SHA512:
-  metadata.gz: ec550f409511efd94ecaf381a5f5e1e73ba6c1000a0d663f142dc7582d487db8ee2cb04c51c7b709c81dd3b3416f5a1a8c1cd206a740fad4f9787e5d0b80c364
-  data.tar.gz: 40a0f820f59e823896b9b20b4b1c4c85f1f4c05902b2692e37c2c6821004831298834a4d1712b209cff6afba8c73da08f174d38ba5fbad18fd83f4230c9b8532
+  metadata.gz: a05517f26f04375b0241986354bd9008960f1d57dc3146580165b17d608a4d4e47056605993b40d94f856d07729c7a4e9547fc1fce9a21214c578fa53fa6fe69
+  data.tar.gz: 7bdf2a597339e02a8fc40a919b79edbddd4de11aabe7a997979e07cb4e280f69632649b82ef1cd5009869a7cd252009683dbafb264a027d7abb01604bf8f86a5

data/README.md

@@ -1,28 +1,111 @@
-## VirtualModule
+# VirtualModule
 
-If you feel good with Ruby's language design but *totally* disappointed with Ruby's computational performance such as terribly slow speed of loop or Math functions, then VirtualModule may save you a little.
+If you have ever felt at all disappointed seeing Ruby's computational performance particularly for large-scale scientific computation, VirtualModule may save you a little. It offers you a way to run your arbitrary Ruby code on the other language VM process (currently only Julia is supported). What make VirtualModule possible to do this is [ruby2julia transpiler](https://github.com/remore/julializer) (which is at very early stage of development) and IPC using [msgpack](http://msgpack.org/).
 
-With VirtualModule, you can run your Ruby code on the other language VM process(currently only Julia is supported).
+Let's take a look at results of [benchmark program](https://github.com/remore/virtual_module/blob/master/doc/benchmark_loop_performance/add_two_floating_point_numbers_cumulatively.rb) of which scores were taken on my MacBook Pro (OSX10.11, Core i5 2.9GHz, 8GB). Essentially this benchmarking program just repeats adding an integer value to floating point value cumulatively, which is represented as `x=2.5; 1.upto(N){|i| x=x+i}` in Ruby. If you run this program with setting of smaller number of loops (upto 10,000,000 times), VirtualModule doesn't make sense at all as you can see:
+
+![Benchmarking results](https://raw.githubusercontent.com/remore/virtual_module/master/doc/assets/benchmark-result-from-1e1-to-1e7.png "Graphs of average execution time vs number of loops(MacBook Pro)")
+
+However as the number of loops gets bigger(upto 1,000,000,000 times), VirtualModule significantly reduces total execution time.
+
+![Benchmarking results](https://raw.githubusercontent.com/remore/virtual_module/master/doc/assets/benchmark-result-from-1e1-to-1e9.png "Graphs of average execution time vs number of loops(MacBook Pro)")
+
+(For the sake of honor of python and cython, here is [another benchmarking results](https://raw.githubusercontent.com/remore/virtual_module/master/doc/assets/benchmark-result-from-1e1-to-1e9-with-ubuntu.png) measured on Ubuntu 16 on DigitalOcean (8CPUs, 16GB Mem). It records better score for both python2.7 and Cython than the one with my MacBook Pro, although I'm not sure why this happens. Does anyone can guess why this happens?)
+
+#### An experiment with word2vec
+
+The other example here is a [prototype word2vec implementation in Ruby](https://github.com/remore/virtual_module/blob/master/example/word2vec.rb), which is just partially ported (CBOW model only) from original C implementation ([the one created by Tomas Mikolov at Google](https://code.google.com/archive/p/word2vec/)). Since it's not optimized very well yet due to my limited time to work on this example, it doesn't record decent score in terms of speed yet, but it reveals at least the fact that VirtualModule will reduces total execution time considerably even in real world example.
+
+![Benchmarking results](https://raw.githubusercontent.com/remore/virtual_module/master/doc/assets/benchmark-result-of-word2vec-performance.png "Graphs of average execution time vs filesize of training data(Ubuntu16)")
+
+(A few record values resulted in 0 because it took too much time, actually more than three days to complete due to exponential explosion happened.)
+
+Of course the vector binary file format generated through VirtualModule is compatible with original C implementation. Here is [sample generated vector files](https://github.com/remore/remore.github.com/tree/master/virtual_module/sample_output) saved onto GitHUb, and command logs are as shown below:
 
 ```
+$ cd example
+$ ruby word2vec.rb --output /tmp/vectors.bin --train ../doc/benchmark_word2vec/training_data/10mb.txt --size 20 --window 10 --negative 5 --sample 1e-4 --binary 1 --iter 3 --debug 0 > /dev/null 2>&1
+$ python
+Python 2.7.12 (default, Jul  1 2016, 15:12:24)
+[GCC 5.4.0 20160609] on linux2
+Type "help", "copyright", "credits" or "license" for more information.
+>>> import gensim
+>>> model = gensim.models.Word2Vec.load_word2vec_format('/tmp/vectors.bin', binary=True)
+>>> model.most_similar("japan")
+[(u'netherlands', 0.9741939902305603), (u'china', 0.9712631702423096), (u'county', 0.9686408042907715), (u'spaniards', 0.9669440388679504), (u'vienna', 0.9614173769950867), (u'abu', 0.9587018489837646), (u'korea', 0.9565504789352417), (u'canberra', 0.954473614692688), (u'erupts', 0.9540712833404541), (u'prefecture', 0.9534248113632202)]
+```
+
+## Usage and How It Works
+
+To write Ruby code using VirtualModule is something like to write Python code using Cython. Basic concept is to separate large-scale computation algorithm, and isolate them as a module accessible from the base program. Like Cython, VirtualModule is NOT comprehensive approach too but gives you an opportunity to reduce execution time in exchange for [the hard limitation of Ruby syntax due to ruby2julia transipiler](https://github.com/remore/julializer#supported-classes-and-syntax).
+
+```
+require 'virtual_module'
 vm = VirtualModule.new(<<EOS)
 def hi
-  "ho"
+  "yo"
 end
 
-# @param [Fixnum] num
-def hello(num)
-  num*num
+def init_table(list)
+  for i in 0..list.size-1
+    list[i]+=Random.rand
+  end
+  list
 end
 EOS
-p vm.hi #"ho"
+p vm.hi # "yo"
 include vm
-p hello(33) #1089
+p init_table([1,20]) # [1.3066601775641218, 20.17001189249985]
 ```
 
-## Other explanation
+As this sample snippet shows, VirtualModule is a Module generator which has only one public api named `VirtualModule#new`. Since `VirtualModule#new` returns a instance of Module class, user can simply include `vm` object that's why `#init_table` method is called in the context of `self`.
+
+In detail, when you run VirtualModule, following processing are happening internally. It's just a typical procedures of an RPC call.
 
-TBC
+ 1. VirtualModule calls `Julializer#ruby2julia` to transpile `#hi` and `#init_table` methods into raw Julia code
+ 2. Prepare glue code to send and receive parameters from/to Julia and add this to (1)
+ 3. Run (2) through Julia VM process and pass the parameters using msgpack-ruby and MsgPack.jl
+ 4. Receive returned value passed from Julia
+
+What makes VirtualModule different from normal RPC call is `#virtual_module_eval` method. It offers a convenient way to eval arbitary code as if it's executed on the context of `self`. Most important note here is by this way, you can even pass type information about arguments like below.
+
+```
+# An example to pass Array{Float64,1} type array to Julia
+class FloatArray < Array
+end
+float_list = FloatArray.new([1.0234, 9.9999])
+vm.virtual_module_eval("float_list = init_table(float_list)")
+puts float_list # [1.5765191145261321, 10.270808112990153] is calculated as Array{Float64,1} in Julia, which means much faster computation than Array{Any,1}!
+```
+
+## Requirement
+
+- Mac OSX, Linux and Ruby (MRI) 2.1 and higher are supported.
+- Either Julia or Docker installed on your machine is a must, although installing Julia is highly recommended for the performance reason.
+
+## Installation
+
+Simply run the following command.
+
+```
+$ gem install virtual_module
+$ julia -e 'Pkg.add("MsgPack"); Pkg.add("MsgPackRpcServer")'
+```
+
+## Features
+
+Following features are implemented already:
+
+- File-based IPC using [msgpack-ruby](https://github.com/msgpack/msgpack-ruby) and [MsgPack.jl](https://github.com/kmsquire/MsgPack.jl/)
+- RPC-based IPC using [msgpack-ruby](https://github.com/msgpack/msgpack-ruby) and [MsgPackRpcServer.jl](https://github.com/remore/MsgPackRpcServer.jl)
+
+features to be implemented in the future (if needed/requested) are listed as follows:
+
+- A bug fix for `#virtual_module_eval`: return value has a bug whenever argument has more than two lines.
+- Improve julializer (transpiler) much better
+  - word2vec implementation is another interesting option to improve
+- Other programming language support such as golang, ruby itself etc.
 
 ## License
+
 MIT

data/lib/virtual_module/bridge.jl

@@ -1,3 +1,6 @@
+import Base.+
+import Base.*
+
 function +(x::ASCIIString, y::ASCIIString)
   string(x, y)
 end

data/lib/virtual_module/version.rb

@@ -1,3 +1,3 @@
 module VirtualModule
-  VERSION = "0.1.0"
+  VERSION = "0.2.1"
 end

data/lib/virtual_module.rb

@@ -10,41 +10,58 @@ module VirtualModule
   require 'virtual_module/version'
 
   class << self
-    def new(methods, config={})
-      if !config.is_a?(BaseBuilder)
-        config[:lang] = :julia if config[:lang].nil?
-        vm_builder = instance_eval("#{config[:lang].to_s.capitalize}Builder").new(config)
-      else
-        vm_builder = config
-      end
+    def new(methods, **args)
+      option = {:lang=>:julia, :pkgs=>[], :ipc=>:file}.merge(args)
+      vm_builder = Builder.new(option[:lang], option[:pkgs], option[:ipc])
       vm_builder.add(methods)
       vm_builder.build
     end
   end
 
-  class BaseBuilder
-    def initialize(config={})
-      @source = []
-      if !config.is_a?(BaseIpcInterface)
-        props = {:ipc=>:file, :work_dir=>nil}
-        props.each do |k,v|
-          instance_variable_set("@#{k}", config[k] || v)
+  module SexpParser
+    def extract_defs(s)
+      if s.instance_of?(Array) && s[0].instance_of?(Symbol) then
+        if [:def].include?(s[0])
+          "#{s[1][1]},"
+        else
+          s.map{|e| extract_defs(e)}.join
         end
-        @ipc = instance_eval("#{@ipc.to_s.capitalize}IpcInterface").new(props.merge(config))
-      else
-        @ipc = config
+      elsif s.instance_of?(Array) && s[0].instance_of?(Array) then
+        s.map{|e| extract_defs(e)}.join
+      end
+    end
+
+    def extract_args(s)
+      if s.instance_of?(Array) && s[0].instance_of?(Symbol) then
+        if [:vcall, :var_field].include?(s[0])
+          "#{s[1][1]},"
+        else
+          s.map{|e| extract_args(e)}.join
+        end
+      elsif s.instance_of?(Array) && s[0].instance_of?(Array) then
+        s.map{|e| extract_args(e)}.join
       end
     end
+  end
+
+  class Builder
+    include SexpParser
 
-    def add(methods)
-      @source << methods
-      @ipc.reset get_compiled_code
+    def initialize(lang=:julia, pkgs=[], ipc=:file)
+      @provider = instance_eval("#{lang.capitalize}SourceProvider").new(pkgs)
+      @ipc = instance_eval("#{ipc.to_s.capitalize}IpcInterface").new(@provider)
+    end
+
+    def add(methods="")
+      @provider.source << methods
+      @provider.compile
+      @ipc.reset @provider
     end
 
     def build
       vm_builder = self
       ipc = @ipc
-      vm = Module.new{
+      @vm = Module.new{
         @vm_builder = vm_builder
         @ipc = ipc
         def self.virtual_module_eval(*args)
@@ -53,56 +70,50 @@ module VirtualModule
         def self.method_missing(key, *args, &block)
           @vm_builder.send(:call, key, *args)
         end
+        def self.___get_serialized___
+          @ipc.serialized
+        end
       }
-      extract_defs(Ripper.sexp(@source.join(";"))).split(",").each{|e|
-        vm.class_eval {
+      extract_defs(Ripper.sexp(@provider.source.join(";"))).split(",").each{|e|
+        @vm.class_eval {
           define_method e.to_sym, Proc.new { |*args|
             vm_builder.call(e.to_sym, *args)
           }
         }
       }
-      vm
+      @vm
     end
 
     def call(name, *args)
-      Dir.mktmpdir(nil, Dir.home) do |tempdir|
-        @work_dir = @ipc.work_dir = tempdir
+      begin
         @ipc.call(name, *args)
+      rescue StandardError => e
+        @ipc.serialized = e.message
+        @vm
       end
     end
 
-    private
-      def extract_defs(s)
-        if s.instance_of?(Array) && s[0].instance_of?(Symbol) then
-          if [:def].include?(s[0])
-            "#{s[1][1]},"
-          else
-            s.map{|e| extract_defs(e)}.join
-          end
-        elsif s.instance_of?(Array) && s[0].instance_of?(Array) then
-          s.map{|e| extract_defs(e)}.join
-        end
+    def virtual_module_eval(script, auto_binding=true)
+      vars, type_info, params = inspect_local_vars(binding.of_caller(2), script)
+      @provider.compile(vars, type_info, params, script, auto_binding)
+      @ipc.reset @provider
+      evaluated = self.call(:___main, params, type_info)
+      if auto_binding
+        binding.of_caller(2).eval(evaluated[1].map{|k,v| "#{k}=#{v};" if !v.nil? }.join)
       end
 
-      def extract_args(s)
-        if s.instance_of?(Array) && s[0].instance_of?(Symbol) then
-          if [:vcall, :var_field].include?(s[0])
-            "#{s[1][1]},"
-          else
-            s.map{|e| extract_args(e)}.join
-          end
-        elsif s.instance_of?(Array) && s[0].instance_of?(Array) then
-          s.map{|e| extract_args(e)}.join
-        end
-      end
+      return evaluated[0]
+    end
 
+    alias_method :virtual_instance_eval, :virtual_module_eval
+    alias_method :virtual_eval, :virtual_module_eval
+
+    private
       def inspect_local_vars(context, script)
         vars = extract_args(Ripper.sexp(script)).split(",").uniq.map{|e| e.to_sym} & context.eval("local_variables")
-        #p "vars=#{vars}"
 
         type_info = {}
         type_info[:params] = context.eval("Hash[ *#{vars}.collect { |e| [ e, eval(e.to_s).class.to_s ] }.flatten ]").select{|k,v| ["FloatArray", "IntArray"].include?(v)}
-        #p "type_info=#{type_info}"
 
         params = context.eval(<<EOS)
 require 'msgpack'
@@ -110,16 +121,67 @@ ___params = {}
 ___params = local_variables.map{|e| [e, eval(e.to_s)]}.each{|e| ___params[e[0]]=e[1] if #{vars}.include?(e[0])}[0][1]
 ___params
 EOS
-        #File.write("#{@work_dir}.polykit.params", params)
         [vars, type_info, params]
       end
   end
 
+  class BaseSourceProvider
+    attr_accessor :source
+    def initialize(pkgs)
+      @source = load_packages(pkgs)
+      @compiled_lib = ""
+    end
 
-  class JuliaBuilder < BaseBuilder
-    def virtual_module_eval(script, auto_binding=true)
-      vars, type_info, params = inspect_local_vars(binding.of_caller(2), script)
-      boot_script = <<EOS
+    def load_packages(pkgs)
+      [] #to be overrieded
+    end
+  end
+
+  class JuliaSourceProvider < BaseSourceProvider
+    def load_packages(pkgs)
+      pkgs.map{|e| "import #{e}"}
+    end
+
+    def main_loop(input_queue_path, output_queue_path)
+      <<EOS
+using MsgPack
+while true
+  source = open( "#{input_queue_path}", "r" ) do fp
+    readall(fp)
+  end
+  result = eval(parse(source))
+  open( "#{output_queue_path}", "w" ) do fp
+    try
+      write(fp,pack(result))
+    catch
+      serialize(fp,result)
+    end
+  end
+end
+EOS
+    end
+
+    def lib_script(ipc=nil)
+      if ipc!=:rpc
+        @compiled_lib
+      else
+        <<EOS
+import MsgPackRpcServer
+module RemoteFunctions
+#{@compiled_lib}
+end
+MsgPackRpcServer.run(parse(ARGS[1]), RemoteFunctions)
+EOS
+      end
+    end
+
+    def compile(vars=nil, type_info=nil, params=nil, script=nil, auto_binding=nil)
+      @compiled_lib = File.read(
+        File.dirname(__FILE__)+"/virtual_module/bridge.jl") + ";" +
+        Julializer.ruby2julia(@source.join(";\n")
+      )
+      if !vars.nil? && !type_info.nil? && !params.nil? && !script.nil? && !auto_binding.nil?
+        @compiled_lib += <<EOS
   function ___convert_type(name, typename, params)
     if length(findin(#{type_info[:params].keys.map{|e| e.to_s}},[name]))>0
       if typename=="FloatArray"
@@ -135,125 +197,124 @@ EOS
   function ___main(params, type_info)
     #{vars.map{|e| e.to_s + '=___convert_type("'+e.to_s+'","'+(type_info[:params][e]||"")+'", params)'}.join(";")}
     ##{vars.map{|e| 'println("'+e.to_s+'=", typeof('+e.to_s+'))' }.join(";")}
-    ___evaluated = #{Julializer.ruby2julia(script)}
+    ___evaluated = (#{Julializer.ruby2julia(script)})
 
     #{vars.map{|e| 'params["'+e.to_s+'"]='+e.to_s }.join(";") if auto_binding}
 
     (___evaluated,#{auto_binding ? "params" : "-1" })
   end
 EOS
-
-#      File.write("#{@work_dir}.polykit.rb", <<EOS)
-#  require 'msgpack-rpc'
-#  client = MessagePack::RPC::Client.new('127.0.0.1', #{@port})
-#  client.timeout = #{@timeout}
-#  p client.call(:___main, #{params}, #{type_info})
-#EOS
-
-      @ipc.reset get_compiled_code + ";#{boot_script}"
-      evaluated = self.call(:___main, params, type_info)
-      if auto_binding
-        binding.of_caller(2).eval(evaluated[1].map{|k,v| "#{k}=#{v};" if !v.nil? }.join)
       end
-
-      return evaluated[0]
     end
 
-    alias_method :virtual_instance_eval, :virtual_module_eval
-    alias_method :virtual_eval, :virtual_module_eval
-
-    private
-      def get_compiled_code
-        File.read(File.dirname(__FILE__)+"/virtual_module/bridge.jl") + ";" + Julializer.ruby2julia(@source.join(";\n"))
+    def generate_message(input_queue_path, name, *args)
+      script = ""
+      params = []
+      args.each_with_index do |arg, i|
+        type = arg.class == Module ? "serialized" : "msgpack"
+        File.write("#{input_queue_path}.#{i}.#{type}", arg.class == Module ? arg.___get_serialized___ : MessagePack.pack(arg))
+        params << "params_#{i}"
+        val = arg.class == Module ? "deserialize(fp)" : "unpack(readall(fp))"
+        script += "#{params.last} =open( \"#{input_queue_path}.#{i}.#{type}\", \"r\" ) do fp; #{val}; end;"
       end
+      script += "#{name}(#{params.join(',')});"
+    end
 
   end
 
   class BaseIpcInterface
-    INPUT_ARGS = "virtualmodule-input.msgpack"
-    OUTPUT_ARGS = "virtualmodule-output.msgpack"
-    ENTRYPOINT_SCRIPT = "virtualmodule-entrypoint.jl"
-    LIB_SCRIPT = "virtualmodule-lib.jl"
+    LIB_SCRIPT = "vm-lib"
 
     attr_accessor :work_dir
+    attr_accessor :serialized
 
-    def initialize(config)
-      #do nothing
+    def initialize(provider)
+      @provider = provider
+      @work_dir = Dir.mktmpdir(nil, Dir.home)
     end
     def call(name, *args)
       #do nothing
     end
-    def reset(source)
-      #do nothing
+    def reset(provider)
+      @provider = provider
     end
   end
 
   class FileIpcInterface < BaseIpcInterface
+    INPUT = "vm-input"
+    OUTPUT = "vm-output"
+    MAIN_LOOP = "vm-main"
+
+    def initialize(provider)
+      super
+      File.mkfifo("#{@work_dir}/#{INPUT}")
+      File.mkfifo("#{@work_dir}/#{OUTPUT}")
+      at_exit do
+        Process.kill(:INT, @pid) if !@pid.nil?
+        FileUtils.remove_entry @work_dir if File.directory?(@work_dir)
+      end
+    end
+
     def call(name, *args)
-      File.write("#{@work_dir}/#{LIB_SCRIPT}", @lib_source)
-      File.write("#{@work_dir}/#{INPUT_ARGS}", MessagePack.pack(args))
-
-      if is_installed?(:julia)
-        File.write("#{@work_dir}/#{ENTRYPOINT_SCRIPT}", generate_entrypoint(@work_dir, "#{@work_dir}/#{OUTPUT_ARGS}", name, *args))
-        command = "julia --depwarn=no -L #{@work_dir}/#{LIB_SCRIPT} #{@work_dir}/#{ENTRYPOINT_SCRIPT}"
-      elsif is_installed?(:docker)
-        File.write("#{@work_dir}/#{ENTRYPOINT_SCRIPT}", generate_entrypoint("/opt/", "/opt/#{OUTPUT_ARGS}", name, *args))
-        command = "docker run -v #{@work_dir}/:/opt/ remore/virtual_module julia --depwarn=no -L /opt/virtualmodule-lib.jl /opt/virtualmodule-entrypoint.jl"
+      #require 'byebug'
+      #byebug
+      if Helper.is_installed?(:julia)
+        enqueue @provider.generate_message("#{@work_dir}/#{INPUT}", name, *args)
+      elsif Helper.is_installed?(:docker)
+        enqueue @provider.generate_message("/opt/#{INPUT}", name, *args)
       else
         raise Exception.new("Either julia or docker command is required to run virtual_module")
       end
-      out, err, status = Open3.capture3(command)
       #byebug
-      # FIXME: Only For Debug
-      # printf out, err
-      MessagePack.unpack(File.read("#{@work_dir}/#{OUTPUT_ARGS}"))
+      response = dequeue
+      begin
+        MessagePack.unpack(response)
+      rescue
+        raise StandardError.new(response)
+      end
     end
 
     def reset(source)
-      @lib_source = source
+      super
+      restart_server_process
     end
 
     private
-      def is_installed?(command)
-        Open3.capture3("which #{command.to_s}")[0].size > 0
-      end
-
-      def generate_entrypoint(basedir, target, name, *args)
-        if args.count>0
-          script =<<EOS
-using MsgPack
-params = open( "#{basedir}/#{INPUT_ARGS}", "r" ) do fp
-  readall( fp )
-end
-result = #{name}(unpack(params)...)
-EOS
+      def restart_server_process
+        Process.kill(:KILL, @pid) if !@pid.nil?
+        File.write("#{@work_dir}/#{LIB_SCRIPT}", @provider.lib_script)
+        File.write("#{@work_dir}/#{MAIN_LOOP}", @provider.main_loop("#{@work_dir}/#{INPUT}", "#{@work_dir}/#{OUTPUT}"))
+        if Helper.is_installed?(:julia)
+          command = "julia --depwarn=no -L #{@work_dir}/#{LIB_SCRIPT} #{@work_dir}/#{MAIN_LOOP}"
+        elsif Helper.is_installed?(:docker)
+          command = "docker run -v #{@work_dir}/:/opt/ remore/virtual_module julia --depwarn=no -L /opt/#{LIB_SCRIPT} /opt/#{MAIN_LOOP}"
         else
-          script =<<EOS
-using MsgPack
-result = #{name}()
-EOS
+          raise Exception.new("Either julia or docker command is required to run virtual_module")
         end
+        @pid = Process.spawn(command, :err => :out,:out => "/dev/null") # , :pgroup => Process.pid)
+        Process.detach @pid
+      end
 
-        script + ";" + <<EOS
-open( "#{target}", "w" ) do fp
-  write( fp, pack(result) )
-end
-EOS
+      def enqueue(message)
+        File.write("#{@work_dir}/#{INPUT}", message)
       end
 
+      def dequeue
+        File.open("#{@work_dir}/#{OUTPUT}", 'r'){|f| f.read}
+      end
   end
 
   class RpcIpcInterface < BaseIpcInterface
-    def initialize(config={})
+    def initialize(provider)
+      super
       init_connection
-      props = {:server=>"127.0.0.1", :port=>8746, :timeout=>10}
-      props.each do |k,v|
-        instance_variable_set("@#{k}", config[k] || v)
-      end
+      @server = "127.0.0.1"
+      @port = 8746
+      @timeout = 10
     end
 
     def call(name, *args)
-      restart_server_process @lib_source
+      restart_server_process
       while `echo exit | telnet #{@server} #{@port} 2>&1`.chomp[-5,5]!="host." do
         sleep(0.05)
       end
@@ -262,36 +323,34 @@ EOS
       args.count>0 ? @client.call(name, *args) : @client.call(name)
     end
 
-    def reset(source)
-      @lib_source = source
-    end
-
     private
       def init_connection
         @pid = nil
         @client.close if !@client.nil?
         @client = nil
+        at_exit do
+          @client.close if !@client.nil?
+          Process.kill(:INT, @pid) if !@pid.nil?
+          FileUtils.remove_entry @work_dir if File.directory?(@work_dir)
+        end
       end
 
-      def restart_server_process(source)
-        Process.kill(:TERM, @pid) if !@pid.nil?
+      def restart_server_process
+        Process.kill(:KILL, @pid) if !@pid.nil?
         `lsof -wni tcp:#{@port} | cut -f 4 -d ' ' | sed -ne '2,$p' | xargs kill -9`
         init_connection
-        compiled = <<EOS
-import MsgPackRpcServer
-module RemoteFunctions
-#{source}
-end
-MsgPackRpcServer.run(#{@port}, RemoteFunctions)
-EOS
-        File.write("#{@work_dir}/#{LIB_SCRIPT}", compiled)
-        @pid = Process.spawn("julia --depwarn=no #{@work_dir}/#{LIB_SCRIPT}")
+        File.write("#{@work_dir}/#{LIB_SCRIPT}", @provider.lib_script(:rpc))
+        @pid = Process.spawn("julia --depwarn=no #{@work_dir}/#{LIB_SCRIPT} #{@port}", :err => :out,:out => "/dev/null") #, :pgroup=>Process.pid)
+        Process.detach @pid
       end
+  end
 
-      at_exit do
-        @client.close if !@client.nil?
-        Process.kill(:TERM, @pid) if !@pid.nil?
+  module Helper
+    class << self
+      def is_installed?(command)
+        Open3.capture3("which #{command.to_s}")[0].size > 0
       end
+    end
   end
 
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: virtual_module
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Kei Sawada(@remore)
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-08-24 00:00:00.000000000 Z
+date: 2016-09-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: msgpack