donkey 0.1.0

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2009 Howard Yeh
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.textile

@@ -0,0 +1,162 @@
+
+Asynchronous Service Stages (ASS) is a way to
+organize distributed services by decoupling the
+what and how of computation from the when and
+where. Built on top of RabbitMQ (an implementation
+of AMQP), ASS helps you to build robust and
+scalable distributed applications.
+
+End of project pimping, let's get started.
+
+h1. Install
+
+You need
+
+(1) Erlang
+(2) RabbitMQ
+(3) AMQP gem
+
+Thread.new { EM.run }
+AMQP.start
+
+^C to exit
+
+
+h1. The Basics
+
+A service component is a ruby script that
+communicates with RabbitMQ. You need to define the
+AMQP server your ASS depends on. Something like this,
+
+
+require 'rubygems'
+require 'ass'
+AMQP.start(:host => 'localhost',
+           #:vhost => "/ass-test",
+           :logging => false) do
+  # ASS definition
+end
+
+
+To start a server
+
+server = ASS.new("echo")
+# => #<ASS::Server echo>
+
+But it doesn't do anything yet. You define the
+behaviour of the server by setting its
+callback. The callback can be a class, so that for
+each client request an object is created from the
+class to process the request. Like so,
+
+
+server.react(SomeReactorClass)
+
+
+However, often you just want something simple. The
+react method can take a block and construct an
+anonymous callback class from which the server
+creates an callback object for each request. Here
+we ask the server to react to @foo@ or @bar@.
+
+
+server.react {
+  def foo(input)
+    [:server,:foo,input]
+  end
+
+  def oof(input)
+    [:server,:oof,input]
+  end
+}
+
+
+The react method accepts for the callback either
+a Class, a Module, a block, or any object. When an
+object is used, it's considered a singleton, which
+is used to process all the requests.
+
+Now that we have a server, we need to get a client
+so to call the server. Because the call is
+asynchronous (the client doesn't wait for the
+result), to process the result when it gets back,
+we need to define callback for the client (just as
+we did for the server). For each call to the
+remote server, the result is processed at the
+client side by a method of the same name,
+
+
+client = server.client.react {
+  def foo(output)
+    p [:client,:foo,output]
+  end
+  def oof(output)
+    p [:client,:oof,output]
+  end
+}
+
+c.call(:foo,42)
+c.call(:oof,24)
+
+# [:client,:foo,[:server,:foo,42]]
+# [:client,:foo,[:server,:foo,24]]
+
+
+> ruby server.rb
+> ruby client.rb
+
+> ruby server.rb
+^C
+
+While the server is down, the requests the client
+is making is queued by the underlying message
+middleware (RabbitMQ), so in some future time when
+we restart the server, we wouldn't lose any
+request. Let's restart the server.
+
+> ruby server.rb
+
+See that the server caught up with all the pending
+requests. To increase service capacity, we can
+just increase the number of server instances.
+
+> ruby server.rb
+
+Now the load is distributed between these two
+instances. We can also start more clients to
+handle more load.
+
+> ruby client.rb
+
+You can see requests coming in from two clients.
+
+
+h1. Service Configuration
+
+-how the name of a service map to AMQP entities.
+-various options for different functional characteristics.
+
+-using routing_key
+-using reply_to
+
+
+rabbitmqctl list_exchanges
+rabbitmqctl list_queues
+
+
+h1. RPC service
+
+The RPC client provides a synchronous API to the
+asynchronous services. This is so the users of an
+ASS application don't have to bother with the
+difficulties of asynchronous programming style
+with callbacks.
+
+The RPC client is intended to be used as the
+gateway into some reliable internal
+services. While the internal services coordinated
+with ASS needs to be robust to component failures,
+there's no such requirements for gateways. It is
+ok for a gateway to fail, and fail in delivering a
+response, as long as the internal services carry
+out the task without a hitch.

data/Rakefile

@@ -0,0 +1,57 @@
+require 'rubygems'
+require 'rake'
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = "donkey"
+    gem.summary = "Asynchronous Service Stages for Distributed Services"
+    gem.email = "hayeah@gmail.com"
+    gem.homepage = "http://github.com/hayeah/donkey"
+    gem.authors = ["Howard Yeh"]
+    gem.add_dependency "amqp"
+    gem.files = FileList["[A-Z]*", "{lib,test}/**/*"]
+    # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end
+
+require 'rake/testtask'
+Rake::TestTask.new(:test) do |test|
+  test.libs << 'lib' << 'test'
+  test.pattern = 'test/**/*_test.rb'
+  test.verbose = true
+end
+
+begin
+  require 'rcov/rcovtask'
+  Rcov::RcovTask.new do |test|
+    test.libs << 'test'
+    test.pattern = 'test/**/*_test.rb'
+    test.verbose = true
+  end
+rescue LoadError
+  task :rcov do
+    abort "RCov is not available. In order to run rcov, you must: sudo gem install spicycode-rcov"
+  end
+end
+
+
+task :default => :test
+
+require 'rake/rdoctask'
+Rake::RDocTask.new do |rdoc|
+  if File.exist?('VERSION.yml')
+    config = YAML.load(File.read('VERSION.yml'))
+    version = "#{config[:major]}.#{config[:minor]}.#{config[:patch]}"
+  else
+    version = ""
+  end
+
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title = "ass #{version}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:major: 0
+:minor: 1
+:patch: 0

data/lib/ass.rb

@@ -0,0 +1,125 @@
+$:.unshift File.expand_path(File.dirname(File.expand_path(__FILE__)))
+require 'mq'
+
+module ASS; end
+require 'ass/amqp' # monkey patch stolen from nanite.
+require 'ass/serializers'
+
+require 'ass/server' # monkey patch stolen from nanite.
+require 'ass/callback_factory'
+require 'ass/actor'
+require 'ass/rpc'
+require 'ass/client'
+
+require 'ass/topic'
+
+module ASS
+
+  class << self
+
+    #MQ = nil
+    def start(settings={})
+      raise "should have one ASS per eventmachine" if EM.reactor_running? == true # allow ASS to restart if EM is not running.
+      EM.run {
+        @serializer = settings.delete(:format) || ::Marshal
+        raise "Object Serializer must respond to :load and :dump" unless @serializer.respond_to?(:load) && @serializer.respond_to?(:dump)
+        @mq = ::MQ.new(AMQP.start(settings))
+        # ASS and its worker threads (EM.threadpool) should share the same MQ instance.
+        yield if block_given?
+      }
+    end
+
+    def stop
+      AMQP.stop{ EM.stop }
+      true
+    end
+
+    def mq
+      @mq
+    end
+
+    def serializer
+      @serializer
+    end
+
+    def server(name,opts={},&block)
+      s = ASS::Server.new(name,opts)
+      if block
+        s.react(&block)
+      end
+      s
+    end
+
+    def actor(name,opts={},&block)
+      s = ASS::Actor.new(name,opts)
+      if block
+        s.react(&block)
+      end
+      s
+    end
+
+    def rpc(opts={})
+      ASS::RPC.new(opts)
+    end
+
+    # the opts is used to initiate an RPC
+    def client(opts={})
+      ASS::Client.new(opts)
+    end
+
+    # maybe move cast and call into ASS::Server's class methods
+    def cast(name,method,data,opts,meta)
+      call(name,method,data,opts.merge(:reply_to => nil),meta)
+    end
+    
+    def call(name,method,data,opts,meta)
+      # make sure the payload hash use string
+      # keys. Serialization format might not
+      # preserve type.
+      payload = {
+        #:type => type,
+        "method" => method,
+        "data" => data,
+        "meta" => meta,
+      }
+      payload.merge("version" => opts[:version]) if opts.has_key?(:version)
+      payload.merge("meta" => opts[:meta]) if opts.has_key?(:meta)
+      dummy_exchange(name).publish(ASS.serializer.dump(payload),opts)
+      true
+    end
+
+    # this would create a dummy MQ exchange object
+    # for the sole purpose of publishing the
+    # message. Will not clobber existing server
+    # already started in the process.
+    def dummy_exchange(name)
+      @mq.direct(name,:no_declare => true)
+    end
+  end
+  
+  
+  #   def self.topic(name,opts={})
+  #     ASS::Topic.new(name,opts)
+  #   end
+
+  
+
+  
+  # def self.peep(server_name,callback=nil,&block)
+#     callback = block if callback.nil?
+#     callback = Module.new {
+#       def server(*args)
+#         p [:server,args]
+#       end
+
+#       def client(*args)
+#         p [:client,args]
+#       end
+#     }
+#     ASS::Peeper.new(server_name,callback)
+#   end
+
+  # assumes server initializes it with an exclusive and auto_delete queue.
+  
+  
+end

data/lib/ass/actor.rb

@@ -0,0 +1,50 @@
+class ASS::Actor
+  # this class is a thin layer over ASS::Server
+  def initialize(name,opts={},&block)
+    @server = ASS.server(name,opts)
+    if block
+      react(&block)
+    end
+  end
+
+  def queue(opts={})
+    @server.queue(opts)
+    self
+  end
+
+  def react(callback=nil,opts=nil,&block)
+    if block
+      opts = callback
+      callback = block
+    end
+    opts = {} if opts.nil?
+    callback_factory = ASS::CallbackFactory.new(callback)
+    server = @server # for closure capturing, needed for callback_factory
+    @server.react(opts) {
+      define_method(:on_call) do |_|
+        raise "can't call an actor with method set to nil" if payload["method"].nil?
+        callback_object = callback_factory.callback_for(server,header,payload)
+        callback_object.send(payload["method"],payload["data"])
+      end
+
+      define_method(:on_error) do |e,_|
+        callback_object = callback_factory.callback_for(server,header,payload)
+        if callback_object.respond_to?(:on_error)
+          callback_object.on_error(e,payload["data"])
+        else
+          raise e
+        end
+      end
+    }
+    self
+  end
+
+  def call(*args)
+    @server.call(*args)
+  end
+
+  def cast(*args)
+    @server.cast(*args)
+  end
+  
+end

data/lib/ass/amqp.rb

@@ -0,0 +1,19 @@
+
+# monkey patch to the amqp gem that adds :no_declare => true option for new 
+# Exchange objects. This allows us to send messeages to exchanges that are
+# declared by the mappers and that we have no configuration priviledges on.
+# temporary until we get this into amqp proper
+MQ::Exchange.class_eval do
+  def initialize mq, type, name, opts = {}
+    @mq = mq
+    @type, @name, @opts = type, name, opts
+    @mq.exchanges[@name = name] ||= self
+    @key = opts[:key]
+
+    @mq.callback{
+      @mq.send AMQP::Protocol::Exchange::Declare.new({ :exchange => name,
+                                                 :type => type,
+                                                 :nowait => true }.merge(opts))
+    } unless name == "amq.#{type}" or name == ''  or opts[:no_declare]
+  end
+end

data/lib/ass/callback_factory.rb

@@ -0,0 +1,95 @@
+class ASS::CallbackFactory
+
+  module ServiceMethods
+    def resend
+      throw(:__ass_resend)
+    end
+    
+    def discard
+      throw(:__ass_discard)
+    end
+    
+    def header
+      @__header__
+    end
+
+    def payload
+      @__payload__
+    end
+
+    def method
+      @__method__
+    end
+
+    def data
+      @__data__
+    end
+
+    def meta
+      @__meta__
+    end
+
+    def version
+      @__version__
+    end
+
+    def call(name,method,data=nil,opts={},meta=nil)
+      @__service__.call(name,method,data,opts,meta)
+    end
+
+    def cast(name,method,data=nil,opts={},meta=nil)
+      @__service__.cast(name,method,data,opts,meta)
+    end
+  end
+  
+  def initialize(callback)
+    @factory = build_factory(callback)
+  end
+
+  def callback_for(server,header,payload)
+    # method,data
+    if @factory.is_a? Class
+      if @factory.respond_to? :version
+        klass = @factory.get_version(payload["version"])
+      else
+        klass = @factory
+      end
+      obj = klass.new
+    else
+      obj = @factory
+    end
+    obj.instance_variable_set("@__service__",server)
+    obj.instance_variable_set("@__header__",header)
+    obj.instance_variable_set("@__payload__",payload)
+    obj.instance_variable_set("@__method__",payload["method"])
+    obj.instance_variable_set("@__data__",payload["data"])
+    obj.instance_variable_set("@__meta__",payload["meta"])
+    obj.instance_variable_set("@__version__",payload["version"])
+    obj
+  end
+
+  private
+
+  def build_factory(callback)
+    c = case callback
+        when Proc
+          Class.new &callback
+        when Class
+          callback
+        when Module
+          Class.new { include callback }
+        else
+          raise "can build factory from one of Proc, Class, Module"
+        end
+    case c
+    when Class
+      c.instance_eval { include ServiceMethods }
+    else
+      c.extend ServiceMethods
+    end
+    c
+  end
+  
+
+  
+end