serf 0.6.1 → 0.7.0

data/Gemfile

@@ -12,17 +12,23 @@ gem 'uuidtools', '>= 2.1.2'
 # Add dependencies to develop your gem here.
 # Include everything needed to run rake, tests, features, etc.
 group :development, :test do
-  gem 'rspec', '~> 2.3.0'
-  gem 'yard', '~> 0.6.0'
-  gem 'bundler', '~> 1.0.0'
-  gem 'jeweler', '~> 1.6.4'
+  gem "rspec", "~> 2.8.0"
+  gem "yard", "~> 0.7.5"
+  gem "bundler", "~> 1.0.22"
+  gem "jeweler", "~> 1.8.3"
   gem 'simplecov', '>= 0'
 
+  # For our testing
+  gem 'log4r', '~> 1.1.10'
+
   # Soft Dependencies
   #gem 'log4r', '~> 1.1.9'
   gem 'msgpack', '>= 0.4.6'
   #gem 'multi_json', '~> 1.0.3'
 
   # For Server Side of things
+
+  # EventMachine is now optional runner
   gem 'eventmachine', '>= 0.12.10'
+  gem 'girl_friday', '~> 0.9.7'
 end

data/Gemfile.lock

@@ -1,46 +1,56 @@
 GEM
   remote: http://rubygems.org/
   specs:
-    activesupport (3.2.0)
+    activesupport (3.2.2)
       i18n (~> 0.6)
       multi_json (~> 1.0)
+    connection_pool (0.1.0)
     diff-lcs (1.1.3)
     eventmachine (0.12.10)
+    girl_friday (0.9.7)
+      connection_pool (~> 0.1.0)
     git (1.2.5)
     i18n (0.6.0)
-    jeweler (1.6.4)
+    jeweler (1.8.3)
       bundler (~> 1.0)
       git (>= 1.2.5)
       rake
+      rdoc
+    json (1.6.5)
+    log4r (1.1.10)
     msgpack (0.4.6)
-    multi_json (1.0.4)
+    multi_json (1.1.0)
     rake (0.9.2.2)
-    rspec (2.3.0)
-      rspec-core (~> 2.3.0)
-      rspec-expectations (~> 2.3.0)
-      rspec-mocks (~> 2.3.0)
-    rspec-core (2.3.1)
-    rspec-expectations (2.3.0)
+    rdoc (3.12)
+      json (~> 1.4)
+    rspec (2.8.0)
+      rspec-core (~> 2.8.0)
+      rspec-expectations (~> 2.8.0)
+      rspec-mocks (~> 2.8.0)
+    rspec-core (2.8.0)
+    rspec-expectations (2.8.0)
       diff-lcs (~> 1.1.2)
-    rspec-mocks (2.3.0)
-    simplecov (0.5.4)
-      multi_json (~> 1.0.3)
+    rspec-mocks (2.8.0)
+    simplecov (0.6.1)
+      multi_json (~> 1.0)
       simplecov-html (~> 0.5.3)
     simplecov-html (0.5.3)
     uuidtools (2.1.2)
-    yard (0.6.8)
+    yard (0.7.5)
 
 PLATFORMS
   ruby
 
 DEPENDENCIES
   activesupport (>= 3.2.0)
-  bundler (~> 1.0.0)
+  bundler (~> 1.0.22)
   eventmachine (>= 0.12.10)
+  girl_friday (~> 0.9.7)
   i18n (>= 0.6.0)
-  jeweler (~> 1.6.4)
+  jeweler (~> 1.8.3)
+  log4r (~> 1.1.10)
   msgpack (>= 0.4.6)
-  rspec (~> 2.3.0)
+  rspec (~> 2.8.0)
   simplecov
   uuidtools (>= 2.1.2)
-  yard (~> 0.6.0)
+  yard (~> 0.7.5)

data/README.md

@@ -15,7 +15,7 @@ manner.
 
 Fundamentally, a service creates an abstraction of:
 1. Messages
-2. Handlers
+2. Handlers/Commands
 
 Messages are the representation of data, documents, etc that are
 marshalled from client to service, which represents the business
@@ -29,22 +29,22 @@ Handlers are the code that is executed over Messages.
 Handlers may process Command Messages.
 Handlers may process observed Events that 3rd party services emit.
 
-Services SHOULD declare a manifest to map messages to handlers, which
-allows Serf to route messages and to determine blocking or non-blocking modes.
-
 Serf App and Channels
 =====================
 
 A Serf App is a Rack-like application that accepts an ENV hash as input.
 This ENV hash is simply the hash representation of a message to be processed.
-The Serf App, as configured by registered manifests, will:
-1. route the ENV to the proper handler
-2. run the handler in blocking or non-blocking mode.
-  a. The handler's serf call method will parse the ENV into a message object
-    if the handler registered a Message class.
-3. publish the handler's results to results or error channels.
-  a. These channels are normally message queuing channels.
-  b. We only require the channel instance to respond to the 'publish'
+
+The Serf App, as configured by DSL, will:
+1. route the ENV to the proper Endpoint
+2. The endpoint will create a handler instance.
+  a. The handler's build class method will be given an ENV, which
+    may be handled as fit. The Command class will help by parsing it
+    into a Message object as registered by the implementing subclass.
+3. Push the handler's results to a response channel.
+  a. Raised errors are caught and pushed to an error channel.
+  b. These channels are normally message queuing channels.
+  c. We only require the channel instance to respond to the 'push'
     method and accept a message (or message hash) as the argument.
 4. return the handler's results to the caller if it is blocking mode.
   a. In non-blocking mode, an MessageAcceptedEvent is returned instead
@@ -65,12 +65,14 @@ Service Libraries
     hashes that define at least one attribute: 'kind'.
 2. Serialization of the messages SHOULD BE Json or MessagePack (I hate XML).
   a. `to_hash` is also included.
-3. Handler methods MUST receive a message as either as an options hash
-  (with symbolized keys) or an instance of a declared Message.
-4. Handler methods MUST return zero or more messages.
+3. Handlers MUST implement the 'build' class method, which
+  MUST receive the ENV Hash as the first parameter, followed by supplemental
+  arguments as declared in the DSL.
+4. Handler methods SHOULD return zero or more messages.
+  a. Raised errors are caught and pushed to error channels.
 5. Handler methods SHOULD handle catch their business logic exceptions and
   return them as specialized messages that can be forwarded down error channels.
-  Uncaught exceptions that are then caught by Serf are published as
+  Uncaught exceptions that are then caught by Serf are pushed as
   generic Serf::CaughtExceptionEvents, and are harder to deal with.
 
 
@@ -107,27 +109,28 @@ aware that:
     hash that is to be parsed into a message object.
 
 
-Example
-=======
+Example With GirlFriday
+=======================
 
     # Require our libraries
-    require 'active_model'
     require 'log4r'
     require 'json'
 
     require 'serf/builder'
+    require 'serf/command'
     require 'serf/message'
     require 'serf/middleware/uuid_tagger'
+    require 'serf/util/options_extraction'
 
     # create a simple logger for this example
     outputter = Log4r::FileOutputter.new(
       'fileOutputter',
       filename: 'log.txt')
-    ['top_level',
+    ['tick',
       'serf',
-      'handler',
-      'results_channel',
-      'error_channel'].each do |name|
+      'hndl',
+      'resp',
+      'errr'].each do |name|
       logger = Log4r::Logger.new name
       logger.outputters = outputter
     end
@@ -135,131 +138,307 @@ Example
     # Helper class for this example to receive result or error messages
     # and pipe it into our logger.
     class MyChannel
-      def initialize(logger)
+      def initialize(logger, error=false)
         @logger = logger
+        @error = error
       end
-      def publish(message)
-        @logger.info "#{message}"
+      def push(message)
+        if @error
+          @logger.fatal "#{message}"
+        else
+          @logger.debug "#{message}"
+        end
       end
     end
 
     # my_lib/my_message.rb
+    # This class is stripped down minimal functionality that
+    # ActiveModel or Aequitas/Virtus implements.
     class MyMessage
+      include Serf::Util::OptionsExtraction
       include Serf::Message
-      include ActiveModel::Validations
 
       attr_accessor :data
 
-      validates_presence_of :data
-
-      def initialize(options={})
-        @data = options[:data]
+      def initialize(*args)
+        extract_options! args
+        self.data = opts :data
       end
 
-      # We define this for Serf::Message serialization.
       def attributes
         { 'data' => data }
       end
 
+      def valid?
+        !data.nil?
+      end
+
+      def full_error_messages
+        'Data is blank' if data.nil?
+      end
     end
 
-    # my_lib/my_handler.rb
-    class MyHandler
+    # my_lib/my_overloaded_command.rb
+    class MyOverloadedCommand
+      include Serf::Command
 
-      def initialize(options={})
-        @logger = options[:logger]
+      self.request_factory = MyMessage
+
+      def initialize(*args)
+        super
+        raise "Constructor Error: #{opts(:name)}" if opts :raises_in_new
       end
 
-      def submit_my_message(message)
-        @logger.info "In Submit My Message: #{message.to_json}"
-        # Validate message because we have implement my_message with it.
-        unless message.valid?
-          raise ArgumentError, message.errors.full_messages.join(',')
-        end
+      def call
+        # Set our logger
+        logger = ::Log4r::Logger['hndl']
+
+        # Just our name to sort things out
+        name = opts! :name
+        logger.info "#{name}: #{request.to_json}"
+
+        raise "Forcing Error in #{name}" if opts(:raises, false)
+
         # Do work Here...
         # And return 0 or more messages as result. Nil is valid response.
-        return { kind: 'my_message_results' }
+        return { kind: "#{name}_result", input: request.to_hash }
+      end
+
+      def inspect
+        "MyOverloadedCommand: #{opts(:name,'notnamed')}, #{request.to_json}"
+      end
+
+    end
+
+    # Create a new builder for this serf app.
+    builder = Serf::Builder.new do
+      # Include some middleware
+      use Serf::Middleware::UuidTagger
+
+      # Create response and error channels for the handler result messages.
+      response_channel MyChannel.new(::Log4r::Logger['resp'])
+      error_channel MyChannel.new(::Log4r::Logger['errr'], true)
+
+      # We pass in a logger to our Serf code: Serfer and Runners.
+      logger ::Log4r::Logger['serf']
+
+      runner :direct
+
+      match 'my_message'
+      run MyOverloadedCommand, name: 'my_message_command'
+
+      match 'raise_error_message'
+      run MyOverloadedCommand, name: 'foreground_raises_error', raises: true
+      run MyOverloadedCommand, name: 'constructor_error', raises_in_new: true
+
+      match 'other_message'
+      run MyOverloadedCommand, name: 'foreground_other_message'
+
+      runner :girl_friday
+
+      # This message kind is handled by multiple handlers.
+      match 'other_message'
+      run MyOverloadedCommand, name: 'background_other_message'
+      run MyOverloadedCommand, name: 'background_raises error', raises: true
+
+      match /^events\/.*$/
+      run MyOverloadedCommand, name: 'regexp_matched_command'
+
+      # Optionally define a not found handler...
+      # Defaults to raising an ArgumentError, 'Not Found'.
+      #not_found lambda {|x| puts x}
+    end
+    app = builder.to_app
+
+    # Start event machine loop.
+    logger = ::Log4r::Logger['tick']
+
+    logger.info "Start Tick #{Thread.current.object_id}"
+
+    # This will submit a 'my_message' message (as a hash) to the Serf App.
+    # NOTE: We should get an error message pushed to the error channel
+    #  because no 'data' field was put in my_message as required
+    #  And the Result should have a CaughtExceptionEvent.
+    logger.info "BEG MyMessage w/o Data"
+    results = app.call 'kind' => 'my_message'
+    logger.info "END MyMessage w/o Data: #{results.size} #{results.to_json}"
+
+    # Here is good result
+    logger.info "BEG MyMessage w/ Data"
+    results = app.call 'kind' => 'my_message', 'data' => '1'
+    logger.info "END MyMessage w/ Data: #{results.size} #{results.to_json}"
+
+    # Here is a result that will raise an error in foreground
+    # We should get two event messages in the results because we
+    # mounted two commands to the raise_error_message kind.
+    # Each shows errors being raised in two separate stages.
+    # 1. Error in creating the instance of the command.
+    # 2. Error when the command was executed by the foreground runner.
+    logger.info "BEG RaisesErrorMessage"
+    results = app.call 'kind' => 'raise_error_message', 'data' => '2'
+    logger.info "END RaisesErrorMessage: #{results.size} #{results.to_json}"
+
+    # This submission will be executed by THREE commands.
+    #   One in the foreground, two in the background.
+    #
+    # The foreground results should be:
+    # * MessageAcceptedEvent
+    # * And return result of one command call
+    #
+    # The error channel should output an error from one background command.
+    logger.info "BEG OtherMessage"
+    results = app.call 'kind' => 'other_message', 'data' => '3'
+    logger.info "END OtherMessage: #{results.size} #{results.to_json}"
+
+    # This will match a regexp call
+    logger.info "BEG Regexp"
+    results = app.call 'kind' => 'events/my_event', 'data' => '4'
+    logger.info "END Regexp Results: #{results.size} #{results.to_json}"
+
+    begin
+      # Here, we're going to submit a message that we don't handle.
+      # By default, an exception will be raised.
+      app.call 'kind' => 'unhandled_message_kind'
+    rescue => e
+      logger.warn "Caught in Tick: #{e.inspect}"
+    end
+    logger.info "End Tick #{Thread.current.object_id}"
+
+Example With EventMachine
+=========================
+
+    # Require our libraries
+    require 'log4r'
+    require 'json'
+
+    require 'serf/builder'
+    require 'serf/command'
+    require 'serf/message'
+    require 'serf/middleware/uuid_tagger'
+    require 'serf/util/options_extraction'
+
+    # create a simple logger for this example
+    outputter = Log4r::FileOutputter.new(
+      'fileOutputter',
+      filename: 'log.txt')
+    ['tick',
+      'serf',
+      'hndl',
+      'resp',
+      'errr'].each do |name|
+      logger = Log4r::Logger.new name
+      logger.outputters = outputter
+    end
+
+    # Helper class for this example to receive result or error messages
+    # and pipe it into our logger.
+    class MyChannel
+      def initialize(logger, error=false)
+        @logger = logger
+        @error = error
+      end
+      def push(message)
+        if @error
+          @logger.fatal "#{message}"
+        else
+          @logger.debug "#{message}"
+        end
       end
+    end
 
-      def submit_other_message(message={})
-        # The message here is the ENV hash because we didn't declare
-        # an :as option with `receives`.
-        @logger.info "In Submit OtherMessage: #{message.inspect.to_s}"
-        return [
-          { kind: 'other_message_result1' },
-          { kind: 'other_message_result2' }
-        ]
+    # my_lib/my_message.rb
+    # This class is stripped down minimal functionality that
+    # ActiveModel or Aequitas/Virtus implements.
+    class MyMessage
+      include Serf::Util::OptionsExtraction
+      include Serf::Message
+
+      attr_accessor :data
+
+      def initialize(*args)
+        extract_options! args
+        self.data = opts :data
       end
 
-      def raises_error(message={})
-        @logger.info 'In Raises Error, about to raise error'
-        raise 'My Handler Runtime Error'
+      def attributes
+        { 'data' => data }
       end
 
-      def regexp_matched(message={})
-        @logger.info "RegExp Matched #{message.inspect}"
-        nil
+      def valid?
+        !data.nil?
       end
 
-      def call(message={})
-        @logger.info "A message matched an empty action part in the target"
-        nil
+      def full_error_messages
+        'Data is blank' if data.nil?
       end
     end
 
-    # my_lib/routes.rb
-    ROUTES = {
-      # Declare a matcher and a list of routes to endpoints.
-      # We can declare a single route.
-      'my_message' => {
-        # Declares which handler and action (method) of the handler
-        # to call. The handler part is the name of the handler used
-        # to register an actual handler object.
-        target: 'my_handler#submit_my_message',
-
-        # Define a parser that will build up a message object.
-        # Default: nil, no parsing done.
-        # Or name of registered parser to use.
-        message_parser: 'my_parser',
-
-        # Default is process in foreground.
-        #background: false
-      },
-      # This message kind is handled by multiple handlers.
-      'other_message' => [{
-        target: 'my_handler#submit_other_message',
-        background: true
-      }, {
-        target: 'my_handler#raises_error',
-        background: true
-      },
-        # This is just a string route defining the target, nothing else.
-        # The handler is 'my_handler' and an empty (or missing) action
-        # part defaults to the 'call' method of the handler.
-        'my_handler'
-      ],
-      /^events\/.*$/ => 'my_handler#regexp_matched'
-    }
+    # my_lib/my_overloaded_command.rb
+    class MyOverloadedCommand
+      include Serf::Command
+
+      self.request_factory = MyMessage
+
+      def initialize(*args)
+        super
+        raise "Constructor Error: #{opts(:name)}" if opts :raises_in_new
+      end
+
+      def call
+        # Set our logger
+        logger = ::Log4r::Logger['hndl']
+
+        # Just our name to sort things out
+        name = opts! :name
+        logger.info "#{name}: #{request.to_json}"
+
+        raise "Forcing Error in #{name}" if opts(:raises, false)
+
+        # Do work Here...
+        # And return 0 or more messages as result. Nil is valid response.
+        return { kind: "#{name}_result", input: request.to_hash }
+      end
+
+      def inspect
+        "MyOverloadedCommand: #{opts(:name,'notnamed')}, #{request.to_json}"
+      end
+
+    end
 
     # Create a new builder for this serf app.
     builder = Serf::Builder.new do
       # Include some middleware
       use Serf::Middleware::UuidTagger
 
-      # Registers routes from different service libary manifests.
-      routes ROUTES
-
-      # Can define arguments to pass to the 'my_handler' initialize method.
-      handler 'my_handler', MyHandler.new(logger: ::Log4r::Logger['handler'])
-      message_parser 'my_parser', MyMessage
-
-      # Create result and error channels for the handler result messages.
-      error_channel MyChannel.new(::Log4r::Logger['error_channel'])
-      results_channel MyChannel.new(::Log4r::Logger['results_channel'])
+      # Create response and error channels for the handler result messages.
+      response_channel MyChannel.new(::Log4r::Logger['resp'])
+      error_channel MyChannel.new(::Log4r::Logger['errr'], true)
 
       # We pass in a logger to our Serf code: Serfer and Runners.
       logger ::Log4r::Logger['serf']
 
+      runner :direct
+
+      match 'my_message'
+      run MyOverloadedCommand, name: 'my_message_command'
+
+      match 'raise_error_message'
+      run MyOverloadedCommand, name: 'foreground_raises_error', raises: true
+      run MyOverloadedCommand, name: 'constructor_error', raises_in_new: true
+
+      match 'other_message'
+      run MyOverloadedCommand, name: 'foreground_other_message'
+
+      runner :event_machine
+
+      # This message kind is handled by multiple handlers.
+      match 'other_message'
+      run MyOverloadedCommand, name: 'background_other_message'
+      run MyOverloadedCommand, name: 'background_raises error', raises: true
+
+      match /^events\/.*$/
+      run MyOverloadedCommand, name: 'regexp_matched_command'
+
       # Optionally define a not found handler...
       # Defaults to raising an ArgumentError, 'Not Found'.
       #not_found lambda {|x| puts x}
@@ -267,7 +446,7 @@ Example
     app = builder.to_app
 
     # Start event machine loop.
-    logger = ::Log4r::Logger['top_level']
+    logger = ::Log4r::Logger['tick']
     EM.run do
       # On the next tick
       EM.next_tick do
@@ -277,28 +456,46 @@ Example
         # NOTE: We should get an error message pushed to the error channel
         #  because no 'data' field was put in my_message as required
         #  And the Result should have a CaughtExceptionEvent.
-        results = app.call('kind' => 'my_message')
-        logger.info "In Tick, MyMessage Results: #{results.inspect}"
+        logger.info "BEG MyMessage w/o Data"
+        results = app.call 'kind' => 'my_message'
+        logger.info "END MyMessage w/o Data: #{results.size} #{results.to_json}"
 
         # Here is good result
-        results = app.call('kind' => 'my_message', 'data' => '1234')
-        logger.info "In Tick, MyMessage Results: #{results.inspect}"
-
-        # This will submit 'other_message' to be handled in foreground
-        # Because we declared the 'other_message' kind to be handled async
-        # in each route config, we should get a MessageAcceptedEvent as
-        # the results.
-        results = app.call('kind' => 'other_message')
-        logger.info "In Tick, OtherMessage Results: #{results.inspect}"
-
-        # This will match a regexp
-        results = app.call('kind' => 'events/my_event')
-        logger.info "In Tick, Regexp Results: #{results.inspect}"
+        logger.info "BEG MyMessage w/ Data"
+        results = app.call 'kind' => 'my_message', 'data' => '1'
+        logger.info "END MyMessage w/ Data: #{results.size} #{results.to_json}"
+
+        # Here is a result that will raise an error in foreground
+        # We should get two event messages in the results because we
+        # mounted two commands to the raise_error_message kind.
+        # Each shows errors being raised in two separate stages.
+        # 1. Error in creating the instance of the command.
+        # 2. Error when the command was executed by the foreground runner.
+        logger.info "BEG RaisesErrorMessage"
+        results = app.call 'kind' => 'raise_error_message', 'data' => '2'
+        logger.info "END RaisesErrorMessage: #{results.size} #{results.to_json}"
+
+        # This submission will be executed by THREE commands.
+        #   One in the foreground, two in the background.
+        #
+        # The foreground results should be:
+        # * MessageAcceptedEvent
+        # * And return result of one command call
+        #
+        # The error channel should output an error from one background command.
+        logger.info "BEG OtherMessage"
+        results = app.call 'kind' => 'other_message', 'data' => '3'
+        logger.info "END OtherMessage: #{results.size} #{results.to_json}"
+
+        # This will match a regexp call
+        logger.info "BEG Regexp"
+        results = app.call 'kind' => 'events/my_event', 'data' => '4'
+        logger.info "END Regexp Results: #{results.size} #{results.to_json}"
 
         begin
           # Here, we're going to submit a message that we don't handle.
           # By default, an exception will be raised.
-          app.call('kind' => 'unhandled_message_kind')
+          app.call 'kind' => 'unhandled_message_kind'
         rescue => e
           logger.warn "Caught in Tick: #{e.inspect}"
         end