right_support 1.4.1 → 2.0.0

data/README.rdoc

@@ -9,57 +9,76 @@ SystemLogger is a rewrite of the seattle.rb SyslogLogger class that features the
 * Inherits from standard Ruby Logger class for guaranteed compatibility
 * Can be configured with options about how to handle newlines, ANSI escape codes, etc
 
-It is very similar to SystemLogger, with a few differences (e.g. it is a child class of Logger instead of
-merely being duck-type compatible). You use it as follows.
+We also provide Log::Mixin, a powerful tool that lets you sprinkle a logger
+into any object or class, and supports per-class or per-instance custom
+loggers!
 
-  @logger = SystemLogger.new('my_cool_app', :split=>true, :color=>false)
-  @logger.info "Hello world\nThis will appear on separate lines\nand without \e[33;0mbeautiful colors" 
+  class Jet < Aircraft
+    include RightSupport::Log.Mixin
+  end
 
-CustomLogger is a Rack middleware that allows a Rack app to use any log sink it wishes. The
-stock Rack logger middleware is inflexible and gives the end user no control over which logger is used
-or where the log entries go.
+  #most jets log to the syslog
+  Jet.logger = RightSupport::Log::SystemLogger.new('black box',
+                 :facility=>'local0')
 
-  require 'right_support/rack/custom_logger'
+  #but MY jet
+  @my_jet.logger = Logger.new(StringIO.new)
 
-  my_logger = MyAwesomeLogger.new
-  use RightSupport::Rack::CustomLogger, Logger::INFO, my_logger
+==== Logging for Rack Applications
 
-=== String Manipulation
+RightSupport also provides Rack middleware that allows a Rack app to use any log sink it wishes. The
+stock Rack logger middleware is inflexible and gives the end user no control over which logger is used
+or where the log entries go. Example of usage on Sinatra based application:
+
+  class MyApp < Sinatra::Base
+    ...
+    # Specify which logger will be used as default logger
+    LOGGER =\
+      if ENV['RACK_ENV'].downcase == 'development'
+        Logger.new(STDOUT)
+      else
+        RightSupport::Log::SystemLogger.new("MyApp", {:facility => "local0"})
+      end    
+
+    # use the RequestLogger via LogSetter to log requests for application
+    use RightSupport::Rack::LogSetter, {:logger => LOGGER}
+    use RightSupport::Rack::RequestLogger    
+
+    # set logger and mix Log::Mixin
+    RightSupport::Log::Mixin.default_logger = LOGGER
+    include RightSupport::Log::Mixin
+    ...
+  end
 
-StringExtensions contains String#camelize, which is only defined if the
-ActiveSupport gem cannot be loaded. It also has some RightScale-specific
-methods:
-* String#snake_case
-* String#to_const
-* String#to_const_path
+After that all rack requests to MyApp will be logged, and since we mixed Log::Mixin we can use:
 
-Net::StringEncoder applies and removes URL-escape, base64 and other encodings.
+  logger.info "Hello world\nThis will appear on separate lines\nand without \e[33;0mbeautiful colors"
 
-=== Input Validation
+=== Networking Stuff
 
-Validation  contains several format-checkers that can be used to validate your
-web app's models before saving, check for preconditions in your controllers,
-and so forth.
+==== HTTP Client
 
-You can use it as a mixin by including the appropriate child module of
-RightSupport::Validation.
+We provide a very thin wrapper around the rest-client gem that enables simple but
+robust rest requests with a timeout, headers, etc.
 
-  class AwesomenessGenerator < ActiveRecord::Base
-    include RightSupport::Validation::OpenSSL
+HTTPClient is interface-compatible with the RestClient module, but allows an
+optional timeout to be specified as an extra parameter.
 
-    before_save do |record|
-      errors[:foo] = 'Hey, that's not a key!' unless pem_public_key?(record.foo)
-    end
-  end
+   # Create a wrapper object
+   @client = RightSupport::Net::HTTPClient.new
 
-But you really don't want to do that, do you? Instead, you want to call the module
-methods of RightSupport::Validation, which contains all of the same mixin methods,
-but does not pollute the dispatch table of your application classes.
+   # Default timeout is 5 seconds
+   @client.get('http://localhost')
 
-  the_key = STDIN.read
-  raise ArgumentError unless RightSupport::Validation.ssh_public_key?(the_key)
+   # Make sure the HTTPClient request fails after 1 second so we can report an error
+   # and move on!
+   @client.get('http://localhost', {:headers => {'X-Hello'=>'hi!'}, :timeout => 1)}
 
-=== Client-Side Load Balancer
+   # HTTPClient transforms String or Hash :query into query string, for example;
+   @client.get('http://localhost/moo', {:query=>{:a=>{:b=>:c}}} )
+   # the url that would be requested is http://localhost/moo?a[b]=c
+
+==== Client-Side Load Balancer
 
 RequestBalancer randomly chooses endpoints for a network request, which lets
 you perform easy client-side load balancing:
@@ -71,26 +90,23 @@ you perform easy client-side load balancing:
     REST.get(url)
   end
 
-The balancer will keep trying requests until one of them succeeds without throwing
-any exceptions. (NB: a nil return value counts as success!!)
+The balancer will keep trying requests until one of them succeeds without
+throwing any exceptions. (NB: a nil return value counts as success!!)
 
-=== HTTP Client
+==== HTTP Request Tracking for Rack
 
-We provide a very thin wrapper around the rest-client gem that enables simple but
-robust rest requests with a timeout, headers, etc.
+Correlate data flows across your entire architecture with the RequestTracker
+middleware, which uses custom HTTP headers to "tag" every Web request with
+a UUID, and works with RequestLogger to log "begin" and "end" lines containing
+the UUID.
 
-HTTPClient is interface-compatible with the RestClient module, but allows an
-optional timeout to be specified as an extra parameter.
-   
-   # Create a wrapper object
-   @client = RightSupport::Net::HTTPClient.new
-   
-   # Default timeout is 5 seconds
-   @client.get('http://localhost')
+If your app consumes the UUID and passes it on to HTTP services that it
+invokes, the same request UUID will appear in all logs and you can grep for
+the "big picture" instead of wasting time paging between logs.
 
-   # Make sure the HTTPClient request fails after 1 second so we can report an error
-   # and move on!
-   @client.get('http://localhost', {:headers => {'X-Hello'=>'hi!'}, :timeout => 1)}
+To use this functionality you need:
+
+  use RightSupport::Rack::RequestTracker
 
 === Statistics Gathering
 
@@ -107,27 +123,37 @@ Profile your compute-heavy and network activities using a Stats::Activity counte
 
   puts "Only %.1f lines/sec? You are a slow typist!" % [stats.avg_rate]
 
-=== Configuration
+=== Input Validation
 
-RightSupport::Config contains functionality, which provides possibility 
-to use human-readable yaml configuration files. For example, you have following yaml 
-file '/tmp/features_config.yml', with content:
+Validation  contains several format-checkers that can be used to validate your
+web app's models before saving, check for preconditions in your controllers,
+and so forth.
 
-  eat: 
-    khlav kalash: YES!
-  speak: 
-    klingonese: false
-    belarusian: true
+You can use it as a mixin by including the appropriate child module of
+RightSupport::Validation.
 
-Then, if you would add configuration in you class, like:
+  class AwesomenessGenerator < ActiveRecord::Base
+    include RightSupport::Validation::OpenSSL
 
-  class SweetestClass    
-    CONFIG = RightSupport::Config.features('/tmp/features_config.yml')
+    before_save do |record|
+      errors[:foo] = 'Hey, that's not a key!' unless pem_public_key?(record.foo)
+    end
   end
 
-* RightSupport::Config.features receives file` path, io or string.
+But you really don't want to do that, do you? Instead, you want to call the module
+methods of RightSupport::Validation, which contains all of the same mixin methods,
+but does not pollute the dispatch table of your application classes.
 
-Now you can call CONFIG['feature_group']['feature'], like:
+  the_key = STDIN.read
+  raise ArgumentError unless RightSupport::Validation.ssh_public_key?(the_key)
 
-* SweetestClass::CONFIG['eat']['khlav kalash'] would return 'YES!'
-* SweetestClass::CONFIG['speak']['belarusian'] would return true
+=== String Manipulation
+
+StringExtensions contains String#camelize, which is only defined if the
+ActiveSupport gem cannot be loaded. It also has some RightScale-specific
+methods:
+* String#snake_case
+* String#to_const
+* String#to_const_path
+
+Net::StringEncoder applies and removes URL-escape, base64 and other encodings.

data/lib/right_support.rb

@@ -2,8 +2,8 @@
 require 'thread'
 
 require 'right_support/ruby'
+require 'right_support/data'
 require 'right_support/crypto'
-require 'right_support/config'
 require 'right_support/db'
 require 'right_support/log'
 require 'right_support/net'

data/lib/right_support/{config.rb → data.rb}

@@ -1,5 +1,5 @@
 #
-# Copyright (c) 2012 RightScale Inc
+# Copyright (c) 2011 RightScale Inc
 #
 # Permission is hereby granted, free of charge, to any person obtaining
 # a copy of this software and associated documentation files (the
@@ -22,13 +22,11 @@
 
 module RightSupport
   #
-  # A namespace for configuration tools.
+  # A namespace for data-processing tools.
   #
-  module Config
+  module Data
 
   end
 end
 
-require 'right_support/config/feature_set'
-require 'right_support/config/yaml_config'
-require 'right_support/config/recursive_true_class'
+require 'right_support/data/uuid'

data/lib/right_support/data/uuid.rb

@@ -0,0 +1,175 @@
+#
+# Copyright (c) 2011 RightScale Inc
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+require 'set'
+
+module RightSupport::Data
+  module UUID
+    # Exception that indicates a given implementation is unavailable.
+    class Unavailable < Exception; end
+
+    # The base class for all UUID implementations. Subclasses must override
+    # #initialize such that they raise an exception if the implementation is
+    # not available, and #generate such that it returns a String containing
+    # a UUID.
+    class Implementation
+      @subclasses = Set.new
+
+      def self.inherited(klass)
+        @subclasses << klass
+      end
+
+      # Determine which UUID implementations are available in this Ruby VM.
+      #
+      # === Return
+      # available(Array):: the available implementation classes
+      def self.available
+        avail = []
+
+        @subclasses.each do |eng|
+          begin
+            eng.new #if it initializes, it's available!
+            avail << eng
+          rescue Exception => e
+            #must not be available!
+          end
+        end
+      end
+
+      # Determine the default UUID implementation in this Ruby VM.
+      #
+      # === Return
+      # available(Array):: the available implementation classes
+      def self.default
+        #completely arbitrary
+        available.first
+      end
+
+      def generate
+        raise Unavailable, "This implementation is currently not available"
+      end
+    end
+
+    # An implementation that uses the SimpleUUID gem.
+    class SimpleUUID < Implementation
+      def initialize
+        require 'simple_uuid'
+        generate
+      end
+
+      def generate
+        ::SimpleUUID::UUID.new.to_guid
+      end
+    end
+
+    # An implementation that uses UUIDTools v1.
+    class UUIDTools1 < Implementation
+      def initialize
+        require 'uuidtools'
+        generate
+      end
+
+      def generate
+        ::UUID.timestamp_create.to_s
+      end
+    end
+
+    # An implementation that uses UUIDTools v2.
+    class UUIDTools2 < Implementation
+      def initialize
+        require 'uuidtools'
+        generate
+      end
+
+      def generate
+        ::UUIDTools::UUID.timestamp_create.to_s
+      end
+    end
+
+    # An implementation that uses the "uuid" gem.
+    class UUIDGem < Implementation
+      def initialize
+        require 'uuid'
+        generate
+      end
+
+      def generate
+        ::UUID.new.to_s
+      end
+    end
+
+    module_function
+
+    def generate
+      impl = implementation
+      raise Unavailable, "No implementation is available" unless impl
+      impl.generate
+    end
+
+    # Return the implementation that will be used to generate UUIDs.
+    #
+    # === Return
+    # The implementation object.
+    #
+    # === Raise
+    # Unavailable:: if no suitable implementation is available
+    #
+    def implementation
+      return @implementation if @implementation
+
+      if (defl = Implementation.default)
+        self.implementation = defl
+      else
+        raise Unavailable, "No implementation is available"
+      end
+
+      @implementation
+    end
+
+    # Set the implementation that will be used to generate UUIDs.
+    #
+    # === Parameters
+    # klass(inherits-from Implementation):: the implementation class
+    #
+    # === Return
+    # The class that was passed as a parameter
+    #
+    # === Raise
+    # Unavailable:: if the specified implementation is not available
+    # ArgumentError:: if klass is not a subclass of Implementation, or is not available
+    #
+    def implementation=(klass)
+      if klass.is_a?(Implementation)
+        @implementation = klass
+      elsif klass.is_a?(Class) && klass.ancestors.include?(Implementation)
+        @implementation = klass.new
+      elsif klass.nil?
+        @implementation = nil
+      else
+        raise ArgumentError, "Expected Implementation instance or subclass, got #{klass}"
+      end
+    rescue Exception => e
+      raise Unavailable, "#{klass} is not available due to a #{e.class}: #{e.message}"
+    end
+  end
+  
+end

data/lib/right_support/log.rb

@@ -31,7 +31,6 @@ end
 
 require 'right_support/log/system_logger'
 require 'right_support/log/filter_logger'
-require 'right_support/log/tag_logger'
 require 'right_support/log/null_logger'
 require 'right_support/log/multiplexer'
 require 'right_support/log/exception_logger'

data/lib/right_support/log/system_logger.rb

@@ -23,7 +23,7 @@
 require 'logger'
 
 module RightSupport::Log
-  if_require_succeeds('syslog') do
+  if require_succeeds?('syslog')
     # A logger that forwards log entries to the Unix syslog facility, but complies
     # with the interface of the Ruby Logger object and faithfully translates log
     # severities and other concepts. Provides optional cleanup/filtering in order

data/lib/right_support/net.rb

@@ -32,7 +32,7 @@ end
 require 'right_support/net/address_helper'
 require 'right_support/net/http_client'
 require 'right_support/net/string_encoder'
-require 'right_support/net/balancing'
+require 'right_support/net/lb'
 require 'right_support/net/request_balancer'
 require 'right_support/net/ssl'
 require 'right_support/net/dns'

data/lib/right_support/net/http_client.rb

@@ -1,12 +1,12 @@
 module RightSupport::Net
-  if_require_succeeds('right_http_connection') do
+  if require_succeeds?('right_http_connection')
     #nothing, nothing at all! just need to make sure
     #that RightHttpConnection gets loaded before
     #rest-client, so the Net::HTTP monkey patches
     #take effect.
   end
 
-  if_require_succeeds('restclient') do
+  if require_succeeds?('restclient')
     HAS_REST_CLIENT = true
   end
 
@@ -24,31 +24,74 @@ module RightSupport::Net
   #
   #
   # HTTPClient is a thin wrapper around the RestClient::Request class, with a few minor changes to its
-  # interface:
-  #  * initializer accepts some default request options that can be overridden per-request
-  #  * it has discrete methods for get/put/post/delete, instead of a single "request" method
-  #
-  #   # create an instance ot HTTPClient with some default request options
+  # interface, namely:
+  #  * initializer accepts some default request options that can be overridden
+  #    per-request
+  #  * it has discrete methods for get/put/post/delete, instead of a single
+  #    "request" method
+  #  * it supports explicit :query and :payload options for query-string and
+  #    request-body, and understands the Rails convention for encoding a
+  #    nested Hash into request parameters.
+  #
+  # == Request Parameters
+  # You can include a query-string with your request by passing the :query
+  # option to any of the request methods. You can pass a Hash, which will
+  # be translated to a URL-encoded query string using the Rails convention
+  # for nesting. Or, you can pass a String which will be appended verbatim
+  # to the URL. (In this case, don't forget to CGI.escape your string!)
+  #
+  # To include a form with your request, pass the :payload option to any
+  # request method. You can pass a Hash, which will be translated to an
+  # x-www-form-urlencoded request body using the Rails convention for
+  # nesting. Or, you can pass a String which will be appended verbatim
+  # to the URL. You can even use a binary String combined with a
+  # suitable request-content-type header to pass binary data in the
+  # payload. (In this case, be very careful about String encoding under
+  # Ruby 1.9!)
+  #
+  # == Usage Examples
+  #
+  #   # Create an instance ot HTTPClient with some default request options
   #   @client = HTTPClient.new()
   #
-  #   # GET
+  #   # Simple GET
   #   xml = @client.get 'http://example.com/resource'
-  #   # and, with timeout of 5 seconds...
-  #   jpg = @client.get 'http://example.com/resource', {:accept => 'image/jpg', :timeout => 5}
-  #
-  #   # authentication and SSL
-  #   @client.get 'https://user:password@example.com/private/resource'
   #
-  #   # POST or PUT with a hash sends parameters as a urlencoded form body
-  #   @client.post 'http://example.com/resource', {:param1 => 'one'}
+  #   # And, with timeout of 5 seconds...
+  #   jpg = @client.get 'http://example.com/resource',
+  #     {:accept => 'image/jpg', :timeout => 5}
   #
-  #   # nest hash parameters, add a timeout of 10 seconds (and specify "no extra headers")
-  #   @client.post 'http://example.com/resource', {:payload => {:nested => {:param1 => 'one'}}, :timeout => 10}
+  #   # Doing some client authentication and SSL.
+  #   @client.get 'https://user:password@example.com/private/resource'
+  #   
+  #   # The :query option will be encoded as a URL query-string using Rails
+  #   # nesting convention (e.g. "a[b]=c" for this case).
+  #   @client.get 'http://example.com', :query=>{:a=>{:b=>'c'}}
+  #
+  #   # The :payload option specifies the request body. You can specify a raw
+  #   # payload:
+  #   @client.post 'http://example.com/resource', :payload=>'hi hi hi lol lol'
+  #
+  #   # Or, you can specify a Hash payload which will be translated to a
+  #   # x-www-form-urlencoded request body using the Rails nesting convention.
+  #   # (e.g. "a[b]=c" for this case)
+  #   @client.post 'http://example.com/resource', :payload=>{:d=>{:e=>'f'}}
+  #
+  #   # You can specify query and/or payload for any HTTP verb, even if it
+  #   # defies convention  (be careful!)
+  #   @client.post 'http://example.com/resource',
+  #     :query   => {:query_string_param=>'hi'}
+  #     :payload => {:form_param=>'hi'}, :timeout => 10
   #
   #   # POST and PUT with raw payloads
-  #   @client.post 'http://example.com/resource', {:payload => 'the post body', :headers => {:content_type => 'text/plain'}}
-  #   @client.post 'http://example.com/resource.xml', {:payload => xml_doc}
-  #   @client.put 'http://example.com/resource.pdf', {:payload => File.read('my.pdf'), :headers => {:content_type => 'application/pdf'}}
+  #   @client.post 'http://example.com/resource',
+  #     {:payload => 'the post body',
+  #      :headers => {:content_type => 'text/plain'}}
+  #   @client.post 'http://example.com/resource.xml',
+  #     {:payload => xml_doc}
+  #   @client.put 'http://example.com/resource.pdf',
+  #     {:payload => File.read('my.pdf'),
+  #      :headers => {:content_type => 'application/pdf'}}
   #
   #   # DELETE
   #   @client.delete 'http://example.com/resource'
@@ -58,18 +101,20 @@ module RightSupport::Net
   #   res.code                    # => 200
   #   res.headers[:content_type]  # => 'image/jpg'
   class HTTPClient
-    
-    DEFAULT_TIMEOUT       = 5
-    DEFAULT_OPEN_TIMEOUT  = 2
-    
+    # The default options for every request; can be overridden by options
+    # passed to #initialize or to the individual request methods (#get,
+    # #post, and so forth).
+    DEFAULT_OPTIONS = {
+      :timeout      => 5,
+      :open_timeout => 2,
+      :headers      => {}
+    }
+
     def initialize(defaults = {})
-      @defaults = defaults.clone
-      @defaults[:timeout]      ||= DEFAULT_TIMEOUT
-      @defaults[:open_timeout] ||= DEFAULT_OPEN_TIMEOUT
-      @defaults[:headers]      ||= {}
+      @defaults = DEFAULT_OPTIONS.merge(defaults)
     end
 
-    def get(*args)
+    def get(*args)      
       request(:get, *args)
     end
 
@@ -94,7 +139,8 @@ module RightSupport::Net
   # === Options
   # This method can accept any of the options that RestClient::Request can accept, since
   # all options are proxied through after merging in defaults, etc. Interesting options:
-  # * :payload - hash containing the request body (e.g. POST or PUT parameters)
+  # * :query - hash containing a query string (GET parameters) as a Hash or String
+  # * :payload - hash containing the request body (POST or PUT parameters) as a Hash or String
   # * :headers - hash containing additional HTTP request headers
   # * :cookies - will replace possible cookies in the :headers
   # * :user and :password - for basic auth, will be replaced by a user/password available in the url
@@ -108,13 +154,53 @@ module RightSupport::Net
   #
     def request(type, url, options={}, &block)
       options = @defaults.merge(options)
+
+      # Handle query-string option which is implemented by us, not by rest-client.
+      # (rest-client version of this, :headers={:params=>...}) but it
+      # is broken in many ways and not suitable for use!)
+      if query = options.delete(:query)
+        url = process_query_string(url, query)
+      end
+
       options.merge!(:method => type, :url => url)
 
       request_internal(options, &block)
     end
 
-    protected
-    
+    private
+
+    # Process a query-string option and append it to the URL as a properly
+    # encoded query string. The input must be either a String or Hash.
+    #
+    # === Parameters
+    # url(String):: the URL to request, including any query-string parameters
+    # query(Hash|String):: the URL params, that will be added to URL, Hash or String
+    #
+    # === Return
+    # Returns url with concated with parameters.
+    def process_query_string(url='', query={})
+      url_params = ''
+
+      if query.kind_of?(String)
+        url_params = query.gsub(/^\?/, '')
+      elsif query.kind_of?(Hash)
+        if require_succeeds?('addressable/uri')
+          uri = Addressable::URI.new
+          uri.query_values = query
+          url_params = uri.query
+        else
+          url_params = query.collect { |k, v| "#{CGI.escape(k.to_s)}=#{CGI.escape(v.to_s)}" }.join('&')
+        end
+      else
+        raise ArgumentError.new("Parameter query should be String or Hash")
+      end
+      unless (url+url_params)[/\?/]
+        url_params = '?' + url_params unless url_params.empty?
+      end
+
+      url + url_params
+    end
+
     # Wrapper around RestClient::Request.execute -- see class documentation for details.
     def request_internal(options, &block)
       if HAS_REST_CLIENT