batch-kit 0.3

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: a3d2112789225de19ecd9118d3cec6a9d5b88798
+  data.tar.gz: 9e3df2077d4dd002c2238dc097643ed95258938c
+SHA512:
+  metadata.gz: 78c81cba988369e2f053885f7fd8c3eebbfb963b8c6860e15241dc557ba39578bd84411936364b7ff5ea3206cd1b2827393b2ac0a70e5c7d909dc5002a3c9433
+  data.tar.gz: a77407066d3b7168ccc1c9af9cb0ffdc090bd16210dec372eac58ad9b615046ca4f9c94f2f5e139ae8036975142ac25a4147704fd2afb775fe29aa445544723a

data/LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2013, Adam Gardiner
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,165 @@
+# Batch Kit
+
+Batch Kit is a framework for creating batch jobs, i.e. small utility programs
+that are often run in batch via a scheduler to perform some kind of repetitive
+task. Using batch-kit, you can develop batch jobs from Ruby programs that are
+simple to write, but which are:
+
+- __Robust__: Batch jobs handle any uncaught exceptions gracefully, ensuring the
+  exception is reported, resources are freed, and the job itself exits with a
+  non-zero error code.
+- __Configurable__: Batch Kit jobs make it easy to define the command-line
+  arguments the job can take, and also make it simple to use configuration files
+  in either property or YAML format.
+- __Secure__: Configuration files support encryption of sensitive items such as
+  passwords.
+- __Measured__: Batch jobs can use a database to gather statistics of runs, such
+  as the number of runs of each job, the average, minimum, and maximum duration,
+  arguments passed to the job, log output, etc.
+
+To provide these capabilities, the batch kit framework provides:
+
+- A job framework, which can be used to turn any class into a batch job. This
+  can be done either by extending the {BatchKit::Job} class, or by including the
+  {BatchKit::ActsAsJob} module. The job framework allows for new or existing job
+  and task methods to be created. Both job and task methods add 
+  {https://en.wikipedia.org/wiki/Advice_(programming) advices} that wrap
+  the logic of the method with exception handlers, as well as gathering
+  statistics about the status and duration of the task or job. These can be
+  persisted to a database for job reporting.
+
+- A facade over the Log4r and java.util.logging log frameworks, allowing
+  sophisticated logging with colour output to the console, and persistent
+  output to log files and to a database.
+
+- A configuration class ({BatchKit::Config}), which supports either property or
+  YAML-based configuration files. The {BatchKit::Config} class extends Hash,
+  providing:
+
+    + __Flexible Access__: keys are case-insensitive, can be accessed using
+      either strings or symbols, and can be accessed using BatchKit::Config#[]
+      or accessor-style methods
+    + __Support for Placeholder Variables__: substitution variables can be used
+      in the configuration file, and will be expanded either from higher-level
+      properties in the configuration tree, or left to be resolved at a later
+      time.
+    + __Encryption__: any property value can be encrypted, and will be stored in
+      memory in encrypted form, and only be decrypted when accessed explicitly.
+      Encryption is performed using AES-128 bit encryption via a separate
+      master key (which should not be stored in the same configuration file).
+
+- A resource manager class that can be used to ensure the cleanup of any
+  resource that has an acquire/release pair of methods. Use of the
+  {BatchiKit::ResourceManager} class ensures that a job releases all resources 
+  in both success and error outcomes. Support is also provided for locking
+  resources, such that concurrent or discrete jobs that share a resource can
+  coordinate their use.
+
+- Helpers: helper modules are provided for common batch tasks, such as zipping
+  files, sending emails, archiving files etc.
+
+## Example Usage
+
+The simplest way to use the batch kit framework is to create a class for your job
+that extends the {BatchKit::Job} class.
+
+```
+require 'batch-kit/job'
+
+class MyJob < BatchKit::Job
+```
+
+Next, use the {BatchKit::Configurable::ClassMethods#configure configure} method
+to add any configuration file(s) your job needs to read to load configuration
+settings that control its behaviour:
+
+```
+configure 'my_config.yaml'
+```
+
+The job configuration is now available from both the class itself, and instances
+of the class, via the {BatchKit::Job.config #config} method. Making the
+configuration available from the class allows it to be used while defining the
+class, e.g. when defining default values for command-line arguments your job
+provides.
+
+Command-line arguments are supported in batch kit jobs via the use of the
+{https://github.com/agardiner/arg-parser arg-parser} gem. This provides a
+DSL for defining various different kinds of command-line arguments:
+
+```
+positional_arg :spec, 'A path to a specification file',
+    default: config.default_spec
+flag_arg :verbose, 'Output more details during processing'
+```
+
+When your job is run, you will be able to access the values supplied on the
+command line via the job's provided {BatchKit::Job#arguments #arguments}
+accessor.
+
+```
+if arguments.verbose
+    # Do something verbosely
+end
+```
+
+We now come to the meat of our job - the tasks it is going to perform when
+run. Tasks are simply methods, but have added functionality wrapped around
+them. To define a task, there are two approaches that can be used:
+
+```
+desc 'A one-line description for my task'
+task :method1 do |param1|
+    # Define the steps this method is to perform
+    ...
+end
+
+def method2(param1, param2)
+    # Another task method
+    ...
+end
+task :method2, 'A short description for my task method'
+```
+
+Both methods are equivalent, and both leave your class with a method named
+for the task (which is then invoked like any other method) - so use
+whichever appraoch you prefer.
+
+While performing actions in your job, you can make use of the
+{BatchKit::Job#log #log} method to output logging at various levels:
+
+```
+log.config "Spec: #{arguments.spec}"
+log.info "Doing some work now"
+log.detail "Here are some more detailed messages about what we are doing"
+log.warn "Oh-oh, looks like trouble ahead"
+log.error "Oh no, an exception has occurred!"
+```
+
+Finally, we need a method that will act as the main entry point to the job. We
+define a job method much like a task, but there should only be one job method
+in our class:
+
+```
+job 'This job does XYZ' do
+    p1, p2, p3 = ...
+    method1(p1)
+    method2(p2, p3)
+end
+```
+
+As with tasks, we can use the {BatchKit::ActsAsJob#job #job} DSL method above to
+define the main entry method, or we can pass a symbol identifying an existing
+method in our class to be the job entry point.
+
+Finally, to allow our job to run when it is passed as the main program to the
+Ruby engine, we call the {Batch::Job.run run} method on our class at the end of
+our script:
+
+```
+MyJob.run
+```
+
+This instructs the batch kit framework to instantiate our job class, parse any
+command-line arguments, and then invoke our job entry method to start processing.
+

data/lib/batch-kit.rb

@@ -0,0 +1,9 @@
+# BatchKit is the namespace for the framework of functionality that includes:
+#  - BatchKit:Job
+#  - BatchKit::Config
+#  - BatchKit::Database
+#  - Core Ruby class extensions
+#  - Helper methods for common job requirements
+class BatchKit; end
+
+require 'batch-kit/job'

data/lib/batch-kit/arguments.rb

@@ -0,0 +1,57 @@
+require 'arg_parser'
+require 'color-console'
+
+
+class BatchKit
+
+    # Defines a module for adding argument parsing to a job via the arg-parser
+    # gem, and displaying help via the color-console gem.
+    module Arguments
+
+        include ArgParser::DSL
+
+
+        # Adds an arguments accessor that returns the results from parsing the
+        # command-line.
+        attr_reader :arguments
+
+
+        # Parse command-line arguments
+        def parse_arguments(args = ARGV, show_usage_on_error = true)
+            if defined?(BatchKit::Job) && self.is_a?(BatchKit::Job)
+                args_def.title ||= self.job.name.titleize
+                args_def.purpose ||= self.job.description
+            elsif defined?(BatchKit::Sequence) && self.is_a?(BatchKit::Sequence)
+                args_def.title ||= self.sequence.name.titleize
+                args_def.purpose ||= self.sequence.description
+            end
+            arg_parser = ArgParser::Parser.new(args_def)
+            @arguments = arg_parser.parse(args)
+            if @arguments == false
+                if arg_parser.show_help?
+                    arg_parser.definition.show_help(nil, Console.width || 80).each do |line|
+                        Console.puts line, :cyan
+                    end
+                    exit
+                else
+                    arg_parser.errors.each{ |error| Console.puts error, :red }
+                    if show_usage_on_error
+                        arg_parser.definition.show_usage(nil, Console.width || 80).each do |line|
+                            Console.puts line, :yellow
+                        end
+                    end
+                    exit(99)
+                end
+            end
+            @arguments
+        end
+
+
+        # Add class methods when module is included
+        def self.included(base)
+            base.extend(ClassMethods)
+        end
+
+    end
+
+end

data/lib/batch-kit/config.rb

@@ -0,0 +1,517 @@
+class BatchKit
+
+    # Defines a class for managing configuration properties; essentially, this
+    # is a hash that is case and Strng/Symbol insensitive with respect to keys;
+    # this means a value can be retrieved using any mix of case using either a
+    # String or Symbol as the lookup key.
+    #
+    # In addition, there are some further conveniences added on:
+    # - Items can be accessed by either [] (using a String or Symbol key) or as
+    #   methods on the Config object, i.e. the following are all equivalent:
+    #
+    #     config['Foo']
+    #     config[:foo]
+    #     config.foo
+    #
+    # - Contents can be loaded from:
+    #   - an existing Hash object
+    #   - a properties file using [Section] and KEY=VALUE syntax
+    #   - a YAML file
+    # - String values can contain placeholder variables that will be replaced
+    #   when the item is added to the Config collection. Placeholder variables
+    #   are denoted by ${<variable>} or %{<variable>} syntax, where <variable>
+    #   is the name of another configuration variable or a key in a supplied
+    #   expansion properties hash.
+    # - Support for encrypted values. These are decrypted on the fly, provided
+    #   a decryption key has been set on the Config object.
+    #
+    # Internally, the case and String/Symbol insensitivity is managed by
+    # maintaining a second Hash that converts the lower-cased symbol value
+    # of all keys to the actual keys used to store the object. When looking up
+    # a value, we use this lookup Hash to find the actual key used, and then
+    # lookup the value.
+    #
+    # @note As this Config object is case and String/Symbol insensitive,
+    #   different case and type keys that convert to the same lookup key are
+    #   considered the same. The practical implication is that you can't have
+    #   two different values in this Config object where the keys differ only
+    #   in case and/or String/Symbol class.
+    class Config < Hash
+
+        # Create a new Config object, and initialize it from the specified
+        # +file+.
+        #
+        # @param file [String] A path to a properties or YAML file to load.
+        # @param props [Hash, Config] An optional Hash (or Config) object to
+        #   seed this Config object with.
+        # @param options [Hash] An options hash.
+        # @option options [Boolean] :raise_on_unknown_var Whether to raise an
+        #   error if an unrecognised placeholder variable is encountered in the
+        #   file.
+        # @option options [Boolean] :use_erb If true, the contents of +file+
+        #   is first run through ERB.
+        # @option options [Binding] :binding The binding to use when evaluating
+        #   expressions in ERB
+        # @return [Config] A new Config object populated from +file+ and
+        #   +props+, where placeholder variables have been expanded.
+        def self.load(file, props = nil, options = {})
+            cfg = self.new(props, options)
+            cfg.load(file, options)
+            cfg
+        end
+
+
+        # Converts a +str+ in the form of a properties file into a Hash.
+        def self.properties_to_hash(str)
+            hsh = props = {}
+            str.each_line do |line|
+                line.chomp!
+                if match = /^\s*\[([A-Za-z0-9_ ]+)\]\s*$/.match(line)
+                    # Section heading
+                    props = hsh[match[1]] = {}
+                elsif match = /^\s*([A-Za-z0-9_\.]+)\s*=\s*([^#]+)/.match(line)
+                    # Property setting
+                    val = match[2]
+                    props[match[1]] = case val
+                    when /^\d+$/ then val.to_i
+                    when /^\d*\.\d+$/ then val.to_f
+                    when /^:/ then val.intern
+                    when /false/i then false
+                    when /true/i then true
+                    else val
+                    end
+                end
+            end
+            hsh
+        end
+
+
+        # Expand any ${<variable>} or %{<variable>} placeholders in +str+ from
+        # either the supplied +props+ hash or the system environment variables.
+        # The props hash is assumed to contain string or symbol keys matching the
+        # variable name between ${ and } (or %{ and }) delimiters.
+        # If no match is found in the supplied props hash or the environment, the
+        # default behaviour returns the string with the placeholder variable still in
+        # place, but this behaviour can be overridden to cause an exception to be
+        # raised if desired.
+        #
+        # @param str [String] A String to be expanded from 0 or more placeholder
+        #   substitutions
+        # @param properties [Hash, Array<Hash>] A properties Hash or array of Hashes
+        #   from which placeholder variable values can be looked up.
+        # @param raise_on_unknown_var [Boolean] Whether or not an exception should
+        #   be raised if no property is found for a placeholder expression. If false,
+        #   unrecognised placeholder variables are left in the returned string.
+        # @return [String] A new string with placeholder variables replaced by
+        #   the values in +props+.
+        def self.expand_placeholders(str, properties, raise_on_unknown_var = false)
+            chain = properties.is_a?(Hash) ? [properties] : properties.reverse
+            str.gsub(/(?:[$%])\{([a-zA-Z0-9_]+)\}/) do
+                case
+                when src = chain.find{ |props| props.has_key?($1) ||
+                                       props.has_key?($1.intern) ||
+                                       props.has_key?($1.downcase.intern) }
+                    src[$1] || src[$1.intern] || src[$1.downcase.intern]
+                when ENV[$1] then ENV[$1]
+                when raise_on_unknown_var
+                    raise KeyError, "No value supplied for placeholder variable '#{$&}'"
+                else
+                    $&
+                end
+            end
+        end
+
+
+        # Create a Config object, optionally initialized from +hsh+.
+        #
+        # @param hsh [Hash] An optional Hash to seed this Config object with.
+        # @param options [Hash] An options hash.
+        def initialize(hsh = nil, options = {})
+            super(nil)
+            @lookup_keys = {}
+            @decryption_key = nil
+            merge!(hsh, options) if hsh
+        end
+
+
+        # Read a properties or YAML file at the path specified in +path+, and
+        # load the contents to this Config object.
+        #
+        # @param path [String] The path to the properties or YAML file to be
+        #   loaded.
+        # @param options [Hash] An options hash.
+        # @option options [Boolean] @raise_on_unknown_var Whether to raise an
+        #   error if an unrecognised placeholder variable is encountered in the
+        #   file.
+        def load(path, options = {})
+            props = case File.extname(path)
+            when /\.yaml/i then self.load_yaml(path, options)
+            else self.load_properties(path, options)
+            end
+        end
+
+
+        # Process a property file, returning its contents as a Hash.
+        # Only lines of the form KEY=VALUE are processed, and # indicates the start
+        # of a comment. Property files can contain sections, denoted by [SECTION].
+        #
+        # @example
+        #   If a properties file contains the following:
+        #
+        #     FOO=Bar             # This is a comment
+        #     BAR=${FOO}\Baz
+        #
+        #     [BAT]
+        #     Car=Ford
+        #
+        #   Then we would return a Config object containing the following:
+        #
+        #     {'FOO' => 'Bar', 'BAR' => 'Bar\Baz', 'BAT' => {'Car' => 'Ford'}}
+        #
+        #   This config content could be accessed via #[] or as properties of
+        #   the Config object, e.g.
+        #
+        #     cfg[:foo]    # => 'Bar'
+        #     cfg.bar      # => 'Bar\Baz'
+        #     cfg.bat.car  # => 'Ford'
+        #
+        # @param prop_file [String] A path to the properties file to be parsed.
+        # @param options [Hash] An options hash.
+        # @option options [Boolean] :use_erb If true, the contents of +prop_file+
+        #   is first passed through ERB before being processed. This allows for
+        #   the use of <% %> and <%= %> directives in +prop_file+. The binding
+        #   passed to ERB is the value of any :binding option specified, or else
+        #   this Config object. If not specified, ERB is used if the file is
+        #   found to contain the string '<%'.
+        # @option options [Binding] :binding The binding for ERB to use when
+        #   processing expressions. Defaults to this Config instance if not
+        #   specified.
+        # @return [Hash] The parsed contents of the file as a Hash.
+        def load_properties(prop_file, options = {})
+            str = read_file(prop_file, options)
+            hsh = self.class.properties_to_hash(str)
+            self.merge!(hsh, options)
+        end
+
+
+        # Load the YAML file at +yaml_file+.
+        #
+        # @param yaml_file [String] A path to a YAML file to be loaded.
+        # @param options [Hash] An options hash.
+        # @option options [Boolean] :use_erb If true, the contents of +yaml_file+
+        #   are run through ERB before being parsed as YAML. This allows for use
+        #   of <% %> and <%= %> directives in +yaml_file+. The binding passed to
+        #   ERB is the value of any :binding option specified, or else this
+        #   Config object. If not specified, ERB is used if the file is found to
+        #   contain the string '<%'.
+        # @option options [Binding] :binding The binding for ERB to use when
+        #   processing expressions. Defaults to this Config instance if not
+        #   specified.
+        # @return [Object] The results of parsing the YAML contents of +yaml_file+.
+        def load_yaml(yaml_file, options = {})
+            require 'yaml'
+            str = read_file(yaml_file, options)
+            yaml = YAML.load(str)
+            self.merge!(yaml, options)
+        end
+
+
+        # Save this config object as a YAML file at +yaml_file+.
+        #
+        # @param yaml_file [String] A path to the YAML file to be saved.
+        # @param options [Hash] An options hash.
+        def save_yaml(yaml_file, options = {})
+            require 'yaml'
+            str = self.to_yaml
+            File.open(yaml_file, 'wb'){ |f| f.puts(str) }
+        end
+
+
+        # Ensure that Config objects are saved as normal hashes when writing
+        # YAML.
+        def encode_with(coder)
+            coder.represent_map nil, self
+        end
+
+
+        # Merge the contents of the specified +hsh+ into this Config object.
+        #
+        # @param hsh [Hash] The Hash object to merge into this Config object.
+        # @param options [Hash] An options hash.
+        # @option options [Boolean] :raise_on_unknown_var Whether to raise an
+        #   exception if an unrecognised placeholder variable is encountered in
+        #   +hsh+.
+        def merge!(hsh, options = {})
+            if hsh && !hsh.is_a?(Hash)
+                raise ArgumentError, "Only Hash objects can be merged into Config (got #{hsh.class.name})"
+            end
+            hsh && hsh.each do |key, val|
+                self[key] = convert_val(val, options[:raise_on_unknown_var])
+            end
+            if hsh.is_a?(Config)
+                @decryption_key = hsh.instance_variable_get(:@decryption_key) unless @decryption_key
+            end
+            self
+        end
+
+
+        # Merge the contents of the specified +hsh+ into a new Config object.
+        #
+        # @param hsh [Hash] The Hash object to merge with this Config object.
+        # @param options [Hash] An options hash.
+        # @option options [Boolean] @raise_on_unknown_var Whether to raise an
+        #   error if an unrecognised placeholder variable is encountered in the
+        #   file.
+        # @return A new Config object with the combined contents of this Config
+        #   object plus the contents of +hsh+.
+        def merge(hsh, options = {})
+            cfg = self.dup
+            cfg.merge!(hsh, options)
+            cfg
+        end
+
+
+        # If set, encrypted strings (only) will be decrypted when accessed via
+        # #[] or #method_missing (for property-like access, e.g. +cfg.password+).
+        #
+        # @param key [String] The master encryption key used to encrypt sensitive
+        #   values in this Config object.
+        def decryption_key=(key)
+            require_relative 'encryption'
+            self.each do |_, val|
+                val.decryption_key = key if val.is_a?(Config)
+            end
+            @decryption_key = key
+        end
+        alias_method :encryption_key=, :decryption_key=
+
+
+        # Recursively encrypts the values of all keys in this Config object that
+        # match +key_pat+.
+        #
+        # Note: +key_pat+ will be compared against the standardised key values of
+        # each object (i.e. lowercase, with spaces converted to _).
+        #
+        # @param key_pat [Regexp|String] A regular expression to be used to identify
+        #   the keys that should be encrypted, e.g. /password/ would encrypt all
+        #   values that have "password" in their key.
+        # @param master_key [String] The master key that should be used when
+        #   encrypting. If not specified, uses the current value of the
+        #   +decryption_key+ set for this Config object.
+        def encrypt(key_pat, master_key = @decryption_key)
+            key_pat = Regexp.new(key_pat, true) if key_pat.is_a?(String)
+            raise ArgumentError, "key_pat must be a Regexp or String" unless key_pat.is_a?(Regexp)
+            raise ArgumentError, "No master key has been set or passed" unless master_key
+            require_relative 'encryption'
+            self.each do |key, val|
+                if Config === val
+                    val.encrypt(key_pat, master_key)
+                else
+                    if @decryption_key && val.is_a?(String) && val =~ /!AES:([a-zA-Z0-9\/+=]+)!/
+                        # Decrypt using old master key
+                        val = self[key]
+                        self[key] = val
+                    end
+                    if val.is_a?(String) && convert_key(key) =~ key_pat
+                        self[key] = "!AES:#{Encryption.encrypt(master_key, val).strip}!"
+                    end
+                end
+            end
+            @decryption_key = master_key
+        end
+
+
+        # Override #[] to be agnostic as to the case of the key, and whether it
+        # is a String or a Symbol.
+        def [](key)
+            key = @lookup_keys[convert_key(key)]
+            val = super(key)
+            if @decryption_key && val.is_a?(String) && val =~ /!AES:([a-zA-Z0-9\/+=]+)!/
+                begin
+                    val = Encryption.decrypt(@decryption_key, $1)
+                rescue Exception => ex
+                    raise "An error occurred while decrypting the value for key '#{key}': #{ex.message}"
+                end
+            end
+            val
+        end
+
+
+        # Override #[]= to be agnostic as to the case of the key, and whether it
+        # is a String or a Symbol.
+        def []=(key, val)
+            std_key = convert_key(key)
+            if @lookup_keys[std_key] != key
+                delete(key)
+                @lookup_keys[std_key] = key
+            end
+            super key, val
+        end
+
+
+        # Override #delete to be agnostic as to the case of the key, and whether
+        # it is a String or a Symbol.
+        def delete(key)
+            key = @lookup_keys.delete(convert_key(key))
+            super key
+        end
+
+
+        # Override #has_key? to be agnostic as to the case of the key, and whether
+        # it is a String or a Symbol.
+        def has_key?(key)
+            key = @lookup_keys[convert_key(key)]
+            super key
+        end
+        alias_method :include?, :has_key?
+
+
+        # Override #fetch to be agnostic as to the case of the key, and whether it
+        # is a String or a Symbol.
+        def fetch(key, *rest)
+            key = @lookup_keys[convert_key(key)] || key
+            super
+        end
+
+
+        # Override #clone to also clone contents of @lookup_keys.
+        def clone
+            copy = super
+            copy.instance_variable_set(:@lookup_keys, @lookup_keys.clone)
+            copy
+        end
+
+
+        # Override #dup to also clone contents of @lookup_keys.
+        def dup
+            copy = super
+            copy.instance_variable_set(:@lookup_keys, @lookup_keys.dup)
+            copy
+        end
+
+
+        # Override Hash core extension method #to_cfg and just return self.
+        def to_cfg
+            self
+        end
+
+
+        # Override method_missing to respond to method calls with the value of the
+        # property, if this Config object contains a property of the same name.
+        def method_missing(name, *args)
+            if name =~ /^(.+)\?$/
+                has_key?($1)
+            elsif has_key?(name)
+                self[name]
+            elsif has_key?(name.to_s.gsub('_', ''))
+                self[name.to_s.gsub('_', '')]
+            elsif name =~ /^(.+)=$/
+                self[$1]= args.first
+            else
+                raise ArgumentError, "No configuration entry for key '#{name}'"
+            end
+        end
+
+
+        # Override respond_to? to indicate which methods we will accept.
+        def respond_to?(name)
+            if name =~ /^(.+)\?$/
+                has_key?($1)
+            elsif has_key?(name)
+                true
+            elsif has_key?(name.to_s.gsub('_', ''))
+                true
+            elsif name =~ /^(.+)=$/
+                true
+            else
+                super
+            end
+        end
+
+
+        # Expand any ${<variable>} or %{<variable>} placeholders in +str+ from
+        # this Config object or the system environment variables.
+        # This Config object is assumed to contain string or symbol keys matching
+        # the variable name between ${ and } (or %{ and }) delimiters.
+        # If no match is found in the supplied props hash or the environment, the
+        # default behaviour is to raise an exception, but this can be overriden
+        # to leave the placeholder variable still in place if desired.
+        #
+        # @param str [String] A String to be expanded from 0 or more placeholder
+        #   substitutions
+        # @param raise_on_unknown_var [Boolean] Whether or not an exception should
+        #   be raised if no property is found for a placeholder expression. If false,
+        #   unrecognised placeholder variables are left in the returned string.
+        # @return [String] A new string with placeholder variables replaced by
+        #   the values in +props+.
+        def expand_placeholders(str, raise_on_unknown_var = true)
+            self.class.expand_placeholders(str, self, raise_on_unknown_var)
+        end
+
+
+        # Reads a template file at +template_name+, and expands any substitution
+        # variable placeholder strings from this Config object.
+        #
+        # @param template_name [String] The path to the template file containing
+        #   placeholder variables to expand from this Config object.
+        # @return [String] The contents of the template file with placeholder
+        #   variables replaced by the content of this Config object.
+        def read_template(template_name, raise_on_unknown_var = true)
+            template = IO.read(template_name)
+            expand_placeholders(template, raise_on_unknown_var)
+        end
+
+
+        private
+
+
+        # Reads the contents of +file+ into a String. The +file+ is passed
+        # through ERB if the :use_erb option is true, or if the options does
+        # not contain the :use_erb key and the file does contain the string
+        # '<%'.
+        def read_file(file, options)
+            str = IO.read(file)
+            if (options.has_key?(:use_erb) && options[:use_erb]) || str =~ /<%/
+                require 'erb'
+                str = ERB.new(str).result(options[:binding] || binding)
+            end
+            str
+        end
+
+
+        # Convert the supplied key to a lower-case symbol representation, which
+        # is the key to the @lookup_keys hash.
+        def convert_key(key)
+            key.to_s.downcase.gsub(' ', '_').intern
+        end
+
+
+        # Convert a value before merging it into the Config. This consists of
+        # these tasks:
+        #   - Converting Hashes to Config objects
+        #   - Propogating decryption keys to child Config objects
+        #   - Expanding placeholder variables in strings
+        def convert_val(val, raise_on_unknown_var, parents = [self])
+            case val
+            when Config then val
+            when Hash
+                cfg = Config.new
+                cfg.instance_variable_set(:@decryption_key, @decryption_key)
+                new_parents = parents.clone
+                new_parents << cfg
+                val.each do |k, v|
+                    cfg[k] = convert_val(v, raise_on_unknown_var, new_parents)
+                end
+                cfg
+            when Array
+                val.map{ |v| convert_val(v, raise_on_unknown_var, parents) }
+            when /[$%]\{[a-zA-Z0-9_]+\}/
+                self.class.expand_placeholders(val, parents, raise_on_unknown_var)
+            else val
+            end
+        end
+
+    end
+
+end
+