configtoolkit 1.2.0 → 2.0.0

data/FAQ.txt

@@ -1,7 +1,8 @@
 === Why can't I use a normal Ruby Array rather than a ConstrainedArray in my configuration's specification?
 You can, but you probably don't want to.  Actually, underneath, the
-ConfigToolkit converts Array parameters into ConstrainedArray parameters with
-no constraints (<code>ConstrainedArray.new(Object, nil, nil)</code>).
+ConfigToolkit converts Array parameters into ConfigToolkit::ConstrainedArray
+parameters with no constraints
+(ConfigToolkit::ConstrainedArray.new(Object, nil, nil)).
 If you do this and therefore don't specify an element class constraint
 for the array, the ConfigToolkit will accept *any* value for an element
 of the array.  If this really is what you want, then go wild!
@@ -11,30 +12,34 @@ Every time a parameter's setter is called, the new value is validated
 against the parameter's specification in order to ensure that it is
 of the proper class (or can be converted to the proper class) *before* the
 parameter is set.  In addition, if a custom validation block was specified
-for the paramater, then this is called with the new value *before*
-the paramter is set.  Since all parameter values loaded from a file (during
-the load method) are set with parameter setters, all values loaded from
-a file receive this validation.
-
-In addition, after a load operation and before a dump operation, the
-configuration is checked in order to ensure that all required parameters have
-been set.  If an optional parameter is found that has not been set, then
-it is set with its associated default value.  Finally, the validate_all_values
-method is called if it exists.
-
-The above ensures that parameters are set with only valid values, and that
-the configuration as a whole is valid right after load and right before
+for the parameter, then this is called with the new value *before*
+the parameter is set.  Since all parameter values loaded from a file (during
+a ConfigToolkit::BaseConfig#load call) are set with parameter setters,
+all values loaded from a file receive this validation.
+
+In addition, ConfigToolkit::BaseConfig#enforce_specs is called at the end of
+ConfigToolkit::BaseConfig#load and at the start of
+ConfigToolkit::BaseConfig#dump.  This method checks that all required
+parameters have been set (a ConfigToolkit::Error will be raised if not).
+In addition, it sets any optional parameters that have not been set and that
+have default values to their associated default values.  Finally, it calls
+ConfigToolkit::BaseConfig#validate_all_values in order to enforce any
+constraints between the values of different parameters.
+
+The above checks ensure that parameters are set with only valid values, and
+that the configuration as a whole is valid right after load and right before
 dump operations.
 
 === How/why does the ConfigToolkit convert parameter values from Strings to other classes?
-Relatively complex languages like YAML support typed data.  Simple
-configuration formats like key/value pairs, however, do not support typed data;
-every parameter value is read as a String.  Since the ConfigToolkit *knows* the
+Relatively sophisticated configuration formats like YAML configuration files
+support typed data (see YAML.txt).  Simple configuration formats like
+key-value files, however, do not support typed data; every parameter value
+is read as a String (see KeyValue.txt).  Since the ConfigToolkit *knows* the
 specified class of a parameter value, however, it can attempt to convert the
 String to the specified class.  Unfortunately, Ruby does not have a standard
-way to convert Strings to other classes (whereas it does have a standard way to
-convert classes into Strings, the to_s method).  So, the ConfigToolkit does
-the best that it can and provides this conversion for the most common
+way to convert Strings to other classes (whereas it does have a standard way
+to convert classes into Strings, the +to_s+ method).  So, the ConfigToolkit
+does the best that it can and provides this conversion for the most common
 types:
 * Integers
 * Floats
@@ -48,8 +53,91 @@ arbitrary types via some kind of extension mechanism, please feel free to
 request the enhancement.
 
 === What kind of nesting is supported by the ConfigToolkit?
-ConfigToolkit::BaseConfig child classes can be nested as parameters
-within other ConfigToolkit::BaseConfig child classes arbitrarily deep.  In
+Configuration classes (ConfigToolkit::BaseConfig child classes) can be nested
+as parameters within other configuration classes arbitrarily deep (README.txt
+has an example of nesting one configuration within another).  In
 addition, one can have ConfigToolkit::ConstrainedArray parameters with
-ConfigToolkit::BaseConfig element classes (an array of configuration objects
-as a parameter).
+configuration classes (ConfigToolkit::BaseConfig child classes) as element
+classes (an array of configurations as a parameter value).
+
+=== Which is the "best" configuration file format?
+This depends, of course, on what one wants to do.  README.txt references
+documentation that provides background on each format that the ConfigToolkit
+supports.  There is no "best" format but here are some guidelines:
+* Key-value formats are primitive but support probably is present in all
+  languages (and, if not, easily can be written).  A former employer had
+  C++ and Perl code that read this format.
+* Libraries to read YAML files exist for most popular languages, and the format
+  is very flexible.
+* The Ruby configuration file format only is supported by Ruby (although, using
+  an embedded Ruby interpreter, it also probably could be read by C) but is
+  the most expressive, as arbitrary code can calculate configuration parameter
+  values when the file is loaded.
+
+YAML offers a good balance of expressiveness, features, cross-language support,
+cross-platform support, and ease of editing. I would recommend YAML files in
+all cases, except:
+* The file *ONLY* is going to be used by a Ruby application (and this *NEVER*
+  will change), and the configuration must be very dynamic, in which case Ruby
+  configuration files are appropriate.
+* The file is going to be used in a wide variety of environments by Ruby and
+  non-Ruby applications.  The non-Ruby applications in particular are going to
+  be running in environments in which a YAML library is not and never can be 
+  made available.  In this case, a key-value format is best, but the format
+  cannot use key-value extensions like Hashes, Arrays, or include directives
+  (since writing support for these in a key-value implementation in another
+   language is non-trivial)
+
+=== How can configuration parameters be documented with rdoc?
+Unfortunately, +rdoc+ cannot document configuration parameters by default.  One
+can tell it to treat ConfigToolkit::BaseConfig.add_required_param and
+ConfigToolkit::BaseConfig.add_optional_param as attribute specifiers via
+command-line options (the -A option), but this produces slightly broken
+output because +rdoc+ misinterprets the second and third (if present) arguments
+to the method as additional attributes (since +attr_reader+, for instance,
+can take multiple Symbols as arguments, each of which becomes an attribute).
+The +configtoolkit+ gem includes small patch to +rdoc+ (*only* to version 2.0.0,
+available as a gem at http://rubyforge.org/projects/rdoc/) that fixes this for
+ConfigToolkit::BaseConfig.add_required_param and
+ConfigToolkit::BaseConfig.add_optional_param:
+======+rdoc_support/rdoc.patch+:
+ --- parse_rb.rb	2008-07-14 12:25:06.000000000 -0400
+ +++ /usr/lib/ruby/gems/1.8/gems/rdoc-2.0.0/lib/rdoc/parsers/parse_rb.rb	2008-07-14 12:32:02.000000000 -0400
+ @@ -2505,6 +2505,21 @@
+        rw = @options.extra_accessor_flags[tk.name]
+      end
+  
+ +    #
+ +    # This is a temporary hack.
+ +    # At some point in the (near) future, this should be specifiable via
+ +    # rdoc's command-line options.
+ +    # 
+ +    # Unlike the attr* family of functions, only the first argument of 
+ +    # the add*param functions is an attribute; the other arguments
+ +    # constrain the attribute.
+ +    #
+ +    if((tk.name == "add_optional_param") || (tk.name == "add_required_param"))
+ +      if(args.size > 1)
+ +        args[1, args.size - 1] = nil
+ +      end
+ +    end
+ +    
+      for name in args
+        att = RDoc::Attr.new get_tkread, name, rw, comment
+        context.add_attribute att
+
+The patch should be applied to +rdoc-2.0.0/lib/rdoc/parsers/parse_rb.rb+ in the
+Ruby gems directory with the +patch+ utility.
+
+After applying this patch, +rdoc+ should register the parameters as attributes
+with this command-line option:
+ --accessor=add_optional_param=rw,add_required_param=rw
+
+As with standard attributes, +rdoc+ will display a comment block above the
+call to add a new parameter in the generated HTML along with the attribute.  The
+ConfigToolkit::KeyValueConfig documentation was produced like this.
+
+I realize that this is an unsatisfactory solution and have it on the
++configtoolkit+ TODO list (see README.txt) to craft a better one (one that
+does not require patching rdoc).  If this is particularly important to you,
+please let me know so that I can prioritize this work.

data/Hash.txt

@@ -0,0 +1,128 @@
+== LOADING AND DUMPING CONFIGURATIONS FROM AND TO HASHES:
+Reader classes extending the ConfigToolkit::Reader interface implement
+a +read+ method (see ConfigToolkit::Reader#read) that reads configuration
+from some underlying data source and returns it to ConfigToolkit::BaseConfig
+as a Hash.  Writer classes extending the ConfigToolkit::Writer interface
+implement a +write+ method (see ConfigToolkit::Writer#write)
+that receives a configuration Hash from ConfigToolkit::BaseConfig and writes it
+to some data sink.  ConfigToolkit::HashReader and ConfigToolkit::HashWriter
+implement this interface in order to allow programmers to specify and access
+configurations as Hashes directly.  ConfigToolkit::HashReader is particularly
+useful in that it allows ConfigToolkit::BaseConfig#load to be run on data
+specified programatically in a Hash.
+
+== EXAMPLE:
+======+examples/hash_example.rb+:
+ #!/usr/bin/env ruby
+ 
+ #
+ # These example programs use the 'relative' gem in order to
+ # express paths relative to __FILE__ cleanly, allowing them to be run from
+ # any directory.  In particular, they use the require_relative
+ # method to load examples/machineconfig.rb and
+ # File.expand_path_relative_to_caller in order to refer to
+ # configuration files within the examples directory.
+ #
+ require 'rubygems'
+ require 'relative'
+ require_relative 'machineconfig'
+ 
+ require 'configtoolkit/hashreader'
+ require 'configtoolkit/hashwriter'
+ 
+ #
+ # The first configuration has no containing object name.
+ #
+ config_hash = {
+   :num_cpus => 32,
+   :os => {
+     :name => "AIX",
+     :version => 5.3
+   },
+   :behind_firewall => false,
+   :contains_sensitive_data => false,
+   :addresses => [
+                  URI("http://default.designingpatterns.com"),
+                  URI("http://apple.designingpatterns.com")
+                 ]
+ }
+ reader = ConfigToolkit::HashReader.new(config_hash)
+ first_config = MachineConfig.load(reader)
+ print("The first config:\n#{first_config}\n")
+ 
+ #
+ # The second configuration has "production.www" as a containing
+ # object name and therefore the configuration is mapped to by
+ # :www in a Hash, which in turn is mapped to by :production
+ # in a containing Hash.
+ #
+ config_hash = {
+   :production => {
+     :www => {
+       :num_cpus => 64,
+       :os => {
+         :name => "Solaris",
+         :version => 10.0
+       },
+       :contains_sensitive_data => true,
+       :addresses => [
+                      URI("http://www.designingpatterns.com"),
+                      URI("http://tokyo.designingpatterns.com")
+                     ]
+     }
+   }
+ }
+ reader = ConfigToolkit::HashReader.new(config_hash)
+ second_config = MachineConfig.load(reader, "production.www")
+ print("The second config:\n#{second_config}\n")
+ 
+ #
+ # After the call to dump, the config_hash instance member of the HashWriter
+ # instance contains the Hash representation of second_config.
+ #
+ writer = ConfigToolkit::HashWriter.new()
+ second_config.dump(writer)
+ print("The Hash dumped from the second config:\n")
+ print("#{writer.config_hash.inspect}\n")
+
+This program loads two configurations, one with a containing object and one
+without a containing object, directly from Hashes.  Note how the nested
++OSConfig+ parameter was represented as a nested Hash.  Also note how the
+second configuration's containing object was represented by nesting the
+configuration's Hash within Hashes representing the containing object, one
+Hash for each member of the containing object (the Hash for +production+
+contained a Hash for +www+ that in turn contained the configuration Hash).
+When run, the program produces:
+======The output of <code>examples/hash_example.rb</code>:
+ The first config:
+ {
+     addresses              : [
+         http://default.designingpatterns.com,
+         http://apple.designingpatterns.com
+     ]
+     num_cpus               : 32
+     os                     : {
+         version: 5.3
+         name   : AIX
+     }
+     behind_firewall        : false
+     contains_sensitive_data: false
+ }
+ 
+ The second config:
+ production.www: {
+     addresses              : [
+         http://www.designingpatterns.com,
+         http://tokyo.designingpatterns.com
+     ]
+     num_cpus               : 64
+     os                     : {
+         version: 10.0
+         name   : Solaris
+     }
+     behind_firewall        : true
+     contains_sensitive_data: true
+ }
+ 
+ The Hash dumped from the second config:
+ {:production=>{:www=>{:addresses=>[#<URI::HTTP:0xb7e2f8d0 URL:http://www.designingpatterns.com>, #<URI::HTTP:0xb7e2f880 URL:http://tokyo.designingpatterns.com>], :num_cpus=>64, :os=>{:version=>10.0, :name=>"Solaris"}, :behind_firewall=>true, :contains_sensitive_data=>true}}}

data/History.txt

@@ -1,6 +1,49 @@
+=== 2.0.0 / 2008-07-03
+This version of the ConfigToolkit has a ton of enhancements.  I really would
+welcome input from the community about it!
+* IMPORTANT INTERFACE CHANGE: ConfigToolkit::BaseConfig#load_group now
+  returns a Hash mapping Symbols to config instances, not
+  a Hash mapping Strings to config instances.
+* IMPORTANT INTERFACE CHANGE: The Hash passed to ConfigToolkit::Writer#write
+  uses Symbols instead of Strings for parameter names, unless
+  the writer returns +false+ from
+  ConfigToolkit::Writer#require_symbol_parameter_names?.
+* IMPORTANT INTERFACE CHANGE: ConfigToolkit::Writer#write must
+  accept two arguments, the configuration hash and the containing
+  object name.
+* Add support for Ruby configuration files through the
+  ConfigToolkit::RubyReader class (allowing the full power of Ruby to be
+  used within configuration files; the format is very similar to that used by
+  some of the Rails configuration files).  See Ruby.txt for more details.
+* Add support for key-value configuration files through the
+  ConfigToolkit::KeyValueReader and the ConfigToolkit::KeyValueWriter.
+  See KeyValue.txt for more details.
+* Add support for loading and dumping configurations directly to and from
+  Hashes through the ConfigToolkit::HashReader and ConfigToolkit::HashWriter
+  (allowing configurations to be loaded programatically).
+  See Hash.txt for more details.
+* Provide a way for +rdoc+ to detect and generate documentation for each
+  parameter in a configuration class, just as it does for attributes (see
+  FAQ.txt for how to enable this).
+* Allow a block to be passed to ConfigToolkit::BaseConfig.new for
+  initialization of the configuration parameters.
+* Allow optional parameters to not have default values, meaning that
+  optional parameters sometimes can be absent.  A method to delete
+  the value of an optional parameter now is generated for each optional
+  parameter (+clear_param_name+).
+* A predicate method now is generated for each parameter that returns
+  whether or not a value for the parameter is present (+param_name?+).
+* Greatly improve the documentation.  The YAML configuration file format
+  is described by YAML.txt; the key-value configuration file format is
+  described by KeyValue.txt; the Ruby configuration file format is described
+  by Ruby.txt; programatically loading configurations from and dumping
+  configurations to Hashes is described in Hash.txt.
+* Many example programs exist in the +examples+ subdirectory (and are
+  integrated into the documentation).
+
 === 1.2.0 / 2008-05-20
-Add BaseConfig.load_group, which allows a hash of configs to be loaded with
-one call.  Corrected a bug in how array parameters are printed.
+Add ConfigToolkit::BaseConfig#load_group, which allows a hash of configs to 
+be loaded with one call.  Corrected a bug in how array parameters are printed.
 
 === 1.1.0 / 2008-05-14
 Finish the README.txt; this is the first production release!

data/KeyValue.txt

@@ -0,0 +1,235 @@
+== KEY-VALUE CONFIGURATION FILES:
+Key-value configuration files are quite common, although they come in a
+myriad of different formats.  +/etc/sysctl.conf+ on my system (CentOS 5.1), for
+instance, uses "=" characters to divide keys and values.
++/etc/nsswitch.conf+, on the other hand, uses ":" characters to divide keys
+and values.
+
+The ConfigToolkit accomodates this diversity by allowing
+ConfigToolkit::KeyValueReader and ConfigToolkit::KeyValueWriter
+instances to be configured themselves with the format of the file to
+be read or written.  Configuration is accomplished through the
+ConfigToolkit::KeyValueConfig class, which allows various key-value
+format characteristics to be specified (including key-value
+delimiter).  Every ConfigToolkit::KeyValueConfig parameter has a
+default value, so only parameters with values different than the
+default need to be specified when creating a new
+ConfigToolkit::KeyValueConfig instance.  A full list of the parameters
+can been seen in the ConfigToolkit::KeyValueConfig documentation.
+
+The ConfigToolkit also extends the key-value format to allow hashes
+and arrays (nested to any depth) to be values.  In addition, ConfigToolkit
+key-value files can include other key-value files through the use of include
+directives.  Include directives can handle absolute and relative (to the
+configuration file currently being processed) file paths.  Finally,
+ConfigToolkit key-value files support specifying containing objects
+in keys.  Of course, if a key-value configuration file also must be read by
+a program written in a different language, these extensions cannot be
+used.
+
+All configuration for a single parameter must be contained on a single line,
+whatever the ConfigToolkit::KeyValueConfig.  This can make
+complicated confgurations containing hashes and arrays messy; YAML
+represents such configurations more easily (see YAML.txt for details).
+The Ruby configuration file format also offers a key-value like configuration
+syntax but allows hashes and arrays to be spread across multiple lines (as
+this is allowed by the Ruby language; see Ruby.txt for details); this may 
+an acceptable alternative if only Ruby applications ever will need to read
+the file (see FAQ.txt for more discussion about which format to choose for
+a new file).
+
+== EXAMPLE:
+The following configuration file contains one configuration in a common
+key-value format (the ConfigToolkit's default key-value format, in fact):
+======+examples/key_value_example.normal1.cfg+:
+ #
+ # This file contains MachineConfig configurations
+ # (see examples/machineconfig.rb for the configuration's specification).
+ #
+ 
+ ############################################################
+ # First configuration
+ num_cpus = 32
+ os = {name => AIX, version => 5.3}
+ behind_firewall = no
+ contains_sensitive_data = no
+ addresses = [http://default.designingpatterns.com, http://apple.designingpatterns.com]
+ ############################################################
+ 
+ #
+ # Include a second configuration from a different file.
+ # Note that the path to the second file is relative to
+ # *this* file (they are in the same directory).
+ #
+ *include key_value_example.normal2.cfg
+
+It includes a second key-value configuration file containing another
+configuration (in the same key-value format):
+======+examples/key_value_example.normal2.cfg+:
+ #
+ # This file contains MachineConfig configurations
+ # (see examples/machineconfig.rb for the configuration's specification).
+ #
+ 
+ ############################################################
+ # Second configuration (nested in the production.www
+ # containing object).  Note how all of the keys are prefixed
+ # with production.www in order to indicate that they are
+ # within the production.www containing object.
+ production.www.num_cpus = 64
+ production.www.os = {name => Solaris, version => 10.0}
+ production.www.contains_sensitive_data = yes
+ production.www.addresses = [http://www.designingpatterns.com, http://tokyo.designingpatterns.com]
+ ############################################################
+
+Note that the second configuration has a +production.www+ containing object,
+and so that all keys are prefixed with +production.www+.  The first
+configuration also is represented with a different key-value
+format (the format uses ":" instead of "=" to separate keys and
+values, "<<-->>" instead of "=>" to separate hash keys and values, and
+";" instead of "#" to prefix comments) in this file:
+======+examples/key_value_example.wacky.cfg+:
+ ;
+ ; This file contains MachineConfig configurations
+ ; (see examples/machineconfig.rb for the configuration's specification).
+ ;
+ 
+ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+ ; First configuration, but with a wacky format
+ num_cpus: 32
+ os: {name <<-->> AIX, version <<-->> 5.3}
+ behind_firewall: no
+ contains_sensitive_data: no
+ addresses: [http://default.designingpatterns.com, http://apple.designingpatterns.com]
+ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+The following program loads these configurations and writes the second
+configuration in a new key-value format:
+======+examples/key_value_example.rb+:
+ #!/usr/bin/env ruby
+ 
+ #
+ # These example programs use the 'relative' gem in order to
+ # express paths relative to __FILE__ cleanly, allowing them to be run from
+ # any directory.  In particular, they use the require_relative
+ # method to load examples/machineconfig.rb and
+ # File.expand_path_relative_to_caller in order to refer to
+ # configuration files within the examples directory.
+ #
+ require 'rubygems'
+ require 'relative'
+ require_relative 'machineconfig'
+ 
+ require 'configtoolkit/keyvaluereader'
+ require 'configtoolkit/keyvaluewriter'
+ 
+ #
+ # The key-value format used by this file is the default format.
+ #
+ NORMAL_CONFIGURATION_FILE = 
+   File.expand_path_relative_to_caller("key_value_example.normal1.cfg")
+ 
+ #
+ # The first configuration has no containing object name.
+ #
+ reader = ConfigToolkit::KeyValueReader.new(NORMAL_CONFIGURATION_FILE)
+ first_config = MachineConfig.load(reader)
+ print("The first config:\n#{first_config}\n")
+ 
+ #
+ # The second configuration has "production.www" as a containing
+ # object name.  Note that the second configuration is contained in a different
+ # file that the first file includes.
+ #
+ reader = ConfigToolkit::KeyValueReader.new(NORMAL_CONFIGURATION_FILE)
+ second_config = MachineConfig.load(reader, "production.www")
+ print("The second config:\n#{second_config}\n")
+ 
+ #
+ # Configure a new, admittedly wacky, key-value format.
+ # Note how the block passed to the constructor initializes
+ # only a few of the KeyValueConfig parameters; the other
+ # parameters will be assigned their default values.
+ #
+ key_value_config = ConfigToolkit::KeyValueConfig.new() do |config|
+   config.key_value_delimiter = ":"
+   config.comment_line_prefix = ";"
+   config.hash_key_value_delimiter = "<<-->>"
+ end
+ 
+ #
+ # Read the first configuration from a file containing it in the new
+ # format.
+ #
+ WACKY_CONFIGURATION_FILE = 
+   File.expand_path_relative_to_caller("key_value_example.wacky.cfg")
+ reader = ConfigToolkit::KeyValueReader.new(WACKY_CONFIGURATION_FILE,
+                                            key_value_config)
+ first_config = MachineConfig.load(reader)
+ print("The first config sourced from a different key-value format:\n")
+ print("#{first_config}\n")
+ 
+ #
+ # Write second_config in the new format to a string stream, the contents
+ # of which will be printed.
+ #
+ string_stream = StringIO.new()
+ writer = ConfigToolkit::KeyValueWriter.new(string_stream, key_value_config)
+ second_config.dump(writer)
+ print("The second configuration dumped to a new key-value file format:\n")
+ print("#{string_stream.string}")
+
+When run, it produces:
+======The output of <code>examples/key_value_example.rb</code>:
+ The first config:
+ {
+     behind_firewall        : false
+     num_cpus               : 32
+     addresses              : [
+         http://default.designingpatterns.com,
+         http://apple.designingpatterns.com
+     ]
+     contains_sensitive_data: false
+     os                     : {
+         name   : AIX
+         version: 5.3
+     }
+ }
+ 
+ The second config:
+ production.www: {
+     behind_firewall        : true
+     num_cpus               : 64
+     addresses              : [
+         http://www.designingpatterns.com,
+         http://tokyo.designingpatterns.com
+     ]
+     contains_sensitive_data: true
+     os                     : {
+         name   : Solaris
+         version: 10.0
+     }
+ }
+ 
+ The first config sourced from a different key-value format:
+ {
+     behind_firewall        : false
+     num_cpus               : 32
+     addresses              : [
+         http://default.designingpatterns.com,
+         http://apple.designingpatterns.com
+     ]
+     contains_sensitive_data: false
+     os                     : {
+         name   : AIX
+         version: 5.3
+     }
+ }
+ 
+ The second configuration dumped to a new key-value file format:
+ production.www.behind_firewall : true
+ production.www.num_cpus : 64
+ production.www.addresses : [http://www.designingpatterns.com, http://tokyo.designingpatterns.com]
+ production.www.contains_sensitive_data : true
+ production.www.os : {name <<-->> Solaris, version <<-->> 10.0}
+