lorj 1.0.1 → 1.0.2

data/lib/overview.md

@@ -0,0 +1,120 @@
+As described by the [concept](concept_md.html), lorj is a combination of 4 blocks:
+
+* The main of your application
+* a Config system
+* a Generic process
+* a controller
+
+Those blocks communicate through lorj Core, except the config system which can be
+accessible from anywhere.
+
+
+# Lorj configuration
+To run your application with lorj, you will certainly need to configure lorj
+()Lorj::Core), like the application name, the application path, log file name, etc...
+
+For details about the lorj configuration, please read documentation on PrcLib.
+
+
+# Config system
+About the config system, lorj provides 2 differents predefined configuration classes.
+
+* Lorj::Config  - Basic config systems. Data are retrieved in **application defaults**, then **local config**, then **runtime**.
+* Lorj::Account - Based on Lorj::Config, it introduces an **account** file, between **local config** and **runtime**.
+
+Those 2 classes are based on a generic PRC::BaseConfig(or SectionConfig), and PRC::CoreConfig.
+PRC::CoreConfig introduce a way to get data from different source, organized by layers.
+It automates the notion of, 'if not there here, go to the next one, until default'
+A config layer, can be a instant memory data, created on demand, or any kind of
+local yaml file, loaded at startup or runtime.
+
+For details, see PRC::CoreConfig
+
+Lorj has implemented some classes, predefining:
+For Lorj::Config : 3 layers
+* Application defaults (defaults.yaml). The file is loaded thanks to Lorj PrcLib configuration
+  This layer is used to predefine lorj data model, to setup the account file.
+* local data. Refer to a user local application config file.
+* runtime. A lorj process can get/set any data on needs.
+
+Details available from Lorj::Config
+
+For Lorj::Account: 4 layers
+* Application defaults (defaults.yaml). The file is loaded thanks to Lorj PrcLib configuration
+  This layer is used to predefine lorj data model, to setup the account file.
+* local data. Refer to a user local application config file.
+* account. Depending on needs, lorj can setup an account file to preload some data
+  to help the process or the dedicated controller to do the work.
+  Ex: with a cloud process, a cloud controller will need some credentials. lorj setup
+      will be able to ask those information to the user, save them to the account file
+      and Lorj::Account will be able to load them for process execution.
+* runtime. A lorj process can get/set any data on needs.
+
+Details available from Lorj::Account, and BaseDefinition.setup
+
+
+# Generic process
+
+The generic Process is your application process.
+
+To simplify the vision, imagine a human as an application
+Then the application process is the brain.
+
+But to keep the power of lorj, a process must respect some rules:
+* A process defines a data model which is known only by it self.
+* A process should never have to deal with a controller data model.
+  lorj provides a mapping feature to map process data with controller data.
+* A process should never deal with how a controller manipulate data.
+  A controller can have his own independant process
+  Ex: If your process deals with booting a server, the process do not take care on
+  how to ensure the network is already configured to access internet.
+
+For details, see Lorj::BaseProcess
+
+
+# Controller
+
+A controller enhance a generic Process with how to really execute what the
+process define to do.
+
+In the simple human application vision, a controller is the human body.
+It gives you the way to walk, for example.
+
+You can consider that the 'brain' process can be enhanced in 2 differents areas:
+* Do and execute a task:
+  This is like, human muscles
+* how to do a controller task that the brain do not care of.
+  This is like how to automate several muscles to execute a simple task 'walk'
+
+For details, see Lorj::BaseController
+
+
+# enhanced logger system
+
+There is nothing clever here. Globally, you should be able to redefine the lorj
+logging system by any thing, if you redefine PrcLib logging messages.
+This said, Lorj has implemented a logging system based on logger. It helps to
+save any debug info to a rotated log file system, even if the data is not printed out
+to the default output.
+
+This helps in debugging the lorj application.
+
+
+# Implementation examples
+
+##Students examples:
+The main lorj README has a learn by examples documentation. This could help to start.
+
+## Forj cli
+forj cli is the first implementation of lorj.
+
+Forj cli is a command line interface to start a DevOps solution in the cloud.
+As forj has been developped with cloud agnostics in mind, the cli has implemented
+a Cloud process to start the first box called 'Maestro'. And lorj help forj cli
+to boot maestro on any kind of cloud, as soon as a controller exists.
+
+It ensures that everything is configured on the cloud provider to boot up a box
+with all features needed, like network connected to internet, image available,
+NAT preconfigured, with port open, etc...
+
+See [forj cli gem](https://rubygems.org/gems/forj)

data/lib/prc.rb

@@ -15,26 +15,96 @@
 require 'fileutils'
 require 'logger'
 
-# General lorj library. Used as library data configuration
+#
+# PrcLib module
+#
+# This module helps to configure the lorj library.
+# It implements also a Logging class based on logger.
+#
+# For details about this class capabilities, see PrcLib::Logging
+#
 # List of possible library settings:
-# PrcLib.log          : PrcLib::Logging object. Used internally by PrcLib
-#                       logging system.
-#                       This object is automatically created as soon as
-#                       a message is printed out
-# PrcLib.core_level   : lorj debug level. from 0 to 5.
-# PrcLib.pdata_path   : Define the private data local directory. Usually used
-#                       for any private keys, passwords, etc...
-#                       By default: ~/.config/<app_name>
-# PrcLib.data_path    : Define the data local directory.
-#                       By default: ~/.<app_name>
-# PrcLib.app_name     : Define the application name. By default 'lorj'
-# PrcLib.app_defaults : Used by Lorj::Config to load application default data.
-#                       By default nil.
-# PrcLib.log_file     : Define the log file name used.
-#                       By default, defined as ~/.<app_name>/<app_name>.log
-# PrcLib.level        : logger level used.
-#                       Can be set at runtime, with PrcLib.set_level
-# PrcLib.model        : Model loaded.
+# - PrcLib.log
+#
+#   PrcLib::Logging object. Used internally by PrcLib logging system.
+#
+#   This object is automatically created as soon as a message is printed out
+# - PrcLib.core_level
+#
+#   Initialize lorj debug level. from 0 to 5.
+#
+#   ex:
+#
+#    PrcLib.core_level = 4
+# - PrcLib.pdata_path
+#
+#   Define the private data local directory. Usually used
+#   for any private keys, passwords, etc...
+#
+#   By default: ~/.config/<app_name>
+#
+#   ex:
+#
+#    PrcLib.pdata_path = File.join('~', '.private_myapp')
+# - PrcLib.data_path
+#
+#   Define the data local directory.
+#
+#   By default: ~/.<app_name>
+#
+#   ex:
+#
+#    PrcLib.data_path = File.join('/etc', 'myapp')
+#
+# - PrcLib.app_name
+#
+#   Define the application name. By default 'lorj'.
+#   By default, this setting configure PrcLib.data_path and PrcLib.pdata_path
+#   automatically, except if you set it before.
+#
+#   ex:
+#
+#    PrcLib.app_name = 'myapp'
+# - PrcLib.app_defaults
+#
+#   Used by Lorj::Config to identify application defaults and your application
+#   data model data.
+#
+#   By default nil.
+#   Ex:
+#
+#    puts PrcLib.app_defaults[:data] # To get value of the predefined :data key.
+# - PrcLib.log_file
+#
+#   Define the log file name used.
+#
+#   By default, defined as ~/.<app_name>/<app_name>.log
+# - PrcLib.level
+#   logger level used. It can be updated at runtime.
+#
+#   Ex:
+#
+#    PrcLib.level = Logger::FATAL
+# - PrcLib.model
+#
+#   Model loaded.
+#
+# - PrcLib.log_file
+#
+#   Initialize a log file name instead of default one.
+#
+#   Ex:
+#
+#    PrcLib.log_file = "mylog.file.log"
+#
+# - PrcLib.controller_path
+#
+#   Provides the default controller path.
+#
+# - PrcLib.process_path
+#
+#   Provides the default process path.
+#
 module PrcLib
   # Check if dir exists and is fully accessible (rwx)
   def self.dir_exists?(path)
@@ -71,8 +141,7 @@ module PrcLib
   # Define module data for lorj library configuration
   class << self
     attr_accessor :log, :core_level
-    attr_reader :pdata_path, :data_path, :app_defaults, :log_file, :level,
-                :model
+    attr_reader :pdata_path, :data_path, :app_defaults, :log_file, :level
   end
 
   module_function
@@ -83,7 +152,7 @@ module PrcLib
   end
 
   def app_name
-    @app_name = 'Lorj' unless @app_name
+    self.app_name = 'Lorj' unless @app_name
     @app_name
   end
 
@@ -109,10 +178,14 @@ module PrcLib
     PrcLib.ensure_dir_exists(@data_path)
   end
 
-  # TODO: Low. Be able to support multiple model.
   def app_name=(v)
     @app_name = v unless @app_name
-    @model = Lorj::Model.new
+  end
+
+  # TODO: Low. Be able to support multiple model.
+  def model
+    @model = Lorj::Model.new if @model.nil?
+    @model
   end
 
   # TODO: Support for several defaults, depending on controllers loaded.

data/lib/prc_core_config.rb

@@ -212,15 +212,16 @@ module PRC
 
     def layers
       result = []
-      @config_layers.each { | layer | result << layer[:name] }
+      @config_layers.each { |layer| result << layer[:name] }
       result
     end
 
     def to_s
       data = "Configs list ordered:\n"
-      @config_layers.each do | layer |
+      @config_layers.each do |layer|
         data += format("---- Config : %s ----\noptions: ", layer[:name])
 
+        data += 'predefined, ' if layer[:init].is_a?(TrueClass)
         if layer[:set]
           data += 'data RW '
         else
@@ -333,7 +334,7 @@ module PRC
 
       return nil if keys.length == 0 || keys[0].nil? || config_layers[0].nil?
 
-      config_layers.each_index do | index |
+      config_layers.each_index do |index|
         config = @config_layers[index][:config]
 
         data_options = options.clone
@@ -381,7 +382,7 @@ module PRC
 
     def _do_where?(config_layers, keys, options, data_opts)
       layer_indexes = []
-      config_layers.each_index do | index |
+      config_layers.each_index do |index|
         config = config_layers[index][:config]
 
         data_options = options.clone
@@ -421,7 +422,7 @@ module PRC
 
       return nil if keys.length == 0 || keys[0].nil? || config_layers[0].nil?
 
-      config_layers.each_index do | layer_index |
+      config_layers.each_index do |layer_index|
         layer = config_layers[layer_index]
 
         data_options = options.clone
@@ -650,12 +651,88 @@ module PRC
       initialize_layers(config_layers)
     end
 
+    # This function add a config layer at runtime.
+    # The new layer added at runtime, can be removed at runtime
+    # with layer_remove
+    # The name MUST be different than other existing config layer names
+    #
+    # *Args*
+    # - +options+ : Hash data
+    #   - :name     : Required. Name of the layer to add
+    #   - :index    : Config position to use. 0 is the default. 0 is the first
+    #                 Config layer use by get.
+    #   - :config   : A Config instance of class type PRC::BaseConfig
+    #   - :set      : Boolean. True if is authorized to set a variable.
+    #   - :load     : Boolean. True if is authorized to load from a file.
+    #   - :save     : Boolean. True if is authorized to save to a file.
+    #   - :file_set : Boolean. True if is authorized to change the file name.
+    #
+    # *returns*
+    # - true if layer is added.
+    # OR
+    # - nil : if layer name already exist
+    def layer_add(options)
+      layer = CoreConfig.define_layer(options)
+
+      layer[:init] = false # Runtime layer
+
+      index = 0
+      index = options[:index] if options[:index].is_a?(Fixnum)
+      names = []
+      @config_layers.each { |alayer| names << alayer[:name] }
+
+      return nil if names.include?(layer[:name])
+      @config_layers.insert(index, layer)
+      true
+    end
+
+    # Function to remove a runtime layer.
+    # You cannot remove a predefined layer, created during CoreConfig
+    # instanciation.
+    # *Args*
+    # - +options+ : Hash data
+    #   - +:name+ : Name of the layer to remove.
+    #   - +:index+: Index of the layer to remove.
+    #
+    # At least, :name or :index is required.
+    # If both; :name and :index are set, :name is used.
+    # *return*
+    # - true if layer name is removed.
+    # OR
+    # - nil : if not found or invalid.
+    def layer_remove(options)
+      index = layer_index(options[:name])
+      index = options[:index] if index.nil?
+
+      return nil if index.nil?
+
+      layer = @config_layers[index]
+
+      return nil if layer.nil? || layer[:init]
+
+      @config_layers.delete_at(index)
+      true
+    end
+
+    # Function to define layer options.
+    # By default, :set is true and :config is attached to a new PRC::BaseConfig
+    # instance.
+    #
+    # Supported options:
+    #   - :config   : optional. See `Defining Config layer instance` for details
+    #   - :name     : required. String. Name of the config layer.
+    #                 Warning! unique name on layers is no tested.
+    #   - :set      : boolean. True if authorized. Default is True.
+    #   - :load     : boolean. True if authorized. Default is False.
+    #   - :save     : boolean. True if authorized. Default is False.
+    #   - :file_set : boolean. True if authorized to update a filename.
+    #                 Default is False.
     def self.define_layer(options = {})
       attributes = [:name, :config, :set, :load, :save, :file_set]
 
       layer = {}
 
-      attributes.each do | attribute |
+      attributes.each do |attribute|
         if options.key?(attribute)
           layer[attribute] = options[attribute]
         else
@@ -674,6 +751,48 @@ module PRC
       layer
     end
 
+    # layer_indexes function
+    #
+    # * *Args*
+    # - +:name+ : layer to identify.
+    #
+    # * *Returns*
+    #   first index found or nil.
+    #
+    def layer_indexes(names)
+      names = [names] if names.is_a?(String)
+      return nil unless names.is_a?(Array)
+
+      layers = []
+
+      names.each do |name|
+        index = layer_index(name)
+        layers << index unless index.nil?
+      end
+      return layers if layers.length > 0
+      nil
+    end
+
+    # layer_index function
+    #
+    # * *Args*
+    # - +:name+ : layer to identify.
+    #
+    # * *Returns*
+    #   first index found or nil.
+    #
+    def layer_index(name)
+      return nil unless name.is_a?(String)
+      return nil if @config_layers.nil?
+
+      @config_layers.each_index do |index|
+        return index if @config_layers[index][:name] == name
+      end
+      nil
+    end
+
+    private
+
     # Function to initialize Config layers.
     #
     # *Args*
@@ -705,7 +824,7 @@ module PRC
     def initialize_layers(config_layers = nil)
       @config_layers = []
 
-      config_layers.each do | layer |
+      config_layers.each do |layer|
         next unless layer.is_a?(Hash) && layer.key?(:config) &&
                     layer[:config].is_a?(BaseConfig)
         next unless layer[:name].is_a?(String)
@@ -714,55 +833,18 @@ module PRC
       @config_layers.reverse!
     end
 
+    # Function to initialize a predefined layer
+    # Used internally by initialize_layers.
     def initialize_layer(layer)
       newlayer = { :config => layer[:config], :name => layer[:name] }
       newlayer[:set] = layer[:set].boolean? ? layer[:set] : true
       newlayer[:load] = layer[:load].boolean? ? layer[:load] : false
       newlayer[:save] = layer[:save].boolean? ? layer[:save] : false
       newlayer[:file_set] = layer[:file_set].boolean? ? layer[:file_set] : false
+      newlayer[:init] = true
       newlayer
     end
 
-    # layer_indexes function
-    #
-    # * *Args*
-    # - +:name+ : layer to identify.
-    #
-    # * *Returns*
-    #   first index found or nil.
-    #
-    def _layer_indexes(names)
-      names = [names] if names.is_a?(String)
-      return nil unless names.is_a?(Array)
-
-      layers = []
-
-      names.each do | name |
-        index = layer_index(name)
-        layers << index unless index.nil?
-      end
-      return layers if layers.length > 0
-      nil
-    end
-
-    # layer_index function
-    #
-    # * *Args*
-    # - +:name+ : layer to identify.
-    #
-    # * *Returns*
-    #   first index found or nil.
-    #
-    def layer_index(name)
-      return nil unless name.is_a?(String)
-      return nil if @config_layers.nil?
-
-      @config_layers.each_index do | index |
-        return index if @config_layers[index][:name] == name
-      end
-      nil
-    end
-
     # Check and returns values of options required and optionnal.
     #
     # * *Args*
@@ -781,12 +863,12 @@ module PRC
 
       result = [[], []]
 
-      required.each do | key |
+      required.each do |key|
         return nil unless options.key?(key)
         result[0] << options[key]
       end
 
-      optionnal.each { | key | result[1] << options[key] }
+      optionnal.each { |key| result[1] << options[key] }
 
       result
     end
@@ -837,7 +919,7 @@ module PRC
 
     # Setting indexes from names or indexes.
     def _set_indexes(result)
-      names_indexes = _layer_indexes(result[1][1])
+      names_indexes = layer_indexes(result[1][1])
       # replaced indexes by names indexes if exists.
       result[1][0] = names_indexes if names_indexes
       result[1].delete_at(1)
@@ -846,7 +928,7 @@ module PRC
     def _set_data_opts(result)
       data_opts = []
 
-      result[0][0].each_index do | layer_index |
+      result[0][0].each_index do |layer_index|
         data_options = result[1][0][layer_index] if result[1][0].is_a?(Array)
         data_options = {} unless data_options.is_a?(Hash)
         data_opts << data_options
@@ -863,7 +945,7 @@ module PRC
         config_layers = @config_layers
       else
         config_layers = []
-        result[1][0].each do | index |
+        result[1][0].each do |index|
           config_layers << @config_layers[index] if index.is_a?(Fixnum)
         end
         config_layers = @config_layers if config_layers.length == 0

data/lib/providers/hpcloud/compute.rb

@@ -54,9 +54,9 @@ module HPCompute
 
     result = []
     addresses = oComputeConnect.addresses.all
-    addresses.each do | oElem |
+    addresses.each do |oElem|
       is_found = true
-      sQuery.each do | key, value |
+      sQuery.each do |key, value|
         if !oElem.attributes.key?(key) || oElem.attributes[key] != value
           is_found = false
           break
@@ -76,7 +76,7 @@ module HPCompute
     addresses = oComputeConnect.addresses.all
     address = nil
     # Search for an available IP
-    addresses.each do | oElem |
+    addresses.each do |oElem|
       if oElem.fixed_ip.nil?
         address = oElem
         break

data/lib/providers/hpcloud/hpcloud.rb

@@ -74,12 +74,12 @@ class Hpcloud
   obj_needs :data, :addr_map,   :mapping => :remote_ip_prefix
   obj_needs :data, :sg_id,      :mapping => :security_group_id
 
-  get_attr_mapping :dir,      :direction
-  get_attr_mapping :proto,    :protocol
-  get_attr_mapping :port_min, :port_range_min
-  get_attr_mapping :port_max, :port_range_max
-  get_attr_mapping :addr_map, :remote_ip_prefix
-  get_attr_mapping :sg_id,    :security_group_id
+  def_attr_mapping :dir,      :direction
+  def_attr_mapping :proto,    :protocol
+  def_attr_mapping :port_min, :port_range_min
+  def_attr_mapping :port_max, :port_range_max
+  def_attr_mapping :addr_map, :remote_ip_prefix
+  def_attr_mapping :sg_id,    :security_group_id
 
   define_obj :keypairs
 
@@ -95,11 +95,11 @@ class Hpcloud
   obj_needs :data,   :external_gateway_id, :mapping => [:external_gateway_info,
                                                         'network_id']
 
-  get_attr_mapping :gateway_network_id, [:external_gateway_info, 'network_id']
+  def_attr_mapping :gateway_network_id, [:external_gateway_info, 'network_id']
 
   # ************************************ SERVER Object
   define_obj :server
-  get_attr_mapping :status, :state
+  def_attr_mapping :status, :state
   attr_value_mapping :create, 'BUILD'
   attr_value_mapping :boot,   :boot
   attr_value_mapping :active, 'ACTIVE'
@@ -108,12 +108,12 @@ class Hpcloud
   define_obj :server_log
 
   # Excon::Response object type
-  get_attr_mapping :output,  'output'
+  def_attr_mapping :output,  'output'
 
   # ************************************* Public IP Object
   define_obj :public_ip
-  get_attr_mapping :server_id, :instance_id
-  get_attr_mapping :public_ip, :ip
+  def_attr_mapping :server_id, :instance_id
+  def_attr_mapping :public_ip, :ip
 
   # defines setup Cloud data (:account => true for setup)
   define_data(:account_id,
@@ -368,7 +368,7 @@ class HpcloudController # rubocop: disable Metrics/ClassLength
   def query_each(oFogObject)
     case oFogObject.class.to_s
     when 'Fog::HP::Network::Networks'
-      oFogObject.each { | value | yield(value) }
+      oFogObject.each { |value| yield(value) }
     else
       controller_error "'%s' is not a valid list for 'each'",
                        oFogObject.class
@@ -439,7 +439,7 @@ class HpcloudController # rubocop: disable Metrics/ClassLength
       # not know why...
       search_services = services.rh_get(:service_catalog)
       service = nil
-      service_to_find.each do | sServiceElem |
+      service_to_find.each do |sServiceElem|
         if search_services.key?(sServiceElem)
           service = sServiceElem
           break
@@ -450,7 +450,7 @@ class HpcloudController # rubocop: disable Metrics/ClassLength
                        service_to_find if service.nil?
       result = services.rh_get(:service_catalog, service).keys
       result.delete('name')
-      result.each_index do | iIndex |
+      result.each_index do |iIndex|
         result[iIndex] = result[iIndex].to_s if result[iIndex].is_a?(Symbol)
       end
       return result

data/lib/providers/hpcloud/network.rb

@@ -36,10 +36,10 @@ module HPNetwork
 
   def self.create_subnetwork(oNetworkConnect, oNetwork, name)
     oNetworkConnect.subnets.create(
-       :network_id => oNetwork.id,
-       :name => name,
-       :cidr => get_next_subnet(oNetworkConnect),
-       :ip_version => '4'
+      :network_id => oNetwork.id,
+      :name => name,
+      :cidr => get_next_subnet(oNetworkConnect),
+      :ip_version => '4'
     )
   end
 

data/lib/providers/hpcloud/security_groups.rb

@@ -47,7 +47,7 @@ module HPKeyPairs
     keypairs.each do |sElem|
       is_selected = true
       attributes = sElem.instance_variable_get(:@attributes)
-      sQuery.each do | key, value |
+      sQuery.each do |key, value|
         if attributes[key] != value
           is_selected = false
           break