sfn-parameters 0.1.2 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 383bf03f04f48d1ab57c52aa83ade4785857822d
-  data.tar.gz: 0bd07943bc258049c41c36992a064d8505887c34
+  metadata.gz: fa98393dbb5772d3b8058e38d07d604ad0bd14e4
+  data.tar.gz: 4eb0b9d98743d33e4cab470c4e0034fc6f38baa3
 SHA512:
-  metadata.gz: eda7b3f8b62025688e4b4c9989747fbc83634ba64a5ea6e59c9dc5a73bf1230ae701983c7a17e55f12a6b95d38097234ec64d90029d3618555e3639493a1b5f1
-  data.tar.gz: 7f4e6716911e5fecea3988dd33491183cceab6a24f4323b9e6abc27f7c8aee427c3c386894f019cff15bf848aec15f4ce8597d19ea6970f098c1f4fdc851c421
+  metadata.gz: 0ab1532d9ed57477fe43dd480c91fa1cbdf047c12216be462f341b5db45265b58dc4570acb4c5b4b745df6e6f55dfb79af47ef0f76128bba51436ae5fdc1c786
+  data.tar.gz: dd16da459bc56af432aa2fa0528032f64b9210f16684ce409c21e5e32f1698f0a03a940127673ee03707a580606d41ecccc6eb9df97822211a1d849e869a0c6a

data/CHANGELOG.md

@@ -1,3 +1,7 @@
+# v0.2.0
+* Add support for single stack workflow (#1)
+* Add encryption foundation with initial OpenSSL implementation (#2)
+
 # v0.1.2
 * Relax sfn constraint to allow 2.0 version
 * Fix nested stack parameter prefix generation

data/README.md

@@ -4,15 +4,25 @@ Provides automatic assignment of stack parameter information.
 Currently supported workflows:
 
 * Infrastructure Mode
+* Stacks Mode
+
+This callback also supports optional encryption of stack
+parameter files. Current implementations:
+
+* OpenSSL
 
 ## Usage
 
-### Infrastructure Mode
+Make the callback available by adding it to the bundle via the
+project's Gemfile:
 
-Infrastructure mode assumes a single template which describes
-an entire infrastructure (generally via nested templates). A
-configuration file can provide all information required for the
-root stack, as well as all descendant stacks.
+~~~ruby
+group :sfn do
+  gem 'sfn-parameters'
+end
+~~~
+
+### Parameters
 
 #### Enable
 
@@ -23,12 +33,39 @@ configuration file. First the callback must be enabled:
 Configuration.new do
   callbacks do
     require ['sfn-parameters']
-    default ['parameters_infrastructure']
+    default ['parameters_infrastructure'] # or ['parameters_stacks']
   end
 end
 ~~~
 
-#### Configure
+#### File format
+
+The basic structure of the file in JSON:
+
+~~~json
+{
+  "parameters": {},
+  "compile_parameters": {},
+  "apply_stacks": [],
+  "stacks": {}
+}
+~~~
+
+Break down of the keys:
+
+* `parameters` - Run time parameters sent to the orchestration API
+* `compile_parameters` - Compile time parameters used to generate the template
+* `apply_stacks` - List of stacks whose outputs should be applied
+* `stacks`- Nested stack information
+
+#### Infrastructure Mode
+
+Infrastructure mode assumes a single template which describes
+an entire infrastructure (generally via nested templates). A
+configuration file can provide all information required for the
+root stack, as well as all descendant stacks.
+
+##### Configure
 
 Some optional configuration is available via the `.sfn` file
 to control the behavior of the callback:
@@ -45,7 +82,7 @@ end
 * `directory` - Relative path from repository root to directory containing configuration files
 * `destination` - Name of file holding configuration minus the extension
 
-#### Functionality
+##### Functionality
 
 One file will contain the configuration information required
 for the stack operation (create/update). The location of this
@@ -61,26 +98,6 @@ The contents of the file will be processed using the bogo-config
 library. This allows defining the file in a serialization format
 (JSON, YAML, XML) or as a Ruby file.
 
-#### File format
-
-The basic structure of the file in JSON:
-
-~~~json
-{
-  "parameters": {},
-  "compile_parameters": {},
-  "apply_stacks": [],
-  "stacks": {}
-}
-~~~
-
-Break down of the keys:
-
-* `parameters` - Run time parameters sent to the orchestration API
-* `compile_parameters` - Compile time parameters used to generate the template
-* `apply_stacks` - List of stacks whose outputs should be applied
-* `stacks`- Nested stack information
-
 ##### Example
 
 ~~~json
@@ -101,6 +118,121 @@ Break down of the keys:
 }
 ~~~
 
+#### Stacks Mode
+
+The stacks mode assumes multiple stacks represented by multiple templates. Each stack
+will have a corresponding parameters file which matches the stack name.
+
+##### Configure
+
+Some optional configuration is available via the `.sfn` file
+to control the behavior of the callback:
+
+~~~ruby
+Configuration.new do
+  sfn_parameters do
+    directory 'stacks'
+  end
+end
+~~~
+
+* `directory` - Relative path from repository root to directory containing configuration files
+
+##### Functionality
+
+One file will contain the configuration information required
+for a single stack operation (create/update). The location of this
+file is generated using the configuration value provided
+above with the stack name. For example, if working with a stack
+named `my-test-stack`, the file will be expected at:
+
+~~~
+REPO_ROOT/stacks/my-test-stack.{rb,xml,json,yaml,yml}
+~~~
+
+The contents of the file will be processed using the bogo-config
+library. This allows defining the file in a serialization format
+(JSON, YAML, XML) or as a Ruby file.
+
+##### Example
+
+~~~json
+{
+  "parameters": {
+    "stack_creator": "chris"
+  },
+  "apply_stacks": [
+    "networking"
+  ]
+}
+~~~
+
+### Encryption
+
+This callback also supports encrypting stack parameter information for storage. The callback
+adds a `parameters` command for handling encryption/decryption.
+
+#### Configuration
+
+Encryption configuration is controlled within the `.sfn` file:
+
+~~~ruby
+Configuration.new
+  sfn_parameters do
+    safe do
+      key 'MY-SECRET-KEY'
+      type 'ssl'
+      cipher 'AES-256-CBC'
+      iterations 10000
+      salt 'sfn~parameters~crypt~salt'
+    end
+  end
+end
+~~~
+
+##### Default options
+
+* `type` - Safe type for encryption (default: ssl)
+
+##### OpenSSL options
+
+* `key` - **REQUIRED** - Secret shared key
+* `cipher` - Cipher to be used (default: 'AES-256-CBC')
+* `iterations` - Modify computation length (default: 10000)
+* `salt` - Random string (default: random bytes)
+
+#### Commands
+
+##### Create a new file
+
+Create a new parameters file and automatically lock it when complete:
+
+~~~
+$ sfn parameters create my-test-stack
+~~~
+
+##### Update an existing file
+
+Edit the file and only lock the file if it was previously locked:
+
+~~~
+$ sfn parameters edit my-test-stack
+~~~
+
+##### Lock an existing file
+
+~~~
+$ sfn parameters lock my-test-stack
+~~~
+
+##### Unlock an existing file
+
+~~~
+$ sfn parameters unlock my-test-stack
+~~~
+
+_NOTE: Full paths can also be used when defining parameters file._
+
 # Info
 
 * Repository: https://github.com/sparkleformation/sfn-parameters

data/lib/sfn-parameters/command.rb

@@ -0,0 +1,167 @@
+require 'sfn-parameters'
+
+module Sfn
+  class Command
+    # Parameters command
+    class Parameters < Command
+
+      include SfnParameters::Utils
+      include Sfn::CommandModule::Base
+
+      # Execute parameters action request
+      def execute!
+        action, item = arguments[0,2]
+        ui.info "Running parameters action #{ui.color(action.to_s, :bold)}"
+        case action.to_sym
+        when :lock
+          item = validate_item(item)
+          ui.print " Locking #{ui.color(item, :bold)}... "
+          content = load_json(File.read(item)).to_smash
+          if(content[:sfn_parameters_lock])
+            ui.puts ui.color('no-op', :yellow)
+            ui.warn "Item is already locked! (#{item})"
+          else
+            thing = lock_content(content)
+            val = format_json(thing)
+            File.write(item, val)
+            ui.puts ui.color('locked', :blue)
+          end
+        when :unlock
+          item = validate_item(item)
+          ui.print " Unlocking #{ui.color(item, :bold)}... "
+          content = load_json(File.read(item)).to_smash
+          if(content[:sfn_parameters_lock])
+            content = unlock_content(content)
+            content.delete(:sfn_lock_enabled)
+            File.write(item, format_json(content))
+            ui.puts ui.color('unlocked', :green)
+          else
+            ui.puts ui.color('no-op', :yellow)
+            ui.warn "Item is already unlocked! (#{item})"
+          end
+        when :show
+          item = validate_item(item)
+          content = load_json(File.read(item)).to_smash
+          if(content[:sfn_parameters_lock])
+            ui.print ui.color(' *', :bold)
+            ui.print " Unlocking #{ui.color(item, :bold)} for display... "
+            content = unlock_content(content)
+            content.delete(:sfn_lock_enabled)
+            ui.puts ui.color('unlocked', :green)
+          end
+          ui.puts format_json(content)
+        when :create, :edit
+          unless(ENV['EDITOR'])
+            raise ArgumentError.new '$EDITOR must be set for create/edit commands!'
+          end
+          begin
+            item = validate_item(item)
+          rescue ArgumentError
+            new_item = true
+            item = new_item(item)
+          end
+          FileUtils.mkdir_p(File.dirname(item))
+          tmp = Bogo::EphemeralFile.new('sfn-parameters')
+          content = new_item ? Smash.new : load_json(File.read(item)).to_smash
+          if(content[:sfn_parameters_lock])
+            ui.print ui.color(' *', :bold)
+            ui.print " Unlocking #{ui.color(item, :bold)} for edit... "
+            content = unlock_content(content)
+            ui.puts ui.color('unlocked', :green)
+          end
+          lock_enabled = content.delete(:sfn_lock_enabled) || new_item
+          tmp.write(format_json(content))
+          tmp.flush
+          system("#{ENV['EDITOR']} #{tmp.path}")
+          tmp.rewind
+          content = load_json(tmp.read).to_smash
+          ui.print ui.color(' *', :bold)
+          if(lock_enabled)
+            ui.print " Locking #{ui.color(item, :bold)} for storage... "
+            content = lock_content(content)
+            ui.puts ui.color('locked', :blue)
+          else
+            ui.puts " Storing #{ui.color(item, :bold)} for storage... #{ui.color('unlocked', :yellow)}"
+          end
+          File.write(item, format_json(content))
+          tmp.close
+        else
+          ArgumentError.new "Unsupported action received `#{action}`. " \
+            "Allowed: lock, unlock, show, create, edit"
+        end
+      end
+
+      # Expand path for new item if required
+      #
+      # @param item [String]
+      # @return [String]
+      def new_item(item)
+        unless(item.include?(File::SEPARATOR))
+          prefixes = [
+            config.get(:sfn_parameters, :directory),
+            'infrastructure',
+            'stacks'
+          ].compact
+          prefix = prefixes.find_all do |dir|
+            File.directory?(dir)
+          end
+          if(prefix.size > 1)
+            raise ArgumentError.new "Unable to auto-determine directory for item! Multiple directories found. " \
+              "(detected: #{prefix.join(', ')})"
+          elsif(prefix.empty?)
+            raise ArgumentError.new "No existing parameter directories found. Please create required directory. " \
+              "(checked: #{prefixes.join(', ')})"
+          end
+          File.join(prefix.first, "#{item}.json")
+        end
+      end
+
+      # Validate existence of requested item. Expand path
+      # if only name given
+      #
+      # @param item [String]
+      # @return [String]
+      def validate_item(item)
+        items = [
+          item,
+          File.join(
+            config.fetch(
+              :sfn_parameters, :directory, 'stacks'
+            ),
+            item
+          ),
+          File.join(
+            config.fetch(
+              :sfn_parameters, :directory, 'stacks'
+            ),
+            "#{item}.json"
+          ),
+          File.join(
+            config.fetch(
+              :sfn_parameters, :directory, 'infrastructure'
+            ),
+            item
+          ),
+          File.join(
+            config.fetch(
+              :sfn_parameters, :directory, 'infrastructure'
+            ),
+            "#{item}.json"
+          )
+        ]
+        valid = items.find_all do |file|
+          File.exist?(file)
+        end
+        if(valid.empty?)
+          raise ArgumentError.new "Failed to locate item `#{item}`!"
+        elsif(valid.size > 1)
+          raise ArgumentError.new "Multiple matches detected for item `#{item}`. " \
+            "(Matches: #{valid.join(', ')})"
+        else
+          valid.first
+        end
+      end
+
+    end
+  end
+end

data/lib/sfn-parameters/config.rb

@@ -0,0 +1,8 @@
+require 'sfn-parameters'
+
+module Sfn
+  class Config
+    class Parameters < Config
+    end
+  end
+end

data/lib/sfn-parameters/infrastructure.rb

@@ -5,6 +5,9 @@ module Sfn
     # Auto load stack parameters for infrastructure pattern
     class ParametersInfrastructure < Callback
 
+      include Sfn::Utils::JSON
+      include SfnParameters::Utils
+
       # Valid file extensions for configuration file
       VALID_EXTENSIONS = ['.rb', '.xml', '.json', '.yaml', '.yml']
 
@@ -37,9 +40,10 @@ module Sfn
         if(paths.size > 1)
           raise ArgumentError.new "Multiple parameter file matches encountered! (#{paths.join(', ')})"
         elsif(paths.empty?)
-          raise ArgumentError.new 'No parameter file matches found!'
+          Smash.new
+        else
+          unlock_content(Bogo::Config.new(paths.first).data)
         end
-        Bogo::Config.new(paths.first).data
       end
 
       # Process the given hash and set configuration values

data/lib/sfn-parameters/safe/ssl.rb

@@ -0,0 +1,97 @@
+require 'sfn-parameters'
+
+module SfnParameters
+  # Safe storage
+  class Safe
+
+    # OpenSSL based Safe implementation
+    class Ssl < Safe
+
+      # Default cipher
+      DEFAULT_CIPHER='AES-256-CBC'
+      # Maximum computation iteration length
+      CRYPT_ITER=10000
+      # Default length of generated key
+      CRYPT_KEY_LENGTH=32
+
+      # Create OpenSSL backed safe
+      #
+      # @param args [Hash]
+      # @option :args [String] :cipher name of cipher
+      # @option :args [String] :key shared key value
+      # @option :args [Integer] :iterations generation interation
+      # @option :args [String] :salt value for salting
+      # @option :args [Integer] :key_length length of generated key
+      # @return [self]
+      def initialize(*_)
+        super
+        unless(arguments[:salt])
+          arguments[:salt] = OpenSSL::Random.random_bytes(16)
+        end
+        unless(arguments[:key])
+          raise ArgumentError.new 'Required `:key` argument unset for `Safe::Ssl`!'
+        end
+      end
+
+      # Lock a given value for storage
+      #
+      # @param value [String] value to lock
+      # @return [Hash] locked content in form {:iv, :content}
+      def lock(value)
+        cipher = build(arguments[:salt])
+        new_iv = cipher.random_iv
+        cipher.iv = new_iv
+        result = cipher.update(value) + cipher.final
+        Smash.new(
+          :iv => Base64.urlsafe_encode64(new_iv),
+          :cipher => arguments.fetch(:cipher, DEFAULT_CIPHER),
+          :content => Base64.urlsafe_encode64(result),
+          :salt => Base64.urlsafe_encode64(arguments[:salt]),
+          :sfn_parameters_lock => Bogo::Utility.snake(self.class.name.split('::').last)
+        )
+      end
+
+      # Unlock a given value for access
+      #
+      # @param value [Hash] content to unlock
+      # @option :value [String] :iv initialization vector value
+      # @option :value [String] :salt random salt value
+      # @option :value [String] :content stored content
+      # @return [String]
+      def unlock(value)
+        value = value.to_smash
+        o_cipher = arguments[:cipher]
+        arguments[:cipher] = value[:cipher] if value[:cipher]
+        cipher = build(
+          Base64.urlsafe_decode64(value[:salt]),
+          Base64.urlsafe_decode64(value[:iv])
+        )
+        arguments[:cipher] = o_cipher
+        string = Base64.urlsafe_decode64(value[:content])
+        cipher.update(string) + cipher.final
+      end
+
+      protected
+
+      # Build a new cipher
+      #
+      # @param iv [String] initialization vector
+      # @param salt [String] random value
+      # @return [OpenSSL::Cipher]
+      def build(salt=nil, iv=nil)
+        cipher = OpenSSL::Cipher.new(arguments[:cipher] || DEFAULT_CIPHER)
+        iv ? cipher.decrypt : cipher.encrypt
+        key = OpenSSL::PKCS5.pbkdf2_hmac_sha1(
+          arguments[:key],
+          salt,
+          arguments.fetch(:iterations, CRYPT_ITER),
+          arguments.fetch(:key_length, CRYPT_KEY_LENGTH)
+        )
+        cipher.iv = iv if iv
+        cipher.key = key
+        cipher
+      end
+
+    end
+  end
+end

data/lib/sfn-parameters/safe.rb

@@ -0,0 +1,56 @@
+require 'sfn-parameters'
+
+module SfnParameters
+  # Safe storage
+  class Safe
+
+    autoload :Ssl, 'sfn-parameters/safe/ssl'
+
+    # @return [Hash] safe configuration
+    attr_reader :arguments
+
+    # Create a new safe
+    #
+    # @param args [Hash]
+    # @return [self]
+    def initialize(args={})
+      @arguments = args.to_smash
+    end
+
+    # Lock a given value for storage
+    #
+    # @param value [String] value to lock
+    # @return [Hash]
+    def lock(value)
+      raise NotImplementedError
+    end
+
+    # Unlock a given value for access
+    #
+    # @param value [Hash] content to unlock
+    # @return [String]
+    def unlock(value)
+      raise NotImplementedError
+    end
+
+    class << self
+
+      # Build a new safe instance
+      #
+      # @param args [Hash] arguments for safe instance
+      # @option args [String] :type type of safe
+      # @return [Safe]
+      def build(args={})
+        args = args.to_smash
+        type = Bogo::Utility.camel(args.fetch(:type, 'ssl'))
+        if(const_defined?(type))
+          const_get(type).new(args)
+        else
+          raise ArgumentError.new "Unknown safe type provided `#{type}`."
+        end
+      end
+
+    end
+
+  end
+end

data/lib/sfn-parameters/stacks.rb

@@ -0,0 +1,26 @@
+require 'sfn-parameters'
+
+module Sfn
+  class Callback
+    # Auto load stack parameters for single stack pattern
+    class ParametersStacks < ParametersInfrastructure
+
+      # Load the configuration file
+      #
+      # @param stack_name [String]
+      # @return [Smash]
+      def load_file_for(stack_name)
+        root_path = config.fetch(:sfn_parameters, :directory, 'stacks')
+        paths = Dir.glob(File.join(root_path, "#{stack_name}{#{VALID_EXTENSIONS.join(',')}}")).map(&:to_s)
+        if(paths.size > 1)
+          raise ArgumentError.new "Multiple parameter file matches encountered! (#{paths.join(', ')})"
+        elsif(paths.empty?)
+          Smash.new
+        else
+          unlock_content(Bogo::Config.new(paths.first).data)
+        end
+      end
+
+    end
+  end
+end

data/lib/sfn-parameters/utils.rb

@@ -0,0 +1,37 @@
+require 'sfn-parameters'
+
+module SfnParameters
+  # Common helper methods
+  module Utils
+
+    # Lock the given content
+    #
+    # @param content [Hash] content to lock
+    # @return [Hash] locked content
+    def lock_content(content)
+      content = content.to_smash
+      content.merge!(:sfn_lock_enabled => true)
+      safe = SfnParameters::Safe.build(
+        config.fetch(:sfn_parameters, :safe, Smash.new)
+      )
+      safe.lock(dump_json(content))
+    end
+
+    # Unlock given content
+    #
+    # @param content [Hash] content to unlock
+    # @return [Hash] unlocked content
+    def unlock_content(content)
+      content = content.to_smash
+      if(content[:sfn_parameters_lock])
+        safe = SfnParameters::Safe.build(
+          config.fetch(:sfn_parameters, :safe, Smash.new)
+        )
+        load_json(safe.unlock(content)).to_smash.merge(:sfn_lock_enabled => true)
+      else
+        content
+      end
+    end
+
+  end
+end

data/lib/sfn-parameters/version.rb

@@ -1,3 +1,3 @@
 module SfnParameters
-  VERSION = Gem::Version.new('0.1.2')
+  VERSION = Gem::Version.new('0.2.0')
 end

data/lib/sfn-parameters.rb

@@ -1,3 +1,12 @@
+require 'sfn'
+
+module SfnParameters
+  autoload :Safe, 'sfn-parameters/safe'
+  autoload :Utils, 'sfn-parameters/utils'
+end
+
 require 'sfn-parameters/version'
 require 'sfn-parameters/infrastructure'
-#require 'sfn-parameters/single-stack'
+require 'sfn-parameters/stacks'
+require 'sfn-parameters/config'
+require 'sfn-parameters/command'

data/sfn-parameters.gemspec

@@ -10,6 +10,6 @@ Gem::Specification.new do |s|
   s.description = 'SparkleFormation Parameters Callback'
   s.license = 'Apache-2.0'
   s.require_path = 'lib'
-  s.add_dependency 'sfn', '>= 1.0.0', '< 3.0'
+  s.add_dependency 'sfn', '>= 3.0', '< 4.0'
   s.files = Dir['{lib,bin,docs}/**/*'] + %w(sfn-parameters.gemspec README.md CHANGELOG.md LICENSE)
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sfn-parameters
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.2.0
 platform: ruby
 authors:
 - Chris Roberts
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-01-22 00:00:00.000000000 Z
+date: 2016-05-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sfn
@@ -16,20 +16,20 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 1.0.0
+        version: '3.0'
     - - "<"
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '4.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 1.0.0
+        version: '3.0'
     - - "<"
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '4.0'
 description: SparkleFormation Parameters Callback
 email: code@chrisroberts.org
 executables: []
@@ -40,7 +40,13 @@ files:
 - LICENSE
 - README.md
 - lib/sfn-parameters.rb
+- lib/sfn-parameters/command.rb
+- lib/sfn-parameters/config.rb
 - lib/sfn-parameters/infrastructure.rb
+- lib/sfn-parameters/safe.rb
+- lib/sfn-parameters/safe/ssl.rb
+- lib/sfn-parameters/stacks.rb
+- lib/sfn-parameters/utils.rb
 - lib/sfn-parameters/version.rb
 - sfn-parameters.gemspec
 homepage: http://github.com/sparkleformation/sfn-parameters