frise 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 41db8000684dfb7ee23cc7f26a790a57bb38ad76
+  data.tar.gz: 782cf2ff2d3866a754950fd8edd3735a1bb1f425
+SHA512:
+  metadata.gz: c37c19e1f9edbad815b30d52f867d4344fd3b3da8a376e30236814a47ddd579b889459b1a702190fe5d1c1d2b99afdc3abef685f54a062616ea632872640a5e2
+  data.tar.gz: c9c87e93dbd3b361a5632431ac8f83650749c7d1de090c3397299b403cda7e1a986767356d8795dc3cf037f9461227a8287dd3870df79a573031ee5928d31931

data/.gitignore

@@ -0,0 +1,11 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+
+*.iml

data/.rspec

@@ -0,0 +1 @@
+--require spec_helper

data/.rubocop.yml

@@ -0,0 +1,2 @@
+Metrics:
+  Enabled: false

data/.travis.yml

@@ -0,0 +1,6 @@
+language: ruby
+rvm:
+  - 2.1
+  - 2.2
+  - 2.3
+  - 2.4

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in frise.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,13 @@
+Copyright 2017 ShiftForward, S.A. [http://www.shiftforward.eu]
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not
+use this file except in compliance with the License. You may obtain a copy of
+the License at
+
+    [http://www.apache.org/licenses/LICENSE-2.0]
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations under
+the License.

data/README.md

@@ -0,0 +1,131 @@
+# Frise
+[![Build Status](https://travis-ci.org/ShiftForward/frise.svg?branch=master)](https://travis-ci.org/ShiftForward/frise)
+[![Coverage Status](https://coveralls.io/repos/github/ShiftForward/frise/badge.svg?branch=master)](https://coveralls.io/github/ShiftForward/frise?branch=master)
+
+Frise is a library for loading configuration files as native Ruby structures. Besides reading and
+parsing the files themselves, it also:
+
+- Completes it with default values specified in another file or set of files;
+- Interprets [Liquid](https://shopify.github.io/liquid) templates in configs and defaults;
+- Validates the loaded config according to a schema file or set of files.
+
+## Install
+
+```
+gem install frise
+```
+
+## Usage
+
+### Basic configs
+
+The simplest example would be to load [a simple configuration](example/config.yml) from a file:
+
+```ruby
+require 'frise'
+require 'pp'
+
+loader = Frise::Loader.new
+loader.load('example/config.yml')
+# => {"movies"=>
+#     [{"title"=>"The Shawshank Redemption",
+#       "year"=>1994,
+#       "categories"=>["Crime", "Drama"],
+#       "rating"=>9.3},
+#      {"title"=>"The Godfather",
+#       "year"=>1972,
+#       "director"=>"Francis Ford Coppola",
+#       "categories"=>["Crime", "Drama"],
+#       "rating"=>9.2}]}
+```
+
+Currently Frise only supports YAML files, but it may support JSON and other formats in the future.
+
+### Default values
+
+By defining directories where the files with default values can be found (in this example,
+[example/_defaults](example/_defaults)), Frise can handle its application internally on load time:
+
+```ruby
+loader = Frise::Loader.new(defaults_load_paths: ['example/_defaults'])
+loader.load('example/config.yml')
+# => {"movies"=>
+#     [{"title"=>"The Shawshank Redemption",
+#       "year"=>1994,
+#       "categories"=>["Crime", "Drama"],
+#       "rating"=>9.3,
+#       "director"=>"N/A"},
+#      {"title"=>"The Godfather",
+#       "year"=>1972,
+#       "director"=>"Francis Ford Coppola",
+#       "categories"=>["Crime", "Drama"],
+#       "rating"=>9.2}],
+#    "ui"=>
+#     {"default_movie"=>"The Shawshank Redemption",
+#      "filter_from"=>1972,
+#      "filter_to"=>1994}}
+```
+
+Note that files with default values follow exactly the same structure of the config file itself.
+Special values such as `$all` allow users to define default values for all elements of an object or
+array. Liquid templates are also used to define some defaults as a function of other objects of the
+config.
+
+### Schemas
+
+Additionally, configuration files can also be validated against a schema. By specifying
+`schema_load_paths`, users can provide directories where schema files such as
+[example/_schemas/config.yml](example/_schemas/config.yml) can be found:
+
+```ruby
+loader = Frise::Loader.new(
+  schema_load_paths: ['example/_schemas'],
+  defaults_load_paths: ['example/_defaults'])
+
+loader.load('example/config.yml')
+# {"movies"=>
+#   [{"title"=>"The Shawshank Redemption",
+#     "year"=>1994,
+#     "categories"=>["Crime", "Drama"],
+#     "rating"=>9.3,
+#     "director"=>"N/A"},
+#    {"title"=>"The Godfather",
+#     "year"=>1972,
+#     "director"=>"Francis Ford Coppola",
+#     "categories"=>["Crime", "Drama"],
+#     "rating"=>9.2}],
+#  "ui"=>
+#   {"default_movie"=>"The Shawshank Redemption",
+#    "filter_from"=>1972,
+#    "filter_to"=>1994}}
+```
+
+If this config is loaded without the defaults instead, there are now required values that are
+missing and Frise by default prints a summary of the errors and terminates the program:
+
+
+```ruby
+loader = Frise::Loader.new(schema_load_paths: ['example/_schemas'])
+
+loader.load('example/config.yml')
+# 2 config error(s) found:
+#  - At movies.0.director: missing required value
+#  - At ui: missing required value
+```
+
+Once more, the structure of the schema mimics the structure of the config itself, making it easy to
+write schemas and to create a config scaffold from its schema later.
+
+Users can check a whole range of properties in config values besides their type: optional values,
+hashes with validated keys, hashes with unknown keys and even custom validations are also supported.
+The [specification](spec/frise/validator_spec.rb) of the validator provides various examples of
+schemas and describes the behavior of each value (more documentation will be written soon).
+
+### Other features
+
+Users can also define custom code to be run before defaults and schemas are applied and can even do
+each of the steps separately. Additionally, defaults and schemas can be loaded at a specific path
+inside an existing Ruby object. The [Loader](lib/frise/loader.rb) class provides high-level methods
+to access those features, while lower-level functionality can be accessed through
+[Parser](lib/frise/parser.rb), [DefaultsLoader](lib/frise/defaults_loader.rb) and
+[Validator](lib/frise/validator.rb).

data/Rakefile

@@ -0,0 +1,13 @@
+require 'bundler/gem_tasks'
+
+def load_if_available(req_path)
+  require req_path
+  yield
+rescue LoadError
+  false # req not available
+end
+
+load_if_available('rspec/core/rake_task') { RSpec::Core::RakeTask.new(:spec) }
+load_if_available('rubocop/rake_task') { RuboCop::RakeTask.new(:rubocop) }
+
+task default: %i[rubocop spec]

data/frise.gemspec

@@ -0,0 +1,31 @@
+# coding: utf-8
+
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'frise/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = 'frise'
+  spec.version       = Frise::VERSION
+  spec.authors       = ['ShiftForward']
+  spec.email         = ['info@shiftforward.eu']
+
+  spec.summary       = 'Ruby config library with schema validation, default values and templating'
+  spec.homepage      = 'https://github.com/ShiftForward/frise'
+  spec.license       = 'Apache-2.0'
+
+  spec.files         = `git ls-files -z`.split("\x0").reject do |f|
+    f.match(%r{^(test|spec|features|example)/})
+  end
+  spec.require_paths = ['lib']
+  spec.required_ruby_version = '>= 2.1.0'
+
+  spec.add_dependency 'liquid', '~> 3.0'
+
+  spec.add_development_dependency 'bundler', '~> 1.14'
+  spec.add_development_dependency 'coveralls', '~> 0.8.21'
+  spec.add_development_dependency 'simplecov', '~> 0.14.1'
+  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'rspec', '~> 3.4'
+  spec.add_development_dependency 'rubocop', '~> 0.49.1'
+end

data/lib/frise/defaults_loader.rb

@@ -0,0 +1,69 @@
+require 'frise/parser'
+
+module Frise
+  # Provides the logic for merging config defaults into pre-loaded configuration objects.
+  #
+  # The merge_defaults and merge_defaults_at entrypoint methods provide ways to read files with
+  # defaults and apply them to configuration objects.
+  module DefaultsLoader
+    class << self
+      def widened_class(obj)
+        class_name = obj.class.to_s
+        return 'Boolean' if %w[TrueClass FalseClass].include? class_name
+        return 'Integer' if %w[Fixnum Bignum].include? class_name
+        class_name
+      end
+
+      def merge_defaults_obj(config, defaults)
+        if defaults.nil?
+          config
+
+        elsif config.nil?
+          if defaults.class != Hash then defaults
+          elsif defaults['$optional'] then nil
+          else merge_defaults_obj({}, defaults)
+          end
+
+        elsif defaults.class == Array && config.class == Array
+          defaults + config
+
+        elsif defaults.class == Hash && defaults['$all'] && config.class == Array
+          config.map { |elem| merge_defaults_obj(elem, defaults['$all']) }
+
+        elsif defaults.class == Hash && config.class == Hash
+          new_config = {}
+          (config.keys + defaults.keys).uniq.each do |key|
+            next if key.start_with?('$')
+            new_config[key] = config[key]
+            new_config[key] = merge_defaults_obj(new_config[key], defaults[key]) if defaults.key?(key)
+            new_config[key] = merge_defaults_obj(new_config[key], defaults['$all']) unless new_config[key].nil?
+            new_config.delete(key) if new_config[key].nil?
+          end
+          new_config
+
+        elsif widened_class(defaults) != widened_class(config)
+          raise "Cannot merge config #{config.inspect} (#{widened_class(config)}) " \
+            "with default #{defaults.inspect} (#{widened_class(defaults)})"
+
+        else
+          config
+        end
+      end
+
+      def merge_defaults_obj_at(config, at_path, defaults)
+        at_path.reverse.each { |key| defaults = { key => defaults } }
+        merge_defaults_obj(config, defaults)
+      end
+
+      def merge_defaults(config, defaults_file, symbol_table = config)
+        defaults = Parser.parse(defaults_file, symbol_table)
+        merge_defaults_obj(config, defaults)
+      end
+
+      def merge_defaults_at(config, at_path, defaults_file, symbol_table = config)
+        defaults = Parser.parse(defaults_file, symbol_table)
+        merge_defaults_obj_at(config, at_path, defaults)
+      end
+    end
+  end
+end

data/lib/frise/loader.rb

@@ -0,0 +1,60 @@
+require 'frise/defaults_loader'
+require 'frise/parser'
+require 'frise/validator'
+
+module Frise
+  # The entrypoint for loading configs from files according to the conventions defined for Frise.
+  #
+  # The load method loads a configuration file, merges it with the applicable defaults and validates
+  # its schema. Other methods in Loader perform only parts of the process.
+  class Loader
+    def initialize(schema_load_paths: [], defaults_load_paths: [], pre_loaders: [], validators: nil)
+      @schema_load_paths = schema_load_paths
+      @defaults_load_paths = defaults_load_paths
+      @pre_loaders = pre_loaders
+      @validators = validators
+    end
+
+    def load(config_file, exit_on_fail = true, symbol_table = {})
+      config = Parser.parse(config_file, symbol_table)
+      config_name = File.basename(config_file)
+
+      @pre_loaders.each do |pre_loader|
+        config = pre_loader.call(config)
+      end
+
+      config = merge_defaults(config, config_name, symbol_table)
+      validate(config, config_name, exit_on_fail)
+    end
+
+    def merge_defaults(config, defaults_name, symbol_table = {})
+      merge_defaults_at(config, [], defaults_name, symbol_table)
+    end
+
+    def merge_defaults_at(config, at_path, defaults_name, symbol_table = {})
+      @defaults_load_paths.map do |defaults_dir|
+        defaults_file = File.join(defaults_dir, defaults_name)
+        config = DefaultsLoader.merge_defaults_at(
+          config, at_path, defaults_file, symbol_table.merge(config)
+        )
+      end
+      config
+    end
+
+    def validate(config, schema_name, exit_on_fail = true)
+      validate_at(config, [], schema_name, exit_on_fail)
+    end
+
+    def validate_at(config, at_path, schema_name, exit_on_fail = true)
+      @schema_load_paths.map do |schema_dir|
+        schema_file = File.join(schema_dir, schema_name)
+        errors = Validator.validate_at(config, at_path, schema_file,
+                                       validators: @validators,
+                                       print: exit_on_fail,
+                                       fatal: exit_on_fail)
+        return nil if errors.any?
+      end
+      config
+    end
+  end
+end

data/lib/frise/parser.rb

@@ -0,0 +1,17 @@
+require 'liquid'
+require 'yaml'
+
+module Frise
+  # Provides a static parse method for reading a config from a YAML file applying the required
+  # transformations.
+  module Parser
+    class << self
+      def parse(file, symbol_table = nil)
+        return {} unless File.file? file
+        content = File.open(file).read
+        content = Liquid::Template.parse(content).render symbol_table if symbol_table
+        YAML.safe_load(content, [], [], true) || {}
+      end
+    end
+  end
+end

data/lib/frise/validator.rb

@@ -0,0 +1,179 @@
+#!/usr/bin/env ruby
+
+require 'frise/parser'
+require 'set'
+
+module Frise
+  # Checks if a pre-loaded config object conforms to a schema file.
+  #
+  # The validate and validate_at static methods read schema files and validates config objects
+  # against the parsed schema. They can optionally be initialized with a set of user-defined
+  # validators that can be used in the schema files for custom validations.
+  class Validator
+    attr_reader :errors
+
+    def initialize(root, validators = nil)
+      @root = root
+      @validators = validators
+      @errors = []
+    end
+
+    def widened_class(obj)
+      class_name = obj.class.to_s
+      return 'Boolean' if %w[TrueClass FalseClass].include? class_name
+      return 'Integer' if %w[Fixnum Bignum].include? class_name
+      class_name
+    end
+
+    def add_validation_error(path, msg)
+      logged_path = path.empty? ? '<root>' : path[1..-1]
+      @errors << "At #{logged_path}: #{msg}"
+    end
+
+    def get_full_schema(schema)
+      case schema
+      when Hash then schema
+      when Symbol then { type: 'Object', validate: schema }
+      when Array then
+        if schema.size == 1
+          { type: 'Array', all: schema[0] }
+        else
+          (raise "Invalid schema: #{schema.inspect}")
+        end
+      when String then
+        if schema.end_with?('?')
+          { type: schema[0..-2], optional: true }
+        else
+          { type: schema }
+        end
+      else raise "Invalid schema: #{schema.inspect}"
+      end
+    end
+
+    def validate_optional(full_schema, obj, path)
+      if obj.nil?
+        add_validation_error(path, 'missing required value') unless full_schema[:optional]
+        return false
+      end
+      true
+    end
+
+    def get_expected_types(full_schema)
+      type_key = full_schema.fetch(:type, 'Hash')
+      allowed_types = %w[Hash Array String Integer Float Object]
+      return [Object.const_get(type_key)] if allowed_types.include?(type_key)
+      return [TrueClass, FalseClass] if type_key == 'Boolean'
+      raise "Invalid expected type in schema: #{type_key}"
+    end
+
+    def validate_type(full_schema, obj, path)
+      expected_types = get_expected_types(full_schema)
+      unless expected_types.any? { |typ| obj.is_a?(typ) }
+        type_key = full_schema.fetch(:type, 'Hash')
+        add_validation_error(path, "expected #{type_key}, found #{widened_class(obj)}")
+        return false
+      end
+      true
+    end
+
+    def validate_custom(full_schema, obj, path)
+      if full_schema[:validate]
+        begin
+          @validators.method(full_schema[:validate]).call(@root, obj)
+        rescue StandardError => ex
+          add_validation_error(path, ex.message)
+        end
+      end
+      true
+    end
+
+    def validate_spec_keys(full_schema, obj, path, processed_keys)
+      full_schema.each do |spec_key, spec_value|
+        next if spec_key.is_a?(Symbol)
+        validate_object("#{path}.#{spec_key}", obj[spec_key], spec_value)
+        processed_keys << spec_key
+      end
+      true
+    end
+
+    def validate_remaining_keys(full_schema, obj, path, processed_keys)
+      expected_types = get_expected_types(full_schema)
+      if expected_types.size == 1 && expected_types[0].ancestors.member?(Enumerable)
+        hash = obj.is_a?(Hash) ? obj : Hash[obj.map.with_index { |x, i| [i, x] }]
+        hash.each do |key, value|
+          if full_schema[:all_keys] && !key.is_a?(Symbol)
+            validate_object(path, key, full_schema[:all_keys])
+          end
+
+          next if processed_keys.member? key
+          if full_schema[:all]
+            validate_object("#{path}.#{key}", value, full_schema[:all])
+          elsif !full_schema[:allow_unknown_keys]
+            add_validation_error(path, "unknown key: #{key}")
+          end
+        end
+      end
+      true
+    end
+
+    def validate_object(path, obj, schema)
+      full_schema = get_full_schema(schema)
+
+      return unless validate_optional(full_schema, obj, path)
+      return unless validate_type(full_schema, obj, path)
+      return unless validate_custom(full_schema, obj, path)
+
+      processed_keys = Set.new
+      return unless validate_spec_keys(full_schema, obj, path, processed_keys)
+      validate_remaining_keys(full_schema, obj, path, processed_keys)
+    end
+
+    def self.parse_symbols(obj)
+      case obj
+      when Array then obj.map { |e| parse_symbols(e) }
+      when Hash then Hash[obj.map { |k, v| [parse_symbols(k), parse_symbols(v)] }]
+      when String then obj.start_with?('$') ? obj[1..-1].to_sym : obj
+      else obj
+      end
+    end
+
+    def self.validate_obj(config, schema, options = {})
+      validator = Validator.new(config, options[:validators])
+      validator.validate_object('', config, schema)
+
+      if validator.errors.any?
+        if options[:print]
+          puts "#{validator.errors.length} config error(s) found:"
+          validator.errors.each do |error|
+            puts " - #{error}"
+          end
+        end
+
+        exit 1 if options[:fatal]
+        raise ValidationError.new(validator.errors), 'Invalid configuration' if options[:raise_error]
+      end
+      validator.errors
+    end
+
+    def self.validate(config, schema_file, options = {})
+      schema = parse_symbols(Parser.parse(schema_file))
+      validate_obj(config, schema, options)
+    end
+
+    def self.validate_at(config, at_path, schema_file, options = {})
+      schema = parse_symbols(Parser.parse(schema_file))
+      at_path.reverse.each { |key| schema = { key => schema, :allow_unknown_keys => true } }
+      validate_obj(config, schema, options)
+    end
+  end
+
+  # An error resulting of the validation of a config. The list of errors can be inspected using the
+  # errors method.
+  class ValidationError < StandardError
+    attr_reader :errors
+
+    def initialize(errors)
+      @errors = errors
+    end
+  end
+end

data/lib/frise/version.rb

@@ -0,0 +1,3 @@
+module Frise
+  VERSION = '0.1.0'.freeze
+end

data/lib/frise.rb

@@ -0,0 +1,5 @@
+require 'frise/defaults_loader'
+require 'frise/loader'
+require 'frise/parser'
+require 'frise/validator'
+require 'frise/version'

metadata

@@ -0,0 +1,157 @@
+--- !ruby/object:Gem::Specification
+name: frise
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- ShiftForward
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2017-08-10 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: liquid
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.14'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.14'
+- !ruby/object:Gem::Dependency
+  name: coveralls
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.8.21
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.8.21
+- !ruby/object:Gem::Dependency
+  name: simplecov
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.14.1
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.14.1
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.4'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.4'
+- !ruby/object:Gem::Dependency
+  name: rubocop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.49.1
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.49.1
+description: 
+email:
+- info@shiftforward.eu
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".rubocop.yml"
+- ".travis.yml"
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- frise.gemspec
+- lib/frise.rb
+- lib/frise/defaults_loader.rb
+- lib/frise/loader.rb
+- lib/frise/parser.rb
+- lib/frise/validator.rb
+- lib/frise/version.rb
+homepage: https://github.com/ShiftForward/frise
+licenses:
+- Apache-2.0
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.1.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.11
+signing_key: 
+specification_version: 4
+summary: Ruby config library with schema validation, default values and templating
+test_files: []