process_settings 0.4.0.pre.11 → 0.4.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: ae29f9011f41199d20fb3ea7d752808f073747d3
-  data.tar.gz: f223b8d3815773cd89bf77d586fba51ffbfcf4cb
+  metadata.gz: 191a509df72fd317ad7475301c1c1f9ed2b9234d
+  data.tar.gz: 32913ebed29db8091d71c1e669d75964f0b2170f
 SHA512:
-  metadata.gz: fdb5acfd1b2729d0973d48cf59af213bb37fe6551a2acc22a20c4bbbfb465b9856e0309306d1e0797c0b475ba3c6274acf53e485820cc7ce7f43f369ba4c0878
-  data.tar.gz: 7b7cfd35fff4fc5beadd5dfe57c73fb572ef80e2c350027162cf9c668d7482ed59318eb318e359c39cba08f2f94a6aa6368a9e432f5869786caf4b7b535cf87a
+  metadata.gz: b09368b7c8d4d091e955d492be4231cc909719e544d170484e79f1ee27af606c42e927e87f465e7cb7587b02ecbe7750a0ff2bee4f2d91a75dd4dfd8cfc6f327
+  data.tar.gz: 7aa5af5281aec6c0c9ec14f9c47c9bdf5e276ba7d6ceeb348dc2e62eebddb5f0553eeb13bcc6271c0a3ef7c957f6faaa6fbee27515ce007eed79c0ec5cf306bc

data/README.md

@@ -1,9 +1,7 @@
-# ProcessSettings
-This gem provides dynamic settings for Ruby processes. The settings are stored in YAML.
-Settings are managed in a git repo, in separate YAML files for each concern (for example, each micro-service). Each YAML file can be targeted based on matching context values (for example, `service_name`).
-
-
-The context can be either static to the process (for example, `service_name` or `data_center`) or dynamic (for example, the current web request `domain`).
+# ProcessSettings [![Build Status](https://travis-ci.org/Invoca/process_settings.svg?branch=master)](https://travis-ci.org/Invoca/process_settings) [![Coverage Status](https://coveralls.io/repos/github/Invoca/process_settings/badge.svg?branch=master)](https://coveralls.io/github/Invoca/process_settings?branch=master) [![Gem Version](https://badge.fury.io/rb/process_settings.svg)](https://badge.fury.io/rb/process_settings)
+This gem provides dynamic settings for Linux processes. These settings are stored in JSON.
+They including a targeting notation so that each settings group can be targeted based on matching context values.
+The context can be either static to the process (for example, service_name or data_center) or dynamic (for example, the domain of the current web request).
 
 ## Installation
 To install this gem directly on your machine from rubygems, run the following:
@@ -13,151 +11,27 @@ gem install process_settings
 
 To install this gem in your bundler project, add the following to your Gemfile:
 ```ruby
-gem 'process_settings', '~> 0.4'
+gem 'process_settings', '~> 0.3'
 ```
 
-## Usage
-The `ProcessSettings::Monitor` and related classes can be freely created and used at any time.
-But typical usage is through the `ProcessSettings::Monitor.instance`.
-That should be configured at process startup before use.
-### Configuration
-Before using `ProcessSettings::Monitor.instance`, you must first configure the path to the combined process settings file on disk,
-and provide a logger.
+To use an unreleased version, add it to your Gemfile for Bundler:
 ```ruby
-require 'process_settings'
-
-ProcessSettings::Monitor.file_path = "/etc/process_settings/combined_process_settings.yml"
-ProcessSettings::Monitor.logger = logger
-```
-### Monitor Initialization
-The `ProcessSettings::Monitor` is a hybrid singleton. The class attribute `instance` returns
-the current instance. If not already set, this is lazy-created based on the above configuration.
-
-The monitor should be initialized with static (unchanging) context for your process:
-```
-ProcessSettings::Monitor.static_context = {
-  "service_name" => "frontend",
-  "data_center" => "AWS-US-EAST-1"
-}
-```
-The `static_context` is important because it is used to pre-filter settings for the process.
-For example, a setting that is targeted to `service_name: frontend` will match the above static context and
-be simplified to `true`. In other processes with a different `service_name`, such a targeted setting will be
-simplified to `false` and removed from memory.
-
-Note that the `static_context` as well as `dynamic_context` must use strings, not symbols, for both keys and values.
-
-### Reading Settings
-For the following section, consider this `combined_process_settings.yml` file:
-```
----
-- filename: frontend.yml
-  settings:
-    frontend:
-      log_level: info
-- filename: frontend-microsite.yml
-  target:
-    domain: microsite.example.com
-  settings:
-    frontend:
-      log_level: debug
-- meta:
-    version: 27
-    END: true
-```
-
-To read a setting, application code should call the `[]` method on the `ProcessSettings` class. For example:
-```
-log_level = ProcessSettings['frontend', 'log_level']
-=> "info"
-```
-#### ProcessSettings[] interface
-The `ProcessSettings[]` method delegates to `ProcessSettings::Monitor#[]` on the `instance`.
-
-`[]` interface:
-
-```
-[](*path, dynamic_context: {}, required: true)
-```
-
-|argument|description|
-|--------|-------------|
-|_path_    |A series of 1 or more comma-separated strings forming a path to navigate the `settings` hash, starting at the top.|
-|`dynamic_context:` |An optional hash of dynamic settings, used to target the settings. This will automatically be deep-merged with the static context. It may not contradict the static context. |
-|`required:` |A boolean indicating if the setting is required to be present. If a setting is missing, then if `required` is truthy, a `ProcesssSettings::SettingsPathNotFound` exception will be raised. Otherwise, `nil` will be returned. Default: `true`.
-
-Example with `dynamic_context`:
-```
-log_level = ProcessSettings['frontend', 'log_level',
-                            dynamic_context: { "domain" => "microsite.example.com" }
-                           ]
-=> "debug"
-```
-
-Example with `required: true` (default) that was not found:
+gem 'process_settings', git: 'git@github.com:Invoca/process_settings'
 ```
-http_version = ProcessSettings['frontend', 'http_version']
-
-exception raised!
 
-ProcessSettings::SettingsPathNotFound: No settings found for path ["frontend", "http_version"]
+## Usage
+### Initialization
+To use the contextual logger, all you need to do is initailize the object with your existing logger
+```ruby
+require 'process_settings'
 ```
 
-Here is the same example with `required: false`, applying a default value of `2`:
-```
-http_version = ProcessSettings['frontend', 'http_version', required: false] || 2
-```
+TODO: Fill in here how to use the Monitor's instance method to get current settings, how to register for on_change callbacks, etc.
 
 ### Dynamic Settings
 
-The `ProcessSettings::Monitor` loads settings changes dynamically whenever the file changes,
-by using the [listen](https://github.com/guard/listen) gem which in turn uses the `INotify` module of the Linux kernel, or `FSEvents` on MacOS. There is no need to restart the process or send it a signal to tell it to reload changes.
-
-There are two ways to get access the latest settings from inside the process:
-
-#### Read Latest Setting Through `ProcessSettings[]`
-
-The simplest approach--as shown above--is to read the latest settings at any time through `ProcessSettings[]` (which delegates to `ProcessSettings::Monitor.instance`):
-```
-http_version = ProcessSettings['frontend', 'http_version']
-```
-
-#### Register an `on_change` Callback
-Alternatively, if you need to execute some code when there is a change, register a callback with `ProcessSettings::Monitor#on_change`:
-```
-ProcessSettings::Monitor.instance.on_change do
-  logger.level = ProcessSettings['frontend', 'log_level']
-end
-```
-Note that all callbacks run sequentially on the shared change monitoring thread, so please be considerate!
-
-There is no provision for unregistering callbacks. Instead, replace the `instance` of the monitor with a new one.
-
-## Targeting
-Each settings YAML file has an optional `target` key at the top level, next to `settings`.
-
-If there is no `target` key, the target defaults to `true`, meaning all processes are targeted for these settings. (However, the settings may be overridden by other YAML files. See "Precedence" below.)
-
-### Hash Key-Values Are AND'd
-To `target` on context values, provide a hash of key-value pairs. All keys must match for the target to be met. For example, consider this target hash:
-```
-target:
-  service_name: frontend
-  data_center: AWS-US-EAST-1
-```
-This will be applied in any process that has `service_name == "frontend"` AND is running in `data_center == "AWS-US-EAST-1"`.
-
-### Multiple Values Are OR'd
-Values may be set to an array, in which case the key matches if _any_ of the values matches. For example, consider this target hash:
-```
-target:
-  service_name: [frontend, auth]
-  data_center: AWS-US-EAST-1
-```
-This will be applied in any process that has (`service_name == "frontend"` OR `service_name == "auth"`) AND `data_center == "AWS-US-EAST-1"`.
+In order to load changes dynamically, `ProcessSettings` relies on INotify module of the Linux kernel. On kernels that do not have this module (MacOS for example), you will see a warning on STDERR that changes will not be loaded while the process runs.
 
-### Precedence
-The settings YAML files are always combined in alphabetical order by file path. Later settings take precedence over the earlier ones.
 
 ## Contributions
 

data/bin/combine_process_settings

@@ -32,16 +32,15 @@ def parse_options(argv)
   options.verbose = false
   option_parser = OptionParser.new(argv) do |opt|
     opt.on('-v', '--verbose', 'Verbose mode.') { options.verbose = true }
-    opt.on('-n', '--version=VERSION', 'Set version number.') { |value| options.version = value.to_i unless value.empty? }
     opt.on('-r', '--root_folder=ROOT') { |o| options.root_folder = o }
     opt.on('-o', '--output=FILENAME', 'Output file.') { |o| options.output_filename = o }
     opt.on('-i', '--initial=FILENAME', 'Initial settings file for version inference.') { |o| options.initial_filename = o }
   end
 
-  if option_parser.parse! && options.root_folder && options.output_filename && (options.version || options.initial_filename)
+  if option_parser.parse! && options.root_folder && options.output_filename && (ENV['BUILD_NUMBER'] || options.initial_filename)
     options
   else
-    warn "usage: #{PROGRAM_NAME} -r staging|production -o combined_process_settings.yml [--version=VERSION] [-i initial_combined_process_settings.yml] (-i required if --version= not set)"
+    warn "usage: #{PROGRAM_NAME} -r staging|production -o combined_process_settings.yml [-i initial_combined_process_settings.yml] (required if BUILD_NUMBER not set)"
     option_parser.summarize(STDERR)
     exit(1)
   end
@@ -83,7 +82,7 @@ options = parse_options(ARGV.dup)
 
 combined_settings = read_and_combine_settings(Pathname.new(options.root_folder) + SETTINGS_FOLDER)
 
-version_number = options.version || default_version_number(options.initial_filename)
+version_number = ENV['BUILD_NUMBER']&.to_i || default_version_number(options.initial_filename)
 combined_settings << end_marker(version_number)
 
 yaml = combined_settings.to_yaml

data/bin/diff_process_settings

@@ -34,6 +34,6 @@ system("rm -f tmp/combined_process_settings-A.yml tmp/combined_process_settings-
 system("sed '/^- meta:$/,$d' #{input_files[0]} > tmp/combined_process_settings-A.yml")
 system("sed '/^- meta:$/,$d' #{input_files[1]} > tmp/combined_process_settings-B.yml")
 
-system("diff -c tmp/combined_process_settings-A.yml tmp/combined_process_settings-B.yml | sed '1,3d'")
+system("diff tmp/combined_process_settings-A.yml tmp/combined_process_settings-B.yml")
 
 system("rm -f tmp/combined_process_settings-A.yml tmp/combined_process_settings-B.yml")

data/bin/process_settings_for_services

@@ -107,7 +107,7 @@ if correlation_scorecard.any? { |_correllation_key, scorecard| scorecard[true].a
     system("rm -f tmp/old.yml tmp/new.yml")
     File.write("tmp/old.yml", ProcessSettings.plain_hash(best_correlation.last['__changes__']['old']).to_yaml)
     File.write("tmp/new.yml", ProcessSettings.plain_hash(best_correlation.last['__changes__']['new']).to_yaml)
-    STDOUT << `diff -c tmp/old.yml tmp/new.yml | sed '1,3d'`
+    STDOUT << `diff tmp/old.yml tmp/new.yml`
   else
     puts "No best correlation found?"
   end

data/lib/process_settings/hash_path.rb

@@ -2,24 +2,13 @@
 
 module ProcessSettings
   # Module for mixing into `Hash` or other class with `[]` that you want to be able to index
-  # with a hash path like:  hash.mine('honeypot', 'answer_odds')
+  # with a hash path like:  hash['app.service_name' => 'frontend']
   module HashPath
-    # returns the value found at the given path
-    # if the path is not found:
-    #   if a block is given, it is called and its value returned (may also raise an exception from the block)
-    #   else, the not_found_value: is returned
-    def mine(*path_array, not_found_value: nil)
-      path_array.is_a?(Enumerable) && path_array.size > 0 or raise ArgumentError, "path must be 1 or more keys; got #{path_array.inspect}"
-      path_array.reduce(self) do |hash, key|
-        if hash.has_key?(key)
-          hash[key]
-        else
-          if block_given?
-            break yield
-          else
-            break not_found_value
-          end
-        end
+    def [](key)
+      if key.is_a?(Hash)
+        HashPath.hash_at_path(self, key)
+      else
+        super
       end
     end
 
@@ -42,6 +31,24 @@ module ProcessSettings
           hash[path]
         end
       end
+
+      def set_hash_at_path(hash, path)
+        path.is_a?(Hash) or raise ArgumentError, "got unexpected non-hash value (#{hash[path]}"
+        case path.size
+        when 0
+          hash
+        when 1
+          path_key, path_value = path.first
+          if path_value.is_a?(Hash)
+            set_hash_at_path(remaining_hash, remaining_path)
+          else
+            hash[path_key] = path_value
+          end
+        else
+          raise ArgumentError, "path may have at most 1 key (got #{path.inspect})"
+        end
+        hash
+      end
     end
   end
 end

data/lib/process_settings/monitor.rb

@@ -7,8 +7,6 @@ require 'listen'
 require 'active_support'
 
 module ProcessSettings
-  class SettingsPathNotFound < StandardError; end
-
   class Monitor
     attr_reader :file_path, :min_polling_seconds, :logger
     attr_reader :static_context, :untargeted_settings, :statically_targeted_settings
@@ -28,8 +26,8 @@ module ProcessSettings
 
       path = File.dirname(@file_path)
 
-      @listener = file_change_notifier.to(path) do |modified, added, _removed|
-        if modified.include?(@file_path) || added.include?(@file_path)
+      @listener = file_change_notifier.to(path) do |modified, _added, _removed|
+        if modified.include?(@file_path)
           @logger.info("ProcessSettings::Monitor file #{@file_path} changed. Reloading.")
           load_untargeted_settings
 
@@ -56,10 +54,7 @@ module ProcessSettings
     end
 
     # Assigns a new static context. Recomputes statically_targeted_settings.
-    # Keys must be strings or integers. No symbols.
     def static_context=(context)
-      self.class.ensure_no_symbols(context)
-
       @static_context = context
 
       load_statically_targeted_settings(force_retarget: true)
@@ -67,32 +62,20 @@ module ProcessSettings
 
     # Returns the process settings value at the given `path` using the given `dynamic_context`.
     # (It is assumed that the static context was already set through static_context=.)
-    # If nothing set at the given `path`:
-    #   if required, raises SettingsPathNotFound
-    #   else returns nil
-    def targeted_value(*path, dynamic_context:, required: true)
+    # Returns `nil` if nothing set at the given `path`.
+    def targeted_value(path, dynamic_context)
       # Merging the static context in is necessary to make sure that the static context isn't shifting
       # this can be rather costly to do every time if the dynamic context is not changing
       # TODO: Warn in the case where dynamic context was attempting to change a static value
       # TODO: Cache the last used dynamic context as a potential optimization to avoid unnecessary deep merges
       full_context = dynamic_context.deep_merge(static_context)
-      result = statically_targeted_settings.reduce(:not_found) do |latest_result, target_and_settings|
+      statically_targeted_settings.reduce(nil) do |result, target_and_settings|
         # find last value from matching targets
         if target_and_settings.target.target_key_matches?(full_context)
-          if (value = target_and_settings.process_settings.json_doc.mine(*path, not_found_value: :not_found)) != :not_found
-            latest_result = value
+          unless (value = HashPath.hash_at_path(target_and_settings.process_settings, path)).nil?
+            result = value
           end
         end
-        latest_result
-      end
-
-      if result == :not_found
-        if required
-          raise SettingsPathNotFound, "no settings found for path #{path.inspect}"
-        else
-          nil
-        end
-      else
         result
       end
     end
@@ -141,20 +124,6 @@ module ProcessSettings
         @logger = new_logger
         Listen.logger = new_logger
       end
-
-      def ensure_no_symbols(value)
-        case value
-        when Symbol
-          raise ArgumentError, "symbol value #{value.inspect} found--should be String"
-        when Hash
-          value.each do |k, v|
-            k.is_a?(Symbol) and raise ArgumentError, "symbol key #{k.inspect} found--should be String"
-            ensure_no_symbols(v)
-          end
-        when Array
-          value.each { |v| ensure_no_symbols(v) }
-        end
-      end
     end
 
     # Calls all registered on_change callbacks. Rescues any exceptions they may raise.

data/lib/process_settings/replace_versioned_file.rb

@@ -5,48 +5,31 @@ require_relative 'targeted_settings'
 module ProcessSettings
   # This class will override a file with a higher version file; it accounts for minor version number use
   module ReplaceVersionedFile
-    class SourceVersionOlderError < StandardError; end
-    class FileDoesNotExistError < StandardError; end
-
     class << self
-      # Contracts
-      #   source_file_path must be present
-      #   destination_file_path must be present
-      #   source_file_path must exist on filesystem
-      #   source file version cannot be older destination version
       def replace_file_on_newer_file_version(source_file_path, destination_file_path)
         source_file_path.to_s      == '' and raise ArgumentError, "source_file_path not present"
         destination_file_path.to_s == '' and raise ArgumentError, "destination_file_path not present"
-        File.exist?(source_file_path) or raise FileDoesNotExistError, "source file '#{source_file_path}' does not exist"
-        validate_source_version_is_not_older(source_file_path, destination_file_path)
 
         if source_version_is_newer?(source_file_path, destination_file_path)
-          FileUtils.cp(source_file_path, destination_file_path)
+          FileUtils.mv(source_file_path, destination_file_path)
+        elsif source_file_path != destination_file_path # make sure we're not deleting destination file
+          if File.exist?(source_file_path)
+            FileUtils.remove_file(source_file_path) # clean up, remove left over file
+          end
         end
       end
 
       private
 
       def source_version_is_newer?(source_file_path, destination_file_path)
-        if File.exist?(destination_file_path)
-          source_version      = ProcessSettings::TargetedSettings.from_file(source_file_path, only_meta: true).version
-          destination_version = ProcessSettings::TargetedSettings.from_file(destination_file_path, only_meta: true).version
-
-          source_version.to_f > destination_version.to_f
-        else
-          true
-        end
-      end
-
-      def validate_source_version_is_not_older(source_file_path, destination_file_path)
-        if File.exist?(destination_file_path)
-          source_version      = ProcessSettings::TargetedSettings.from_file(source_file_path, only_meta: true).version
-          destination_version = ProcessSettings::TargetedSettings.from_file(destination_file_path, only_meta: true).version
-
-          if source_version.to_f < destination_version.to_f
-            raise SourceVersionOlderError,
-                  "source file '#{source_file_path}' is version #{source_version}"\
-                  " and destination file '#{destination_file_path}' is version #{destination_version}"
+        if File.exist?(source_file_path)
+          if File.exist?(destination_file_path)
+            source_version      = ProcessSettings::TargetedSettings.from_file(source_file_path, only_meta: true).version
+            destination_version = ProcessSettings::TargetedSettings.from_file(destination_file_path, only_meta: true).version
+
+            Gem::Version.new(source_version) > Gem::Version.new(destination_version)
+          else
+            true
           end
         end
       end

data/lib/process_settings/settings.rb

@@ -21,5 +21,9 @@ module ProcessSettings
     def eql?(rhs)
       self == rhs
     end
+
+    def [](key)
+      @json_doc[key]
+    end
   end
 end

data/lib/process_settings/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ProcessSettings
-  VERSION = '0.4.0.pre.11'
+  VERSION = '0.4.0.1'
 end

data/lib/process_settings.rb

@@ -7,8 +7,8 @@ require 'process_settings/monitor'
 
 module ProcessSettings
   class << self
-    def [](*path, dynamic_context: {}, required: true)
-      Monitor.instance.targeted_value(*path, dynamic_context: dynamic_context, required: required)
+    def [](value, dynamic_context = {})
+      Monitor.instance.targeted_value(value, dynamic_context)
     end
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: process_settings
 version: !ruby/object:Gem::Version
-  version: 0.4.0.pre.11
+  version: 0.4.0.1
 platform: ruby
 authors:
 - Invoca
@@ -96,9 +96,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">"
+  - - ">="
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
 rubyforge_project: 
 rubygems_version: 2.6.13