cloud-config 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 22afde03ef83a2749759d80cbdd646a070348effbd70fb7311c92a52dae05869
-  data.tar.gz: 16737f0d99ddc1a3bffdf0cad9be1525558efab1565cd21a9552124c1217d838
+  metadata.gz: a5d99c6f38f07031a7da456fa285c345a87ed6f124a534d2b5fe004540743e85
+  data.tar.gz: cba8c49a6fcc40d3cfac3f451826a3fddaaf6a0d8a7271033fbc7e833dc47cbb
 SHA512:
-  metadata.gz: 1bae934a5af84caeb67f8ba48b9eef08e3f7990ebe26bd577089da7f5d22a253044c499c7c4da0ac4ef92421c934830db4a2b5f21ba52da3892aeb4132eb2a45
-  data.tar.gz: 4d528c9e6b3c3e46919bc035424de7e572e87f0eedaaa8a29d38f9aa7fe325f72246ca1b45dc7803ba252c1523d6284f37015e1f3d80e2fe5918ea8838fa7e5d
+  metadata.gz: b883d58ca786602fe85135a77a31959c00dcd7d3af7534e63889320fb7d1d739285bb048fa6f2c72fb309c8ecf9ed4bc95da513be39a9d073a14ec768c4cb7c7
+  data.tar.gz: 0d61a08ecbba70c9ae743cf8fa84c68772da1a6a3566716fb9f17ed564472150ef5c6dcbc38e0f6b9722437327feef6cac85c6aca02e1bcd2c49ab0812c003fb

data/CHANGELOG.md

@@ -1,5 +1,18 @@
 ## [Unreleased]
 
-## [0.1.0] - 2023-01-15
+## [0.2.0] - 2023-05-20
+
+Bug fixes:
+
+- Fix in memory cache expiry
+
+Enhancements:
+
+- Reset cache for single key
+- Lookup providers by configuration key
+
+## [Released]
+
+## [0.1.0] - 2023-05-17
 
 - Initial release

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    cloud-config (0.1.0)
+    cloud-config (0.2.0)
       parallel
 
 GEM

data/README.md

@@ -2,7 +2,9 @@
 
 A library to modernise the way applications fetch configuration. Typically an application will use environment variables and settings files to store and retrieve configurations, but there are many issues with this approach. Often environment variables create a security risk and settings files hard code infrastructure dependencies into the code. Changing configuration settings will usually require redeploying large parts of the infrastructure and perhaps even need to go through the application deployment lifecycle.
 
-A modern approach stores configuration remotely, often using key/value databases. Storing configurations in a single database, separately from the codebase reduces infrastructure dependency. Configuration updates can automatically sync with any application, without requiring redeployments. Security risk is greatly reduced, since the configurations can be securely stored. Another goal with this approach is creating an environmentless codebase. The application no longer needs to know which environment it's running in, since all the configuration is handled by the infrastucture.
+A modern approach stores configuration remotely, often using key/value databases. Storing configurations in a single database, separately from the codebase reduces infrastructure dependency. Configuration updates can automatically sync with any application, without requiring redeployments. Security risk is greatly reduced, since configurations can be securely stored. Another goal with this approach is creating an environmentless codebase. The application no longer needs to know which environment it's running in, since all the configuration is handled by the infrastucture.
+
+Another common problem is password rotation. A typical application will need to be restarted or even redeployed when the configuration settings change, such as rotating passwords. CloudConfig can handle this elegantly, improving the uptime and resilience of the application.
 
 ## Installation
 
@@ -16,7 +18,7 @@ If bundler is not being used to manage dependencies, install the gem by executin
 
 ## Usage
 
-CloudConfig can be configured to fetch keys from multiple providers. Since CloudConfig will need to know which provider has the correct key, all the keys will need to be preconfigured. An example configuration may look like
+CloudConfig can be configured to fetch keys from multiple providers. Since CloudConfig will need to know which provider has the correct key, all the keys will need to be preconfigured. Duplicate keys will be overriden with the latest provider. An example configuration may look like
 
 ```ruby
 CloudConfig.configure do
@@ -27,6 +29,18 @@ CloudConfig.configure do
 end
 ```
 
+Fetch a setting with
+
+```ruby
+url = CloudConfig.get(:api_url)
+```
+
+If caching has been configured, the cache can be reset using
+
+```ruby
+url = CloudConfig.get(:api_url, reset_cache: true)
+```
+
 ### Configuration Options
 
 ```ruby
@@ -66,7 +80,7 @@ end
 
 ### Preloading
 
-CloudConfig can preload all the keys configured for preload enabled providers. A cache client must be configured, otherwise
+CloudConfig can preload all the keys configured for preload enabled providers. A cache client must also be configured, otherwise
 preloading won't do anything.
 
 ```ruby
@@ -74,6 +88,7 @@ CloudConfig.configure do
   cache_client CloudConfig::Cache::InMemory.new
 
   provider :in_memory, preload: true do
+    setting :key1
   end
 end
 ```
@@ -84,9 +99,77 @@ Call preload to cache all the keys.
 CloudConfig.preload
 ```
 
+### Custom Providers
+
+CloudConfig comes equipped with a set of providers for fetching configuration from remote datastores. The limit set of providers will not be sufficient for most real world applications, so creating custom providers will be necessary. There is a very simple interface consisting of 3 methods necessary for creating a custom provider.
+
+```ruby
+class CustomProvider
+  # Define initialize with a params argument.
+  def initialize(params = {}); end
+
+  # Define `get` for fetching keys from the remote datastore
+  def get(key); end
+
+  # Define `set` for storing keys in the remote datastore
+  def set(key, value); end
+end
+```
+
+Specify the class when configuring the custom provider.
+
+```ruby
+CloudConfig.configure do
+  provider :custom_provider, provider_class: CustomProvider do
+  end
+end
+```
+
+### Custom Cache
+
+Defining a custom cache client will require an interface of 3 methods.
+
+```ruby
+class CustomCache
+  # Define `key?` for checking whether the key exists in the cache client.
+  # This check allows nil values to be cached.
+  def key?(key); end
+
+  # Define `get` for fetching keys from the cache
+  def get(key); end
+
+  # Define `set` for storing keys in the cache
+  def set(key, value); end
+end
+```
+
+Configure CloudConfig to use the cache in the usual way
+
+```ruby
+CloudConfig.configure do
+  cache_client CustomCache.new
+end
+```
+
+## Connection Pooling
+
+CloudConfig does not do any connection pooling. It is the responsibility of the application to handle connection pooling. For example, Rails handles its own connection pools, so CloudConfig will not attempt to interfere with the pool.
+
+## Examples
+
+There are some example configurations in the [examples](examples) folder. In the terminal, execute
+
+    $ examples/001_parameter_store.rb
+
+These examples make use of AWS infrastructure, so make sure the AWS enviroment variables have been correctly configured.
+
+    $ export AWS_ACCESS_KEY_ID=
+    $ export AWS_SECRET_ACCESS_KEY=
+    $ export AWS_REGION=
+
 ## Development
 
-It's recommended to use Docker when working with the library. Assuming Docker is installed, run
+It's recommended to use Docker when working with this library. Assuming Docker is installed, run
 
     $ docker-compose run config /bin/bash
 

data/cloud-config.gemspec

@@ -9,16 +9,19 @@ Gem::Specification.new do |spec|
   spec.email = ['hypernova2002@gmail.com']
 
   spec.summary = 'A library for dynamically reloading settings from cloud services'
-  # spec.description = "TODO: Write a longer description or delete this line."
-  # spec.homepage = "TODO: Put your gem's website or public repo URL here."
+  spec.description = <<~DOC
+        Modernise applications by fetching and caching remote settings. No longer store settings
+    in the codebase or as environment variables.
+  DOC
+  spec.homepage = 'http://github.com/hypernova2002/cloud-config'
   spec.license = 'MIT'
   spec.required_ruby_version = '>= 3.1.0'
 
   # spec.metadata["allowed_push_host"] = "TODO: Set to your gem server 'https://example.com'"
 
-  # spec.metadata["homepage_uri"] = spec.homepage
-  # spec.metadata["source_code_uri"] = "TODO: Put your gem's public repo URL here."
-  # spec.metadata["changelog_uri"] = "TODO: Put your gem's CHANGELOG.md URL here."
+  spec.metadata['homepage_uri'] = spec.homepage
+  spec.metadata['source_code_uri'] = spec.homepage
+  spec.metadata['changelog_uri'] = "#{spec.homepage}/CHANGELOG.md"
 
   # Specify which files should be added to the gem when it is released.
   # The `git ls-files -z` loads the files in the RubyGem that have been added into git.

data/examples/004_handling_configuration_changes.rb

@@ -0,0 +1,45 @@
+#! /usr/bin/env ruby
+
+require 'bundler/inline'
+
+gemfile do
+  source 'https://rubygems.org'
+
+  gem 'faraday'
+
+  gem 'cloud-config', path: '..'
+end
+
+require 'faraday'
+require 'cloud-config/cache/in_memory'
+require 'cloud-config/providers/in_memory'
+
+CloudConfig.configure do
+  cache_client CloudConfig::Cache::InMemory.new
+
+  provider :in_memory, preload: true do
+    setting :url, cache: 60
+  end
+end
+
+CloudConfig.set(:url, 'https://httpstat.us/503')
+
+url = CloudConfig.get(:url)
+
+# Mimick a configuration change
+CloudConfig.providers_by_key[:url].provider.set(:url, 'https://httpstat.us/200')
+
+req = Faraday.new do |f|
+  f.response :raise_error
+  f.response :logger
+end
+
+begin
+  req.get(url)
+rescue Faraday::ServerError
+  url = CloudConfig.get(:url, reset_cache: true)
+
+  sleep 1
+
+  retry
+end

data/lib/cloud-config/cache/in_memory.rb

@@ -55,7 +55,7 @@ module CloudConfig
       # @param value [Object] Value of the key
       # @option options [Hash] :expire_in Time in seconds until key expires
       def set(key, value, options = {})
-        expire_in = options.fetch(:expire_in) { DEFAULT_EXPIRE }
+        expire_in = options[:expire_in] || DEFAULT_EXPIRE
 
         expire_at = Time.now + expire_in
 

data/lib/cloud-config/cache.rb

@@ -30,7 +30,7 @@ module CloudConfig
       #
       # @return [Object] Value of the key
       def with_cache(key, options = {})
-        return cache.get(key) if cache&.key?(key)
+        return cache.get(key) if !options[:reset_cache] && cache&.key?(key)
 
         value = yield
 

data/lib/cloud-config/providers/aws_parameter_store.rb

@@ -34,10 +34,6 @@ module CloudConfig
       def set(key, value)
         client.put_parameter(name: key, value:)
       end
-
-      # def client
-      #   @client ||= Aws::SSM::Client.new
-      # end
     end
   end
 end

data/lib/cloud-config/providers.rb

@@ -14,6 +14,10 @@ module CloudConfig
       #   @return [Hash<Symbol,ProviderConfig>]
       attr_reader :providers
 
+      # @!attribute [r] Fetch provider configuration by setting key
+      #   @return [Hash<Symbol,ProviderConfig>]
+      attr_reader :providers_by_key
+
       # Add a provider to the list of provider configurations.
       #
       # @param provider_name [Symbol] Name of the provider
@@ -25,6 +29,18 @@ module CloudConfig
         provider_config.instance_eval(&blk) if blk
         @providers ||= {}
         @providers[provider_name] = provider_config
+
+        update_provider_keys(provider_config)
+      end
+
+      # Update {#providers_by_key} with settings from the provider config.
+      #
+      # @param provider_config [ProviderConfig] Provider config containing the keys
+      def update_provider_keys(provider_config)
+        @providers_by_key ||= {}
+        provider_config.settings.each_key do |key|
+          @providers_by_key[key] = provider_config
+        end
       end
     end
   end

data/lib/cloud-config/version.rb

@@ -2,5 +2,5 @@
 
 module CloudConfig
   # Version of CloudConfig
-  VERSION = '0.1.0'
+  VERSION = '0.2.0'
 end

data/lib/cloud-config.rb

@@ -35,14 +35,15 @@ module CloudConfig
   # Fetch the value of a key using the appropriate provider.
   #
   # @param key [String,Symbol] Key to lookup
+  # @param reset_cache [Boolean] Whether the cache for the key should be reset
   #
   # @return [Object] Value of the key
-  def get(key)
-    provider_config = providers.values.find { |provider| provider.settings.key?(key) }
+  def get(key, reset_cache: false)
+    provider_config = providers_by_key[key]
 
     raise MissingKey, 'Key not found' if provider_config.nil?
 
-    load_key(provider_config, key)
+    load_key(provider_config, key, reset_cache:)
   end
 
   # Set the value of a key with the configured provider.
@@ -50,7 +51,7 @@ module CloudConfig
   # @param key [String,Symobl] Key to update
   # @param value [Object] Value of key
   def set(key, value)
-    provider_config = providers.values.find { |provider| provider.settings.key?(key) }
+    provider_config = providers_by_key[key]
 
     raise MissingKey, 'Key not found' if provider_config.nil?
 
@@ -84,10 +85,11 @@ module CloudConfig
   #
   # @param provider_config [CloudConfig::ProviderConfig] provider configuration
   # @param key [String,Symbol] Key to fetch
+  # @param reset_cache [Boolean] Whether the cache for the key should be reset
   #
   # @return [Object] Value of the key
-  def load_key(provider_config, key)
-    with_cache(key, expire_in: provider_config.settings[key][:cache]) do
+  def load_key(provider_config, key, reset_cache: false)
+    with_cache(key, reset_cache:, expire_in: provider_config.settings[key][:cache]) do
       provider_config.provider.get(key)
     end
   end
@@ -95,6 +97,9 @@ module CloudConfig
   # Reset the {CloudConfig} configuration
   def reset!
     @cache = nil
-    @provider = nil
+    @providers = {}
+    @providers_by_key = {}
   end
 end
+
+CloudConfig.reset!

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: cloud-config
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - hypernova2002
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-05-17 00:00:00.000000000 Z
+date: 2023-05-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: parallel
@@ -206,7 +206,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description: 
+description: |2
+      Modernise applications by fetching and caching remote settings. No longer store settings
+  in the codebase or as environment variables.
 email:
 - hypernova2002@gmail.com
 executables: []
@@ -228,6 +230,7 @@ files:
 - examples/001_parameter_store.rb
 - examples/002_parameter_store_with_cache.rb
 - examples/003_parameter_store_with_preload.rb
+- examples/004_handling_configuration_changes.rb
 - lib/cloud-config.rb
 - lib/cloud-config/cache.rb
 - lib/cloud-config/cache/in_memory.rb
@@ -241,10 +244,13 @@ files:
 - lib/cloud-config/providers/yaml_file.rb
 - lib/cloud-config/version.rb
 - sig/cloud-config.rbs
-homepage: 
+homepage: http://github.com/hypernova2002/cloud-config
 licenses:
 - MIT
 metadata:
+  homepage_uri: http://github.com/hypernova2002/cloud-config
+  source_code_uri: http://github.com/hypernova2002/cloud-config
+  changelog_uri: http://github.com/hypernova2002/cloud-config/CHANGELOG.md
   rubygems_mfa_required: 'true'
 post_install_message: 
 rdoc_options: []