consult 0.6.0 → 0.7.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA256:
-  metadata.gz: aa590a02bf4aceb2c3b04e21c5737d1b7e86160db8cb1251c79dcf612eba6e9f
-  data.tar.gz: 29e80de8a03c8c52eb156dcabaed67b43bad22997e6c3532994d177afce5d4b0
+SHA1:
+  metadata.gz: 1e1427b4b6792bfd9723be2d22edacda36e0349e
+  data.tar.gz: a993feb5698c03865b75ce377ea0c8222d256672
 SHA512:
-  metadata.gz: 8d6aebe85871c232f97e9b760c811b18e98488149214fe9c7b3f618b601e69323ba6f2c30ff270afd6d68a5783492407cbc1446e455efccc1d031c0a880a3a44
-  data.tar.gz: 532d19aaf161adb255f1179958814d03f1d9ced811c4405dcd2f66244589aa58c08e83583b84720feb6a8d6187f3a991076d7d4524c81b27bcddaf38716354e3
+  metadata.gz: b9e9b8cef692b5713baaa1cbd0267c00041ac17e2499c3a5b217d816abdb869ff14e05f36ad218c10ca23cb6cf28ab66c0a33244ab095a976591262e8de777d4
+  data.tar.gz: 5488c26112399997e8818345aad315a711b5cd09759bbbeaba2e076846bae36a0795def1af07293c4e6616a15a501e5023e3494713f346158e1439c490186362

data/CHANGELOG.md

@@ -1,7 +1,12 @@
-# 0.6.0
+#### 0.7.0
+
+* consult.yml configuration structure has changed, to enable environment specific configuration blocks (see readme for example)
+* Improve safety around edge cases
+
+#### 0.6.0
 
 * Fixed tests to account for versioned key-value stores in Vault 0.10+
 
-# 0.5.0
+#### 0.5.0
 
 Initial release.

data/README.md

@@ -1,14 +1,18 @@
 # Consult
 
+Generate configuration and secrets for Rails apps automatically from [Consul](https://github.com/hashicorp/consul) & [Vault](https://github.com/hashicorp/vault).
+
 [![Gem Version](https://badge.fury.io/rb/consult.svg)](https://badge.fury.io/rb/consult)
 [![Build Status](https://travis-ci.org/veracross/consult.svg?branch=master)](https://travis-ci.org/veracross/consult)
 [![Maintainability](https://api.codeclimate.com/v1/badges/d7b048b7edd9f27c83b9/maintainability)](https://codeclimate.com/github/veracross/consult/maintainability)
 
-Render configuration and secrets from [Consul] and [Vault] with ERB templates.
+## Background
+
+This gem is a spiritual sibling to [Consul Template](https://github.com/hashicorp/consul-template), but specifically intended for use in Ruby/Rails environments. It does not have the same features as Consul Template; it is intended for simpler scenarios. Most importantly, leases and configuration changes are _not_ watched to automatically re-render. Consult is intended for more static or medium-to-long lived application configuration.
 
-This gem is a spiritual sibling to [Consul Template], but specifically intended for use in Ruby or Rails environments. It does not have the same features as Consul Template; it is intended for simpler scenarios. Most importantly, leases and configuration changes are _not_ watched to automatically re-render. Consult is intended for more static or medium-to-long lived configuration.
+We use Consul Template for server level configuration, but application level configuration is more tricky. It is difficult to solve the problem of fetching configuration and secrets in a consistent way in development, staging, and production. For example, we wanted to avoid having Consul Template used in production, but some other custom solution in development.
 
-If this gem is included in a Rails, the template will render on Rails boot. Because of this, configuration or credential changes can be picked up by restarting your app.
+With Consult the process is the same in all environments.
 
 This gem is considered _beta_. At Veracross, we are just beginning to use it in staging environments.
 
@@ -30,63 +34,73 @@ Or install it yourself as:
 
 ## Usage
 
-Using Consult requires a configuration YAML file and a series of templates. The configuration file serves as a manifest of templates and their settings, along with optional connection settings to Vault and Consul.
+Using Consult requires a configuration YAML file and a series of template files. The configuration file serves as a manifest of templates and their settings, along with optional connection settings to Vault and Consul.
+
+Pre-existing copies of files generated by Consult (such as `secrets.yml`, `database.yml`, etc) should be removed from your app's source control and added to your `.gitignore`. Only keep your templates in source control, not the generated files!
+
+If this gem is included in a Rails, the templates will render on Rails boot. Configuration or credential changes can be picked up by restarting your app.
 
 ### Configuration
 
 ```yaml
-# The environment you are operating it. Defaults to ENV['RAILS_ENV'] or Rails.env if Rails is present.
+# Optional; Consult will render this specific environment, if set
+# Defaults to ENV['RAILS_ENV'] or Rails.env if Rails is present
 env: test
 
-# Optional
-consul:
-  # Prefers `CONSUL_HTTP_ADDR` environment variable
-  address: http://0.0.0.0:8500
-  # Prefers `CONSUL_HTTP_TOKEN` environment variable, or a ~/.consul-token file.
-  # Setting a token here is not best practice because consul tokens should have a relatively short TTL
-  # and be read from the environment, but this is convenient for testing.
-  token: 5d3f1c66-d405-4ad1-b634-ea30be4fb539
-
-# Optional
-vault:
-  # Prefers `VAULT_ADDR` environment variable
-  address: http://0.0.0.0:8200
-  # Prefers `VAULT_TOKEN` environment variable, or a ~/.vault-token file
-  # Setting a token here is not best practice because vault tokens should have a relatively short TTL
-  # and be read from the environment, but this is convenient for testing.
-  token: 8fcd5aed-3eb9-412d-8923-1397af7aede2
-
-# Enumerate the templates.
-templates:
-  database:
-    # Relative paths are assumed to be in #{Rails.root}.
-    # Path to the template
-    path: config/templates/database.yml.erb
-    # Destination for the rendered template
-    dest: config/database.yml
-    # Which environments to render this template in
-    environments: all
-    # If the file is less than this old, do not re-render
-    ttl: 3600 # seconds
-
-  secrets:
-    path: config/templates/secrets.yml.erb
-    dest: config/secrets.yml
-    environments: test
-
-  should_be_excluded:
-    path: config/templates/fake.yml.erb
-    dest: config/fake.yml
-    environments: production # won't be rendered because it doesn't match `env` at the top
-```
-
-### Conventions
+# "shared" is the base configuration used for all environments by default
+# note: you do NOT need to use yaml merge syntax to have shared configuration included for specific environments
+shared:
+  # Optional
+  consul:
+    # Prefers `CONSUL_HTTP_ADDR` environment variable
+    address: http://0.0.0.0:8500
+    # Prefers `CONSUL_HTTP_TOKEN` environment variable, or a ~/.consul-token file.
+    # Setting a token here is not best practice because consul tokens should have a relatively short TTL
+    # and be read from the environment, but this is convenient for testing.
+    token: 5d3f1c66-d405-4ad1-b634-ea30be4fb539
+
+  # Optional
+  vault:
+    # Prefers `VAULT_ADDR` environment variable
+    address: http://0.0.0.0:8200
+    # Prefers `VAULT_TOKEN` environment variable, or a ~/.vault-token file
+    # Setting a token here is not best practice because vault tokens should have a relatively short TTL
+    # and be read from the environment, but this is convenient for testing.
+    token: 8fcd5aed-3eb9-412d-8923-1397af7aede2
+
+  # Enumerate the templates.
+  templates:
+    database:
+      # Relative paths are assumed to be in #{Rails.root}.
+      # Path to the template
+      path: config/templates/database.yml
+      # Destination for the rendered template
+      dest: config/database.yml
+      # If the file is less than this old, do not re-render
+      ttl: 3600 # seconds
+
+# environment specific configuration
+# NOTE: environment keys will be deep merged with the "shared" configuration
+test:
+  templates:
+    secrets:
+      path: config/templates/secrets.yml
+      dest: config/secrets.yml
 
-Because you can use this to fetch secrets or render out things like `database.yml`, you should delete `secrets.yml` and `database.yml` from your app and add them to `.gitignore`. Only keep your templates in source control.
+production:
+  # example: override vault token in production
+  vault:
+    token: 1397af7aede2-8923-412d-3eb9-8fcd5aed
+  templates:
+    # excluded from non-production environments
+    should_be_excluded:
+      path: config/templates/fake.yml
+      dest: config/fake.yml
+```
 
 ### Templates
 
-Templates are ERB files, and as such can do anything ERB can do. However, Consult does provide a few helper functions.
+Templates files are processed with ERB. As such, they can do anything ERB can do. Consult also provides a few helper functions.
 
 Note that under the hood, Consult is using [Diplomat](https://github.com/WeAreFarmGeek/diplomat) and the [Vault Gem](https://github.com/hashicorp/vault-ruby). Consul objects are therefore Diplomat objects, and likewise Vault objects are Vault Gem objects. See their API docs for more information. Diplomat generally returns structs with title cased properties.
 
@@ -203,10 +217,10 @@ password: <%= s.data[:password] %>
 
 #### More Full Examples
 
-Render multiple servers into a database.yml file, keyed by their name.
+Render multiple servers into a `database.yml` file, keyed by their name.
 
 ```yml
-# database.yml.erb
+# database.yml
 <% service("postgres").each do |node| %>
 '<%= node.Node %>':
   host: <%= node.Address %>
@@ -240,7 +254,7 @@ Yields something like
 #### Secrets
 
 ```yml
-# secrets.yml.erb
+# secrets.yml
 shared:
   rollbar_token: <%= secret('secrets/third_party').data[:rollbar] %>
   scout_token: <%= secret('secrets/third_party').data[:scout] %>
@@ -261,12 +275,6 @@ Rollbar.configure do |config|
 end
 ```
 
-## Why we built this
-
-We use [Consul Template] for server-level configuration, but application level configuration is more tricky. It is difficult to solve the problem of fetching secrets and config in a consistent way in both development and production. We wanted to avoid having [Consul Template] in use in production, but some other custom solution in development.
-
-Using this gem, the implementation is the same in dev and prod, and it is frictionless since the templates render when Rails boots.
-
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment. See below for testing instructions.
@@ -288,7 +296,3 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/veracr
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
-
-[Vault]: https://github.com/hashicorp/vault
-[Consul]: https://github.com/hashicorp/consul
-[Consul Template]: https://github.com/hashicorp/consul-template

data/lib/consult.rb

@@ -23,9 +23,12 @@ module Consult
     def load(config_dir: nil)
       root directory: config_dir
       yaml = root.join('config', 'consult.yml')
-      @config = yaml.exist? ? YAML.safe_load(ERB.new(yaml.read).result, [], [], true) : {}
-      @config.deep_symbolize_keys!
-      @templates = @config[:templates]&.map { |name, config| Template.new(name, config) }
+
+      @all_config = yaml.exist? ? YAML.safe_load(ERB.new(yaml.read).result, [], [], true).to_h : {}
+      @all_config.deep_symbolize_keys!
+
+      @config = @all_config[:shared].to_h.deep_merge @all_config[env&.to_sym].to_h
+      @templates = @config[:templates]&.map { |name, config| Template.new(name, config) } || []
 
       configure_consul
       configure_vault
@@ -58,11 +61,11 @@ module Consult
     end
 
     def root(directory: nil)
-      @_root ||= directory ? Pathname.new(directory) : (!!defined?(Rails) && ::Rails.root)
+      @_root ||= directory ? Pathname.new(directory) : (defined?(::Rails) && ::Rails.root)
     end
 
     def env
-      @config[:env] || ENV['RAILS_ENV'] || Rails.env
+      @all_config[:env] || ENV['RAILS_ENV'] || (defined?(::Rails) && ::Rails.root)
     end
 
     # Return only the templates that are relevant for the current environment

data/lib/consult/template.rb

@@ -34,7 +34,7 @@ module Consult
     end
 
     def should_render?
-      (@config[:environments] == 'all' || @config[:environments].include?(Consult.env)) && expired?
+      expired?
     end
 
     def expired?

data/lib/consult/version.rb

@@ -1,3 +1,3 @@
 module Consult
-  VERSION = '0.6.0'
+  VERSION = '0.7.1'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: consult
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.7.1
 platform: ruby
 authors:
 - Jeff Fraser
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2018-07-09 00:00:00.000000000 Z
+date: 2018-07-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -219,7 +219,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.7.3
+rubygems_version: 2.6.14
 signing_key: 
 specification_version: 4
 summary: Manage consul/vault backed template files in Ruby.