TerraformDevKit 0.1.14 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: a94f0642cf970c5994db6fc663a2f2add47810e4
-  data.tar.gz: 8fa17cf1bf78f83abe26094616904692db12fdf2
+  metadata.gz: 3fdb3ffa0d6129a11bf2307a2e9016bbf82d779e
+  data.tar.gz: f3f531b2f14178df98c4463184ede09081b577bc
 SHA512:
-  metadata.gz: daf674ba9d45403f922f157eccb704ac0581470c801c9d8c86d90b1c3dab5a3899993706931b42712b9994fe3b104d232b8ddb17d816184a887ff5f844ff3285
-  data.tar.gz: 9171fd82c7de355a543053b1fd95bbd7f868ec2cb39547b4dec5faeb6510a820ec929e3931fec624683de7f8f924b694e767e44f20bb8c837708b0708bdbf64a
+  metadata.gz: f396925aa0ae75fe3687d94cc1c0266701e936880585f721cfaf5175c0e38787352f56d7f8bb958030b0bf79340ca884d5f4e9133c1033d659981def38610e8d
+  data.tar.gz: 47a6b8d4e74f95fb23c40a980325161641e7e6ac2dc64962dee09474a88baa52d46270562136d790ab46657fb946db7d548b1cdc2c3ded5ed6181238ee2f47d7

data/.rspec

@@ -1,2 +1,2 @@
---format documentation
---color
+--format documentation
+--color

data/.travis.yml

@@ -1,8 +1,8 @@
-sudo: false
-language: ruby
-rvm:
-  - 2.1.6
-before_install: gem install bundler -v 1.14.6
-os:
-  - linux
-  - osx
+sudo: false
+language: ruby
+rvm:
+  - 2.1.6
+before_install: gem install bundler -v 1.14.6
+os:
+  - linux
+  - osx

data/Gemfile

@@ -1,4 +1,4 @@
-source 'https://rubygems.org'
-
-# Specify your gem's dependencies in TerraformDevKit.gemspec
-gemspec
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in TerraformDevKit.gemspec
+gemspec

data/README.md

@@ -1,200 +1,198 @@
-# TerraformDevKit
-
-[![Build Status](https://travis-ci.org/vistaprint/TerraformDevKit.svg?branch=master)](https://travis-ci.org/vistaprint/TerraformDevKit) [![Build status](https://ci.appveyor.com/api/projects/status/74s4yd7dmuwg5tmn/branch/master?svg=true)](https://ci.appveyor.com/project/Vistaprint/terraformdevkit/branch/master)
-
-Set of scripts to ease development and testing with [Terraform](https://www.terraform.io/).
-
-The script collection includes support for:
-
-* Managing AWS credentials
-* Backing up the state from a failed Terraform execution
-* Executing external commands
-* Simple configuration management
-* Simple reading and writing to AWS DynamoDB
-* Multiplatform tools
-* Making simple HTTP requests
-* Retrying a block of code
-* Terraform environment management
-* Locally installing Terraform and [Terragrunt](https://github.com/gruntwork-io/terragrunt)
-* Filtering Terraform logging messages
-
-Most of these scripts exist to provide support to a module development and testing environment for Terraform: [TerraformModules](https://github.com/vistaprint/TerraformModules). But, they might be useful for other purposes too.
-
-## Installation
-
-Add this line to your application's Gemfile:
-
-```ruby
-gem 'TerraformDevKit'
-```
-
-And then execute:
-
-    $ bundle
-
-Or install it yourself as:
-
-    $ gem install TerraformDevKit
-
-## Usage
-
-To use the library simply import it as:
-
-```ruby
-require 'TerraformDevKit'
-```    
-
-## Managing Terraform Environments
-
-TerraformDevKit provides a set of Rake tasks and Ruby scripts to ease the management of multiple Terraform environments. Three major environment types are supported: `dev`, `test` and `prod`.
-
-There might be many development environments (`dev`), each one with its own name. Development environments use a local Terraform backend. They are intented to be used by developers while adding features to the infrastructure.
-
-Testing (`test`) and production (`prod`) environment use a remote backend. Thus, the Terraform state file is not kept in the local disk, but on S3. This allows multiple developers to easily work on the same infrastructure instance. For safety reasons, operations that affect testing and production environments require manual user input. This is not the case for development environments.
-
-TerraformDevKit expects templated versions (using [Mustache](https://mustache.github.io/)) of the Terraform files. Such files might contain placeholders for several fields such as `Environment`, (AWS) `Region` or (AWS) `Profile`, among others. TerraformDevKit uses the template files to generate the final files that will be consumed by Terraform and Terragrunt. As an example, for the production environment, the resulting files are placed in a directory named `envs/prod`.
-
-### Configuration Files
-
-Configuration files must be placed in a directory named `config`. Each environment type requires a different configuration file. Thus, the following three files must be placed in the `config` directory:
-
-* `config-dev.yml`
-* `config-test.yml`
-* `config-prod.yml`
-
-The first one contains the configuration for **all** the development environments that are created. The other two contain the configuration for the test and production environments, respectively.
-
-A sample configuration files is shown next:
-
-```yaml
-terraform-version: 0.10.8
-terragrunt-version: 0.13.5
-aws:
-  profile: myprofile
-  region: eu-west-1
-```
-
-The AWS profile **must not** be specified for test and production accounts, as users are required to manually type the profile name.
-
-### A Minimal Rakefile
-
-```ruby
-ROOT_PATH = File.dirname(File.expand_path(__FILE__))
-
-spec = Gem::Specification.find_by_name 'TerraformDevKit'
-load "#{spec.gem_dir}/tasks/devkit.rake"
-
-task :custom_test, [:env] do |_, args|
-  # Add some tests here
-end
-```
-
-### Tasks and Hooks
-
-TerraformDevKit provides a set of generic tasks to perform:
-
-* `prepare`: prepares the environment
-* `plan`: shows the plan to create the infrastructure
-* `apply`: creates the infrastructure
-* `destroy`: destroys the infrastructure
-* `clean`: cleans the environment (after destroying the infrastructure)
-* `test`: tests a local environment
-* `preflight`: creates a temporary infrastructure and runs the test task
-
-Additionally, TerraformDevKit allows users to define a set of hooks that will be called during the different steps required to complete the previous list of tasks. The following hooks are available:
-
-* `pre_apply`: invoked before `apply` task runs
-* `post_apply`: invoked after `apply` task runs
-* `pre_destroy`: invoked before `destroy` task runs
-* `post_destroy`: invoked after `destroy` task runs
-* `custom_prepare`: invoked during the preparation process, before terraform is initialized
-* `custom_test`: invoked during as part of the `test` task, right after `apply` completes.
-
-### Sample Terraform/Terragrunt Templates
-
-The following file (`main.tf.mustache`) contains the infrastructure configuration (a single S3 bucket) as well as information related to the AWS provider.
-
-```hcl
-locals {
-  env    = "{{Environment}}"
-}
-
-terraform {
-    backend {{#LocalBackend}}"local"{{/LocalBackend}}{{^LocalBackend}}"s3"{{/LocalBackend}} {}
-}
-
-provider "aws" {
-  profile = "{{Profile}}"
-  region  = "{{Region}}"
-}
-
-resource "aws_s3_bucket" "raw" {
-  bucket = "foo-${local.env}"
-  acl    = "private"
-
-{{#LocalBackend}}
-  force_destroy = true
-{{/LocalBackend}}
-{{^LocalBackend}}
-  lifecycle {
-    prevent_destroy = true
-  }
-{{/LocalBackend}}
-}
-```
-
-An additional file (`terraform.tfvars.mustache`) contains the Terragrunt configuration. Terragrunt is used as it makes it easier to manage a remote backend. While Terraform is able to keep the state file in S3, it does not transparently create the DynamoDB lock table that prevents multiple developers from concurrently modifying the state file. Terragrunt, however, is able to do this without user intervention.
-
-```hcl
-terragrunt = {
-  remote_state {
-  {{#LocalBackend}}
-    backend = "local"
-    config {}
-  {{/LocalBackend}}
-  {{^LocalBackend}}
-    backend = "s3"
-    config {
-      bucket     = "foo-remote-state"
-      key        = "foo-{{Environment}}.tfstate"
-      lock_table = "foo-{{Environment}}-lock-table"
-      encrypt    = true
-      profile    = "{{Profile}}"
-      region     = "{{Region}}"
-    }
-  {{/LocalBackend}}
-  }
-}
-```
-
-### Injecting Additional Variables into Template Files
-
-In addition to the default variables that are passed to Mustache when rendering a template file, users can provide additional variables. To do so, users must register a procedure that receives the environment as a parameter and returns a map with the extra variables and their values. An example is shown next:
-
-```ruby
-TDK::TerraformConfigManager.register_extra_vars_proc(
-  proc do
-    { SumoLogicEndpoint: TDK::Configuration.get('sumologic')['endpoint'] }
-  end
-)
-```
-
-### Updating Modules
-
-Terraform will get the necessary modules every time a new environment is created. Once the modules are cached, there is generally no need to keep updating the modules each time Terraform is executed. When using a module repository it is possible to select a specific version to use (as shown [here](https://www.terraform.io/docs/modules/sources.html#ref)). In such a case, Terraform will automatically update the modules whenever the version number is changed.
-
-When using local modules (e.g., during development process) it might be desirable to update the modules every time Terraform runs. This can be achieved by setting the environment variable `TF_DEVKIT_UPDATE_MODULES` to `true`.
-
-## Development
-
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
-
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
-
-## Contributing
-
-Bug reports and pull requests are welcome on GitHub at https://github.com/vistaprint/TerraformDevKit.
-
-## License
-
-The gem is available as open source under the terms of the [Apache License, Version 2.0](https://opensource.org/licenses/Apache-2.0).
+# TerraformDevKit
+
+[![Build Status](https://travis-ci.org/vistaprint/TerraformDevKit.svg?branch=master)](https://travis-ci.org/vistaprint/TerraformDevKit) [![Build status](https://ci.appveyor.com/api/projects/status/74s4yd7dmuwg5tmn/branch/master?svg=true)](https://ci.appveyor.com/project/Vistaprint/terraformdevkit/branch/master)
+
+Set of scripts to ease development and testing with [Terraform](https://www.terraform.io/).
+
+The script collection includes support for:
+
+* Managing AWS credentials
+* Backing up the state from a failed Terraform execution
+* Executing external commands
+* Simple configuration management
+* Simple reading and writing to AWS DynamoDB
+* Multiplatform tools
+* Making simple HTTP requests
+* Retrying a block of code
+* Terraform environment management
+* Locally installing Terraform
+* Filtering Terraform logging messages
+
+Most of these scripts exist to provide support to a module development and testing environment for Terraform: [TerraformModules](https://github.com/vistaprint/TerraformModules). But, they might be useful for other purposes too.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'TerraformDevKit'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install TerraformDevKit
+
+## Usage
+
+To use the library simply import it as:
+
+```ruby
+require 'TerraformDevKit'
+```    
+
+## Managing Terraform Environments
+
+TerraformDevKit provides a set of Rake tasks and Ruby scripts to ease the management of multiple Terraform environments. Three major environment types are supported: `dev`, `test` and `prod`.
+
+There might be many development environments (`dev`), each one with its own name. Development environments use a local Terraform backend. They are intented to be used by developers while adding features to the infrastructure.
+
+Testing (`test`) and production (`prod`) environment use a remote backend. Thus, the Terraform state file is not kept in the local disk, but on S3. This allows multiple developers to easily work on the same infrastructure instance. For safety reasons, operations that affect testing and production environments require manual user input. This is not the case for development environments.
+
+TerraformDevKit expects templated versions (using [Mustache](https://mustache.github.io/)) of the Terraform files. Such files might contain placeholders for several fields such as `Environment`, (AWS) `Region` or (AWS) `Profile`, among others. TerraformDevKit uses the template files to generate the final files that will be consumed by Terraform. As an example, for the production environment, the resulting files are placed in a directory named `envs/prod`.
+
+### Configuration Files
+
+Configuration files must be placed in a directory named `config`. Each environment type requires a different configuration file. Thus, the following three files must be placed in the `config` directory:
+
+* `config-dev.yml`
+* `config-test.yml`
+* `config-prod.yml`
+
+The first one contains the configuration for **all** the development environments that are created. The other two contain the configuration for the test and production environments, respectively.
+
+A sample configuration files is shown next:
+
+```yaml
+terraform-version: 0.11.0
+project-name: my super cool project
+aws:
+  profile: myprofile
+  region: eu-west-1
+```
+
+The AWS profile **must not** be specified for test and production accounts, as users are required to manually type the profile name.
+
+### A Minimal Rakefile
+
+```ruby
+ROOT_PATH = File.dirname(File.expand_path(__FILE__))
+
+spec = Gem::Specification.find_by_name 'TerraformDevKit'
+load "#{spec.gem_dir}/tasks/devkit.rake"
+
+task :custom_test, [:env] do |_, args|
+  # Add some tests here
+end
+```
+
+### Tasks and Hooks
+
+TerraformDevKit provides a set of generic tasks to perform:
+
+* `prepare`: prepares the environment
+* `plan`: shows the plan to create the infrastructure
+* `apply`: creates the infrastructure
+* `destroy`: destroys the infrastructure
+* `clean`: cleans the environment (after destroying the infrastructure)
+* `test`: tests a local environment
+* `preflight`: creates a temporary infrastructure and runs the test task
+
+Additionally, TerraformDevKit allows users to define a set of hooks that will be called during the different steps required to complete the previous list of tasks. The following hooks are available:
+
+* `pre_apply`: invoked before `apply` task runs
+* `post_apply`: invoked after `apply` task runs
+* `pre_destroy`: invoked before `destroy` task runs
+* `post_destroy`: invoked after `destroy` task runs
+* `custom_prepare`: invoked during the preparation process, before terraform is initialized
+* `custom_test`: invoked during as part of the `test` task, right after `apply` completes.
+
+### Sample Terraform Templates
+
+The following file (`main.tf.mustache`) contains the infrastructure configuration (a single S3 bucket) as well as information related to the AWS provider.
+
+```hcl
+locals {
+  env    = "{{Environment}}"
+}
+
+terraform {
+    backend {{#LocalBackend}}"local"{{/LocalBackend}}{{^LocalBackend}}"s3"{{/LocalBackend}} {}
+}
+
+provider "aws" {
+  profile = "{{Profile}}"
+  region  = "{{Region}}"
+}
+
+resource "aws_s3_bucket" "raw" {
+  bucket = "foo-${local.env}"
+  acl    = "private"
+
+{{#LocalBackend}}
+  force_destroy = true
+{{/LocalBackend}}
+{{^LocalBackend}}
+  lifecycle {
+    prevent_destroy = true
+  }
+{{/LocalBackend}}
+}
+```
+
+The config file requires a `project-name` to be set. This project name is then use to generate the S3 bucket and dynamodb lock table required by terraform to mamage remote state. To use the remote state feature of TerraformDevKit you must add the following section to your `main.tf.mustache` file:
+
+```hcl
+terraform {
+  {
+  {{#LocalBackend}}
+    backend = "local" {}
+  {{/LocalBackend}}
+  {{^LocalBackend}}
+    backend = "s3" {
+      bucket     = "{{ProjectName}}-{{Environment}}-state"
+      key        = "{{ProjectAcronym}}-{{Environment}}.tfstate"
+      lock_table = "{{ProjectAcronym}}-{{Environment}}-lock-table"
+      encrypt    = true
+      profile    = "{{Profile}}"
+      region     = "{{Region}}"
+    }
+  {{/LocalBackend}}
+  }
+}
+```
+
+### Injecting Additional Variables into Template Files
+
+In addition to the default variables that are passed to Mustache when rendering a template file, users can provide additional variables. To do so, users must register a procedure that receives the environment as a parameter and returns a map with the extra variables and their values. An example is shown next:
+
+```ruby
+TDK::TerraformConfigManager.register_extra_vars_proc(
+  proc do
+    { SumoLogicEndpoint: TDK::Configuration.get('sumologic')['endpoint'] }
+  end
+)
+```
+
+### Updating Modules
+
+Terraform will get the necessary modules every time a new environment is created. Once the modules are cached, there is generally no need to keep updating the modules each time Terraform is executed. When using a module repository it is possible to select a specific version to use (as shown [here](https://www.terraform.io/docs/modules/sources.html#ref)). In such a case, Terraform will automatically update the modules whenever the version number is changed.
+
+When using local modules (e.g., during development process) it might be desirable to update the modules every time Terraform runs. This can be achieved by setting the environment variable `TF_DEVKIT_UPDATE_MODULES` to `true`.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/vistaprint/TerraformDevKit.
+
+## License
+
+The gem is available as open source under the terms of the [Apache License, Version 2.0](https://opensource.org/licenses/Apache-2.0).

data/Rakefile

@@ -1,6 +1,6 @@
-require 'bundler/gem_tasks'
-require 'rspec/core/rake_task'
-
-RSpec::Core::RakeTask.new(:spec)
-
-task :default => :spec
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/TerraformDevKit.gemspec

@@ -1,32 +1,33 @@
-# coding: utf-8
-
-lib = File.expand_path('../lib', __FILE__)
-$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
-require 'TerraformDevKit/version'
-
-Gem::Specification.new do |spec|
-  spec.name          = 'TerraformDevKit'
-  spec.version       = TerraformDevKit::VERSION
-  spec.authors       = ['Victor Jimenez']
-  spec.email         = ['vjimenez@vistaprint.com']
-
-  spec.summary       = 'Set of scripts to ease development and testing with Terraform.'
-  spec.homepage      = 'https://github.com/vistaprint/TerraformDevKit'
-  spec.license       = 'Apache-2.0'
-
-  spec.files = `git ls-files -z`.split("\x0").reject do |f|
-    f.match(%r{^(test|spec|features)/})
-  end
-  spec.bindir        = 'exe'
-  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
-  spec.require_paths = ['lib']
-
-  spec.add_development_dependency 'bundler', '~> 1.14'
-  spec.add_development_dependency 'rake', '~> 10.0'
-  spec.add_development_dependency 'rspec', '~> 3.0'
-  spec.add_development_dependency 'webmock', '~> 3.0.1'
-
-  spec.add_runtime_dependency 'aws-sdk', '~> 2.9.37'
-  spec.add_runtime_dependency 'mustache', '~> 1.0.2'
-  spec.add_runtime_dependency 'rubyzip', '~> 1.2.1'
-end
+# coding: utf-8
+
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'TerraformDevKit/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = 'TerraformDevKit'
+  spec.version       = TerraformDevKit::VERSION
+  spec.authors       = ['Victor Jimenez']
+  spec.email         = ['vjimenez@vistaprint.com']
+
+  spec.summary       = 'Set of scripts to ease development and testing with Terraform.'
+  spec.homepage      = 'https://github.com/vistaprint/TerraformDevKit'
+  spec.license       = 'Apache-2.0'
+
+  spec.files = `git ls-files -z`.split("\x0").reject do |f|
+    f.match(%r{^(test|spec|features)/})
+  end
+  spec.bindir        = 'exe'
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ['lib']
+
+  spec.add_development_dependency 'bundler', '~> 1.14'
+  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'rspec', '~> 3.0'
+  spec.add_development_dependency 'webmock', '~> 3.0'
+
+  spec.add_runtime_dependency 'aws-sdk', '~> 2.9'
+  spec.add_runtime_dependency 'mustache', '~> 1.0'
+  spec.add_runtime_dependency 'rainbow', '~> 3.0'
+  spec.add_runtime_dependency 'rubyzip', '~> 1.2'
+end

data/bin/console

@@ -1,14 +1,14 @@
-#!/usr/bin/env ruby
-
-require "bundler/setup"
-require "TerraformDevKit"
-
-# You can add fixtures and/or initialization code here to make experimenting
-# with your gem easier. You can also use a different console, if you like.
-
-# (If you use this, don't forget to add pry to your Gemfile!)
-# require "pry"
-# Pry.start
-
-require "irb"
-IRB.start(__FILE__)
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "TerraformDevKit"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -1,8 +1,8 @@
-#!/usr/bin/env bash
-set -euo pipefail
-IFS=$'\n\t'
-set -vx
-
-bundle install
-
-# Do any other automated setup that you need to do here
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/lib/TerraformDevKit/backup_state.rb

@@ -1,18 +1,18 @@
-require 'fileutils'
-
-require_relative 'zip_file_generator'
-
-module TerraformDevKit
-  class BackupState
-    def self.backup(prefix)
-      backup_path = ENV['TM_STATE_BACKUP_PATH']
-      return if backup_path.nil?
-
-      filename = "#{prefix}failure_state.zip"
-      ZipFileGenerator.new('.', filename).write
-
-      FileUtils.cp(filename, backup_path)
-      puts "Copied state to #{File.join(backup_path, filename)}"
-    end
-  end
-end
+require 'fileutils'
+
+require_relative 'zip_file_generator'
+
+module TerraformDevKit
+  class BackupState
+    def self.backup(prefix)
+      backup_path = ENV['TM_STATE_BACKUP_PATH']
+      return if backup_path.nil?
+
+      filename = "#{prefix}failure_state.zip"
+      ZipFileGenerator.new('.', filename).write
+
+      FileUtils.cp(filename, backup_path)
+      puts "Copied state to #{File.join(backup_path, filename)}"
+    end
+  end
+end