cloudshaper 0.0.4

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 720390f8ef2c81bf799239154777e8b0177a7d4b
+  data.tar.gz: 952a846501c2d3d5e32b34800cd1d70fd9b7e81d
+SHA512:
+  metadata.gz: 610e4fc75444a2e2721aaccf6a0cfc76e8c56a979848a65d500f0d87934661d91f81cc06e2d91744450d742a357a4067d1f2352eb4efae53e8e462d40d76e5a1
+  data.tar.gz: 8def0b2aef1901b0e1b84fba5c3d061aa9bdab0062dca156200efe18f402a2d4bfe3f3c448e8a8b297b44bffaff74170874dfc78f42ebedf1855e148bc8c2608

data/.gitignore

@@ -0,0 +1,4 @@
+*.swp
+*.swo
+coverage
+*.gem

data/.travis.yml

@@ -0,0 +1,3 @@
+language: ruby
+rvm:
+  - 2.1.5

data/Gemfile

@@ -0,0 +1,7 @@
+source 'https://rubygems.org'
+
+gemspec
+
+group :test do
+  gem 'simplecov', '=0.10.0'
+end

data/Gemfile.lock

@@ -0,0 +1,27 @@
+PATH
+  remote: .
+  specs:
+    cloudshaper (0.0.4)
+      rake (~> 10.4)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    docile (1.1.5)
+    json (1.8.2)
+    minitest (5.6.1)
+    rake (10.4.2)
+    simplecov (0.10.0)
+      docile (~> 1.1.0)
+      json (~> 1.8)
+      simplecov-html (~> 0.10.0)
+    simplecov-html (0.10.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  bundler
+  cloudshaper!
+  minitest (~> 5.6)
+  simplecov (= 0.10.0)

data/LICENSE

@@ -0,0 +1,23 @@
+MIT License
+
+The MIT License (MIT)
+
+Copyright (c) [year] [fullname]
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,162 @@
+[![Build Status](https://travis-ci.org/dalehamel/terraform_dsl.svg)](https://travis-ci.org/dalehamel/terraform_dsl)
+
+# Terraform DSL
+
+This is a simple DSL for wrapping hashicorp's [terraform configuration](https://terraform.io/docs/configuration/index.html).
+
+Terraform is an infrastructure-as-code tool for managing infrastructure by defining stack element in a DSL.
+
+This ruby DSL It supports almost identical syntax, and generates JSON that the terraform go application will understand.
+
+## Differences from terraform's HCL configuration
+
+There are a few ways to access stack element variables and attributes for different purposes:
+
+* get(:var) - This gets the actual value of a variable at generation time. This is needed when passing variables to submodules
+* var(:var) - This references the terraform interpolation syntax of a variable.
+* value\_of(:var) - This references the value of a variable for a stack element by type and name using interpolation syntax.
+* id\_of(:var) - This references the id of a stack element by type and name using interpolation syntax.
+
+Since 'module' is a keyword in ruby, and even though it is technically OK to use it as a function name, we use 'submodule' instead of module, to make editors happy and avoid using the keyword.
+* submodules are terraform modules under the hood - you can use normal terraform module syntax if you wish
+* If you reference your source as 'module\_MODULENAME', you can reference another ruby module!
+ * This module will be resolved at runtime, not compile time
+ * Modules nested most deeply are resolved first. The root module is resolved last.
+
+We allow variables to have array values. This is very useful as a flow-control technique to DRY up modules, and leverage the full power of the ruby DSL.
+However, since terraform does not allow variables to have array values, we automatically flatten them to a comma-separated string. This variable may or may not be used by terraform, if you reference it explicity using one of the interpolated accessors mentioned above. If it is not used, it doesn't matter and will simply be ignored.
+
+Keep in mind that when specifying a hash as a value within the DSL, you must use brackets around the braces ( call as ({key: value}) instead of {key: value}, this is an unfortunate but necessary syntax).
+
+# Usage
+
+To use this gem, you will need to:
+
+* define some stack modules
+* configure secrets for your app
+* define those stacks using your modules in yaml
+* create a simple rake file
+
+Once you've done this, you can run various rake tasks to manage your stacks.
+
+If you get lost, take a look at the [sample app](https://github.com/dalehamel/terraform_dsl_sample)
+
+## Stack modules
+
+In terraform, everything is defined in terms of 'modules'. Even if you don't use the 'module' (submodule in our case) keyword, you're implicitly working within the 'root' module.
+
+Create a stack module, like one of our [examples](examples), such as our [simple app](examples/simple_app.rb)
+
+Generally, you need to do:
+
+```
+require 'terraform_dsl'
+```
+
+And then subclass Terraform::StackModule
+
+```
+class MyAwesomeStackModule < Terraform::StackModule
+```
+
+Within that class, define resources using a similar syntax to [terraform's configuration](https://terraform.io/docs/configuration/index.html).
+
+## Submodules
+
+You may even reference other stack modules that you've defined! Just call the module by using "module\_MODULENAME" as the 'source' value.
+
+Using submodules are a fantastic way to DRY up your infrastructure! Since they are just terraform modules under the hood, you can specify variables as input (to configure the module) and specify outputs of the module, so that other modules may interpolate them for their own resources.
+
+## Configuration
+
+### Secrets
+
+Create a file at config/secrets.json that contains secrets needed for you providers.
+
+Specify the secrets as a JSON hash like so:
+
+```
+{
+  "aws": {
+    "AWS_ACCESS_KEY_ID": "ACCESS_KEY",
+    "AWS_SECRET_ACCESS_KEY": "SECRET_KEY"
+  }
+}
+```
+
+For other providers, see [example configs](examples/secretconfig)
+
+**Note** do not commit plaintext secrets.json to your repository. We recommend you use [ejson](https://github.com/Shopify/ejson) to store your secrets, and [capistrano-ejson](https://github.com/Shopify/capistrano-ejson) to decrypt them in production.
+**Note** Secrets are never written to module files, as a safeguared to prevent them from accidentally being committed. Instead, they are passed as environment vairables.
+
+### YAML
+
+After setting up your rakefile, as below, just run:
+
+```
+bundle exec rake terraform:init
+```
+
+This will set up your stacks.yml with an initial config template, which you can customize for your chosen stack module
+
+```
+common:
+  remote:
+    s3:
+      bucket: quartermaster-terraform
+      region: us-east-1
+stacks:
+  - name: teststack
+    uuid: 1433296428_0safh8-Y # must be unique
+    description: just a test stack
+    root: simpleapp
+    variables:
+      flavor: t1.micro
+      key: SOMESSHKEY
+```
+
+You may also specify a 'common' block, that will be merged into all stacks.
+
+Terraform stacks need somewhere to store their state. By default, this will be the local filesystem.
+
+It's highly recommended that you use a [remote backend](https://www.terraform.io/docs/commands/remote-config.html) instead, so that you can share your stacks.
+
+### Tasks
+
+Create a rake file that loads your modules, and calls Terraform::Tasks.loadall.
+
+```
+## Loads terraform tasks and modules
+require 'terraform_dsl'
+Terraform::Tasks.loadall
+
+```
+
+This will add some terraform tasks for managing your terraform stacks:
+
+```
+rake terraform:apply[name]          # Apply pending changes for a stack
+rake terraform:apply_all            # Apply all pending stack changes
+rake terraform:destroy[name]        # Destroy a stack
+rake terraform:get[name]            # Fetch modules for a stack
+rake terraform:get_all              # Fetch modules for all stacks
+rake terraform:init                 # Initialize stacks.yml if it does not exist
+rake terraform:list                 # List all available stacks
+rake terraform:load                 # Loads available stack modules
+rake terraform:plan[name]           # Show pending changes for a stack
+rake terraform:pull[name]           # Pulls stack state from remote location
+rake terraform:pull_all             # Pulls stack states from remote location
+rake terraform:push[name]           # Push stack state to remote location
+rake terraform:push_all             # Push stack states to remote location
+rake terraform:remote_config[name]  # Sets up remote config for a stack
+rake terraform:remote_config_all    # Sets up remote config for all stacks that support it
+rake terraform:show[name]           # Show details about a stack by name
+rake terraform:show_all             # Show all pending stack changes
+rake terraform:uuid                 # Generate a UUID for a stack, so stacks do not clobber each other
+```
+
+# Credits
+
+Inspired by [terraframe](https://github.com/eropple/terraframe), a very similar but less generic and complete terraform DSL.
+
+[license](LICENSE)

data/Rakefile

@@ -0,0 +1,9 @@
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+
+Rake::TestTask.new('test') do |t|
+  t.libs << 'lib' << 'test'
+  t.test_files = FileList['test/*.rb']
+end
+
+task default: :test

data/TODO.md

@@ -0,0 +1,7 @@
+* Allow variables to include arrays natively
+ * Detect arrays, and flatten into strings (terraform doesn't support arrays for variable values :sad\_panda:)
++ Improve test coverage
++ Come up with a way to lock S3 states
++ Improve documentation
++ Strategy to ship modules around
+ + Gems?

data/examples/secretconfig/README.md

@@ -0,0 +1,3 @@
+These are example JSON blobs for configuring various providers.
+
+You will need to define one of these for each provider you wish to use that requires secrets.

data/examples/secretconfig/atlas.json

@@ -0,0 +1,5 @@
+{
+  "atlas": {
+    "ATLAS_TOKEN": "TOKEN"
+  }
+}

data/examples/secretconfig/aws.json

@@ -0,0 +1,6 @@
+{
+  "aws": {
+    "AWS_ACCESS_KEY_ID": "ACCESS_KEY",
+    "AWS_SECRET_ACCESS_KEY": "SECRET_KEY"
+  }
+}

data/examples/secretconfig/cloudflare.json

@@ -0,0 +1,6 @@
+{
+  "cloudflare": {
+    "CLOUDFLARE_EMAIL": "EMAIL",
+    "CLOUDFLARE_TOKEN": "TOKEN"
+  }
+}

data/examples/secretconfig/cloudstack.json

@@ -0,0 +1,7 @@
+{
+  "cloudstack": {
+    "CLOUDSTACK_API_URL": "API_URL",
+    "CLOUDSTACK_API_KEY": "API_KEY",
+    "CLOUDSTACK_SECRET_KEY": "SECRET_KEY"
+  }
+}

data/examples/secretconfig/digitalocean.json

@@ -0,0 +1,5 @@
+{
+  "digitalocean": {
+    "DIGITALOCEAN_TOKEN": "TOKEN"
+  }
+}

data/examples/secretconfig/dme.json

@@ -0,0 +1,6 @@
+{
+  "dme": {
+    "DME_AKEY": "API_KEY",
+    "DME_SKEY": "SECRET_KEY"
+  }
+}

data/examples/secretconfig/heroku.json

@@ -0,0 +1,6 @@
+{
+  "heroku": {
+    "HEROKU_EMAIL": "EMAIL",
+    "HEROKU_API_KEY": "API_KEY"
+  }
+}

data/examples/secretconfig/mailgun.json

@@ -0,0 +1,5 @@
+{
+  "mailgun": {
+    "MAILGUN_API_KEY": "API_KEY"
+  }
+}

data/examples/simple_app.rb

@@ -0,0 +1,61 @@
+class SimpleApp < StackTemplate
+  variable(:flavor) { default 'm1.small' }
+  variable(:ami) { default 'ami-50948838' }
+  variable(:region) { default 'us-east-1' }
+  variable(:availability_zone) { default 'us-east-1b' }
+  variable(:cidr) { default '10.1.2.0/24' }
+  variable(:name) { default 'default' }
+  variable(:key) {}
+
+  provider 'aws' do
+    region var(:region)
+  end
+
+  resource 'aws_security_group', :basic_web do
+    name 'simple_app_web'
+    description 'Basic security group with port 22, 80, 443 open to the world.'
+
+    ingress do
+      from_port 22
+      to_port 22
+      protocol 'tcp'
+      cidr_blocks ['0.0.0.0/0']
+    end
+
+    ingress do
+      from_port 80
+      to_port 80
+      protocol 'tcp'
+      cidr_blocks ['0.0.0.0/0']
+    end
+
+    ingress do
+      from_port 443
+      to_port 443
+      protocol 'tcp'
+      cidr_blocks ['0.0.0.0/0']
+    end
+  end
+
+  resource 'aws_vpc', :simple_vpc do
+    cidr_block var(:cidr)
+    tags ({
+      'Name' => var(:name)
+    })
+  end
+
+  resource 'aws_subnet', :simple_subnet do
+    vpc_id id_of('aws_vpc', :simple_vpc)
+    cidr_block var(:cidr)
+  end
+
+  resource 'aws_instance', :simple_app do
+    security_groups [value_of('aws_security_group', :basic_web, :name)]
+    instance_type var(:flavor)
+    ami var(:ami)
+    key_name var(:key)
+    tags ({
+      'Name' => "#{var(:name)}.ec2.shopify.com"
+    })
+  end
+end

data/lib/tasks/tasks.rb

@@ -0,0 +1,12 @@
+require 'rake'
+
+module Terraform
+  # Loads all rake tasks when terraform_dsl is included by a rake script
+  class Tasks
+    def self.loadall
+      Dir.glob("#{File.join(File.dirname(__dir__), 'tasks')}/*.rake").each { |r| load r }
+      template_path = ENV['TERRAFORM_TEMPLATE_PATH'] || 'templates'
+      Dir.glob("#{File.join(Dir.pwd, template_path)}/*.rb").each { |t| require_relative t }
+    end
+  end
+end

data/lib/tasks/terraform.rake

@@ -0,0 +1,127 @@
+require 'terraform_dsl'
+
+namespace 'terraform' do
+  desc 'Loads available stack modules'
+  task :load do
+    Terraform::Stacks.load
+  end
+
+  desc 'Initialize stacks.yml if it does not exist'
+  task :init do
+    Terraform::Stacks.init
+  end
+
+  desc 'Fetch modules for a stack'
+  task :get, [:name] => :load do |_t, args|
+    stack = Terraform::Stacks.stacks[args[:name]]
+    stack.get
+  end
+
+  desc 'Fetch modules for all stacks'
+  task get_all: :load do
+    Terraform::Stacks.stacks.each do |_name, stack|
+      stack.get
+    end
+  end
+
+  desc 'List all available stacks'
+  task list: :load do
+    Terraform::Stacks.stacks.each do |name, _stack|
+      puts name
+    end
+  end
+
+  desc 'Show details about a stack by name'
+  task :show, [:name] => :load do |_t, args|
+    fail 'Specify a name' unless args[:name]
+    stack = Terraform::Stacks.stacks[args[:name]]
+    puts stack
+    stack.plan
+  end
+
+  desc 'Show all pending stack changes'
+  task show_all: [:load, :get_all] do
+    Terraform::Stacks.stacks.each do |_name, stack|
+      puts stack
+      stack.plan
+    end
+  end
+
+  desc 'Show pending changes for a stack'
+  task :plan, [:name] => :load do |_t, args|
+    fail 'Specify a name' unless args[:name]
+    stack = Terraform::Stacks.stacks[args[:name]]
+    stack.plan
+  end
+
+  desc 'Apply pending changes for a stack'
+  task :apply, [:name] => :load do |_t, args|
+    fail 'Specify a name' unless args[:name]
+    stack = Terraform::Stacks.stacks[args[:name]]
+    stack.apply
+  end
+
+  desc 'Apply all pending stack changes'
+  task apply_all: :load do
+    Terraform::Stacks.stacks.each do |_name, stack|
+      puts stack
+      stack.apply
+    end
+  end
+
+  desc 'Destroy a stack'
+  task :destroy, [:name] => :load do |_t, args|
+    fail 'Specify a name' unless args[:name]
+    stack = Terraform::Stacks.stacks[args[:name]]
+    stack.destroy
+  end
+
+  desc 'Push stack state to remote location'
+  task :push, [:name] => [:load, :remote_config] do |_t, args|
+    stack = Terraform::Stacks.stacks[args[:name]]
+    stack.push
+  end
+
+  desc 'Push stack states to remote location'
+  task push_all: [:load, :remote_config_all] do
+    Terraform::Stacks.stacks.each do |_name, stack|
+      puts stack
+      stack.push
+    end
+  end
+
+  desc 'Pulls stack state from remote location'
+  task :pull, [:name] => [:load, :remote_config] do |_t, args|
+    stack = Terraform::Stacks.stacks[args[:name]]
+    stack.pull
+  end
+
+  desc 'Pulls stack states from remote location'
+  task pull_all: [:load, :remote_config_all] do
+    Terraform::Stacks.stacks.each do |_name, stack|
+      puts stack
+      stack.pull
+    end
+  end
+
+  desc 'Sets up remote config for a stack'
+  task :remote_config, [:name] => [:load] do |_t, args|
+    stack = Terraform::Stacks.stacks[args[:name]]
+    stack.remote_config
+  end
+
+  desc 'Sets up remote config for all stacks that support it'
+  task remote_config_all: :load do
+    Terraform::Stacks.stacks.each do |_name, stack|
+      puts stack
+      stack.remote_config
+    end
+  end
+
+  desc 'Generate a UUID for a stack, so stacks do not clobber each other'
+  task :uuid do
+    uuid = Terraform::Stacks.uuid
+    puts "uuid: #{uuid}"
+    puts 'Add this as a field to a new stack to prevent clobbering stacks with the same name'
+  end
+end