terraspace 0.1.1 → 0.2.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: aaf22ae87af9bb9007e2a7c7a7b2b79f15c9f8303fe47257fc869219c4cb0feb
-  data.tar.gz: f632c63f5fed2f8b8483cdfd04ef7f50a1fe912a2595b5325c7aef7ed9ac944a
+  metadata.gz: bfffe08e9ea588c9c01b6e6bfb0c3399047391acf17eac9d74c8d4643ba4722b
+  data.tar.gz: f887084702cab528ef80c786bbacf5ce5e7e7f56938b96bce2cc4c85da8d0549
 SHA512:
-  metadata.gz: 0463d2d835a44d3c0a2e607f3903856e018881ebac0c96d06324e0387b3908ae7202216f0bdc71b79d2e739e7c1cdf5c600853397418ea7525f9437fce4814d6
-  data.tar.gz: f2017f4f70d426a3f5e4449c788301ab82e287caa35f453244c019580c8b628d4bdd8fc040e078cb8b50cd924dbe3a6b97e63821a59384787d173f751cffbee3
+  metadata.gz: fe9ec67940c66eaf3f091b2e9dc7eaec2252ad8e1b134c11faaf4fc03befdb840c2e5222cf6204d879d6b5f0221f506457245910c236785c0caee8c58528352d
+  data.tar.gz: 725d3091bad4ea62ccb7e9fbb49dcb124c235e5ec810278b301fa859ba3c9721f8b616528cb777dd5e17a233b66bc6804ecef605b922364e8634b3e16a4204c9

data/CHANGELOG.md

@@ -3,8 +3,41 @@
 All notable changes to this project will be documented in this file.
 This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [0.2.3]
+* #37 config.clean_cache option
+
+## [0.2.2]
+* #36 cloud.relative_root setting
+
+## [0.2.1]
+* #35 fix summary bug when ran multiple times with different envs
+
+## [0.2.0]
+* #34 Terraform Cloud and Terraform Enterprise support added.
+* TFC Vars support: JSON and DSL. config.overwrite and config.overwrite_sensitive configs
+* Build all stacks with config/terraform files. Designed to support the TFC VCS-driven workflow.
+* Layer Interface module added. All latest provider plugins like terraspace\_plugin_aws make use of this module.
+* Backend pattern expansion auto-detects the provider bakcend. The `expansion` method replaces the `backend_expand` method. `backend_expand` is deprecated.
+* New expander variables: TYPE_INSTANCE, INSTANCE, CACHE_ROOT. Also added strip trailing - and / behavior.
+* Timeout for terraform init. The default timeout is 10m and will then print out the terraform init log.
+* Terraspace 0.2.x is compatible with terraspace\_plugin_aws 0.2.x, terraspace\_plugin_google 0.2.x, and terraspace\_plugin_azurerm 0.2.x
+* New commands: terraspace list, terraspace cloud list, terraspace cloud setup, terraspace cloud destroy, terraspace new shim, terraspace new git_hook
+* terraspace list: list of modules and stacks
+* terraspace cloud list: shows list of TFC workspaces
+* terraspace cloud setup: setups up TFC workspace for VCS-driven workflow. This automatically happens for the TFC CLI-driven workflow.
+* terraspace cloud destroy: destroys the TFC workspace associated with the stack. Can also use the `terraspace down demo --destroy-workspace` option.
+* terraspace new shim: An quick way to generate a terraspace shim.
+* terraspace new git_hook: An quick way to set up a git pre-push hook for the TFC VCS-driven workflow.
+* terraspace down: works even if there's no app/stacks folder. A fake stack is built.
+* terraspace build: terraspace build placeholder concept.
+* terraspace build: only builds now. auto bucket backend creation and terraform init is is still automatically called by terraform up, etc.
+* terraspace up: --reconfigure option. This is useful if upgrading Terraform version.
+
+## [0.1.2]
+* #33 rspec-terraspace dependency added
+
 ## [0.1.1]
-- #32 terraspace summary --short option
+* #32 terraspace summary --short option
 
 ## [0.1.0]
-- Initial release
+* Initial release

data/README.md

@@ -1,3 +1,7 @@
+<div align="center">
+  <a href="https://terraspace.cloud"><img src="https://img.boltops.com/boltops/logos/terraspace-dark-v2.png" /></a>
+</div>
+
 # Terraspace
 
 [![Gem Version](https://badge.fury.io/rb/terraspace.png)](http://badge.fury.io/rb/terraspace)
@@ -8,6 +12,10 @@ The Terraform Framework.
 
 Official Docs Site: [terraspace.cloud](https://terraspace.cloud)
 
+Introduction Video:
+
+[![Watch the video](https://img.boltops.com/boltops/tools/terraspace/terraspace-youtube.png)](https://www.youtube.com/watch?v=O87t5q22YNc)
+
 ## Quick Start
 
 Here are commands to get started:
@@ -17,8 +25,8 @@ Here are commands to get started:
     terraspace up demo
     terraspace down demo
 
-* The `new` command generates a starter project. 
-* The `up` command creates an s3 bucket. 
+* The `new` command generates a starter project.
+* The `up` command creates an s3 bucket.
 * The `down` command cleans up and deletes the bucket.
 
 The default plugin is aws. Other plugins also supported are: google and azurerm.
@@ -28,10 +36,10 @@ The default plugin is aws. Other plugins also supported are: google and azurerm.
 Create infrastructure:
 
     $ terraspace up demo -y
-    Building .terraspace-cache/dev/stacks/demo
-    Current directory: .terraspace-cache/dev/stacks/demo
-    => terraform init -get > /dev/null
-    Built in .terraspace-cache/dev/stacks/demo
+    Building .terraspace-cache/us-west-2/dev/stacks/demo
+    Current directory: .terraspace-cache/us-west-2/dev/stacks/demo
+    => terraform init -get > /tmp/terraspace/out/terraform-init20200824-21379-bkfvnh.out
+    Built in .terraspace-cache/us-west-2/dev/stacks/demo
     => terraform apply -auto-approve
     random_pet.bucket: Creating...
     random_pet.bucket: Creation complete after 0s [id=amusing-mouse]
@@ -48,10 +56,10 @@ Create infrastructure:
 Destroy infrastructure:
 
     $ terraspace down demo -y
-    Building .terraspace-cache/dev/stacks/demo
-    Current directory: .terraspace-cache/dev/stacks/demo
-    => terraform init -get > /dev/null
-    Built in .terraspace-cache/dev/stacks/demo
+    Building .terraspace-cache/us-west-2/dev/stacks/demo
+    Current directory: .terraspace-cache/us-west-2/dev/stacks/demo
+    => terraform init -get > /tmp/terraspace/out/terraform-init20200824-21379-bkfvnh.out
+    Built in .terraspace-cache/us-west-2/dev/stacks/demo
     => terraform destroy -auto-approve
     random_pet.bucket: Refreshing state... [id=amusing-mouse]
     module.bucket.aws_s3_bucket.this: Refreshing state... [id=bucket-amusing-mouse]
@@ -67,9 +75,9 @@ Destroy infrastructure:
 
 * [Config Structure](https://terraspace.cloud/docs/config/): A common config structure that gets materializes with the deployed module. Configs can be dynamically controlled to keep your code DRY. You can override the settings if needed, like for using existing backends. See: [Existing Backends](https://terraspace.cloud/docs/state/existing/).
 * [Generators](https://terraspace.cloud/docs/generators/): Built-in generators to quickly create the starter module. Focus on code instead of boilerplate structure.
-* [Tfvars](https://terraspace.cloud/docs/tfvars/): Use the same code with different tfvars to create multiple environments. Terraspace conventionally loads tfvars from the `tfvars` folder.
-* [Layering](https://terraspace.cloud/docs/tfvars/layering/): Rich layering support. This allows you to build different environments like dev and prod with the same code.
+* [Tfvars](https://terraspace.cloud/docs/tfvars/) & [Layering](https://terraspace.cloud/docs/tfvars/layering/): Use the same code with different tfvars to create multiple environments. Terraspace conventionally loads tfvars from the `tfvars` folder. Rich layering support allows you to build different environments like dev and prod with the same code.  Examples are in [Full Layering](https://terraspace.cloud/docs/tfvars/full-layering/).
 * [Testing](https://terraspace.cloud/docs/testing/): A testing framework that allows you to create test harnesses, deploy real-resources, and have higher confidence that your code works.
 * [Configurable CLI](https://terraspace.cloud/docs/cli/): Configurable [CLI Hooks](https://terraspace.cloud/docs/cli/hooks/) and [CLI Args](https://terraspace.cloud/docs/cli/args/) allow you to adjust the underlying terraform command.
+* [Terraform Cloud and Terraform Enterprise Support](https://terraspace.cloud/docs/cloud/): TFC and TFE are both supported. Terraspace adds additional conveniences to make working with Terraform Cloud Workspaces easier.
 
 For more info: [terraspace.cloud](https://terraspace.cloud)

data/lib/templates/base/git_hook/hook.sh

@@ -0,0 +1,5 @@
+#!/bin/bash -ex
+echo "Running: terraspace build placeholder"
+<%= terraspace_build_commands %>
+git add .terraspace-cache
+git commit -m 'update .terraspace-cache' || true

data/lib/templates/base/project/.gitignore

@@ -14,7 +14,6 @@ terraform.tfvars
 *.gem
 *.rbc
 .idea
-.terraspace-cache
 /.bundle
 /.config
 /.yardoc

data/lib/templates/base/shim/terraspace

@@ -0,0 +1,7 @@
+#!/bin/bash
+<%= switch_ruby_version_line %>
+if [ -f config/app.rb ]; then
+  exec bundle exec terraspace "$@"
+else
+  exec terraspace "$@"
+fi

data/lib/templates/hcl/project/config/terraform/backend.tf.tt

@@ -3,8 +3,8 @@
 # terraform {
 #   backend "s3" {
 #     bucket         = "<%= options[:bucket] || "terraform-state-:ACCOUNT-:REGION-:ENV" %>"
-#     key            = "<%%= "s3", ":REGION/:ENV/:BUILD_DIR/terraform.tfstate" %> # IE: us-west-2/dev/ec2/modules/vpc/terraform.tfstate
-#     region         = "<%%= "s3", ":REGION" %>"
+#     key            = "<%%= expansion(":REGION/:ENV/:BUILD_DIR/terraform.tfstate") %> # IE: us-west-2/dev/ec2/modules/vpc/terraform.tfstate
+#     region         = "<%%= expansion(":REGION") %>"
 #     encrypt        = true
 #     dynamodb_table = "terraform_locks"
 #   }
@@ -13,6 +13,6 @@
 # terraform {
 #   backend "gcs" {
 #     bucket = "<%= options[:bucket] || "terraform-state-:PROJECT-:REGION-:ENV" %>"
-#     prefix = "<%%= backend_expand("gcs", ":REGION/:ENV/:BUILD_DIR") %>" # IE: us-central1/dev/modules/vm
+#     prefix = "<%%= expansion(":REGION/:ENV/:BUILD_DIR") %>" # IE: us-central1/dev/modules/vm
 #   }
 # }

data/lib/templates/plugin/lib/templates/hcl/project/config/terraform/backend.tf.tt

@@ -1,6 +1,6 @@
 terraform {
   backend "PROVIDER_BACKEND" {
-    bucket = "<%%= backend_expand("PROVIDER_BACKEND", "terraform-state-REPLACE_ME-REPLACE_ME-:ENV") %>" # expanded by terraspace
-    prefix = "<%%= backend_expand("PROVIDER_BACKEND", "REPLACE_ME/:ENV/:BUILD_DIR") %>" # expanded by terraspace
+    bucket = "<%%= expansion("terraform-state-REPLACE_ME-REPLACE_ME-:ENV") %>" # expanded by terraspace
+    prefix = "<%%= expansion("REPLACE_ME/:ENV/:BUILD_DIR") %>" # expanded by terraspace
   }
 }

data/lib/terraspace/app.rb

@@ -14,6 +14,14 @@ module Terraspace
       config.logger = Logger.new($stdout)
       config.logger.level = :info
       config.hooks = Hooks.new
+      config.cloud = ActiveSupport::OrderedOptions.new
+      config.cloud.overwrite = true
+      config.cloud.overwrite_sensitive = true
+      config.cloud.relative_root = nil
+      config.build = ActiveSupport::OrderedOptions.new
+      config.build.cache_root = nil # defaults to /full/path/to/.terraspace-cache
+      config.build.cache_dir = ":CACHE_ROOT/:REGION/:ENV/:BUILD_DIR"
+      config.build.clean_cache = nil # defaults to /full/path/to/.terraspace-cache
       config
     end
 

data/lib/terraspace/builder.rb

@@ -3,25 +3,16 @@ module Terraspace
     def run
       Terraspace::CLI::CheckSetup.check!
       @mod.root_module = true
-      Compiler::Cleaner.new(@mod, @options).clean
-      build_dir = Util.pretty_path(@mod.cache_build_dir)
+      Compiler::Cleaner.new(@mod, @options).clean if clean?
+      build_dir = Util.pretty_path(@mod.cache_dir)
       logger.info "Building #{build_dir}"
 
       build_all("modules") # build all modules and stacks as dependencies
       build_all("stacks")
-      build_root_module    # build root module at the end
-
-      auto_create_backend
-      Terraform::Runner.new("init", @options).run if !auto? && @options[:init] != false # will run on @options[:init].nil?
-      build_remote_dependencies # runs after terraform init, which downloads remote modules
+      build_root_module
       logger.info "Built in #{build_dir}"
     end
 
-    def auto?
-      # command is only passed from CLI in the update specifically for this check
-      @options[:auto] && @options[:command] == "update"
-    end
-
     def build_root_module
       Compiler::Builder.new(@mod).build
     end
@@ -35,22 +26,12 @@ module Terraspace
 
         consider_stacks = type_dir == "stacks"
         mod = Mod.new(mod_name, consider_stacks: consider_stacks)
-        next if root?(mod) # will build root module at the end
 
         Compiler::Builder.new(mod).build
         built << mod_name
       end
     end
 
-    def root?(mod)
-      mod.name == @mod.name && mod.type == @mod.type
-    end
-
-    def auto_create_backend
-      return unless @options[:command] == "update"
-      Compiler::Backend.new(@mod).create
-    end
-
     def local_paths(type_dir)
       dirs("app/#{type_dir}/*") + dirs("vendor/#{type_dir}/*")
     end
@@ -59,29 +40,9 @@ module Terraspace
       Dir.glob("#{Terraspace.root}/#{path}")
     end
 
-    # Currently only handles remote modules only one-level deep.
-    def build_remote_dependencies
-      modules_json_path = "#{@mod.cache_build_dir}/.terraform/modules/modules.json"
-      return unless File.exist?(modules_json_path)
-
-      initialized_modules = JSON.load(IO.read(modules_json_path))
-      # For example of structure see spec/fixtures/initialized/modules.json
-      initialized_modules["Modules"].each do |meta|
-        build_remote_mod(meta)
-      end
-    end
-
-    def build_remote_mod(meta)
-      return if local_source?(meta["Source"])
-      return if meta['Dir'] == '.' # root is already built
-
-      remote_mod = Mod::Remote.new(meta, @mod)
-      Compiler::Builder.new(remote_mod).build
-    end
-
-  private
-    def local_source?(s)
-       s =~ %r{^\.} || s =~ %r{^/}
+    def clean?
+      clean_cache = Terraspace.config.build.clean_cache
+      clean_cache.nil? ? true : clean_cache
     end
   end
 end

data/lib/terraspace/cli.rb

@@ -24,20 +24,25 @@ module Terraspace
     init_option = Proc.new {
       option :init, type: :boolean, default: true, desc: "Instance of stack"
     }
+    reconfigure_option = Proc.new {
+      option :reconfigure, type: :boolean, desc: "Add terraform -reconfigure option"
+    }
 
     desc "new SUBCOMMAND", "new subcommands"
     long_desc Help.text(:new)
     subcommand "new", New
 
-    desc "build MODULE", "build"
+    desc "cloud SUBCOMMAND", "cloud subcommands"
+    long_desc Help.text(:cloud)
+    subcommand "cloud", Cloud
+
+    desc "build STACK", "build"
     long_desc Help.text(:build)
     option :quiet, type: :boolean, default: true, desc: "quiet output"
-    auto_option.call
-    init_option.call
-    input_option.call
     instance_option.call
+    yes_option.call
     def build(mod)
-      Build.new(options.merge(mod: mod)).run
+      Terraspace::Builder.new(@options.merge(mod: mod)).run
     end
 
     desc "bundle", "bundle"
@@ -58,22 +63,25 @@ module Terraspace
       Clean.new(options).run
     end
 
-    desc "console", "console .terraspace-cache dir"
+    desc "console STACK", "console .terraspace-cache dir"
     long_desc Help.text(:console)
     instance_option.call
     def console(mod)
       Commander.new("console", options.merge(mod: mod)).run
     end
 
-    desc "down MODULE", "down"
+    desc "down STACK", "down"
     long_desc Help.text(:down)
     instance_option.call
     yes_option.call
+    reconfigure_option.call
+    option :destroy_workspace, type: :boolean, desc: "Also destroy the Cloud workspace. Only applies when using Terraform Cloud remote backend."
     def down(mod)
-      Commander.new("destroy", options.merge(mod: mod)).run
+      Commander.new("destroy", options.merge(mod: mod, command: "down")).run
+      Terraspace::Terraform::Cloud::Workspace.new(options.merge(mod: mod)).destroy if @options[:destroy_workspace]
     end
 
-    desc "info MODULE", "info"
+    desc "info STACK", "info"
     long_desc Help.text(:info)
     format_option.call
     instance_option.call
@@ -81,31 +89,38 @@ module Terraspace
       Info.new(options.merge(mod: mod)).run
     end
 
-    desc "plan MODULE", "plan module"
+    desc "list", "list stacks and modules"
+    long_desc Help.text(:list)
+    def list
+      List.new(options).run
+    end
+
+    desc "plan STACK", "plan stack"
     long_desc Help.text(:plan)
     auto_option.call
     input_option.call
     instance_option.call
     out_option.call
+    reconfigure_option.call
     def plan(mod)
       Commander.new("plan", options.merge(mod: mod)).run
     end
 
-    desc "providers MODULE", "providers"
+    desc "providers STACK", "providers"
     long_desc Help.text(:providers)
     instance_option.call
     def providers(mod)
       Commander.new("providers", options.merge(mod: mod)).run
     end
 
-    desc "refresh", "refresh"
+    desc "refresh STACK", "refresh"
     long_desc Help.text(:refresh)
     instance_option.call
     def refresh(mod)
       Commander.new("refresh", options.merge(mod: mod)).run
     end
 
-    desc "seed MODULE", "seed"
+    desc "seed STACK", "seed"
     long_desc Help.text(:seed)
     option :yes, aliases: :y, type: :boolean, desc: "bypass prompts and force overwrite files"
     option :where, desc: "where to create file. either under app or seed folder structure. values: app or stack"
@@ -124,7 +139,7 @@ module Terraspace
       Summary.new(options).run
     end
 
-    desc "show MODULE", "show"
+    desc "show STACK", "show"
     long_desc Help.text(:show)
     instance_option.call
     def show(mod)
@@ -137,7 +152,7 @@ module Terraspace
       Test.new(options).run
     end
 
-    desc "output MODULE", "output"
+    desc "output STACK", "output"
     long_desc Help.text(:output)
     format_option.call
     instance_option.call
@@ -146,20 +161,21 @@ module Terraspace
       Commander.new("output", options.merge(mod: mod)).run
     end
 
-    desc "update MODULE", "Update infrasturcture. IE: apply plan"
+    desc "update STACK", "Update infrasturcture. IE: apply plan"
     long_desc Help.text(:update)
     auto_option.call
     init_option.call
     input_option.call
     instance_option.call
     yes_option.call
+    reconfigure_option.call
     option :plan, desc: "Execution plan that can be used to only execute a pre-determined set of actions."
     option :var_files, type: :array, desc: "list of var files"
     def update(mod)
-      Commander.new("apply", options.merge(mod: mod, command: "update")).run
+      Commander.new("apply", options.merge(mod: mod)).run
     end
 
-    desc "validate MODULE", "validate"
+    desc "validate STACK", "validate"
     long_desc Help.text(:validate)
     instance_option.call
     def validate(mod)

data/lib/terraspace/cli/build/placeholder.rb

@@ -0,0 +1,40 @@
+# The placeholder stack is a special stack that is useful in case there are no app/stacks.
+# We build a placeholder back just so we have backend.tf to be used to grab info.
+# It's useful for the summary command.
+module Terraspace::CLI::Build
+  class Placeholder
+    include Terraspace::Util::Logging
+
+    def initialize(options={})
+      @options = options
+    end
+
+    # Grab the last module and build that.
+    # Assume the backend key has the same prefix
+    # Note: Tried building a empty "null" stack but with TFC a null space workspace is created, which is undesired.
+    def build
+      return if ENV['TS_SUMMARY_BUILD'] == '0'
+
+      mod = @options[:mod]
+      if !mod or mod == "placeholder"
+        logger.info "Building one of the modules to get backend.tf info"
+        mod = find_mod
+      end
+      Terraspace::Builder.new(@options.merge(mod: mod, init: false)).run # generate and init
+      Terraspace::Mod.new(mod, @options) # mod metadata
+    end
+
+    # Used by: terraspace build placeholder
+    def find_mod
+      mod_path = Dir.glob("{app,vendor}/{modules,stacks}/*").last
+      unless mod_path
+        logger.info <<~EOL
+          No modules or stacks found.
+          Unable to determine the backend state path without at least one module.
+        EOL
+        exit 0
+      end
+      File.basename(mod_path) # mod name
+    end
+  end
+end