terraspace 0.3.2 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 471e59d4e8478c78dd4de1190397e89548e68646fa6fcec02748442ecf9b282e
-  data.tar.gz: 65250fbef38cef2731688e9c75877747a2a567ecd0d9ba40ce30b2c86c5c73e6
+  metadata.gz: 6b99a86e9b9b829fa0ac1b04a6774b02a43e67642219ea3f22aeb663869bfec6
+  data.tar.gz: f2b265d5e13a86949ddb753e991fc8784c77361909162123dfd7d06d0c1c3555
 SHA512:
-  metadata.gz: 400b78be5d433399119713533a7b56fd7b143b253cae7e96d344273d74784ba9fe877a2a92293f237cec9f0642694256bd04c0b0ad0bef73050d205603884f92
-  data.tar.gz: eac5b2042f655964d08454d7b5ea3ba40012145fed1c560bf116757c91857479ef7430cd1b7a2be661e828dbaf12a7cd0323014fb847a13299ef8fdc01cc8125
+  metadata.gz: 7fd4d65aac675d439e5e23089db75f937e3f50c72158099d1088112db67ed97c93afef6c47e0555cdde5a6dfe3e175ede017fc9e509c5419128c53ee114de91d
+  data.tar.gz: 4ef8b0354c0e8dd14ccd074bc6ecd2cda96f80c846bfd3dc790dce4cf20f36efbe236020eff587555032ec1dd869ec405cde649a348e232e747ba7900ad79393

data/.cody/aws/bin/build.sh

@@ -10,4 +10,6 @@ export PATH=~/bin:$PATH # ~/bin/terraspace wrapper
 set -x
 terraspace new project infra --examples
 cd infra
+terraspace new bootstrap_test
+terraspace new project_test demo --examples
 terraspace test

data/.cody/azurerm/bin/build.sh

@@ -17,4 +17,6 @@ export ARM_TENANT_ID=$(cat ~/.azure/app-client.json | jq -r '.tenant_id')
 set -x
 terraspace new project infra --examples --plugin azurerm
 cd infra
+terraspace new bootstrap_test
+terraspace new project_test demo --examples --plugin azurerm
 terraspace test

data/.cody/google/bin/build.sh

@@ -14,4 +14,6 @@ export GOOGLE_PROJECT=$(cat $GOOGLE_APPLICATION_CREDENTIALS | jq -r '.project_id
 set -x
 terraspace new project infra --examples --plugin google
 cd infra
+terraspace new bootstrap_test
+terraspace new project_test demo --examples --plugin google
 terraspace test

data/CHANGELOG.md

@@ -3,6 +3,44 @@
 All notable changes to this project will be documented in this file.
 This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [0.4.0]
+* improve hooks: allow multiple hooks of same type, change path hooks/terraform.rb
+* improve hooks: introduce terraspace-level as well as terraform-level hooks
+* improve hooks: can take Ruby block or shell script
+* improve auto init: reinit when module source changed, improve auto init: generalize retry check
+* fix already_init? detection for case when stack does use module
+* terraspace bundler options: can set any option now
+* add `terraspace all init` command
+* improve terraspace clean, prompt user and add `-y` option
+* bundle: check if in terraspace project
+* Rename NullObject to Unresolved
+* improve error message when stacks not found. give `terraspace list` hint
+* terraspace list. change default to `--type stack`
+* change summary option to --details
+* include terraspace version in generated Gemfile
+* logs command to handle viewing, and clean logs to clean
+
+## [0.3.6]
+* #44 improve logs management commands: `terraspace logs remove` and `terraspace logs truncate`
+
+## [0.3.5]
+* #43 rename `terraform_output` helper to `output`. Keep `terraform_output` for backwards compatibility
+* to_ruby natural interface to access output with full power of Ruby
+* output formatters removed in favor for `.to_ruby` method.
+
+## [0.3.4]
+* #42 update cli docs and bug fixes
+* fix console by using system instead of popen3
+* fix build for edge case when app/modules exist but app/stacks do not
+* terraspace new project: do not generate spec folder by default
+* improve all output summary
+* remove redundant `terraspace cloud setup`, instead use: `terraspace cloud sync`
+* improve terraspace info output
+* fix integration test pipeline
+
+## [0.3.3]
+* #41 fix `terraspace build` and `terraspace seed` when bucket doesnt exist yet and there are dependenices defined.
+
 ## [0.3.2]
 * #40 fix backend auto creation
 

data/README.md

@@ -81,14 +81,14 @@ To choose multiple stacks to deploy
 
     terraspace all up instance vpc
 
-When you use the all command, the dependency graph is calculated and the stacks are deployed in the right order. To learn more: [Deploy Multiple Stacks](https://terraspace.cloud/docs/dependencies/deploy-all/).
+When you use the all command, the dependency graph is calculated and the stacks are deployed in the right order. To learn more: [Deploy Multiple Stacks](https://terraspace.cloud/docs/intro/deploy-all/).
 
 ## Features
 
 * [Config Structure](https://terraspace.cloud/docs/config/): A common config structure that gets materializes with the deployed module. Configs can be dynamically controlled to keep your code DRY. You can override the settings if needed, like for using existing backends. See: [Existing Backends](https://terraspace.cloud/docs/state/existing/).
 * [Generators](https://terraspace.cloud/docs/generators/): Built-in generators to quickly create the starter module. Focus on code instead of boilerplate structure.
 * [Tfvars](https://terraspace.cloud/docs/tfvars/) & [Layering](https://terraspace.cloud/docs/tfvars/layering/): Use the same code with different tfvars to create multiple environments. Terraspace conventionally loads tfvars from the `tfvars` folder. Rich layering support allows you to build different environments like dev and prod with the same code.  Examples are in [Full Layering](https://terraspace.cloud/docs/tfvars/full-layering/).
-* [Deploy Multiple Stacks](https://terraspace.cloud/docs/dependencies/deploy-all/): The ability to deploy multiple stacks with a single command. Terraspace calculates the [dependency graph](https://terraspace.cloud/docs/dependencies/) and deploys stacks in the right order. You can also target specific stacks and deploy [subgraphs](https://terraspace.cloud/docs/dependencies/subgraphs/).
+* [Deploy Multiple Stacks](https://terraspace.cloud/docs/intro/deploy-all/): The ability to deploy multiple stacks with a single command. Terraspace calculates the [dependency graph](https://terraspace.cloud/docs/dependencies/) and deploys stacks in the right order. You can also target specific stacks and deploy [subgraphs](https://terraspace.cloud/docs/dependencies/subgraphs/).
 * [Configurable CLI](https://terraspace.cloud/docs/cli/): Configurable [CLI Hooks](https://terraspace.cloud/docs/cli/hooks/) and [CLI Args](https://terraspace.cloud/docs/cli/args/) allow you to adjust the underlying terraform command.
 * [Testing](https://terraspace.cloud/docs/testing/): A testing framework that allows you to create test harnesses, deploy real-resources, and have higher confidence that your code works.
 * [Terraform Cloud and Terraform Enterprise Support](https://terraspace.cloud/docs/cloud/): TFC and TFE are both supported. Terraspace adds additional conveniences to make working with Terraform Cloud Workspaces easier.

data/lib/templates/base/project/README.md

@@ -14,4 +14,4 @@ To deploy individual stacks:
 
 ## Terrafile
 
-To add more modules for use, you can add them to the [Terrafile](https://terraspace.cloud/docs/terrafile/).
+To use more modules, add them to the [Terrafile](https://terraspace.cloud/docs/terrafile/).

data/lib/terraspace.rb

@@ -6,6 +6,7 @@ require "active_support/core_ext/class"
 require "active_support/core_ext/hash"
 require "active_support/core_ext/string"
 require "active_support/ordered_options"
+require "cli-format"
 require "deep_merge/rails_compat"
 require "dsl_evaluator"
 require "fileutils"
@@ -26,6 +27,7 @@ module Terraspace
   extend Core # for Terraspace.root
   class Error < StandardError; end
   class InitRequiredError < Error; end
+  class BucketNotFoundError < Error; end
 end
 
 Terraspace::Booter.boot

data/lib/terraspace/all/preview.rb

@@ -22,7 +22,7 @@ module Terraspace::All
       @batches.map do |batch|
         i += 1
         batch.map do |stack|
-          command = "   terraspace #{@command}"
+          command = "    terraspace #{@command}"
           ljust = command.size + max_name_size + 1
           command = "#{command} #{stack.name}"
           command.ljust(ljust, ' ') + " # batch #{i}"

data/lib/terraspace/all/runner.rb

@@ -105,6 +105,7 @@ module Terraspace::All
       set_log_path!(mod_name)
       name = command_map(@command)
       o = @options.merge(mod: mod_name, yes: true, build: false, input: false)
+      o.merge!(quiet: false) if @command == "init" # noisy so can filter and summarize output
       case @command
       when "up"
         Terraspace::CLI::Up.new(o).run

data/lib/terraspace/all/summary.rb

@@ -1,6 +1,6 @@
 module Terraspace::All
   class Summary
-    include Terraspace::CLI::Log::Concern
+    include Terraspace::CLI::Logs::Concern
     include Terraspace::Util::Logging
 
     def initialize(data={})
@@ -35,6 +35,13 @@ module Terraspace::All
       end
     end
 
+    def init
+      @lines.select! do |line|
+        line.include?("successfully initialized") || # success
+        line.include?("Error: ")  # error
+      end
+    end
+
     # Example 1:
     #     [2020-09-07T14:45:14 #23340 terraspace plan b1]: No changes. Infrastructure is up-to-date.
     # Example 2:
@@ -51,7 +58,15 @@ module Terraspace::All
     end
 
     def output
-      @lines # pass through all output info
+      if @lines.grep(/No outputs found/).empty?
+        @lines.select! do |line|
+          line.include?(" = ") # looks like output
+        end
+      else
+        @lines.select! do |line|
+          line.include?("No outputs found")
+        end
+      end
     end
 
     def show
@@ -65,6 +80,18 @@ module Terraspace::All
       @lines.unshift(summary) # add to top
     end
 
+    # [2020-09-19T19:36:33 #14387 terraspace providers c1]: => terraform providers
+    # [2020-09-19T19:36:33 #14387 terraspace providers c1]:
+    # [2020-09-19T19:36:33 #14387 terraspace providers c1]: Providers required by configuration:
+    # [2020-09-19T19:36:33 #14387 terraspace providers c1]: .
+    # [2020-09-19T19:36:33 #14387 terraspace providers c1]: └── provider[registry.terraform.io/hashicorp/random]
+    # [2020-09-19T19:36:33 #14387 terraspace providers c1]:
+    def providers
+      @lines.select! do |line|
+        line.include?("provider[")
+      end
+    end
+
     # [2020-09-07T14:53:36 #31106 terraspace show b1]: Outputs:
     # [2020-09-07T14:53:36 #31106 terraspace show b1]:
     # [2020-09-07T14:53:36 #31106 terraspace show b1]: length = 1

data/lib/terraspace/app.rb

@@ -9,6 +9,8 @@ module Terraspace
     end
 
     def defaults
+      ts_logger = Logger.new(ENV['TS_LOG_PATH'] || $stderr)
+
       config = ActiveSupport::OrderedOptions.new
       config.all = ActiveSupport::OrderedOptions.new
       config.all.concurrency = 5
@@ -21,14 +23,16 @@ module Terraspace
       config.build.cache_dir = ":CACHE_ROOT/:REGION/:ENV/:BUILD_DIR"
       config.build.cache_root = nil # defaults to /full/path/to/.terraspace-cache
       config.build.clean_cache = nil # defaults to /full/path/to/.terraspace-cache
+      config.bundle = ActiveSupport::OrderedOptions.new
+      config.bundle.logger = ts_logger
       config.cloud = ActiveSupport::OrderedOptions.new
       config.cloud.auto_sync = true
-      config.cloud.working_dir_prefix = nil
       config.cloud.hostname = nil
       config.cloud.vars = ActiveSupport::OrderedOptions.new
       config.cloud.vars.overwrite = true
       config.cloud.vars.overwrite_sensitive = true
       config.cloud.vars.show_message = "create"
+      config.cloud.working_dir_prefix = nil
       config.cloud.workspace = ActiveSupport::OrderedOptions.new
       config.cloud.workspace.attrs = ActiveSupport::OrderedOptions.new
       config.hooks = Hooks.new
@@ -36,14 +40,14 @@ module Terraspace
       config.init.mode = "auto" # auto, never, always
       config.log = ActiveSupport::OrderedOptions.new
       config.log.root = Terraspace.log_root
-      config.logger = Logger.new(ENV['TS_LOG_PATH'] || $stderr)
-      config.logger.level = ENV['TS_LOG_LEVEL'] || :info
+      config.logger = ts_logger
       config.logger.formatter = Logger::Formatter.new
-      config.test_framework = "rspec"
+      config.logger.level = ENV['TS_LOG_LEVEL'] || :info
       config.terraform = ActiveSupport::OrderedOptions.new
       config.terraform.plugin_cache = ActiveSupport::OrderedOptions.new
-      config.terraform.plugin_cache.enabled = true
       config.terraform.plugin_cache.dir = ENV['TF_PLUGIN_CACHE_DIR'] || "#{Terraspace.tmp_root}/plugin_cache"
+      config.terraform.plugin_cache.enabled = true
+      config.test_framework = "rspec"
       config
     end
 

data/lib/terraspace/builder.rb

@@ -1,6 +1,8 @@
 module Terraspace
   class Builder < Terraspace::CLI::Base
+    include Compiler::CommandsConcern
     include Compiler::DirsConcern
+    include Hooks::Concern
 
     attr_reader :graph
 
@@ -13,11 +15,14 @@ module Terraspace
       placeholder_stack_message
       logger.info "Building #{build_dir}" unless @options[:quiet] # from terraspace all
 
-      build_unresolved
-      auto_create_backend
-      batches = build_batches
-      build_all
-      logger.info "Built in #{build_dir}" unless @options[:quiet] # from terraspace all
+      batches = nil
+      run_hooks("terraspace.rb", "build") do
+        build_unresolved
+        auto_create_backend
+        batches = build_batches
+        build_all
+        logger.info "Built in #{build_dir}" unless @options[:quiet] # from terraspace all
+      end
       batches
     end
 
@@ -58,16 +63,10 @@ module Terraspace
     # Auto create after build_unresolved since will need to run state pull for dependencies
     def auto_create_backend
       return if Terraspace.config.auto_create_backend == false
-      return unless create_backend?
+      return unless requires_backend?
       Terraspace::Compiler::Backend.new(@mod).create
     end
 
-    def create_backend?
-      commands = %w[down init output plan providers refresh show up validate]
-      commands.include?(ARGV[0]) ||                  # IE: terraspace up
-      ARGV[0] == "all" && commands.include?(ARGV[1]) # IE: terraspace all up
-    end
-
     def clean
       Compiler::Cleaner.new(@mod, @options).clean if clean?
     end

data/lib/terraspace/cli.rb

@@ -32,19 +32,19 @@ module Terraspace
     long_desc Help.text(:all)
     subcommand "all", All
 
+    desc "clean SUBCOMMAND", "clean subcommands"
+    long_desc Help.text(:clean)
+    subcommand "clean", Clean
+
     desc "cloud SUBCOMMAND", "cloud subcommands"
     long_desc Help.text(:cloud)
     subcommand "cloud", Cloud
 
-    desc "logs SUBCOMMAND", "logs management subcommands"
-    long_desc Help.text(:logs)
-    subcommand "logs", Logs
-
     desc "new SUBCOMMAND", "new subcommands"
     long_desc Help.text(:new)
     subcommand "new", New
 
-    desc "build [STACK]", "build"
+    desc "build [STACK]", "Build project."
     long_desc Help.text(:build)
     option :quiet, type: :boolean, desc: "quiet output"
     instance_option.call
@@ -53,32 +53,26 @@ module Terraspace
       Terraspace::Builder.new(options.merge(mod: mod)).run # building any stack builds them all
     end
 
-    desc "bundle", "bundle"
+    desc "bundle", "Bundle with Terrafile."
     long_desc Help.text(:bundle)
     def bundle(*args)
       Bundle.new(options.merge(args: args)).run
     end
 
-    desc "check_setup", "check_setup"
+    desc "check_setup", "Check setup."
     long_desc Help.text(:check_setup)
     def check_setup
       CheckSetup.new(options).run
     end
 
-    desc "clean", "clean .terraspace-cache dir"
-    long_desc Help.text(:clean)
-    def clean
-      Clean.new(options).run
-    end
-
-    desc "console STACK", "console .terraspace-cache dir"
+    desc "console STACK", "Run console in built terraform project."
     long_desc Help.text(:console)
     instance_option.call
     def console(mod)
-      Commander.new("console", options.merge(mod: mod)).run
+      Commander.new("console", options.merge(mod: mod, shell: "system")).run
     end
 
-    desc "down STACK", "down"
+    desc "down STACK", "Destroy infrastructure stack."
     long_desc Help.text(:down)
     instance_option.call
     yes_option.call
@@ -88,39 +82,39 @@ module Terraspace
       Down.new(options.merge(mod: mod)).run
     end
 
-    desc "info STACK", "info"
+    desc "info STACK", "Get info about stack."
     long_desc Help.text(:info)
-    format_option.call
     instance_option.call
+    option :format, desc: "Output formats: #{CliFormat.formats.join(', ')}"
     def info(mod)
       Info.new(options.merge(mod: mod)).run
     end
 
-    desc "init STACK", "init"
+    desc "init STACK", "Run init in built terraform project."
     long_desc Help.text(:init)
     instance_option.call
     def init(mod)
       Commander.new("init", options.merge(mod: mod, quiet: false)).run
     end
 
-    desc "list", "list stacks and modules"
+    desc "list", "List stacks and modules."
     long_desc Help.text(:list)
-    option :type, aliases: %w[t], desc: "Type: stack or module. Default all"
+    option :type, default: "stack", aliases: %w[t], desc: "Type: stack, module, or all"
     def list
       List.new(options).run
     end
 
-    desc "log [ACTION] [STACK]", "The all log command allows you to view multiple logs."
-    long_desc Help.text("log")
+    desc "logs [ACTION] [STACK]", "View and tail logs."
+    long_desc Help.text("logs")
     option :timestamps, aliases: %w[t], type: :boolean, desc: "Whether or not to show the leading timestamp. Defaults to timestamps for multiple logs, and no timestamp if a single log is specified. Note: In follow mode, timestamp always shown"
     option :follow, aliases: %w[f], type: :boolean, desc: "Follow the log in live tail fashion. Must specify a stack if using this option."
     option :limit, aliases: %w[n], default: 10, type: :numeric, desc: "Number of lines to limit showing. Only applies in no-follow mode."
     option :all, aliases: %w[a], type: :boolean, desc: "All mode turns off the limit. Defaults to all if a single log is specified. Only applies in no-follow mode."
-    def log(action=nil, stack=nil)
-      Log.new(@options.merge(action: action, stack: stack)).run
+    def logs(action=nil, stack=nil)
+      Logs.new(@options.merge(action: action, stack: stack)).run
     end
 
-    desc "plan STACK", "plan stack"
+    desc "plan STACK", "Plan stack."
     long_desc Help.text(:plan)
     auto_option.call
     input_option.call
@@ -131,21 +125,21 @@ module Terraspace
       Commander.new("plan", options.merge(mod: mod)).run
     end
 
-    desc "providers STACK", "providers"
+    desc "providers STACK", "Show providers."
     long_desc Help.text(:providers)
     instance_option.call
     def providers(mod)
       Commander.new("providers", options.merge(mod: mod)).run
     end
 
-    desc "refresh STACK", "refresh"
+    desc "refresh STACK", "Run refresh."
     long_desc Help.text(:refresh)
     instance_option.call
     def refresh(mod)
       Commander.new("refresh", options.merge(mod: mod)).run
     end
 
-    desc "seed STACK", "seed"
+    desc "seed STACK", "Build starer seed tfvars file."
     long_desc Help.text(:seed)
     option :yes, aliases: :y, type: :boolean, desc: "bypass prompts and force overwrite files"
     option :where, desc: "where to create file. either under app or seed folder structure. values: app or stack"
@@ -155,16 +149,16 @@ module Terraspace
       Seed.new(options.merge(mod: mod)).run
     end
 
-    desc "summary", "Summary of resources"
-    long_desc Help.text(:clean)
+    desc "summary", "Summarize resources."
+    long_desc Help.text(:summary)
     option :mod, desc: "Module to build to generate a backend file for discovery. By default the last module is used. Usually, it wont matter."
     init_option.call
-    option :short, aliases: %w[s], type: :boolean, desc: "Only show statefiles"
+    option :details, type: :boolean, desc: "Show details of the listed resources"
     def summary
       Summary.new(options).run
     end
 
-    desc "show STACK", "show"
+    desc "show STACK", "Run show."
     long_desc Help.text(:show)
     instance_option.call
     option :plan, desc: "path to created.plan"
@@ -173,13 +167,13 @@ module Terraspace
       Commander.new("show", options.merge(mod: mod)).run
     end
 
-    desc "test", "test"
+    desc "test", "Run test."
     long_desc Help.text(:test)
     def test
       Test.new(options).run
     end
 
-    desc "output STACK", "output"
+    desc "output STACK", "Run output."
     long_desc Help.text(:output)
     format_option.call
     instance_option.call
@@ -188,8 +182,8 @@ module Terraspace
       Commander.new("output", options.merge(mod: mod)).run
     end
 
-    desc "up STACK", "Deploy infrastructure. IE: terraform apply"
-    long_desc Help.text(:update)
+    desc "up STACK", "Deploy infrastructure stack."
+    long_desc Help.text(:up)
     auto_option.call
     init_option.call
     input_option.call
@@ -202,7 +196,7 @@ module Terraspace
       Up.new(options.merge(mod: mod)).run
     end
 
-    desc "validate STACK", "validate"
+    desc "validate STACK", "Validate stack."
     long_desc Help.text(:validate)
     instance_option.call
     def validate(mod)
@@ -221,7 +215,7 @@ module Terraspace
       Completer::Script.generate
     end
 
-    desc "version", "prints version"
+    desc "version", "Prints version."
     def version
       puts VERSION
     end