terraspace-bundler 0.2.0 → 0.3.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 500271ff2f8e5799a55febd3ec25261bcf4126bff456b83360ae0e2a96bb6357
-  data.tar.gz: ee45f81805b893d5287fdea6777c9b28ac39936273806c3d97275f6c314a8cd1
+  metadata.gz: 120ab895b524973b2a5a6367f76d8d1f8794cbc5957abae90ab11532833b8620
+  data.tar.gz: 55c69457dbd4fd1bdfce68b1432bcab70fea3d28805f28fc06961fe1c534c149
 SHA512:
-  metadata.gz: 94af4d341ed3e43445bf2eb0eae1e3383fe064a6df708c449658b8eb5a958163a225be2ddae121356573ae7e099e87669c5cb9c54808423b9db685faf27d7837
-  data.tar.gz: d68d1825f89b91ac4d9ad7525a4b4b95cc709ce1e2f3c10d83aa6ad5c6b27a89b3c6ae1ad9bc1ee3179b08345ac73fc551d1467769f702cc43d69b6f3b38b577
+  metadata.gz: 024b2f949df4eccfbdd5eaa7ce5b56a60ffc8f48f98044edd38408bd6f87a6f1e87f6b4fbcab54871884489c8742a7a92586eea27233e169cd2ea06c154bb8dc
+  data.tar.gz: 2270365d4fea228f1170601e2b02e54e9ccad230ed1dee044b9db134c0bd957f0e50fee715530ec9b1e976f174078e183cb0d598ca24eaf92ebe7f4ccc2d53e9

data/CHANGELOG.md

@@ -3,6 +3,28 @@
 All notable changes to this project will be documented in this file.
 This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [0.3.4] - 2021-04-15
+- [#11](https://github.com/boltops-tools/terraspace-bundler/pull/11) improve registry dectection and gitlab repository support
+
+## [0.3.3] - 2021-02-24
+- [#8](https://github.com/boltops-tools/terraspace-bundler/pull/8) fix https sources
+- [#9](https://github.com/boltops-tools/terraspace-bundler/pull/9) Git checkout retry with v
+
+## [0.3.2]
+- #4 only purge and export explicit mods
+- are you sure prompt for purge_cache
+- clean up update mode
+
+## [0.3.1]
+- #3 all_stacks: only consider dirs
+
+## [0.3.0]
+- #2 bundle improvements
+- change `base_clone_url` default from `git@github.com:` to `https://github.com/`
+- mod-level stack option
+- terrafile-level stack_options
+- rename terrafile-level option to export_to
+
 ## [0.2.0]
 - #1 Rework implementation. Better Terrafile syncing
 - clearer implementation: Terrafile and Lockfile classes build standard mods object.

data/README.md

@@ -1,5 +1,9 @@
 # terraspace-bundler
 
+[![Gem Version](https://badge.fury.io/rb/terraspace-bundler.png)](http://badge.fury.io/rb/terraspace-bundler)
+
+[![BoltOps Badge](https://img.boltops.com/boltops/badges/boltops-badge.png)](https://www.boltops.com)
+
 Bundles terraform modules based on a `Terrafile` to the `vendor/modules` folder. Used by the [Terraspace Terraform Framework](https://terraspace.cloud/).
 
 ## Usage
@@ -7,14 +11,13 @@ Bundles terraform modules based on a `Terrafile` to the `vendor/modules` folder.
 Create a `Terrafile` file:
 
 ```ruby
-org "boltopspro" # set default org
+org "boltops-tools" # set default org
 
 # GitHub repo with default org
-mod "elb", source: "terraform-aws-elb", version: "master"
-mod "ec2", source: "terraform-aws-ec2", version: "branch-name"
+mod "s3", source: "terraform-aws-s3", version: "master"
 
 # GitHub repo with explicit org
-mod "rds", source: "boltopspro/terraform-aws-rds", version: "v0.1.0"
+# mod "elasticache", source: "boltopspro/terraform-aws-elasticache"
 
 # Terraform registry
 mod "sg", source: "terraform-aws-modules/security-group/aws", version: "3.10.0"
@@ -26,90 +29,7 @@ Running `terraspace bundle` creates the `Terrafile.lock` file, which locks the v
 
     terraspace bundle
 
-## Updating
-
-To update all the locked versions in `Terrafile.lock` run `terraspace bundle update`. You can also delete the `Terrafile.lock` file first.
-
-    terraspace bundle update
-
-You can selectively update multiple modules:
-
-    terraspace bundle update MODS
-    terraspace bundle update mod1 mod2
-
-## List & Info
-
-    terraspace bundle list
-    terraspace bundle info mod1
-
-## DSL Methods
-
-Some of these are methods that apply globally at the Terrafile level. Some of these are options that apply the at the mod method level.
-
-### Terrafile level
-
-Name | Description | Default
---- |  --- | ---
-base_clone_url | Base clone url to use | git@gihtub.com:
-export_path |  Where the modules get exported to saved to. | vendor/modules
-export_purge | Where or not to clean up all existing exported modules folder first. | true
-
-### Mod Method level
-
-Name | Description | Default
---- |  --- | ---
-subfolder | The subfolder where the module lives within the repo. | nil
-export_to | Overrides the export_path Terrafile level option. With this one-off option, other modules in this folder will not be purged. | nil
-
-Examples of some of the options are below:
-
-### base url
-
-The base url used for clone is `git@github.com:`. You can change it with `base_url`, example:
-
-```ruby
-base_url "https://github.com/"  # default base url for git clone
-# ...
-```
-
-### export_path
-
-The default export path is `vendor/modules`, you can change it:
-
-```ruby
-export_path "app/modules"
-# ...
-```
-
-## prune
-
-The `terraspace bundle` commands removes all files in the export_path, `vendor/modules`, by default. You can disable this behavior with:
-
-```ruby
-export_purge false
-# ...
-```
-
-### subfolder
-
-The default export path is `vendor/modules`, you can change it:
-
-```ruby
-# ...
-mod "s3", source: "terraform-aws-s3", subfolder: "path/to/module/s3"
-```
-
-## Config Options
-
-You can also configure some behavior with `TB.config`. IE: `TB.config.terrafile = "/path/to/Terrafile"`
-
-Name | Description | Default
---- | --- | ---
-base_clone_url | Base git url to use for cloning | git@github.com:
-export_path | Where to export the modules to. Can also be set with the env var TB_EXPORT_PATH | vendor/modules
-export_purge | Whether or not to prune all the modules | true
-logger | Logger instance
-terrafile | The Terrafile path. Can also be set with the env TB_TERRAFILE || "Terrafile"
+For more detailed usage instructions refer to the [Terraspace Terrafile docs](https://terraspace.cloud/docs/terrafile/options/)
 
 ## Installation
 
@@ -123,7 +43,7 @@ To install:
 * Handles updating the `Terrafile.lock` based on the `Terrafile`
 * Others running the `terraspace bundle install` will install the exact same module versions based the `Terrafile.lock`.
 * To update `Terraform.lock` run `terraspace bundle update`.
-* The repos are downloaded to `/tmp/terraspace-bundler` area as a cache. Delete the cache by running `terraspace bundle purge_cache`.
+* The repos are downloaded to the `/tmp/terraspace-bundler` area as a cache. Delete the cache by running `terraspace bundle purge_cache`.
 
 ## Contributing
 

data/lib/terraspace_bundler.rb

@@ -7,6 +7,7 @@ require "dsl_evaluator"
 require "fileutils"
 require "memoist"
 require "rainbow/ext/string"
+require "singleton"
 require "terraspace_bundler/version"
 require "yaml"
 

data/lib/terraspace_bundler/cli/bundle.rb

@@ -22,11 +22,12 @@ class TerraspaceBundler::CLI
     long_desc Help.text("bundle/install")
     terrafile_option.call
     def install
-      Runner.new(options).run
+      TB::Runner.new(options).run
     end
 
     desc "purge_cache", "Purge cache."
     long_desc Help.text("bundle/purge_cache")
+    option :yes, aliases: :y, type: :boolean, desc: "bypass are you sure prompt"
     def purge_cache
       PurgeCache.new(options).run
     end
@@ -35,7 +36,8 @@ class TerraspaceBundler::CLI
     long_desc Help.text("bundle/update")
     terrafile_option.call
     def update(*mods)
-      Runner.new(options.merge(mods: mods, mode: "update")).run
+      TB.update_mode = true
+      TB::Runner.new(options.merge(mods: mods)).run
     end
   end
 end

data/lib/terraspace_bundler/cli/help/bundle/info.md

@@ -5,4 +5,4 @@
         sha: bad676a426f5d3ddb48e674bebd6fd52f063a8b4
         source: org/terraform-random-pet
         type: git
-        url: git@github.com:org/terraform-random-pet
+        url: https://github.com/org/terraform-random-pet

data/lib/terraspace_bundler/cli/purge_cache.rb

@@ -2,10 +2,28 @@ class TerraspaceBundler::CLI
   class PurgeCache < Base
     include TB::Mod::PathConcern
     include TB::Util::Logging
+    include TB::Util::Sure
 
     def run
-      FileUtils.rm_rf(tmp_root)
-      logger.info "Removed #{tmp_root}"
+      paths = [tmp_root]
+      are_you_sure?(paths)
+      paths.each do |path|
+        FileUtils.rm_rf(path)
+        logger.info "Removed #{path}"
+      end
+
+    end
+
+    def are_you_sure?(paths)
+      pretty_paths = paths.map { |p| "    #{p}" }.join("\n")
+      message = <<~EOL.chomp
+        Will remove these folders and all their files:
+
+        #{pretty_paths}
+
+        Are you sure?
+      EOL
+      sure?(message) # from Util::Sure
     end
   end
 end

data/lib/terraspace_bundler/config.rb

@@ -5,11 +5,12 @@ module TerraspaceBundler
 
     def config
       config = ActiveSupport::OrderedOptions.new
-      config.base_clone_url = "git@github.com:"
-      config.export_path = ENV['TB_EXPORT_PATH'] || "vendor/modules"
+      config.base_clone_url = "https://github.com/"
+      config.export_to = ENV['TB_EXPORT_TO'] || "vendor/modules"
       config.export_purge = ENV['TB_EXPORT_PRUNE'] == '0' ? false : true
       config.lockfile = "#{config.terrafile}.lock"
       config.logger = new_logger
+      config.stack_options = {dest: "app/stacks", purge: nil, examples: "examples"}  # Note: Important purge is nil not false so can fallback to Terrafile-level stack_options
       config.terrafile = ENV['TB_TERRAFILE'] || "Terrafile"
       config
     end

data/lib/terraspace_bundler/core.rb

@@ -22,5 +22,15 @@ module TerraspaceBundler
       dsl
     end
     memoize :dsl
+
+    @@update_mode = false
+    def update_mode
+      @@update_mode
+    end
+    alias_method :update_mode?, :update_mode
+
+    def update_mode=(v)
+      @@update_mode = v
+    end
   end
 end

data/lib/terraspace_bundler/dsl/syntax.rb

@@ -1,28 +1,32 @@
 class TerraspaceBundler::Dsl
   module Syntax
     def org(url)
-      global[:org] = url
+      config.org = url
     end
     alias_method :user, :org
 
     def base_clone_url(value)
-      TB.config.base_clone_url = value
+      config.base_clone_url = value
     end
 
-    def export_path(path)
-      global[:export_path] = path
+    def export_to(path)
+      config.export_to = path
     end
 
     def export_purge(value)
-      TB.config.export_purge = value
+      config.export_purge = value
     end
 
-    def mod(*args, **options)
-      meta[:mods] << {args: args, options: options}
+    def stack_options(value={})
+      config.stack_options.merge!(value)
+    end
+
+    def config
+      TB.config
     end
 
-    def global
-      meta[:global]
+    def mod(*args, **options)
+      meta[:mods] << {args: args, options: options}
     end
   end
 end

data/lib/terraspace_bundler/exporter.rb

@@ -8,36 +8,42 @@ module TerraspaceBundler
     end
 
     def run
-      purge
-      lockfile.mods.each do |mod|
+      purge_all
+      mods.each do |mod|
+        logger.info "Exporting #{mod.name}"
+        purge(mod)
         export(mod)
       end
     end
 
+    def mods
+      mods = lockfile.mods
+      if TB.update_mode? && !@options[:mods].empty?
+        mods.select! { |mod| @options[:mods].include?(mod.name) }
+      end
+      mods
+    end
+
     def export(mod)
       downloader = Mod::Downloader.new(mod)
+      downloader.run
       downloader.switch_version(mod.sha)
-
-      stage_path = stage_path(mod.full_repo)
-      stage_path = "#{stage_path}/#{mod.subfolder}" if mod.subfolder
-      mod_path = mod_path(mod)
-      FileUtils.rm_rf(mod_path)
-      FileUtils.mkdir_p(File.dirname(mod_path))
-      FileUtils.cp_r(stage_path, mod_path)
-      FileUtils.rm_rf("#{mod_path}/.git")
-      logger.debug "Exported: #{mod_path}"
-    end
-
-    def mod_path(mod)
-      name = mod.name
-      export_to = mod.export_to || TB.config.export_path
-      "#{export_to}/#{name}"
+      copy = Copy.new(mod)
+      copy.mod
+      copy.stacks
+      logger.debug "Exported: #{copy.mod_path}"
     end
 
   private
-    def purge
+    def purge_all
+      return if TB.update_mode?
       return unless TB.config.export_purge
-      FileUtils.rm_rf(TB.config.export_path)
+      FileUtils.rm_rf(TB.config.export_to)
+    end
+
+    def purge(mod)
+      mod_path = get_mod_path(mod)
+      FileUtils.rm_rf(mod_path)
     end
 
     def lockfile

data/lib/terraspace_bundler/exporter/base.rb

@@ -0,0 +1,6 @@
+class TerraspaceBundler::Exporter
+  class Base
+    include TB::Mod::PathConcern
+    include TB::Util::Logging
+  end
+end

data/lib/terraspace_bundler/exporter/copy.rb

@@ -0,0 +1,25 @@
+class TerraspaceBundler::Exporter
+  class Copy < Base
+    def initialize(mod)
+      @mod = mod
+    end
+
+    def mod
+      FileUtils.rm_rf(mod_path)
+      FileUtils.mkdir_p(File.dirname(mod_path))
+      FileUtils.cp_r(src_path, mod_path)
+      FileUtils.rm_rf("#{mod_path}/.git")
+    end
+
+    def stacks
+      Stacks.new(@mod).export
+    end
+
+    # src path is from the stage area
+    def src_path
+      path = stage_path(@mod.full_repo) # from PathConcern
+      path = "#{path}/#{@mod.subfolder}" if @mod.subfolder
+      path
+    end
+  end
+end

data/lib/terraspace_bundler/exporter/stacks.rb

@@ -0,0 +1,14 @@
+class TerraspaceBundler::Exporter
+  class Stacks < Base
+    def initialize(mod)
+      @mod = mod
+      @stacks = mod.stacks || []
+    end
+
+    def export
+      @stacks.each do |options|
+        Stack.new(@mod, options).export
+      end
+    end
+  end
+end

data/lib/terraspace_bundler/exporter/stacks/base.rb

@@ -0,0 +1,5 @@
+class TerraspaceBundler::Exporter::Stacks
+  class Base < TerraspaceBundler::Exporter::Base
+    include TB::Util::Logging
+  end
+end

data/lib/terraspace_bundler/exporter/stacks/rewrite.rb

@@ -0,0 +1,57 @@
+class TerraspaceBundler::Exporter::Stacks
+  class Rewrite < Base
+    def initialize(stack)
+      @stack = stack
+    end
+
+    def run
+      expr = "#{@stack.dest}/*.tf"
+      Dir.glob(expr).each do |path|
+        next unless File.file?(path) # skip symlinks and dirs
+        replace(path)
+      end
+    end
+
+    def replace(path)
+      lines = IO.readlines(path)
+      new_lines = new_lines(lines)
+      text = new_lines.join('')
+      IO.write(path, text)
+    end
+
+    def new_lines(lines)
+      lines.map do |line|
+        # leading spaces or no space. but cannot have any other chars
+        # can have space betweens source and =
+        line =~ /^\s*source\s*=/ ? new_line(line) : line
+      end
+    end
+
+    def new_line(line)
+      # marker comment so files dont get updated twice when purge is false
+      terraspace_comment = "# updated by terraspace"
+      return line unless line.include?("../")
+      return line if line.include?(terraspace_comment)
+
+      md = line.match(/^\s*source\s*=\s*"(.*)?"/)
+      unless md
+        logger.error "ERROR: unable to find the module source".color(:red)
+        logger.error line
+        exit 1
+      end
+      # Lots of cases covered by specs
+      #
+      #     ../..
+      #     ../../
+      #     ../../modules/iam-user
+      #     ../../../modules/compute_instance
+      #
+      source_value = md[1] # IE: ""../../../modules/compute_instance"
+      dirs = source_value.split('/')
+      count = dirs.count('..')
+      mod_dir = dirs[count..-1].join('/')
+      full_dir = [@stack.mod.name, mod_dir].reject(&:empty?).join('/')
+      %Q|  source = "../../modules/#{full_dir}" #{terraspace_comment}\n|
+    end
+  end
+end