simplygenius-atmos 0.7.1 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9aaf452ae6008398a6c4d8c49060cf619642eb416b1c5f5a598f46b8682fd45e
-  data.tar.gz: 2ab6e70ece053abb50ead60f1231dbd367bc2328a88286747aacbeef875cd93d
+  metadata.gz: 5761d35141b55d13b0e372081647d0a898aa0dda202d12c02f4c6728d671600f
+  data.tar.gz: b2b0650b44c07c757f106b7f8f4c55ac3a0b44457f337c06620adb934340db16
 SHA512:
-  metadata.gz: 279577af65179d0755e33cf157185ce06856ca4d9916c723e3474e766f5f30bf5d01eb2ac969cd9e8d712808b64318a30877f0bca5312181667d4c415ca9e69c
-  data.tar.gz: 712356cea2ce2c9ff66ddea22910c6c4e9a44b7c29eb910cf2dae4669bb28ea91bddb74d32fa6c7472714457a72beaa852517ba563613a1236a47a9f3401bd27
+  metadata.gz: 889d94f2c5743bccab0a8e53d76a936585f4410bd1a18ee9df46036da8a3b718833a73f8aaed475ae1278708532f113f4f96b7dc43833bef8b5bab7fac0520d6
+  data.tar.gz: 87c5c8896b562ed95898584284a741bb0d7dc903e0365e197054f73e00c795eaf4200a6c0ec4cfccc8a66f4aa5f15c0ede4e8e7c16b628ce83cdf6eb96deb2ef

data/README.md

@@ -6,7 +6,7 @@
 
 Atmos(phere) - Breathe easier with terraform
 
-Atmos provides a layer of organization on top of terraform for creating cloud system architectures with Amazon Web Services.  It handles the plumbing so you can focus on your application.  The core atmos runtime is free and open-source, with a business friendly license (Apache).  It provides some basic recipes to help get you going with a service oriented architecture implemented with AWS Elastic Container Services.  For more in-depth recipes, please check out the Atmos Pro offering.
+Atmos provides a layer of organization on top of terraform for creating cloud system architectures with Amazon Web Services.  It handles the plumbing so you can focus on your application.  The core atmos runtime is free and open-source, with a business friendly license (Apache).  It provides some basic recipes to help get you going with a service oriented architecture implemented with AWS Elastic Container Services.  For more in-depth recipes, please check out the [Atmos Pro](https://simplygenius.com/atmos-pro) offering.
 
 In a nutshell, you provide the green, atmos delivers the rest:
 
@@ -18,7 +18,7 @@ In a nutshell, you provide the green, atmos delivers the rest:
 
 * The command line is king.  Using a CLI to iterate on and manage core infrastructure has always been more effective for me, so I aim to make things as convenient and usable as possible from there.
 
-* No lock-in.  Since atmos just provides you convenience on top of using terraform, and the templates for defining your infrastructure are 99% in terraform, its possible to migrate away (albeit giving up all the convenience atmos provides) if your goals ever diverge from those of atmos.
+* No lock-in.  Since atmos just provides you convenience on top of using terraform, and the templates for defining your infrastructure are entirely in terraform, its possible to migrate away (albeit giving up all the convenience atmos provides) if your goals ever diverge from those of atmos.
 
 * Your infrastructure is an important product.  It should have its own repo and be tracked under configuration management, not just clickety-click-clicked on in a UI and promptly forgotten what it is you actually did to get there.  The only guarantee you have, is that things are going to need to change, and you'll be much better off with a system that allows you to iterate easily.  Atmos gets you started with minimal up-front knowledge, but provides a path for your infrastructure to evolve.
 
@@ -48,7 +48,7 @@ To install as a gem:
  * gem install simplygenius-atmos
  * verify: `atmos --help`
 
-To install/run as a docker image:
+To install/run as a docker image (experimental):
  * curl -sL https://raw.githubusercontent.com/simplygenius/atmos/master/exe/atmos-docker > /usr/local/bin/atmos
  * chmod +x /usr/local/bin/atmos
  * verify: `atmos --help`
@@ -216,4 +216,4 @@ The gem is available as open source under the terms of the [Apache 2.0 License](
 
 # About Us
 
-Simply Genius LLC is an independently run organization in Boston, MA USA.  Its Chief Everything is Matt Conway, a software engineer and executive with more than 20 years of experience in the Boston Tech Startup scene.  Atmos is his attempt at providing the world with the same tools, techniques, mindset and philosophy that he strives for when building a system architecture for a new startup.
+[Simply Genius LLC](https://simplygenius.com) is an independently run organization in Boston, MA USA.  Its Chief Everything is Matt Conway, a software engineer and executive with more than 20 years of experience in the Boston Tech Startup scene.  Atmos is his attempt at providing the world with the same tools, techniques, mindset and philosophy that he strives for when building a system architecture for a new startup.

data/exe/atmos

@@ -1,4 +1,4 @@
 #!/usr/bin/env ruby
 
-require_relative '../lib/atmos/cli'
-Atmos::CLI.run
+require_relative '../lib/simplygenius/atmos/cli'
+SimplyGenius::Atmos::CLI.run

data/lib/{atmos.rb → simplygenius/atmos.rb}

@@ -1,12 +1,14 @@
 require_relative 'atmos/version'
-
-require_relative 'atmos/logging'
-Atmos::Logging.setup_logging(false, false, nil)
-
-require_relative 'atmos/config'
 require 'active_support/core_ext/string'
 require 'active_support/concern'
 
-module Atmos
-  mattr_accessor :config
+module SimplyGenius
+  module Atmos
+    mattr_accessor :config
+  end
 end
+
+require_relative 'atmos/logging'
+# SimplyGenius::Atmos::Logging.setup_logging(false, false, nil)
+
+require_relative 'atmos/config'

data/lib/simplygenius/atmos/cli.rb

@@ -0,0 +1,116 @@
+require_relative '../atmos'
+require_relative '../atmos/ui'
+require_relative '../atmos/plugins/prompt_notify'
+require 'clamp'
+require 'sigdump/setup'
+
+Dir.glob(File.join(File.join(__dir__, 'commands'), '*.rb')) do |f|
+  require_relative "commands/#{File.basename(f).sub(/\.rb$/, "")}"
+end
+
+module SimplyGenius
+  module Atmos
+
+    # The command line interface to atmos
+    class CLI < Clamp::Command
+
+      include GemLogger::LoggerSupport
+
+      def self.description
+        desc = <<-DESC
+          Atmos version #{VERSION}
+    
+          Runs The atmos command line application
+    
+          e.g.
+    
+          atmos --help
+        DESC
+        desc.split("\n").collect(&:strip).join("\n")
+      end
+
+      option ["-d", "--debug"],
+             :flag, "debug output\n",
+             default: false
+
+      option ["-c", "--[no-]color"],
+             :flag, "colorize output (or not)\n (default: $stdout.tty?)"
+
+      option ["-e", "--atmos-env"],
+             'ENV', "The atmos environment\n",
+             environment_variable: 'ATMOS_ENV', default: 'ops'
+
+      option ["-l", "--load-path"],
+             "PATH", "adds additional paths to ruby load path",
+             multivalued: true
+
+      def default_color?
+         $stdout.tty?
+      end
+
+      option ["-l", "--[no-]log"],
+             :flag, "log to file in addition to terminal (or not)\n",
+             default: true
+
+
+      subcommand "new", "Sets up a new atmos project",
+                 Commands::New
+      subcommand "generate", "Generate recipes for the repository",
+                 Commands::Generate
+      subcommand "bootstrap", "Init cloud provider for use with atmos",
+                 Commands::Bootstrap
+      subcommand "init", "Run terraform init",
+                 Commands::Init
+      subcommand "plan", "Run terraform plan",
+                 Commands::Plan
+      subcommand "apply", "Run terraform apply",
+                 Commands::Apply
+      subcommand "destroy", "Run terraform destroy",
+                 Commands::Destroy
+      subcommand "terraform", "Run all other terraform commands",
+                 Commands::Terraform
+      subcommand "account", "Account management commands",
+                 Commands::Account
+      subcommand "user", "User management commands",
+                 Commands::User
+      subcommand "otp", "Otp tools",
+                 Commands::Otp
+      subcommand "secret", "Secret management commands",
+                 Commands::Secret
+      subcommand "auth_exec", "Authenticated exec",
+                 Commands::AuthExec
+      subcommand "container", "Container tools",
+                 Commands::Container
+
+      subcommand "version", "Display version" do
+        def execute
+          logger.info "Atmos Version #{VERSION}"
+        end
+      end
+
+      subcommand "config", "Display expanded config for atmos_env" do
+        def execute
+          logger.info YAML.dump(Atmos.config.to_h)
+        end
+      end
+
+      # hook into clamp lifecycle to force logging setup even when we are calling
+      # a subcommand
+      def parse(arguments)
+        super
+        if Atmos.config.nil?
+          Atmos.config = Config.new(atmos_env)
+          log = Atmos.config.is_atmos_repo? && log? ? "atmos.log" : nil
+          Logging.setup_logging(debug?, color?, log)
+          UI.color_enabled = color?
+
+          Atmos.config.add_user_load_path(*load_path_list)
+          Atmos.config.plugin_manager.register_output_filter(:stdout, Plugins::PromptNotify)
+          Atmos.config.plugin_manager.load_plugins
+        end
+      end
+
+    end
+
+  end
+end

data/lib/simplygenius/atmos/commands/account.rb

@@ -0,0 +1,69 @@
+require_relative 'base_command'
+require_relative '../../atmos/settings_hash'
+require 'climate_control'
+
+module SimplyGenius
+  module Atmos
+    module Commands
+
+      class Account < BaseCommand
+
+        def self.description
+          "Manages accounts/envs in the cloud provider"
+        end
+
+        subcommand "create", "Create a new account" do
+
+          option ["-s", "--source-env"],
+                 "SOURCE_ENV", "Base the new env on a clone of the given one\n"
+
+          option ["-e", "--email"],
+                 "EMAIL", "override default email used for new account\n"
+
+          option ["-n", "--name"],
+                 "NAME", "override default name used for new account\n"
+
+          parameter "ENV",
+                    "The name of the new env to create"
+
+          def execute
+
+            Atmos.config.provider.auth_manager.authenticate(ENV) do |auth_env|
+              ClimateControl.modify(auth_env) do
+
+                config = YAML.load_file(Atmos.config.config_file)
+
+                if config['environments'][env]
+                  signal_usage_error "Env '#{env}' is already present in atmos config"
+                end
+
+                source = {}
+                if source_env.present?
+                  source = config['environments'][source_env]
+                  if source.blank?
+                    signal_usage_error "Source env '#{source_env}' does not exist"
+                  end
+                  source = source.clone
+                end
+
+                account = Atmos.config.provider.account_manager.create_account(env, name: name, email: email)
+                logger.info "Account created: #{display account}"
+
+                source['account_id'] = account[:account_id].to_s
+
+                new_yml = SettingsHash.add_config(
+                    Atmos.config.config_file,
+                    "environments.#{env}", source
+                )
+                logger.info("Writing out new atmos.yml containing new account")
+                File.write(Atmos.config.config_file, new_yml)
+              end
+            end
+          end
+        end
+
+      end
+
+    end
+  end
+end

data/lib/simplygenius/atmos/commands/apply.rb

@@ -0,0 +1,24 @@
+require_relative 'terraform'
+
+module SimplyGenius
+  module Atmos
+    module Commands
+
+      class Apply < Terraform
+
+        def self.description
+          "Runs terraform apply"
+        end
+
+        def execute
+          args = ["apply"]
+          args << "--get-modules" unless Atmos.config["disable_auto_modules"].to_s == "true"
+          @terraform_arguments.insert(0, *args)
+          super
+        end
+
+      end
+
+    end
+  end
+end

data/lib/simplygenius/atmos/commands/auth_exec.rb

@@ -0,0 +1,34 @@
+require_relative 'base_command'
+require 'climate_control'
+
+module SimplyGenius
+  module Atmos
+    module Commands
+
+      class AuthExec < BaseCommand
+
+        def self.description
+          "Exec subprocess with an authenticated environment"
+        end
+
+        option ["-r", "--role"],
+               'ROLE', "overrides assume role name\n"
+
+        parameter "COMMAND ...", "command to exec", :attribute_name => :command
+
+        def execute
+          Atmos.config.provider.auth_manager.authenticate(ENV, role: role) do |auth_env|
+            result = system(auth_env, *command)
+            if ! result
+              logger.error("Process failed: #{command}")
+              exit(1)
+            end
+          end
+
+        end
+
+      end
+
+    end
+  end
+end

data/lib/simplygenius/atmos/commands/base_command.rb

@@ -0,0 +1,16 @@
+require_relative '../../atmos'
+require_relative '../../atmos/ui'
+require 'clamp'
+
+module SimplyGenius
+  module Atmos
+    module Commands
+
+      class BaseCommand < Clamp::Command
+        include GemLogger::LoggerSupport
+        include UI
+      end
+
+    end
+  end
+end

data/lib/simplygenius/atmos/commands/bootstrap.rb

@@ -0,0 +1,76 @@
+require_relative 'base_command'
+
+module SimplyGenius
+  module Atmos
+    module Commands
+
+      class Bootstrap < BaseCommand
+
+        def self.description
+          "Sets up the initial aws account for use by atmos"
+        end
+
+        option ["-f", "--force"],
+               :flag, "forces bootstrap\n"
+
+        def execute
+
+          tf_init_dir = File.join(Atmos.config.tf_working_dir('bootstrap'), '.terraform')
+          tf_initialized = File.exist?(tf_init_dir)
+          backend_initialized = File.exist?(File.join(tf_init_dir, 'terraform.tfstate'))
+
+          rebootstrap_msg = <<~EOF
+            Bootstrap should only be performed when provisioning an account for the first
+            time.  Try 'atmos terraform init'
+          EOF
+
+          if !force? && tf_initialized
+            signal_usage_error(rebootstrap_msg)
+          end
+
+          Atmos.config.provider.auth_manager.authenticate(ENV, bootstrap: true) do |auth_env|
+            begin
+              exe = TerraformExecutor.new(process_env: auth_env, working_group: 'bootstrap')
+
+              skip_backend = true
+              skip_secrets = true
+              if backend_initialized
+                skip_backend = false
+                skip_secrets = false
+              end
+
+              # Cases
+              # 1) bootstrap of new account - success
+              # 2) repeating bootstrap of new account due to failure partway - success
+              # 3) try to rebootstrap existing account on fresh checkout - should fail trying to create resources of same name, check output for this?
+              # 4) bootstrap new account with no-default secrets
+
+              # Need to init before we can create the resources to store state in bootstrap
+              exe.run("init", "-input=false", "-lock=false",
+                      skip_backend: true, skip_secrets: true)
+
+              # Bootstrap to create the resources needed to store state
+              exe.run("apply", "-input=false",
+                      skip_backend: true, skip_secrets: true)
+
+              # Need to init to setup the backend state after we create the resources
+              # to store state in bootstrap
+              exe.run("init", "-input=false", "-force-copy", skip_secrets: true)
+
+              # Might as well init the non-bootstrap case as well once the state
+              # storage has been setup in bootstrap
+              exe = TerraformExecutor.new(process_env: auth_env)
+              exe.run("init", "-input=false", skip_secrets: true)
+
+            rescue TerraformExecutor::ProcessFailed => e
+              logger.error(e.message)
+              logger.error(rebootstrap_msg)
+            end
+          end
+        end
+
+      end
+
+    end
+  end
+end

data/lib/simplygenius/atmos/commands/container.rb

@@ -0,0 +1,62 @@
+require_relative 'base_command'
+require_relative '../../atmos/settings_hash'
+require 'climate_control'
+
+module SimplyGenius
+  module Atmos
+    module Commands
+
+      class Container < BaseCommand
+
+        def self.description
+          "Manages containers in the cloud provider"
+        end
+
+        subcommand "deploy", "Deploy a container" do
+
+          option ["-c", "--cluster"],
+                 "CLUSTER", "The cluster name\n",
+                 required: true
+
+          option ["-r", "--role"],
+                 "ROLE", "The role to assume when deploying\n"
+
+          option ["-i", "--image"],
+                 "IMAGE", "The local container image to deploy\nDefaults to service/task name"
+
+          option ["-t", "--task"],
+                 :flag, "Deploy as a task, not a service\n"
+
+          option ["-v", "--revision"],
+                 "REVISION", "Use as the remote image revision\n"
+
+          parameter "NAME",
+                    "The name of the service (or task) to deploy"
+
+          def default_image
+            name
+          end
+
+          def execute
+            Atmos.config.provider.auth_manager.authenticate(ENV, role: role) do |auth_env|
+              ClimateControl.modify(auth_env) do
+                mgr = Atmos.config.provider.container_manager
+
+                result = mgr.push(name, image, revision: revision)
+                if task?
+                  result = result.merge(mgr.deploy_task(name, result[:remote_image]))
+                else
+                  result = result.merge(mgr.deploy_service(cluster, name, result[:remote_image]))
+                end
+
+                logger.info "Container deployed:\n #{display result}"
+              end
+            end
+          end
+        end
+
+      end
+
+    end
+  end
+end