lono 3.5.0 → 4.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ea871b8860abb2c1fbfc39edeae04a90c43f46780dd8fd4c37a1b97ca6480370
-  data.tar.gz: d52b9f70968c615adb50d16b3c84327b2300123b049a48abfa3affd2e5d6699a
+  metadata.gz: 813d3d801fda95c8b91d3664909e105efa0233aa589324dd8480ff70d8364e83
+  data.tar.gz: 47425854afafa70350676c4e2a8af6b409c10f272d0547e679d45bec168ed0e5
 SHA512:
-  metadata.gz: 93bff1d68869ca94ba8ddc8014ceef8f403962d3f0f04261a6461c76e5e354fd2780e8c50e0a1cdcdb458ef5820ee2683014249a163b4d13e7a3f2e4afe0483b
-  data.tar.gz: cf812cbf0e9eb23190ddeb615d5d7471bcd19ea31b1094e80ec642ac3b04fa15a2123c8cf43431cf1e8331d41ae3b34e4bbd665167d7fbced8a22c4717a76526
+  metadata.gz: c8c3bcc99b77b005f58d93d6893e44e74badf2bb86dc44dda78d0d364f003c4473a847ad7875ccbd9fac77177e884214dfdd0c546e80b0d5e9013a4652bacf2a
+  data.tar.gz: '049cebb1112d4ddacbe892965b615fffd93b0fc96ced048aae471d4798d42852f544f3a2ba51989ea711a88661cb3ccb76104afac4c5e204cf21efe52f7736b4'

data/.gitignore

@@ -1,10 +1,21 @@
-.DS_Store
 *.gem
+*.rbc
 .bundle
+.config
+.DS_Store
+.yardoc
+_yardoc
 coverage
+doc/
 Gemfile.lock
-pkg
-tmp
+InstalledFiles
+lib/bundler/man
 output
-spec/project
+pkg
+rdoc
 spec/fixtures/my_project/templates/aws-waf-security-automations.yml
+spec/project
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/.rspec

@@ -1,2 +1,3 @@
 --color
 --format documentation
+--require spec_helper

data/CHANGELOG.md

@@ -3,6 +3,20 @@
 All notable changes to this project will be documented in this file.
 This project *tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [4.0.0]
+- lono upgrade4 command
+- simplified lono project structure
+- app/scripts upload support and extract_scripts helper
+- new settings.yml format: environment support, simplified s3_folder and aws_profiles options
+- revamped starter projects
+- added focus to lono import flow
+- move lono summary up to top-level cli
+- cli auto-completion support
+- improved YAML parse error: shows error line immediately to user of output template
+- better ERB template error rendering with render_me_pretty gem
+- remove json support. focus on yaml.
+- lono user_data command
+
 ## [3.5.0]
 - Using Lono.root instead of scattered project_root parameters
 - prefer use of Lono.env over LONO_ENV internally
@@ -65,7 +79,7 @@ This project *tries* to adhere to [Semantic Versioning](http://semver.org/), eve
 - Format and Extension Detection
 - Custom Helper Support
 - Source Name Convention Support
-- Settings File Support
+- Setting File Support
 
 ## [2.1.0]
 - improve instance_eval error when lono.rb errors, print out line of code and context

data/Gemfile

@@ -1,6 +1,6 @@
 source "https://rubygems.org"
+
+# Specify your gem dependencies in lono.gemspec
 gemspec
 
-group :development do
-  gem 'coveralls', require: false
-end
+gem "codeclimate-test-reporter", group: :test, require: nil

data/Guardfile

@@ -1,10 +1,19 @@
-guard 'bundler' do
-  watch('Gemfile')
+guard "bundler", cmd: "bundle" do
+  watch("Gemfile")
+  watch(/^.+\.gemspec/)
 end
 
-guard 'rspec' do
-  watch(%r{^spec/.+_spec\.rb$})
-  watch(%r{^lib/(.+)\.rb$})     { |m| "spec/lib/#{m[1]}_spec.rb" }
-  watch(%r{^lib/.*/(.+)\.rb$})     { |m| "spec/lib/lono_spec.rb" }
-  # watch('spec/spec_helper.rb')  { "spec" }
-end
+guard :rspec, cmd: "bundle exec rspec" do
+  require "guard/rspec/dsl"
+  dsl = Guard::RSpec::Dsl.new(self)
+
+  # RSpec files
+  rspec = dsl.rspec
+  watch(rspec.spec_helper) { rspec.spec_dir }
+  watch(rspec.spec_support) { rspec.spec_dir }
+  watch(rspec.spec_files)
+
+  # Ruby files
+  ruby = dsl.ruby
+  dsl.watch_spec_files_for(ruby.lib_files)
+end

data/{LICENSE → LICENSE.txt}

@@ -1,4 +1,4 @@
-Copyright (c) 2017 Tung Nguyen
+Copyright (c) 2018 Tung Nguyen
 
 MIT License
 

data/README.md

@@ -1,3 +1,7 @@
+<div align="center">
+  <img src="http://lono.cloud/img/logos/lono-logo.png" style="max-width: 300px" />
+</div>
+
 # Lono
 
 [![Gem Version](https://badge.fury.io/rb/lono.png)](http://badge.fury.io/rb/lono)
@@ -11,14 +15,20 @@
 [3]: https://codeclimate.com/repos/51d7f1407e00a4042c010ab4/badges/5273fe6cdb5a13e58554/gpa.png
 [4]: https://codeclimate.com/repos/51d7f1407e00a4042c010ab4/feed
 
-Lono is a tool to help you easily manage your CloudFormation templates. Lono handles the entire CloudFormation lifecyle. It starts with helping you craft of the templates and helps you all the way to end when you provision of the infrastructure.
+Lono is a tool to help you manage your CloudFormation templates. Lono handles the entire CloudFormation lifecyle. It helps you craft the templates and then provisions of the infrastructure.
 
-* Lono generates CloudFormation templates based on ERB ruby templates in either `yaml` or `json` format.
-* Lono takes simple env-like files to and generates the CloudFormation parameter files.
-* Lono wraps the CloudFormation api calls in a simple interface using the generated files to launch the CloudFormation stacks.
+* Lono generates CloudFormation templates based on ERB ruby templates in `yaml` format.
+* Lono takes simple env-like files and generates the CloudFormation parameter files.
+* Lono provides a simple CLI interface to launch the CloudFormation stacks.
 
 See [lono.cloud](http://lono.cloud) for full lono documentation.
 
+## Important
+
+If you are on version 3, you can run `lono upgrade4` within your project to upgrade it to version 4.  Refer to the [CHANGELOG](CHANGELOG.md).
+
+## Blog Posts
+
 These blog posts also cover lono:
 
 * [Why Generate CloudFormation Templates with Lono](https://medium.com/boltops/why-generate-cloudformation-templates-with-lono-65b8ea5eb87d)
@@ -35,23 +45,21 @@ It only takes a couple of commands to start using lono.
 gem install lono
 lono new infra
 cd infra
-lono generate # not needed but showing for explanation
-lono cfn create example
+lono import https://s3-us-west-2.amazonaws.com/cloudformation-templates-us-west-2/EC2InstanceWithSecurityGroupSample.template --name ec2
+# update the generated params file
+lono cfn create ec2
 ```
-
-This sets up a starter lono project called infra with example templates.  You cd into the folder and call `lono cfn create` which automatically generates the CloudFormation template and parameter files to `output` and `output/params` and launches the stack.
-
 ![Lono flowchart](http://tongueroo.com/images/github-readmes/lono-flowchart.png "Lono flowchart")
 
-### lono cfn summary
+### Lono Cfn Lifecycle Commands
 
-Lono also provides a `lono cfn` management command that allows you to launch stacks from the lono templates.  The `lono cfn` tool automatically runs `lono generate` internally and then launches the CloudFormation stack all in one command.  Provided that you are in a lono project and have a `my-stack` lono template definition.  To create a stack you can simply run:
+Lono provides a `lono cfn` lifecycle command that allows you to launch stacks quickly.  The `lono cfn` tool automatically runs `lono generate` internally and then launches the CloudFormation stack all in one command.  If you are in a lono project and have a `my-stack` lono template definition.  To create a stack you can simply run:
 
 ```
 $ lono cfn create my-stack
 ```
 
-The above command will generate files to `output/my-stack.json` and `output/params/my-stack.txt` and use them to create a CloudFormation stack.  Here are some more examples of cfn commands:
+The above command will generate files to `output/templates/my-stack.json` and `output/params/my-stack.txt` and use them to create a CloudFormation stack.  Here are some more examples of cfn commands:
 
 ```
 $ lono cfn create mystack-$(date +%Y%m%d%H%M%S) --template mystack --params mystack

data/Rakefile

@@ -1,7 +1,6 @@
-#!/usr/bin/env rake
 require "bundler/gem_tasks"
 require "rspec/core/rake_task"
 
-task default: :spec
+task :default => :spec
 
 RSpec::Core::RakeTask.new

data/{bin → exe}/lono

@@ -6,5 +6,6 @@ Signal.trap("INT") {
   sleep 1
   exit
 }
+
 require_relative "../lib/lono"
 Lono::CLI.start

data/lib/lono.rb

@@ -1,10 +1,10 @@
-require 'json'
-require 'yaml'
-require 'pp'
+require 'active_support/core_ext/string'
 require 'colorize'
 require 'fileutils'
-# http://guides.rubyonrails.org/active_support_core_extensions.html#inflections
-require 'active_support/core_ext/string'
+require 'json'
+require 'pp'
+require 'render_me_pretty'
+require 'yaml'
 
 # vendor because need https://github.com/futurechimp/plissken/pull/6 to be merged
 $:.unshift(File.expand_path("../../vendor/plissken/lib", __FILE__))
@@ -19,17 +19,20 @@ module Lono
   autoload :Command, 'lono/command'
   autoload :CLI, 'lono/cli'
   autoload :New, 'lono/new'
+  autoload :Sequence, 'lono/sequence'
   autoload :Template, 'lono/template'
   autoload :Cfn, 'lono/cfn'
   autoload :Param, 'lono/param'
   autoload :Clean, 'lono/clean'
-  autoload :Settings, 'lono/settings'
+  autoload :Setting, 'lono/setting'
   autoload :Importer, 'lono/importer'
   autoload :Inspector, 'lono/inspector'
-  autoload :CurrentRegion, 'lono/current_region'
+  autoload :Completion, 'lono/completion'
+  autoload :Completer, 'lono/completer'
   autoload :Core, 'lono/core'
+  autoload :Upgrade4, 'lono/upgrade4'
+  autoload :Script, 'lono/script'
+  autoload :UserData, 'lono/user_data'
 
   extend Core
 end
-
-Lono.setup!

data/lib/lono/cfn.rb

@@ -1,8 +1,7 @@
 require "thor"
 
 class Lono::Cfn < Lono::Command
-  autoload :Help, 'lono/cfn/help'
-  autoload :AwsServices, 'lono/cfn/aws_services'
+  autoload :AwsService, 'lono/cfn/aws_service'
   autoload :Util, 'lono/cfn/util'
   autoload :CLI, 'lono/cfn/cli'
   autoload :Base, 'lono/cfn/base'
@@ -21,20 +20,19 @@ class Lono::Cfn < Lono::Command
   class_option :template, desc: "override convention and specify the template file to use"
   class_option :param, desc: "override convention and specify the param file to use"
   class_option :lono, type: :boolean, desc: "invoke lono to generate CloudFormation templates", default: true
-  class_option :s3_upload, type: :boolean, desc: "uploads templates to s3 if s3.path detected", default: true
   class_option :capabilities, type: :array, desc: "iam capabilities. Ex: CAPABILITY_IAM, CAPABILITY_NAMED_IAM"
   class_option :iam, type: :boolean, desc: "Shortcut for common IAM capabilities: CAPABILITY_IAM, CAPABILITY_NAMED_IAM"
   class_option :rollback, type: :boolean, desc: "rollback", default: true
 
   desc "create STACK", "create a CloudFormation stack"
   option :randomize_stack_name, type: :boolean, desc: "tack on random string at the end of the stack name", default: nil
-  long_desc Help.create
+  long_desc Lono::Help.text("cfn/create")
   def create(name)
     Create.new(name, options).run
   end
 
   desc "update STACK", "update a CloudFormation stack"
-  long_desc Help.update
+  long_desc Lono::Help.text("cfn/update")
   option :change_set, type: :boolean, default: true, desc: "Uses generated change set to update the stack.  If false, will perform normal update-stack."
   option :diff, type: :boolean, default: true, desc: "Show diff of the source code template changes before continuing."
   option :preview, type: :boolean, default: true, desc: "Show preview of the stack changes before continuing."
@@ -44,14 +42,14 @@ class Lono::Cfn < Lono::Command
   end
 
   desc "delete STACK", "delete a CloudFormation stack"
-  long_desc Help.delete
+  long_desc Lono::Help.text("cfn/delete")
   option :sure, type: :boolean, desc: "Skips are you sure prompt"
   def delete(name)
     Delete.new(name, options).run
   end
 
   desc "preview STACK", "preview a CloudFormation stack update"
-  long_desc Help.preview
+  long_desc Lono::Help.text("cfn/preview")
   option :keep, type: :boolean, desc: "keep the changeset instead of deleting it afterwards"
   option :diff, type: :boolean, default: true, desc: "Show diff of the source code template changes also."
   def preview(name)
@@ -60,13 +58,13 @@ class Lono::Cfn < Lono::Command
   end
 
   desc "diff STACK", "diff of newly generated template vs existing template in AWS"
-  long_desc Help.diff
+  long_desc Lono::Help.text("cfn/diff")
   def diff(name)
     Diff.new(name, options).run
   end
 
   desc "download STACK", "download CloudFormation template from existing stack"
-  long_desc Help.download
+  long_desc Lono::Help.text("cfn/download")
   option :name, desc: "Name you want to save the template as. Default: existing stack name."
   def download(stack_name)
     Download.new(stack_name, options).run

data/lib/lono/cfn/{aws_services.rb → aws_service.rb}

@@ -1,6 +1,6 @@
 require "aws-sdk-cloudformation"
 
-module Lono::Cfn::AwsServices
+module Lono::Cfn::AwsService
   def cfn
     @cfn ||= Aws::CloudFormation::Client.new
   end

data/lib/lono/cfn/base.rb

@@ -1,7 +1,7 @@
 require "lono"
 
 class Lono::Cfn::Base
-  include Lono::Cfn::AwsServices
+  include Lono::Cfn::AwsService
   include Lono::Cfn::Util
 
   attr_reader :randomize_stack_name
@@ -42,7 +42,7 @@ class Lono::Cfn::Base
     puts "  #{command_with_iam(capabilities)}"
 
     puts "Please confirm (y/n)"
-    confirm = $stdin.gets
+    $stdin.gets
   end
 
   def command_with_iam(capabilities)
@@ -51,24 +51,35 @@ class Lono::Cfn::Base
 
   def generate_all
     if @options[:lono]
+      build_scripts
       generate_templates
-      upload_templates if @options[:s3_upload] and !@options[:noop]
+      unless @options[:noop]
+        upload_scripts
+        upload_templates
+      end
     end
     params = generate_params(mute: @options[:mute_params])
     check_for_errors
     params
   end
 
+  def build_scripts
+    Lono::Script::Build.new.run
+  end
+
   def generate_templates
-    Lono::Template::DSL.new(pretty: true).run
+    Lono::Template::DSL.new.run
   end
 
+  # only upload templates if s3_folder configured in settings
   def upload_templates
-    # only upload templates if s3.path configured in settings
-    settings = Lono::Settings.new
-    return unless settings.s3_path
+    Lono::Template::Upload.new.run if s3_folder
+  end
 
-    Lono::Template::Upload.new(pretty: true).run
+  # only upload templates if s3_folder configured in settings
+  def upload_scripts
+    return unless s3_folder
+    Lono::Script::Upload.new.run
   end
 
   def generate_params(options={})
@@ -97,13 +108,13 @@ class Lono::Cfn::Base
   def check_files
     errors, warns = [], []
     unless File.exist?(@template_path)
-      warns << "Template file missing: could not find #{@template_path}"
+      errors << "Template file missing: could not find #{@template_path}"
     end
     # Examples:
     #   @param_path = params/prod/ecs.txt
     #              => output/params/prod/ecs.json
     output_param_path = @param_path.sub(/\.txt/, '.json')
-    output_param_path = "#{Lono.root}/output/#{output_param_path}"
+    output_param_path = "#{Lono.config.output_path}/#{output_param_path}"
     if @options[:param] && !File.exist?(output_param_path)
       warns << "Parameters file missing: could not find #{output_param_path}"
     end
@@ -117,7 +128,7 @@ class Lono::Cfn::Base
   # Type - :param or :template
   def get_source_path(path, type)
     if path.nil?
-      default_convention_path = convention_path(@stack_name, type)
+      convention_path(@stack_name, type) # default convention
     else
       # convention path based on the input from the user
       convention_path(path, type)
@@ -127,36 +138,15 @@ class Lono::Cfn::Base
   def convention_path(name, type)
     path = case type
     when :template
-      format = detect_format
-      "#{Lono.root}/output/#{name}.#{format}"
+      "#{Lono.config.output_path}/templates/#{name}.yml"
     when :param
-      "#{Lono.root}/params/#{Lono.env}/#{name}.txt"
+      "#{Lono.config.params_path}/#{Lono.env}/#{name}.txt"
     else
       raise "hell: dont come here"
     end
     path.sub(/^\.\//, '')
   end
 
-  # Returns String with value of "yml" or "json".
-  def detect_format
-    formats = Dir.glob("#{Lono.root}/templates/**/*").
-                map { |path| path.sub(/\.erb$/, '') }.
-                map { |path| File.extname(path) }.
-                reject { |s| s.empty? }. # reject ""
-                select { |s| s.include?("yml") || s.include?("json") }.
-                uniq
-    if formats.size > 1
-      puts "ERROR: Detected multiple formats: #{formats.join(", ")}".colorize(:red)
-      puts "All the output files must use the same format.  Either all json or all yml."
-      exit 1
-    elsif formats.size == 1
-      formats.first.sub(/^\./,'')
-    else
-      puts "WARN: Did not detect any template formats. Defaulting to yml.  Please check the config and templates folders.".colorize(:yellow)
-      "yml" # default format
-    end
-  end
-
   # All CloudFormation states listed here:
   # http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-describing-stacks.html
   def stack_status(stack_name)
@@ -164,15 +154,15 @@ class Lono::Cfn::Base
     return false if @options[:noop]
 
     resp = cfn.describe_stacks(stack_name: stack_name)
-    status = resp.stacks[0].stack_status
+    resp.stacks[0].stack_status
   end
 
-  def exist_unless_updatable(status)
+  def exit_unless_updatable!(status)
     return true if testing_update?
     return false if @options[:noop]
 
     unless status =~ /_COMPLETE$/
-      puts "Cannot create a change set for the stack because the #{@stack_name} is not in an updatable state.  Stack status: #{status}"
+      puts "Cannot create a change set for the stack because the #{@stack_name} is not in an updatable state.  Stack status: #{status}".colorize(:red)
       quit(1)
     end
   end
@@ -202,7 +192,7 @@ class Lono::Cfn::Base
     end
 
     # otherwise use the settings preference
-    settings = Lono::Settings.new
+    settings = Lono::Setting.new
     settings.data['randomize_stack_name']
   end
 
@@ -212,4 +202,17 @@ class Lono::Cfn::Base
       ["CAPABILITY_IAM", "CAPABILITY_NAMED_IAM"]
     end
   end
+
+  def show_parameters(params, meth=nil)
+    params = params.clone.compact
+    params[:template_body] = "Hidden due to size... View at: #{@template_path}"
+    to = meth || "AWS API"
+    puts "Parameters passed to #{to}:"
+    puts YAML.dump(params.deep_stringify_keys)
+  end
+
+  def s3_folder
+    setting = Lono::Setting.new
+    setting.s3_folder
+  end
 end