lono 8.0.0.pre.rc3 → 8.0.0.pre.rc4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 99ed66c36726e513ebe394291384f039d6dbbc7beef5a376b28b3b84121f1a42
-  data.tar.gz: ec1902278f840c29539d54fffa9167565a27f1837a4ee2795d8db86badf0030d
+  metadata.gz: 963246c282376afe61a61f60ad1ec0ccb17ee194420d972802868f8574fbc97a
+  data.tar.gz: 3e8b4c478d5ee57744ff05ad45023592dffab08825a5af2e044a8656680b0512
 SHA512:
-  metadata.gz: 522056fb098c360c2ad59d99dfabaa723aeec157ca5012938c36efcb130e1bf6c2974d158e3c2f864e258c5d4b065d8e949667291f6af7d779bad91d2b451a0f
-  data.tar.gz: 18d17bbf1fcdd0f9d7c59f87b83e463f277958fb34cfb07d5918904b76ae44ecd2995f7e36f09d0a99c15d01070b469abf44bcfc95ec8f413aa54fdb0058fc6c
+  metadata.gz: f8ea0aa2e31ea1dad60b69d7993ef1b6eb9781c1ee0d04770ed2a1eb0fec9387703e79b95faf1a04918685d147ff884a7f712420891dbb50f9091bdf3439fa9b
+  data.tar.gz: f73dd80095b287c9be478a68c4de4181fdd0657153d69dc0c72145497555a7c542096e8bd0fb5b85877589251d878cd2dee7cace277614917dcab383991de30f

data/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,84 @@
+---
+name: Reproducible Bug Report
+about: Is something not working as expected?
+title: ''
+labels: 'bug'
+assignees: ''
+
+---
+
+<!--
+Hi! Thanks for considering to file a bug with Lono. Please take the time to
+answer the basic questions. Please try to be as detailed as possible.
+To be sensitive to everyone's time, if not enough details are provided, the
+issue may be closed without comment. If you repeatedly fail to provide enough
+details, you may be blocked from ever submitting issues to Lono again.
+Please use your best judgment. 👍
+
+If you are unsure this is a bug in Lono, please consider asking your question at:
+https://community.boltops.com
+
+Thanks!
+-->
+
+## Checklist
+
+<!--
+Make sure that you've done all of these.  To mark a checkbox done, replace [ ] with [x]. Or after you create the issue you can click the checkbox.
+-->
+
+- [ ] Upgrade Lono: Are you using the latest version of Lono? This allows Lono to fix issues fast. There's an Upgrading Guide: https://lono.cloud/docs/upgrading/
+- [ ] Reproducibility: Are you reporting a bug others will be able to reproduce and not asking a question. If you're unsure or want to ask a question, do so on https://community.boltops.com
+- [ ] Code sample: Have you put together a code sample to reproduce the issue and make it available?  Code samples help speed up fixes dramatically. If it's an easily reproducible issue, then code samples are not needed. If you're unsure, please include a code sample.
+
+## My Environment
+
+<!-- Please fill out the table below with debugging info to help: -->
+
+| Software         | Version |
+| ---------------- | ------- |
+| Operating System |         |
+| Lono             |         |
+| Ruby             |         |
+
+---
+
+## Expected Behaviour
+
+<!--
+What is it you expected to happen? This should be a description of how the functionality you tried to use is supposed to work. Try to keep this to one-paragraph.
+-->
+
+## Current Behavior
+
+<!--
+Describe the details of the bug.  Try to keep this to one-paragraph.
+-->
+
+## Step-by-step reproduction instructions
+
+<!--
+Be sure to include any steps you took for the problem to exist. This is likely the longest part of the report.
+
+Please include any logs you think relevant here. If the logs are long (more than 50 lines) please make a gist of the logs and link to it. https://gist.github.com
+
+With long logs, you can also use the <details> tag to keep the report readable. Example:
+
+<details>
+ <summary>Summary Goes Here</summary>
+
+ ...this is hidden, collapsable content. start with a blank line to get terminal output to format right...
+</details>
+-->
+
+## Code Sample
+
+<!--
+Please provide a code repository, gist, code snippet or sample files to reproduce the issue.
+-->
+
+## Solution Suggestion
+
+<!--
+Please provide possible solutions. If you can't think of anything, feel free to omit. Please be kind and add helpful possible solutions.  For example, "Fix it!" is not a helpful solution suggestion. We are mere mortals. Please be constructive.
+-->

data/.github/ISSUE_TEMPLATE/documentation.md

@@ -0,0 +1,12 @@
+---
+name: Documentation
+about: Found a typo or something that isn't crystal clear in the docs?
+title: ''
+labels: docs
+assignees: ''
+
+---
+
+The Lono Docs are in the [lono-docs repo](https://github.com/boltops-tools/lono-docs). Please submit a PR there. Thanks!
+
+For documentation changes to the lono code base itself, like code comments. Please submit a PR here. Thanks!

data/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,64 @@
+---
+name: New Feature Suggestion
+about: Want to add a feature to Lono?
+title: ''
+labels: feature
+assignees: ''
+
+---
+
+<!--
+Hi! Thanks for considering to file a feature request with Lono. Please take the time to answer the basic questions. Please try to be as detailed as possible.
+
+Thanks!
+-->
+
+## Summary
+
+<!--
+A one-paragraph explanation of the feature.
+-->
+
+## Motivation
+
+<!--
+Why do you want to see this feature in Lono? What use cases does it support?
+
+How the feature would be relevant to 80% or more of Lono users.
+-->
+
+## Guide-level explanation
+
+<!--
+Explain the proposal as if it was already included in the project and you were teaching it to another programmer. That generally means:
+
+- Introducing new named concepts.
+- Explaining the feature largely in terms of examples.
+- If applicable, provide sample error messages, deprecation warnings, or upgrade guidance.
+
+If this is a small feature, you may omit this section.
+-->
+
+## Reference-level explanation
+
+<!--
+This is the technical portion of the feature request. Explain the design in sufficient detail that:
+
+- Its interaction with other features is clear.
+- It is reasonably clear how the feature would be implemented.
+- Corner cases are dissected by example.
+
+If you do not know how to answer this, you can omit it. No worries!
+-->
+
+## Drawbacks
+
+<!--
+Why should we *not* do this?
+-->
+
+## Unresolved Questions
+
+<!--
+What related issues do you consider out of scope for this feature that could be addressed in the future independently of the solution that comes out of this feature?
+-->

data/.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,14 @@
+---
+name: Question
+about: Have any questions about how Lono works?
+title: ''
+labels: 'question'
+assignees: ''
+
+---
+
+The Lono issue tracker IS NOT for usage questions! Please post your question on our dedicated forum at https://community.boltops.com
+
+To be sensitive to everyone's time, we may close issues asking questions without comment. If you repeatedly post questions in the issues tracker, you may be blocked from ever submitting issues to Lono again. Please use your best judgment. 👍
+
+Posting your questions in the Lono community forum benefits others by grouping questions in a dedicated place. Here are some additional options also https://lono.cloud/support/ 😁

data/.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,7 @@
+Please fill out one of the templates on https://github.com/boltops-tools/lono/issues/new/choose
+
+If you want to ask a question please do so in the Lono category in the BoltOps Community forum: https://community.boltops.com
+
+To be sensitive to everyone's time, we may close issues asking questions without comment. Posting your questions in the Lono community forum is the best place. It also benefits others by making the questions easier to find. Here are some additional options also https://lono.cloud/support/ 👌
+
+Thank you!

data/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,50 @@
+<!--
+Thanks for creating a Pull Request! Before you submit, please make sure you've done the following:
+
+- I read the contributing document at https://lono.cloud/docs/contributing/
+-->
+
+<!--
+Make our lives easier! Choose one of the following by uncommenting it:
+-->
+
+<!-- This is a 🐞 bug fix. -->
+<!-- This is a 🙋‍♂️ feature or enhancement. -->
+<!-- This is a 🧐 documentation change. -->
+
+<!--
+Before you submit this pull request, make sure to have a look at the following checklist. To mark a checkbox done, replace [ ] with [x]. Or after you create the issue you can click the checkbox.
+
+If you don't know how to do some of these, that's fine!  Submit your pull request and we will help you out on the way.
+-->
+
+- [ ] I've added tests (if it's a bug, feature or enhancement)
+- [ ] I've adjusted the documentation (if it's a feature or enhancement)
+- [ ] The test suite passes (run `bundle exec rspec` to verify this)
+
+## Summary
+
+<!--
+Provide a description of what your pull request changes.
+-->
+
+## Context
+
+<!--
+Is this related to any GitHub issue(s) or another relevant link?
+-->
+
+## How to Test
+
+<!--
+Please provide instructions on how to test the fix. This speeds up reviewing the PR. If testing requires a demo Lono project, please provide an example repo.
+-->
+
+
+## Version Changes
+
+<!--
+Which semantic version change would you recommend?
+If you don't know, feel free to omit it.
+-->
+

data/CHANGELOG.md

@@ -1,5 +1,38 @@
 # Change Log
 
+## [8.0.0.rc4] - 2022-03-19
+
+Notable:
+
+* lono app and role concept
+* extra layering support
+* hooks support: project and blueprint level
+
+Misc:
+
+* add lono bucket output
+* alias blueprint with -b
+* change params extension to .env
+* config.hooks.show = true default
+* config.layering.show = true option
+* extra layering support
+* github templates
+* improve simplier new helper cli interface
+* improve squeezer: edge case Array with nil elements
+* hooks generator
+* lambda layer
+* layering full mode default
+* lazy create lono stack and lono bucket
+* lono extra support
+* lono iam
+* lono summary
+* lono up: edge case fix continue after rollback
+* parse_for_layering_show
+* remove extensions and add new hook generator
+* show configset layering
+* update dsl_evaluator dependency to at least 0.3.0
+* version structure check
+
 ## [8.0.0.rc3] - 2022-03-06
 
 * lono files support for lambda functions

data/lib/lono/app.rb

@@ -20,8 +20,13 @@ module Lono
       config.deny.envs = nil
       config.deny.regions = nil
 
+      config.hooks = ActiveSupport::OrderedOptions.new
+      config.hooks.show = true
+
       config.layering = ActiveSupport::OrderedOptions.new
       config.layering.names = {}
+      config.layering.mode = "full" # simple or fulll. simple does not include region, account, and region/account layers
+      config.layering.show = show_layers?
 
       config.log = ActiveSupport::OrderedOptions.new
       config.log.root = Lono.log_root
@@ -31,9 +36,7 @@ module Lono
 
       config.names = ActiveSupport::OrderedOptions.new
       config.names.output = ActiveSupport::OrderedOptions.new
-      config.names.output.expand = true
-      config.names.output.stack = ":BLUEPRINT-:ENV" # does not include APP by default. Think this is more common
-      config.names.stack = ":APP-:BLUEPRINT-:ENV"
+      config.names.stack = ":APP-:ROLE-:BLUEPRINT-:ENV-:EXTRA"
 
       config.plan = ActiveSupport::OrderedOptions.new
       config.plan.changeset = true
@@ -55,6 +58,37 @@ module Lono
       config
     end
 
+    def show_layers?
+      ENV['LONO_SHOW_LAYERS'] || parse_for_layering_show
+    end
+    private :show_layers?
+
+    # Some limitations:
+    #
+    # * Only parsing one file: .ufo/config.rb
+    # * If user is using Ruby code that cannot be parse will fallback to default
+    #
+    # Think it's worth it so user only has to configure
+    #
+    #     config.layering.show = true
+    #
+    def parse_for_layering_show
+      return false if Lono::Command.non_project_command?
+      lines = IO.readlines("#{Lono.root}/config/app.rb")
+      config_line = lines.find { |l| l =~ /config\.layering.show.*=/ && l !~ /^\s+#/ }
+      return false unless config_line # default is false
+      config_value = config_line.gsub(/.*=/,'').strip.gsub(/["']/,'')
+      config_value != "false" && config_value != "nil"
+    rescue Exception => e
+      if ENV['LONO_DEBUG']
+        puts "#{e.class} #{e.message}".color(:yellow)
+        puts "WARN: Unable to parse for config.layering.show".color(:yellow)
+        puts "Using default: config.layering.show = false"
+      end
+      false
+    end
+    memoize :parse_for_layering_show
+
     def lono_logger
       Logger.new(ENV['LONO_LOG_PATH'] || $stderr)
     end

data/lib/lono/builder/allow.rb

@@ -1,4 +1,4 @@
-module Lono::Builder
+class Lono::Builder
   class Allow < Lono::CLI::Base
     def check!
       Env.new(@options).check!

data/lib/lono/builder/configset/definition/context.rb

@@ -8,18 +8,59 @@ class Lono::Builder::Configset::Definition
     end
 
     # Docs: https://lono.cloud/docs/configsets/layering/
-    # Layering is simple enough to keep in this method.
+    # Think layering is simple enough to keep in this method for now.
+    # Consider trying to combine with Lono::Layering::Layer for uniformity though.
     def load_vars
-      logger.debug "Layers for configset #{@configset.name}:"
-      evaluate_layer("#{@configset.root}/vars.rb") # source defaults
-      evaluate_layer("#{Lono.root}/config/configsets/#{@configset.name}/vars.rb") # source overrides
-      evaluate_layer("#{@blueprint.root}/config/configsets/vars/#{@configset.name}.rb") # blueprint overrides
-      evaluate_layer("#{Lono.root}/config/blueprints/#{@blueprint.name}/configsets/vars/#{@configset.name}.rb") # user overrides
+      layers.each do |layer|
+        evaluate_file(layer)
+      end
     end
 
-    def evaluate_layer(path)
-      logger.debug "    #{pretty_path(path)}" if File.exist?(path) || ENV['LONO_SHOW_ALL_LAYERS']
-      evaluate_file(path)
+    def layers
+      paths = [
+        "#{@configset.root}/vars.rb", # source defaults
+        "#{Lono.root}/config/configsets/#{@configset.name}/vars.rb", # source overrides
+        "#{@blueprint.root}/config/configsets/vars/#{@configset.name}.rb", # blueprint overrides
+        "#{Lono.root}/config/blueprints/#{@blueprint.name}/configsets/vars/#{@configset.name}.rb", # user overrides
+      ]
+      show_layers(paths)
+      paths
+    end
+
+    def show_layers(paths)
+      if ENV['LONO_LAYERS_ALL']
+        show_all_layers(paths)
+      else
+        show_existing_layers(paths)
+      end
+    end
+
+    def show_all_layers(paths)
+      show_header
+      paths.each do |path|
+        show_layer(path)
+      end
+    end
+
+    def show_existing_layers(paths)
+      existing = paths.select { |path| File.exist?(path) }
+      return if existing.empty?
+      show_header
+      existing.each do |path|
+        show_layer(path)
+      end
+    end
+
+    def show_header
+      logger.info "    Configset Layers #{@configset.name}"
+    end
+
+    def show_layer(path)
+      logger.info "        #{pretty_path(path)}"
+    end
+
+    def show_layers?
+      Lono.config.layering.show || ENV['LONO_LAYERS']
     end
 
     def load_helpers

data/lib/lono/builder/configset/definition/dsl/syntax/content.rb

@@ -1,7 +1,7 @@
 module Lono::Builder::Configset::Definition::Dsl::Syntax
   module Content
     def content_file(path)
-      content_path = "#{file_root}/content"
+      content_path = "#{content_file_root}/content"
       file = "#{content_path}/#{path}"
       if File.exist?(file)
         IO.read(file)
@@ -10,17 +10,13 @@ module Lono::Builder::Configset::Definition::Dsl::Syntax
       end
     end
 
-    def file_root
+  private
+    def content_file_root
       if @configset
         @configset.root
       else
         @blueprint.root
       end
     end
-
-    def s3_key(name)
-      Lono::Configset::S3File::Registry.register(name, blueprint: @blueprint, configset: @configset, root: @root)
-      "file://configset/#{@configset}/#{name}"
-    end
   end
 end

data/lib/lono/builder/context.rb

@@ -1,4 +1,4 @@
-module Lono::Builder
+class Lono::Builder
   module Context
     include DslEvaluator
 

data/lib/lono/builder/dsl/finalizer/files/build.rb

@@ -11,27 +11,19 @@ class Lono::Builder::Dsl::Finalizer::Files
       clean
       validate!
       build_files
-      # build_layers
     end
 
     def build_files
       Lono::Files.files.each do |file| # using singular file, but is like a "file_list"
         file.build
+        file.build_lambda_layer(@cfn)
         file.compress
-        # Note: Uploading files happen right before create_stack or execute_change_set
-        # after user confirms action, instead of part of the build process
+        # UploadNote: Instead of part of the build process here, uploading happen as part
+        # of create_stack or execute_change_set after user confirms action. IE:
+        #   Cfn::Deploy#create and Cfn::Deploy#update
       end
     end
 
-    # TODO: LambdaLayer support
-    # def build_layers
-    #   layer_items = Registry.layers
-    #   layer_items.each do |item|
-    #     LambdaLayer.new(@blueprint, item).build
-    #   end
-    # end
-    #
-
     def clean
       FileUtils.rm_rf(@output_path)
     end

data/lib/lono/builder/dsl/finalizer/files/replace.rb

@@ -8,9 +8,14 @@ class Lono::Builder::Dsl::Finalizer::Files
     end
 
     def replacements
-      Lono::Files.files.inject({}) do |result, file|
+      data = Lono::Files.files.inject({}) do |result, file|
         result.merge(file.marker => file.s3_key)
       end
+      # Edge case when bucket is created for the first time and files_bucket is
+      # not available yet. So we call it at this point.
+      Lono::S3::Bucket.ensure_exist
+      data["LONO://S3_BUCKET"] = Lono::S3::Bucket.name
+      data
     end
 
     def update_template!(hash)

data/lib/lono/builder/dsl/helpers/files.rb

@@ -1,7 +1,7 @@
 module Lono::Builder::Dsl::Helpers
   module Files
-    def files(path)
-      Lono::Files.register(@options.merge(path: path))
+    def files(path, o={})
+      Lono::Files.register(@options.merge(path: path).merge(o))
     end
   end
 end

data/lib/lono/builder/dsl/helpers/s3.rb

@@ -3,14 +3,12 @@ module Lono::Builder::Dsl::Helpers
     def s3_bucket
       Lono::S3::Bucket.name
     end
-    alias_method :lono_bucket_name, :s3_bucket
-    alias_method :files_bucket, :s3_bucket
+    alias_method :lono_bucket, :s3_bucket
 
-    def s3_key(name, options={})
-      default = {type: "file"}
-      options.reverse_merge!(default)
-      "file://app/files/#{options[:type]}/#{name}" # placeholder for post processing
+    # Edge case when bucket is created for the first time and files_bucket is
+    # not available yet to be use by Finalizer::Files::Replace#replacements
+    def files_bucket
+      "LONO://S3_BUCKET"
     end
-    alias_method :file_s3_key, :s3_key
   end
 end

data/lib/lono/builder/dsl/helpers/stack.rb

@@ -15,7 +15,7 @@ module Lono::Builder::Dsl::Helpers
       if o
         o.output_value
       else
-        logger.info "WARN2: NOT FOUND: output #{key} for stack #{stack_name}"
+        logger.info "WARN: STACK OUTPUT NOT FOUND: output #{key} for stack #{stack_name}"
         nil
       end
     end
@@ -35,10 +35,8 @@ module Lono::Builder::Dsl::Helpers
     end
 
     def stack_name(blueprint)
-      return blueprint unless Lono.config.names.output.expand
       names = Lono::Names.new(blueprint: blueprint)
-      # explicit expansion pattern provided by user
-      pattern = blueprint.include?(':') ? blueprint : Lono.config.names.output.stack
+      pattern = blueprint # blueprint can be a pattern provided by user. IE: :BLUEPRINT-:ENV
       names.expansion(pattern)
     end
 

data/lib/lono/builder/dsl/helpers/template_file.rb

@@ -21,7 +21,7 @@ module Lono::Builder::Dsl::Helpers
     class TempleFileNotFoundError < StandardError; end
     def template_file_missing(path)
       message = "ERROR: path #{pretty_path(path)} not found"
-      caller_line = caller.find { |l| l =~ %r{/blueprints/} } # TODO: show code itself
+      caller_line = caller.find { |l| l =~ %r{/blueprints/} }
       logger.error message.color(:red)
       logger.error "Called from:"
       logger.error "    #{pretty_path(caller_line)}"
@@ -33,7 +33,7 @@ module Lono::Builder::Dsl::Helpers
       if File.exist?(path)
         RenderMePretty.result(path, context: self)
       else
-        lines = caller.select { |l| l.include?(Lono.root.to_s) } # TODO: show code itself
+        lines = caller.select { |l| l.include?(Lono.root.to_s) }
         caller_line = pretty_path(lines.first)
         message =<<~EOL
           WARN: #{pretty_path(path)} does not exist

data/lib/lono/builder/dsl/syntax/core/squeezer.rb

@@ -9,7 +9,9 @@ module Lono::Builder::Dsl::Syntax::Core
 
       case data
       when Array
-        data.map! { |v| squeeze(v) }
+        # .compact prevents infinite loop when data = [nil] on accident
+        # IE: data[:key] = [nil]
+        data.compact.map! { |v| squeeze(v) }
       when Hash
         data.each_with_object({}) do |(k,v), squeezed|
           # only remove nil and empty Array values within Hash structures

data/lib/lono/builder/dsl/syntax/fn.rb

@@ -102,8 +102,6 @@ module Lono::Builder::Dsl::Syntax
               else
                 item
               end
-      # list.map!(&:camelize) unless options[:autoformat] == false # TODO: maybe add as an option.
-      # feel this may be to destructive since am going with auto_camelize false for resources now.
       args = [list[0], list[1..-1].join('.')]
       { "Fn::GetAtt" => args }
     end

data/lib/lono/builder/dsl.rb

@@ -1,4 +1,4 @@
-module Lono::Builder
+class Lono::Builder
   class Dsl < Lono::CLI::Base
     attr_reader :results
     def run

data/lib/lono/builder/param.rb

@@ -1,4 +1,4 @@
-module Lono::Builder
+class Lono::Builder
   class Param < Lono::CLI::Base
     attr_reader :env_path, :base_path # set when build is called
     include Lono::Builder::Dsl::Syntax

data/lib/lono/builder/template.rb

@@ -1,6 +1,4 @@
-require "yaml"
-
-module Lono::Builder
+class Lono::Builder
   class Template < Lono::CLI::Base
     extend Memoist
 

data/lib/lono/builder.rb

@@ -0,0 +1,39 @@
+module Lono
+  class Builder < Lono::CLI::Base
+    include Lono::Hooks::Concern
+
+    def all
+      check_allow!
+
+      parameters = nil
+      run_hooks("build") do
+        clean
+        template_builder.run # build with placeholders IE: LONO://app/files/index.rb
+        parameters = param_builder.build  # Writes the json file in CamelCase keys format
+      end
+
+      logger.info "" # newline
+      parameters
+    end
+    memoize :all
+    alias_method :parameters, :all
+
+    def clean
+      Lono::CLI::Clean.new(@options.merge(mute: true)).run
+    end
+
+    def check_allow!
+      Lono::Builder::Allow.new(@options).check!
+    end
+
+    def param_builder
+      Lono::Builder::Param.new(@options)
+    end
+    memoize :param_builder
+
+    def template_builder
+      Lono::Builder::Template.new(@options) # write templates to disk
+    end
+    memoize :template_builder
+  end
+end