jets 3.2.1 → 3.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8fedba281ee9d6ac31a44184b49bbc8fe15d309a3f4d58142cfa25c4ddebc153
-  data.tar.gz: e73ee950202ea1786c881b43abc95c03bc4c3e7ef44b04265997eca2f4b9211b
+  metadata.gz: 12689dd61b554f0b5a049ee578cf14566593788a3a2ebb7353ea26a5f7bd803c
+  data.tar.gz: c54ebb36356ac71a8e9d211eb9187e761eba9aca2c650abd50e55ffdbf14f6d3
 SHA512:
-  metadata.gz: 57070368542b8a767fbf8553f0c627b36432e691ae07f1f41488a0d8af830bffd1ed84a71633fe971df6b45bcf79ff8862f2d4513d22e4a1981b633f36cbb8bd
-  data.tar.gz: 32ba83eb4d4e87a8c92b5d7ec8d573ae2b36ff558fedbae7bb33f1ebe5dbcb36baf2f64b89b2a05d2e56e52bcd2145964a0770f7a56efdb48d1a925498c7f177
+  metadata.gz: 9834c8acaa032e002820e7ba7855f62f318160942287e2c1d23b31f4746d62cbb92253c97097bc94ca6d463044b95d7ef97f5030c6c8105682155ec6d568941d
+  data.tar.gz: 707f58aae8c4b8730db0a6e160830eea8a45fc840300a1f1e2a79ffb656118774446e24f1b91722e974bd0f5126a1b755c68eaea6147a8403d2aad76cfc4e697

data/CHANGELOG.md

@@ -3,6 +3,9 @@
 All notable changes to this project will be documented in this file.
 This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/).
 
+## [3.2.2] - 2023-05-19
+- [#640](https://github.com/boltops-tools/jets/pull/640) Base path mapping CloudFormation custom resource and lambda function hardening
+
 ## [3.2.1] - 2023-05-03
 - [#637](https://github.com/boltops-tools/jets/pull/637) Readme: Replace "splat out" with "had" a baby
 - [#638](https://github.com/boltops-tools/jets/pull/638) feature: update an array of reserved variables in aws lambda

data/lib/jets/internal/app/functions/jets/base_path.rb

@@ -1,13 +1,49 @@
-require 'bundler/setup'
+begin
+  require 'bundler/setup'
+  # When require bundler/setup fails, AWS Lambda won't be able to load base_path_mapping
+  # So we'll require base_path_mapping within begin/rescue block so that it does not also
+  # fail the entire lambda function.
+  require 'jets/internal/app/functions/jets/base_path_mapping'
+rescue Exception => e
+  # Note: rescue LoadError is not enough in AWS Lambda environment
+  # Actual exceptions:
+  #   require bundler/setup: Ruby exception "Bundler::GemNotFound"
+  #   require base_path_mappnig: AWS Lambda reported error "errorType": "Init<LoadError>"
+  # Will use a generic rescue Exception though in case error changes in the future.
+  puts "WARN: #{e.class} #{e.message}"
+  puts <<~EOL
+    Could not require bundler/setup.
+    This can happen for weird timeout missing error. Example:
+
+        Could not find timeout-0.3.2 in locally installed gems
+
+    Happens when the gem command is out-of-date on old versions of ruby 2.7.
+    See: https://community.boltops.com/t/could-not-find-timeout-0-3-1-in-any-of-the-sources/996
+  EOL
+end
 require 'cfn_response'
-require 'jets/internal/app/functions/jets/base_path_mapping'
 
 STAGE_NAME = "<%= @stage_name %>"
 
 def lambda_handler(event:, context:)
   cfn = CfnResponse.new(event, context)
   cfn.response do
-    mapping = BasePathMapping.new(event, STAGE_NAME)
+    # Super edge case: mapping is nil when require bundler/setup fails
+    begin
+      mapping = BasePathMapping.new(event, STAGE_NAME)
+    rescue NameError => e
+      puts "ERROR: #{e.class} #{e.message}"
+      puts error_message
+    end
+
+    # This is the "second pass" of CloudFormation when it tries to delete the BasePathMapping during rollback
+    if mapping.nil? && event['RequestType'] == "Delete"
+      cfn.success # so that CloudFormation can continue the delete process from a rollback
+      delay
+      return
+    end
+
+    # Normal behavior when mapping is not nil when bundler/setup loads successfully
     case event['RequestType']
     when "Create", "Update"
       mapping.update
@@ -16,3 +52,57 @@ def lambda_handler(event:, context:)
     end
   end
 end
+
+def delay
+  puts "Delaying 60 seconds to allow user some time to see lambda function logs."
+  60.times do
+    puts Time.now
+    sleep 1
+  end
+end
+
+def error_message
+  <<~EOL
+  This is ultimately the result of require bundler/setup failing to load.
+  On the CloudFormation first pass, the BasePathMapping fails to CREATE.
+  CloudFormation does a rollback and tries to delete the BasePathMapping.
+
+  Jets will send a success response to CloudFormation so it can continue and delete
+  BasePathMapping on the rollback. Otherwise, CloudFormation ends up in the terminal
+  UPDATE_FAILED state and the CloudFormation console provides 3 options:
+
+      1) retry 2) update 3) rollback.
+
+  The only option that seems to work is rollback to get it out of UPDATE_FAILED to
+  UPDATE_ROLLBACK_COMPLETE. But then, if we `jets deploy` again without fixing the
+  require bundler/setup issue, we'll end back up in the UPDATE_FAILED state.
+
+  Will handle this error so we can continue the stack because we do not want it to fail
+  and never be able to send the CfnResponse. Then we have to wait hours for a CloudFormation timeout.
+  Sometimes deleting the APP-dev-ApiDeployment20230518230443-EXAMPLEY8YQP0 stack
+  allows the cloudformation stacks to continue, but usually, it'll only just slightly speed up the rollback.
+
+  Some examples of actual rollback times:
+
+  When left alone, the rollback takes about 2.5 hours.
+
+      2023-05-19 05:39:25  User Initiated
+      2023-05-19 08:01:48  UPDATE_ROLLBACK_COMPLETE
+
+  When deleting the APP-dev-ApiDeployment20230518230443-EXAMPLEY8YQP0 stack, it takes about 1.5 hours.
+
+      2023-05-19 06:25:41  User Initiated
+      2023-05-19 07:47:03  UPDATE_ROLLBACK_COMPLETE
+
+  Rescuing and handling the error here allows the CloudFormation stack to continue and finish the rollback process.
+  It takes the rollback time down to about 3 minutes. Example:
+
+      2023-05-19 16:34:19 User Initiated
+      2023-05-19 16:37:34 UPDATE_ROLLBACK_COMPLETE
+
+  Note: The first cloudformation CREATE pass sends FAILED Status to CloudFormation,
+  and the second cloudformation DELETE pass sends SUCCESS Status to CloudFormation.
+
+  Related: https://community.boltops.com/t/could-not-find-timeout-0-3-1-in-any-of-the-sources/996
+  EOL
+end

data/lib/jets/internal/app/functions/jets/base_path_mapping.rb

@@ -28,8 +28,28 @@ class BasePathMapping
   # Cannot use update_base_path_mapping to update the base_mapping because it doesnt
   # allow us to change the rest_api_id. So we delete and create.
   def update
-    delete(true)
-    create
+    puts "BasePathMapping update"
+    if rest_api_changed?
+      delete(true)
+      create
+    else
+      puts "BasePathMapping update: rest_api_id #{rest_api_id} did not change. Skipping."
+    end
+
+    puts "BasePathMapping update complete"
+  end
+
+  def rest_api_changed?
+    puts "BasePathMapping checking if rest_api_id changed"
+    mapping = current_base_path_mapping
+    return true unless mapping
+    mapping.rest_api_id != rest_api_id
+  end
+
+  def current_base_path_mapping
+    resp = apigateway.get_base_path_mapping(base_path: "(none)", domain_name: domain_name)
+  rescue Aws::APIGateway::Errors::NotFoundException
+    return nil
   end
 
   # Dont delete the newly created base path mapping unless this is an operation
@@ -39,10 +59,14 @@ class BasePathMapping
   end
 
   def delete(fail_silently=false)
-    apigateway.delete_base_path_mapping(
+    puts "BasePathMapping delete"
+    options = {
       domain_name: domain_name, # required
       base_path: base_path.empty? ? '(none)' : base_path,
-    )
+    }
+    puts "BasePathMapping delete options #{options.inspect}"
+    apigateway.delete_base_path_mapping(options)
+    puts "BasePathMapping delete complete"
   # https://github.com/tongueroo/jets/issues/255
   # Used to return: Aws::APIGateway::Errors::NotFoundException
   # Now returns: Aws::APIGateway::Errors::InternalFailure
@@ -52,12 +76,41 @@ class BasePathMapping
   end
 
   def create
-    apigateway.create_base_path_mapping(
+    puts "BasePathMapping create"
+    options = {
       domain_name: domain_name, # required
       base_path: base_path,
       rest_api_id: rest_api_id, # required
       stage: @stage_name,
-    )
+    }
+    puts "BasePathMapping create options #{options.inspect}"
+    apigateway.create_base_path_mapping(options)
+    puts "BasePathMapping create complete"
+  rescue Aws::APIGateway::Errors::ServiceError => e
+    puts "ERROR: #{e.class}: #{e.message}"
+    puts "BasePathMapping create failed"
+    if e.message.include?("Invalid domain name identifier specified")
+      puts <<~EOL
+        This super edge case error seems to happen when the cloudformation stack does a rollback
+        because the BasePathMapping custom resource fails. This has happened with a strange combination of
+        ruby 2.7 and the timeout gem not being picked up in the AWS Lambda runtime environment
+        Specifically, when jets deploy was used with a rubygems install that is out-of-date.
+        See: https://community.boltops.com/t/could-not-find-timeout-0-3-1-in-any-of-the-sources/996
+
+        The new base path mapping is not created correctly and the old base path mapping is not properly deleted.
+        The old ghost base mapping interferes with the new base path mapping.
+        The workaround solution seems to require removing all the config.domain settings and deploying again. Example:
+
+        config/application.rb
+
+            config.domain.cert_arn = "arn:aws:acm:us-west-2:111111111111:certificate/EXAMPLE1-a3de-4fe7-b72e-4cc153c5303e"
+            config.domain.hosted_zone_name = "example.com"
+
+        Comment out those settings, deploy, then uncomment and deploy again.
+        If there's a better workaround, please let us know.
+      EOL
+    end
+    raise(e)
   end
 
   def deployment_stack
@@ -110,3 +163,9 @@ class BasePathMapping
     options
   end
 end
+
+if __FILE__ == $0
+  event = JSON.load(File.read(ARGV[0]))
+  context = nil # stub out
+  BasePathMapping.new(event, context).update
+end

data/lib/jets/version.rb

@@ -1,3 +1,3 @@
 module Jets
-  VERSION = "3.2.1"
+  VERSION = "3.2.2"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: jets
 version: !ruby/object:Gem::Version
-  version: 3.2.1
+  version: 3.2.2
 platform: ruby
 authors:
 - Tung Nguyen
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-05-03 00:00:00.000000000 Z
+date: 2023-05-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: actionmailer
@@ -1103,7 +1103,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.6
+rubygems_version: 3.4.13
 signing_key: 
 specification_version: 4
 summary: Ruby Serverless Framework