uc3-sam-sceptre 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: bf7d0e753bbcbfc977d046ad6beee0d8837459749b24c4518b4071e653c2b2d0
+  data.tar.gz: bf7717c153cc8414bab7e6a79fdf21759358e906525e69823d7a3cbace98f1b3
+SHA512:
+  metadata.gz: 11a60179364b3915ada6b464a6fa2fea14bbc1f59d6af4fb4e428b8432b4a0785eb2e739210e0703d59d49c277c4e3fd06a48cca7a2560083e40c14d37c799be
+  data.tar.gz: fbca8e612283e589377a04093427851b2e652403aeb92ce07be76e584ff02fad65ad237007826cbce6ba8cb490fadefca3015d4ca1befd7f87a641a14056a7a8

data/README.md

@@ -0,0 +1,247 @@
+# UC3 SAM Sceptre gem
+
+This gem provides functionality that can be used to help Sceptre trigger a SAM build and/or SAM deploy
+to create an image and push it to an ECR repository or build the ZIP archive and put it into an S3 bucket
+and then run the CloudFormation create/update/delete.
+
+## Prerequisites
+
+This gem assumes that you have organized your Sceptre config by environment! For example:
+```
+my_project
+   |
+    ------ config
+   |         |
+   |          ---- dev
+   |         |      |
+   |         |       ----- s3.yaml
+   |         |
+   |          ---- prd
+   |                |
+   |                 ----- s3.yaml
+   |
+    ------ templates
+              |
+               ------ s3.yaml
+```
+
+This gem assumes that you have access to run CloudFormation tasks in 2 AWS accounts: `dev` and `prd`
+
+If you specify `env: prd` or `env: stg` when initializing the Proxy, your resources will be constucted in the `prd` AWS account otherwise they will be created in the `dev` account.
+
+Since SAM is not directly connected to Sceptre and cannot therefore access any of Sceptre's resolvers, you will need to ensure access to any SAM template parameters defined in your template.
+
+This gem will retrieve values from either the SSM parameter store or exported CloudFormation stack outputs. The SSM parameters and stack outputs **MUST** live within the same region that you are running your SAM deploy!
+
+For example, given the following SAM template:
+```
+AWSTemplateFormatVersion: '2010-09-09'
+Transform: 'AWS::Serverless-2016-10-31'
+
+Parameters:
+  VpcId:
+    Type: 'AWS::EC2::VPC::Id'
+```
+
+You would need to make the `VpcId` available as an SSM parameter or as an exported output from one of your other Sceptre managed CloudFormation stacks like this:
+```
+Outputs:
+  VpcId:
+    Value: !Ref Vpc
+    Export:
+      Name: VpcId
+```
+
+You should place your SAM template and the Lambda code under the Sceptre project's `templates` directory. For example:
+```
+my_project
+    |
+     ------ config
+    |          |
+    |           ----- dev
+    |                   |
+    |                    ----- api-gateway.yaml                     # Your Sceptre config for the API Gateway that
+    |                                                               # includes a hook to run the SAM build + deploy
+    |
+     ------ templates
+               |
+                ------- api-gateway.yaml                            # Your SAM template that builds the API Gateway
+               |
+               |
+                ------- lambdas
+                         |
+                          ----- src
+                                 |
+                                  ------ my_lambda
+                                 |          |
+                                 |           ------ file(s)         # Your Lambda source
+                                 |
+                                  ------- Gemfile                   # Gemfile that includes this gem
+                                 |
+                                  ------- sam_build_deploy.rb       # The Ruby script that initializes and uses this gem
+                                 |
+                                  ------- template.yaml             # Your SAM template for the Lambda(s)
+```
+
+## Usage
+
+This gem is intended to be invoked from a Ruby script that you create and store in the same directory as your SAM template. The script can be run manually at any time and via a Sceptre hook.
+
+Since CloudFormation requires your Lambda code to exist in an S3 bucket or an ECR when you create your stack, we advise that you at least add this script as a hook to an appropriate Sceptre config. For example as an `after_create` hook on the Sceptre config for the S3 bucket or ECR that will be used to store your Lambda code.
+
+### Build a Ruby script
+Create a Gemfile that is adjacent to your SAM template (see the location of the `Gemfile` file in the directory structure outlined above). This file should look like this:
+```
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
+
+gem 'uc3-sam-sceptre'
+```
+
+Create a Ruby script that is adjacent to your SAM template (see the location of the `sam_build_deploy.rb` file in the directory structure outlined above). (See the exmaples below)
+
+The following arguments can be used in your Ruby script when initializing the SAM proxy:
+```
+# REQUIRED arguments:
+# ------------------------
+service: 'dmp'                                # REQUIRED - The service name
+subservice: 'hub'                             # REQUIRED - The subservice name
+git_repo: 'https://github.com/MYORG/my-repo'  # REQUIRED - The location of your SAM code
+stack_suffix: 'docker-lambdas'                # REQUIRED - Used to generate the CloudFormation stack name
+admin_email_key: 'AdminEmail'                 # REQUIRED - An SSM parameter name or a CF stack export name
+
+# Include one of the following:
+ecr_uri_key: 'MyEcrRepoUri                    # REQUIRED if Lambda is a Docker Image - An SSM parameter name or a CF stack export name
+s3_arn_key: 'S3PrivateBucketArn'              # REQUIRED if Lambda is NOT a Docker Image - An SSM parameter name or a CF stack export name
+
+# An entry for each of your SAM template parameters.
+#   - Fetchable will be looked up in either the SSM paramter store or available CloudFormation stack exports
+#   - Static will be used as-is
+#
+fetchable_cf_params: [
+  {
+    stack_export_name: 'CloudFrontDistroId'   # REQUIRED - An SSM parameter name or a CF stack export name
+    template_param_name: 'CFDistributionId'   # (defaults to :stack_export_name) The name of the parameter your SAM template is looking for
+  }
+],
+static_cf_params: [
+  {
+    template_param_name: 'Version'          # REQUIRED - The name of the parameter your SAM template is looking for
+    value: 'v0'                             # REQUIRED - The value of the parameter
+  }
+]
+
+# OPTIONAL arguments:
+# ------------------------
+program: 'uc3'                                # The group that owns the service (default: 'uc3')
+env: 'stg'                                    # The environment (default: 'dev')
+region: 'us-east-1'                           # The AWS region (default: 'us-west-2')
+auto: false                                   # Whether or not to run the SAM deploy in `--guided` mode (default: true)
+```
+
+The `:program, :service, :subservice, and :env` are used to:
+- Create the SAM CloudFormation stack name (along with the `:stack_suffix`) For example: `uc3-dmp-hub-dev-foo-bar`
+- Help find the values of parameters in the `:fetchable_cf_params` by limiting the stacks to those that share your Sceptre project's naming conventions. For example: `uc3-dmp-hub-dev`
+- Create resource tags along with the `:git_repo` and the value found for the `:admin_email_key` to tag all resources created by SAM with tags for `Program, Service, Subservice, Environment, CodeRepo and Contact`
+
+The entries within the `:fetchable_cf_params` and `:static_cf_params` will all be passed to the SAM deploy as `--parameter-overrides`
+
+This gem will automatically append the `:program + :service + :subservice + :env` to the names of the SSM parameter keys you provide (unless you provided the fully qulified key name). So for example if you specify `/uc3/dmp/hub/dev/AdminEmail`, it will be used as-is; if you provide `AdminEmail` then the gem will prepend `/uc3/dmp/hub/dev` to the name.
+
+#### Build the bundle
+
+You will need to run `bundle install` so that this gem is installed and ready
+
+#### Add the Sceptre hook
+
+Recommended but optional.
+
+Add a hook to an existing Sceptre config that will run this script. Since CloudFormation requires your Lambda code to already exist in an S3 bucket or an ECR when you are creating your SAM stacks. For example here is a hook that will run a SAM build and deploy immediately after an S3 bucket is created:
+
+```
+# config/dev/s3.yaml
+template:
+  path: s3.yaml
+  type: file
+
+parameters:
+  LogBucketObjectLifeSpan: '30'
+
+hooks:
+  after_create:
+    # Build the Lambda code and store in the bucket so it is available downstream
+    - !cmd './templates/lambdas/src/sam_build_deploy.rb'
+```
+
+### Examples Ruby scripts:
+The following is an example of deploying to a Lambda that is a Docker image into an ECR.
+```
+# frozen_string_literal: true
+
+require 'uc3-sam-sceptre'
+
+params = {
+  service: 'dmp',
+  subservice: 'hub',
+  git_repo: 'https://github.com/CDLUC3/dmp-hub-cfn',
+  stack_suffix: 'docker-lambdas',
+  admin_email_key: 'AdminEmail',
+
+  ecr_uri_key: 'EcrRepositoryUri',
+
+  fetchable_cf_params: [
+    { stack_export_name: 'VpcId' },
+    { stack_export_name: 'SubnetA' },
+    { stack_export_name: 'SubnetB' },
+    { stack_export_name: 'SubnetC' },
+    { stack_export_name: 'DeadLetterQueueArn' },
+    { stack_export_name: 'LambdaSecGroupId' },
+    { stack_export_name: 'SnsTopicEmailArn', template_param_name: 'SnsEmailTopicArn' }
+    { stack_export_name: 'DomainName' },
+    { stack_export_name: 'RdsHost' },
+    { stack_export_name: 'RdsPort' },
+    { stack_export_name: 'RdsDbName' }
+  ],
+  static_cf_params: [
+    { template_param_name: 'Version', value: 'v0' }
+  ]
+}
+
+proxy = Uc3SamSceptre::Proxy.new(params)
+proxy.build
+proxy.deploy
+```
+
+The following is an example of deploying a Lambda that is NOT a Docker image into a ZIP archive stored in S3
+```
+# frozen_string_literal: true
+
+require 'uc3-sam-sceptre'
+
+params = {
+  service: 'dmp',
+  subservice: 'hub',
+  git_repo: 'https://github.com/CDLUC3/dmp-hub-cfn',
+  ecr_uri_key: 'EcrRepositoryUri',
+  stack_suffix: 'docker-lambdas',
+  admin_email_key: 'AdminEmail',
+
+  s3_arn_key: 'S3PrivateBucketArn',
+
+  fetchable_cf_params: [
+    { stack_export_name: 'CloudFrontDistroId' },
+    { stack_export_name: 'S3PublicBucketArn' },
+    { stack_export_name: 'DynamoTableName' },
+    { stack_export_name: 'SnsTopicEmailArn', template_param_name: 'SnsEmailTopicArn' }
+    { stack_export_name: 'DomainName' }
+  ],
+  static_cf_params: [
+    { template_param_name: 'Version', value: 'v0' }
+  ]
+}
+
+proxy = Uc3SamSceptre::Proxy.new(params)
+proxy.build
+proxy.deploy
+```

data/lib/uc3-sam-sceptre/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module Uc3SamSceptre
+  VERSION = '0.0.1'
+end

data/lib/uc3-sam-sceptre.rb

@@ -0,0 +1,145 @@
+# rubocop:disable Naming/FileName
+# frozen_string_literal: true
+
+require 'aws-sdk-cloudformation'
+require 'aws-sdk-ssm'
+
+module Uc3SamSceptre
+  class ProxyError < StandardError; end
+
+  # Ruby proxy for running SAM dynamically
+  class Proxy
+    DEFAULT_REGION = 'us-west-2'
+    DEFAULT_PROGRAM = 'uc3'
+    DEFAULT_ENV = 'dev'
+
+    MSG_INVALID_PARAMS = 'Invalid initialization params. Expected at least the [:program, :service, :subservice, /
+                          :env, :git_repo, :stack_suffix, :admin_email_key] and one of the [:ecr_uri_key, :s3_arn_key]
+                          and at least one of the following arrays: [:fetchable_cf_params, :static_cf_params]. /
+                          See the README.md for examples.'
+
+    attr_accessor :region, :program, :service, :subservice, :env, :git_repo, :stack_suffix, :admin_email_key, # Required
+                  :ecr_uri_key, :s3_arn_key, # Require one of these
+                  :ssm_args, :cf_args, :static_args, # Require at least one of these
+                  :account_env, :prefix, :ssm_key_prefix, :ssm_client, :cf_client, :stack_exports
+
+    # Initialize the SAM proxy
+    def initialize(**params)
+      @region = params.fetch(:region, DEFAULT_REGION)
+      @program = params.fetch(:program, DEFAULT_PROGRAM)
+      @service = params[:service]
+      @subservice = params[:subservice]
+      @env = params.fetch(:env, DEFAULT_ENV)
+      @git_repo = params[:git_repo]
+      @stack_suffix = params[:stack_suffix]
+      @ecr_uri_key = params[:ecr_uri_key]
+      @s3_arn_key = params[:s3_arn_key]
+      @admin_email_key = params[:admin_email_key]
+
+      @auto = params.fetch(:auto, false)
+
+      # Determine which AWS account we are working with, dev or prd (stg and prd are shared)
+      @account_env = %w[stg prd].include?(@env) ? 'prd' : 'dev'
+      @prefix = "#{@program}-#{@service}-#{@subservice}-#{@env}"
+      @ssm_key_prefix = params.fetch(:ssm_key_prefix, "/#{@prefix.gsub('-', '/')}/")
+
+      @fetchable_cf_params = params.fetch(:fetchable_cf_params, [])
+      @static_cf_params = params.fetch(:static_cf_params, [])
+
+      raise ProxyError, MSG_INVALID_PARAMS if @service.nil? || @subservice.nil? || @git_repo.nil? || @stack_suffix.nil? ||
+                                              (@ecr_uri_key.nil? && @s3_arn_key.nil?) ||
+                                              (@fetchable_cf_params.empty? && @static_cf_params.empty?)
+
+      @ssm_client = Aws::SSM::Client.new(region: DEFAULT_REGION)
+      @cf_client = Aws::CloudFormation::Client.new(region: DEFAULT_REGION)
+
+      @stack_exports = cf_client.list_exports.exports
+      @stack_name = "#{@prefix}-#{@stack_suffix}"
+    rescue Aws::Errors::ServiceError => e
+      raise ClientError, "Unable to connect to AWS resource. - #{e.message}"
+    end
+
+    # Run a SAM build
+    def build
+      system 'sam build'
+    end
+
+    # Run a SAM deploy
+    def deploy(guided: false)
+      args = [
+        "--stack name #{@stack_name}",
+        "--confirm-changeset #{!@auto}",
+        '--capabilities CAPABILITY_NAMED_IAM',
+        '--disable-rollback false',
+        "--tags #{_sam_tags(admin_email: _locate_value(key: @admin_email_key)}",
+        "--parameter-overrides #{_build_deploy_overrides}"
+      ]
+      # Add the S3 or ECR details depending on what we're working with
+      args << "--s3-bucket #{_locate_value(key: @s3_arn_key)&..gsub('arn:aws:s3:::', '')}" unless @s3_arn_key.nil?
+      args << "--s3-prefix #{@stack_name}" unles @s3_arn_key.nil?
+      args << "--image-repository #{_locate_value(key: @ecr_uri_key)}" unless @ecr_uri_key.nil?
+
+      args << "--guided" if guided
+      system "sam deploy #{args.join(' ')}"
+    end
+
+    private
+
+    # Construct the AWS tags that SAM will use when building resources
+    def _sam_tags(admin_email: nil)
+      tags = [
+        "Program=#{@program}",
+        "Service=#{@service}",
+        "Subservice=#{@subservice}",
+        "Environment=#{@env}",
+        "CodeRepo=#{@git_repo}"
+      ]
+      tags << "Contact=#{admin_email}" unless admin_email.nil?
+      tags
+    end
+
+    # Construct the SAM deploy arguments
+    def _build_deploy_overrides
+      overrides = [_sam_param(key: 'Env', value: @env)]
+
+      @fetchable_cf_params.each do |hash|
+        val = _locate_value(name: hash[:stack_name])
+        msg = "Unable to locate value for #{hash[:stack_export_name]}! Make sure you have exported it from the CF Stack."
+        raise ProxyError, msg if val.nil?
+
+        overrides << { _sam_param(key: hash.fetch(:template_param_name, hash[:stack_export_name]), value: val) }
+      end
+
+      @static_cf_params.each do { |hash| overrides << _sam_param(key: hash[:template_param_name], value: hash[:value]) }
+      overrides
+    end
+
+    # Convert the Hash key and value into SAM deploy args
+    def _sam_param(key:, value:)
+      return '' if key.nil? || value.nil?
+
+      "ParameterKey=#{key},ParameterValue=#{value}"
+    end
+
+    # Search for the key in the following order: Stack exports, SSM parameters
+    def _locate_value(key:)
+      val = _fetch_stack_export(name: key)
+      return val unless val.nil?
+
+      _fetch_ssm_parameter(key: key)
+    end
+
+    # Fetches an SSM parameter key and returns the value
+    def _fetch_ssm_parameter(key:)
+      name = key.start_with?(@ssm_key_prefix) ? key : "#{@ssm_key_prefix}#{key}"
+      @ssm_client.get_parameter(name: name, with_decryption: true)&.parameter&.value
+    end
+
+    # Fetch a CloudFormation stack export
+    def _fetch_stack_export(name:)
+      key = name.to_s.downcase.strip
+      @stack_exports.select { |exp| exp[:exporting_stack_id].include?(@prefix) && exp[:name].downcase.strip == key }.first
+    end
+  end
+end
+# rubocop:enable Naming/FileName

metadata

@@ -0,0 +1,145 @@
+--- !ruby/object:Gem::Specification
+name: uc3-sam-sceptre
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+platform: ruby
+authors:
+- Brian Riley
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2023-06-04 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: aws-sdk-s3cloudformation
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.78'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.78'
+- !ruby/object:Gem::Dependency
+  name: aws-sdk-ssm
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.150'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.150'
+- !ruby/object:Gem::Dependency
+  name: byebug
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 11.1.3
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 11.1.3
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 3.9.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 3.9.0
+- !ruby/object:Gem::Dependency
+  name: rubocop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 1.50.2
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 1.50.2
+- !ruby/object:Gem::Dependency
+  name: rubocop-performance
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 1.17.1
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 1.17.1
+- !ruby/object:Gem::Dependency
+  name: rubocop-rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 2.20.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 2.20.0
+description: Allows you to kick off a SAM build/deploy from Sceptre
+email:
+- brian.riley@ucop.edu
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- lib/uc3-sam-sceptre.rb
+- lib/uc3-sam-sceptre/version.rb
+homepage: https://github.com/CDLUC3/dmp-hub-cfn/blob/main/src/sam/gems/uc3-sam-sceptre
+licenses:
+- MIT
+metadata:
+  rubygems_mfa_required: 'false'
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '2.7'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.1.6
+signing_key: 
+specification_version: 4
+summary: Functionality to trigger SAM build and deploy using Sceptre hooks.
+test_files: []