cfndsl 0.15.0 → 1.0.0

data/Gemfile

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 source 'https://rubygems.org'
 
 gemspec

data/README.md

@@ -1,9 +1,9 @@
 cfndsl
 ======
 
-[![Build Status](https://travis-ci.org/stevenjack/cfndsl.png?branch=master)](https://travis-ci.org/stevenjack/cfndsl)
+[![Build Status](https://travis-ci.org/cfndsl/cfndsl.png?branch=master)](https://travis-ci.org/cfndsl/cfndsl)
 [![Gem Version](https://badge.fury.io/rb/cfndsl.png)](http://badge.fury.io/rb/cfndsl)
-[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/stevenjack/cfndsl?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
+[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/cfndsl/cfndsl?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) [![Join the chat at https://gitter.im/cfndsl/cfndsl](https://badges.gitter.im/cfndsl/cfndsl.svg)](https://gitter.im/cfndsl/cfndsl?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
 [AWS Cloudformation](http://docs.amazonwebservices.com/AWSCloudFormation/latest/UserGuide/GettingStarted.html) templates are an incredibly powerful way to build
 sets of resources in Amazon's AWS environment. Unfortunately, because
@@ -24,8 +24,22 @@ templates by running ruby.
 
 ## Getting Started
 
+ruby version > 2.4 is required to run cfndsl, you should look at using rbenv example for installing with rbenv
+
+    rbenv exec gem install cfndsl
+
+Example for doing it system wide Ruby
+
     sudo gem install cfndsl
 
+Update the the cloudformation specification to the latest version.
+
+    cfndsl -u
+
+or update to a specific version
+
+    cfndsl -u 7.1.0
+
 Now write a template in the dsl
 
 ```ruby
@@ -288,7 +302,7 @@ end
 
 ### Template Resources
 
-Cfndsl creates accessor methods for all of the resources listed [here](https://github.com/stevenjack/cfndsl/blob/master/lib/cfndsl/aws/types.yaml) and [here](https://github.com/stevenjack/cfndsl/blob/master/lib/cfndsl/os/types.yaml). If a resource is missing, or if you prefer to explicitly enter a resource in a template, you can do so. Keep in mind that since you are using the generic `Resource` class, you will also need to explicitly set the `Type` and that you no longer have access to the helper methods defined on that particular class, so you will have to use the `Property` method to set them.
+Cfndsl creates accessor methods for all of the resources listed [here](https://github.com/cfndsl/cfndsl/blob/master/lib/cfndsl/aws/types.yaml) and [here](https://github.com/cfndsl/cfndsl/blob/master/lib/cfndsl/os/types.yaml). If a resource is missing, or if you prefer to explicitly enter a resource in a template, you can do so. Keep in mind that since you are using the generic `Resource` class, you will also need to explicitly set the `Type` and that you no longer have access to the helper methods defined on that particular class, so you will have to use the `Property` method to set them.
 
 ```ruby
 CloudFormation do
@@ -428,16 +442,15 @@ The cfndsl command line program now accepts some command line options.
 Usage: cfndsl [options] FILE
     -o, --output FILE                Write output to file
     -y, --yaml FILE                  Import yaml file as local variables
-    -r, --ruby FILE                  Evaluate ruby file before template
     -j, --json FILE                  Import json file as local variables
     -p, --pretty                     Pretty-format output JSON
     -f, --format FORMAT              Specify the output format (JSON default)
     -D, --define "VARIABLE=VALUE"    Directly set local VARIABLE as VALUE
     -v, --verbose                    Turn on verbose ouptut
-    -b, --disable-binding            Disable binding configuration
     -m, --disable-deep-merge         Disable deep merging of yaml
     -s, --specification-file FILE    Location of Cloudformation Resource Specification file
-    -u, --update-specification       Update the Cloudformation Resource Specification file
+    -u [VERSION],                    Update the Resource Specification file to latest, or specific version
+        --update-specification
     -g RESOURCE_TYPE,RESOURCE_LOGICAL_NAME,
         --generate                   Add resource type and logical name
     -l, --list                       List supported resources
@@ -554,38 +567,87 @@ would generate a template with 5 instances declared.
 Specifying multiple -y options will default deep_merge all the yaml in the order specified.
 You can disable this with -m.
 
-Finally, the -r option gives you the opportunity to execute some
-arbitrary ruby code in the evaluation context before the cloudformation
-template is evaluated (this is not available if `--disable-binding` is used).
-
 ### Rake task
 Simply add the following to your `Rakefile`:
 
 ```ruby
 require 'cfndsl/rake_task'
 
-CfnDsl::RakeTask.new do |t|
-  t.cfndsl_opts = {
-    verbose: true,
-    files: [{
-      filename: 'templates/application.rb',
-      output: 'application.json'
-    }],
-    extras: [
-      [ :yaml, 'templates/default_params.yml' ]
-    ]
-  }
+namespace(:cfndsl) do
+  CfnDsl::RakeTask.new do |t|
+    # Use a custom specification file
+    t.specification(file: 'tmp/cloudformation_resources.json')
+
+    desc 'Generate CloudFormation Json'
+    t.json(name: :json, files: FileList.new('sample/*.rb'), pathmap: 'tmp/%f.json')
+    
+    # Generate yaml output, loading an extra file that matches the source file
+    t.yaml(name: :yaml, files: 'sample/t1.rb', pathmap: 'tmp/%f.yaml', extras: '%X.yaml')
+  end
 end
 ```
 
 And then use rake to generate the cloudformation:
 
 ```bash
-$ bin/rake generate
+$ bin/rake cfndsl:generate
+```
+
+### Embedded Ruby
+```ruby
+
+# Optionally before requiring 'cfndsl' set global options
+require `cfndsl/globals`
+CfnDsl.specification_file(file) # Override location of json spec file 
+CfnDsl.disable_deep_merge       # Prevent monkey patching of Hash with deep merge capabilities
+
+require `cfndsl`
+
+# As a function that takes a block of DSL
+template = CloudFormation do
+  # Some CfnDsl
+end
+puts JSON.pretty_generate(template.validate)
+
+# As a class that is a template
+class MyTemplate < CfnDsl::CloudFormationTemplate  
+  def initialize 
+    super do
+      # Base DSL
+    end  
+  end
+  
+  def instance(logical_id)
+    this = self
+    EC2_Instance(logical_id) do
+      self.class.name # << 'CfnDsl::ResourceDefinition' not 'MyTemplate' !
+      InstanceType this.instance_type  
+    end 
+  end
+   
+  #do not make this private! 
+  def instance_type
+    't3.large'      
+  end
+end
+
+# As a builder class
+class Builder
+  include CfnDsl::CloudFormation
+
+  def model
+    this = self # the DSL blocks are executed via instance_eval
+    ref_param = Ref('param')  # and other Fn::*
+    template = CloudFormation do
+      # Some DSL
+    end
+    template.validate.to_json
+  end
+end
 ```
 
 ### Generating CloudFormation resources from cfndsl
-By supplying the -g paramater you are now able to generate cloudformation resources for supported objects, for a list of supported resources run cfndsl -l
+By supplying the -g parameter you are now able to generate cloudformation resources for supported objects, for a list of supported resources run cfndsl -l
 
 Example
 ```bash

data/Rakefile

@@ -1,9 +1,12 @@
+# frozen_string_literal: true
+
 require 'bundler/gem_tasks'
 require 'rspec/core/rake_task'
 require 'cfndsl/version'
 require 'rubocop/rake_task'
 require 'yamllint/rake_task'
 require 'github_changelog_generator/task'
+require 'cfndsl/rake_task'
 
 RSpec::Core::RakeTask.new
 RuboCop::RakeTask.new
@@ -16,16 +19,37 @@ end
 
 YamlLint::RakeTask.new do |t|
   t.paths = %w[
-    lib/cfndsl/aws/types.yaml
-    lib/cfndsl/os/types.yaml
     sample/t1.yaml
     .travis.yml
     .rubocop.yml
   ]
 end
 
-task default: %i[spec rubocop yamllint]
+task default: %i[clean spec rubocop yamllint samples:generate]
+
+# Test our own rake task and samples
+
+directory 'tmp'
 
+namespace :samples do
+  source_files = FileList.new('sample/*.rb') { |fl| fl.exclude('**/circular.rb') }
+
+  CfnDsl::RakeTask.new do |t|
+    t.specification(file: 'tmp/cloudformation_resources.json')
+    desc 'Generate CloudFormation Json'
+    t.json(name: :json, files: source_files, pathmap: 'tmp/%f.json', pretty: true, extras: FileList.new('sample/*.yaml'))
+    t.yaml(name: :yaml, files: 'sample/t1.rb', pathmap: 'tmp/%f.yaml', extras: '%X.yaml')
+  end
+end
+
+CLEAN.add 'tmp/*.rb.{json,yaml}', 'tmp/cloudformation_resources.json'
+
+CfnDsl::RakeTask.new do |t|
+  desc 'Update Embedded Cloudformation specification'
+  t.specification(name: :update_cfn_spec, file: CfnDsl::LOCAL_SPEC_FILE, version: 'latest')
+end
+
+# TODO: Bump should ensure we have the latest upstream resource spec
 task :bump, :type do |_, args|
   type = args[:type].downcase
   version_path = 'lib/cfndsl/version.rb'
@@ -35,9 +59,7 @@ task :bump, :type do |_, args|
 
   raise unless types.include?(type)
 
-  if `git rev-parse --abbrev-ref HEAD`.strip != 'master'
-    raise "Looks like you're trying to create a release in a branch, you can only create one in 'master'"
-  end
+  raise "Looks like you're trying to create a release in a branch, you can only create one in 'master'" if `git rev-parse --abbrev-ref HEAD`.strip != 'master'
 
   version_segments = CfnDsl::VERSION.split('.').map(&:to_i)
 

data/TODO.md

@@ -0,0 +1,18 @@
+# 1.0.0 Release Changes
+
+## Prepare for Machine Ingestable Types
+
+## ~~Remove OS Heat Support~~
+
+## Remove Ruby/Constant Interpolation
+
+## ~~Better Ruby Api~~
+
+## 100% Test Coverage
+
+## Use Ruby Contracts for Resource Validation
+
+## Style Guide/Wiki/README/Examples Update
+
+## Fully implement ExternalParameters as a Mash
+

data/UPGRADING.md

@@ -0,0 +1,22 @@
+# Major version upgrades
+
+## 0.x to 1.x
+
+### Deprecations
+
+* FnFormat => FnSub
+* addTag => add_tag
+* checkRefs => check_refs
+* Ruby versions < 2.4
+* Legacy cfndsl resource specification files
+
+### Validation
+
+* Tighter validation including for null values and cyclic references in Resources, Outputs, Rules and Conditions
+* Tighter definition of duplicates. eg. Route must be EC2_Route because another service now has a Route resource.
+* Requires the specification file to exist at the time it is explicitly set with CfnDsl.specification_file=
+
+#### Spec version 
+
+The AWS cloudformation spec will be regularly updated on every release
+

data/cfndsl.gemspec

@@ -1,25 +1,26 @@
-# coding: utf-8
+# frozen_string_literal: true
 
-lib = File.expand_path('../lib', __FILE__)
+lib = File.expand_path('lib', __dir__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'cfndsl/version'
 
 Gem::Specification.new do |s|
-  s.name          = 'cfndsl'
-  s.version       = CfnDsl::VERSION
-  s.summary       = 'AWS Cloudformation DSL'
-  s.description   = 'DSL for creating AWS Cloudformation templates'
-  s.authors       = ['Steven Jack', 'Chris Howe']
-  s.email         = ['stevenmajack@gmail.com', 'chris@howeville.com']
-  s.files         = `git ls-files`.split($INPUT_RECORD_SEPARATOR)
-  s.homepage      = 'https://github.com/stevenjack/cfndsl'
-  s.license       = 'MIT'
-  s.test_files    = s.files.grep(%r{^(test|spec|features)/})
-  s.require_paths = ['lib']
+  s.name                  = 'cfndsl'
+  s.version               = CfnDsl::VERSION
+  s.summary               = 'AWS Cloudformation DSL'
+  s.description           = 'DSL for creating AWS Cloudformation templates'
+  s.authors               = ['Steven Jack', 'Chris Howe', 'Travis Dempsey', 'Greg Cockburn']
+  s.email                 = ['stevenmajack@gmail.com', 'chris@howeville.com', 'dempsey.travis@gmail.com', 'gergnz@gmail.com']
+  s.files                 = `git ls-files`.split($INPUT_RECORD_SEPARATOR)
+  s.homepage              = 'https://github.com/cfndsl/cfndsl'
+  s.license               = 'MIT'
+  s.test_files            = s.files.grep(%r{^(test|spec|features)/})
+  s.require_paths         = ['lib']
+  s.bindir                = 'exe'
+  s.required_ruby_version = '~> 2.3'
 
   s.executables << 'cfndsl'
 
-  s.add_development_dependency 'bundler', '~> 1.13'
-
-  s.post_install_message = "'addTag' is now deprecated in favour of 'add_tag'. 'addTag' will be removed in the next major version."
+  s.add_development_dependency 'bundler', '~> 2.1'
+  s.add_runtime_dependency 'hana', '~> 1.3'
 end

data/exe/cfndsl

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'cfndsl/runner'
+CfnDsl::Runner.invoke!

data/lib/cfndsl.rb

@@ -1,114 +1,4 @@
-require 'forwardable'
-require 'json'
+# frozen_string_literal: true
 
-require 'deep_merge/deep_merge'
-require 'cfndsl/globals'
-require 'cfndsl/module'
-require 'cfndsl/errors'
-require 'cfndsl/ref_check'
-require 'cfndsl/jsonable'
-require 'cfndsl/properties'
-require 'cfndsl/update_policy'
-require 'cfndsl/creation_policy'
-require 'cfndsl/conditions'
-require 'cfndsl/mappings'
-require 'cfndsl/resources'
-require 'cfndsl/parameters'
-require 'cfndsl/outputs'
-require 'cfndsl/patches'
-require 'cfndsl/specification'
-require 'cfndsl/aws/cloud_formation_template'
-require 'cfndsl/os/heat_template'
-require 'cfndsl/external_parameters'
-require 'cfndsl/version'
-
-# CfnDsl
-module CfnDsl
-  # This function handles the eval of the template file and returns the
-  # results. It does this with a ruby "eval", but it builds up a customized
-  # binding environment before it calls eval. The environment can be
-  # customized by passing a list of customizations in the extras parameter.
-  #
-  # These customizations are expressed as an array of pairs of
-  # (type,filename). They are evaluated in the order they appear in the
-  # extras array. The types are as follows
-  #
-  # :yaml - the second element is treated as a file name, which is loaded
-  #         as a yaml file. The yaml file should contain a top level
-  #         dictionary. Each of the keys of the top level dictionary is
-  #         used as a local variable in the evalation context.
-  #
-  # :json - the second element is treated as a file name, which is loaded
-  #         as a json file. The yaml file should contain a top level
-  #         dictionary. Each of the keys of the top level dictionary is
-  #         used as a local variable in the evalation context.
-  #
-  # :ruby - the second element is treated as a file name which is evaluated
-  #         as a ruby file. Any assigments (or other binding affecting
-  #         side effects) will persist into the context where the template
-  #         is evaluated
-  #
-  # :raw  - the second element is treated as a ruby statement and is
-  #         evaluated in the binding context, similar to the contents of
-  #         a ruby file.
-  #
-  # Note that the order is important, as later extra sections can overwrite
-  # or even undo things that were done by earlier sections.
-
-  # rubocop:disable all
-  def self.eval_file_with_extras(filename, extras = [], logstream = nil)
-    b = binding
-    params = CfnDsl::ExternalParameters.refresh!
-    extras.each do |type, file|
-      case type
-      when :yaml, :json
-        klass_name = type.to_s.upcase
-        logstream.puts("Loading #{klass_name} file #{file}") if logstream
-        params.load_file file
-        params.add_to_binding(b, logstream) unless disable_binding?
-      when :ruby
-        if disable_binding?
-          logstream.puts("Interpreting Ruby files was disabled. #{file} will not be read") if logstream
-        else
-          logstream.puts("Running ruby file #{file}") if logstream
-          b.eval(File.read(file), file)
-        end
-      when :raw
-        params.set_param(*file.split('='))
-        unless disable_binding?
-          logstream.puts("Running raw ruby code #{file}") if logstream
-          b.eval(file, 'raw code')
-        end
-      end
-    end
-
-    logstream.puts("Loading template file #{filename}") if logstream
-    b.eval(File.read(filename), filename)
-  end
-end
-
-def CloudFormation(&block)
-  x = CfnDsl::CloudFormationTemplate.new
-  x.declare(&block)
-  invalid_references = x.check_refs
-  if invalid_references
-    abort invalid_references.join("\n")
-  elsif CfnDsl::Errors.errors?
-    abort CfnDsl::Errors.errors.join("\n")
-  else
-    return x
-  end
-end
-
-def Heat(&block)
-  x = CfnDsl::HeatTemplate.new
-  x.declare(&block)
-  invalid_references = x.check_refs
-  if invalid_references
-    abort invalid_references.join("\n")
-  elsif CfnDsl::Errors.errors?
-    abort CfnDsl::Errors.errors.join("\n")
-  else
-    return x
-  end
-end
+require_relative 'cfndsl/cloudformation'
+require_relative 'cfndsl/cfnlego'