cfoo 0.0.1 → 0.0.2

data/README.md

@@ -4,7 +4,11 @@
 [![Coverage Status](https://coveralls.io/repos/drrb/cfoo/badge.png?branch=master)](https://coveralls.io/r/drrb/cfoo)
 [![Code Climate](https://codeclimate.com/github/drrb/cfoo.png)](https://codeclimate.com/github/drrb/cfoo)
 
-Write your CloudFormation templates in a YAML-based markup language
+[![Gem Version](https://badge.fury.io/rb/cfoo.png)](https://badge.fury.io/rb/cfoo)
+[![Dependency Status](https://gemnasium.com/drrb/cfoo.png)](https://gemnasium.com/drrb/cfoo)
+
+Cfoo (pronounced "sifu") lets you write your [CloudFormation](https://aws.amazon.com/cloudformation)
+templates [in YAML](#templates), and makes it easier with some [helpers](#shortcuts).
 
 ## Installation
 
@@ -14,9 +18,17 @@ Cfoo can be installed as a Ruby Gem
 
 ## Usage
 
-Process some Cfoo templates from the command line
+1. Write your CloudFormation templates using Cfoo YAML 
 
-    $ cfoo web-server-template.yml database_template.yml
+2. Turn your Cfoo templates into normal CloudFormation templates
+```terminal
+$ cfoo web-server.template.yml database.template.yml > web-stack.template.json
+```
+
+3. Create your stack with CloudFormation
+```terminal
+$ cfn-create-stack --stack-name WebStack -f web-stack.template.json
+```
 
 ## Templates
 
@@ -72,7 +84,41 @@ Properties:
     /opt/aws/bin/cfn-signal -e 0 -r "cfn-init completed" '$(WaitHandle)'
 ```
 
-### Shortcuts
+## Projects
+
+Using Cfoo, it is possible to split your templates up into logical components that will
+combined to form your CloudFormation template.
+
+First, create a directory in your project directory called `modules`. For each module,
+create some Cfoo templates defining the different parts of your app. Your project
+structure will look like this:
+
+```
+my-web-app
+└── modules
+    ├── application
+    │   ├── app_servers.yml
+    │   ├── database.yml
+    │   └── public_load_balancer.yml
+    ├── instances
+    │   └── instance_mappings.yml
+    └── network
+        ├── bastion.yml
+        ├── cfn_user.yml
+        ├── dns.yml
+        ├── nat.yml
+        ├── private_subnet.yml
+        ├── public_subnet.yml
+        └── vpc.yml
+```
+
+Use Cfoo to generate your project's CloudFormation template
+
+```terminal
+$ cfoo > web-app.template.json
+```
+
+## Shortcuts
 
 Cfoo allows you to simplify CloudFormation [intrinsic function](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html)
 references using its own shorthand
@@ -106,31 +152,61 @@ Cfoo Shortcut: `$(HostedZone).`
 ### YAML Types
 
 Cfoo gives you the option of using YAML custom data-types where it helps to make your templates easier to read.
-The table below uses inline YAML lists, but multiline lists can also be used.
 
 ##### Reference
 
-CloudFormation: `{ "Ref" : "InstanceType" }`
+CloudFormation:
+```json
+{ "Ref" : "InstanceType" }
+```
 
-YAML Type: `!Ref InstanceType`
+YAML Type:
+```yaml
+!Ref InstanceType
+```
 
 ##### Mapping Reference
 
-CloudFormation: `{ "FindInMap" : [ "SubnetConfig", "VPC", "CIDR" ] }`
+CloudFormation:
+```json
+{ "FindInMap" : [ "SubnetConfig", "VPC", "CIDR" ] }
+```
 
-YAML Type: `!FindInMap [ SubnetConfig, VPC, CIDR ]`
+YAML Type:
+```yaml
+!FindInMap [ SubnetConfig, VPC, CIDR ]
+```
 
 ##### Attribute Reference
 
-CloudFormation: `{ "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] }`
+CloudFormation:
+```json
+{ "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] }
+```
 
-YAML Type: `!GetAtt [ Ec2Instance, PublicIp ]`
+YAML Type:
+```yaml
+!GetAtt [ Ec2Instance, PublicIp ]
+```
 
 ##### Base64 String
 
-CloudFormation: `{ "Fn::Base64" : "#!/bin/bash\necho 'Running script...'" }`
+CloudFormation:
+```json
+{ "Fn::Base64" : "#!/bin/bash\necho 'Running script...'" }
+```
+
+YAML Type:
+```yaml
+!Base64 "#!/bin/bash\necho 'running script...'"
+```
 
-YAML Type: `!Base64 "#!/bin/bash\necho 'running script...'"`
+Alternative YAML Type:
+```yaml
+!Base64 |
+    #!/bin/bash
+    echo 'running script...'
+```
 
 ## Goals
 
@@ -144,9 +220,8 @@ Cfoo aims to let developers simplify CloudFormation templates by:
 
 ### Secondary Goals
 
-Cfoo aims (subject to Primary Goals) to:
+Cfoo also aims (subject to Primary Goals) to:
 
-- allow all aspects of CloudFormation templates to be expressed in YAML (so you don't need to use JSON)
 - allow inclusion existing JSON templates (so you don't have to switch all at once)
 
 ### Non-goals

data/TODO

@@ -2,13 +2,12 @@ Features
 ----------
 More tests for EL parsing
 Spaces in EL?
-More than [A-Za-z:] in EL identifiers?
+Allow lone dollar signs in strings? (escaping is only necessary if the dollar is in front of a bracket)
 Shorthand for AWS variables (e.g. { "Ref" : "AWS::StackId" } ) 
-Shorthand for Fn::GetAZs
-Better errors when parsing fails
+Allow arrays as function args (e.g. $(Join("", [1,2,3])))
 Split into resources/mappings/parameters/etc by putting templates into directories with those names
-Allow directory for raw templates
 Allow traditional JSON CloudFormation templates (including modules, splitting, etc)
+Expose parser, for use with Fog (github fog/fog)
 Vim highlighting
 Use multi-json instead of json gem (spec.add_dependency 'multi_json', '~> 1.0')
 Disablable warnings about naming conventions
@@ -23,7 +22,6 @@ Tests
 
 Tasks
 ----------
-Refactor EL parser
 
 Bugs
 ----------

data/bin/cfoo

@@ -11,13 +11,8 @@ end
 
 filenames = ARGV
 
-yaml_parser = Cfoo::YamlParser.new
-file_system = Cfoo::FileSystem.new(".", yaml_parser)
-project = Cfoo::Project.new(file_system)
-parser = Cfoo::Parser.new(file_system)
-processor = Cfoo::Processor.new(parser, project)
-renderer = Cfoo::Renderer.new
-cfoo = Cfoo::Cfoo.new(processor, renderer, STDOUT)
+factory = Cfoo::Factory.new(STDOUT, STDERR)
+cfoo = factory.cfoo
 
 if filenames.empty?
     cfoo.build_project

data/features/error_reporting.feature

@@ -0,0 +1,23 @@
+Feature: Error reporting
+    As a Cfoo user
+    I want to get useful parse errors
+    So that I can debug my templates
+
+    Scenario: YAML Parse error
+        Given I have a file "bad_yaml.yml" containing
+        """
+        EntryPoint: Key: Value:
+        """
+        When I process "bad_yaml.yml"
+        Then I should see an error containing "bad_yaml.yml"
+        And I should see an error containing "line"
+        And I should see an error containing "col"
+
+    Scenario: EL Parse error
+        Given I have a file "bad_el.yml" containing
+        """
+        EntryPoint: $(
+        """
+        When I process "bad_el.yml"
+        Then I should see an error containing "Source: $("
+        And I should see an error containing "Location: bad_el.yml line 1, column 13"

data/features/function_expansion.feature

@@ -0,0 +1,139 @@
+Feature: Expand Function Calls
+    As a CloudFormation user
+    I want to use an expression language as a shorthand for function calls
+    So that I my templates are easier to read
+
+    Scenario: Function with no parameters
+        Given I have a file "autoscaling_group.yml" containing
+        """
+        WebServerGroup:
+           Type: AWS::AutoScaling::AutoScalingGroup
+           Properties:
+               AvailabilityZones: $(GetAZs())
+        """
+        When I process "autoscaling_group.yml"
+        Then the output should match JSON
+        """
+        {
+            "AWSTemplateFormatVersion" : "2010-09-09",
+            "WebServerGroup" : {
+               "Type" : "AWS::AutoScaling::AutoScalingGroup",
+               "Properties" : {
+                 "AvailabilityZones" : { "Fn::GetAZs" : "" }
+               }
+            }
+        }
+        """
+
+    Scenario: Function with one parameter
+        Given I have a file "autoscaling_group.yml" containing
+        """
+        WebServerGroup:
+           Type: AWS::AutoScaling::AutoScalingGroup
+           Properties:
+               AvailabilityZones: $(GetAZs(us-east-1))
+        """
+        When I process "autoscaling_group.yml"
+        Then the output should match JSON
+        """
+        {
+            "AWSTemplateFormatVersion" : "2010-09-09",
+            "WebServerGroup" : {
+               "Type" : "AWS::AutoScaling::AutoScalingGroup",
+               "Properties" : {
+                 "AvailabilityZones" : { "Fn::GetAZs" : "us-east-1" }
+               }
+            }
+        }
+        """
+
+    Scenario: Function with multiple parameters
+        Given I have a file "autoscaling_group.yml" containing
+        """
+        FrontendFleet:
+            Type: AWS::AutoScaling::AutoScalingGroup
+            Properties:
+                AvailabilityZones:
+                    - $(GetAtt(PrivateSubnet, AvailabilityZone))
+        """
+        When I process "autoscaling_group.yml"
+        Then the output should match JSON
+        """
+        {
+            "AWSTemplateFormatVersion" : "2010-09-09",
+            "FrontendFleet" : {
+                "Type" : "AWS::AutoScaling::AutoScalingGroup",
+                "Properties" : {
+                    "AvailabilityZones" : [{ "Fn::GetAtt" : [ "PrivateSubnet", "AvailabilityZone" ] }]
+                }
+            }
+        }
+        """
+
+    Scenario: Function with no spaces between arguments
+        Given I have a file "autoscaling_group.yml" containing
+        """
+        FrontendFleet:
+            Type: AWS::AutoScaling::AutoScalingGroup
+            Properties:
+                AvailabilityZones:
+                    - $(GetAtt(PrivateSubnet,AvailabilityZone))
+        """
+        When I process "autoscaling_group.yml"
+        Then the output should match JSON
+        """
+        {
+            "AWSTemplateFormatVersion" : "2010-09-09",
+            "FrontendFleet" : {
+                "Type" : "AWS::AutoScaling::AutoScalingGroup",
+                "Properties" : {
+                    "AvailabilityZones" : [{ "Fn::GetAtt" : [ "PrivateSubnet", "AvailabilityZone" ] }]
+                }
+            }
+        }
+        """
+
+    Scenario: Function with lots of spaces between and around arguments
+        Given I have a file "autoscaling_group.yml" containing
+        """
+        FrontendFleet:
+            Type: AWS::AutoScaling::AutoScalingGroup
+            Properties:
+                AvailabilityZones:
+                    - $(GetAtt(  PrivateSubnet  ,  AvailabilityZone  ))
+        """
+        When I process "autoscaling_group.yml"
+        Then the output should match JSON
+        """
+        {
+            "AWSTemplateFormatVersion" : "2010-09-09",
+            "FrontendFleet" : {
+                "Type" : "AWS::AutoScaling::AutoScalingGroup",
+                "Properties" : {
+                    "AvailabilityZones" : [{ "Fn::GetAtt" : [ "PrivateSubnet", "AvailabilityZone" ] }]
+                }
+            }
+        }
+        """
+
+    Scenario: Function with embedded EL
+        Given I have a file "app_servers.yml" containing
+        """
+        AppServerLaunchConfig:
+            Type: AWS::AutoScaling::LaunchConfiguration
+            Properties:
+                ImageId: $(FindInMap(AWSRegionArch2AMI, $(AWS::Region), $(AWSInstanceType2Arch[$(FrontendInstanceType)][Arch])))
+        """
+        When I process "app_servers.yml"
+        Then the output should match JSON
+        """
+        {
+            "AWSTemplateFormatVersion" : "2010-09-09",
+            "AppServerLaunchConfig" : {
+                "Type": "AWS::AutoScaling::LaunchConfiguration",
+                "Properties" : {
+                    "ImageId": { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" }, { "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "FrontendInstanceType" }, "Arch" ] } ] }
+                }
+            }
+        }
+        """

data/features/step_definitions/cfoo_steps.rb

@@ -26,7 +26,7 @@ When /^I build the project$/ do
     cfoo.build_project
 end
 
-Then(/^the output should match JSON$/) do |expected_json|
+Then /^the output should match JSON$/ do |expected_json|
     begin
         expected = JSON.parse(expected_json)
     rescue StandardError => e
@@ -42,6 +42,14 @@ Then(/^the output should match JSON$/) do |expected_json|
     actual.should == expected
 end
 
+Then /^I should see "(.*?)"$/ do |expected_output|
+    stdout.messages.join("\n").should include expected_output
+end
+
+Then /^I should see an error containing "(.*?)"$/ do |expected_output|
+    stderr.messages.join("\n").should include expected_output
+end
+
 def write_file(filename, content)
     file_fqn = resolve_file(filename)
     mkdir_p(File.dirname file_fqn)
@@ -59,7 +67,7 @@ def resolve_file(filename)
 end
 
 def cfoo
-    @cfoo ||= Cfoo::Cfoo.new(processor, renderer, stdout)
+    @cfoo ||= Cfoo::Cfoo.new(processor, renderer, stdout, stderr)
 end
 
 def processor
@@ -92,6 +100,10 @@ def stdout
     @output ||= Output.new
 end
 
+def stderr
+    @output ||= Output.new
+end
+
 After do                                                                             
     rm_rf project_root if File.directory? project_root
 end

data/lib/cfoo.rb

@@ -1,5 +1,8 @@
+require "cfoo/array"
 require "cfoo/cfoo"
+require "cfoo/constants"
 require "cfoo/el_parser"
+require "cfoo/factory"
 require "cfoo/file_system"
 require "cfoo/module"
 require "cfoo/parser"
@@ -8,4 +11,5 @@ require "cfoo/project"
 require "cfoo/renderer"
 require "cfoo/version"
 require "cfoo/yaml"
+require "cfoo/yaml_parser"
 

data/lib/cfoo/array.rb

@@ -0,0 +1,13 @@
+class Array
+    def join_adjacent_strings
+        return clone if empty?
+        self[1..-1].inject([first]) do |combined_parts, part|
+            previous = combined_parts.pop
+            if previous.class == String && part.class == String
+                combined_parts << previous + part
+            else
+                combined_parts << previous << part
+            end
+        end
+    end
+end

data/lib/cfoo/cfoo.rb

@@ -1,15 +1,19 @@
 module Cfoo
     class Cfoo
-        def initialize(processor, renderer, stdout)
-            @processor, @renderer, @stdout = processor, renderer, stdout
+        def initialize(processor, renderer, stdout, stderr)
+            @processor, @renderer, @stdout, @stderr = processor, renderer, stdout, stderr
         end
 
         def process(*filenames)
             @stdout.puts(@renderer.render @processor.process(*filenames))
+        rescue Exception => error
+            @stderr.puts error
         end
 
         def build_project
             @stdout.puts(@renderer.render @processor.process_all)
+        rescue Exception => error
+            @stderr.puts error
         end
     end
 end