rubycfn 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c43cabdbb5f495f514c1ebc4a349558b5c9d8333
-  data.tar.gz: d5440442930fa154a50f53499b29f33c7caeac69
+  metadata.gz: b0849cb139ac3748d7b632536c72f4cfdd24cbdb
+  data.tar.gz: a2dfc5d8d89ea55f7ca693ebd264f71f4b37efe7
 SHA512:
-  metadata.gz: 47448e9fa021ecb199521f3550d1c4f5b505dc6c34700dea376518f5972416601852f73f716372010e90d19fa3e7afe71b222eadcdf14a3e63099bc12f39e786
-  data.tar.gz: d27ca79b9e457f496e5435e659845df1870a9fb140806a4f777d991d88fae306d685b7c47a70acb92989d125b229df88abd1b6692fc8833e5fb288f760c2c838
+  metadata.gz: d84760f33c8903f332e4b78c969768bf550a7b25ae50c4df22f0cdf0b9572ba7439c1cfc6ffa8b19bc93b9e4f647c841d2b5d783b46c41c772d263186b78f908
+  data.tar.gz: 281ed98ef84978bc640d4ab7ffd475fa2c9d30777e81007aca22ee3dded7ffe53e36804abb744fd68f9d7492a5cff7b20b1f1f0317408a9c9bec2b8d445b7c43

data/CHANGELOG.md

@@ -2,7 +2,11 @@
 All notable changes to Rubycfn will be documented in this file.
 This project uses [Semantic Versioning](http://semver.org/).
 
-## 0.2.1 (Next Release)
+## 0.2.2 (Next Release)
+
+## 0.2.1
+  * Fixed bug in VPC compound resource. Resource names are now camel cased -- [@dennisvink][@dennisvink]
+  * Updated README.md -- [@dennisvink][@dennisvink]
 
 ## 0.2.0
   * Added support for GCP templates -- [@dennisvink][@dennisvink]

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    rubycfn (0.2.0)
+    rubycfn (0.2.1)
       activesupport (~> 5.1.5)
       dotenv (~> 2.4.0)
       json (~> 2.1.0)

data/README.md

@@ -1,24 +1,62 @@
 # RubyCfn
 
-RubyCfn is a light-weight tiny CloudFormation DSL to make expressing
-CloudFormation as Ruby code a bit more pleasing to the eye.
+[RubyCfn](https://rubycfn.com/) is a light-weight tiny CloudFormation, Deployment Manager and ARM DSL to make expressing
+cloud vendor templates as Ruby code a bit more pleasing to the eye.
 
-## Installation
+## Quick start
 
-Type: `gem install rubycfn`
+Install Rubycfn:
+`gem install rubycfn`
 
-or, create a Gemfile with this content:
+Starting a new Rubycfn project:
+`rubycfn`
+$ rubycfn
+__________ ____ __________________.___._________ _____________________
+\______   \    |   \______   \__  |   |\_   ___ \\_   _____/\______   \
+ |       _/    |   /|    |  _//   |   |/    \  \/ |    __)   |    |  _/
+ |    |   \    |  / |    |   \\____   |\     \____|     \    |    |   \
+ |____|_  /______/  |______  // ______| \______  /\___  /    |______  /
+        \/                 \/ \/               \/     \/            \/ [v0.2.1]
+Project name? example
+Account ID? 1234567890
+Select region EU (Frankfurt)
 
-```
-source "https://rubygems.org"
+Installing project dependencies:
+`bundle`
 
-gem "rubycfn", "~> 0.2.0"
+Updating project dependencies:
+`bundle update`
 
-```
+Compiling Rubycfn project:
+`rake compile`
+
+Running Rubycfn unit tests:
+`rake spec`
+
+Running tests and compiling:
+`rake`
+
+Converting CloudFormation JSON template to Rubycfn:
+`./cfn2rubycfn /path/to/cloudformation_template.json`
+
+## Philosophy
 
-## Starting a new project
+Standardisation is key to keep your engineering team agile. Time spent on projects that deviate from a standard implementation is time taken away from delivering value. Custom implementations are detrimental to a team’s velocity and scalability. It hinders knowledge sharing as a select few have knowledge about the specifics of such a custom implementation, and because the wheel is reinvented many times over proper testing is tedious at best. We’ve automated best practices and ensured that new projects automatically incorporate our principles. Our tooling has been built with cloud engineer happiness in mind.
 
-Typing `rubycfn` at the prompt and answering its questions generates a new sample project.
+## Overview of Rubycfn
+
+RubyCfn is an abstraction layer around several Cloud templates such as CloudFormation (AWS), Deployment Manager (GCP) and ARM (Azure). Rubycfn projects are set up for easy grouping of resources that have a mutual cohesion, and structured in such a way to make it easy for developers to quickly find what they need. Rubycfn is a so-called ‘DSL’ on top of these template formats and presents templates as code that is friendly to the eye and easy to read and understand. In addition of being an alternate representation of a template, Rubycfn allows you to combine template generation with programming logic making it far more versatile than what the respective cloud providers offer in their templates. Last but not least Rubycfn enforces code quality by testing the generated templates against unit tests, checking if the expected resources and their configuration matches with what was actually generated, and by LINTing the templates and the underlying code that generates the templates.
+
+Out of the box Rubycfn comes with a CI/CD pipeline. It’s a serverless pipeline running on Amazon Web Services (AWS), which you can fully configure using the complimentary `buildspec.yml`.  The CI/CD pipeline is linked to a Github repository, and a change in this repository triggers the CI/CD pipeline to execute the steps you’ve defined in the buildspec.yml.
+
+Typically a commit to your application GIT repository triggers the build process and the following things happen:
+- Application code is checked out
+- Infrastructure as code (Rubycfn project) is checked out
+- Rubycfn project is ‘built’, kicking off unit tests against the underlying code, and against the resulting build artifact (template(s))
+- Application (unit) tests are ran
+- The build artifact is stored (versioned), so you can use it as input for your delivery pipeline
+- The artifact may or may not include the application code. A part of the build process could - for example - also be that the application is dockerized and pushed to a docker registry.
+- The resulting artifact is the complete recipe to deploy the application and associated resources to AWS, GCP or Azure.
 
 ## Example code
 
@@ -199,6 +237,210 @@ or...
 Paste the CloudFormation output in [cfnflip.com](https://cfnflip.com/) to
 convert it to YAML format ;)
 
+## The anatomy of a Rubycfn project
+
+When you start a new Rubycfn project it comes structured out of the box. We standardise this structure so that it’s uniform from project to project. A colleague Cloud Engineer needs to be able to take over or troubleshoot a project immediately without having to learn the inner working of the project first. This allows us to remain agile. In addition, by not having to rely on pre-existing knowledge about a project (and thus having knowledge of a project with just a few select people), we promote synergy.
+
+You start a new project by typing `rubycfn` at the prompt. This will ask you a couple of questions about the project - such as the project’s name. The entire project is then generated for you. It then looks like this:
+
+-rw-r--r--   1 binx  staff   166 Oct 17 16:06 .env
+-rw-r--r--   1 binx  staff    81 Oct 17 16:06 .env.test
+-rw-r--r--   1 binx  staff   246 Oct 17 16:06 Gemfile
+-rw-r--r--   1 binx  staff   346 Oct 17 16:06 Rakefile
+drwxr-xr-x   2 binx  staff    64 Oct 17 16:06 build
+-rwxrwxrwx   1 binx  staff  3223 Oct 17 16:06 cfn2rubycfn
+drwxr-xr-x   3 binx  staff    96 Oct 17 16:06 config
+-rw-r--r--   1 binx  staff    15 Oct 17 16:06 format.vim
+drwxr-xr-x   6 binx  staff   192 Oct 17 16:06 lib
+drwxr-xr-x   4 binx  staff   128 Oct 17 16:06 spec
+
+First the flat files:
+
+`.env` and `.env.test` are files where you store environment variables that you may want to use in your project code. The difference between the .env and the .env.test file is that the .env file is the “global” environment variable file, whereas the .env.test file is an environment-specific environment variable file, that can override values you’ve specified in your .env file (or add new environment variables, for that matter). You’d typically have a .env.test, .env.acceptance and a .env.production file for things like instance sizing.
+
+Example:
+$ cat .env.test
+# ENV vars for test environment
+APPLICATION_INSTANCE_CLASS="t2.micro"
+
+To make use of - for example - .env.production as source, you can either override the ENVIRONMENT variable in the .env file, setting it to production, or you can invoke rake as: `ENVIRONMENT="production" rake`.
+
+The `Gemfile` is a collection of Ruby dependencies. By running `bundle` you install the dependencies.
+
+The `Rakefile` contain the tasks that are performed when you type the `rake` command. It consists of a `compile` task and a `spec` task. You can invoke the tasks individually by typing `rake compile` or `rake spec`, but by default the `spec` task is ran first, and then the `compile` task. A “spec” is another word for unit test. It’s important to run the unit test first, so that if a test fails no template is generated. If you just run `rake` both tasks are executed sequentially, provided the specs throw no error.
+
+`cfn2rubycfn` is a small helper script that converts AWS CloudFormation scripts to Rubycfn code. This allows you to migrate your existing projects over to Rubycfn quickly. It exports the converted CloudFormation script to `generated.rb`, a ready to use module for your stacks.
+
+Example:
+$ cat sample.json
+{
+  "AWSTemplateFormatVersion": "2010-09-09",
+  "Resources": {
+    "ApiGatewayRestApi": {
+      "Properties": {
+        "Name": "trigger-github-webhook"
+      },
+      "Type": "AWS::ApiGateway::RestApi"
+    }
+  }
+}
+
+$ ./cfn2rubycfn sample.json
+Reformatting code...
+
+$ cat generated.rb
+module ConvertedStack
+  module Main
+    extend ActiveSupport::Concern
+    included do
+      resource :api_gateway_rest_api,
+        type: "AWS::ApiGateway::RestApi" do |r|
+        r.property(:name) { "trigger-github-webhook" }
+      end
+    end
+  end
+end
+
+The `format.vim` file is used by the CloudFormation conversion script to reindent the file after conversion. You can make changes to the file to reflect your preferred style of code indentation.
+
+Onto the subdirectories:
+
+The `build` directory is where all the compiled templates end up.
+
+Example:
+$ ls -al build/
+total 24
+drwxr-xr-x   4 binx  staff   128 Oct 17 16:27 .
+drwxr-xr-x  16 binx  staff   512 Oct 17 16:27 ..
+-rw-r--r--   1 binx  staff  3197 Oct 17 16:27 test-gcp-demostack.json
+-rw-r--r--   1 binx  staff  4152 Oct 17 16:27 test-aws-demostack.json
+
+The `spec` directory is where your unit tests live. These are not your application unit tests. A Rubycfn project lives in another universe than your application code. These unit tests test your expectations of the generated templates against reality. Such tests typically check for the existence and absence of particular resources, and values of properties. The tests are always executed when you run the `rake` command or the `rake spec` command. By default a project comes with unit tests that amongst other things check for the generation of the CI/CD pipeline and if it’s been configured correctly.
+
+Example:
+      context "Codebuild Service Role" do
+        let(:code_build_service_role) { resources["CodeBuildDemoServiceRole"] }
+        subject { code_build_service_role }
+
+        it { should have_key "Properties" }
+
+        context "Code build service role properties" do
+          let(:code_build_service_role_properties) { code_build_service_role["Properties"] }
+          subject { code_build_service_role_properties }
+
+          it { should have_key "AssumeRolePolicyDocument" }
+          it { should have_key "Path" }
+          it { should have_key "Policies" }
+
+          context "Code build service role policy document" do
+            let(:policy_document) { code_build_service_role_properties["Policies"][0]["PolicyDocument"] }
+            subject { policy_document }
+
+            it { should have_key "Statement" }
+
+            context "Code build service role actions" do
+              let(:statement) { policy_document["Statement"][0]["Action"] }
+              subject { statement }
+
+              it { should eq %w(logs:CreateLogGroup logs:CreateLogStream logs:PutLogEvents) }
+            end
+          end
+        end
+      end
+
+The `config` directory contains your buildspec.yml, which contain all the instructions for the build pipeline. It’s also possible to source the buildspec.yml from another source, such as the application repository.
+
+And finally, the `lib` directory contains all the relevant project code. The `lib` directory consists of several files and directories. The `lib/main.rb` is the bootstrapper that ties everything together. The `lib/compile.rb` is responsible for compiling the templates and writing them to the build directory.
+
+There are two directories under `lib`, namely `shared_concerns` and `stacks`. Concerns in this context simply mean Modules. Rubycfn relies on the ActiveConcern gem which makes modularisation of code very easy. The `shared_concerns` directory contains modules that can be used by several stacks. By default it has a `global_variables` module containing the following:
+
+module Concerns
+  module GlobalVariables
+    extend ActiveSupport::Concern
+
+    included do
+      variable :environment,
+               default: "test",
+               global: true,
+               value: ENV["ENVIRONMENT"]
+    end
+  end
+end
+
+This module exposes a variable `environment`, which defaults to `test` if not set. It sources the value from the ENVIRONMENT environment variable. This variable can be used throughout your project at any place you see fit.
+
+The `stacks` directory is a container for all stacks that you want to generate. There is no limitation to the amount of stacks that it supports. By default, it comes with a single stack for your project:
+
+Example `lib/stacks/demo_stack.rb`:
+module DemoStack
+  extend ActiveSupport::Concern
+  include Rubycfn
+
+  included do
+    include DemoStack::Main
+    include DemoStack::CICD
+  end
+end
+
+Our stack file consists of two modules: Main and CICD. When compiling the code, the combined result of the Main and CICD module will be written to the DemoStack json file in the build/ directory. Modularising stacks allows for separation of code by cohesion or any other logic you deem appropriate.
+
+The stacks directory also contains a directory that is named the same, minus the .rb extension: `lib/stacks/demo_stack/`
+
+All the stack modules live inside this directory. The modules that make up the stack are the actual implementation of the resources, parameters and outputs.
+
+An example of such a module:
+module DemoStack
+  module Main
+    extend ActiveSupport::Concern
+    included do
+      import(
+        path: "cloudsql.jinja"
+      )
+
+      resource :api_gateway_rest_api,
+        type: "AWS::ApiGateway::RestApi" do |r|
+        r.property(:name) { "#{environment}-webhook" }
+      end
+
+      resource "cloudsql",
+        type: "GCP::cloudsql.jinja" do |r|
+        r.property("database") do
+          {
+            "name": "#{environment}"
+          }
+        end
+        r.property("dbUser") do
+          {
+            "password": "test123_"
+          }
+        end
+        r.property("failover") { true }
+        r.property("readReplicas") { 1 }
+      end
+    end
+  end
+end
+
+When compiling the project, Rubycfn will recognise the resources for the different Cloud providers and write them to separate template files. When building a cloud agnostic solution, you implement the resources for the respective vendors. To decide which resources are deployed to which cloud provider, and to be able to switch them quickly, you can wrap the resources with `if` statements, but a better solution is to utilise the `amount` property for resources:
+
+      variable :cloudsql_vendor,
+               default: "GCP",
+               value: ENV["CLOUDSQL_VENDOR"]
+
+      resource "cloudsql",
+        amount: cloudsql_vendor == "GCP" ? 1 : 0,
+        type: "GCP::cloudsql.jinja" do |r|
+            ...
+      end
+
+      resource "cloudsql",
+        amount: cloudsql_vendor == "ARM" ? 1 : 0,
+        type: "ARM::MSSQL" do |r|
+			...
+      end
+
+By implementing resources in the above way you can switch particular resources to another cloud vendor by simply updating the environment variable in your CI/CD pipeline, while still being able to use the same variables and the same ecosystem.
+
 ## License
 
 MIT License

data/lib/compound/vpc.rb

@@ -32,7 +32,7 @@ module RubyCfn
 
           yield self if block_given? # Variable overrides
 
-          resource "#{prefix}_vpc#{suffix}",
+          resource "#{prefix}_vpc#{suffix}".cfnize,
                    type: "AWS::EC2::VPC" do |r, index|
             r.property(:cidr_block) { cidr_block }
             r.property(:enable_dns_support) { enable_dns_support }
@@ -40,22 +40,22 @@ module RubyCfn
             r.property(:instance_tenancy) { instance_tenancy } unless instance_tenancy.empty?
           end
 
-          resource "#{prefix}_internet_gateway#{suffix}",
+          resource "#{prefix}_internet_gateway#{suffix}".cfnize,
                    type: "AWS::EC2::InternetGateway"
 
-          resource "#{prefix}_route#{suffix}",
+          resource "#{prefix}_route#{suffix}".cfnize,
                    type: "AWS::EC2::Route" do |r, index|
             r.property(:destination_cidr_block) { "0.0.0.0/0" }
             r.property(:gateway_id) { "#{prefix}_internet_gateway#{suffix}".cfnize.ref }
             r.property(:route_table_id) { "#{prefix}_route_table#{suffix}".cfnize.ref }
           end
 
-          resource "#{prefix}_route_table#{suffix}",
+          resource "#{prefix}_route_table#{suffix}.cfnize",
                    type: "AWS::EC2::RouteTable" do |r, index|
             r.property(:vpc_id) { "#{prefix}_vpc#{suffix}".cfnize.ref }
           end
 
-          resource "#{prefix}_vpc_gateway_attachment#{suffix}",
+          resource "#{prefix}_vpc_gateway_attachment#{suffix}".cfnize,
                    type: "AWS::EC2::VPCGatewayAttachment" do |r, index|
             r.property(:internet_gateway_id) { "#{prefix}_internet_gateway#{suffix}".cfnize.ref }
             r.property(:vpc_id) { "#{prefix}_vpc#{suffix}".cfnize.ref }

data/lib/rubycfn/version.rb

@@ -1,4 +1,4 @@
 # Rubycfn version
 module Rubycfn
-  VERSION = "0.2.0"
+  VERSION = "0.2.1"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rubycfn
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Dennis Vink
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-10-17 00:00:00.000000000 Z
+date: 2018-10-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: neatjson