stax 0.0.3 → 0.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 0444c2ae4f050d73ba7cfd720eca476667102bf2
-  data.tar.gz: 764f80db2e413eade8840ef8f04c33d93003e190
+SHA256:
+  metadata.gz: 6fb67f05d66f75c3ee95f2f244189b9f0a24cf84c510280a02f17325b0f6c29b
+  data.tar.gz: e8a9c6ab1f3b869f36c688ffc7c9caf69799ed9fd9730f23beabd29a750ddfa2
 SHA512:
-  metadata.gz: bb84fb5b04019ef607f8f4a6658414fdda2b3dbf641a31bbedadfb785ab20f1e76c8d039529c71a52d5f18419ead4c5594297bdfc7303efef8ccb4f8d0ef0191
-  data.tar.gz: f65676b1870954bcc39c5d53a0ecd7b238d5b2fafd7bd2ea5761fc993dece0e33c7b8c2205ebf6a61021b148ba01f84f8856a523fd4d4c2fa6ff3bbfcea6fb41
+  metadata.gz: e31cf509ed73dbfc5045f5d2971d47894be9194f0a78d33812544ee5e854f15095c32ff3d4be868a168cf4d29475e9bf2bd03338f8f9043a6b43774bbecf3fa0
+  data.tar.gz: fad98ced6ce0cced2e8fa45405e6e3f5dd47d168ab5134a41c66e42cea280f2b36d9169b11d367aeb19e88cfbac3b093aa72541b619ca3de2a3b52c933dff9f2

data/README.md

@@ -1,12 +1,71 @@
 # Stax
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/stax`. To experiment with that code, run `bin/console` for an interactive prompt.
+Stax is a highly-opionated framework for managing Cloudformation
+stacks along with all the crappy glue code you need around them.
 
-TODO: Delete this and the text above, and describe your gem
+Stax is built as a set of ruby classes, with configuration based
+around sub-classing and monkey-patching.
+
+For now, Stax reads template files written using the
+[cfer](https://github.com/seanedwards/cfer) ruby wrapper. It should be
+straightforward to change to raw json/yaml, or a different wrapper by
+re-implementing the methods in `lib/stax/stack/crud.rb`.
+
+## Concepts
+
+### Application
+
+You have an application in a git repo. Our example will be called
+`website`. The Stax infrastructure code and cloudformation templates
+live in the same repo as the application code.
+
+### Branch
+
+You can check out any branch of your repo and deploy a
+fully-operational infrastructure using Stax.
+
+Example branches might be `prod`, `stg`, `dev`, or perhaps your model
+is `release/37`, `feature/38`, `hotfix/1234`, etc.
+
+### Stacks
+
+Each deployable branch consists of one or more actual cloudformation
+stacks. For example, our website app may consist of stacks `vpc`, `db`
+and `app`.
+
+In our experience it is better to build multiple coupled
+cloudformation stacks, handling discrete parts of an application
+infrastructure, rather than having a single giant template.
+
+These stacks are connected via their outputs and input parameters. For
+example, `vpc` stack outputs its subnet IDs, which are passed as
+parameters to the `app` stack. Stax is designed to handle this wiring
+for us.
+
+### Extensions
+
+Stax is intended to be modified to handle all the hackery needed in
+real-world app deployments. Each stack is modeled by subclassing the
+`Stax::Stack` class, and you are encouraged to monkey-patch methods,
+for example to perform extra work before/after creating or destroying
+stacks.
 
 ## Installation
 
-Add this line to your application's Gemfile:
+We like to keep all infrastructure code in a subdirectory `ops` of
+application repos, but you can use any layout you like.
+
+Example directory structure:
+
+```
+ops/
+├── Gemfile
+├── Staxfile
+├── cf/
+├── lib/
+```
+
+Add this line to your `ops/Gemfile`:
 
 ```ruby
 gem 'stax'
@@ -14,27 +73,160 @@ gem 'stax'
 
 And then execute:
 
-    $ bundle
+```
+$ cd ops
+$ bundle
+$ bundle exec stax version
+```
 
-Or install it yourself as:
+## Usage
 
-    $ gem install stax
+Add each of your stacks to `ops/Staxfile`:
 
-## Usage
+```ruby
+stack :vpc
+stack :db
+stack :app
+```
 
-TODO: Write usage instructions here
+Run stax to see it has created subcommands for each of your stacks:
+
+```
+$ bundle exec stax
+Commands:
+  stax app             # app stack
+  stax create          # meta create task
+  stax db              # db stack
+  stax delete          # meta delete task
+  stax help [COMMAND]  # Describe available commands or one specific command
+  stax ls              # list stacks for this branch
+  stax version         # show version
+  stax vpc             # vpc stack
+
+```
+
+with the standard create/update/delete tasks for each:
+
+```
+$ bundle exec stax vpc
+Commands:
+  stax vpc create           # create stack
+  stax vpc delete           # delete stack
+  stax vpc events           # show all events for stack
+  stax vpc exists           # test if stack exists
+  stax vpc generate         # generate cloudformation template
+  stax vpc help [COMMAND]   # Describe subcommands or one specific subcommand
+  stax vpc id [LOGICAL_ID]  # get physical ID from resource logical ID
+  stax vpc outputs          # show stack outputs
+  stax vpc parameters       # show stack input parameters
+  stax vpc protection       # show/set termination protection for stack
+  stax vpc resources        # list resources for this stack
+  stax vpc tail             # tail stack events
+  stax vpc template         # get template of existing stack from cloudformation
+  stax vpc update           # update stack
+```
+
+## Cloudformation templates
+
+Stax will load template files from the path relative to its `Staxfile`
+as `cf/$stack.rb`, e.g. `cf/vpc.rb`. Modify this using the method `Stax::Stack::cfer_template`.
+See `examples` for a typical setup.
+
+Simply control stacks using the relevant subcommands:
+
+```
+$ stax vpc create
+$ stax vpc update
+$ stax vpc delete
+```
+
+By default Stax will name stacks as `$app-$branch-$stack`. For our
+example we will have e.g. `website-master-vpc`, `website-master-db`,
+etc.
+
+To change this scheme modify the methods `Stax::Base::stack_prefix`
+and/or `Stax::Stack::stack_name`.
+
+## Stack parameters
+
+For any given stack, subclass `Stax::Stack` and return define a hash of
+parameters from the method `cfer_parameters`. For example:
+
+```ruby
+module Stax
+  class App < Stack
+    no_commands do
+
+      def cfer_parameters
+        {
+          vpc: stack(:vpc).stack_name,  # how to reference other stacks
+          db:  stack(:db).stack_name,
+          ami: 'ami-e582d29f',
+        }
+      end
+
+    end
+  end
+end
+```
+
+Note, `Stax::Stack` objects are subclassed from
+[Thor](https://github.com/erikhuda/thor), and any non-CLI command
+methods must be defined inside a `no_commands` block. See examples for
+clearer illustration of this.
+
+## Adding and modifying tasks
+
+A strong underlying assumption of Stax is that you will always need
+extra non-cloudformation glue code to handle edge-cases in your
+infrastructure. This is handled by sub-classing and monkey-patching
+`Stax::Stack`.
+
+For example, in our `Stax::App` class:
+
+```ruby
+module Stax
+  class App < Stack
+
+    desc 'create', 'create stack'
+    def create
+      ensure_stack :vpc, :db   # make sure vpc and db stacks are created first
+      super                    # create the stack
+      notify_slack()           # define and call any extra code you need
+    end
+
+    desc 'delete', 'delete stack'
+    def delete
+      super                    # delete the stack
+      cleanup_code()           # do some extra work
+      notify_slack()           # etc
+    end
+
+  end
+end
+```
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bin/setup` to install
+dependencies. You can also run `bin/console` for an interactive prompt
+that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `bundle exec rake
+install`. To release a new version, update the version number in
+`version.rb`, and then run `bundle exec rake release`, which will
+create a git tag for the version, push git commits and tags, and push
+the `.gem` file to [rubygems.org](https://rubygems.org).
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/stax. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
-
+Bug reports and pull requests are welcome on GitHub at
+https://github.com/[USERNAME]/stax. This project is intended to be a
+safe, welcoming space for collaboration, and contributors are expected
+to adhere to the [Contributor
+Covenant](http://contributor-covenant.org) code of conduct.
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
+The gem is available as open source under the terms of the [MIT
+License](http://opensource.org/licenses/MIT).

data/lib/stax.rb

@@ -14,8 +14,10 @@ require 'stax/cfer'
 require 'stax/stack'
 require 'stax/stack/cfn'
 require 'stax/stack/crud'
+require 'stax/stack/changeset'
 require 'stax/stack/parameters'
 require 'stax/stack/outputs'
+require 'stax/stack/imports'
 require 'stax/stack/resources'
 
 require 'stax/mixin/ec2'
@@ -25,6 +27,7 @@ require 'stax/mixin/sg'
 require 'stax/mixin/s3'
 require 'stax/mixin/asg'
 require 'stax/mixin/ecs'
+require 'stax/mixin/ecr'
 require 'stax/mixin/sqs'
 require 'stax/mixin/kms'
 require 'stax/mixin/ssm'
@@ -35,4 +38,6 @@ require 'stax/mixin/lambda'
 require 'stax/mixin/dynamodb'
 require 'stax/mixin/logs'
 require 'stax/mixin/apigw'
-require 'stax/mixin/firehose'
+require 'stax/mixin/firehose'
+require 'stax/mixin/codebuild'
+require 'stax/mixin/codepipeline'

data/lib/stax/aws/asg.rb

@@ -16,6 +16,7 @@ module Stax
 
         def instances(names)
           ids = describe(names).map(&:instances).flatten.map(&:instance_id)
+          return [] if ids.empty? # below call will return all instances in a/c if this empty
           paginate(:auto_scaling_instances) do |token|
             client.describe_auto_scaling_instances(instance_ids: ids, next_token: token)
           end

data/lib/stax/aws/cfn.rb

@@ -13,6 +13,7 @@ module Stax
       ]
 
       COLORS = {
+        ## stack status
         CREATE_COMPLETE:      :green,
         DELETE_COMPLETE:      :green,
         UPDATE_COMPLETE:      :green,
@@ -21,6 +22,10 @@ module Stax
         UPDATE_FAILED:        :red,
         ROLLBACK_IN_PROGRESS: :red,
         ROLLBACK_COMPLETE:    :red,
+        ## resource action
+        Add:    :green,
+        Modify: :yellow,
+        Remove: :red,
       }
 
       class << self
@@ -55,8 +60,6 @@ module Stax
           paginate(:stack_events) do |token|
             client.describe_stack_events(stack_name: name, next_token: token)
           end
-        rescue ::Aws::CloudFormation::Errors::ValidationError => e
-          puts e.message
         end
 
         def id(name, id)
@@ -72,7 +75,7 @@ module Stax
         end
 
         def exists?(name)
-          Cfn.describe(name) && true
+          Aws::Cfn.describe(name) && true
         rescue ::Aws::CloudFormation::Errors::ValidationError
           false
         end
@@ -87,14 +90,72 @@ module Stax
           outputs(name)[key]
         end
 
+        ## list of this stack output exports
+        def exports(name)
+          describe(name).outputs.select(&:export_name)
+        end
+
+        ## list of stacks that import from this one
+        def imports(name)
+          paginate(:imports) do |next_token|
+            client.list_imports(export_name: name, next_token: next_token)
+          end
+        rescue ::Aws::CloudFormation::Errors::ValidationError
+          []
+        end
+
+        def validate(opt)
+          client.validate_template(opt)
+        end
+
+        def create(opt)
+          client.create_stack(opt)&.stack_id
+        end
+
+        def update(opt)
+          client.update_stack(opt)&.stack_id
+        end
+
         def delete(name)
           client.delete_stack(stack_name: name)
         end
 
+        def cancel(name)
+          client.cancel_update_stack(stack_name: name)
+        end
+
         def protection(name, enable)
           client.update_termination_protection(stack_name: name, enable_termination_protection: enable)
         end
 
+        def get_policy(opt)
+          client.get_stack_policy(opt).stack_policy_body
+        end
+
+        def set_policy(opt)
+          client.set_stack_policy(opt)
+        end
+
+        def list_change_sets(name)
+          paginate(:summaries) do |next_token|
+            client.list_change_sets(stack_name: name, next_token: next_token)
+          end
+        end
+
+        def changes(opt)
+          paginate(:changes) do |next_token|
+            client.describe_change_set(opt.merge(next_token: next_token))
+          end
+        end
+
+        def changeset(opt)
+          client.create_change_set(opt)
+        end
+
+        def execute(opt)
+          client.execute_change_set(opt)
+        end
+
       end
 
     end

data/lib/stax/aws/codebuild.rb

@@ -0,0 +1,41 @@
+module Stax
+  module Aws
+    class Codebuild < Sdk
+
+      class << self
+
+        def client
+          @_client ||= ::Aws::CodeBuild::Client.new
+        end
+
+        def projects(names)
+          client.batch_get_projects(names: names).projects
+        end
+
+        ## returns ids of num most recent builds for project
+        def builds_for_project(name, num = 100)
+          count = 0
+          next_token = nil
+          builds = []
+          loop do
+            r = client.list_builds_for_project(project_name: name, next_token: next_token)
+            builds += r.ids
+            break if (count += r.ids.count) >= num
+            break if (next_token = r.next_token).nil?
+          end
+          builds.first(num)
+        end
+
+        def builds(ids)
+          client.batch_get_builds(ids: ids).builds
+        end
+
+        def start(opt)
+          client.start_build(opt).build
+        end
+
+      end
+
+    end
+  end
+end

data/lib/stax/aws/codepipeline.rb

@@ -0,0 +1,44 @@
+module Stax
+  module Aws
+    class Codepipeline < Sdk
+
+      class << self
+
+        def client
+          @_client ||= ::Aws::CodePipeline::Client.new
+        end
+
+        def stages(name)
+          client.get_pipeline(name: name).pipeline.stages
+        end
+
+        def executions(name, num = nil)
+          opt = {pipeline_name: name, max_results: num}
+          token = nil
+          summaries = []
+          loop do
+            s = client.list_pipeline_executions(opt.merge(next_token: token))
+            summaries += s.pipeline_execution_summaries
+            break if (token = s.next_token).nil?
+            break if summaries.count >= num
+          end
+          summaries.first(num)
+        end
+
+        def execution(name, id)
+          client.get_pipeline_execution(pipeline_name: name, pipeline_execution_id: id).pipeline_execution
+        end
+
+        def state(name)
+          client.get_pipeline_state(name: name)
+        end
+
+        def start(name)
+          client.start_pipeline_execution(name: name).pipeline_execution_id
+        end
+
+      end
+
+    end
+  end
+end