outliers 0.0.0 → 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 96ee1fb7ec544e936e6a55b6c0d4bac0b8a9baea
+  data.tar.gz: 098ddc59e2cd429d938785b42336b592823e38a1
+SHA512:
+  metadata.gz: 13c134dbeb4901b3016502c188cbfacf41aef8c8f11db1f1d8a85d8f81b08937759fed1c257039b5259130b46756d66e7b19174d742f43acbfe2ecaf2a28ca2b
+  data.tar.gz: 49dc924d5ea0694f3e8c4e98abae6ea93f327bf2c828309a375e4269430f14117b7169fc4e5dba91562fa8c81bc93e920917d2ab5bfdb73cea5ef63f6ffdb1ed

data/.ruby-gemset

@@ -0,0 +1 @@
+outliers

data/.ruby-version

@@ -0,0 +1 @@
+2.0.0

data/.travis.yml

@@ -0,0 +1,5 @@
+rvm:
+  - 1.9.2
+  - 1.9.3
+  - 2.0.0
+script: "rake spec"

data/CHANGELOG.md

@@ -0,0 +1,3 @@
+## 0.0.1
+
+* Initial release

data/LICENSE.txt

@@ -1,22 +1,13 @@
-Copyright (c) 2013 Brett Weaver
+Copyright 2013 Brett Weaver
 
-MIT License
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
 
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-"Software"), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
+   http://www.apache.org/licenses/LICENSE-2.0
 
-The above copyright notice and this permission notice shall be
-included in all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
-LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
-OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
-WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/README.md

@@ -1,24 +1,191 @@
 # Outliers
 
-TODO: Write a gem description
+Outliers is a framework for verifying configuration of resources.
+
+## Overview
+
+* Applications and teams rely on multiple service providers (AWS, Github, etc).
+* Providers deliver like resources with complex configuration (EC2 Instances, S3 Buckets, etc).
+* Resource configuration can be verified (launched from given AMI, contain private objects, etc).
+* Those not passing verifications, are flagged as Outliers.
 
 ## Installation
 
-Add this line to your application's Gemfile:
+Install the gem:
+
+    gem install outliers
+
+## Setup
 
-    gem 'outliers'
+Create **~/outliers.yml** with a list of credentials in the following format:
 
-And then execute:
+    credential_name:
+      key1: value1
+      key2: value2
 
-    $ bundle
+Multiple accounts can be specified, to add a prod and preprod AWS account:
 
-Or install it yourself as:
+    aws_pre_prod:
+      region: us-east-1
+      access_key_id: YYY
+      secret_access_key: XXX
 
-    $ gem install outliers
+    aws_prod:
+      region: us-east-1
+      access_key_id: AAA
+      secret_access_key: BBB
+
+Depending on the provider, different keys and values are required. For a list of providers:
+
+    outliers providers
 
 ## Usage
 
-TODO: Write usage instructions here
+Outliers can be used in two modes, as a **CLI** or a **DSL**.
+
+The CLI is good for testing and quick verifications.
+
+The DSL can be used to build up comprehensive list of verifications for a project or company.
+
+### CLI
+
+To verify all EC2 instances are in a VPC:
+
+    outliers evaluate -c aws_prod -p aws_ec2 -r instance -v vpc
+
+Credential keys can be overriden. To specify region us-west-1.
+
+    outliers evaluate -c aws_prod -p aws_ec2 -r instance -v vpc -c region=us-west-1
+
+Verifications may require arguments. To verify your RDS databases have a 2 day retention period:
+
+    outliers evaluate -c aws_prod -p aws_rds -r db_instance -v backup_retention_period -a days=2
+
+Verifications may accept multiple values for an argument. Values are separated by commas.
+
+To verify EC2 instances are launched from a list of known images:
+
+    outliers evaluate -c aws_prod -p aws_ec2 -r instance -v valid_image_id -a image_ids=ami-12345678,ami-87654321
+
+To only target a specific resource:
+
+    outliers evaluate -c aws_prod -p aws_ec2 -r instance -t i-87654321
+
+To exclude resources that are known exceptions:
+
+    outliers evaluate -c aws_prod -p aws_ec2 -r instance -e i-12345678
+
+### DSL
+
+To run Outliers as a DSL
+
+* Create a directory to store your evaluations. 
+* Evalutions are read from from files within the directory.
+* All files ending in **.rb** will be processed.
+* Each file can have one or more evaluation blocks. 
+
+To process a directory:
+
+    outliers process -d /home/user/outliers
+
+To verify all instances are in a VPC, create the file **ec2.rb** and add the following block:
+
+    evaluate do |e|
+      e.connect 'aws_prod', provider: 'aws_ec2'
+      e.resources 'instance'
+      e.verify 'vpc'
+    end
+
+Files can have multiple evaluations, to add a validation that overrides the region:
+
+    evaluate do |e|
+      e.connect 'aws_prod', provider: 'aws_ec2'
+      e.resources 'instance'
+      e.verify 'vpc'
+    end
+
+    evaluate do |e|
+      e.connect 'aws_prod', provider: 'aws_ec2', region: 'us-west-1'
+      e.resources 'instance'
+      e.verify 'vpc'
+    end
+
+The DSL supports any valid Ruby code. To iterate over multiple regions:
+
+    ['us-west-1', 'us-west-2', 'us-east-1'].each do |region|
+      evaluate do |e|
+        e.connect 'aws_prod', provider: 'aws_ec2', region: region
+        e.resources 'instance'
+        e.verify 'vpc'
+      end
+    end
+
+Evaluations can be given names to help identify Outliers in results.
+
+    evaluate "validate_database_retention_period" do |e|
+      e.connect 'aws_prod', provider: 'aws_rds', region: 'us-west-1'
+      e.resources 'db_instance'
+      e.verify 'backup_retention_period', days: 2
+    end
+
+To pass arguments to a verification:
+
+    evaluate do |e|
+      e.connect 'aws_prod', provider: 'aws_rds', region: 'us-west-1'
+      e.resources 'db_instance'
+      e.verify 'backup_retention_period', days: 2
+    end
+
+To pass multiple arguments, specify them as an array:
+
+    evaluate do |e|
+      e.connect 'aws_prod', provider: 'aws_ec2', region: 'us-west-1'
+      e.resources 'instance'
+      e.verify 'valid_image_id', image_ids: ['ami-12345678','ami-87654321']
+    end
+
+To only target a specific resource:
+
+    evaluate do |e|
+      e.connect 'aws_prod', provider: 'aws_ec2', region: 'us-west-1'
+      e.resources 'instance', 'i-12345678'
+      e.verify 'valid_image_id', image_ids: ['ami-12345678','ami-87654321']
+    end
+
+To target multiple resources, you can pass an array:
+
+    evaluate do |e|
+      e.connect 'aws_prod', provider: 'aws_ec2', region: 'us-west-1'
+      e.resources 'instance', ['i-12345678', 'i-abcdef12']
+      e.verify 'valid_image_id', image_ids: ['ami-12345678','ami-87654321']
+    end
+
+Sometimes you want to exclude resources that are known exceptions, to exclude an instance from the VPC validation:
+
+    evaluate do |e|
+      e.connect 'aws_prod', provider: 'aws_ec2', region: 'us-west-1'
+      e.resources 'instance'
+      e.exclude 'i-12345678'
+      e.verify 'valid_image_id', image_ids: ['ami-12345678','ami-87654321']
+    end
+
+### Help
+
+For a list of providers and required credentials:
+
+    outliers providers
+
+For a list of resources, and available verifications, for a given provider:
+
+    outliers resources -p PROVIDER_NAME
+
+For a fule list of commands run:
+
+    outliers -h
+
+For full help on a command, append -h:
+
+    outliers evaluate -h
 
 ## Contributing
 

data/Rakefile

@@ -1 +1,8 @@
 require "bundler/gem_tasks"
+
+require "rspec/core/rake_task"
+
+desc 'Run specs'
+RSpec::Core::RakeTask.new do |t|
+  t.rspec_opts = %w(--color)
+end

data/bin/outliers

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+
+require 'outliers'
+require 'outliers/cli'
+
+Outliers::CLI.start

data/lib/outliers/cli/evaluate.rb

@@ -0,0 +1,115 @@
+module Outliers
+  module CLI
+    class Evaluate
+      def evaluate
+        @options = { arguments: [], exclude: [], credentials: [], target_resources: [] }
+        @credentials = {}
+
+        option_parser.parse!
+
+        Outliers.config_path @options[:config]
+
+        @logger = Outliers.logger
+        @run    = Run.new
+
+        load_credentials
+
+        begin
+          @run.evaluate "Validationf from CLI." do |e|
+            e.connect 'cli'
+            e.resources @options[:resources], target_resources
+            e.exclude @options[:exclude] if @options[:exclude].any?
+            e.verify @options[:verification], arguments
+          end
+        rescue Outliers::Exceptions::Base => e
+          @logger.error e.message
+          exit 1
+        end
+
+        exit 1 if @run.results.first.failed?
+      end
+
+      def command_name
+        'evaluate'
+      end
+
+      def command_summary
+        'Evaluate the given verification.'
+      end
+
+      private
+
+      def load_credentials
+        credentials_name = @options[:credential_name]
+        credentials = Credentials.load_from_file("#{ENV['HOME']}/.outliers.yml").fetch credentials_name
+        credentials.merge! 'provider' => @options[:provider]
+
+        if @options[:credentials].any?
+          @options[:credentials].each do |c|
+            key = c.split('=').first
+            value =  c.split('=').last
+            credentials.merge! key => value
+          end
+        end
+
+        @run.credentials = { 'cli' => credentials }
+      end
+
+      def target_resources
+        @options[:target_resources]
+      end
+      
+      def arguments
+        arguments = {}
+
+        @options[:arguments].each do |a|
+          key = a.split('=').first
+          value = a.split('=').last
+          value = value.split(',') if value.include?(',')
+          arguments.merge! key => value
+        end
+
+        arguments
+      end
+
+      def option_parser
+        OptionParser.new do |opts|
+          opts.banner = "Usage: outliers evaluate [options]"
+
+          opts.on("-a", "--argument [NAME]", "Equals seperated key and value to pass as argument to verification (can be specified multiple times).") do |o|
+            @options[:arguments] << o
+          end
+
+          opts.on("-c", "--credential_name [CREDENTIAL_NAME]", "Name to load from credentials file.") do |o|
+            @options[:credential_name] = o
+          end
+
+          opts.on("-e", "--exclude [EXCLUDE]", "Exclude resources in collection with the given key (can be specified multiple times).") do |o|
+            @options[:exclude] << o
+          end
+
+          opts.on("-p", "--provider [PROVIDER]", "Provider of target resources.") do |o|
+            @options[:provider] = o
+          end
+
+          opts.on("-r", "--resources [RESOURCES]", "Name of resources collection to evaluate.") do |o|
+            @options[:resources] = o
+          end
+
+          opts.on("-t", "--target_resources [TARGET_RESOURCES]", "Target resources with key name (can be specified more than once).") do |o|
+            @options[:target_resources] << o
+          end
+
+          opts.on("-v", "--verification [VERIFICATION]", "Verification to perform against collection of resources.") do |o|
+            @options[:verification] = o
+          end
+
+          opts.on("--credentials [CREDENTIALS]", "Equals seperated key and value to include as credentials (can be specified multiple times).") do |o|
+            @options[:credentials] << o
+          end
+        end
+      end
+
+    end
+  end
+end

data/lib/outliers/cli/process.rb

@@ -0,0 +1,56 @@
+module Outliers
+  module CLI
+    class Process
+      def process
+        @options = {}
+
+        option_parser.parse!
+
+        Outliers.config_path @options[:directory]
+
+        @logger = Outliers.logger
+        @run    = Run.new
+
+        begin
+          @run.credentials = Credentials.load_from_file "#{ENV['HOME']}/.outliers.yml"
+          @run.process_evaluations_in_config_folder
+        rescue Outliers::Exceptions::Base => e
+          @logger.error e.message
+          exit 1
+        end
+
+        passed = @run.passed.count
+        failed = @run.failed.count
+
+        @run.failed.each do |f|
+          @logger.info "Evaluation '#{f.description}' failed."
+        end
+
+        @logger.info "(#{failed} evaluations failed, #{passed} evaluations passed.)"
+
+        exit 1 unless failed.zero?
+      end
+
+      def command_name
+        'process'
+      end
+
+      def command_summary
+        'Process evaluations in config folder.'
+      end
+
+      private
+
+      def option_parser
+        OptionParser.new do |opts|
+          opts.banner = "Usage: outliers process [options]"
+
+          opts.on("-d", "--directory [DIRECTORY]", "Directory containing evaluations to load.") do |o|
+            @options[:directory] = o
+          end
+        end
+      end
+
+    end
+  end
+end

data/lib/outliers/cli/providers.rb

@@ -0,0 +1,29 @@
+module Outliers
+  module CLI
+    class Providers
+      def providers
+        option_parser.parse!
+        list = Outliers::Providers.name_map
+        list.each_pair do |k,v|
+          puts k
+          v.credential_arguments.each_pair { |k,v| puts "  #{k}: #{v}" }
+        end
+      end
+
+      def command_name
+        'providers'
+      end
+
+      def command_summary
+        'List available providers.'
+      end
+
+      def option_parser
+        OptionParser.new do |opts|
+          opts.banner = "Usage: outliers providers"
+        end
+      end
+
+    end
+  end
+end

data/lib/outliers/cli/resources.rb

@@ -0,0 +1,57 @@
+module Outliers
+  module CLI
+    class Resources
+      def resources
+        @options = {}
+
+        option_parser.parse!
+
+        provider = @options[:provider]
+
+        @logger = Outliers.logger
+
+        unless provider
+          @logger.error "Required parameter 'provider' not specified."
+          exit 1
+        end
+
+        all = Outliers::Resources.collections
+
+        list = all.select { |r| r.to_human =~ /^#{provider}_.*$/ }
+
+        if list.any?
+          list.each do |r|
+            name = r.to_human
+            name.slice! provider
+            name[0] = ''
+            puts name
+            r.verifications.each { |v| puts "  #{v[:name]}(#{v[:args]}) #{v[:description]}" }
+          end
+        else
+          puts "No resources found for '#{provider}'."
+        end
+      end
+
+      def command_name
+        'resources'
+      end
+
+      def command_summary
+        'List available resources for a provider.'
+      end
+
+      private
+
+      def option_parser
+        OptionParser.new do |opts|
+          opts.banner = "Usage: outliers resources [options]"
+
+          opts.on("-p", "--provider [PROVIDER]", "Provider to list resources.") do |o|
+            @options[:provider] = o
+          end
+        end
+      end
+
+    end
+  end
+end

data/lib/outliers/cli.rb

@@ -0,0 +1,78 @@
+require 'optparse'
+
+require 'outliers/cli/evaluate'
+require 'outliers/cli/process'
+require 'outliers/cli/providers'
+require 'outliers/cli/resources'
+
+module Outliers
+  module CLI
+
+    def self.start
+      cmd = ARGV.shift
+
+      case cmd
+      when 'evaluate'
+        begin
+          CLI::Evaluate.new.evaluate
+        rescue OptionParser::MissingArgument => e
+          puts e.message
+          exit 1
+        end
+      when 'providers'
+        begin
+          CLI::Providers.new.providers
+        rescue OptionParser::MissingArgument => e
+          puts e.message
+          exit 1
+        end
+      when 'process'
+        begin
+          CLI::Process.new.process
+        rescue OptionParser::MissingArgument => e
+          puts e.message
+          exit 1
+        end
+      when 'resources'
+        begin
+          CLI::Resources.new.resources
+        rescue OptionParser::MissingArgument => e
+          puts e.message
+          exit 1
+        end
+      when '-v'
+        puts OUTLIERS::VERSION
+      else
+        puts "Unknown command: '#{cmd}'."
+        puts ''
+        usage
+        exit 1
+      end
+    end
+
+    def self.usage
+      puts 'Usage: outliers command'
+      puts ''
+      puts 'Append -h for help on specific subcommand.'
+      puts ''
+
+      puts 'Commands:'
+      commands.each do |cmd|
+        $stdout.printf "    %-#{length_of_longest_command}s      %s\n",
+                       cmd.command_name,
+                       cmd.command_summary
+      end
+    end
+
+    def self.commands
+      return @commands if @commands
+      klasses   = Outliers::CLI.constants
+      @commands = klasses.map { |klass| Outliers::CLI.const_get(klass).new }
+    end
+
+    def self.length_of_longest_command
+      commands.map { |c| c.command_name.length }.max
+    end
+
+  end
+end