outliers 0.1.1 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 370c7538e31dc0846ae0de97c21b9eadc6cafe22
4
- data.tar.gz: ecb6077e1e2451e3f7d19b58ab0d5ebedf673178
3
+ metadata.gz: ca2a95007c8c6b9d85847b7c857e8e5f54e72859
4
+ data.tar.gz: 273c267dc763814b17162cf394cc453e77ba2253
5
5
  SHA512:
6
- metadata.gz: 5ef91692770636f84be52db5fa1989d7d9d3c54bda9fa206b60e63fc1b7186cfc3959c02287af1ae8b575ba6fe8a7f191d9d866bd43c4c15a394174ca485dacd
7
- data.tar.gz: 8ab846d7b565ccaf4dfb4afaf377971e058dbc99f5d1fda119c57fe9a1d53a46dd7ca95d60eaa16acfe19ab2db18540b489ab8f1f8b1d8c73bd5edafeef28640
6
+ metadata.gz: efbb5c48979eafc1001695502ad892ae799517497b789e6b56d034b44e0e3cdf74fa40c950cf77e9fd329cd0afbe1fb4ab612497c6b0a33ab7d00e95c1b75cba
7
+ data.tar.gz: a2a0da57894add6de86c37a749d172a8174476b12f1d8665ebe68df9b11c31271bab95de1c1171f401cc4f8a2619e9726ad426f62b66914ec0e1b280f8aa74e4
data/CHANGELOG.md CHANGED
@@ -1,3 +1,10 @@
1
+ ## 0.2.0
2
+
3
+ * Added filters
4
+ * Added multiple verificaiton support
5
+ * Remove all but AWS resources
6
+ * Multiple bug fixes
7
+
1
8
  ## 0.1.1
2
9
 
3
10
  * Bug fix in evaluate CLI.
data/README.md CHANGED
@@ -6,11 +6,17 @@ Outliers is a framework for verifying configuration of resources.
6
6
 
7
7
  ## Overview
8
8
 
9
- * Applications and teams rely on multiple service providers (AWS, Github, etc).
9
+ * Applications and teams rely on multiple service providers (AWS, etc).
10
10
  * Providers deliver like resources with complex configuration (EC2 Instances, S3 Buckets, etc).
11
11
  * Resource configuration can be verified (launched from given AMI, contain private objects, etc).
12
+ * The resources can be included or excluded by their ID (Instance ID, Object Key, etc).
13
+ * Resources can be included in the list by matching a filter (Instance has tag 'x' with value 'y').
12
14
  * Those not passing verifications, are flagged as Outliers.
13
15
 
16
+ ## Requirements
17
+
18
+ * Ruby 1.9.3 or greater.
19
+
14
20
  ## Installation
15
21
 
16
22
  Install the gem:
@@ -19,6 +25,8 @@ Install the gem:
19
25
 
20
26
  ## Setup
21
27
 
28
+ **Currently Outliers only supports AWS**
29
+
22
30
  Create **~/outliers.yml** with a list of credentials in the following format:
23
31
 
24
32
  credential_name:
@@ -37,7 +45,9 @@ Multiple accounts can be specified, to add a prod and preprod AWS account:
37
45
  access_key_id: AAA
38
46
  secret_access_key: BBB
39
47
 
40
- Depending on the provider, different keys and values are required. For a list of providers:
48
+ Depending on the provider, different keys and values are required.
49
+
50
+ For a list of providers:
41
51
 
42
52
  outliers providers
43
53
 
@@ -77,6 +87,10 @@ To exclude resources that are known exceptions:
77
87
 
78
88
  outliers evaluate -c aws_prod -p aws_ec2 -r instance -e i-12345678
79
89
 
90
+ Resources have attributes which can be used to filter target resources. To filter instances who have tag 'Name' equal to 'web'.
91
+
92
+ outliers evaluate -c aws_prod -p aws_ec2 -r instance -f 'tag=Name:web'
93
+
80
94
  ### DSL
81
95
 
82
96
  To run Outliers as a DSL
@@ -122,6 +136,16 @@ The DSL supports any valid Ruby code. To iterate over multiple regions:
122
136
  end
123
137
  end
124
138
 
139
+ Evaluations can run multiple verifications. To validate instances are in a VPC, running and using a valid image:
140
+
141
+ evaluate do
142
+ connect 'aws_prod', provider: 'aws_ec2', region: 'us-west-1'
143
+ resources 'instance'
144
+ verify 'vpc'
145
+ verify 'running'
146
+ verify 'valid_image_id', image_ids: ['ami-12345678','ami-87654321']
147
+ end
148
+
125
149
  Evaluations can be given names to help identify Outliers in results.
126
150
 
127
151
  evaluate "validate_database_retention_period" do
@@ -171,6 +195,15 @@ Sometimes you want to exclude resources that are known exceptions, to exclude an
171
195
  verify 'valid_image_id', image_ids: ['ami-12345678','ami-87654321']
172
196
  end
173
197
 
198
+ Resources have attributes which can be used to filter target resources. To filter instances who have tag 'Name' equal to 'web'.
199
+
200
+ evaluate do
201
+ connect 'aws_prod', provider: 'aws_ec2', region: 'us-west-1'
202
+ resources 'instance'
203
+ filter tag: 'Name:web'
204
+ verify 'valid_image_id', image_ids: ['ami-12345678','ami-87654321']
205
+ end
206
+
174
207
  ### Help
175
208
 
176
209
  For a list of providers and required credentials:
@@ -2,7 +2,7 @@ module Outliers
2
2
  module CLI
3
3
  class Evaluate
4
4
  def evaluate
5
- @options = { arguments: [], exclude: [], credentials: [], target_resources: [] }
5
+ @options = { arguments: [], exclude: [], filter: [], credentials: [], target_resources: [] }
6
6
  @credentials = {}
7
7
 
8
8
  option_parser.parse!
@@ -15,6 +15,7 @@ module Outliers
15
15
  load_credentials
16
16
 
17
17
  @options[:parsed_arguments] = parse_arguments
18
+ @options[:parsed_filters] = parse_filters
18
19
 
19
20
  # Make options available global
20
21
  # Required to read by instance_eval in @run.evaluate
@@ -25,6 +26,7 @@ module Outliers
25
26
  connect 'cli'
26
27
  resources @@options[:resource], @@options[:target_resources]
27
28
  exclude @@options[:exclude] if @@options[:exclude].any?
29
+ filter @@options[:parsed_filters] if @@options[:filter].any?
28
30
  verify @@options[:verification], @@options[:parsed_arguments]
29
31
  end
30
32
  rescue Outliers::Exceptions::Base => e
@@ -61,6 +63,19 @@ module Outliers
61
63
  @run.credentials = { 'cli' => credentials }
62
64
  end
63
65
 
66
+ def parse_filters
67
+ filters = {}
68
+
69
+ @options[:filter].each do |a|
70
+ key = a.split('=').first
71
+ value = a.split('=').last
72
+ filters.merge! key => value
73
+ end
74
+
75
+ filters
76
+ end
77
+
78
+
64
79
  def parse_arguments
65
80
  arguments = {}
66
81
 
@@ -90,6 +105,10 @@ module Outliers
90
105
  @options[:exclude] << o
91
106
  end
92
107
 
108
+ opts.on("-f", "--fitler [FILTER]", "Equals seperated filter name and value (can be specified multiple times).") do |o|
109
+ @options[:filter] << o
110
+ end
111
+
93
112
  opts.on("-p", "--provider [PROVIDER]", "Provider of target resources.") do |o|
94
113
  @options[:provider] = o
95
114
  end
@@ -22,8 +22,11 @@ module Outliers
22
22
  passed = @run.passed.count
23
23
  failed = @run.failed.count
24
24
 
25
+ @logger.info "Evaluations completed."
26
+
25
27
  @run.failed.each do |f|
26
- @logger.info "Evaluation '#{f.description}' failed."
28
+ @logger.info "Evaluation '#{f.evaluation}' verification '#{f.verification}' of '#{f.resource}' failed."
29
+ @logger.debug "Failing resource IDs '#{f.failing_resources.map{|r| r.id}.join(', ')}'"
27
30
  end
28
31
 
29
32
  @logger.info "(#{failed} evaluations failed, #{passed} evaluations passed.)"
@@ -25,7 +25,10 @@ module Outliers
25
25
  name.slice! provider
26
26
  name[0] = ''
27
27
  puts name
28
- r.verifications.each { |v| puts " #{v[:name]}(#{v[:args]}) #{v[:description]}" }
28
+ puts " Verifications:"
29
+ r.verifications.each { |v| puts " #{v[:name]}(#{v[:args]}) #{v[:description]}" }
30
+ puts " Filters:"
31
+ r.filters.each { |v| puts " #{v[:name]}(#{v[:args]}) #{v[:description]}" }
29
32
  end
30
33
  else
31
34
  puts "No resources found for '#{provider}'."
@@ -17,11 +17,19 @@ module Outliers
17
17
  Outliers::Verifications::Shared.verifications + self.resource_class.verifications
18
18
  end
19
19
 
20
+ def self.filters
21
+ []
22
+ end
23
+
20
24
  def self.resource_class
21
25
  array = self.to_s.gsub(/Collection$/, '').split('::')
22
26
  array.inject(Object) {|o,c| o.const_get c}
23
27
  end
24
28
 
29
+ def to_s
30
+ self.class.to_human
31
+ end
32
+
25
33
  def initialize(provider)
26
34
  @targets = []
27
35
  @provider = provider
@@ -40,13 +48,39 @@ module Outliers
40
48
  @all = save
41
49
  end
42
50
 
51
+ def filter(args)
52
+ name = args.keys.first
53
+ value = args.fetch name
54
+
55
+ logger.info "Applying filter '#{name}' with value '#{value}'."
56
+
57
+ unless self.public_methods.include? "filter_#{name}".to_sym
58
+ raise Exceptions::UnknownFilter.new "Unknown filter '#{name}'."
59
+ end
60
+
61
+ filtered_list = self.public_send "filter_#{name}", value
62
+
63
+ logger.warn "No resources match filter." unless filtered_list.any?
64
+
65
+ @all = filtered_list
66
+ end
67
+
43
68
  def verify(name, arguments={})
44
69
  name << "?" unless name =~ /^.*\?$/
45
70
 
46
- logger.debug "Verifying resources '#{all_by_key.join(', ')}'."
71
+ unless all.any?
72
+ return { failing_resources: [], passing_resources: [] }
73
+ end
74
+
75
+ logger.info "Verifying '#{name}'."
76
+ logger.debug "Target resources '#{all_by_key.join(', ')}'."
77
+
78
+ unless verification_exists? name
79
+ raise Exceptions::UnknownVerification.new "Unkown verification '#{name}'."
80
+ end
47
81
 
48
82
  if collection_verification? name
49
- send_verification self, name, arguments
83
+ send_collection_verification name, arguments
50
84
  else
51
85
  send_resources_verification name, arguments
52
86
  end
@@ -66,6 +100,13 @@ module Outliers
66
100
 
67
101
  private
68
102
 
103
+ def verification_exists?(name)
104
+ m = resource_class.instance_methods - resource_class.class.instance_methods
105
+ m += Outliers::Verifications::Shared.instance_methods
106
+ m -= [:source, :id, :method_missing]
107
+ m.include? name.to_sym
108
+ end
109
+
69
110
  def all_by_key
70
111
  all.map {|r| r.public_send key}
71
112
  end
@@ -82,17 +123,6 @@ module Outliers
82
123
  self.public_methods.include? name.to_sym
83
124
  end
84
125
 
85
- def send_resources_verification(verification, arguments)
86
- set_target_resources verification if targets.any?
87
-
88
- results = map do |resource|
89
- result = send_verification resource, verification, arguments
90
- logger.debug "Verification of resource '#{resource.id}' #{result ? 'passed' : 'failed'}."
91
- result
92
- end
93
- !results.include? false
94
- end
95
-
96
126
  def set_target_resources(verification)
97
127
  logger.info "Verifying target '#{targets.join(', ')}'."
98
128
 
@@ -105,12 +135,27 @@ module Outliers
105
135
  @all
106
136
  end
107
137
 
138
+ def send_resources_verification(verification, arguments)
139
+ set_target_resources verification if targets.any?
140
+
141
+ failing_resources = reject do |resource|
142
+ result = send_verification resource, verification, arguments
143
+ logger.debug "Verification of resource '#{resource.id}' #{result ? 'passed' : 'failed'}."
144
+ result
145
+ end
146
+ { failing_resources: failing_resources, passing_resources: all - failing_resources }
147
+ end
148
+
149
+ def send_collection_verification(verification, arguments)
150
+ failing_resources = send_verification(self, verification, arguments)
151
+ { failing_resources: failing_resources, passing_resources: all - failing_resources }
152
+ end
153
+
108
154
  def send_verification(object, verification, arguments)
109
155
  if object.method(verification).arity.zero?
110
156
  if arguments.any?
111
157
  raise Outliers::Exceptions::NoArgumentRequired.new "Verification '#{verification}' does not require an arguments."
112
158
  end
113
-
114
159
  object.public_send verification
115
160
  else
116
161
  if arguments.none?
@@ -26,7 +26,7 @@ module Outliers
26
26
  targets_array = Array(targets)
27
27
 
28
28
  if targets_array.any?
29
- logger.info "Verifying against '#{targets_array.join(', ')}' from '#{name}' collection."
29
+ logger.info "Verifying '#{targets_array.join(', ')}' from '#{name}' collection."
30
30
  collection.targets = targets_array
31
31
  end
32
32
  collection
@@ -36,14 +36,22 @@ module Outliers
36
36
  collection.exclude_by_key Array(exclusions)
37
37
  end
38
38
 
39
+ def filter(args)
40
+ collection.filter args.keys_to_s
41
+ end
42
+
39
43
  def verify(verification, arguments={})
40
- collection.load_all
44
+ @resources_loaded ||= collection.load_all
41
45
 
42
- r = collection.verify verification, arguments.keys_to_sym
46
+ verification_result = collection.verify verification, arguments.keys_to_sym
43
47
 
44
- result = Outliers::Result.new :description => @name, :passed => r
48
+ result = Outliers::Result.new evaluation: @name,
49
+ failing_resources: verification_result.fetch(:failing_resources),
50
+ passing_resources: verification_result.fetch(:passing_resources),
51
+ resource: @collection,
52
+ verification: verification
45
53
 
46
- logger.info "Evaluation '#{result}'."
54
+ logger.info "Verification '#{verification}' #{result}."
47
55
 
48
56
  @run.results << result
49
57
  end
@@ -12,6 +12,9 @@ module Outliers
12
12
  class ArgumentRequired < Base
13
13
  end
14
14
 
15
+ class InvalidBucket < Base
16
+ end
17
+
15
18
  class NoArgumentRequired < Base
16
19
  end
17
20
 
@@ -24,10 +27,10 @@ module Outliers
24
27
  class UnknownVerification < Base
25
28
  end
26
29
 
27
- class UnknownProvider < Base
30
+ class UnknownFilter < Base
28
31
  end
29
32
 
30
- class InvalidBucket < Base
33
+ class UnknownProvider < Base
31
34
  end
32
35
 
33
36
  class UnsupportedRegion < Base
@@ -0,0 +1,38 @@
1
+ module Outliers
2
+ module Filters
3
+ module Aws
4
+ module Ec2
5
+ module Tags
6
+
7
+ def filter_tag(value)
8
+ tag_name = value.split(':').first
9
+ tag_value = value.split(':').last
10
+ logger.info "Filtering by tag '#{tag_name}' equals '#{tag_value}'."
11
+ all.select do |r|
12
+ if r.tags.has_key? tag_name
13
+ value = r.tags[tag_name]
14
+ result = value == tag_value
15
+ logger.debug "'#{r.id}' has tag with value '#{value}'. #{result ? 'Matches' : 'Does not match'} filter."
16
+ result
17
+ else
18
+ logger.debug "'#{r.id}' does not have tag '#{tag_name}'"
19
+ false
20
+ end
21
+ end
22
+ end
23
+
24
+ module_function
25
+
26
+ def self.filters
27
+ [
28
+ { name: 'tag',
29
+ description: 'Filter instances tagged with the given tag name and value.',
30
+ args: 'TAG_NAME:VALUE"' }
31
+ ]
32
+ end
33
+
34
+ end
35
+ end
36
+ end
37
+ end
38
+ end
@@ -0,0 +1 @@
1
+ require 'outliers/filters/aws/ec2/tags'
@@ -0,0 +1 @@
1
+ require 'outliers/filters/aws/ec2'
@@ -0,0 +1 @@
1
+ require 'outliers/filters/aws'
@@ -1,5 +1,4 @@
1
1
  require 'outliers/providers/aws'
2
- require 'outliers/providers/github'
3
2
 
4
3
  module Outliers
5
4
  module Providers
@@ -8,7 +8,7 @@ module Outliers
8
8
  end
9
9
 
10
10
  def self.verifications
11
- {}
11
+ []
12
12
  end
13
13
 
14
14
  def initialize(source)
@@ -0,0 +1,13 @@
1
+ module Outliers
2
+ module Resources
3
+ module Aws
4
+ module Ec2
5
+ class Image < Resource
6
+ def self.key
7
+ 'image_id'
8
+ end
9
+ end
10
+ end
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,22 @@
1
+ module Outliers
2
+ module Resources
3
+ module Aws
4
+ module Ec2
5
+ class ImageCollection < Collection
6
+
7
+ include Outliers::Filters::Aws::Ec2::Tags
8
+
9
+ def load_all
10
+ logger.debug "Loading private images owned by this account."
11
+ connect.images.with_owner(:self).map {|r| resource_class.new r}
12
+ end
13
+
14
+ def self.filters
15
+ Outliers::Filters::Aws::Ec2::Tags.filters
16
+ end
17
+
18
+ end
19
+ end
20
+ end
21
+ end
22
+ end
@@ -4,10 +4,16 @@ module Outliers
4
4
  module Ec2
5
5
  class InstanceCollection < Collection
6
6
 
7
+ include Outliers::Filters::Aws::Ec2::Tags
8
+
7
9
  def load_all
8
10
  connect.instances.map {|r| resource_class.new r}
9
11
  end
10
12
 
13
+ def self.filters
14
+ Outliers::Filters::Aws::Ec2::Tags.filters
15
+ end
16
+
11
17
  end
12
18
  end
13
19
  end
@@ -4,6 +4,8 @@ require 'outliers/resources/aws/ec2/security_group'
4
4
  require 'outliers/resources/aws/ec2/security_group_collection'
5
5
  require 'outliers/resources/aws/ec2/instance'
6
6
  require 'outliers/resources/aws/ec2/instance_collection'
7
+ require 'outliers/resources/aws/ec2/image'
8
+ require 'outliers/resources/aws/ec2/image_collection'
7
9
  require 'outliers/resources/aws/elb/load_balancer'
8
10
  require 'outliers/resources/aws/elb/load_balancer_collection'
9
11
  require 'outliers/resources/aws/iam/user'
@@ -1,5 +1,4 @@
1
1
  require 'outliers/resources/aws'
2
- require 'outliers/resources/github'
3
2
 
4
3
  module Outliers
5
4
  module Resources
@@ -1,11 +1,14 @@
1
1
  module Outliers
2
2
  class Result
3
3
 
4
- attr_reader :description, :passed
4
+ attr_reader :evaluation, :failing_resources, :passing_resources, :resource, :verification
5
5
 
6
6
  def initialize(args)
7
- @description = args[:description]
8
- @passed = args[:passed]
7
+ @evaluation = args[:evaluation]
8
+ @failing_resources = args[:failing_resources]
9
+ @passing_resources = args[:passing_resources]
10
+ @resource = args[:resource]
11
+ @verification = args[:verification]
9
12
  end
10
13
 
11
14
  def to_s
@@ -13,11 +16,11 @@ module Outliers
13
16
  end
14
17
 
15
18
  def passed?
16
- @passed == true
19
+ !failed?
17
20
  end
18
21
 
19
22
  def failed?
20
- !passed?
23
+ @failing_resources.any?
21
24
  end
22
25
 
23
26
  end
@@ -3,15 +3,13 @@ module Outliers
3
3
  module Shared
4
4
 
5
5
  def none_exist?
6
- logger.debug 'Verifying no resources exist.'
7
- logger.debug "Found #{all.empty? ? 'no resources' : all_by_key.join(',')}."
8
- all.empty?
6
+ all
9
7
  end
10
8
 
11
9
  def equals?(args)
12
10
  list = Array(args[:keys])
13
- logger.debug "Verifying '#{list.join(',')}' equals #{all.empty? ? 'no resources' : all_by_key.join(',')}."
14
- list == all_by_key
11
+ logger.debug "Verifying '#{list.join(',')}' equals '#{all.empty? ? 'no resources' : all_by_key.join(',')}'."
12
+ all.reject {|r| list.include? r.id}
15
13
  end
16
14
 
17
15
  module_function
@@ -1,3 +1,3 @@
1
1
  module Outliers
2
- VERSION = "0.1.1"
2
+ VERSION = "0.2.0"
3
3
  end
data/lib/outliers.rb CHANGED
@@ -5,6 +5,7 @@ require "outliers/collection"
5
5
  require "outliers/credentials"
6
6
  require "outliers/exceptions"
7
7
  require "outliers/evaluation"
8
+ require "outliers/filters"
8
9
  require "outliers/provider"
9
10
  require "outliers/providers"
10
11
  require "outliers/resource"
data/outliers.gemspec CHANGED
@@ -23,5 +23,4 @@ Gem::Specification.new do |spec|
23
23
  spec.add_development_dependency "rspec", "~> 2.13.0"
24
24
 
25
25
  spec.add_runtime_dependency "aws-sdk", "1.14.1"
26
- spec.add_runtime_dependency "github_api", "0.10.1"
27
26
  end
@@ -4,8 +4,6 @@ describe Outliers::Collection do
4
4
  let(:provider) { mock 'provider' }
5
5
  let(:resource1) { mock 'resource1' }
6
6
  let(:resource2) { mock 'resource2' }
7
- let(:resource_class) { stub 'resource_class', key: 'name',
8
- verifications_requiring_target: ['target_me?'] }
9
7
 
10
8
  subject { Outliers::Collection.new provider }
11
9
 
@@ -20,7 +18,7 @@ describe Outliers::Collection do
20
18
  context "#to_human" do
21
19
  it "should return the human name for this resource" do
22
20
  expect(Outliers::Resources::Aws::Ec2::SecurityGroupCollection.to_human).to eq('aws_ec2_security_group')
23
- expect(Outliers::Resources::Github::RepoCollection.to_human).to eq('github_repo')
21
+ expect(Outliers::Resources::Aws::S3::BucketCollection.to_human).to eq('aws_s3_bucket')
24
22
  end
25
23
  end
26
24
 
@@ -37,6 +35,19 @@ describe Outliers::Collection do
37
35
  end
38
36
  end
39
37
 
38
+ context "#filter" do
39
+ it "should apply the given filter to resources" do
40
+ subject.should_receive('filter_tag').with('Name:test123').and_return [resource1]
41
+ subject.filter 'tag' => 'Name:test123'
42
+ expect(subject.all).to eq([resource1])
43
+ end
44
+
45
+ it "should raise an exception if the filter does not exist" do
46
+ expect { subject.filter('bogus' => 'Name:test123') }.
47
+ to raise_error Outliers::Exceptions::UnknownFilter
48
+ end
49
+ end
50
+
40
51
  context "#key" do
41
52
  it "should return the key for the resource" do
42
53
  expect(subject.key).to eq('name')
@@ -50,37 +61,52 @@ describe Outliers::Collection do
50
61
  end
51
62
 
52
63
  context "#verify" do
64
+ let(:resource_class) {
65
+ m = [:source, :id, :method_missing, :equals?, :none_exist?, :valid_resource?]
66
+ stub 'resource_class', key: 'name',
67
+ verifications_requiring_target: ['target_me?'],
68
+ instance_methods: m
69
+ }
53
70
  before do
54
71
  Outliers::Collection.stub :resource_class => resource_class
55
72
  end
56
73
 
57
74
  it "should verify the given verification against the colection" do
58
- expect(subject.verify 'none?', {}).to be_false
75
+ expect(subject.verify 'none_exist?', {}).
76
+ to eq( { failing_resources: [resource1, resource2], passing_resources: [] } )
77
+ end
78
+
79
+ it "should raise unkown verification if the verification does not exist" do
80
+ expect { subject.verify 'none', {} }.to raise_error Outliers::Exceptions::UnknownVerification
59
81
  end
60
82
 
61
83
  it "should verify the given verification against the colection with options" do
62
- expect(subject.verify 'equals?', :keys => ['resource1', 'resource2']).to be_true
84
+ expect(subject.verify 'equals?', :keys => ['resource1', 'resource2']).
85
+ to eq( { failing_resources: [], passing_resources: [resource1, resource2] } )
63
86
  end
64
87
 
65
88
  it "should verify the given verification against each resource in the collection" do
66
89
  [resource1, resource2].each {|r| r.define_singleton_method :valid_resource?, lambda { true } }
67
- expect(subject.verify 'valid_resource?').to be_true
90
+ expect(subject.verify 'valid_resource?').
91
+ to eq( { failing_resources: [], passing_resources: [resource1, resource2] } )
68
92
  end
69
93
 
70
94
  it "should appaned a ? to the policy" do
71
- expect(subject.verify 'none').to be_false
95
+ expect(subject.verify 'none_exist').
96
+ to eq( { failing_resources: [resource1, resource2], passing_resources: [] } )
72
97
  end
73
98
 
74
99
  it "should remove all but the target resources if one is required and given" do
75
100
  subject.targets = ['resource1', 'in-valid']
76
- resource1.should_receive(:method).with('target_me?').and_return(stub 'method', :arity => 0)
77
- resource1.should_receive(:target_me?)
78
- expect(subject.verify 'target_me?').to be_true
101
+ resource1.should_receive(:method).with('valid_resource?').and_return(stub 'method', :arity => 0)
102
+ resource1.should_receive(:valid_resource?).and_return false
103
+ expect(subject.verify 'valid_resource?').
104
+ to eq( { failing_resources: [resource1], passing_resources: [] } )
79
105
  end
80
106
 
81
107
  it "should raise an error if the target resources does not exist" do
82
108
  subject.targets = ['in-valid']
83
- expect { subject.verify 'target_me?' }.to raise_error(Outliers::Exceptions::TargetNotFound)
109
+ expect { subject.verify 'valid_resource?' }.to raise_error(Outliers::Exceptions::TargetNotFound)
84
110
  end
85
111
 
86
112
  it "should raise an error if the verification requires arguments and none given" do
@@ -90,13 +116,20 @@ describe Outliers::Collection do
90
116
 
91
117
  it "should raise an error if the verification does not require arguments and arguments are given" do
92
118
  resource1.define_singleton_method :valid_resource?, lambda { true }
93
- expect { subject.verify 'valid_resource?', 'unneeded argument' => 3 }.to raise_error(Outliers::Exceptions::NoArgumentRequired)
119
+ expect { subject.verify 'valid_resource?', 'unneeded argument' => 3 }.
120
+ to raise_error(Outliers::Exceptions::NoArgumentRequired)
121
+ end
122
+
123
+ it "should return empty passing and failing arrays if no resources exist in all" do
124
+ subject.stub :load_all => []
125
+ expect(subject.verify 'valid_resource?', {}).to eq( { failing_resources: [], passing_resources: [] } )
94
126
  end
95
127
 
96
128
  it "should verify the given verification against each resource in the collection with options" do
97
129
  resource1.should_receive(:valid_resource?).with('test_arg' => 2).and_return true
98
130
  resource2.should_receive(:valid_resource?).with('test_arg' => 2).and_return true
99
- expect(subject.verify 'valid_resource?', 'test_arg' => 2).to be_true
131
+ expect(subject.verify 'valid_resource?', 'test_arg' => 2).
132
+ to eq( { failing_resources: [], passing_resources: [resource1, resource2] } )
100
133
  end
101
134
  end
102
135
 
@@ -2,7 +2,6 @@ require 'spec_helper'
2
2
 
3
3
  describe Outliers::Evaluation do
4
4
  let(:run) { mock 'run' }
5
- let(:result) { mock 'result' }
6
5
  let(:connect) { subject.connect('test_credentials_1') }
7
6
  let(:resources) { subject.resources('security_group') }
8
7
  subject { Outliers::Evaluation.new :run => run, :name => 'test' }
@@ -73,21 +72,63 @@ describe Outliers::Evaluation do
73
72
  end
74
73
  end
75
74
 
75
+ context "#filter" do
76
+ it "should apply the given filter to the collection" do
77
+ resources.should_receive(:filter).with('tag' => 'Name:test123')
78
+ subject.filter 'tag' => 'Name:test123'
79
+ end
80
+
81
+ it "should convert keys in the args hash to strings" do
82
+ resources.should_receive(:filter).with('tag' => 'Name:test123')
83
+ subject.filter tag: 'Name:test123'
84
+ end
85
+ end
86
+
76
87
  context "#verify" do
88
+ let(:result1) { mock 'result1' }
89
+ let(:result2) { mock 'result2' }
90
+ let(:verification_response) { ( { passing_resources: ['1', '2'], failing_resources: ['3', '4'] } ) }
91
+
77
92
  before do
78
- resources.should_receive(:load_all)
79
- Outliers::Result.should_receive(:new).with(description: 'test', :passed => true).and_return result
80
- run.should_receive(:results).and_return([])
93
+ resources.should_receive(:load_all).and_return ['resource1', 'resource2']
94
+ run.stub results: []
81
95
  end
82
96
 
83
97
  it "should verify the given method" do
84
- resources.should_receive(:verify).with('test_verification?', {}).and_return(true)
85
- expect(subject.verify('test_verification?', {})).to eq([result])
98
+ resources.should_receive(:verify).with('test_verification?', {}).and_return verification_response
99
+ Outliers::Result.should_receive(:new).with(evaluation: 'test',
100
+ passing_resources: ['1','2'],
101
+ failing_resources: ['3','4'],
102
+ resource: resources,
103
+ verification: 'test_verification?').and_return result1
104
+ expect(subject.verify('test_verification?', {})).to eq([result1])
86
105
  end
87
106
 
88
107
  it "should convert all options to symbols" do
89
- resources.should_receive(:verify).with('test_verification?', :test => false).and_return(true)
90
- expect(subject.verify('test_verification?', { 'test' => false } )).to eq([result])
108
+ resources.should_receive(:verify).with('test_verification?', :test => false).and_return verification_response
109
+ Outliers::Result.should_receive(:new).with(evaluation: 'test',
110
+ passing_resources: ['1','2'],
111
+ failing_resources: ['3','4'],
112
+ resource: resources,
113
+ verification: 'test_verification?').and_return result1
114
+ expect(subject.verify('test_verification?', { 'test' => false } )).to eq([result1])
115
+ end
116
+
117
+ it "should run verify multiple times in given evaluation" do
118
+ resources.should_receive(:verify).with('test_verification1?', :test => false).and_return verification_response
119
+ resources.should_receive(:verify).with('test_verification2?', :test => true).and_return verification_response
120
+ Outliers::Result.should_receive(:new).with(evaluation: 'test',
121
+ passing_resources: ['1','2'],
122
+ failing_resources: ['3','4'],
123
+ resource: resources,
124
+ verification: 'test_verification1?').and_return result1
125
+ Outliers::Result.should_receive(:new).with(evaluation: 'test',
126
+ passing_resources: ['1','2'],
127
+ failing_resources: ['3','4'],
128
+ resource: resources,
129
+ verification: 'test_verification2?').and_return result2
130
+ expect(subject.verify('test_verification1?', { 'test' => false })).to eq [result1]
131
+ expect(subject.verify('test_verification2?', { 'test' => true })).to eq [result1, result2]
91
132
  end
92
133
  end
93
134
 
@@ -0,0 +1,27 @@
1
+ require 'spec_helper'
2
+
3
+ describe Outliers::Filters::Aws::Ec2::Tags do
4
+ subject do
5
+ object = Object.new
6
+ object.extend Outliers::Filters::Aws::Ec2::Tags
7
+ object
8
+ end
9
+
10
+ let(:logger) { stub 'logger', debug: true, info: true }
11
+ let(:tags1) { mock 'tags1' }
12
+ let(:tags2) { mock 'tags2' }
13
+ let(:resource1) { stub 'resource1', tags: tags1, id: 'resource1' }
14
+ let(:resource2) { stub 'resource2', tags: tags2, id: 'resource2' }
15
+
16
+ before do
17
+ subject.stub :logger => logger
18
+ subject.stub :all => [resource1, resource2]
19
+ end
20
+
21
+ it "should return the list of instances filtered by the given tag name and value" do
22
+ tags1.should_receive(:has_key?).with('Name').and_return true
23
+ tags2.should_receive(:has_key?).with('Name').and_return false
24
+ tags1.should_receive(:[]).with('Name').and_return 'test123'
25
+ expect(subject.filter_tag('Name:test123')).to eq([resource1])
26
+ end
27
+ end
@@ -4,25 +4,25 @@ describe Outliers::Provider do
4
4
  subject { Outliers::Provider }
5
5
 
6
6
  context "#connect_to" do
7
- let(:credentials) { ( { :name => "test_credentials_1",
8
- "provider" => "github",
9
- "token" => "abc" } ) }
7
+ let(:credentials) { ( { :name => "test_credentials_1",
8
+ "provider" => "aws_ec2",
9
+ "secret_access_key" => "abc",
10
+ "access_key_id" => "123" } ) }
10
11
 
11
12
  it "should connect to the provider specified in the given credentials" do
12
- expect(subject.connect_to(credentials).class).to eq(Outliers::Providers::Github)
13
+ expect(subject.connect_to(credentials).class).to eq(Outliers::Providers::Aws::Ec2)
13
14
  end
14
15
 
15
16
  it "should set the credentials instance variable" do
16
17
  expect(subject.connect_to(credentials).credentials).
17
- to eq({ :name => "test_credentials_1", "provider" => "github", "token" => "abc" })
18
+ to eq({ :name => "test_credentials_1",
19
+ "provider" => "aws_ec2",
20
+ "secret_access_key" => "abc",
21
+ "access_key_id" => "123" })
18
22
  end
19
23
  end
20
24
 
21
25
  context "#to_human" do
22
- it "should return the name a human would use to access the provider" do
23
- expect(Outliers::Providers::Github.to_human).to eq('github')
24
- end
25
-
26
26
  it "should return the name a human would use to access the provider" do
27
27
  expect(Outliers::Providers::Aws::Rds.to_human).to eq('aws_rds')
28
28
  end
data/spec/results_spec.rb CHANGED
@@ -2,7 +2,12 @@ require 'spec_helper'
2
2
 
3
3
  describe Outliers::Result do
4
4
  context "passing" do
5
- subject { Outliers::Result.new description: 'stuff', passed: true }
5
+ subject { Outliers::Result.new evaluation: 'evalme',
6
+ failing_resources: [],
7
+ passing_resources: ['key1', 'key2'],
8
+ resource: 'instance',
9
+ verification: 'vpc' }
10
+
6
11
  it "should return passed" do
7
12
  expect(subject.to_s).to eq 'passed'
8
13
  end
@@ -11,18 +16,26 @@ describe Outliers::Result do
11
16
  expect(subject.passed?).to be_true
12
17
  end
13
18
 
14
- it "should return false for passing verification" do
19
+ it "should return false for failing verification" do
15
20
  expect(subject.failed?).to be_false
16
21
  end
22
+
23
+ it "should return the result information" do
24
+ expect(subject.passing_resources).to eq(['key1', 'key2'])
25
+ end
17
26
  end
18
27
 
19
28
  context "failing" do
20
- subject { Outliers::Result.new description: 'stuff', passed: false }
21
- it "should return passed" do
29
+ subject { Outliers::Result.new evaluation: 'evalme',
30
+ failing_resources: ['key3', 'key4'],
31
+ passing_resources: [],
32
+ resource: 'instance',
33
+ verification: 'vpc' }
34
+ it "should return failed" do
22
35
  expect(subject.to_s).to eq 'failed'
23
36
  end
24
37
 
25
- it "should return false for failing verification" do
38
+ it "should return false for passing verification" do
26
39
  expect(subject.passed?).to be_false
27
40
  end
28
41
 
data/spec/run_spec.rb CHANGED
@@ -47,8 +47,9 @@ describe Outliers::Run do
47
47
  end
48
48
 
49
49
  context "returning results" do
50
- let(:result1) { Outliers::Result.new description: 'result1', passed: true }
51
- let(:result2) { Outliers::Result.new description: 'result2', passed: false }
50
+ let(:result1) { Outliers::Result.new name: 'result1', passing_resources: [], failing_resources: [], evaluation: 'test', verification: 'ver' }
51
+ let(:result2) { Outliers::Result.new name: 'result2', passing_resources: [], failing_resources: ['failed'], evaluation: 'test', verification: 'ver' }
52
+
52
53
  before do
53
54
  subject.results << result1
54
55
  subject.results << result2
@@ -2,6 +2,8 @@ require 'spec_helper'
2
2
 
3
3
  describe Outliers::Verifications::Shared do
4
4
  subject { Object.new.extend Outliers::Verifications::Shared }
5
+ let(:resource1) { stub "resource1", id: 'resource1' }
6
+ let(:resource2) { stub "resource2", id: 'resource2' }
5
7
 
6
8
  before do
7
9
  logger_stub = stub 'logger', :debug => true
@@ -11,25 +13,30 @@ describe Outliers::Verifications::Shared do
11
13
  context "#none_exist?" do
12
14
  it "should be true if no resources returned" do
13
15
  subject.stub :all => []
14
- expect(subject.none_exist?).to be_true
16
+ expect(subject.none_exist?).to eq([])
15
17
  end
16
18
 
17
19
  it "should be false if resources returned" do
18
- subject.stub :all_by_key => ['test']
19
- subject.stub :all => ['test']
20
- expect(subject.none_exist?).to be_false
20
+ subject.stub :all_by_key => ['resource1']
21
+ subject.stub :all => ['resource1']
22
+ expect(subject.none_exist?).to eq(['resource1'])
21
23
  end
22
24
  end
23
25
 
24
26
  context "#equals?" do
25
- it "should verify the list of resources equals the list of keys" do
26
- subject.stub :all_by_key => ['test'], :all => ['test_resource']
27
- expect(subject.equals?(:keys => ['test'])).to be_true
27
+ it "should verify the list of resources equals the list of keys and return no failing reosurces" do
28
+ subject.stub :all_by_key => ['resource1'], :all => [resource1]
29
+ expect(subject.equals?(:keys => ['resource1'])).to eq([])
28
30
  end
29
31
 
30
- it "should verify the list of resources equals the single key" do
31
- subject.stub :all_by_key => ['test'], :all => ['test_resource']
32
- expect(subject.equals?(:keys => 'test')).to be_true
32
+ it "should verify the list of resources equals the single key and return no failing resources" do
33
+ subject.stub :all_by_key => ['resource1'], :all => [resource1]
34
+ expect(subject.equals?(:keys => 'resource1')).to eq([])
35
+ end
36
+
37
+ it "should return resources which do not match the given list" do
38
+ subject.stub :all_by_key => ['resource1', 'resource2'], :all => [resource1, resource2]
39
+ expect(subject.equals?(:keys => 'resource1')).to eq([resource2])
33
40
  end
34
41
  end
35
42
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: outliers
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Brett Weaver
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2013-08-13 00:00:00.000000000 Z
11
+ date: 2013-08-19 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -66,20 +66,6 @@ dependencies:
66
66
  - - '='
67
67
  - !ruby/object:Gem::Version
68
68
  version: 1.14.1
69
- - !ruby/object:Gem::Dependency
70
- name: github_api
71
- requirement: !ruby/object:Gem::Requirement
72
- requirements:
73
- - - '='
74
- - !ruby/object:Gem::Version
75
- version: 0.10.1
76
- type: :runtime
77
- prerelease: false
78
- version_requirements: !ruby/object:Gem::Requirement
79
- requirements:
80
- - - '='
81
- - !ruby/object:Gem::Version
82
- version: 0.10.1
83
69
  description: Configuraiton verification framework.
84
70
  email:
85
71
  - brett@weav.net
@@ -108,6 +94,10 @@ files:
108
94
  - lib/outliers/credentials.rb
109
95
  - lib/outliers/evaluation.rb
110
96
  - lib/outliers/exceptions.rb
97
+ - lib/outliers/filters.rb
98
+ - lib/outliers/filters/aws.rb
99
+ - lib/outliers/filters/aws/ec2.rb
100
+ - lib/outliers/filters/aws/ec2/tags.rb
111
101
  - lib/outliers/mixins.rb
112
102
  - lib/outliers/provider.rb
113
103
  - lib/outliers/providers.rb
@@ -120,12 +110,13 @@ files:
120
110
  - lib/outliers/providers/aws/rds.rb
121
111
  - lib/outliers/providers/aws/s3.rb
122
112
  - lib/outliers/providers/aws/sqs.rb
123
- - lib/outliers/providers/github.rb
124
113
  - lib/outliers/resource.rb
125
114
  - lib/outliers/resources.rb
126
115
  - lib/outliers/resources/aws.rb
127
116
  - lib/outliers/resources/aws/cloud_formation/stack.rb
128
117
  - lib/outliers/resources/aws/cloud_formation/stack_collection.rb
118
+ - lib/outliers/resources/aws/ec2/image.rb
119
+ - lib/outliers/resources/aws/ec2/image_collection.rb
129
120
  - lib/outliers/resources/aws/ec2/instance.rb
130
121
  - lib/outliers/resources/aws/ec2/instance_collection.rb
131
122
  - lib/outliers/resources/aws/ec2/security_group.rb
@@ -142,9 +133,6 @@ files:
142
133
  - lib/outliers/resources/aws/s3/bucket_collection.rb
143
134
  - lib/outliers/resources/aws/sqs/queue.rb
144
135
  - lib/outliers/resources/aws/sqs/queue_collection.rb
145
- - lib/outliers/resources/github.rb
146
- - lib/outliers/resources/github/repo.rb
147
- - lib/outliers/resources/github/repo_collection.rb
148
136
  - lib/outliers/result.rb
149
137
  - lib/outliers/run.rb
150
138
  - lib/outliers/verifications.rb
@@ -154,6 +142,7 @@ files:
154
142
  - spec/collection_spec.rb
155
143
  - spec/credentials_spec.rb
156
144
  - spec/evaluation_spec.rb
145
+ - spec/filters/aws/ec2/tags_spec.rb
157
146
  - spec/fixtures/credentials1.yml
158
147
  - spec/fixtures/credentials2.yml
159
148
  - spec/helpers/fixtures.rb
@@ -194,6 +183,7 @@ test_files:
194
183
  - spec/collection_spec.rb
195
184
  - spec/credentials_spec.rb
196
185
  - spec/evaluation_spec.rb
186
+ - spec/filters/aws/ec2/tags_spec.rb
197
187
  - spec/fixtures/credentials1.yml
198
188
  - spec/fixtures/credentials2.yml
199
189
  - spec/helpers/fixtures.rb
@@ -1,23 +0,0 @@
1
- require "github_api"
2
-
3
- module Outliers
4
- module Providers
5
- class Github < Provider
6
-
7
- def settings(args)
8
- @token = args[:token]
9
- end
10
-
11
- def connect
12
- c = ::Github.new
13
- c.oauth_token = @token if @token
14
- c
15
- end
16
-
17
- def self.credential_arguments
18
- { 'token' => 'Github API token.' }
19
- end
20
-
21
- end
22
- end
23
- end
@@ -1,24 +0,0 @@
1
- module Outliers
2
- module Resources
3
- module Github
4
- class Repo < Resource
5
- def self.verifications
6
- [
7
- { name: 'private',
8
- description: 'Repo is private.' },
9
- { name: 'public',
10
- description: 'Repo is public.' }
11
- ]
12
- end
13
-
14
- def private?
15
- source.private
16
- end
17
-
18
- def public?
19
- !source.private
20
- end
21
- end
22
- end
23
- end
24
- end
@@ -1,13 +0,0 @@
1
- module Outliers
2
- module Resources
3
- module Github
4
- class RepoCollection < Collection
5
-
6
- def load_all
7
- connect.repos.list(:type => 'all').map {|r| resource_class.new r}
8
- end
9
-
10
- end
11
- end
12
- end
13
- end
@@ -1,2 +0,0 @@
1
- require 'outliers/resources/github/repo'
2
- require 'outliers/resources/github/repo_collection'