fog_tracker 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +26 -12
- data/lib/fog_tracker/account_tracker.rb +10 -3
- data/lib/fog_tracker/model.rb +20 -0
- data/lib/fog_tracker/query/query_processor.rb +5 -4
- data/lib/fog_tracker/resource_tracker.rb +22 -4
- data/lib/fog_tracker/tracker.rb +4 -1
- data/lib/fog_tracker/version.rb +1 -1
- data/lib/fog_tracker.rb +2 -1
- data/spec/lib/fog_tracker/resource_tracker_spec.rb +12 -1
- data/spec/lib/fog_tracker/tracker_spec.rb +25 -0
- data/spec/support/_mocks.rb +9 -17
- metadata +20 -19
data/README.md
CHANGED
@@ -28,11 +28,10 @@ Install the Fog Tracker gem (and its dependencies if necessary) from RubyGems
|
|
28
28
|
----------------
|
29
29
|
How is it [done]? (Usage)
|
30
30
|
----------------
|
31
|
-
1)
|
31
|
+
1) Require the gem, and create a `FogTracker::Tracker`. Pass it some account information in a hash, perhaps loaded from a YAML file:
|
32
32
|
|
33
33
|
require 'fog_tracker'
|
34
34
|
tracker = FogTracker::Tracker.new(YAML::load(File.read 'accounts.yml'))
|
35
|
-
tracker.start
|
36
35
|
|
37
36
|
Here are the contents of a sample `accounts.yml`:
|
38
37
|
|
@@ -54,24 +53,39 @@ How is it [done]? (Usage)
|
|
54
53
|
:rackspace_username: XXXXXXXXX
|
55
54
|
:polling_time: 180
|
56
55
|
|
57
|
-
2)
|
56
|
+
2) Call `start` on the Tracker. It will run asynchronously, with one thread per account. At any time, you can call `start` or `stop` on it, and query the resulting collections of Fog Resource objects.
|
57
|
+
|
58
|
+
tracker.start
|
59
|
+
|
60
|
+
3) Access the Fog object collections by calling `Tracker::query`, and passing it a filter-based query String. The query string format is: `account_name::service::provider::collection`
|
61
|
+
|
62
|
+
# get all Compute instances across all accounts and providers
|
63
|
+
tracker.query("*::Compute::*::servers")
|
64
|
+
|
65
|
+
# get all Amazon EC2 Resources, of all types, across all accounts
|
66
|
+
tracker["*::Compute::AWS::*"] # the [] operator is the same as query()
|
67
|
+
|
68
|
+
# get all S3 objects in a given account
|
69
|
+
tracker["my production account::Storage::AWS::files"]
|
58
70
|
|
59
|
-
|
60
|
-
tracker.query("*::Compute::*::servers")
|
71
|
+
If you're tired of calling `each` on the results of every query, pass a single-argument block, and it will be invoked once with each resulting resource:
|
61
72
|
|
62
|
-
|
63
|
-
tracker["*::Compute::AWS::*"] # the [] operator is the same as query()
|
73
|
+
tracker.query("*::*::*::*"){|r| puts "Found #{r.class} #{r.identity}"}
|
64
74
|
|
65
|
-
|
66
|
-
tracker["my production account::Storage::AWS::files"]
|
75
|
+
You can also pass a Proc to the Tracker at initialization, which will be invoked whenever an account's Resources have been updated. It should accept a list of the updated Resources as its first argument:
|
67
76
|
|
68
|
-
|
77
|
+
FogTracker::Tracker.new(YAML::load(File.read 'accounts.yml'),
|
78
|
+
:callback => Proc.new do |resources|
|
79
|
+
puts "Got #{resources.count} resources from account "+
|
80
|
+
resources.first.tracker_account[:name]
|
81
|
+
end
|
82
|
+
).start
|
69
83
|
|
70
|
-
|
84
|
+
The appropriate account information for each Fog resource, read from `accounts.yml` can be obtained by calling its `tracker_account` method.
|
71
85
|
|
72
86
|
|
73
87
|
----------------
|
74
|
-
Who is it? (Contribution
|
88
|
+
Who is it? (Contribution)
|
75
89
|
----------------
|
76
90
|
This Gem was created by Benton Roberts _(benton@bentonroberts.com)_, but draws heavily on the work of the [Fog project](http://fog.io/). Thanks to geemus, and to all Fog contributors.
|
77
91
|
|
@@ -1,13 +1,13 @@
|
|
1
1
|
module FogTracker
|
2
2
|
|
3
|
-
# Tracks a single Fog account
|
3
|
+
# Tracks all collections in a single Fog account
|
4
4
|
class AccountTracker
|
5
5
|
require 'fog'
|
6
6
|
|
7
7
|
attr_reader :name, :account, :log, :delay
|
8
8
|
attr_reader :resource_trackers
|
9
9
|
|
10
|
-
# Creates an object for tracking a single Fog account
|
10
|
+
# Creates an object for tracking all collections in a single Fog account
|
11
11
|
#
|
12
12
|
# ==== Attributes
|
13
13
|
#
|
@@ -41,7 +41,7 @@ module FogTracker
|
|
41
41
|
while true do
|
42
42
|
@log.info "Polling account #{@name}..."
|
43
43
|
@resource_trackers.each {|tracker| tracker.update}
|
44
|
-
@callback.call
|
44
|
+
@callback.call(all_resources) if @callback
|
45
45
|
sleep @delay
|
46
46
|
end
|
47
47
|
rescue Exception => e
|
@@ -82,6 +82,13 @@ module FogTracker
|
|
82
82
|
types.map {|type| type.to_s}
|
83
83
|
end
|
84
84
|
|
85
|
+
# Returns an Array of all this Account's currently tracked Resources
|
86
|
+
def all_resources
|
87
|
+
(@resource_trackers.collect do |tracker|
|
88
|
+
tracker.collection
|
89
|
+
end).flatten
|
90
|
+
end
|
91
|
+
|
85
92
|
private
|
86
93
|
|
87
94
|
# Creates and returns an Array of ResourceTracker objects -
|
@@ -0,0 +1,20 @@
|
|
1
|
+
# add an accessor and a method to decorate Fog::Model instances
|
2
|
+
# with tracker account information
|
3
|
+
module Fog
|
4
|
+
class Model
|
5
|
+
|
6
|
+
# a FogTracker::CollectionTracker
|
7
|
+
attr_accessor :fog_collection_tracker
|
8
|
+
|
9
|
+
# returns a clean copy of the resource's account information
|
10
|
+
# from the its collection tracker
|
11
|
+
def tracker_account
|
12
|
+
if fog_collection_tracker
|
13
|
+
fog_collection_tracker.clean_account_data
|
14
|
+
else
|
15
|
+
Hash.new
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
end
|
20
|
+
end
|
@@ -22,10 +22,11 @@ module FogTracker
|
|
22
22
|
# Returns an Array of Resources, filtered by +query+
|
23
23
|
def execute(query)
|
24
24
|
acct_pattern, svc_pattern, prov_pattern, col_pattern = parse_query(query)
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
25
|
+
filter_by_collection(
|
26
|
+
filter_by_provider(
|
27
|
+
filter_by_service(get_results_by_account(acct_pattern), svc_pattern),
|
28
|
+
prov_pattern),
|
29
|
+
col_pattern)
|
29
30
|
end
|
30
31
|
|
31
32
|
private
|
@@ -1,11 +1,11 @@
|
|
1
1
|
module FogTracker
|
2
2
|
|
3
|
-
# Tracks a single Fog
|
3
|
+
# Tracks a single Fog collection in a single account
|
4
4
|
class ResourceTracker
|
5
5
|
|
6
6
|
attr_accessor :collection
|
7
7
|
|
8
|
-
# Creates an object for tracking a single Fog account
|
8
|
+
# Creates an object for tracking a single Fog collection in a single account
|
9
9
|
#
|
10
10
|
# ==== Attributes
|
11
11
|
#
|
@@ -25,8 +25,26 @@ module FogTracker
|
|
25
25
|
# instances of the relevant resource type, and saves them as @collection
|
26
26
|
def update
|
27
27
|
@log.info "Polling #{@type} on #{@account_name}..."
|
28
|
-
|
29
|
-
@log.info "
|
28
|
+
fog_collection = @account_tracker.connection.send(@type) || Array.new
|
29
|
+
@log.info "Fetching #{fog_collection.count} #{@type} on #{@account_name}."
|
30
|
+
new_collection = Array.new
|
31
|
+
# Here's where most of the network overhead is actually incurred
|
32
|
+
fog_collection.each do |resource|
|
33
|
+
@log.debug "Fetching resource: #{resource.class} #{resource.identity}"
|
34
|
+
resource.fog_collection_tracker = self
|
35
|
+
new_collection << resource
|
36
|
+
@log.debug "Got resource: #{resource.inspect}"
|
37
|
+
end
|
38
|
+
@collection = new_collection
|
39
|
+
end
|
40
|
+
|
41
|
+
# Returns a Hash of account information as resource.tracker_account
|
42
|
+
# a :name parameter is added, and the :credentials are removed
|
43
|
+
def clean_account_data
|
44
|
+
@clean_data ||= @account # generate this data only once per res
|
45
|
+
@clean_data[:name] = @account_name
|
46
|
+
@clean_data[:credentials] = Hash.new
|
47
|
+
@clean_data
|
30
48
|
end
|
31
49
|
|
32
50
|
end
|
data/lib/fog_tracker/tracker.rb
CHANGED
@@ -58,15 +58,18 @@ module FogTracker
|
|
58
58
|
end
|
59
59
|
|
60
60
|
# Returns an array of Resources matching the +query_string+
|
61
|
+
# Calls any block passed for each resulting resource
|
61
62
|
#
|
62
63
|
# ==== Attributes
|
63
64
|
#
|
64
65
|
# * +query_string+ - a String used to filter the discovered Resources
|
65
66
|
# it might look like: "Account Name::Compute::AWS::servers"
|
66
67
|
def query(query_string)
|
67
|
-
FogTracker::Query::QueryProcessor.new(
|
68
|
+
results = FogTracker::Query::QueryProcessor.new(
|
68
69
|
@trackers, :logger => @log
|
69
70
|
).execute(query_string)
|
71
|
+
(results.each {|r| yield r}) if block_given?
|
72
|
+
results
|
70
73
|
end
|
71
74
|
alias :[] :query
|
72
75
|
|
data/lib/fog_tracker/version.rb
CHANGED
data/lib/fog_tracker.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
require 'logger'
|
2
2
|
|
3
|
-
# Load all ruby files from 'fog_tracker'
|
3
|
+
# Load all ruby files from the 'fog' and 'fog_tracker' directories
|
4
|
+
Dir[File.join(File.dirname(__FILE__), "fog/**/*.rb")].each {|f| require f}
|
4
5
|
Dir[File.join(File.dirname(__FILE__), "fog_tracker/**/*.rb")].each {|f| require f}
|
5
6
|
|
6
7
|
module FogTracker
|
@@ -5,7 +5,9 @@ module FogTracker
|
|
5
5
|
describe ResourceTracker do
|
6
6
|
|
7
7
|
before(:each) do
|
8
|
-
@account_tracker =
|
8
|
+
@account_tracker = AccountTracker.new(
|
9
|
+
FAKE_ACCOUNT_NAME, FAKE_ACCOUNT, :logger => LOG
|
10
|
+
)
|
9
11
|
@tracker = ResourceTracker.new(
|
10
12
|
FAKE_COLLECTION, @account_tracker
|
11
13
|
)
|
@@ -20,6 +22,15 @@ module FogTracker
|
|
20
22
|
@account_tracker.connection.should_receive(FAKE_COLLECTION)
|
21
23
|
@tracker.update
|
22
24
|
end
|
25
|
+
it "attaches account information to all resources" do
|
26
|
+
account_tracker = mock_account_tracker(5,5)
|
27
|
+
tracker = ResourceTracker.new(FAKE_COLLECTION, account_tracker)
|
28
|
+
tracker.update
|
29
|
+
tracker.collection.each do |resource|
|
30
|
+
resource.tracker_account[:name].should == FAKE_ACCOUNT_NAME
|
31
|
+
resource.tracker_account[:credentials].should == {}
|
32
|
+
end
|
33
|
+
end
|
23
34
|
end
|
24
35
|
end
|
25
36
|
|
@@ -34,5 +34,30 @@ module FogTracker
|
|
34
34
|
end
|
35
35
|
end
|
36
36
|
|
37
|
+
describe '#query' do
|
38
|
+
it "invokes any passed code block once per resulting Resource" do
|
39
|
+
receiver = double "resrouce callback object"
|
40
|
+
receiver.stub(:callback)
|
41
|
+
@tracker.start
|
42
|
+
sleep THREAD_STARTUP_DELAY
|
43
|
+
res_count = @tracker.query('*::*::*::*').count
|
44
|
+
receiver.should_receive(:callback).exactly(res_count).times
|
45
|
+
@tracker.query('*::*::*::*') {|r| receiver.callback(r)}
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
describe '#start' do
|
50
|
+
it "invokes the Tracker's callback Proc when an account is updated" do
|
51
|
+
receiver = double "account callback object"
|
52
|
+
receiver.stub(:callback)
|
53
|
+
tracker = Tracker.new(ACCOUNTS, :logger => LOG,
|
54
|
+
:callback => Proc.new {|resources| receiver.callback(resources)}
|
55
|
+
)
|
56
|
+
receiver.should_receive(:callback).exactly(ACCOUNTS.size).times
|
57
|
+
tracker.start
|
58
|
+
sleep THREAD_STARTUP_DELAY
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
37
62
|
end
|
38
63
|
end
|
data/spec/support/_mocks.rb
CHANGED
@@ -32,6 +32,10 @@ module FogTracker
|
|
32
32
|
module Query
|
33
33
|
QUERY = {} # Used in query_processor_spec.rb
|
34
34
|
end
|
35
|
+
class MockFogResource < Fog::Model
|
36
|
+
#attr_accessor :tracker_account
|
37
|
+
def identity ; "random-resource-id-#{rand 65536}" end
|
38
|
+
end
|
35
39
|
end
|
36
40
|
|
37
41
|
# Create some fake Fog Resource and Collection Classes
|
@@ -54,8 +58,9 @@ end
|
|
54
58
|
def mock_account_tracker(num_collections = 1, resources_per_collection = 0)
|
55
59
|
fake_account_tracker = double('mock_account_tracker')
|
56
60
|
fake_account_tracker.stub(:account).and_return(Hash.new)
|
57
|
-
fake_account_tracker.stub(:name).and_return(
|
61
|
+
fake_account_tracker.stub(:name).and_return(FogTracker::FAKE_ACCOUNT_NAME)
|
58
62
|
fake_account_tracker.stub(:log).and_return(LOG)
|
63
|
+
fake_account_tracker.stub(:all_resources).and_return(Array.new)
|
59
64
|
fake_account_tracker.stub(:connection).and_return(mock_fog_connection)
|
60
65
|
# create an array of mock ResourceTrackers
|
61
66
|
trackers = (1..num_collections).map do |class_index|
|
@@ -70,23 +75,10 @@ end
|
|
70
75
|
|
71
76
|
def mock_fog_connection
|
72
77
|
fake_fog_connection = double('mock_fog_connection')
|
73
|
-
fake_fog_connection.stub(FogTracker::FAKE_COLLECTION).and_return(
|
74
|
-
|
75
|
-
end
|
76
|
-
|
77
|
-
def mock_resource_tracker
|
78
|
-
fake_resource_tracker = double('mock_resource_tracker')
|
79
|
-
fake_resource_tracker.stub(:update)
|
80
|
-
fake_resource_tracker
|
81
|
-
end
|
82
|
-
|
83
|
-
def mock_fog_resource(resource_class)
|
84
|
-
fake_resource = resource_class.new
|
85
|
-
fake_collection_class =
|
86
|
-
fake_resource.stub(:collection).and_return(
|
87
|
-
collection_class.new
|
78
|
+
fake_fog_connection.stub(FogTracker::FAKE_COLLECTION).and_return(
|
79
|
+
[ FogTracker::MockFogResource.new, FogTracker::MockFogResource.new ]
|
88
80
|
)
|
89
|
-
|
81
|
+
fake_fog_connection
|
90
82
|
end
|
91
83
|
|
92
84
|
def mock_resource_tracker(resource_class, number_of_resources = 0)
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: fog_tracker
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.2
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,11 +9,11 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2012-01-
|
12
|
+
date: 2012-01-27 00:00:00.000000000Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: fog
|
16
|
-
requirement: &
|
16
|
+
requirement: &70121482437700 !ruby/object:Gem::Requirement
|
17
17
|
none: false
|
18
18
|
requirements:
|
19
19
|
- - ! '>='
|
@@ -21,10 +21,10 @@ dependencies:
|
|
21
21
|
version: '0'
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
|
-
version_requirements: *
|
24
|
+
version_requirements: *70121482437700
|
25
25
|
- !ruby/object:Gem::Dependency
|
26
26
|
name: rake
|
27
|
-
requirement: &
|
27
|
+
requirement: &70121482437100 !ruby/object:Gem::Requirement
|
28
28
|
none: false
|
29
29
|
requirements:
|
30
30
|
- - ! '>='
|
@@ -32,10 +32,10 @@ dependencies:
|
|
32
32
|
version: '0'
|
33
33
|
type: :development
|
34
34
|
prerelease: false
|
35
|
-
version_requirements: *
|
35
|
+
version_requirements: *70121482437100
|
36
36
|
- !ruby/object:Gem::Dependency
|
37
37
|
name: rspec
|
38
|
-
requirement: &
|
38
|
+
requirement: &70121482436440 !ruby/object:Gem::Requirement
|
39
39
|
none: false
|
40
40
|
requirements:
|
41
41
|
- - ! '>='
|
@@ -43,10 +43,10 @@ dependencies:
|
|
43
43
|
version: '0'
|
44
44
|
type: :development
|
45
45
|
prerelease: false
|
46
|
-
version_requirements: *
|
46
|
+
version_requirements: *70121482436440
|
47
47
|
- !ruby/object:Gem::Dependency
|
48
48
|
name: rdoc
|
49
|
-
requirement: &
|
49
|
+
requirement: &70121482435700 !ruby/object:Gem::Requirement
|
50
50
|
none: false
|
51
51
|
requirements:
|
52
52
|
- - ! '>='
|
@@ -54,10 +54,10 @@ dependencies:
|
|
54
54
|
version: '0'
|
55
55
|
type: :development
|
56
56
|
prerelease: false
|
57
|
-
version_requirements: *
|
57
|
+
version_requirements: *70121482435700
|
58
58
|
- !ruby/object:Gem::Dependency
|
59
59
|
name: guard
|
60
|
-
requirement: &
|
60
|
+
requirement: &70121482434720 !ruby/object:Gem::Requirement
|
61
61
|
none: false
|
62
62
|
requirements:
|
63
63
|
- - ! '>='
|
@@ -65,10 +65,10 @@ dependencies:
|
|
65
65
|
version: '0'
|
66
66
|
type: :development
|
67
67
|
prerelease: false
|
68
|
-
version_requirements: *
|
68
|
+
version_requirements: *70121482434720
|
69
69
|
- !ruby/object:Gem::Dependency
|
70
70
|
name: guard-rake
|
71
|
-
requirement: &
|
71
|
+
requirement: &70121482433280 !ruby/object:Gem::Requirement
|
72
72
|
none: false
|
73
73
|
requirements:
|
74
74
|
- - ! '>='
|
@@ -76,10 +76,10 @@ dependencies:
|
|
76
76
|
version: '0'
|
77
77
|
type: :development
|
78
78
|
prerelease: false
|
79
|
-
version_requirements: *
|
79
|
+
version_requirements: *70121482433280
|
80
80
|
- !ruby/object:Gem::Dependency
|
81
81
|
name: ruby_gntp
|
82
|
-
requirement: &
|
82
|
+
requirement: &70121482432400 !ruby/object:Gem::Requirement
|
83
83
|
none: false
|
84
84
|
requirements:
|
85
85
|
- - ! '>='
|
@@ -87,7 +87,7 @@ dependencies:
|
|
87
87
|
version: '0'
|
88
88
|
type: :development
|
89
89
|
prerelease: false
|
90
|
-
version_requirements: *
|
90
|
+
version_requirements: *70121482432400
|
91
91
|
description: This gem peridically polls mutiple cloud computing services using the
|
92
92
|
fog gem, asynchronously updating the state of the resulting collections of Fog Resources.
|
93
93
|
email:
|
@@ -107,6 +107,7 @@ files:
|
|
107
107
|
- fog_tracker.gemspec
|
108
108
|
- lib/fog_tracker.rb
|
109
109
|
- lib/fog_tracker/account_tracker.rb
|
110
|
+
- lib/fog_tracker/model.rb
|
110
111
|
- lib/fog_tracker/query/query_processor.rb
|
111
112
|
- lib/fog_tracker/resource_tracker.rb
|
112
113
|
- lib/fog_tracker/tracker.rb
|
@@ -135,7 +136,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
135
136
|
version: '0'
|
136
137
|
segments:
|
137
138
|
- 0
|
138
|
-
hash:
|
139
|
+
hash: -2556488865982976006
|
139
140
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
140
141
|
none: false
|
141
142
|
requirements:
|
@@ -144,10 +145,10 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
144
145
|
version: '0'
|
145
146
|
segments:
|
146
147
|
- 0
|
147
|
-
hash:
|
148
|
+
hash: -2556488865982976006
|
148
149
|
requirements: []
|
149
150
|
rubyforge_project: fog_tracker
|
150
|
-
rubygems_version: 1.8.
|
151
|
+
rubygems_version: 1.8.10
|
151
152
|
signing_key:
|
152
153
|
specification_version: 3
|
153
154
|
summary: Tracks the state of cloud computing resources across multiple accounts with
|