fog_tracker 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -28,11 +28,10 @@ Install the Fog Tracker gem (and its dependencies if necessary) from RubyGems
28
28
  ----------------
29
29
  How is it [done]? (Usage)
30
30
  ----------------
31
- 1) Just require the gem, and create a `FogTracker::Tracker`. Pass it some account information in a hash, perhaps loaded from a YAML file:
31
+ 1) Require the gem, and create a `FogTracker::Tracker`. Pass it some account information in a hash, perhaps loaded from a YAML file:
32
32
 
33
33
  require 'fog_tracker'
34
34
  tracker = FogTracker::Tracker.new(YAML::load(File.read 'accounts.yml'))
35
- tracker.start
36
35
 
37
36
  Here are the contents of a sample `accounts.yml`:
38
37
 
@@ -54,24 +53,39 @@ How is it [done]? (Usage)
54
53
  :rackspace_username: XXXXXXXXX
55
54
  :polling_time: 180
56
55
 
57
- 2) The tracker will run asynchronously, with one thread per account. You can call `start()` and `stop()` on it, and query the resulting collections of Fog Resource objects using a filter-based query:
56
+ 2) Call `start` on the Tracker. It will run asynchronously, with one thread per account. At any time, you can call `start` or `stop` on it, and query the resulting collections of Fog Resource objects.
57
+
58
+ tracker.start
59
+
60
+ 3) Access the Fog object collections by calling `Tracker::query`, and passing it a filter-based query String. The query string format is: `account_name::service::provider::collection`
61
+
62
+ # get all Compute instances across all accounts and providers
63
+ tracker.query("*::Compute::*::servers")
64
+
65
+ # get all Amazon EC2 Resources, of all types, across all accounts
66
+ tracker["*::Compute::AWS::*"] # the [] operator is the same as query()
67
+
68
+ # get all S3 objects in a given account
69
+ tracker["my production account::Storage::AWS::files"]
58
70
 
59
- # get all Compute instances across all accounts and providers
60
- tracker.query("*::Compute::*::servers")
71
+ If you're tired of calling `each` on the results of every query, pass a single-argument block, and it will be invoked once with each resulting resource:
61
72
 
62
- # get all Amazon EC2 Resources, of all types, across all accounts
63
- tracker["*::Compute::AWS::*"] # the [] operator is the same as query()
73
+ tracker.query("*::*::*::*"){|r| puts "Found #{r.class} #{r.identity}"}
64
74
 
65
- # get all S3 objects in a given account
66
- tracker["my production account::Storage::AWS::files"]
75
+ You can also pass a Proc to the Tracker at initialization, which will be invoked whenever an account's Resources have been updated. It should accept a list of the updated Resources as its first argument:
67
76
 
68
- The query string format is: "`account name::service::provider::collection`"
77
+ FogTracker::Tracker.new(YAML::load(File.read 'accounts.yml'),
78
+ :callback => Proc.new do |resources|
79
+ puts "Got #{resources.count} resources from account "+
80
+ resources.first.tracker_account[:name]
81
+ end
82
+ ).start
69
83
 
70
- You can also pass a Proc to the Tracker at initialization, which will be invoked whenever an account's Resources have been updated -- see the API docs for details.
84
+ The appropriate account information for each Fog resource, read from `accounts.yml` can be obtained by calling its `tracker_account` method.
71
85
 
72
86
 
73
87
  ----------------
74
- Who is it? (Contribution / Development)
88
+ Who is it? (Contribution)
75
89
  ----------------
76
90
  This Gem was created by Benton Roberts _(benton@bentonroberts.com)_, but draws heavily on the work of the [Fog project](http://fog.io/). Thanks to geemus, and to all Fog contributors.
77
91
 
@@ -1,13 +1,13 @@
1
1
  module FogTracker
2
2
 
3
- # Tracks a single Fog account in an ActiveRecord database
3
+ # Tracks all collections in a single Fog account
4
4
  class AccountTracker
5
5
  require 'fog'
6
6
 
7
7
  attr_reader :name, :account, :log, :delay
8
8
  attr_reader :resource_trackers
9
9
 
10
- # Creates an object for tracking a single Fog account
10
+ # Creates an object for tracking all collections in a single Fog account
11
11
  #
12
12
  # ==== Attributes
13
13
  #
@@ -41,7 +41,7 @@ module FogTracker
41
41
  while true do
42
42
  @log.info "Polling account #{@name}..."
43
43
  @resource_trackers.each {|tracker| tracker.update}
44
- @callback.call @name if @callback
44
+ @callback.call(all_resources) if @callback
45
45
  sleep @delay
46
46
  end
47
47
  rescue Exception => e
@@ -82,6 +82,13 @@ module FogTracker
82
82
  types.map {|type| type.to_s}
83
83
  end
84
84
 
85
+ # Returns an Array of all this Account's currently tracked Resources
86
+ def all_resources
87
+ (@resource_trackers.collect do |tracker|
88
+ tracker.collection
89
+ end).flatten
90
+ end
91
+
85
92
  private
86
93
 
87
94
  # Creates and returns an Array of ResourceTracker objects -
@@ -0,0 +1,20 @@
1
+ # add an accessor and a method to decorate Fog::Model instances
2
+ # with tracker account information
3
+ module Fog
4
+ class Model
5
+
6
+ # a FogTracker::CollectionTracker
7
+ attr_accessor :fog_collection_tracker
8
+
9
+ # returns a clean copy of the resource's account information
10
+ # from the its collection tracker
11
+ def tracker_account
12
+ if fog_collection_tracker
13
+ fog_collection_tracker.clean_account_data
14
+ else
15
+ Hash.new
16
+ end
17
+ end
18
+
19
+ end
20
+ end
@@ -22,10 +22,11 @@ module FogTracker
22
22
  # Returns an Array of Resources, filtered by +query+
23
23
  def execute(query)
24
24
  acct_pattern, svc_pattern, prov_pattern, col_pattern = parse_query(query)
25
- results_by_account = get_results_by_account(acct_pattern)
26
- results = filter_by_service(results_by_account, svc_pattern)
27
- results = filter_by_provider(results, prov_pattern)
28
- results = filter_by_collection(results, col_pattern)
25
+ filter_by_collection(
26
+ filter_by_provider(
27
+ filter_by_service(get_results_by_account(acct_pattern), svc_pattern),
28
+ prov_pattern),
29
+ col_pattern)
29
30
  end
30
31
 
31
32
  private
@@ -1,11 +1,11 @@
1
1
  module FogTracker
2
2
 
3
- # Tracks a single Fog Resource type for a single Account
3
+ # Tracks a single Fog collection in a single account
4
4
  class ResourceTracker
5
5
 
6
6
  attr_accessor :collection
7
7
 
8
- # Creates an object for tracking a single Fog account
8
+ # Creates an object for tracking a single Fog collection in a single account
9
9
  #
10
10
  # ==== Attributes
11
11
  #
@@ -25,8 +25,26 @@ module FogTracker
25
25
  # instances of the relevant resource type, and saves them as @collection
26
26
  def update
27
27
  @log.info "Polling #{@type} on #{@account_name}..."
28
- @collection = @account_tracker.connection.send(@type)
29
- @log.info "Discovered #{@collection.count} #{@type} on #{@account_name}."
28
+ fog_collection = @account_tracker.connection.send(@type) || Array.new
29
+ @log.info "Fetching #{fog_collection.count} #{@type} on #{@account_name}."
30
+ new_collection = Array.new
31
+ # Here's where most of the network overhead is actually incurred
32
+ fog_collection.each do |resource|
33
+ @log.debug "Fetching resource: #{resource.class} #{resource.identity}"
34
+ resource.fog_collection_tracker = self
35
+ new_collection << resource
36
+ @log.debug "Got resource: #{resource.inspect}"
37
+ end
38
+ @collection = new_collection
39
+ end
40
+
41
+ # Returns a Hash of account information as resource.tracker_account
42
+ # a :name parameter is added, and the :credentials are removed
43
+ def clean_account_data
44
+ @clean_data ||= @account # generate this data only once per res
45
+ @clean_data[:name] = @account_name
46
+ @clean_data[:credentials] = Hash.new
47
+ @clean_data
30
48
  end
31
49
 
32
50
  end
@@ -58,15 +58,18 @@ module FogTracker
58
58
  end
59
59
 
60
60
  # Returns an array of Resources matching the +query_string+
61
+ # Calls any block passed for each resulting resource
61
62
  #
62
63
  # ==== Attributes
63
64
  #
64
65
  # * +query_string+ - a String used to filter the discovered Resources
65
66
  # it might look like: "Account Name::Compute::AWS::servers"
66
67
  def query(query_string)
67
- FogTracker::Query::QueryProcessor.new(
68
+ results = FogTracker::Query::QueryProcessor.new(
68
69
  @trackers, :logger => @log
69
70
  ).execute(query_string)
71
+ (results.each {|r| yield r}) if block_given?
72
+ results
70
73
  end
71
74
  alias :[] :query
72
75
 
@@ -1,3 +1,3 @@
1
1
  module FogTracker
2
- VERSION = "0.1.1"
2
+ VERSION = "0.1.2"
3
3
  end
data/lib/fog_tracker.rb CHANGED
@@ -1,6 +1,7 @@
1
1
  require 'logger'
2
2
 
3
- # Load all ruby files from 'fog_tracker' directory
3
+ # Load all ruby files from the 'fog' and 'fog_tracker' directories
4
+ Dir[File.join(File.dirname(__FILE__), "fog/**/*.rb")].each {|f| require f}
4
5
  Dir[File.join(File.dirname(__FILE__), "fog_tracker/**/*.rb")].each {|f| require f}
5
6
 
6
7
  module FogTracker
@@ -5,7 +5,9 @@ module FogTracker
5
5
  describe ResourceTracker do
6
6
 
7
7
  before(:each) do
8
- @account_tracker = mock_account_tracker
8
+ @account_tracker = AccountTracker.new(
9
+ FAKE_ACCOUNT_NAME, FAKE_ACCOUNT, :logger => LOG
10
+ )
9
11
  @tracker = ResourceTracker.new(
10
12
  FAKE_COLLECTION, @account_tracker
11
13
  )
@@ -20,6 +22,15 @@ module FogTracker
20
22
  @account_tracker.connection.should_receive(FAKE_COLLECTION)
21
23
  @tracker.update
22
24
  end
25
+ it "attaches account information to all resources" do
26
+ account_tracker = mock_account_tracker(5,5)
27
+ tracker = ResourceTracker.new(FAKE_COLLECTION, account_tracker)
28
+ tracker.update
29
+ tracker.collection.each do |resource|
30
+ resource.tracker_account[:name].should == FAKE_ACCOUNT_NAME
31
+ resource.tracker_account[:credentials].should == {}
32
+ end
33
+ end
23
34
  end
24
35
  end
25
36
 
@@ -34,5 +34,30 @@ module FogTracker
34
34
  end
35
35
  end
36
36
 
37
+ describe '#query' do
38
+ it "invokes any passed code block once per resulting Resource" do
39
+ receiver = double "resrouce callback object"
40
+ receiver.stub(:callback)
41
+ @tracker.start
42
+ sleep THREAD_STARTUP_DELAY
43
+ res_count = @tracker.query('*::*::*::*').count
44
+ receiver.should_receive(:callback).exactly(res_count).times
45
+ @tracker.query('*::*::*::*') {|r| receiver.callback(r)}
46
+ end
47
+ end
48
+
49
+ describe '#start' do
50
+ it "invokes the Tracker's callback Proc when an account is updated" do
51
+ receiver = double "account callback object"
52
+ receiver.stub(:callback)
53
+ tracker = Tracker.new(ACCOUNTS, :logger => LOG,
54
+ :callback => Proc.new {|resources| receiver.callback(resources)}
55
+ )
56
+ receiver.should_receive(:callback).exactly(ACCOUNTS.size).times
57
+ tracker.start
58
+ sleep THREAD_STARTUP_DELAY
59
+ end
60
+ end
61
+
37
62
  end
38
63
  end
@@ -32,6 +32,10 @@ module FogTracker
32
32
  module Query
33
33
  QUERY = {} # Used in query_processor_spec.rb
34
34
  end
35
+ class MockFogResource < Fog::Model
36
+ #attr_accessor :tracker_account
37
+ def identity ; "random-resource-id-#{rand 65536}" end
38
+ end
35
39
  end
36
40
 
37
41
  # Create some fake Fog Resource and Collection Classes
@@ -54,8 +58,9 @@ end
54
58
  def mock_account_tracker(num_collections = 1, resources_per_collection = 0)
55
59
  fake_account_tracker = double('mock_account_tracker')
56
60
  fake_account_tracker.stub(:account).and_return(Hash.new)
57
- fake_account_tracker.stub(:name).and_return("fake account tracker")
61
+ fake_account_tracker.stub(:name).and_return(FogTracker::FAKE_ACCOUNT_NAME)
58
62
  fake_account_tracker.stub(:log).and_return(LOG)
63
+ fake_account_tracker.stub(:all_resources).and_return(Array.new)
59
64
  fake_account_tracker.stub(:connection).and_return(mock_fog_connection)
60
65
  # create an array of mock ResourceTrackers
61
66
  trackers = (1..num_collections).map do |class_index|
@@ -70,23 +75,10 @@ end
70
75
 
71
76
  def mock_fog_connection
72
77
  fake_fog_connection = double('mock_fog_connection')
73
- fake_fog_connection.stub(FogTracker::FAKE_COLLECTION).and_return([ 1, 2, 3 ])
74
- fake_fog_connection
75
- end
76
-
77
- def mock_resource_tracker
78
- fake_resource_tracker = double('mock_resource_tracker')
79
- fake_resource_tracker.stub(:update)
80
- fake_resource_tracker
81
- end
82
-
83
- def mock_fog_resource(resource_class)
84
- fake_resource = resource_class.new
85
- fake_collection_class =
86
- fake_resource.stub(:collection).and_return(
87
- collection_class.new
78
+ fake_fog_connection.stub(FogTracker::FAKE_COLLECTION).and_return(
79
+ [ FogTracker::MockFogResource.new, FogTracker::MockFogResource.new ]
88
80
  )
89
- fake_resource
81
+ fake_fog_connection
90
82
  end
91
83
 
92
84
  def mock_resource_tracker(resource_class, number_of_resources = 0)
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fog_tracker
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.1.2
5
5
  prerelease:
6
6
  platform: ruby
7
7
  authors:
@@ -9,11 +9,11 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2012-01-25 00:00:00.000000000Z
12
+ date: 2012-01-27 00:00:00.000000000Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: fog
16
- requirement: &2151921100 !ruby/object:Gem::Requirement
16
+ requirement: &70121482437700 !ruby/object:Gem::Requirement
17
17
  none: false
18
18
  requirements:
19
19
  - - ! '>='
@@ -21,10 +21,10 @@ dependencies:
21
21
  version: '0'
22
22
  type: :runtime
23
23
  prerelease: false
24
- version_requirements: *2151921100
24
+ version_requirements: *70121482437700
25
25
  - !ruby/object:Gem::Dependency
26
26
  name: rake
27
- requirement: &2151920060 !ruby/object:Gem::Requirement
27
+ requirement: &70121482437100 !ruby/object:Gem::Requirement
28
28
  none: false
29
29
  requirements:
30
30
  - - ! '>='
@@ -32,10 +32,10 @@ dependencies:
32
32
  version: '0'
33
33
  type: :development
34
34
  prerelease: false
35
- version_requirements: *2151920060
35
+ version_requirements: *70121482437100
36
36
  - !ruby/object:Gem::Dependency
37
37
  name: rspec
38
- requirement: &2151919280 !ruby/object:Gem::Requirement
38
+ requirement: &70121482436440 !ruby/object:Gem::Requirement
39
39
  none: false
40
40
  requirements:
41
41
  - - ! '>='
@@ -43,10 +43,10 @@ dependencies:
43
43
  version: '0'
44
44
  type: :development
45
45
  prerelease: false
46
- version_requirements: *2151919280
46
+ version_requirements: *70121482436440
47
47
  - !ruby/object:Gem::Dependency
48
48
  name: rdoc
49
- requirement: &2151918500 !ruby/object:Gem::Requirement
49
+ requirement: &70121482435700 !ruby/object:Gem::Requirement
50
50
  none: false
51
51
  requirements:
52
52
  - - ! '>='
@@ -54,10 +54,10 @@ dependencies:
54
54
  version: '0'
55
55
  type: :development
56
56
  prerelease: false
57
- version_requirements: *2151918500
57
+ version_requirements: *70121482435700
58
58
  - !ruby/object:Gem::Dependency
59
59
  name: guard
60
- requirement: &2151917620 !ruby/object:Gem::Requirement
60
+ requirement: &70121482434720 !ruby/object:Gem::Requirement
61
61
  none: false
62
62
  requirements:
63
63
  - - ! '>='
@@ -65,10 +65,10 @@ dependencies:
65
65
  version: '0'
66
66
  type: :development
67
67
  prerelease: false
68
- version_requirements: *2151917620
68
+ version_requirements: *70121482434720
69
69
  - !ruby/object:Gem::Dependency
70
70
  name: guard-rake
71
- requirement: &2151915940 !ruby/object:Gem::Requirement
71
+ requirement: &70121482433280 !ruby/object:Gem::Requirement
72
72
  none: false
73
73
  requirements:
74
74
  - - ! '>='
@@ -76,10 +76,10 @@ dependencies:
76
76
  version: '0'
77
77
  type: :development
78
78
  prerelease: false
79
- version_requirements: *2151915940
79
+ version_requirements: *70121482433280
80
80
  - !ruby/object:Gem::Dependency
81
81
  name: ruby_gntp
82
- requirement: &2151914940 !ruby/object:Gem::Requirement
82
+ requirement: &70121482432400 !ruby/object:Gem::Requirement
83
83
  none: false
84
84
  requirements:
85
85
  - - ! '>='
@@ -87,7 +87,7 @@ dependencies:
87
87
  version: '0'
88
88
  type: :development
89
89
  prerelease: false
90
- version_requirements: *2151914940
90
+ version_requirements: *70121482432400
91
91
  description: This gem peridically polls mutiple cloud computing services using the
92
92
  fog gem, asynchronously updating the state of the resulting collections of Fog Resources.
93
93
  email:
@@ -107,6 +107,7 @@ files:
107
107
  - fog_tracker.gemspec
108
108
  - lib/fog_tracker.rb
109
109
  - lib/fog_tracker/account_tracker.rb
110
+ - lib/fog_tracker/model.rb
110
111
  - lib/fog_tracker/query/query_processor.rb
111
112
  - lib/fog_tracker/resource_tracker.rb
112
113
  - lib/fog_tracker/tracker.rb
@@ -135,7 +136,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
135
136
  version: '0'
136
137
  segments:
137
138
  - 0
138
- hash: 3740439790259369910
139
+ hash: -2556488865982976006
139
140
  required_rubygems_version: !ruby/object:Gem::Requirement
140
141
  none: false
141
142
  requirements:
@@ -144,10 +145,10 @@ required_rubygems_version: !ruby/object:Gem::Requirement
144
145
  version: '0'
145
146
  segments:
146
147
  - 0
147
- hash: 3740439790259369910
148
+ hash: -2556488865982976006
148
149
  requirements: []
149
150
  rubyforge_project: fog_tracker
150
- rubygems_version: 1.8.15
151
+ rubygems_version: 1.8.10
151
152
  signing_key:
152
153
  specification_version: 3
153
154
  summary: Tracks the state of cloud computing resources across multiple accounts with