fog-google 0.0.5 → 0.0.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d34bfdbe9d48d888a666b8097dafa7246fc8ab6f
-  data.tar.gz: 3319d90e425fdf1f9d53d3f3ff06e5b5473f8558
+  metadata.gz: a924fff0c53902ab17cf84ba699c35cfb2139898
+  data.tar.gz: f2a1df01bb2be2dccf652d477c80b38fd20ce86d
 SHA512:
-  metadata.gz: 14805dd5f5e1e0d4411c971b43b6c5cce08692be4a512f5e07c008462f26e6f1dbaa5791110fde190115b5a481c8ef7029a2345c7248c42b156ac48a3ebf0b4a
-  data.tar.gz: a23a1dc7faaa20c0887fe4419a6b6f15d21b96094aa41a31826eb7895ae7295b35d241d6dee14436baf4ca8451be524fb498edfdffc654dc32a1bf1d287a2d81
+  metadata.gz: ab467619990b86e9dd1d9144e928434f9343516a12f589f815879471b841372b2b3b020dea667336d0f5473b8a5f7e813bc5d798a5f72d67208a1fbf89111522
+  data.tar.gz: 1a640d9fd9479034e6445febe59eff84ff22a7033502f4bbf24c9a953210546578a84ccd89fd55707e2c99c9fe17119607b5c5a079aa5b88065624ed3e60745a

data/.gitignore

@@ -12,3 +12,4 @@
 *.o
 *.a
 mkmf.log
+.ruby-version

data/CONTRIBUTING.md

@@ -2,6 +2,8 @@
 
 New contributors are always welcome, and when in doubt please ask questions! We strive to be an open and welcoming community. Please be nice to one another.
 
+I recommend heading over to fog's [CONTRIBUTING](https://github.com/fog/fog/blob/master/CONTRIBUTING.md) and having a look around as well.  It has information and context about the state of the `fog` project as a whole.
+
 ### Coding
 
 * Pick a task:
@@ -9,6 +11,11 @@ New contributors are always welcome, and when in doubt please ask questions! We
   * Review open [issues](https://github.com/fog/fog-google/issues) for things to help on.
   * [Create an issue](https://github.com/fog/fog-google/issues/new) to start a discussion on additions or features.
 * Fork the project, add your changes and tests to cover them in a topic branch.
+  * [Fork](https://github.com/fog/fog-google/fork)
+  * Create your feature branch (`git checkout -b my-new-feature`)
+  * Commit your changes (`git commit -am 'Add some feature'`)
+  * Push to the branch (`git push origin my-new-feature`)
+  * Create a new pull request
 * Commit your changes and rebase against `fog/fog-google` to ensure everything is up to date.
 * [Submit a pull request](https://github.com/fog/fog-google/compare/)
 
@@ -17,4 +24,62 @@ New contributors are always welcome, and when in doubt please ask questions! We
 * Offer feedback on open [issues](https://github.com/fog/fog-google/issues).
 * Organize or volunteer at events.
 
-I recommend heading over to fog's [CONTRIBUTING](https://github.com/fog/fog/blob/master/CONTRIBUTING.md) and having a look around as well.  It has information and context about the state of the `fog` project as a whole.
+## Contributing Code
+
+This document is very much a work in progress.  Sorry about that.
+
+It's worth noting that, if you're looking through the code, and you'd like to know the history of a line, you may not find it in the history of this repository, since most of the code was extracted from [fog/fog](https://github.com/fog/fog).  So, you can look at the history from commit [fog/fog#c596e](https://github.com/fog/fog/tree/c596e710952aa9c90713da3fbfb3027db0608413) backward for more information.
+
+### Development environment
+
+If you're going to be doing any kind of modifications, I highly recommend using [rbenv](https://github.com/sstephenson/rbenv), [ruby-build](https://github.com/sstephenson/ruby-build), (don't forget the [dependencies](https://github.com/sstephenson/ruby-build/wiki#suggested-build-environment)!) and [bundler](http://bundler.io/).
+
+Once you've got that all installed, run
+
+```shell
+$ bundle install
+```
+
+to install the required gems.  You might have to [fight a bit](http://www.nokogiri.org/tutorials/installing_nokogiri.html) to get Nokogiri installed.
+
+Then, you should be ready to go!  If you'd like to drop into an interactive shell, configured with your `:test` credential, use
+
+```shell
+rake console
+```
+
+### Testing
+
+This module is tested with [Minitest](https://github.com/seattlerb/minitest).  Right now, the only tests that exist are live integration tests, found in `test/integration/`.  After completing the installation in the README, (including setting up your credentials and keys,) make sure you have a `:test` credential in `~/.fog`, something like:
+
+```
+test:
+  google_project: my-project
+  google_client_email: xxxxxxxxxxxxx-xxxxxxxxxxxxx@developer.gserviceaccount.com
+  google_key_location: /path/to/my-project-xxxxxxxxxxxxx.p12
+  google_json_key_location: /path/to/my-project-xxxxxxxxxxxxx.json
+```
+
+Note that you need both a `.p12` and a `.json` key file for all the tests to pass.
+
+Then you can run all the live tests:
+
+```shell
+$ rake test
+```
+
+or just one:
+
+```shell
+$ rake test TEST=test/integration/compute/test_servers.rb TESTOPTS="--name=TestServers#test_bootstrap_ssh_destroy"
+```
+
+#### Some notes about the tests as they stand
+
+The live integration tests for resources, (servers, disks, etc.,) have a few components:
+
+- The `TestCollection` **mixin module** lives in `test/helpers/test_collection.rb` and contains the standard tests to run for all resources, (e.g. `test_lifecycle`).  It also calls `cleanup` on the resource's factory during teardown, to make sure that resources are getting destroyed before the next test run.
+- The **factory**, (e.g. `ServersFactory`, in `test/integration/factories/servers_factory.rb`,) automates the creation of resources and/or supplies parameters for explicit creation of resources.  For example, `ServersFactory` initializes a `DisksFactory` to supply disks in order to create servers, and implements the `params` method so that tests can create servers with unique names, correct zones and machine types, and automatically-created disks.  `ServersFactory` inherits the `create` method from `CollectionFactory`, which allows tests to create servers on-demand.
+- The **main test**, (e.g. `TestServers`, in `test/integration/compute/test_servers.rb`,) is the test that actually runs.  It mixes in the `TestCollection` module in order to run the tests in that module, it supplies the `setup` method in which it initializes a `ServersFactory`, and it includes any other tests specific to this collection, (e.g. `test_bootstrap_ssh_destroy`).
+
+If you want to create another resource, you should add live integration tests; all you need to do is create a factory in `test/integration/factories/my_resource_factory.rb` and a main test in `test/integration/compute/test_my_resource.rb` that mixes in `TestCollection`.

data/CONTRIBUTORS.md

@@ -39,3 +39,4 @@
 * Timur Alperovich <timur.alperovich@gmail.com>
 * unknown <bturner_2@pibuk-lp71.pibenchmark.com>
 * Wesley Beary <geemus@gmail.com>
+* Google Inc.

data/README.md

@@ -40,16 +40,33 @@ gem 'fog-google'
 
 And then execute:
 
-    $ bundle
+```shell
+$ bundle
+```
 
 Or install it yourself as:
 
-    $ gem install fog-google
+```shell
+$ gem install fog-google
+```
+
+### Setup
+
+#### Credentials
+
+Follow the [instructions to generate a private key](https://cloud.google.com/storage/docs/authentication#generating-a-private-key).  You can then create a fog credentials file at `~/.fog`, which will look something like this:
+
+```
+my_credential:
+    google_project: my-project
+    google_client_email: xxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@developer.gserviceaccount.com
+    google_json_key_location: /path/to/my-project-xxxxxxxxxxxx.json
+```
+
+#### SSH-ing into instances
+
+If you want to be able to bootstrap SSH-able instances, (using `servers.bootstrap`,) be sure you have a key in `~/.ssh/id_rsa` and `~/.ssh/id_rsa.pub`.
 
 ## Contributing
 
-1. Fork it ( https://github.com/fog/fog-google/fork )
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create a new Pull Request
+See `CONTRIBUTING.md` in this repository.

data/Rakefile

@@ -1,15 +1,13 @@
 require 'bundler/gem_tasks'
 require 'rake/testtask'
-require 'fog/google'
 
 Rake::TestTask.new do |t|
-  t.libs.push %w(spec)
-  t.test_files = FileList['spec/**/*_spec.rb']
-  t.verbose = true
+  t.libs << "test"
+  t.pattern = File.join("test", "**", "test_*.rb")
 end
 
 desc 'Default Task'
-task :default => [ :test, 'test:travis' ]
+task :default => [ 'test:travis' ]
 
 namespace :test do
   mock = ENV['FOG_MOCK'] || 'true'
@@ -32,3 +30,14 @@ namespace :google do
     end
   end
 end
+
+# From http://erniemiller.org/2014/02/05/7-lines-every-gems-rakefile-should-have/
+# with some modification.
+task :console do
+  require 'irb'
+  require 'irb/completion'
+  require 'fog/google'
+  Fog.credential = :test
+  ARGV.clear
+  IRB.start
+end

data/examples/image_create.rb

@@ -2,7 +2,7 @@ def test
   connection = Fog::Compute.new({ :provider => "Google" })
 
   rawdisk = {
-    :source         => nil, # 'http://some_valid_url_to_rootfs_tarball'
+    :source         => nil, # Google Cloud Storage URL pointing to the disk image. (e.g. http://storage.googleapis.com/test/test.tar.gz)
     :container_type => 'TAR',
   }
 

data/examples/launch_micro_instance.rb

@@ -14,7 +14,7 @@ def test
     :machine_type => "f1-micro",
     :zone_name => "us-central1-a",
     :disks => [ disk.get_as_boot_disk(true) ],
-    :user => ENV['USER']
+    :username => ENV['USER']
   }
 
   server = connection.servers.bootstrap params

data/lib/fog/google/compute.rb

@@ -3,7 +3,7 @@ module Fog
     class Google < Fog::Service
       requires :google_project
       recognizes :app_name, :app_version, :google_client_email, :google_key_location, :google_key_string,
-                 :google_client, :google_json_key_location, :google_json_key_string
+                 :google_client, :google_json_key_location, :google_json_key_string, :google_extra_global_projects
 
       GOOGLE_COMPUTE_API_VERSION     = 'v1'
       GOOGLE_COMPUTE_BASE_URL        = 'https://www.googleapis.com/compute/'
@@ -46,6 +46,7 @@ module Fog
       request :list_region_view_resources
       request :list_zone_view_resources
       request :list_target_instances
+      request :list_aggregated_target_instances
 
       request :get_server
       request :get_address
@@ -220,9 +221,11 @@ module Fog
 
       class Mock
         include Fog::Google::Shared
+        attr_reader :extra_global_projects
 
         def initialize(options)
           shared_initialize(options[:google_project], GOOGLE_COMPUTE_API_VERSION, GOOGLE_COMPUTE_BASE_URL)
+          @extra_global_projects = options[:google_extra_global_projects] || []
         end
 
         def self.data(api_version)
@@ -1050,7 +1053,7 @@ module Fog
         include Fog::Google::Shared
 
         attr_accessor :client
-        attr_reader :compute
+        attr_reader :compute, :extra_global_projects
 
         def initialize(options)
           shared_initialize(options[:google_project], GOOGLE_COMPUTE_API_VERSION, GOOGLE_COMPUTE_BASE_URL)
@@ -1059,6 +1062,7 @@ module Fog
           @client = initialize_google_client(options)
           @compute = @client.discovered_api('compute', api_version)
           @resourceviews = @client.discovered_api('resourceviews', 'v1beta1')
+          @extra_global_projects = options[:google_extra_global_projects] || []
         end
       end
 

data/lib/fog/google/models/compute/addresses.rb

@@ -34,6 +34,19 @@ module Fog
           return nil if address.empty?
           new(address.first['addresses'].first)
         end
+
+        def get_by_name(ip_name)
+          names = service.list_aggregated_addresses(:filter => "name eq .*#{ip_name}").body['items']
+          name = names.each_value.select { |region| region.key?('addresses') }
+
+          return nil if name.empty?
+          new(name.first['addresses'].first)
+        end
+
+        def get_by_ip_address_or_name(ip_address_or_name)
+          get_by_ip_address(ip_address_or_name) or get_by_name(ip_address_or_name)
+        end
+
       end
     end
   end

data/lib/fog/google/models/compute/backend_services.rb

@@ -13,10 +13,9 @@ module Fog
         end
 
         def get(identity)
-          response = nil
-          response = service.get_backend_service(identity)
-          return nil if response.nil?
-          new(response.body)
+          if backend_service = service.get_backend_service(identity).body
+            new(backend_service)
+          end
         rescue Fog::Errors::NotFound
           nil
         end

data/lib/fog/google/models/compute/global_forwarding_rules.rb

@@ -13,9 +13,11 @@ module Fog
         end
 
         def get(identity, region='global')
-          response = service.get_global_forwarding_rule(identity, region)
-          return nil if response.nil?
-          new(response.body)
+          if global_forwarding_rule = service.get_global_forwarding_rule(identity, region).body
+            new(global_forwarding_rule)
+          end
+        rescue Fog::Errors::NotFound
+          nil
         end
       end
     end

data/lib/fog/google/models/compute/images.rb

@@ -23,7 +23,7 @@ module Fog
 
         def all
           data = []
-          all_projects = GLOBAL_PROJECTS + [ self.service.project ]
+          all_projects = [ self.service.project ] + global_projects
 
           all_projects.each do |project|
             begin
@@ -44,7 +44,7 @@ module Fog
 
         def get(identity)
           # Search own project before global projects
-          all_projects = [ self.service.project ] + GLOBAL_PROJECTS
+          all_projects = [ self.service.project ] + global_projects
 
           data = nil
           all_projects.each do |project|
@@ -60,6 +60,12 @@ module Fog
           return nil if data.nil?
           new(data)
         end
+
+        private
+
+        def global_projects
+          GLOBAL_PROJECTS + self.service.extra_global_projects
+        end
       end
     end
   end

data/lib/fog/google/models/compute/resource_views.rb

@@ -8,7 +8,7 @@ module Fog
         model Fog::Compute::Google::ResourceView
 
         def all(filters={})
-          if fliters['region'].nil? && filters['zone'].nil?
+          if filters['region'].nil? && filters['zone'].nil?
             data = []
             service.list_regions.body['items'].each do |region|
               data += service.list_region_views(region['name']).body['items'] || []
@@ -17,7 +17,7 @@ module Fog
               data += service.list_zone_views(zone['name']).body['items'] || []
             end
           elsif filters['zone'] 
-            data = service.list_zone_views(fliters['zone']).body['items'] || []
+            data = service.list_zone_views(filters['zone']).body['items'] || []
           else 
             data = service.list_region_views(filters['region']).body['items'] || []
           end

data/lib/fog/google/models/compute/server.rb

@@ -31,6 +31,21 @@ module Fog
         attribute :auto_restart
         attribute :on_host_maintenance
 
+        # Security account scope aliases used by official gcloud utility
+        # List derived from 'gcloud compute instances create --help'
+        GCE_SCOPE_ALIASES = {
+              'compute-ro' => 'compute.readonly',
+              'compute-rw' => 'compute',
+              'computeaccounts-ro' => 'computeaccounts.readonly',
+              'computeaccounts-rw' => 'computeaccounts',
+              'logging-write' => 'logging.write',
+              'sql' => 'sqlservice',
+              'sql-admin' => 'sqlservice.admin',
+              'storage-full' => 'devstorage.full_control',
+              'storage-ro' => 'devstorage.read_only',
+              'storage-rw' => 'devstorage.read_write'
+            }
+
         def image_name=(args)
           Fog::Logger.deprecation("image_name= is no longer used [light_black](#{caller.first})[/]")
         end
@@ -206,6 +221,27 @@ module Fog
           return self.metadata
         end
 
+        def map_service_accounts(scope_array)
+
+          scope_array_expanded = scope_array.map do |e|
+            if GCE_SCOPE_ALIASES[e]
+              GCE_SCOPE_ALIASES[e]
+            else
+              e
+            end
+          end
+
+          scope_array_finalized = scope_array_expanded.map do |e|
+            if e.start_with?("https://")
+              e
+            else
+              "https://www.googleapis.com/auth/#{e}"
+            end
+          end
+
+          return scope_array_finalized
+        end
+
         def reload
           data = service.get_server(self.name, zone_name).body
           self.merge_attributes(data)
@@ -241,9 +277,7 @@ module Fog
             options['serviceAccounts'] = [{
               "kind" => "compute#serviceAccount",
               "email" => "default",
-              "scopes" => service_accounts.map {
-                |w| w.start_with?("https://") ? w : "https://www.googleapis.com/auth/#{w}"
-              }
+              "scopes" => self.map_service_accounts(service_accounts)
             }]
           end
 

data/lib/fog/google/models/compute/target_http_proxies.rb

@@ -13,8 +13,11 @@ module Fog
         end
 
         def get(identity)
-          response = service.get_target_http_proxy(identity)
-          new(response.body) unless response.nil?
+          if target_http_proxy = service.get_target_http_proxy(identity).body
+            new(target_http_proxy)
+          end
+        rescue Fog::Errors::NotFound
+          nil
         end
       end
     end

data/lib/fog/google/models/compute/target_http_proxy.rb

@@ -11,7 +11,7 @@ module Fog
         attribute :id, :aliases => 'id'
         attribute :creation_timestamp, :aliases => 'creationTimestamp'
         attribute :description, :aliases => 'description'
-        attribute :urlMap, :aliases => "urlMap"
+        attribute :urlMap, :aliases => ["urlMap", :url_map]
 
         def save
           requires :name

data/lib/fog/google/models/compute/target_instance.rb

@@ -25,8 +25,8 @@ module Fog
             'instance' => instance,
           }
 
-          data = service.insert_target_instance(name, zone, options).body
-          operation = Fog::Compute::Google::Operations.new(:service => service).get(data['name'])
+          data = service.insert_target_instance(name, zone, options)
+          operation = Fog::Compute::Google::Operations.new(:service => service).get(data.body['name'], data.body['zone'])
           operation.wait_for { !pending? }
           reload
         end