fog-google 1.5.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 95802f48e4fa5a610c96521c65d0a9bb2b33ce54
-  data.tar.gz: 6ae1e7e9a6c7ed8a5a05736857219a6144a4e443
+  metadata.gz: 44ebad93a3aba93d0c9c8465a085e52688d340f7
+  data.tar.gz: 2543bae22b7ad7cc12108b4173c8806f67a3b838
 SHA512:
-  metadata.gz: 9b8f524b235aaf667a4ecaf256a19ca038de9ca6db80e486a29c805dabc73d0cc5b2b95f6a5b17198c268c193a3939825ba6182f6122946ef2ba0ff5bb617c62
-  data.tar.gz: cfa889ab0a734fd5165447f4b7b8234c14287f0e87f2ebae833fa9b34d355f218dcdca6abc89923af196b79c6f6c7ce9ab8c641021699e2a296488e37d144d41
+  metadata.gz: b76ea9d006ba75403d8d7d24f9c42904062dd6a620e42a7cf6d742e6d1f59739a97150230ee246468c0d11db751a908177e62228a3d9cb34dd1ae839ef6c1765
+  data.tar.gz: 64b0fa489bb69896c2abb0130e9671188e22660c006a957f6e788a902b383cb53f6f183be418d4159f196425dddfcb873cc1728584e8654c5b1543d63f7bbb4f

data/.rubocop.yml

@@ -19,6 +19,10 @@ Style/FrozenStringLiteralComment:
 Style/HashSyntax:
   EnforcedStyle: no_mixed_keys
 
+# Do not enforce %i syntax
+Style/SymbolArray:
+  Enabled: false
+
 # HoundCI config
 
 AllCops:

data/CHANGELOG.md

@@ -2,6 +2,41 @@
 All notable changes to this project will be documented in this file.
 The format is loosely based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+## 1.6.0
+
+### User-facing
+
+#### Changed
+
+- \#338 `Fog::Google::SQL` resources are now created and destroyed synchronously by default. 
+You can override it in a standard manner by passing a parameter to async method, e.g.:
+ `Fog::Google::SQL::Instance.create(true)` [temikus]
+- \#367 `Fog::Compute::Google::Server.bootstrap` changes [temikus]
+  - Now creates instances with disks that automatically delete on instance shutdown.
+  - Now creates instances with a public IP address by default.
+
+#### Added
+
+- \#361 `Fog::Compute::Google::Server` now recognises `network_ip` attribute to specify internal IP. [mattimatti]
+
+#### Fixed
+
+- \#338 Fixed SQL Users model workflow [temikus]
+- \#359 Fix whitespace escaping in XML Storage methods [temikus]
+- \#366 Fixing `Server` model to properly accept `:private_key_path` and `:public_key_path` attributes again. [temikus]
+- \#367 `Fog::Compute::Google::Server.bootstrap` parameters are now properly merged with default ones. [tesmikus]
+
+### Development changes
+
+#### Added
+
+- \#338 Major refactor of SQLv1 and SQLv2 tests + a lot of small test fixes/improvements
+   (see PR/commit messages for full set of changes) [temikus]
+
+#### Fixed
+
+- \#363 Fixed flaky Monitoring tests [temikus]
+
 ## 1.5.0
 
 ### User-facing
@@ -110,3 +145,21 @@ PRs that change functionality: #212, #215, #203, #198, #201, #221, #222, #216
 
 Rapid-releasing 0.5.2 due to regression fixed by #190 still present in v0.5.1
 We encourage people using 0.5.1 to upgrade.
+
+## Template to use
+
+## Next release
+
+### User-facing
+
+#### Changed
+
+#### Added
+
+#### Fixed
+
+### Development changes
+
+#### Added
+
+#### Fixed

data/CONTRIBUTING.md

@@ -65,6 +65,13 @@ Then you can run all the live tests:
 $ rake test
 ```
 
+or just one API:
+
+```
+$ rake test:compute
+```
+(See `rake -T` for all available tasks)
+
 or just one file:
 
 ```shell
@@ -79,7 +86,8 @@ For more information on transition, read [#50](https://github.com/fog/fog-google
 
 #### Continuous integration
 
-Currently Google maintains a [Concourse CI](https://concourse-ci.org/) server, running a pipeline defined in `ci` folder. It automatically runs all integration tests against every pull-request marked with `integration` label.
+Currently Google maintains a [Concourse CI](https://concourse-ci.org/) server, running a pipeline defined in `ci` folder. 
+It automatically runs all integration tests against every pull-request marked with `integration` label.
 
 For more information on the pipeline please refer to the [ci README](https://github.com/fog/fog-google/blob/master/ci/README.md).
 

data/MIGRATING.md

@@ -0,0 +1,11 @@
+## fog-google 2.0 -> 3.0:
+
+### SQL:
+- `Fog::Google::SQL::Instance` resources are created synchronously by default. 
+You can override it with `Fog::Google::SQL::Instance.create(true)`
+
+- `Fog::Google::SQL::Instance.destroy` - `async:` named parameter has been replaced with `async` positional parameter.
+You should now call `Fog::Google::SQL::Instance.destroy(false)` to disable async flag, as with other Fog::Google models.
+
+- `Fog::Google::SQL::User.destroy` - `async:` named parameter has been replaced with `async` positional parameter.
+  You should now call `Fog::Google::SQL::User.destroy(false)` to disable async flag, as with other Fog::Google models.

data/README.md

@@ -8,6 +8,7 @@ The main maintainers for the Google sections are @icco, @Temikus and @plribeiro3
 
 **As of v1.0.0, fog-google includes google-api-client as a dependency**
 
+**See [MIGRATING.md](MIGRATING.MD) for migration between major versions**
 
 ## Storage
 
@@ -65,9 +66,22 @@ $ gem install fog-google
 
 ## Testing
 
-The tests in `tests` are deprecated. We are currently working on a migration of tests to `minitest`.
+Integration tests can be kicked off via following rake tasks.
+**Important note:** As those tests are running against real API's YOU WILL BE BILLED.
 
-For your test to be tested with real credentials, a repo maintainer may add the label `integrate` to your PR to run integration tests.
+```
+rake test               # Run all integration tests
+rake test:parallel      # Run all integration tests in parallel
+
+rake test:compute       # Run Compute API tests
+rake test:monitoring    # Run Monitoring API tests
+rake test:pubsub        # Run PubSub API tests
+rake test:sql           # Run SQL API tests
+rake test:storage       # Run Storage API tests
+```
+
+Since some resources can be expensive to test, we have a self-hosted CI server.
+Due to security considerations a repo maintainer needs to add the label `integrate` to kick off the CI.
 
 ## Setup
 

data/ci/pipeline.yml

@@ -1,5 +1,6 @@
 jobs:
   - name: integration-tests-compute
+    serial: true
     plan:
       - get: fog-google-src
         resource: pull-request
@@ -15,8 +16,14 @@ jobs:
           google_project: {{google_project}}
           google_json_key_data: {{google_json_key_data}}
           google_client_email: {{google_client_email}}
-
+        on_failure:
+          put: pull-request
+          params:
+            path: fog-google-src
+            status: failure
+            
   - name: integration-tests-monitoring
+    serial: true
     plan:
       - get: fog-google-src
         resource: pull-request
@@ -32,8 +39,14 @@ jobs:
           google_project: {{google_project}}
           google_json_key_data: {{google_json_key_data}}
           google_client_email: {{google_client_email}}
+        on_failure:
+          put: pull-request
+          params:
+            path: fog-google-src
+            status: failure
 
   - name: integration-tests-pubsub
+    serial: true
     plan:
       - get: fog-google-src
         resource: pull-request
@@ -49,8 +62,14 @@ jobs:
           google_project: {{google_project}}
           google_json_key_data: {{google_json_key_data}}
           google_client_email: {{google_client_email}}
+        on_failure:
+          put: pull-request
+          params:
+            path: fog-google-src
+            status: failure
 
   - name: integration-tests-sql
+    serial: true
     plan:
       - get: fog-google-src
         resource: pull-request
@@ -66,8 +85,14 @@ jobs:
           google_project: {{google_project}}
           google_json_key_data: {{google_json_key_data}}
           google_client_email: {{google_client_email}}
+        on_failure:
+          put: pull-request
+          params:
+            path: fog-google-src
+            status: failure
 
   - name: integration-tests-storage
+    serial: true
     plan:
       - get: fog-google-src
         resource: pull-request
@@ -83,6 +108,11 @@ jobs:
           google_project: {{google_project}}
           google_json_key_data: {{google_json_key_data}}
           google_client_email: {{google_client_email}}
+        on_failure:
+          put: pull-request
+          params:
+            path: fog-google-src
+            status: failure
 
   - name: github-pr-aggregator
     plan:

data/examples/bootstrap.rb

@@ -1,18 +1,27 @@
 # All examples presume that you have a ~/.fog credentials file set up.
 # More info on it can be found here: http://fog.io/about/getting_started.html
+# Code can be ran by simply invoking `ruby bootstrap.rb`
+# Note: this example will require 'net-ssh' gem to be installed
 
 require "bundler"
 Bundler.require(:default, :development)
-# Uncomment this if you want to make real requests to GCE (you _will_ be billed!)
-# WebMock.disable!
 
-def test
-  connection = Fog::Compute.new(:provider => "Google")
+p "Connecting to google..."
+p "======================="
+connection = Fog::Compute.new(:provider => "Google")
 
-  server = connection.servers.bootstrap
+p "Bootstrapping a server..."
+p "========================="
+server = connection.servers.bootstrap
 
-  raise "Could not bootstrap sshable server." unless server.ssh("whoami")
-  raise "Could not delete server." unless server.destroy
-end
+p "Waiting for server to be sshable..."
+p "==================================="
+server.wait_for { sshable? }
 
-test
+p "Trying to send an SSH command..."
+p "================================"
+raise "Could not bootstrap sshable server." unless server.ssh("whoami")
+
+p "Deleting a server..."
+p "===================="
+raise "Could not delete server." unless server.destroy

data/lib/fog/compute/google/models/server.rb

@@ -445,6 +445,11 @@ module Fog
           requires :disks
           requires :zone
 
+          generate_ssh_key_metadata(self.username, self.public_key) if self.public_key
+
+          # XXX HACK This is a relic of 1.0 change that for some reason added those arguments
+          # to `save` method. This is left in place to keep things backwards-compatible
+          # TODO(2.0): Remove arguments from save
           generate_ssh_key_metadata(username, public_key) if public_key
 
           options = attributes.reject { |_, v| v.nil? }
@@ -473,6 +478,10 @@ module Fog
             ]
           end
 
+          if attributes[:network_ip]
+            options[:network_interfaces][0][:network_ip] = attributes[:network_ip]
+          end
+
           data = service.insert_server(name, zone_name, options)
 
           operation = Fog::Compute::Google::Operations

data/lib/fog/compute/google/models/servers.rb

@@ -43,9 +43,6 @@ module Fog
         end
 
         def bootstrap(public_key_path: nil, **opts)
-          user = ENV["USER"]
-          public_key = get_public_key(public_key_path)
-
           name = "fog-#{Time.now.to_i}"
           zone_name = "us-central1-f"
 
@@ -66,17 +63,29 @@ module Fog
             disks = [disk]
           end
 
-          data = opts.merge(
-            :name => name,
-            :zone => zone_name,
-            :disks => disks
-          )
+          # TODO: Remove the network init when #360 is fixed
+          network = { :network => "global/networks/default",
+                      :access_configs => [{ :name => "External NAT",
+                                            :type => "ONE_TO_ONE_NAT" }] }
+
+          # Merge the options with the defaults, overwriting defaults
+          # if an option is provided
+          data = { :name => name,
+                   :zone => zone_name,
+                   :disks => disks,
+                   :network_interfaces => [network],
+                   :public_key => get_public_key(public_key_path),
+                   :username => ENV["USER"] }.merge(opts)
+
           data[:machine_type] = "n1-standard-1" unless data[:machine_type]
 
           server = new(data)
-          server.save(:username => user, :public_key => public_key)
-          # TODO: sshable? was removed, needs to be fixed for tests
-          # server.wait_for { sshable? }
+          server.save
+          server.wait_for { ready? }
+
+          # Set the disk to be autodeleted
+          server.set_disk_auto_delete(true)
+
           server
         end
 
@@ -100,7 +109,7 @@ module Fog
           end
 
           if public_key_path.nil? || public_key_path.empty?
-            raise ArgumentError("cannot bootstrap instance without public key file")
+            raise Fog::Errors::Error.new("Cannot bootstrap instance without a public key")
           end
 
           File.read(File.expand_path(public_key_path))

data/lib/fog/compute/google/models/ssl_certificate.rb

@@ -2,7 +2,7 @@ module Fog
   module Compute
     class Google
       ##
-      # Represents a Subnetwork resource
+      # Represents a SslCertificate resource
       #
       # @see https://cloud.google.com/compute/docs/reference/latest/sslCertificates
       class SslCertificate < Fog::Model

data/lib/fog/google/models/sql/instance.rb

@@ -85,14 +85,21 @@ module Fog
 
         ##
         # Creates a Cloud SQL instance
+        # @param [Boolean] async If the operation must be performed asynchronously
+        #
+        #   This is true by default since SQL instances return Google::Apis::ClientError: invalidState
+        #   whenever an instance is in a transition process (creation, deletion, etc.) which makes it
+        #   hard to operate unless one puts guard clauses on Google::Apis::ClientError everywhere.
+        #
+        #   TODO: Rethink this when API graduates out of beta. (Written as of V1beta4)
         #
         # @return [Fog::Google::SQL::Instance] Instance resource
-        def create
+        def create(async = false)
           requires :identity
 
           data = service.insert_instance(identity, attributes[:tier], attributes)
           operation = Fog::Google::SQL::Operations.new(:service => service).get(data.name)
-          operation.wait_for { !pending? }
+          operation.wait_for { ready? } unless async
           reload
         end
 
@@ -107,11 +114,19 @@ module Fog
         ##
         # Deletes a Cloud SQL instance
         #
-        # @param [Boolean] :async If the operation must be performed asynchronously (true by default)
+        # @param [Boolean] async If the operation must be performed asynchronously (false by default)
+        #   See Fog::Google::SQL::Instance.create on details why default is set this way.
+        #
         # @return [Fog::Google::SQL::Operation] A Operation resource
-        def destroy(async: nil)
+        def destroy(async = false)
           requires :identity
 
+          # TODO(2.0): Add a deprecation warning here, depending on the decision in #27
+          # This is a compatibility fix leftover from breaking named parameter change
+          if async.is_a?(Hash)
+            async = async[:async]
+          end
+
           data = service.delete_instance(identity)
           operation = Fog::Google::SQL::Operations.new(:service => service).get(data.name)
           operation.tap { |o| o.wait_for { ready? } unless async }

data/lib/fog/google/models/sql/user.rb

@@ -15,9 +15,15 @@ module Fog
         attribute :kind
         attribute :project
 
-        def destroy(async: true)
+        def destroy(async = true)
           requires :instance, :name, :host
 
+          # TODO(2.0): Add a deprecation warning here, depending on the decision in #27
+          # This is a compatibility fix leftover from breaking named parameter change
+          if async.is_a?(Hash)
+            async = async[:async]
+          end
+
           resp = service.delete_user(instance, host, name)
           operation = Fog::Google::SQL::Operations.new(:service => service).get(resp.name)
           operation.wait_for { ready? } unless async
@@ -30,9 +36,9 @@ module Fog
           data = attributes
           data[:password] = password unless password.nil?
           if etag.nil?
-            resp = service.update_user(instance, data)
-          else
             resp = service.insert_user(instance, data)
+          else
+            resp = service.update_user(instance, data)
           end
 
           operation = Fog::Google::SQL::Operations.new(:service => service).get(resp.name)

data/lib/fog/google/requests/sql/insert_instance.rb

@@ -7,8 +7,8 @@ module Fog
       # @see https://cloud.google.com/sql/docs/mysql/admin-api/v1beta4/instances/insert
 
       class Real
-        def insert_instance(name, tier, instance = {})
-          instance = ::Google::Apis::SqladminV1beta4::DatabaseInstance.new(instance)
+        def insert_instance(name, tier, options = {})
+          instance = ::Google::Apis::SqladminV1beta4::DatabaseInstance.new(options)
           instance.name = name
           instance.settings = ::Google::Apis::SqladminV1beta4::Settings.new(instance.settings || {})
           instance.settings.tier = tier