google-cloud-bigquery 1.38.1 → 1.42.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b0548a7d1c7cdbc387d819573b26adca5ea813c3ffda5d9a585f5cdb9e359a59
-  data.tar.gz: 42c818041de0fe87d32c9a559dba2c2ee7acfe441fc97fde61ccb6ebf9407c9d
+  metadata.gz: 779c0f4690b370db6eb7811fa6fb8dbca101dff34ff478dcdc51ed1e9c83c083
+  data.tar.gz: d1456024ae30a18c93c05d6d8be7eef88dd3f0610cdbf816cf0c46322c9574be
 SHA512:
-  metadata.gz: abb28b6af3f0b30b2b58dbde4a8b0727419d802cf5144b62bd310295bbdffd3a2dc9dd03c11cbabb2792a440a33f54c803b937659c19d4e65973d0e0dc53862f
-  data.tar.gz: 832ef3b337a073041d6df5de3f4bb1fab20f1dd6ab1c3dcb20078a710bcc1011fec2f8a574fa9e81b9dc391218115848430ec58acb4897b7bb772cf4614fff86
+  metadata.gz: 6a5da0e2565b0c224f46a6330d138c8fb35831e874ebc2364fd3b91578d400354011d1e9df7d2986dbe2398f610776b130b7eb9305af8282bf3a6265dd84aa05
+  data.tar.gz: 0d14faa0d89ba6a114e247c362f8a5d4775ae4dc19de1aad0a64083da7e41fc93ac810e876b434056340b27412bb08063af05edf33355800ca1c528ff02f437b

data/AUTHENTICATION.md

@@ -106,15 +106,6 @@ To configure your system for this, simply:
 **NOTE:** This is _not_ recommended for running in production. The Cloud SDK
 *should* only be used during development.
 
-[gce-how-to]: https://cloud.google.com/compute/docs/authentication#using
-[dev-console]: https://console.cloud.google.com/project
-
-[enable-apis]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/enable-apis.png
-
-[create-new-service-account]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/create-new-service-account.png
-[create-new-service-account-existing-keys]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/create-new-service-account-existing-keys.png
-[reuse-service-account]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/reuse-service-account.png
-
 ## Creating a Service Account
 
 Google Cloud requires a **Project ID** and **Service Account Credentials** to
@@ -124,31 +115,22 @@ connect to most services with google-cloud-bigquery.
 If you are not running this client on Google Compute Engine, you need a Google
 Developers service account.
 
-1. Visit the [Google Developers Console][dev-console].
+1. Visit the [Google Cloud Console](https://console.cloud.google.com/project).
 1. Create a new project or click on an existing project.
-1. Activate the slide-out navigation tray and select **API Manager**. From
+1. Activate the menu in the upper left and select **APIs & Services**. From
    here, you will enable the APIs that your application requires.
 
-   ![Enable the APIs that your application requires][enable-apis]
-
    *Note: You may need to enable billing in order to use these services.*
 
 1. Select **Credentials** from the side navigation.
 
-   You should see a screen like one of the following.
-
-   ![Create a new service account][create-new-service-account]
-
-   ![Create a new service account With Existing Keys][create-new-service-account-existing-keys]
-
-   Find the "Add credentials" drop down and select "Service account" to be
-   guided through downloading a new JSON key file.
-
-   If you want to re-use an existing service account, you can easily generate a
-   new key file. Just select the account you wish to re-use, and click "Generate
-   new JSON key":
+   Find the "Create credentials" drop down near the top of the page, and select
+   "Service account" to be guided through downloading a new JSON key file.
 
-   ![Re-use an existing service account][reuse-service-account]
+   If you want to re-use an existing service account, you can easily generate 
+   a new key file. Just select the account you wish to re-use click the pencil
+   tool on the right side to edit the service account, select the **Keys** tab,
+   and then select **Add Key**.
 
    The key file you download will be used by this library to authenticate API
    requests and should be stored in a secure location.

data/CHANGELOG.md

@@ -1,5 +1,33 @@
 # Release History
 
+### 1.42.0 (2023-01-15)
+
+#### Features
+
+* Added support for authorized dataset ([#19442](https://github.com/googleapis/google-cloud-ruby/issues/19442)) 
+* Added support for tags in dataset ([#19350](https://github.com/googleapis/google-cloud-ruby/issues/19350)) 
+
+### 1.41.0 (2023-01-05)
+
+#### Features
+
+* Add support for partial projection of table metadata 
+#### Bug Fixes
+
+* Fix querying of array of structs in named parameters ([#19466](https://github.com/googleapis/google-cloud-ruby/issues/19466)) 
+
+### 1.40.0 (2022-12-14)
+
+#### Features
+
+* support table snapshot and clone ([#19354](https://github.com/googleapis/google-cloud-ruby/issues/19354)) 
+
+### 1.39.0 (2022-07-27)
+
+#### Features
+
+* Update minimum Ruby version to 2.6 ([#18871](https://github.com/googleapis/google-cloud-ruby/issues/18871)) 
+
 ### 1.38.1 / 2022-01-13
 
 #### Bug Fixes

data/LOGGING.md

@@ -4,7 +4,7 @@ To enable logging for this library, set the logger for the underlying [Google
 API
 Client](https://github.com/google/google-api-ruby-client/blob/master/README.md#logging)
 library. The logger that you set may be a Ruby stdlib
-[`Logger`](https://ruby-doc.org/stdlib/libdoc/logger/rdoc/Logger.html) as
+[`Logger`](https://ruby-doc.org/current/stdlibs/logger/Logger.html) as
 shown below, or a
 [`Google::Cloud::Logging::Logger`](https://googleapis.dev/ruby/google-cloud-logging/latest)
 that will write logs to [Stackdriver

data/OVERVIEW.md

@@ -276,7 +276,7 @@ To follow along with these examples, you will need to set up billing on the
 [Google Developers Console](https://console.developers.google.com).
 
 In addition to CSV, data can be imported from files that are formatted as
-[Newline-delimited JSON](http://jsonlines.org/),
+[Newline-delimited JSON](https://jsonlines.org/),
 [Avro](http://avro.apache.org/),
 [ORC](https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-orc),
 [Parquet](https://parquet.apache.org/) or from a Google Cloud Datastore backup.

data/lib/google/cloud/bigquery/convert.rb

@@ -54,10 +54,9 @@ module Google
         end
 
         def self.format_row row, fields
-          row_pairs = fields.zip(row[:f]).map do |f, v|
+          fields.zip(row[:f]).to_h do |f, v|
             [f.name.to_sym, format_value(v, f)]
           end
-          Hash[row_pairs]
         end
 
         # rubocop:disable all
@@ -123,10 +122,9 @@ module Google
             array_values = json_value.map { |v| to_query_param_value v, type }
             Google::Apis::BigqueryV2::QueryParameterValue.new array_values: array_values
           when Hash
-            struct_pairs = json_value.map do |k, v|
+            struct_values = json_value.to_h do |k, v|
               [String(k), to_query_param_value(v, type)]
             end
-            struct_values = Hash[struct_pairs]
             Google::Apis::BigqueryV2::QueryParameterValue.new struct_values: struct_values
           else
             # Everything else is converted to a string, per the API expectations.
@@ -239,7 +237,7 @@ module Google
             type = extract_array_type type
             value.map { |x| to_json_value x, type }
           elsif Hash === value
-            Hash[value.map { |k, v| [k.to_s, to_json_value(v, type)] }]
+            value.to_h { |k, v| [k.to_s, to_json_value(v, type)] }
           else
             value
           end
@@ -256,17 +254,17 @@ module Google
 
         ##
         # Lists are specified by providing the type code in an array. For example, an array of integers are specified as
-        # `[:INT64]`. Extracts the symbol.
+        # `[:INT64]`. Extracts the symbol/hash.
         def self.extract_array_type type
           return nil if type.nil?
-          unless type.is_a?(Array) && type.count == 1 && type.first.is_a?(Symbol)
-            raise ArgumentError, "types Array #{type.inspect} should include only a single symbol element."
+          unless type.is_a?(Array) && type.count == 1 && (type.first.is_a?(Symbol) || type.first.is_a?(Hash))
+            raise ArgumentError, "types Array #{type.inspect} should include only a single symbol or hash element."
           end
           type.first
         end
 
         def self.to_json_row row
-          Hash[row.map { |k, v| [k.to_s, to_json_value(v)] }]
+          row.to_h { |k, v| [k.to_s, to_json_value(v)] }
         end
 
         def self.resolve_legacy_sql standard_sql, legacy_sql

data/lib/google/cloud/bigquery/copy_job.rb

@@ -17,6 +17,24 @@ require "google/cloud/bigquery/encryption_configuration"
 module Google
   module Cloud
     module Bigquery
+      module OperationType
+        # Different operation types supported in table copy job.
+        # https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#operationtype
+
+        # The source and destination table have the same table type.
+        COPY = "COPY".freeze
+
+        # The source table type is TABLE and the destination table type is SNAPSHOT.
+        SNAPSHOT = "SNAPSHOT".freeze
+
+        # The source table type is SNAPSHOT and the destination table type is TABLE.
+        RESTORE = "RESTORE".freeze
+
+        # The source and destination table have the same table type, but only bill for
+        # unique data.
+        CLONE = "CLONE".freeze
+      end
+
       ##
       # # CopyJob
       #
@@ -46,23 +64,35 @@ module Google
         # The table from which data is copied. This is the table on
         # which {Table#copy_job} was called.
         #
+        # @param [String] view Specifies the view that determines which table information is returned.
+        #   By default, basic table information and storage statistics (STORAGE_STATS) are returned.
+        #   Accepted values include `:unspecified`, `:basic`, `:storage`, and
+        #   `:full`. For more information, see [BigQuery Classes](@todo: Update the link).
+        #   The default value is the `:unspecified` view type.
+        #
         # @return [Table] A table instance.
         #
-        def source
+        def source view: nil
           table = @gapi.configuration.copy.source_table
           return nil unless table
-          retrieve_table table.project_id, table.dataset_id, table.table_id
+          retrieve_table table.project_id, table.dataset_id, table.table_id, metadata_view: view
         end
 
         ##
         # The table to which data is copied.
         #
+        # @param [String] view Specifies the view that determines which table information is returned.
+        #   By default, basic table information and storage statistics (STORAGE_STATS) are returned.
+        #   Accepted values include `:unspecified`, `:basic`, `:storage`, and
+        #   `:full`. For more information, see [BigQuery Classes](@todo: Update the link).
+        #   The default value is the `:unspecified` view type.
+        #
         # @return [Table] A table instance.
         #
-        def destination
+        def destination view: nil
           table = @gapi.configuration.copy.destination_table
           return nil unless table
-          retrieve_table table.project_id, table.dataset_id, table.table_id
+          retrieve_table table.project_id, table.dataset_id, table.table_id, metadata_view: view
         end
 
         ##
@@ -157,7 +187,8 @@ module Google
             job_ref = service.job_ref_from options[:job_id], options[:prefix]
             copy_cfg = Google::Apis::BigqueryV2::JobConfigurationTableCopy.new(
               source_table:      source,
-              destination_table: target
+              destination_table: target,
+              operation_type:    options[:operation_type]
             )
             req = Google::Apis::BigqueryV2::Job.new(
               job_reference: job_ref,

data/lib/google/cloud/bigquery/dataset/access.rb

@@ -61,8 +61,10 @@ module Google
             "user"           => :user_by_email,
             "user_by_email"  => :user_by_email,
             "userByEmail"    => :user_by_email,
-            "view"           => :view
+            "view"           => :view,
+            "dataset"        => :dataset
           }.freeze
+          attr_reader :rules
 
           # @private
           GROUPS = {
@@ -267,6 +269,44 @@ module Google
             add_access_view view
           end
 
+          ##
+          # Add reader access to a dataset.
+          #
+          # @param [Google::Cloud::Bigquery::DatasetAccessEntry, Hash<String,String> ] dataset A DatasetAccessEntry
+          #   or a Hash object. Required
+          #
+          # @example
+          #   require "google/cloud/bigquery"
+          #
+          #   bigquery = Google::Cloud::Bigquery.new
+          #   dataset = bigquery.dataset "my_dataset"
+          #   other_dataset = bigquery.dataset "my_other_dataset", skip_lookup: true
+          #
+          #   params = {
+          #     dataset_id: other_dataset.dataset_id,
+          #     project_id: other_dataset.project_id,
+          #     target_types: ["VIEWS"]
+          #   }
+          #
+          #   dataset.access do |access|
+          #     access.add_reader_dataset params
+          #   end
+          #
+          # @example
+          #   require "google/cloud/bigquery"
+          #
+          #   bigquery = Google::Cloud::Bigquery.new
+          #   dataset = bigquery.dataset "my_dataset"
+          #   other_dataset = bigquery.dataset "my_other_dataset", skip_lookup: true
+          #
+          #   dataset.access do |access|
+          #     access.add_reader_dataset other_dataset.access_entry(target_types: ["VIEWS"])
+          #   end
+          #
+          def add_reader_dataset dataset
+            add_access_dataset dataset
+          end
+
           ##
           # Add writer access to a user.
           #
@@ -610,6 +650,44 @@ module Google
             remove_access_view view
           end
 
+          ##
+          # Removes reader access of a dataset.
+          #
+          # @param [Google::Cloud::Bigquery::DatasetAccessEntry, Hash<String,String> ] dataset A DatasetAccessEntry
+          #   or a Hash object. Required
+          #
+          # @example
+          #   require "google/cloud/bigquery"
+          #
+          #   bigquery = Google::Cloud::Bigquery.new
+          #   dataset = bigquery.dataset "my_dataset"
+          #   other_dataset = bigquery.dataset "my_other_dataset", skip_lookup: true
+          #
+          #   params = {
+          #     dataset_id: other_dataset.dataset_id,
+          #     project_id: other_dataset.project_id,
+          #     target_types: ["VIEWS"]
+          #   }
+          #
+          #   dataset.access do |access|
+          #     access.remove_reader_dataset params
+          #   end
+          #
+          # @example
+          #   require "google/cloud/bigquery"
+          #
+          #   bigquery = Google::Cloud::Bigquery.new
+          #   dataset = bigquery.dataset "my_dataset"
+          #   other_dataset = bigquery.dataset "my_other_dataset", skip_lookup: true
+          #
+          #   dataset.access do |access|
+          #     access.remove_reader_dataset other_dataset.access_entry(target_types: ["VIEWS"])
+          #   end
+          #
+          def remove_reader_dataset dataset
+            remove_access_dataset dataset
+          end
+
           ##
           # Remove writer access from a user.
           #
@@ -951,6 +1029,40 @@ module Google
             lookup_access_view view
           end
 
+          ##
+          # Checks reader access for a dataset.
+          #
+          # @param [Google::Cloud::Bigquery::DatasetAccessEntry, Hash<String,String> ] dataset A DatasetAccessEntry
+          #   or a Hash object. Required
+          #
+          # @example
+          #   require "google/cloud/bigquery"
+          #
+          #   bigquery = Google::Cloud::Bigquery.new
+          #   dataset = bigquery.dataset "my_dataset"
+          #   other_dataset = bigquery.dataset "my_other_dataset", skip_lookup: true
+          #
+          #   params = {
+          #     dataset_id: other_dataset.dataset_id,
+          #     project_id: other_dataset.project_id,
+          #     target_types: ["VIEWS"]
+          #   }
+          #
+          #   dataset.access.reader_dataset? params
+          #
+          # @example
+          #   require "google/cloud/bigquery"
+          #
+          #   bigquery = Google::Cloud::Bigquery.new
+          #   dataset = bigquery.dataset "my_dataset"
+          #   other_dataset = bigquery.dataset "my_other_dataset", skip_lookup: true
+          #
+          #   dataset.access.reader_dataset? other_dataset.access_entry(target_types: ["VIEWS"])
+          #
+          def reader_dataset? dataset
+            lookup_access_dataset dataset
+          end
+
           ##
           # Checks writer access for a user.
           #
@@ -1184,6 +1296,18 @@ module Google
             end
           end
 
+          # @private
+          #
+          # Checks the type of user input and converts it to acceptable format.
+          #
+          def validate_dataset dataset
+            if dataset.is_a? Google::Apis::BigqueryV2::DatasetAccessEntry
+              dataset
+            else
+              Service.dataset_access_entry_from_hash dataset
+            end
+          end
+
           # @private
           def add_access_role_scope_value role, scope, value
             role = validate_role role
@@ -1218,6 +1342,17 @@ module Google
             @rules << Google::Apis::BigqueryV2::Dataset::Access.new(**opts)
           end
 
+          # @private
+          def add_access_dataset dataset
+            # scope is dataset, make sure value is in the right format
+            value = validate_dataset dataset
+            # Remove existing rule for input dataset, if any
+            @rules.reject!(&find_by_scope_and_resource_ref(:dataset, value))
+            # Add new rule for this role, scope, and value
+            opts = { dataset: value }
+            @rules << Google::Apis::BigqueryV2::Dataset::Access.new(**opts)
+          end
+
           # @private
           def remove_access_role_scope_value role, scope, value
             role = validate_role role
@@ -1244,6 +1379,14 @@ module Google
             @rules.reject!(&find_by_scope_and_resource_ref(:view, value))
           end
 
+          # @private
+          def remove_access_dataset dataset
+            # scope is dataset, make sure value is in the right format
+            value = validate_dataset dataset
+            # Remove existing rule for input dataset, if any
+            @rules.reject!(&find_by_scope_and_resource_ref(:dataset, value))
+          end
+
           # @private
           def lookup_access_role_scope_value role, scope, value
             role = validate_role role
@@ -1268,6 +1411,14 @@ module Google
             !(!@rules.detect(&find_by_scope_and_resource_ref(:view, value)))
           end
 
+          # @private
+          def lookup_access_dataset dataset
+            # scope is dataset, make sure value is in the right format
+            value = validate_dataset dataset
+            # Detect existing rule for input dataset, if any
+            !(!@rules.detect(&find_by_scope_and_resource_ref(:dataset, value)))
+          end
+
           # @private
           def find_by_role_and_scope_and_value role, scope, value
             lambda do |a|

data/lib/google/cloud/bigquery/dataset/tag.rb

@@ -0,0 +1,67 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+require "google/apis/bigquery_v2"
+
+module Google
+  module Cloud
+    module Bigquery
+      class Dataset
+        ##
+        # A global tag managed by Resource Manager.
+        #
+        # @see https://cloud.google.com/iam/docs/tags-access-control#definitions
+        #
+        class Tag
+          ##
+          # @private The Google API Client object.
+          attr_accessor :gapi
+
+          ##
+          # @private Create an empty Tag object.
+          def initialize
+            @gapi = Google::Apis::BigqueryV2::Dataset::Tag.new
+          end
+
+          ##
+          # The namespaced friendly name of the tag key, e.g. "12345/environment" where
+          # 12345 is org id.
+          #
+          # @return [String]
+          #
+          def tag_key
+            @gapi.tag_key
+          end
+
+          ##
+          # The friendly short name of the tag value, e.g. "production".
+          #
+          # @return [String]
+          #
+          def tag_value
+            @gapi.tag_value
+          end
+
+          ##
+          # @private Google API Client object.
+          def self.from_gapi gapi
+            new_tag = new
+            new_tag.instance_variable_set :@gapi, gapi
+            new_tag
+          end
+        end
+      end
+    end
+  end
+end