google-cloud-bigquery-storage 1.3.0 → 1.4.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (6) hide show
  1. checksums.yaml +4 -4
  2. data/AUTHENTICATION.md +72 -101
  3. data/README.md +3 -3
  4. data/lib/google/cloud/bigquery/storage/version.rb +1 -1
  5. data/lib/google/cloud/bigquery/storage.rb +15 -14
  6. metadata +5 -5
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: bb96a7402788bdcb37694cecf64b0a8ad01320e0d47fd8df5ae9767e29a1244f
4
- data.tar.gz: a5b91371a509e9f50969a9e3917030b6399a3adc8bf8ed2325e8e1e5f5a5cd1a
3
+ metadata.gz: 5e8e76775cc87e7b22e5560a5cb189ad4a91f2cf8204230f744aef81df8bbe77
4
+ data.tar.gz: e1c6d339f95ce4e2933536b4931ea62bd721819eee1fd7c2f3d4dc4c304068b5
5
5
  SHA512:
6
- metadata.gz: aa6da9f0a46a49e4c2acefb090ae28d61096361cc084016b5df5c22bd5969941b28109def67107936745c6ca046888248a2503a1da9933c5f4cef3becde35b49
7
- data.tar.gz: bac77ac81006dd86112a4cb435c94ee7d5fd76cd14860410ae0c8c50d6097331fe2db87d3c15325be7c647484d367f27b5bcd9d4b2e10441221542189872de6a
6
+ metadata.gz: 8ac0577ab09f5191721e551cd0ca26af257306755653d32e634bb77195bdf92f4c0507f7c4b12b8b148a9622ef9fbeddf20c276a44a7d407efcd84c751a8ca00
7
+ data.tar.gz: 1a72b219bdb71d6f30ad85a2d9648c400397f8d13677eb77ee7ad32df304a31017ef3b8b3aa2445f89243a4208407c358c74c77aec82db89367915a038186b9e
data/AUTHENTICATION.md CHANGED
@@ -1,151 +1,122 @@
1
1
  # Authentication
2
2
 
3
- In general, the google-cloud-bigquery-storage library uses
4
- [Service Account](https://cloud.google.com/iam/docs/creating-managing-service-accounts)
5
- credentials to connect to Google Cloud services. When running within
6
- [Google Cloud Platform environments](#google-cloud-platform-environments) the
7
- credentials will be discovered automatically. When running on other
8
- environments, the Service Account credentials can be specified by providing the
9
- path to the
10
- [JSON keyfile](https://cloud.google.com/iam/docs/managing-service-account-keys)
11
- for the account (or the JSON itself) in
12
- [environment variables](#environment-variables). Additionally, Cloud SDK
13
- credentials can also be discovered automatically, but this is only recommended
14
- during development.
3
+ The recommended way to authenticate to the google-cloud-bigquery-storage library is to use
4
+ [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials).
5
+ To review all of your authentication options, see [Credentials lookup](#credential-lookup).
15
6
 
16
7
  ## Quickstart
17
8
 
18
- 1. [Create a service account and credentials](#creating-a-service-account).
19
- 2. Set the [environment variable](#environment-variables).
9
+ The following example shows how to set up authentication for a local development
10
+ environment with your user credentials.
20
11
 
21
- ```sh
22
- export BIGQUERY_STORAGE_CREDENTIALS=path/to/keyfile.json
23
- ```
24
-
25
- 3. Initialize the client.
12
+ **NOTE:** This method is _not_ recommended for running in production. User credentials
13
+ should be used only during development.
26
14
 
27
- ```ruby
28
- require "google/cloud/bigquery/storage"
15
+ 1. [Download and install the Google Cloud CLI](https://cloud.google.com/sdk).
16
+ 2. Set up a local ADC file with your user credentials:
29
17
 
30
- client = Google::Cloud::Bigquery::Storage.big_query_read
18
+ ```sh
19
+ gcloud auth application-default login
31
20
  ```
32
21
 
33
- ## Credential Lookup
34
-
35
- The google-cloud-bigquery-storage library aims to make authentication
36
- as simple as possible, and provides several mechanisms to configure your system
37
- without requiring **Service Account Credentials** directly in code.
38
-
39
- **Credentials** are discovered in the following order:
40
-
41
- 1. Specify credentials in method arguments
42
- 2. Specify credentials in configuration
43
- 3. Discover credentials path in environment variables
44
- 4. Discover credentials JSON in environment variables
45
- 5. Discover credentials file in the Cloud SDK's path
46
- 6. Discover GCP credentials
47
-
48
- ### Google Cloud Platform environments
22
+ 3. Write code as if already authenticated.
49
23
 
50
- When running on Google Cloud Platform (GCP), including Google Compute Engine
51
- (GCE), Google Kubernetes Engine (GKE), Google App Engine (GAE), Google Cloud
52
- Functions (GCF) and Cloud Run, **Credentials** are discovered automatically.
53
- Code should be written as if already authenticated.
24
+ For more information about setting up authentication for a local development environment, see
25
+ [Set up Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-dev).
54
26
 
55
- ### Environment Variables
27
+ ## Credential Lookup
56
28
 
57
- The **Credentials JSON** can be placed in environment variables instead of
58
- declaring them directly in code. Each service has its own environment variable,
59
- allowing for different service accounts to be used for different services. (See
60
- the READMEs for the individual service gems for details.) The path to the
61
- **Credentials JSON** file can be stored in the environment variable, or the
62
- **Credentials JSON** itself can be stored for environments such as Docker
63
- containers where writing files is difficult or not encouraged.
29
+ The google-cloud-bigquery-storage library provides several mechanisms to configure your system.
30
+ Generally, using Application Default Credentials to facilitate automatic
31
+ credentials discovery is the easist method. But if you need to explicitly specify
32
+ credentials, there are several methods available to you.
64
33
 
65
- The environment variables that google-cloud-bigquery-storage
66
- checks for credentials are configured on the service Credentials class (such as
67
- `::Google::Cloud::Bigquery::Storage::V1::BigQueryRead::Credentials`):
34
+ Credentials are accepted in the following ways, in the following order or precedence:
68
35
 
69
- * `BIGQUERY_STORAGE_CREDENTIALS` - Path to JSON file, or JSON contents
70
- * `BIGQUERY_STORAGE_KEYFILE` - Path to JSON file, or JSON contents
71
- * `GOOGLE_CLOUD_CREDENTIALS` - Path to JSON file, or JSON contents
72
- * `GOOGLE_CLOUD_KEYFILE` - Path to JSON file, or JSON contents
73
- * `GOOGLE_APPLICATION_CREDENTIALS` - Path to JSON file
36
+ 1. Credentials specified in method arguments
37
+ 2. Credentials specified in configuration
38
+ 3. Credentials pointed to or included in environment variables
39
+ 4. Credentials found in local ADC file
40
+ 5. Credentials returned by the metadata server for the attached service account (GCP)
74
41
 
75
- ```ruby
76
- require "google/cloud/bigquery/storage"
77
-
78
- ENV["BIGQUERY_STORAGE_CREDENTIALS"] = "path/to/keyfile.json"
42
+ ### Configuration
79
43
 
80
- client = Google::Cloud::Bigquery::Storage.big_query_read
81
- ```
44
+ You can configure a path to a JSON credentials file, either for an individual client object or
45
+ globally, for all client objects. The JSON file can contain credentials created for
46
+ [workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation),
47
+ [workforce identity federation](https://cloud.google.com/iam/docs/workforce-identity-federation), or a
48
+ [service account key](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-key).
82
49
 
83
- ### Configuration
50
+ Note: Service account keys are a security risk if not managed correctly. You should
51
+ [choose a more secure alternative to service account keys](https://cloud.google.com/docs/authentication#auth-decision-tree)
52
+ whenever possible.
84
53
 
85
- The path to the **Credentials JSON** file can be configured instead of storing
86
- it in an environment variable. Either on an individual client initialization:
54
+ To configure a credentials file for an individual client initialization:
87
55
 
88
56
  ```ruby
89
57
  require "google/cloud/bigquery/storage"
90
58
 
91
59
  client = Google::Cloud::Bigquery::Storage.big_query_read do |config|
92
- config.credentials = "path/to/keyfile.json"
60
+ config.credentials = "path/to/credentialfile.json"
93
61
  end
94
62
  ```
95
63
 
96
- Or globally for all clients:
64
+ To configure a credentials file globally for all clients:
97
65
 
98
66
  ```ruby
99
67
  require "google/cloud/bigquery/storage"
100
68
 
101
69
  Google::Cloud::Bigquery::Storage.configure do |config|
102
- config.credentials = "path/to/keyfile.json"
70
+ config.credentials = "path/to/credentialfile.json"
103
71
  end
104
72
 
105
73
  client = Google::Cloud::Bigquery::Storage.big_query_read
106
74
  ```
107
75
 
108
- ### Cloud SDK
76
+ ### Environment Variables
109
77
 
110
- This option allows for an easy way to authenticate during development. If
111
- credentials are not provided in code or in environment variables, then Cloud SDK
112
- credentials are discovered.
78
+ You can also use an environment variable to provide a JSON credentials file.
79
+ The environment variable can contain a path to the credentials file or, for
80
+ environments such as Docker containers where writing files is not encouraged,
81
+ you can include the credentials file itself.
113
82
 
114
- To configure your system for this, simply:
83
+ The JSON file can contain credentials created for
84
+ [workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation),
85
+ [workforce identity federation](https://cloud.google.com/iam/docs/workforce-identity-federation), or a
86
+ [service account key](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-key).
115
87
 
116
- 1. [Download and install the Cloud SDK](https://cloud.google.com/sdk)
117
- 2. Authenticate using OAuth 2.0 `$ gcloud auth login`
118
- 3. Write code as if already authenticated.
88
+ Note: Service account keys are a security risk if not managed correctly. You should
89
+ [choose a more secure alternative to service account keys](https://cloud.google.com/docs/authentication#auth-decision-tree)
90
+ whenever possible.
91
+
92
+ The environment variables that google-cloud-bigquery-storage
93
+ checks for credentials are:
119
94
 
120
- **NOTE:** This is _not_ recommended for running in production. The Cloud SDK
121
- *should* only be used during development.
95
+ * `GOOGLE_CLOUD_CREDENTIALS` - Path to JSON file, or JSON contents
96
+ * `GOOGLE_APPLICATION_CREDENTIALS` - Path to JSON file
122
97
 
123
- ## Creating a Service Account
98
+ ```ruby
99
+ require "google/cloud/bigquery/storage"
124
100
 
125
- Google Cloud requires **Service Account Credentials** to
126
- connect to the APIs. You will use the **JSON key file** to
127
- connect to most services with google-cloud-bigquery-storage.
101
+ ENV["GOOGLE_APPLICATION_CREDENTIALS"] = "path/to/credentialfile.json"
128
102
 
129
- If you are not running this client within
130
- [Google Cloud Platform environments](#google-cloud-platform-environments), you
131
- need a Google Developers service account.
103
+ client = Google::Cloud::Bigquery::Storage.big_query_read
104
+ ```
132
105
 
133
- 1. Visit the [Google Cloud Console](https://console.cloud.google.com/project).
134
- 2. Create a new project or click on an existing project.
135
- 3. Activate the menu in the upper left and select **APIs & Services**. From
136
- here, you will enable the APIs that your application requires.
106
+ ### Local ADC file
137
107
 
138
- *Note: You may need to enable billing in order to use these services.*
108
+ You can set up a local ADC file with your user credentials for authentication during
109
+ development. If credentials are not provided in code or in environment variables,
110
+ then the local ADC credentials are discovered.
139
111
 
140
- 4. Select **Credentials** from the side navigation.
112
+ Follow the steps in [Quickstart](#quickstart) to set up a local ADC file.
141
113
 
142
- Find the "Create credentials" drop down near the top of the page, and select
143
- "Service account" to be guided through downloading a new JSON key file.
114
+ ### Google Cloud Platform environments
144
115
 
145
- If you want to re-use an existing service account, you can easily generate a
146
- new key file. Just select the account you wish to re-use, click the pencil
147
- tool on the right side to edit the service account, select the **Keys** tab,
148
- and then select **Add Key**.
116
+ When running on Google Cloud Platform (GCP), including Google Compute Engine
117
+ (GCE), Google Kubernetes Engine (GKE), Google App Engine (GAE), Google Cloud
118
+ Functions (GCF) and Cloud Run, credentials are retrieved from the attached
119
+ service account automatically. Code should be written as if already authenticated.
149
120
 
150
- The key file you download will be used by this library to authenticate API
151
- requests and should be stored in a secure location.
121
+ For more information, see
122
+ [Set up ADC for Google Cloud services](https://cloud.google.com/docs/authentication/provide-credentials-adc#attached-sa).
data/README.md CHANGED
@@ -16,7 +16,7 @@ for this library, google-cloud-bigquery-storage, to see the convenience methods
16
16
  constructing client objects. Reference documentation for the client objects
17
17
  themselves can be found in the client library documentation for the versioned
18
18
  client gems:
19
- [google-cloud-bigquery-storage-v1](https://googleapis.dev/ruby/google-cloud-bigquery-storage-v1/latest).
19
+ [google-cloud-bigquery-storage-v1](https://cloud.google.com/ruby/docs/reference/google-cloud-bigquery-storage-v1/latest).
20
20
 
21
21
  See also the [Product Documentation](https://cloud.google.com/bigquery/docs/reference/storage)
22
22
  for more usage information.
@@ -37,8 +37,8 @@ In order to use this library, you first need to go through the following steps:
37
37
  ## Enabling Logging
38
38
 
39
39
  To enable logging for this library, set the logger for the underlying [gRPC](https://github.com/grpc/grpc/tree/master/src/ruby) library.
40
- The logger that you set may be a Ruby stdlib [`Logger`](https://ruby-doc.org/stdlib/libdoc/logger/rdoc/Logger.html) as shown below,
41
- or a [`Google::Cloud::Logging::Logger`](https://googleapis.dev/ruby/google-cloud-logging/latest)
40
+ The logger that you set may be a Ruby stdlib [`Logger`](https://ruby-doc.org/current/stdlibs/logger/Logger.html) as shown below,
41
+ or a [`Google::Cloud::Logging::Logger`](https://cloud.google.com/ruby/docs/reference/google-cloud-logging/latest)
42
42
  that will write logs to [Cloud Logging](https://cloud.google.com/logging/). See [grpc/logconfig.rb](https://github.com/grpc/grpc/blob/master/src/ruby/lib/grpc/logconfig.rb)
43
43
  and the gRPC [spec_helper.rb](https://github.com/grpc/grpc/blob/master/src/ruby/spec/spec_helper.rb) for additional information.
44
44
 
data/lib/google/cloud/bigquery/storage/version.rb CHANGED
@@ -21,7 +21,7 @@ module Google
21
21
  module Cloud
22
22
  module Bigquery
23
23
  module Storage
24
- VERSION = "1.3.0"
24
+ VERSION = "1.4.0"
25
25
  end
26
26
  end
27
27
  end
data/lib/google/cloud/bigquery/storage.rb CHANGED
@@ -29,7 +29,7 @@ require "google/cloud/config"
29
29
 
30
30
  # Set the default configuration
31
31
  ::Google::Cloud.configure.add_config! :bigquery_storage do |config|
32
- config.add_field! :endpoint, "bigquerystorage.googleapis.com", match: ::String
32
+ config.add_field! :endpoint, nil, match: ::String
33
33
  config.add_field! :credentials, nil, match: [::String, ::Hash, ::Google::Auth::Credentials]
34
34
  config.add_field! :scope, nil, match: [::Array, ::String]
35
35
  config.add_field! :lib_name, nil, match: ::String
@@ -39,6 +39,7 @@ require "google/cloud/config"
39
39
  config.add_field! :metadata, nil, match: ::Hash
40
40
  config.add_field! :retry_policy, nil, match: [::Hash, ::Proc]
41
41
  config.add_field! :quota_project, nil, match: ::String
42
+ config.add_field! :universe_domain, nil, match: ::String
42
43
  end
43
44
 
44
45
  module Google
@@ -49,9 +50,9 @@ module Google
49
50
  # Create a new client object for BigQueryRead.
50
51
  #
51
52
  # By default, this returns an instance of
52
- # [Google::Cloud::Bigquery::Storage::V1::BigQueryRead::Client](https://googleapis.dev/ruby/google-cloud-bigquery-storage-v1/latest/Google/Cloud/Bigquery/Storage/V1/BigQueryRead/Client.html)
53
- # for version V1 of the API.
54
- # However, you can specify specify a different API version by passing it in the
53
+ # [Google::Cloud::Bigquery::Storage::V1::BigQueryRead::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-bigquery-storage-v1/latest/Google-Cloud-Bigquery-Storage-V1-BigQueryRead-Client)
54
+ # for a gRPC client for version V1 of the API.
55
+ # However, you can specify a different API version by passing it in the
55
56
  # `version` parameter. If the BigQueryRead service is
56
57
  # supported by that API version, and the corresponding gem is available, the
57
58
  # appropriate versioned client will be returned.
@@ -64,7 +65,7 @@ module Google
64
65
  #
65
66
  # @param version [::String, ::Symbol] The API version to connect to. Optional.
66
67
  # Defaults to `:v1`.
67
- # @return [BigQueryRead::Client] A client object for the specified version.
68
+ # @return [::Object] A client object for the specified version.
68
69
  #
69
70
  def self.big_query_read version: :v1, &block
70
71
  require "google/cloud/bigquery/storage/#{version.to_s.downcase}"
@@ -73,17 +74,17 @@ module Google
73
74
  .constants
74
75
  .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
75
76
  .first
76
- package_module = Google::Cloud::Bigquery::Storage.const_get package_name
77
- package_module.const_get(:BigQueryRead).const_get(:Client).new(&block)
77
+ service_module = Google::Cloud::Bigquery::Storage.const_get(package_name).const_get(:BigQueryRead)
78
+ service_module.const_get(:Client).new(&block)
78
79
  end
79
80
 
80
81
  ##
81
82
  # Create a new client object for BigQueryWrite.
82
83
  #
83
84
  # By default, this returns an instance of
84
- # [Google::Cloud::Bigquery::Storage::V1::BigQueryWrite::Client](https://googleapis.dev/ruby/google-cloud-bigquery-storage-v1/latest/Google/Cloud/Bigquery/Storage/V1/BigQueryWrite/Client.html)
85
- # for version V1 of the API.
86
- # However, you can specify specify a different API version by passing it in the
85
+ # [Google::Cloud::Bigquery::Storage::V1::BigQueryWrite::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-bigquery-storage-v1/latest/Google-Cloud-Bigquery-Storage-V1-BigQueryWrite-Client)
86
+ # for a gRPC client for version V1 of the API.
87
+ # However, you can specify a different API version by passing it in the
87
88
  # `version` parameter. If the BigQueryWrite service is
88
89
  # supported by that API version, and the corresponding gem is available, the
89
90
  # appropriate versioned client will be returned.
@@ -99,7 +100,7 @@ module Google
99
100
  #
100
101
  # @param version [::String, ::Symbol] The API version to connect to. Optional.
101
102
  # Defaults to `:v1`.
102
- # @return [BigQueryWrite::Client] A client object for the specified version.
103
+ # @return [::Object] A client object for the specified version.
103
104
  #
104
105
  def self.big_query_write version: :v1, &block
105
106
  require "google/cloud/bigquery/storage/#{version.to_s.downcase}"
@@ -108,8 +109,8 @@ module Google
108
109
  .constants
109
110
  .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
110
111
  .first
111
- package_module = Google::Cloud::Bigquery::Storage.const_get package_name
112
- package_module.const_get(:BigQueryWrite).const_get(:Client).new(&block)
112
+ service_module = Google::Cloud::Bigquery::Storage.const_get(package_name).const_get(:BigQueryWrite)
113
+ service_module.const_get(:Client).new(&block)
113
114
  end
114
115
 
115
116
  ##
@@ -129,7 +130,7 @@ module Google
129
130
  # * `timeout` (*type:* `Numeric`) -
130
131
  # Default timeout in seconds.
131
132
  # * `metadata` (*type:* `Hash{Symbol=>String}`) -
132
- # Additional gRPC headers to be sent with the call.
133
+ # Additional headers to be sent with the call.
133
134
  # * `retry_policy` (*type:* `Hash`) -
134
135
  # The retry policy. The value is a hash with the following keys:
135
136
  # * `:initial_delay` (*type:* `Numeric`) - The initial delay in seconds.
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: google-cloud-bigquery-storage
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.3.0
4
+ version: 1.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Google LLC
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-07-01 00:00:00.000000000 Z
11
+ date: 2024-01-15 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: google-cloud-bigquery-storage-v1
@@ -16,7 +16,7 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: '0.8'
19
+ version: '0.25'
20
20
  - - "<"
21
21
  - !ruby/object:Gem::Version
22
22
  version: 2.a
@@ -26,7 +26,7 @@ dependencies:
26
26
  requirements:
27
27
  - - ">="
28
28
  - !ruby/object:Gem::Version
29
- version: '0.8'
29
+ version: '0.25'
30
30
  - - "<"
31
31
  - !ruby/object:Gem::Version
32
32
  version: 2.a
@@ -188,7 +188,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
188
188
  - !ruby/object:Gem::Version
189
189
  version: '0'
190
190
  requirements: []
191
- rubygems_version: 3.3.14
191
+ rubygems_version: 3.5.3
192
192
  signing_key:
193
193
  specification_version: 4
194
194
  summary: API Client library for the BigQuery Storage API