google-cloud-dataproc 2.3.0 → 2.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/AUTHENTICATION.md +72 -101
- data/README.md +1 -1
- data/lib/google/cloud/dataproc/version.rb +1 -1
- data/lib/google/cloud/dataproc.rb +76 -7
- metadata +5 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c72b99287e161c9444f7eddd525f5b25113329f63fe936aa5f6c267b0dc9daad
|
4
|
+
data.tar.gz: d94216d2bd442d0129c2829d1dc616aeec6e518d22e563438571b4186fd570d8
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 077046dea4c9b7475e0e8630d4da4fa6e2f36a9cef77450bf94ca51065dc6b318e43da9be35c0fa7f691a4a161e512edb8d39fbf22ce3a6386475af082b545a4
|
7
|
+
data.tar.gz: 662837e14d5554df87781f794b360f0499557af73673dc5f395376c8567eabc3d13feeed17c27adb4aab4409fa293d91e8254f664c6c78836cc010bc9c880566
|
data/AUTHENTICATION.md
CHANGED
@@ -1,151 +1,122 @@
|
|
1
1
|
# Authentication
|
2
2
|
|
3
|
-
|
4
|
-
[
|
5
|
-
|
6
|
-
[Google Cloud Platform environments](#google-cloud-platform-environments) the
|
7
|
-
credentials will be discovered automatically. When running on other
|
8
|
-
environments, the Service Account credentials can be specified by providing the
|
9
|
-
path to the
|
10
|
-
[JSON keyfile](https://cloud.google.com/iam/docs/managing-service-account-keys)
|
11
|
-
for the account (or the JSON itself) in
|
12
|
-
[environment variables](#environment-variables). Additionally, Cloud SDK
|
13
|
-
credentials can also be discovered automatically, but this is only recommended
|
14
|
-
during development.
|
3
|
+
The recommended way to authenticate to the google-cloud-dataproc library is to use
|
4
|
+
[Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials).
|
5
|
+
To review all of your authentication options, see [Credentials lookup](#credential-lookup).
|
15
6
|
|
16
7
|
## Quickstart
|
17
8
|
|
18
|
-
|
19
|
-
|
9
|
+
The following example shows how to set up authentication for a local development
|
10
|
+
environment with your user credentials.
|
20
11
|
|
21
|
-
|
22
|
-
|
23
|
-
```
|
24
|
-
|
25
|
-
3. Initialize the client.
|
12
|
+
**NOTE:** This method is _not_ recommended for running in production. User credentials
|
13
|
+
should be used only during development.
|
26
14
|
|
27
|
-
|
28
|
-
|
15
|
+
1. [Download and install the Google Cloud CLI](https://cloud.google.com/sdk).
|
16
|
+
2. Set up a local ADC file with your user credentials:
|
29
17
|
|
30
|
-
|
18
|
+
```sh
|
19
|
+
gcloud auth application-default login
|
31
20
|
```
|
32
21
|
|
33
|
-
|
34
|
-
|
35
|
-
The google-cloud-dataproc library aims to make authentication
|
36
|
-
as simple as possible, and provides several mechanisms to configure your system
|
37
|
-
without requiring **Service Account Credentials** directly in code.
|
38
|
-
|
39
|
-
**Credentials** are discovered in the following order:
|
40
|
-
|
41
|
-
1. Specify credentials in method arguments
|
42
|
-
2. Specify credentials in configuration
|
43
|
-
3. Discover credentials path in environment variables
|
44
|
-
4. Discover credentials JSON in environment variables
|
45
|
-
5. Discover credentials file in the Cloud SDK's path
|
46
|
-
6. Discover GCP credentials
|
47
|
-
|
48
|
-
### Google Cloud Platform environments
|
22
|
+
3. Write code as if already authenticated.
|
49
23
|
|
50
|
-
|
51
|
-
|
52
|
-
Functions (GCF) and Cloud Run, **Credentials** are discovered automatically.
|
53
|
-
Code should be written as if already authenticated.
|
24
|
+
For more information about setting up authentication for a local development environment, see
|
25
|
+
[Set up Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-dev).
|
54
26
|
|
55
|
-
|
27
|
+
## Credential Lookup
|
56
28
|
|
57
|
-
The
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
**Credentials JSON** file can be stored in the environment variable, or the
|
62
|
-
**Credentials JSON** itself can be stored for environments such as Docker
|
63
|
-
containers where writing files is difficult or not encouraged.
|
29
|
+
The google-cloud-dataproc library provides several mechanisms to configure your system.
|
30
|
+
Generally, using Application Default Credentials to facilitate automatic
|
31
|
+
credentials discovery is the easist method. But if you need to explicitly specify
|
32
|
+
credentials, there are several methods available to you.
|
64
33
|
|
65
|
-
|
66
|
-
checks for credentials are configured on the service Credentials class (such as
|
67
|
-
`::Google::Cloud::Dataproc::V1::AutoscalingPolicyService::Credentials`):
|
34
|
+
Credentials are accepted in the following ways, in the following order or precedence:
|
68
35
|
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
36
|
+
1. Credentials specified in method arguments
|
37
|
+
2. Credentials specified in configuration
|
38
|
+
3. Credentials pointed to or included in environment variables
|
39
|
+
4. Credentials found in local ADC file
|
40
|
+
5. Credentials returned by the metadata server for the attached service account (GCP)
|
74
41
|
|
75
|
-
|
76
|
-
require "google/cloud/dataproc"
|
77
|
-
|
78
|
-
ENV["DATAPROC_CREDENTIALS"] = "path/to/keyfile.json"
|
42
|
+
### Configuration
|
79
43
|
|
80
|
-
client
|
81
|
-
|
44
|
+
You can configure a path to a JSON credentials file, either for an individual client object or
|
45
|
+
globally, for all client objects. The JSON file can contain credentials created for
|
46
|
+
[workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation),
|
47
|
+
[workforce identity federation](https://cloud.google.com/iam/docs/workforce-identity-federation), or a
|
48
|
+
[service account key](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-key).
|
82
49
|
|
83
|
-
|
50
|
+
Note: Service account keys are a security risk if not managed correctly. You should
|
51
|
+
[choose a more secure alternative to service account keys](https://cloud.google.com/docs/authentication#auth-decision-tree)
|
52
|
+
whenever possible.
|
84
53
|
|
85
|
-
|
86
|
-
it in an environment variable. Either on an individual client initialization:
|
54
|
+
To configure a credentials file for an individual client initialization:
|
87
55
|
|
88
56
|
```ruby
|
89
57
|
require "google/cloud/dataproc"
|
90
58
|
|
91
59
|
client = Google::Cloud::Dataproc.autoscaling_policy_service do |config|
|
92
|
-
config.credentials = "path/to/
|
60
|
+
config.credentials = "path/to/credentialfile.json"
|
93
61
|
end
|
94
62
|
```
|
95
63
|
|
96
|
-
|
64
|
+
To configure a credentials file globally for all clients:
|
97
65
|
|
98
66
|
```ruby
|
99
67
|
require "google/cloud/dataproc"
|
100
68
|
|
101
69
|
Google::Cloud::Dataproc.configure do |config|
|
102
|
-
config.credentials = "path/to/
|
70
|
+
config.credentials = "path/to/credentialfile.json"
|
103
71
|
end
|
104
72
|
|
105
73
|
client = Google::Cloud::Dataproc.autoscaling_policy_service
|
106
74
|
```
|
107
75
|
|
108
|
-
###
|
76
|
+
### Environment Variables
|
109
77
|
|
110
|
-
|
111
|
-
|
112
|
-
|
78
|
+
You can also use an environment variable to provide a JSON credentials file.
|
79
|
+
The environment variable can contain a path to the credentials file or, for
|
80
|
+
environments such as Docker containers where writing files is not encouraged,
|
81
|
+
you can include the credentials file itself.
|
113
82
|
|
114
|
-
|
83
|
+
The JSON file can contain credentials created for
|
84
|
+
[workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation),
|
85
|
+
[workforce identity federation](https://cloud.google.com/iam/docs/workforce-identity-federation), or a
|
86
|
+
[service account key](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-key).
|
115
87
|
|
116
|
-
|
117
|
-
|
118
|
-
|
88
|
+
Note: Service account keys are a security risk if not managed correctly. You should
|
89
|
+
[choose a more secure alternative to service account keys](https://cloud.google.com/docs/authentication#auth-decision-tree)
|
90
|
+
whenever possible.
|
91
|
+
|
92
|
+
The environment variables that google-cloud-dataproc
|
93
|
+
checks for credentials are:
|
119
94
|
|
120
|
-
|
121
|
-
*
|
95
|
+
* `GOOGLE_CLOUD_CREDENTIALS` - Path to JSON file, or JSON contents
|
96
|
+
* `GOOGLE_APPLICATION_CREDENTIALS` - Path to JSON file
|
122
97
|
|
123
|
-
|
98
|
+
```ruby
|
99
|
+
require "google/cloud/dataproc"
|
124
100
|
|
125
|
-
|
126
|
-
connect to the APIs. You will use the **JSON key file** to
|
127
|
-
connect to most services with google-cloud-dataproc.
|
101
|
+
ENV["GOOGLE_APPLICATION_CREDENTIALS"] = "path/to/credentialfile.json"
|
128
102
|
|
129
|
-
|
130
|
-
|
131
|
-
need a Google Developers service account.
|
103
|
+
client = Google::Cloud::Dataproc.autoscaling_policy_service
|
104
|
+
```
|
132
105
|
|
133
|
-
|
134
|
-
2. Create a new project or click on an existing project.
|
135
|
-
3. Activate the menu in the upper left and select **APIs & Services**. From
|
136
|
-
here, you will enable the APIs that your application requires.
|
106
|
+
### Local ADC file
|
137
107
|
|
138
|
-
|
108
|
+
You can set up a local ADC file with your user credentials for authentication during
|
109
|
+
development. If credentials are not provided in code or in environment variables,
|
110
|
+
then the local ADC credentials are discovered.
|
139
111
|
|
140
|
-
|
112
|
+
Follow the steps in [Quickstart](#quickstart) to set up a local ADC file.
|
141
113
|
|
142
|
-
|
143
|
-
"Service account" to be guided through downloading a new JSON key file.
|
114
|
+
### Google Cloud Platform environments
|
144
115
|
|
145
|
-
|
146
|
-
|
147
|
-
|
148
|
-
|
116
|
+
When running on Google Cloud Platform (GCP), including Google Compute Engine
|
117
|
+
(GCE), Google Kubernetes Engine (GKE), Google App Engine (GAE), Google Cloud
|
118
|
+
Functions (GCF) and Cloud Run, credentials are retrieved from the attached
|
119
|
+
service account automatically. Code should be written as if already authenticated.
|
149
120
|
|
150
|
-
|
151
|
-
|
121
|
+
For more information, see
|
122
|
+
[Set up ADC for Google Cloud services](https://cloud.google.com/docs/authentication/provide-credentials-adc#attached-sa).
|
data/README.md
CHANGED
@@ -16,7 +16,7 @@ for this library, google-cloud-dataproc, to see the convenience methods for
|
|
16
16
|
constructing client objects. Reference documentation for the client objects
|
17
17
|
themselves can be found in the client library documentation for the versioned
|
18
18
|
client gems:
|
19
|
-
[google-cloud-dataproc-v1](https://
|
19
|
+
[google-cloud-dataproc-v1](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest).
|
20
20
|
|
21
21
|
See also the [Product Documentation](https://cloud.google.com/dataproc)
|
22
22
|
for more usage information.
|
@@ -29,7 +29,7 @@ require "google/cloud/config"
|
|
29
29
|
|
30
30
|
# Set the default configuration
|
31
31
|
::Google::Cloud.configure.add_config! :dataproc do |config|
|
32
|
-
config.add_field! :endpoint,
|
32
|
+
config.add_field! :endpoint, nil, match: ::String
|
33
33
|
config.add_field! :credentials, nil, match: [::String, ::Hash, ::Google::Auth::Credentials]
|
34
34
|
config.add_field! :scope, nil, match: [::Array, ::String]
|
35
35
|
config.add_field! :lib_name, nil, match: ::String
|
@@ -39,6 +39,7 @@ require "google/cloud/config"
|
|
39
39
|
config.add_field! :metadata, nil, match: ::Hash
|
40
40
|
config.add_field! :retry_policy, nil, match: [::Hash, ::Proc]
|
41
41
|
config.add_field! :quota_project, nil, match: ::String
|
42
|
+
config.add_field! :universe_domain, nil, match: ::String
|
42
43
|
end
|
43
44
|
|
44
45
|
module Google
|
@@ -48,7 +49,7 @@ module Google
|
|
48
49
|
# Create a new client object for AutoscalingPolicyService.
|
49
50
|
#
|
50
51
|
# By default, this returns an instance of
|
51
|
-
# [Google::Cloud::Dataproc::V1::AutoscalingPolicyService::Client](https://
|
52
|
+
# [Google::Cloud::Dataproc::V1::AutoscalingPolicyService::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-AutoscalingPolicyService-Client)
|
52
53
|
# for a gRPC client for version V1 of the API.
|
53
54
|
# However, you can specify a different API version by passing it in the
|
54
55
|
# `version` parameter. If the AutoscalingPolicyService service is
|
@@ -83,7 +84,7 @@ module Google
|
|
83
84
|
# Create a new client object for BatchController.
|
84
85
|
#
|
85
86
|
# By default, this returns an instance of
|
86
|
-
# [Google::Cloud::Dataproc::V1::BatchController::Client](https://
|
87
|
+
# [Google::Cloud::Dataproc::V1::BatchController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-BatchController-Client)
|
87
88
|
# for a gRPC client for version V1 of the API.
|
88
89
|
# However, you can specify a different API version by passing it in the
|
89
90
|
# `version` parameter. If the BatchController service is
|
@@ -117,7 +118,7 @@ module Google
|
|
117
118
|
# Create a new client object for ClusterController.
|
118
119
|
#
|
119
120
|
# By default, this returns an instance of
|
120
|
-
# [Google::Cloud::Dataproc::V1::ClusterController::Client](https://
|
121
|
+
# [Google::Cloud::Dataproc::V1::ClusterController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-ClusterController-Client)
|
121
122
|
# for a gRPC client for version V1 of the API.
|
122
123
|
# However, you can specify a different API version by passing it in the
|
123
124
|
# `version` parameter. If the ClusterController service is
|
@@ -152,7 +153,7 @@ module Google
|
|
152
153
|
# Create a new client object for JobController.
|
153
154
|
#
|
154
155
|
# By default, this returns an instance of
|
155
|
-
# [Google::Cloud::Dataproc::V1::JobController::Client](https://
|
156
|
+
# [Google::Cloud::Dataproc::V1::JobController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-JobController-Client)
|
156
157
|
# for a gRPC client for version V1 of the API.
|
157
158
|
# However, you can specify a different API version by passing it in the
|
158
159
|
# `version` parameter. If the JobController service is
|
@@ -186,7 +187,7 @@ module Google
|
|
186
187
|
# Create a new client object for NodeGroupController.
|
187
188
|
#
|
188
189
|
# By default, this returns an instance of
|
189
|
-
# [Google::Cloud::Dataproc::V1::NodeGroupController::Client](https://
|
190
|
+
# [Google::Cloud::Dataproc::V1::NodeGroupController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-NodeGroupController-Client)
|
190
191
|
# for a gRPC client for version V1 of the API.
|
191
192
|
# However, you can specify a different API version by passing it in the
|
192
193
|
# `version` parameter. If the NodeGroupController service is
|
@@ -217,11 +218,79 @@ module Google
|
|
217
218
|
service_module.const_get(:Client).new(&block)
|
218
219
|
end
|
219
220
|
|
221
|
+
##
|
222
|
+
# Create a new client object for SessionController.
|
223
|
+
#
|
224
|
+
# By default, this returns an instance of
|
225
|
+
# [Google::Cloud::Dataproc::V1::SessionController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-SessionController-Client)
|
226
|
+
# for a gRPC client for version V1 of the API.
|
227
|
+
# However, you can specify a different API version by passing it in the
|
228
|
+
# `version` parameter. If the SessionController service is
|
229
|
+
# supported by that API version, and the corresponding gem is available, the
|
230
|
+
# appropriate versioned client will be returned.
|
231
|
+
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
232
|
+
# the `transport` parameter.
|
233
|
+
#
|
234
|
+
# ## About SessionController
|
235
|
+
#
|
236
|
+
# The `SessionController` provides methods to manage interactive sessions.
|
237
|
+
#
|
238
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
239
|
+
# Defaults to `:v1`.
|
240
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
241
|
+
# @return [::Object] A client object for the specified version.
|
242
|
+
#
|
243
|
+
def self.session_controller version: :v1, transport: :grpc, &block
|
244
|
+
require "google/cloud/dataproc/#{version.to_s.downcase}"
|
245
|
+
|
246
|
+
package_name = Google::Cloud::Dataproc
|
247
|
+
.constants
|
248
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
249
|
+
.first
|
250
|
+
service_module = Google::Cloud::Dataproc.const_get(package_name).const_get(:SessionController)
|
251
|
+
service_module = service_module.const_get(:Rest) if transport == :rest
|
252
|
+
service_module.const_get(:Client).new(&block)
|
253
|
+
end
|
254
|
+
|
255
|
+
##
|
256
|
+
# Create a new client object for SessionTemplateController.
|
257
|
+
#
|
258
|
+
# By default, this returns an instance of
|
259
|
+
# [Google::Cloud::Dataproc::V1::SessionTemplateController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-SessionTemplateController-Client)
|
260
|
+
# for a gRPC client for version V1 of the API.
|
261
|
+
# However, you can specify a different API version by passing it in the
|
262
|
+
# `version` parameter. If the SessionTemplateController service is
|
263
|
+
# supported by that API version, and the corresponding gem is available, the
|
264
|
+
# appropriate versioned client will be returned.
|
265
|
+
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
266
|
+
# the `transport` parameter.
|
267
|
+
#
|
268
|
+
# ## About SessionTemplateController
|
269
|
+
#
|
270
|
+
# The SessionTemplateController provides methods to manage session templates.
|
271
|
+
#
|
272
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
273
|
+
# Defaults to `:v1`.
|
274
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
275
|
+
# @return [::Object] A client object for the specified version.
|
276
|
+
#
|
277
|
+
def self.session_template_controller version: :v1, transport: :grpc, &block
|
278
|
+
require "google/cloud/dataproc/#{version.to_s.downcase}"
|
279
|
+
|
280
|
+
package_name = Google::Cloud::Dataproc
|
281
|
+
.constants
|
282
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
283
|
+
.first
|
284
|
+
service_module = Google::Cloud::Dataproc.const_get(package_name).const_get(:SessionTemplateController)
|
285
|
+
service_module = service_module.const_get(:Rest) if transport == :rest
|
286
|
+
service_module.const_get(:Client).new(&block)
|
287
|
+
end
|
288
|
+
|
220
289
|
##
|
221
290
|
# Create a new client object for WorkflowTemplateService.
|
222
291
|
#
|
223
292
|
# By default, this returns an instance of
|
224
|
-
# [Google::Cloud::Dataproc::V1::WorkflowTemplateService::Client](https://
|
293
|
+
# [Google::Cloud::Dataproc::V1::WorkflowTemplateService::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-WorkflowTemplateService-Client)
|
225
294
|
# for a gRPC client for version V1 of the API.
|
226
295
|
# However, you can specify a different API version by passing it in the
|
227
296
|
# `version` parameter. If the WorkflowTemplateService service is
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: google-cloud-dataproc
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.
|
4
|
+
version: 2.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Google LLC
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2024-02-06 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: google-cloud-core
|
@@ -30,7 +30,7 @@ dependencies:
|
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: '0.
|
33
|
+
version: '0.24'
|
34
34
|
- - "<"
|
35
35
|
- !ruby/object:Gem::Version
|
36
36
|
version: 2.a
|
@@ -40,7 +40,7 @@ dependencies:
|
|
40
40
|
requirements:
|
41
41
|
- - ">="
|
42
42
|
- !ruby/object:Gem::Version
|
43
|
-
version: '0.
|
43
|
+
version: '0.24'
|
44
44
|
- - "<"
|
45
45
|
- !ruby/object:Gem::Version
|
46
46
|
version: 2.a
|
@@ -189,7 +189,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
189
189
|
- !ruby/object:Gem::Version
|
190
190
|
version: '0'
|
191
191
|
requirements: []
|
192
|
-
rubygems_version: 3.
|
192
|
+
rubygems_version: 3.5.3
|
193
193
|
signing_key:
|
194
194
|
specification_version: 4
|
195
195
|
summary: API Client library for the Cloud Dataproc API
|