google-cloud-dataproc 2.3.0 → 2.5.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/AUTHENTICATION.md +72 -101
- data/README.md +1 -1
- data/lib/google/cloud/dataproc/version.rb +1 -1
- data/lib/google/cloud/dataproc.rb +76 -7
- metadata +5 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c72b99287e161c9444f7eddd525f5b25113329f63fe936aa5f6c267b0dc9daad
|
4
|
+
data.tar.gz: d94216d2bd442d0129c2829d1dc616aeec6e518d22e563438571b4186fd570d8
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 077046dea4c9b7475e0e8630d4da4fa6e2f36a9cef77450bf94ca51065dc6b318e43da9be35c0fa7f691a4a161e512edb8d39fbf22ce3a6386475af082b545a4
|
7
|
+
data.tar.gz: 662837e14d5554df87781f794b360f0499557af73673dc5f395376c8567eabc3d13feeed17c27adb4aab4409fa293d91e8254f664c6c78836cc010bc9c880566
|
data/AUTHENTICATION.md
CHANGED
@@ -1,151 +1,122 @@
|
|
1
1
|
# Authentication
|
2
2
|
|
3
|
-
|
4
|
-
[
|
5
|
-
|
6
|
-
[Google Cloud Platform environments](#google-cloud-platform-environments) the
|
7
|
-
credentials will be discovered automatically. When running on other
|
8
|
-
environments, the Service Account credentials can be specified by providing the
|
9
|
-
path to the
|
10
|
-
[JSON keyfile](https://cloud.google.com/iam/docs/managing-service-account-keys)
|
11
|
-
for the account (or the JSON itself) in
|
12
|
-
[environment variables](#environment-variables). Additionally, Cloud SDK
|
13
|
-
credentials can also be discovered automatically, but this is only recommended
|
14
|
-
during development.
|
3
|
+
The recommended way to authenticate to the google-cloud-dataproc library is to use
|
4
|
+
[Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials).
|
5
|
+
To review all of your authentication options, see [Credentials lookup](#credential-lookup).
|
15
6
|
|
16
7
|
## Quickstart
|
17
8
|
|
18
|
-
|
19
|
-
|
9
|
+
The following example shows how to set up authentication for a local development
|
10
|
+
environment with your user credentials.
|
20
11
|
|
21
|
-
|
22
|
-
|
23
|
-
```
|
24
|
-
|
25
|
-
3. Initialize the client.
|
12
|
+
**NOTE:** This method is _not_ recommended for running in production. User credentials
|
13
|
+
should be used only during development.
|
26
14
|
|
27
|
-
|
28
|
-
|
15
|
+
1. [Download and install the Google Cloud CLI](https://cloud.google.com/sdk).
|
16
|
+
2. Set up a local ADC file with your user credentials:
|
29
17
|
|
30
|
-
|
18
|
+
```sh
|
19
|
+
gcloud auth application-default login
|
31
20
|
```
|
32
21
|
|
33
|
-
|
34
|
-
|
35
|
-
The google-cloud-dataproc library aims to make authentication
|
36
|
-
as simple as possible, and provides several mechanisms to configure your system
|
37
|
-
without requiring **Service Account Credentials** directly in code.
|
38
|
-
|
39
|
-
**Credentials** are discovered in the following order:
|
40
|
-
|
41
|
-
1. Specify credentials in method arguments
|
42
|
-
2. Specify credentials in configuration
|
43
|
-
3. Discover credentials path in environment variables
|
44
|
-
4. Discover credentials JSON in environment variables
|
45
|
-
5. Discover credentials file in the Cloud SDK's path
|
46
|
-
6. Discover GCP credentials
|
47
|
-
|
48
|
-
### Google Cloud Platform environments
|
22
|
+
3. Write code as if already authenticated.
|
49
23
|
|
50
|
-
|
51
|
-
|
52
|
-
Functions (GCF) and Cloud Run, **Credentials** are discovered automatically.
|
53
|
-
Code should be written as if already authenticated.
|
24
|
+
For more information about setting up authentication for a local development environment, see
|
25
|
+
[Set up Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-dev).
|
54
26
|
|
55
|
-
|
27
|
+
## Credential Lookup
|
56
28
|
|
57
|
-
The
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
**Credentials JSON** file can be stored in the environment variable, or the
|
62
|
-
**Credentials JSON** itself can be stored for environments such as Docker
|
63
|
-
containers where writing files is difficult or not encouraged.
|
29
|
+
The google-cloud-dataproc library provides several mechanisms to configure your system.
|
30
|
+
Generally, using Application Default Credentials to facilitate automatic
|
31
|
+
credentials discovery is the easist method. But if you need to explicitly specify
|
32
|
+
credentials, there are several methods available to you.
|
64
33
|
|
65
|
-
|
66
|
-
checks for credentials are configured on the service Credentials class (such as
|
67
|
-
`::Google::Cloud::Dataproc::V1::AutoscalingPolicyService::Credentials`):
|
34
|
+
Credentials are accepted in the following ways, in the following order or precedence:
|
68
35
|
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
36
|
+
1. Credentials specified in method arguments
|
37
|
+
2. Credentials specified in configuration
|
38
|
+
3. Credentials pointed to or included in environment variables
|
39
|
+
4. Credentials found in local ADC file
|
40
|
+
5. Credentials returned by the metadata server for the attached service account (GCP)
|
74
41
|
|
75
|
-
|
76
|
-
require "google/cloud/dataproc"
|
77
|
-
|
78
|
-
ENV["DATAPROC_CREDENTIALS"] = "path/to/keyfile.json"
|
42
|
+
### Configuration
|
79
43
|
|
80
|
-
client
|
81
|
-
|
44
|
+
You can configure a path to a JSON credentials file, either for an individual client object or
|
45
|
+
globally, for all client objects. The JSON file can contain credentials created for
|
46
|
+
[workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation),
|
47
|
+
[workforce identity federation](https://cloud.google.com/iam/docs/workforce-identity-federation), or a
|
48
|
+
[service account key](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-key).
|
82
49
|
|
83
|
-
|
50
|
+
Note: Service account keys are a security risk if not managed correctly. You should
|
51
|
+
[choose a more secure alternative to service account keys](https://cloud.google.com/docs/authentication#auth-decision-tree)
|
52
|
+
whenever possible.
|
84
53
|
|
85
|
-
|
86
|
-
it in an environment variable. Either on an individual client initialization:
|
54
|
+
To configure a credentials file for an individual client initialization:
|
87
55
|
|
88
56
|
```ruby
|
89
57
|
require "google/cloud/dataproc"
|
90
58
|
|
91
59
|
client = Google::Cloud::Dataproc.autoscaling_policy_service do |config|
|
92
|
-
config.credentials = "path/to/
|
60
|
+
config.credentials = "path/to/credentialfile.json"
|
93
61
|
end
|
94
62
|
```
|
95
63
|
|
96
|
-
|
64
|
+
To configure a credentials file globally for all clients:
|
97
65
|
|
98
66
|
```ruby
|
99
67
|
require "google/cloud/dataproc"
|
100
68
|
|
101
69
|
Google::Cloud::Dataproc.configure do |config|
|
102
|
-
config.credentials = "path/to/
|
70
|
+
config.credentials = "path/to/credentialfile.json"
|
103
71
|
end
|
104
72
|
|
105
73
|
client = Google::Cloud::Dataproc.autoscaling_policy_service
|
106
74
|
```
|
107
75
|
|
108
|
-
###
|
76
|
+
### Environment Variables
|
109
77
|
|
110
|
-
|
111
|
-
|
112
|
-
|
78
|
+
You can also use an environment variable to provide a JSON credentials file.
|
79
|
+
The environment variable can contain a path to the credentials file or, for
|
80
|
+
environments such as Docker containers where writing files is not encouraged,
|
81
|
+
you can include the credentials file itself.
|
113
82
|
|
114
|
-
|
83
|
+
The JSON file can contain credentials created for
|
84
|
+
[workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation),
|
85
|
+
[workforce identity federation](https://cloud.google.com/iam/docs/workforce-identity-federation), or a
|
86
|
+
[service account key](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-key).
|
115
87
|
|
116
|
-
|
117
|
-
|
118
|
-
|
88
|
+
Note: Service account keys are a security risk if not managed correctly. You should
|
89
|
+
[choose a more secure alternative to service account keys](https://cloud.google.com/docs/authentication#auth-decision-tree)
|
90
|
+
whenever possible.
|
91
|
+
|
92
|
+
The environment variables that google-cloud-dataproc
|
93
|
+
checks for credentials are:
|
119
94
|
|
120
|
-
|
121
|
-
*
|
95
|
+
* `GOOGLE_CLOUD_CREDENTIALS` - Path to JSON file, or JSON contents
|
96
|
+
* `GOOGLE_APPLICATION_CREDENTIALS` - Path to JSON file
|
122
97
|
|
123
|
-
|
98
|
+
```ruby
|
99
|
+
require "google/cloud/dataproc"
|
124
100
|
|
125
|
-
|
126
|
-
connect to the APIs. You will use the **JSON key file** to
|
127
|
-
connect to most services with google-cloud-dataproc.
|
101
|
+
ENV["GOOGLE_APPLICATION_CREDENTIALS"] = "path/to/credentialfile.json"
|
128
102
|
|
129
|
-
|
130
|
-
|
131
|
-
need a Google Developers service account.
|
103
|
+
client = Google::Cloud::Dataproc.autoscaling_policy_service
|
104
|
+
```
|
132
105
|
|
133
|
-
|
134
|
-
2. Create a new project or click on an existing project.
|
135
|
-
3. Activate the menu in the upper left and select **APIs & Services**. From
|
136
|
-
here, you will enable the APIs that your application requires.
|
106
|
+
### Local ADC file
|
137
107
|
|
138
|
-
|
108
|
+
You can set up a local ADC file with your user credentials for authentication during
|
109
|
+
development. If credentials are not provided in code or in environment variables,
|
110
|
+
then the local ADC credentials are discovered.
|
139
111
|
|
140
|
-
|
112
|
+
Follow the steps in [Quickstart](#quickstart) to set up a local ADC file.
|
141
113
|
|
142
|
-
|
143
|
-
"Service account" to be guided through downloading a new JSON key file.
|
114
|
+
### Google Cloud Platform environments
|
144
115
|
|
145
|
-
|
146
|
-
|
147
|
-
|
148
|
-
|
116
|
+
When running on Google Cloud Platform (GCP), including Google Compute Engine
|
117
|
+
(GCE), Google Kubernetes Engine (GKE), Google App Engine (GAE), Google Cloud
|
118
|
+
Functions (GCF) and Cloud Run, credentials are retrieved from the attached
|
119
|
+
service account automatically. Code should be written as if already authenticated.
|
149
120
|
|
150
|
-
|
151
|
-
|
121
|
+
For more information, see
|
122
|
+
[Set up ADC for Google Cloud services](https://cloud.google.com/docs/authentication/provide-credentials-adc#attached-sa).
|
data/README.md
CHANGED
@@ -16,7 +16,7 @@ for this library, google-cloud-dataproc, to see the convenience methods for
|
|
16
16
|
constructing client objects. Reference documentation for the client objects
|
17
17
|
themselves can be found in the client library documentation for the versioned
|
18
18
|
client gems:
|
19
|
-
[google-cloud-dataproc-v1](https://
|
19
|
+
[google-cloud-dataproc-v1](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest).
|
20
20
|
|
21
21
|
See also the [Product Documentation](https://cloud.google.com/dataproc)
|
22
22
|
for more usage information.
|
@@ -29,7 +29,7 @@ require "google/cloud/config"
|
|
29
29
|
|
30
30
|
# Set the default configuration
|
31
31
|
::Google::Cloud.configure.add_config! :dataproc do |config|
|
32
|
-
config.add_field! :endpoint,
|
32
|
+
config.add_field! :endpoint, nil, match: ::String
|
33
33
|
config.add_field! :credentials, nil, match: [::String, ::Hash, ::Google::Auth::Credentials]
|
34
34
|
config.add_field! :scope, nil, match: [::Array, ::String]
|
35
35
|
config.add_field! :lib_name, nil, match: ::String
|
@@ -39,6 +39,7 @@ require "google/cloud/config"
|
|
39
39
|
config.add_field! :metadata, nil, match: ::Hash
|
40
40
|
config.add_field! :retry_policy, nil, match: [::Hash, ::Proc]
|
41
41
|
config.add_field! :quota_project, nil, match: ::String
|
42
|
+
config.add_field! :universe_domain, nil, match: ::String
|
42
43
|
end
|
43
44
|
|
44
45
|
module Google
|
@@ -48,7 +49,7 @@ module Google
|
|
48
49
|
# Create a new client object for AutoscalingPolicyService.
|
49
50
|
#
|
50
51
|
# By default, this returns an instance of
|
51
|
-
# [Google::Cloud::Dataproc::V1::AutoscalingPolicyService::Client](https://
|
52
|
+
# [Google::Cloud::Dataproc::V1::AutoscalingPolicyService::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-AutoscalingPolicyService-Client)
|
52
53
|
# for a gRPC client for version V1 of the API.
|
53
54
|
# However, you can specify a different API version by passing it in the
|
54
55
|
# `version` parameter. If the AutoscalingPolicyService service is
|
@@ -83,7 +84,7 @@ module Google
|
|
83
84
|
# Create a new client object for BatchController.
|
84
85
|
#
|
85
86
|
# By default, this returns an instance of
|
86
|
-
# [Google::Cloud::Dataproc::V1::BatchController::Client](https://
|
87
|
+
# [Google::Cloud::Dataproc::V1::BatchController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-BatchController-Client)
|
87
88
|
# for a gRPC client for version V1 of the API.
|
88
89
|
# However, you can specify a different API version by passing it in the
|
89
90
|
# `version` parameter. If the BatchController service is
|
@@ -117,7 +118,7 @@ module Google
|
|
117
118
|
# Create a new client object for ClusterController.
|
118
119
|
#
|
119
120
|
# By default, this returns an instance of
|
120
|
-
# [Google::Cloud::Dataproc::V1::ClusterController::Client](https://
|
121
|
+
# [Google::Cloud::Dataproc::V1::ClusterController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-ClusterController-Client)
|
121
122
|
# for a gRPC client for version V1 of the API.
|
122
123
|
# However, you can specify a different API version by passing it in the
|
123
124
|
# `version` parameter. If the ClusterController service is
|
@@ -152,7 +153,7 @@ module Google
|
|
152
153
|
# Create a new client object for JobController.
|
153
154
|
#
|
154
155
|
# By default, this returns an instance of
|
155
|
-
# [Google::Cloud::Dataproc::V1::JobController::Client](https://
|
156
|
+
# [Google::Cloud::Dataproc::V1::JobController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-JobController-Client)
|
156
157
|
# for a gRPC client for version V1 of the API.
|
157
158
|
# However, you can specify a different API version by passing it in the
|
158
159
|
# `version` parameter. If the JobController service is
|
@@ -186,7 +187,7 @@ module Google
|
|
186
187
|
# Create a new client object for NodeGroupController.
|
187
188
|
#
|
188
189
|
# By default, this returns an instance of
|
189
|
-
# [Google::Cloud::Dataproc::V1::NodeGroupController::Client](https://
|
190
|
+
# [Google::Cloud::Dataproc::V1::NodeGroupController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-NodeGroupController-Client)
|
190
191
|
# for a gRPC client for version V1 of the API.
|
191
192
|
# However, you can specify a different API version by passing it in the
|
192
193
|
# `version` parameter. If the NodeGroupController service is
|
@@ -217,11 +218,79 @@ module Google
|
|
217
218
|
service_module.const_get(:Client).new(&block)
|
218
219
|
end
|
219
220
|
|
221
|
+
##
|
222
|
+
# Create a new client object for SessionController.
|
223
|
+
#
|
224
|
+
# By default, this returns an instance of
|
225
|
+
# [Google::Cloud::Dataproc::V1::SessionController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-SessionController-Client)
|
226
|
+
# for a gRPC client for version V1 of the API.
|
227
|
+
# However, you can specify a different API version by passing it in the
|
228
|
+
# `version` parameter. If the SessionController service is
|
229
|
+
# supported by that API version, and the corresponding gem is available, the
|
230
|
+
# appropriate versioned client will be returned.
|
231
|
+
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
232
|
+
# the `transport` parameter.
|
233
|
+
#
|
234
|
+
# ## About SessionController
|
235
|
+
#
|
236
|
+
# The `SessionController` provides methods to manage interactive sessions.
|
237
|
+
#
|
238
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
239
|
+
# Defaults to `:v1`.
|
240
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
241
|
+
# @return [::Object] A client object for the specified version.
|
242
|
+
#
|
243
|
+
def self.session_controller version: :v1, transport: :grpc, &block
|
244
|
+
require "google/cloud/dataproc/#{version.to_s.downcase}"
|
245
|
+
|
246
|
+
package_name = Google::Cloud::Dataproc
|
247
|
+
.constants
|
248
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
249
|
+
.first
|
250
|
+
service_module = Google::Cloud::Dataproc.const_get(package_name).const_get(:SessionController)
|
251
|
+
service_module = service_module.const_get(:Rest) if transport == :rest
|
252
|
+
service_module.const_get(:Client).new(&block)
|
253
|
+
end
|
254
|
+
|
255
|
+
##
|
256
|
+
# Create a new client object for SessionTemplateController.
|
257
|
+
#
|
258
|
+
# By default, this returns an instance of
|
259
|
+
# [Google::Cloud::Dataproc::V1::SessionTemplateController::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-SessionTemplateController-Client)
|
260
|
+
# for a gRPC client for version V1 of the API.
|
261
|
+
# However, you can specify a different API version by passing it in the
|
262
|
+
# `version` parameter. If the SessionTemplateController service is
|
263
|
+
# supported by that API version, and the corresponding gem is available, the
|
264
|
+
# appropriate versioned client will be returned.
|
265
|
+
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
266
|
+
# the `transport` parameter.
|
267
|
+
#
|
268
|
+
# ## About SessionTemplateController
|
269
|
+
#
|
270
|
+
# The SessionTemplateController provides methods to manage session templates.
|
271
|
+
#
|
272
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
273
|
+
# Defaults to `:v1`.
|
274
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
275
|
+
# @return [::Object] A client object for the specified version.
|
276
|
+
#
|
277
|
+
def self.session_template_controller version: :v1, transport: :grpc, &block
|
278
|
+
require "google/cloud/dataproc/#{version.to_s.downcase}"
|
279
|
+
|
280
|
+
package_name = Google::Cloud::Dataproc
|
281
|
+
.constants
|
282
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
283
|
+
.first
|
284
|
+
service_module = Google::Cloud::Dataproc.const_get(package_name).const_get(:SessionTemplateController)
|
285
|
+
service_module = service_module.const_get(:Rest) if transport == :rest
|
286
|
+
service_module.const_get(:Client).new(&block)
|
287
|
+
end
|
288
|
+
|
220
289
|
##
|
221
290
|
# Create a new client object for WorkflowTemplateService.
|
222
291
|
#
|
223
292
|
# By default, this returns an instance of
|
224
|
-
# [Google::Cloud::Dataproc::V1::WorkflowTemplateService::Client](https://
|
293
|
+
# [Google::Cloud::Dataproc::V1::WorkflowTemplateService::Client](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest/Google-Cloud-Dataproc-V1-WorkflowTemplateService-Client)
|
225
294
|
# for a gRPC client for version V1 of the API.
|
226
295
|
# However, you can specify a different API version by passing it in the
|
227
296
|
# `version` parameter. If the WorkflowTemplateService service is
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: google-cloud-dataproc
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.
|
4
|
+
version: 2.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Google LLC
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2024-02-06 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: google-cloud-core
|
@@ -30,7 +30,7 @@ dependencies:
|
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: '0.
|
33
|
+
version: '0.24'
|
34
34
|
- - "<"
|
35
35
|
- !ruby/object:Gem::Version
|
36
36
|
version: 2.a
|
@@ -40,7 +40,7 @@ dependencies:
|
|
40
40
|
requirements:
|
41
41
|
- - ">="
|
42
42
|
- !ruby/object:Gem::Version
|
43
|
-
version: '0.
|
43
|
+
version: '0.24'
|
44
44
|
- - "<"
|
45
45
|
- !ruby/object:Gem::Version
|
46
46
|
version: 2.a
|
@@ -189,7 +189,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
189
189
|
- !ruby/object:Gem::Version
|
190
190
|
version: '0'
|
191
191
|
requirements: []
|
192
|
-
rubygems_version: 3.
|
192
|
+
rubygems_version: 3.5.3
|
193
193
|
signing_key:
|
194
194
|
specification_version: 4
|
195
195
|
summary: API Client library for the Cloud Dataproc API
|