google-cloud-dataproc-v1 0.7.1 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.yardopts +1 -1
- data/AUTHENTICATION.md +7 -25
- data/README.md +6 -1
- data/lib/google/cloud/dataproc/v1/autoscaling_policies_pb.rb +5 -2
- data/lib/google/cloud/dataproc/v1/batch_controller/client.rb +1 -0
- data/lib/google/cloud/dataproc/v1/batches_pb.rb +2 -1
- data/lib/google/cloud/dataproc/v1/cluster_controller/client.rb +1 -0
- data/lib/google/cloud/dataproc/v1/cluster_controller/paths.rb +0 -19
- data/lib/google/cloud/dataproc/v1/clusters_pb.rb +15 -9
- data/lib/google/cloud/dataproc/v1/job_controller/client.rb +1 -0
- data/lib/google/cloud/dataproc/v1/jobs_pb.rb +2 -1
- data/lib/google/cloud/dataproc/v1/operations_pb.rb +2 -1
- data/lib/google/cloud/dataproc/v1/shared_pb.rb +61 -1
- data/lib/google/cloud/dataproc/v1/version.rb +1 -1
- data/lib/google/cloud/dataproc/v1/workflow_template_service/client.rb +2 -1
- data/lib/google/cloud/dataproc/v1/workflow_template_service/paths.rb +0 -19
- data/lib/google/cloud/dataproc/v1/workflow_templates_pb.rb +2 -1
- data/lib/google/cloud/dataproc/v1.rb +2 -0
- data/proto_docs/google/api/resource.rb +10 -71
- data/proto_docs/google/cloud/dataproc/v1/clusters.rb +57 -25
- data/proto_docs/google/cloud/dataproc/v1/jobs.rb +12 -0
- data/proto_docs/google/cloud/dataproc/v1/shared.rb +219 -0
- data/proto_docs/google/cloud/dataproc/v1/workflow_templates.rb +1 -1
- data/proto_docs/google/protobuf/any.rb +3 -3
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ebf8588586e1596fe59cf789a1cbb4039a36ca1d6ed8daa2d79cec58f4ce57d5
|
4
|
+
data.tar.gz: 47785392184b79d8152e5fe51918021d08fb756b9effd5d3d05b6d7ca0e34616
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ac3a25ca0fd30bd5f04c0984fc333af9d50e7dffbbef75d1093f43d43fdc8369b5236fff5be973c9b20f9ab439fb982647446eabc6ce7281a9998bc3dd297d87
|
7
|
+
data.tar.gz: 18a9ef81778a4f33ada7b4bf1e11f9695b21c847cc14c24812b15e0adc6f9395010cb099bd4b93a3a534f978796c9addd3471c9ccc7201164ff2c6b7923fbda1
|
data/.yardopts
CHANGED
data/AUTHENTICATION.md
CHANGED
@@ -120,15 +120,6 @@ To configure your system for this, simply:
|
|
120
120
|
**NOTE:** This is _not_ recommended for running in production. The Cloud SDK
|
121
121
|
*should* only be used during development.
|
122
122
|
|
123
|
-
[gce-how-to]: https://cloud.google.com/compute/docs/authentication#using
|
124
|
-
[dev-console]: https://console.cloud.google.com/project
|
125
|
-
|
126
|
-
[enable-apis]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/enable-apis.png
|
127
|
-
|
128
|
-
[create-new-service-account]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/create-new-service-account.png
|
129
|
-
[create-new-service-account-existing-keys]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/create-new-service-account-existing-keys.png
|
130
|
-
[reuse-service-account]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/reuse-service-account.png
|
131
|
-
|
132
123
|
## Creating a Service Account
|
133
124
|
|
134
125
|
Google Cloud requires **Service Account Credentials** to
|
@@ -139,31 +130,22 @@ If you are not running this client within
|
|
139
130
|
[Google Cloud Platform environments](#google-cloud-platform-environments), you
|
140
131
|
need a Google Developers service account.
|
141
132
|
|
142
|
-
1. Visit the [Google
|
133
|
+
1. Visit the [Google Cloud Console](https://console.cloud.google.com/project).
|
143
134
|
2. Create a new project or click on an existing project.
|
144
|
-
3. Activate the
|
135
|
+
3. Activate the menu in the upper left and select **APIs & Services**. From
|
145
136
|
here, you will enable the APIs that your application requires.
|
146
137
|
|
147
|
-
![Enable the APIs that your application requires][enable-apis]
|
148
|
-
|
149
138
|
*Note: You may need to enable billing in order to use these services.*
|
150
139
|
|
151
140
|
4. Select **Credentials** from the side navigation.
|
152
141
|
|
153
|
-
|
154
|
-
|
155
|
-
![Create a new service account][create-new-service-account]
|
156
|
-
|
157
|
-
![Create a new service account With Existing Keys][create-new-service-account-existing-keys]
|
158
|
-
|
159
|
-
Find the "Add credentials" drop down and select "Service account" to be
|
160
|
-
guided through downloading a new JSON key file.
|
142
|
+
Find the "Create credentials" drop down near the top of the page, and select
|
143
|
+
"Service account" to be guided through downloading a new JSON key file.
|
161
144
|
|
162
145
|
If you want to re-use an existing service account, you can easily generate a
|
163
|
-
new key file. Just select the account you wish to re-use,
|
164
|
-
|
165
|
-
|
166
|
-
![Re-use an existing service account][reuse-service-account]
|
146
|
+
new key file. Just select the account you wish to re-use, click the pencil
|
147
|
+
tool on the right side to edit the service account, select the **Keys** tab,
|
148
|
+
and then select **Add Key**.
|
167
149
|
|
168
150
|
The key file you download will be used by this library to authenticate API
|
169
151
|
requests and should be stored in a secure location.
|
data/README.md
CHANGED
@@ -37,7 +37,7 @@ request = ::Google::Cloud::Dataproc::V1::CreateAutoscalingPolicyRequest.new # (r
|
|
37
37
|
response = client.create_autoscaling_policy request
|
38
38
|
```
|
39
39
|
|
40
|
-
View the [Client Library Documentation](https://
|
40
|
+
View the [Client Library Documentation](https://cloud.google.com/ruby/docs/reference/google-cloud-dataproc-v1/latest)
|
41
41
|
for class and method documentation.
|
42
42
|
|
43
43
|
See also the [Product Documentation](https://cloud.google.com/dataproc)
|
@@ -69,6 +69,11 @@ module GRPC
|
|
69
69
|
end
|
70
70
|
```
|
71
71
|
|
72
|
+
|
73
|
+
## Google Cloud Samples
|
74
|
+
|
75
|
+
To browse ready to use code samples check [Google Cloud Samples](https://cloud.google.com/docs/samples).
|
76
|
+
|
72
77
|
## Supported Ruby Versions
|
73
78
|
|
74
79
|
This library is supported on Ruby 2.5+.
|
@@ -1,13 +1,14 @@
|
|
1
1
|
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
2
2
|
# source: google/cloud/dataproc/v1/autoscaling_policies.proto
|
3
3
|
|
4
|
+
require 'google/protobuf'
|
5
|
+
|
4
6
|
require 'google/api/annotations_pb'
|
5
7
|
require 'google/api/client_pb'
|
6
8
|
require 'google/api/field_behavior_pb'
|
7
9
|
require 'google/api/resource_pb'
|
8
10
|
require 'google/protobuf/duration_pb'
|
9
11
|
require 'google/protobuf/empty_pb'
|
10
|
-
require 'google/protobuf'
|
11
12
|
|
12
13
|
Google::Protobuf::DescriptorPool.generated_pool.build do
|
13
14
|
add_file("google/cloud/dataproc/v1/autoscaling_policies.proto", :syntax => :proto3) do
|
@@ -22,8 +23,10 @@ Google::Protobuf::DescriptorPool.generated_pool.build do
|
|
22
23
|
end
|
23
24
|
end
|
24
25
|
add_message "google.cloud.dataproc.v1.BasicAutoscalingAlgorithm" do
|
25
|
-
optional :yarn_config, :message, 1, "google.cloud.dataproc.v1.BasicYarnAutoscalingConfig"
|
26
26
|
optional :cooldown_period, :message, 2, "google.protobuf.Duration"
|
27
|
+
oneof :config do
|
28
|
+
optional :yarn_config, :message, 1, "google.cloud.dataproc.v1.BasicYarnAutoscalingConfig"
|
29
|
+
end
|
27
30
|
end
|
28
31
|
add_message "google.cloud.dataproc.v1.BasicYarnAutoscalingConfig" do
|
29
32
|
optional :graceful_decommission_timeout, :message, 5, "google.protobuf.Duration"
|
@@ -1,6 +1,8 @@
|
|
1
1
|
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
2
2
|
# source: google/cloud/dataproc/v1/batches.proto
|
3
3
|
|
4
|
+
require 'google/protobuf'
|
5
|
+
|
4
6
|
require 'google/api/annotations_pb'
|
5
7
|
require 'google/api/client_pb'
|
6
8
|
require 'google/api/field_behavior_pb'
|
@@ -9,7 +11,6 @@ require 'google/cloud/dataproc/v1/shared_pb'
|
|
9
11
|
require 'google/longrunning/operations_pb'
|
10
12
|
require 'google/protobuf/empty_pb'
|
11
13
|
require 'google/protobuf/timestamp_pb'
|
12
|
-
require 'google/protobuf'
|
13
14
|
|
14
15
|
Google::Protobuf::DescriptorPool.generated_pool.build do
|
15
16
|
add_file("google/cloud/dataproc/v1/batches.proto", :syntax => :proto3) do
|
@@ -24,25 +24,6 @@ module Google
|
|
24
24
|
module ClusterController
|
25
25
|
# Path helper methods for the ClusterController API.
|
26
26
|
module Paths
|
27
|
-
##
|
28
|
-
# Create a fully-qualified Cluster resource string.
|
29
|
-
#
|
30
|
-
# The resource will be in the following format:
|
31
|
-
#
|
32
|
-
# `projects/{project}/locations/{location}/clusters/{cluster}`
|
33
|
-
#
|
34
|
-
# @param project [String]
|
35
|
-
# @param location [String]
|
36
|
-
# @param cluster [String]
|
37
|
-
#
|
38
|
-
# @return [::String]
|
39
|
-
def cluster_path project:, location:, cluster:
|
40
|
-
raise ::ArgumentError, "project cannot contain /" if project.to_s.include? "/"
|
41
|
-
raise ::ArgumentError, "location cannot contain /" if location.to_s.include? "/"
|
42
|
-
|
43
|
-
"projects/#{project}/locations/#{location}/clusters/#{cluster}"
|
44
|
-
end
|
45
|
-
|
46
27
|
##
|
47
28
|
# Create a fully-qualified Service resource string.
|
48
29
|
#
|
@@ -1,6 +1,8 @@
|
|
1
1
|
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
2
2
|
# source: google/cloud/dataproc/v1/clusters.proto
|
3
3
|
|
4
|
+
require 'google/protobuf'
|
5
|
+
|
4
6
|
require 'google/api/annotations_pb'
|
5
7
|
require 'google/api/client_pb'
|
6
8
|
require 'google/api/field_behavior_pb'
|
@@ -10,7 +12,6 @@ require 'google/longrunning/operations_pb'
|
|
10
12
|
require 'google/protobuf/duration_pb'
|
11
13
|
require 'google/protobuf/field_mask_pb'
|
12
14
|
require 'google/protobuf/timestamp_pb'
|
13
|
-
require 'google/protobuf'
|
14
15
|
|
15
16
|
Google::Protobuf::DescriptorPool.generated_pool.build do
|
16
17
|
add_file("google/cloud/dataproc/v1/clusters.proto", :syntax => :proto3) do
|
@@ -18,6 +19,7 @@ Google::Protobuf::DescriptorPool.generated_pool.build do
|
|
18
19
|
optional :project_id, :string, 1
|
19
20
|
optional :cluster_name, :string, 2
|
20
21
|
optional :config, :message, 3, "google.cloud.dataproc.v1.ClusterConfig"
|
22
|
+
optional :virtual_cluster_config, :message, 10, "google.cloud.dataproc.v1.VirtualClusterConfig"
|
21
23
|
map :labels, :string, :string, 8
|
22
24
|
optional :status, :message, 4, "google.cloud.dataproc.v1.ClusterStatus"
|
23
25
|
repeated :status_history, :message, 7, "google.cloud.dataproc.v1.ClusterStatus"
|
@@ -39,14 +41,17 @@ Google::Protobuf::DescriptorPool.generated_pool.build do
|
|
39
41
|
optional :lifecycle_config, :message, 17, "google.cloud.dataproc.v1.LifecycleConfig"
|
40
42
|
optional :endpoint_config, :message, 19, "google.cloud.dataproc.v1.EndpointConfig"
|
41
43
|
optional :metastore_config, :message, 20, "google.cloud.dataproc.v1.MetastoreConfig"
|
42
|
-
optional :gke_cluster_config, :message, 21, "google.cloud.dataproc.v1.GkeClusterConfig"
|
43
44
|
end
|
44
|
-
add_message "google.cloud.dataproc.v1.
|
45
|
-
optional :
|
45
|
+
add_message "google.cloud.dataproc.v1.VirtualClusterConfig" do
|
46
|
+
optional :staging_bucket, :string, 1
|
47
|
+
optional :auxiliary_services_config, :message, 7, "google.cloud.dataproc.v1.AuxiliaryServicesConfig"
|
48
|
+
oneof :infrastructure_config do
|
49
|
+
optional :kubernetes_cluster_config, :message, 6, "google.cloud.dataproc.v1.KubernetesClusterConfig"
|
50
|
+
end
|
46
51
|
end
|
47
|
-
add_message "google.cloud.dataproc.v1.
|
48
|
-
optional :
|
49
|
-
optional :
|
52
|
+
add_message "google.cloud.dataproc.v1.AuxiliaryServicesConfig" do
|
53
|
+
optional :metastore_config, :message, 1, "google.cloud.dataproc.v1.MetastoreConfig"
|
54
|
+
optional :spark_history_server_config, :message, 2, "google.cloud.dataproc.v1.SparkHistoryServerConfig"
|
50
55
|
end
|
51
56
|
add_message "google.cloud.dataproc.v1.EndpointConfig" do
|
52
57
|
map :http_ports, :string, :string, 1
|
@@ -119,6 +124,7 @@ Google::Protobuf::DescriptorPool.generated_pool.build do
|
|
119
124
|
optional :boot_disk_type, :string, 3
|
120
125
|
optional :boot_disk_size_gb, :int32, 1
|
121
126
|
optional :num_local_ssds, :int32, 2
|
127
|
+
optional :local_ssd_interface, :string, 4
|
122
128
|
end
|
123
129
|
add_message "google.cloud.dataproc.v1.NodeInitializationAction" do
|
124
130
|
optional :executable_file, :string, 1
|
@@ -272,8 +278,8 @@ module Google
|
|
272
278
|
module V1
|
273
279
|
Cluster = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.Cluster").msgclass
|
274
280
|
ClusterConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.ClusterConfig").msgclass
|
275
|
-
|
276
|
-
|
281
|
+
VirtualClusterConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.VirtualClusterConfig").msgclass
|
282
|
+
AuxiliaryServicesConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.AuxiliaryServicesConfig").msgclass
|
277
283
|
EndpointConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.EndpointConfig").msgclass
|
278
284
|
AutoscalingConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.AutoscalingConfig").msgclass
|
279
285
|
EncryptionConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.EncryptionConfig").msgclass
|
@@ -1,6 +1,8 @@
|
|
1
1
|
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
2
2
|
# source: google/cloud/dataproc/v1/jobs.proto
|
3
3
|
|
4
|
+
require 'google/protobuf'
|
5
|
+
|
4
6
|
require 'google/api/annotations_pb'
|
5
7
|
require 'google/api/client_pb'
|
6
8
|
require 'google/api/field_behavior_pb'
|
@@ -8,7 +10,6 @@ require 'google/longrunning/operations_pb'
|
|
8
10
|
require 'google/protobuf/empty_pb'
|
9
11
|
require 'google/protobuf/field_mask_pb'
|
10
12
|
require 'google/protobuf/timestamp_pb'
|
11
|
-
require 'google/protobuf'
|
12
13
|
|
13
14
|
Google::Protobuf::DescriptorPool.generated_pool.build do
|
14
15
|
add_file("google/cloud/dataproc/v1/jobs.proto", :syntax => :proto3) do
|
@@ -1,9 +1,10 @@
|
|
1
1
|
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
2
2
|
# source: google/cloud/dataproc/v1/operations.proto
|
3
3
|
|
4
|
+
require 'google/protobuf'
|
5
|
+
|
4
6
|
require 'google/api/field_behavior_pb'
|
5
7
|
require 'google/protobuf/timestamp_pb'
|
6
|
-
require 'google/protobuf'
|
7
8
|
|
8
9
|
Google::Protobuf::DescriptorPool.generated_pool.build do
|
9
10
|
add_file("google/cloud/dataproc/v1/operations.proto", :syntax => :proto3) do
|
@@ -1,12 +1,15 @@
|
|
1
1
|
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
2
2
|
# source: google/cloud/dataproc/v1/shared.proto
|
3
3
|
|
4
|
-
require 'google/api/field_behavior_pb'
|
5
4
|
require 'google/protobuf'
|
6
5
|
|
6
|
+
require 'google/api/field_behavior_pb'
|
7
|
+
|
7
8
|
Google::Protobuf::DescriptorPool.generated_pool.build do
|
8
9
|
add_file("google/cloud/dataproc/v1/shared.proto", :syntax => :proto3) do
|
9
10
|
add_message "google.cloud.dataproc.v1.RuntimeConfig" do
|
11
|
+
optional :version, :string, 1
|
12
|
+
optional :container_image, :string, 2
|
10
13
|
map :properties, :string, :string, 3
|
11
14
|
end
|
12
15
|
add_message "google.cloud.dataproc.v1.EnvironmentConfig" do
|
@@ -32,6 +35,54 @@ Google::Protobuf::DescriptorPool.generated_pool.build do
|
|
32
35
|
add_message "google.cloud.dataproc.v1.RuntimeInfo" do
|
33
36
|
map :endpoints, :string, :string, 1
|
34
37
|
optional :output_uri, :string, 2
|
38
|
+
optional :diagnostic_output_uri, :string, 3
|
39
|
+
end
|
40
|
+
add_message "google.cloud.dataproc.v1.GkeClusterConfig" do
|
41
|
+
optional :gke_cluster_target, :string, 2
|
42
|
+
repeated :node_pool_target, :message, 3, "google.cloud.dataproc.v1.GkeNodePoolTarget"
|
43
|
+
end
|
44
|
+
add_message "google.cloud.dataproc.v1.KubernetesClusterConfig" do
|
45
|
+
optional :kubernetes_namespace, :string, 1
|
46
|
+
optional :kubernetes_software_config, :message, 3, "google.cloud.dataproc.v1.KubernetesSoftwareConfig"
|
47
|
+
oneof :config do
|
48
|
+
optional :gke_cluster_config, :message, 2, "google.cloud.dataproc.v1.GkeClusterConfig"
|
49
|
+
end
|
50
|
+
end
|
51
|
+
add_message "google.cloud.dataproc.v1.KubernetesSoftwareConfig" do
|
52
|
+
map :component_version, :string, :string, 1
|
53
|
+
map :properties, :string, :string, 2
|
54
|
+
end
|
55
|
+
add_message "google.cloud.dataproc.v1.GkeNodePoolTarget" do
|
56
|
+
optional :node_pool, :string, 1
|
57
|
+
repeated :roles, :enum, 2, "google.cloud.dataproc.v1.GkeNodePoolTarget.Role"
|
58
|
+
optional :node_pool_config, :message, 3, "google.cloud.dataproc.v1.GkeNodePoolConfig"
|
59
|
+
end
|
60
|
+
add_enum "google.cloud.dataproc.v1.GkeNodePoolTarget.Role" do
|
61
|
+
value :ROLE_UNSPECIFIED, 0
|
62
|
+
value :DEFAULT, 1
|
63
|
+
value :CONTROLLER, 2
|
64
|
+
value :SPARK_DRIVER, 3
|
65
|
+
value :SPARK_EXECUTOR, 4
|
66
|
+
end
|
67
|
+
add_message "google.cloud.dataproc.v1.GkeNodePoolConfig" do
|
68
|
+
optional :config, :message, 2, "google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodeConfig"
|
69
|
+
repeated :locations, :string, 13
|
70
|
+
optional :autoscaling, :message, 4, "google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodePoolAutoscalingConfig"
|
71
|
+
end
|
72
|
+
add_message "google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodeConfig" do
|
73
|
+
optional :machine_type, :string, 1
|
74
|
+
optional :preemptible, :bool, 10
|
75
|
+
optional :local_ssd_count, :int32, 7
|
76
|
+
repeated :accelerators, :message, 11, "google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodePoolAcceleratorConfig"
|
77
|
+
optional :min_cpu_platform, :string, 13
|
78
|
+
end
|
79
|
+
add_message "google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodePoolAcceleratorConfig" do
|
80
|
+
optional :accelerator_count, :int64, 1
|
81
|
+
optional :accelerator_type, :string, 2
|
82
|
+
end
|
83
|
+
add_message "google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodePoolAutoscalingConfig" do
|
84
|
+
optional :min_node_count, :int32, 2
|
85
|
+
optional :max_node_count, :int32, 3
|
35
86
|
end
|
36
87
|
add_enum "google.cloud.dataproc.v1.Component" do
|
37
88
|
value :COMPONENT_UNSPECIFIED, 0
|
@@ -66,6 +117,15 @@ module Google
|
|
66
117
|
SparkHistoryServerConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.SparkHistoryServerConfig").msgclass
|
67
118
|
PeripheralsConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.PeripheralsConfig").msgclass
|
68
119
|
RuntimeInfo = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.RuntimeInfo").msgclass
|
120
|
+
GkeClusterConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.GkeClusterConfig").msgclass
|
121
|
+
KubernetesClusterConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.KubernetesClusterConfig").msgclass
|
122
|
+
KubernetesSoftwareConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.KubernetesSoftwareConfig").msgclass
|
123
|
+
GkeNodePoolTarget = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.GkeNodePoolTarget").msgclass
|
124
|
+
GkeNodePoolTarget::Role = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.GkeNodePoolTarget.Role").enummodule
|
125
|
+
GkeNodePoolConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.GkeNodePoolConfig").msgclass
|
126
|
+
GkeNodePoolConfig::GkeNodeConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodeConfig").msgclass
|
127
|
+
GkeNodePoolConfig::GkeNodePoolAcceleratorConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodePoolAcceleratorConfig").msgclass
|
128
|
+
GkeNodePoolConfig::GkeNodePoolAutoscalingConfig = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.GkeNodePoolConfig.GkeNodePoolAutoscalingConfig").msgclass
|
69
129
|
Component = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.Component").enummodule
|
70
130
|
FailureAction = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("google.cloud.dataproc.v1.FailureAction").enummodule
|
71
131
|
end
|
@@ -171,6 +171,7 @@ module Google
|
|
171
171
|
|
172
172
|
@operations_client = Operations.new do |config|
|
173
173
|
config.credentials = credentials
|
174
|
+
config.quota_project = @quota_project_id
|
174
175
|
config.endpoint = @config.endpoint
|
175
176
|
end
|
176
177
|
|
@@ -214,7 +215,7 @@ module Google
|
|
214
215
|
# Required. The resource name of the region or location, as described
|
215
216
|
# in https://cloud.google.com/apis/design/resource_names.
|
216
217
|
#
|
217
|
-
# * For `projects.regions.workflowTemplates
|
218
|
+
# * For `projects.regions.workflowTemplates.create`, the resource name of the
|
218
219
|
# region has the following format:
|
219
220
|
# `projects/{project_id}/regions/{region}`
|
220
221
|
#
|
@@ -24,25 +24,6 @@ module Google
|
|
24
24
|
module WorkflowTemplateService
|
25
25
|
# Path helper methods for the WorkflowTemplateService API.
|
26
26
|
module Paths
|
27
|
-
##
|
28
|
-
# Create a fully-qualified Cluster resource string.
|
29
|
-
#
|
30
|
-
# The resource will be in the following format:
|
31
|
-
#
|
32
|
-
# `projects/{project}/locations/{location}/clusters/{cluster}`
|
33
|
-
#
|
34
|
-
# @param project [String]
|
35
|
-
# @param location [String]
|
36
|
-
# @param cluster [String]
|
37
|
-
#
|
38
|
-
# @return [::String]
|
39
|
-
def cluster_path project:, location:, cluster:
|
40
|
-
raise ::ArgumentError, "project cannot contain /" if project.to_s.include? "/"
|
41
|
-
raise ::ArgumentError, "location cannot contain /" if location.to_s.include? "/"
|
42
|
-
|
43
|
-
"projects/#{project}/locations/#{location}/clusters/#{cluster}"
|
44
|
-
end
|
45
|
-
|
46
27
|
##
|
47
28
|
# Create a fully-qualified Location resource string.
|
48
29
|
#
|
@@ -1,6 +1,8 @@
|
|
1
1
|
# Generated by the protocol buffer compiler. DO NOT EDIT!
|
2
2
|
# source: google/cloud/dataproc/v1/workflow_templates.proto
|
3
3
|
|
4
|
+
require 'google/protobuf'
|
5
|
+
|
4
6
|
require 'google/api/annotations_pb'
|
5
7
|
require 'google/api/client_pb'
|
6
8
|
require 'google/api/field_behavior_pb'
|
@@ -11,7 +13,6 @@ require 'google/longrunning/operations_pb'
|
|
11
13
|
require 'google/protobuf/duration_pb'
|
12
14
|
require 'google/protobuf/empty_pb'
|
13
15
|
require 'google/protobuf/timestamp_pb'
|
14
|
-
require 'google/protobuf'
|
15
16
|
|
16
17
|
Google::Protobuf::DescriptorPool.generated_pool.build do
|
17
18
|
add_file("google/cloud/dataproc/v1/workflow_templates.proto", :syntax => :proto3) do
|
@@ -33,11 +33,7 @@ module Google
|
|
33
33
|
# // For Kubernetes resources, the format is {api group}/{kind}.
|
34
34
|
# option (google.api.resource) = {
|
35
35
|
# type: "pubsub.googleapis.com/Topic"
|
36
|
-
#
|
37
|
-
# pattern: "projects/{project}/topics/{topic}"
|
38
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Project"
|
39
|
-
# parent_name_extractor: "projects/{project}"
|
40
|
-
# }
|
36
|
+
# pattern: "projects/{project}/topics/{topic}"
|
41
37
|
# };
|
42
38
|
# }
|
43
39
|
#
|
@@ -45,10 +41,7 @@ module Google
|
|
45
41
|
#
|
46
42
|
# resources:
|
47
43
|
# - type: "pubsub.googleapis.com/Topic"
|
48
|
-
#
|
49
|
-
# - pattern: "projects/{project}/topics/{topic}"
|
50
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Project"
|
51
|
-
# parent_name_extractor: "projects/{project}"
|
44
|
+
# pattern: "projects/{project}/topics/{topic}"
|
52
45
|
#
|
53
46
|
# Sometimes, resources have multiple patterns, typically because they can
|
54
47
|
# live under multiple parents.
|
@@ -58,26 +51,10 @@ module Google
|
|
58
51
|
# message LogEntry {
|
59
52
|
# option (google.api.resource) = {
|
60
53
|
# type: "logging.googleapis.com/LogEntry"
|
61
|
-
#
|
62
|
-
#
|
63
|
-
#
|
64
|
-
#
|
65
|
-
# }
|
66
|
-
# name_descriptor: {
|
67
|
-
# pattern: "folders/{folder}/logs/{log}"
|
68
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Folder"
|
69
|
-
# parent_name_extractor: "folders/{folder}"
|
70
|
-
# }
|
71
|
-
# name_descriptor: {
|
72
|
-
# pattern: "organizations/{organization}/logs/{log}"
|
73
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Organization"
|
74
|
-
# parent_name_extractor: "organizations/{organization}"
|
75
|
-
# }
|
76
|
-
# name_descriptor: {
|
77
|
-
# pattern: "billingAccounts/{billing_account}/logs/{log}"
|
78
|
-
# parent_type: "billing.googleapis.com/BillingAccount"
|
79
|
-
# parent_name_extractor: "billingAccounts/{billing_account}"
|
80
|
-
# }
|
54
|
+
# pattern: "projects/{project}/logs/{log}"
|
55
|
+
# pattern: "folders/{folder}/logs/{log}"
|
56
|
+
# pattern: "organizations/{organization}/logs/{log}"
|
57
|
+
# pattern: "billingAccounts/{billing_account}/logs/{log}"
|
81
58
|
# };
|
82
59
|
# }
|
83
60
|
#
|
@@ -85,48 +62,10 @@ module Google
|
|
85
62
|
#
|
86
63
|
# resources:
|
87
64
|
# - type: 'logging.googleapis.com/LogEntry'
|
88
|
-
#
|
89
|
-
#
|
90
|
-
#
|
91
|
-
#
|
92
|
-
# - pattern: "folders/{folder}/logs/{log}"
|
93
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Folder"
|
94
|
-
# parent_name_extractor: "folders/{folder}"
|
95
|
-
# - pattern: "organizations/{organization}/logs/{log}"
|
96
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Organization"
|
97
|
-
# parent_name_extractor: "organizations/{organization}"
|
98
|
-
# - pattern: "billingAccounts/{billing_account}/logs/{log}"
|
99
|
-
# parent_type: "billing.googleapis.com/BillingAccount"
|
100
|
-
# parent_name_extractor: "billingAccounts/{billing_account}"
|
101
|
-
#
|
102
|
-
# For flexible resources, the resource name doesn't contain parent names, but
|
103
|
-
# the resource itself has parents for policy evaluation.
|
104
|
-
#
|
105
|
-
# Example:
|
106
|
-
#
|
107
|
-
# message Shelf {
|
108
|
-
# option (google.api.resource) = {
|
109
|
-
# type: "library.googleapis.com/Shelf"
|
110
|
-
# name_descriptor: {
|
111
|
-
# pattern: "shelves/{shelf}"
|
112
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Project"
|
113
|
-
# }
|
114
|
-
# name_descriptor: {
|
115
|
-
# pattern: "shelves/{shelf}"
|
116
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Folder"
|
117
|
-
# }
|
118
|
-
# };
|
119
|
-
# }
|
120
|
-
#
|
121
|
-
# The ResourceDescriptor Yaml config will look like:
|
122
|
-
#
|
123
|
-
# resources:
|
124
|
-
# - type: 'library.googleapis.com/Shelf'
|
125
|
-
# name_descriptor:
|
126
|
-
# - pattern: "shelves/{shelf}"
|
127
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Project"
|
128
|
-
# - pattern: "shelves/{shelf}"
|
129
|
-
# parent_type: "cloudresourcemanager.googleapis.com/Folder"
|
65
|
+
# pattern: "projects/{project}/logs/{log}"
|
66
|
+
# pattern: "folders/{folder}/logs/{log}"
|
67
|
+
# pattern: "organizations/{organization}/logs/{log}"
|
68
|
+
# pattern: "billingAccounts/{billing_account}/logs/{log}"
|
130
69
|
# @!attribute [rw] type
|
131
70
|
# @return [::String]
|
132
71
|
# The resource type. It must be in the format of
|
@@ -32,8 +32,18 @@ module Google
|
|
32
32
|
# unique. Names of deleted clusters can be reused.
|
33
33
|
# @!attribute [rw] config
|
34
34
|
# @return [::Google::Cloud::Dataproc::V1::ClusterConfig]
|
35
|
-
#
|
36
|
-
# default values, and values may change
|
35
|
+
# Optional. The cluster config for a cluster of Compute Engine Instances.
|
36
|
+
# Note that Dataproc may set default values, and values may change
|
37
|
+
# when clusters are updated.
|
38
|
+
# @!attribute [rw] virtual_cluster_config
|
39
|
+
# @return [::Google::Cloud::Dataproc::V1::VirtualClusterConfig]
|
40
|
+
# Optional. The virtual cluster config, used when creating a Dataproc cluster that
|
41
|
+
# does not directly control the underlying compute resources, for example,
|
42
|
+
# when creating a [Dataproc-on-GKE
|
43
|
+
# cluster](https://cloud.google.com/dataproc/docs/concepts/jobs/dataproc-gke#create-a-dataproc-on-gke-cluster).
|
44
|
+
# Note that Dataproc may set default values, and values may change when
|
45
|
+
# clusters are updated. Exactly one of config or virtualClusterConfig must be
|
46
|
+
# specified.
|
37
47
|
# @!attribute [rw] labels
|
38
48
|
# @return [::Google::Protobuf::Map{::String => ::String}]
|
39
49
|
# Optional. The labels to associate with this cluster.
|
@@ -155,37 +165,48 @@ module Google
|
|
155
165
|
# @!attribute [rw] metastore_config
|
156
166
|
# @return [::Google::Cloud::Dataproc::V1::MetastoreConfig]
|
157
167
|
# Optional. Metastore configuration.
|
158
|
-
# @!attribute [rw] gke_cluster_config
|
159
|
-
# @return [::Google::Cloud::Dataproc::V1::GkeClusterConfig]
|
160
|
-
# Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to
|
161
|
-
# Kubernetes. Setting this is considered mutually exclusive with Compute
|
162
|
-
# Engine-based options such as `gce_cluster_config`, `master_config`,
|
163
|
-
# `worker_config`, `secondary_worker_config`, and `autoscaling_config`.
|
164
168
|
class ClusterConfig
|
165
169
|
include ::Google::Protobuf::MessageExts
|
166
170
|
extend ::Google::Protobuf::MessageExts::ClassMethods
|
167
171
|
end
|
168
172
|
|
169
|
-
#
|
170
|
-
#
|
171
|
-
#
|
172
|
-
#
|
173
|
-
|
173
|
+
# Dataproc cluster config for a cluster that does not directly control the
|
174
|
+
# underlying compute resources, such as a [Dataproc-on-GKE
|
175
|
+
# cluster](https://cloud.google.com/dataproc/docs/concepts/jobs/dataproc-gke#create-a-dataproc-on-gke-cluster).
|
176
|
+
# @!attribute [rw] staging_bucket
|
177
|
+
# @return [::String]
|
178
|
+
# Optional. A Storage bucket used to stage job
|
179
|
+
# dependencies, config files, and job driver console output.
|
180
|
+
# If you do not specify a staging bucket, Cloud
|
181
|
+
# Dataproc will determine a Cloud Storage location (US,
|
182
|
+
# ASIA, or EU) for your cluster's staging bucket according to the
|
183
|
+
# Compute Engine zone where your cluster is deployed, and then create
|
184
|
+
# and manage this project-level, per-location bucket (see
|
185
|
+
# [Dataproc staging and temp
|
186
|
+
# buckets](https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging-bucket)).
|
187
|
+
# **This field requires a Cloud Storage bucket name, not a `gs://...` URI to
|
188
|
+
# a Cloud Storage bucket.**
|
189
|
+
# @!attribute [rw] kubernetes_cluster_config
|
190
|
+
# @return [::Google::Cloud::Dataproc::V1::KubernetesClusterConfig]
|
191
|
+
# Required. The configuration for running the Dataproc cluster on Kubernetes.
|
192
|
+
# @!attribute [rw] auxiliary_services_config
|
193
|
+
# @return [::Google::Cloud::Dataproc::V1::AuxiliaryServicesConfig]
|
194
|
+
# Optional. Configuration of auxiliary services used by this cluster.
|
195
|
+
class VirtualClusterConfig
|
174
196
|
include ::Google::Protobuf::MessageExts
|
175
197
|
extend ::Google::Protobuf::MessageExts::ClassMethods
|
198
|
+
end
|
176
199
|
|
177
|
-
|
178
|
-
|
179
|
-
|
180
|
-
|
181
|
-
|
182
|
-
|
183
|
-
|
184
|
-
|
185
|
-
|
186
|
-
|
187
|
-
extend ::Google::Protobuf::MessageExts::ClassMethods
|
188
|
-
end
|
200
|
+
# Auxiliary services configuration for a Cluster.
|
201
|
+
# @!attribute [rw] metastore_config
|
202
|
+
# @return [::Google::Cloud::Dataproc::V1::MetastoreConfig]
|
203
|
+
# Optional. The Hive Metastore configuration for this workload.
|
204
|
+
# @!attribute [rw] spark_history_server_config
|
205
|
+
# @return [::Google::Cloud::Dataproc::V1::SparkHistoryServerConfig]
|
206
|
+
# Optional. The Spark History Server configuration for the workload.
|
207
|
+
class AuxiliaryServicesConfig
|
208
|
+
include ::Google::Protobuf::MessageExts
|
209
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
189
210
|
end
|
190
211
|
|
191
212
|
# Endpoint config for this cluster
|
@@ -588,6 +609,13 @@ module Google
|
|
588
609
|
# If one or more SSDs are attached, this runtime bulk
|
589
610
|
# data is spread across them, and the boot disk contains only basic
|
590
611
|
# config and installed binaries.
|
612
|
+
# @!attribute [rw] local_ssd_interface
|
613
|
+
# @return [::String]
|
614
|
+
# Optional. Interface type of local SSDs (default is "scsi").
|
615
|
+
# Valid values: "scsi" (Small Computer System Interface),
|
616
|
+
# "nvme" (Non-Volatile Memory Express).
|
617
|
+
# See [local SSD
|
618
|
+
# performance](https://cloud.google.com/compute/docs/disks/local-ssd#performance).
|
591
619
|
class DiskConfig
|
592
620
|
include ::Google::Protobuf::MessageExts
|
593
621
|
extend ::Google::Protobuf::MessageExts::ClassMethods
|
@@ -640,6 +668,10 @@ module Google
|
|
640
668
|
CREATING = 1
|
641
669
|
|
642
670
|
# The cluster is currently running and healthy. It is ready for use.
|
671
|
+
#
|
672
|
+
# **Note:** The cluster state changes from "creating" to "running" status
|
673
|
+
# after the master node(s), first two primary worker nodes (and the last
|
674
|
+
# primary worker node if primary workers > 2) are running.
|
643
675
|
RUNNING = 2
|
644
676
|
|
645
677
|
# The cluster encountered an error. It is not ready for use.
|
@@ -785,11 +785,23 @@ module Google
|
|
785
785
|
# 4 times within 10 minute window.
|
786
786
|
#
|
787
787
|
# Maximum value is 10.
|
788
|
+
#
|
789
|
+
# **Note:** Currently, this restartable job option is
|
790
|
+
# not supported in Dataproc
|
791
|
+
# [workflow
|
792
|
+
# template](https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template)
|
793
|
+
# jobs.
|
788
794
|
# @!attribute [rw] max_failures_total
|
789
795
|
# @return [::Integer]
|
790
796
|
# Optional. Maximum number of times in total a driver may be restarted as a result of
|
791
797
|
# driver exiting with non-zero code before job is reported failed.
|
792
798
|
# Maximum value is 240.
|
799
|
+
#
|
800
|
+
# **Note:** Currently, this restartable job option is
|
801
|
+
# not supported in Dataproc
|
802
|
+
# [workflow
|
803
|
+
# template](https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template)
|
804
|
+
# jobs.
|
793
805
|
class JobScheduling
|
794
806
|
include ::Google::Protobuf::MessageExts
|
795
807
|
extend ::Google::Protobuf::MessageExts::ClassMethods
|
@@ -22,6 +22,13 @@ module Google
|
|
22
22
|
module Dataproc
|
23
23
|
module V1
|
24
24
|
# Runtime configuration for a workload.
|
25
|
+
# @!attribute [rw] version
|
26
|
+
# @return [::String]
|
27
|
+
# Optional. Version of the batch runtime.
|
28
|
+
# @!attribute [rw] container_image
|
29
|
+
# @return [::String]
|
30
|
+
# Optional. Optional custom container image for the job runtime environment. If
|
31
|
+
# not specified, a default container image will be used.
|
25
32
|
# @!attribute [rw] properties
|
26
33
|
# @return [::Google::Protobuf::Map{::String => ::String}]
|
27
34
|
# Optional. A mapping of property names to values, which are used to configure workload
|
@@ -111,6 +118,9 @@ module Google
|
|
111
118
|
# @!attribute [r] output_uri
|
112
119
|
# @return [::String]
|
113
120
|
# Output only. A URI pointing to the location of the stdout and stderr of the workload.
|
121
|
+
# @!attribute [r] diagnostic_output_uri
|
122
|
+
# @return [::String]
|
123
|
+
# Output only. A URI pointing to the location of the diagnostics tarball.
|
114
124
|
class RuntimeInfo
|
115
125
|
include ::Google::Protobuf::MessageExts
|
116
126
|
extend ::Google::Protobuf::MessageExts::ClassMethods
|
@@ -125,6 +135,215 @@ module Google
|
|
125
135
|
end
|
126
136
|
end
|
127
137
|
|
138
|
+
# The cluster's GKE config.
|
139
|
+
# @!attribute [rw] gke_cluster_target
|
140
|
+
# @return [::String]
|
141
|
+
# Optional. A target GKE cluster to deploy to. It must be in the same project and
|
142
|
+
# region as the Dataproc cluster (the GKE cluster can be zonal or regional).
|
143
|
+
# Format: 'projects/\\{project}/locations/\\{location}/clusters/\\{cluster_id}'
|
144
|
+
# @!attribute [rw] node_pool_target
|
145
|
+
# @return [::Array<::Google::Cloud::Dataproc::V1::GkeNodePoolTarget>]
|
146
|
+
# Optional. GKE NodePools where workloads will be scheduled. At least one node pool
|
147
|
+
# must be assigned the 'default' role. Each role can be given to only a
|
148
|
+
# single NodePoolTarget. All NodePools must have the same location settings.
|
149
|
+
# If a nodePoolTarget is not specified, Dataproc constructs a default
|
150
|
+
# nodePoolTarget.
|
151
|
+
class GkeClusterConfig
|
152
|
+
include ::Google::Protobuf::MessageExts
|
153
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
154
|
+
end
|
155
|
+
|
156
|
+
# The configuration for running the Dataproc cluster on Kubernetes.
|
157
|
+
# @!attribute [rw] kubernetes_namespace
|
158
|
+
# @return [::String]
|
159
|
+
# Optional. A namespace within the Kubernetes cluster to deploy into. If this namespace
|
160
|
+
# does not exist, it is created. If it exists, Dataproc
|
161
|
+
# verifies that another Dataproc VirtualCluster is not installed
|
162
|
+
# into it. If not specified, the name of the Dataproc Cluster is used.
|
163
|
+
# @!attribute [rw] gke_cluster_config
|
164
|
+
# @return [::Google::Cloud::Dataproc::V1::GkeClusterConfig]
|
165
|
+
# Required. The configuration for running the Dataproc cluster on GKE.
|
166
|
+
# @!attribute [rw] kubernetes_software_config
|
167
|
+
# @return [::Google::Cloud::Dataproc::V1::KubernetesSoftwareConfig]
|
168
|
+
# Optional. The software configuration for this Dataproc cluster running on Kubernetes.
|
169
|
+
class KubernetesClusterConfig
|
170
|
+
include ::Google::Protobuf::MessageExts
|
171
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
172
|
+
end
|
173
|
+
|
174
|
+
# The software configuration for this Dataproc cluster running on Kubernetes.
|
175
|
+
# @!attribute [rw] component_version
|
176
|
+
# @return [::Google::Protobuf::Map{::String => ::String}]
|
177
|
+
# The components that should be installed in this Dataproc cluster. The key
|
178
|
+
# must be a string from the KubernetesComponent enumeration. The value is
|
179
|
+
# the version of the software to be installed.
|
180
|
+
# At least one entry must be specified.
|
181
|
+
# @!attribute [rw] properties
|
182
|
+
# @return [::Google::Protobuf::Map{::String => ::String}]
|
183
|
+
# The properties to set on daemon config files.
|
184
|
+
#
|
185
|
+
# Property keys are specified in `prefix:property` format, for example
|
186
|
+
# `spark:spark.kubernetes.container.image`. The following are supported
|
187
|
+
# prefixes and their mappings:
|
188
|
+
#
|
189
|
+
# * spark: `spark-defaults.conf`
|
190
|
+
#
|
191
|
+
# For more information, see [Cluster
|
192
|
+
# properties](https://cloud.google.com/dataproc/docs/concepts/cluster-properties).
|
193
|
+
class KubernetesSoftwareConfig
|
194
|
+
include ::Google::Protobuf::MessageExts
|
195
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
196
|
+
|
197
|
+
# @!attribute [rw] key
|
198
|
+
# @return [::String]
|
199
|
+
# @!attribute [rw] value
|
200
|
+
# @return [::String]
|
201
|
+
class ComponentVersionEntry
|
202
|
+
include ::Google::Protobuf::MessageExts
|
203
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
204
|
+
end
|
205
|
+
|
206
|
+
# @!attribute [rw] key
|
207
|
+
# @return [::String]
|
208
|
+
# @!attribute [rw] value
|
209
|
+
# @return [::String]
|
210
|
+
class PropertiesEntry
|
211
|
+
include ::Google::Protobuf::MessageExts
|
212
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
213
|
+
end
|
214
|
+
end
|
215
|
+
|
216
|
+
# GKE NodePools that Dataproc workloads run on.
|
217
|
+
# @!attribute [rw] node_pool
|
218
|
+
# @return [::String]
|
219
|
+
# Required. The target GKE NodePool.
|
220
|
+
# Format:
|
221
|
+
# 'projects/\\{project}/locations/\\{location}/clusters/\\{cluster}/nodePools/\\{node_pool}'
|
222
|
+
# @!attribute [rw] roles
|
223
|
+
# @return [::Array<::Google::Cloud::Dataproc::V1::GkeNodePoolTarget::Role>]
|
224
|
+
# Required. The types of role for a GKE NodePool
|
225
|
+
# @!attribute [rw] node_pool_config
|
226
|
+
# @return [::Google::Cloud::Dataproc::V1::GkeNodePoolConfig]
|
227
|
+
# Optional. The configuration for the GKE NodePool.
|
228
|
+
#
|
229
|
+
# If specified, Dataproc attempts to create a NodePool with the
|
230
|
+
# specified shape. If one with the same name already exists, it is
|
231
|
+
# verified against all specified fields. If a field differs, the
|
232
|
+
# virtual cluster creation will fail.
|
233
|
+
#
|
234
|
+
# If omitted, any NodePool with the specified name is used. If a
|
235
|
+
# NodePool with the specified name does not exist, Dataproc create a NodePool
|
236
|
+
# with default values.
|
237
|
+
class GkeNodePoolTarget
|
238
|
+
include ::Google::Protobuf::MessageExts
|
239
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
240
|
+
|
241
|
+
# `Role` specifies whose tasks will run on the NodePool. The roles can be
|
242
|
+
# specific to workloads. Exactly one GkeNodePoolTarget within the
|
243
|
+
# VirtualCluster must have 'default' role, which is used to run all workloads
|
244
|
+
# that are not associated with a NodePool.
|
245
|
+
module Role
|
246
|
+
# Role is unspecified.
|
247
|
+
ROLE_UNSPECIFIED = 0
|
248
|
+
|
249
|
+
# Any roles that are not directly assigned to a NodePool run on the
|
250
|
+
# `default` role's NodePool.
|
251
|
+
DEFAULT = 1
|
252
|
+
|
253
|
+
# Run controllers and webhooks.
|
254
|
+
CONTROLLER = 2
|
255
|
+
|
256
|
+
# Run spark driver.
|
257
|
+
SPARK_DRIVER = 3
|
258
|
+
|
259
|
+
# Run spark executors.
|
260
|
+
SPARK_EXECUTOR = 4
|
261
|
+
end
|
262
|
+
end
|
263
|
+
|
264
|
+
# The configuration of a GKE NodePool used by a [Dataproc-on-GKE
|
265
|
+
# cluster](https://cloud.google.com/dataproc/docs/concepts/jobs/dataproc-gke#create-a-dataproc-on-gke-cluster).
|
266
|
+
# @!attribute [rw] config
|
267
|
+
# @return [::Google::Cloud::Dataproc::V1::GkeNodePoolConfig::GkeNodeConfig]
|
268
|
+
# Optional. The node pool configuration.
|
269
|
+
# @!attribute [rw] locations
|
270
|
+
# @return [::Array<::String>]
|
271
|
+
# Optional. The list of Compute Engine
|
272
|
+
# [zones](https://cloud.google.com/compute/docs/zones#available) where
|
273
|
+
# NodePool's nodes will be located.
|
274
|
+
#
|
275
|
+
# **Note:** Currently, only one zone may be specified.
|
276
|
+
#
|
277
|
+
# If a location is not specified during NodePool creation, Dataproc will
|
278
|
+
# choose a location.
|
279
|
+
# @!attribute [rw] autoscaling
|
280
|
+
# @return [::Google::Cloud::Dataproc::V1::GkeNodePoolConfig::GkeNodePoolAutoscalingConfig]
|
281
|
+
# Optional. The autoscaler configuration for this NodePool. The autoscaler is enabled
|
282
|
+
# only when a valid configuration is present.
|
283
|
+
class GkeNodePoolConfig
|
284
|
+
include ::Google::Protobuf::MessageExts
|
285
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
286
|
+
|
287
|
+
# Parameters that describe cluster nodes.
|
288
|
+
# @!attribute [rw] machine_type
|
289
|
+
# @return [::String]
|
290
|
+
# Optional. The name of a Compute Engine [machine
|
291
|
+
# type](https://cloud.google.com/compute/docs/machine-types).
|
292
|
+
# @!attribute [rw] preemptible
|
293
|
+
# @return [::Boolean]
|
294
|
+
# Optional. Whether the nodes are created as [preemptible VM
|
295
|
+
# instances](https://cloud.google.com/compute/docs/instances/preemptible).
|
296
|
+
# @!attribute [rw] local_ssd_count
|
297
|
+
# @return [::Integer]
|
298
|
+
# Optional. The number of local SSD disks to attach to the node, which is limited by
|
299
|
+
# the maximum number of disks allowable per zone (see [Adding Local
|
300
|
+
# SSDs](https://cloud.google.com/compute/docs/disks/local-ssd)).
|
301
|
+
# @!attribute [rw] accelerators
|
302
|
+
# @return [::Array<::Google::Cloud::Dataproc::V1::GkeNodePoolConfig::GkeNodePoolAcceleratorConfig>]
|
303
|
+
# Optional. A list of [hardware
|
304
|
+
# accelerators](https://cloud.google.com/compute/docs/gpus) to attach to
|
305
|
+
# each node.
|
306
|
+
# @!attribute [rw] min_cpu_platform
|
307
|
+
# @return [::String]
|
308
|
+
# Optional. [Minimum CPU
|
309
|
+
# platform](https://cloud.google.com/compute/docs/instances/specify-min-cpu-platform)
|
310
|
+
# to be used by this instance. The instance may be scheduled on the
|
311
|
+
# specified or a newer CPU platform. Specify the friendly names of CPU
|
312
|
+
# platforms, such as "Intel Haswell"` or Intel Sandy Bridge".
|
313
|
+
class GkeNodeConfig
|
314
|
+
include ::Google::Protobuf::MessageExts
|
315
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
316
|
+
end
|
317
|
+
|
318
|
+
# A GkeNodeConfigAcceleratorConfig represents a Hardware Accelerator request
|
319
|
+
# for a NodePool.
|
320
|
+
# @!attribute [rw] accelerator_count
|
321
|
+
# @return [::Integer]
|
322
|
+
# The number of accelerator cards exposed to an instance.
|
323
|
+
# @!attribute [rw] accelerator_type
|
324
|
+
# @return [::String]
|
325
|
+
# The accelerator type resource namename (see GPUs on Compute Engine).
|
326
|
+
class GkeNodePoolAcceleratorConfig
|
327
|
+
include ::Google::Protobuf::MessageExts
|
328
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
329
|
+
end
|
330
|
+
|
331
|
+
# GkeNodePoolAutoscaling contains information the cluster autoscaler needs to
|
332
|
+
# adjust the size of the node pool to the current cluster usage.
|
333
|
+
# @!attribute [rw] min_node_count
|
334
|
+
# @return [::Integer]
|
335
|
+
# The minimum number of nodes in the NodePool. Must be >= 0 and <=
|
336
|
+
# max_node_count.
|
337
|
+
# @!attribute [rw] max_node_count
|
338
|
+
# @return [::Integer]
|
339
|
+
# The maximum number of nodes in the NodePool. Must be >= min_node_count.
|
340
|
+
# **Note:** Quota must be sufficient to scale up the cluster.
|
341
|
+
class GkeNodePoolAutoscalingConfig
|
342
|
+
include ::Google::Protobuf::MessageExts
|
343
|
+
extend ::Google::Protobuf::MessageExts::ClassMethods
|
344
|
+
end
|
345
|
+
end
|
346
|
+
|
128
347
|
# Cluster components that can be activated.
|
129
348
|
module Component
|
130
349
|
# Unspecified component. Specifying this will cause Cluster creation to fail.
|
@@ -522,7 +522,7 @@ module Google
|
|
522
522
|
# Required. The resource name of the region or location, as described
|
523
523
|
# in https://cloud.google.com/apis/design/resource_names.
|
524
524
|
#
|
525
|
-
# * For `projects.regions.workflowTemplates
|
525
|
+
# * For `projects.regions.workflowTemplates.create`, the resource name of the
|
526
526
|
# region has the following format:
|
527
527
|
# `projects/{project_id}/regions/{region}`
|
528
528
|
#
|
@@ -44,7 +44,7 @@ module Google
|
|
44
44
|
# foo = any.unpack(Foo.class);
|
45
45
|
# }
|
46
46
|
#
|
47
|
-
#
|
47
|
+
# Example 3: Pack and unpack a message in Python.
|
48
48
|
#
|
49
49
|
# foo = Foo(...)
|
50
50
|
# any = Any()
|
@@ -54,7 +54,7 @@ module Google
|
|
54
54
|
# any.Unpack(foo)
|
55
55
|
# ...
|
56
56
|
#
|
57
|
-
#
|
57
|
+
# Example 4: Pack and unpack a message in Go
|
58
58
|
#
|
59
59
|
# foo := &pb.Foo{...}
|
60
60
|
# any, err := anypb.New(foo)
|
@@ -75,7 +75,7 @@ module Google
|
|
75
75
|
#
|
76
76
|
#
|
77
77
|
# JSON
|
78
|
-
#
|
78
|
+
#
|
79
79
|
# The JSON representation of an `Any` value uses the regular
|
80
80
|
# representation of the deserialized, embedded message, with an
|
81
81
|
# additional field `@type` which contains the type URL. Example:
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: google-cloud-dataproc-v1
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.10.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Google LLC
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2022-05-13 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: gapic-common
|
@@ -243,7 +243,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
243
243
|
- !ruby/object:Gem::Version
|
244
244
|
version: '0'
|
245
245
|
requirements: []
|
246
|
-
rubygems_version: 3.
|
246
|
+
rubygems_version: 3.3.5
|
247
247
|
signing_key:
|
248
248
|
specification_version: 4
|
249
249
|
summary: API Client library for the Cloud Dataproc V1 API
|