google-cloud-dataflow 0.5.1 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +32 -2
- data/lib/google/cloud/dataflow/version.rb +1 -1
- data/lib/google/cloud/dataflow.rb +216 -0
- metadata +4 -7
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c9a5c687f6cca7322b2fa8021c17eaf5bb070a5380ba27153c32fcadbc6774f9
|
4
|
+
data.tar.gz: a2b85664420e858d35e89e3ef21b49d0b6d0008cba8e42ba1ab0f62028bde4c4
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e864ccf24544372f5acf708b3dcd7089483d5c6f29f4f743f71f56997a6add1f797802a4d1d43e1ebf13da71dea4cfe3cd31ddb896b468a5c827b70eaedcc4d7
|
7
|
+
data.tar.gz: e98aeb2a53b55ca2cc0df6c73978199ee305cf10f0114c4b57514ef39bf566e952538723560e3e33b959e4e7d100f5f69d0f7875da2638d6818d62d0d27fdca3
|
data/README.md
CHANGED
@@ -7,7 +7,7 @@ Dataflow is a managed service for executing a wide variety of data processing pa
|
|
7
7
|
Actual client classes for the various versions of this API are defined in
|
8
8
|
_versioned_ client gems, with names of the form `google-cloud-dataflow-v*`.
|
9
9
|
The gem `google-cloud-dataflow` is the main client library that brings the
|
10
|
-
|
10
|
+
versioned gems in as dependencies, and provides high-level methods for
|
11
11
|
constructing clients. More information on versioned clients can be found below
|
12
12
|
in the section titled *Which client should I use?*.
|
13
13
|
|
@@ -34,9 +34,39 @@ In order to use this library, you first need to go through the following steps:
|
|
34
34
|
1. [Enable the API.](https://console.cloud.google.com/apis/library/dataflow.googleapis.com)
|
35
35
|
1. {file:AUTHENTICATION.md Set up authentication.}
|
36
36
|
|
37
|
+
## Debug Logging
|
38
|
+
|
39
|
+
This library comes with opt-in Debug Logging that can help you troubleshoot
|
40
|
+
your application's integration with the API. When logging is activated, key
|
41
|
+
events such as requests and responses, along with data payloads and metadata
|
42
|
+
such as headers and client configuration, are logged to the standard error
|
43
|
+
stream.
|
44
|
+
|
45
|
+
**WARNING:** Client Library Debug Logging includes your data payloads in
|
46
|
+
plaintext, which could include sensitive data such as PII for yourself or your
|
47
|
+
customers, private keys, or other security data that could be compromising if
|
48
|
+
leaked. Always practice good data hygiene with your application logs, and follow
|
49
|
+
the principle of least access. Google also recommends that Client Library Debug
|
50
|
+
Logging be enabled only temporarily during active debugging, and not used
|
51
|
+
permanently in production.
|
52
|
+
|
53
|
+
To enable logging, set the environment variable `GOOGLE_SDK_RUBY_LOGGING_GEMS`
|
54
|
+
to the value `all`. Alternatively, you can set the value to a comma-delimited
|
55
|
+
list of client library gem names. This will select the default logging behavior,
|
56
|
+
which writes logs to the standard error stream. On a local workstation, this may
|
57
|
+
result in logs appearing on the console. When running on a Google Cloud hosting
|
58
|
+
service such as [Google Cloud Run](https://cloud.google.com/run), this generally
|
59
|
+
results in logs appearing alongside your application logs in the
|
60
|
+
[Google Cloud Logging](https://cloud.google.com/logging/) service.
|
61
|
+
|
62
|
+
Debug logging also requires that the versioned clients for this service be
|
63
|
+
sufficiently recent, released after about Dec 10, 2024. If logging is not
|
64
|
+
working, try updating the versioned clients in your bundle or installed gems:
|
65
|
+
[google-cloud-dataflow-v1beta3](https://cloud.google.com/ruby/docs/reference/google-cloud-dataflow-v1beta3/latest).
|
66
|
+
|
37
67
|
## Supported Ruby Versions
|
38
68
|
|
39
|
-
This library is supported on Ruby
|
69
|
+
This library is supported on Ruby 3.0+.
|
40
70
|
|
41
71
|
Google provides official support for Ruby versions that are actively supported
|
42
72
|
by Ruby Core—that is, Ruby versions that are either in normal maintenance or
|
@@ -58,6 +58,11 @@ module Google
|
|
58
58
|
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
59
59
|
# the `transport` parameter.
|
60
60
|
#
|
61
|
+
# Raises an exception if the currently installed versioned client gem for the
|
62
|
+
# given API version does not support the given transport of the Snapshots service.
|
63
|
+
# You can determine whether the method will succeed by calling
|
64
|
+
# {Google::Cloud::Dataflow.snapshots_available?}.
|
65
|
+
#
|
61
66
|
# ## About Snapshots
|
62
67
|
#
|
63
68
|
# Provides methods to manage snapshots of Google Cloud Dataflow jobs.
|
@@ -79,6 +84,37 @@ module Google
|
|
79
84
|
service_module.const_get(:Client).new(&block)
|
80
85
|
end
|
81
86
|
|
87
|
+
##
|
88
|
+
# Determines whether the Snapshots service is supported by the current client.
|
89
|
+
# If true, you can retrieve a client object by calling {Google::Cloud::Dataflow.snapshots}.
|
90
|
+
# If false, that method will raise an exception. This could happen if the given
|
91
|
+
# API version does not exist or does not support the Snapshots service,
|
92
|
+
# or if the versioned client gem needs an update to support the Snapshots service.
|
93
|
+
#
|
94
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
95
|
+
# Defaults to `:v1beta3`.
|
96
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
97
|
+
# @return [boolean] Whether the service is available.
|
98
|
+
#
|
99
|
+
def self.snapshots_available? version: :v1beta3, transport: :grpc
|
100
|
+
require "google/cloud/dataflow/#{version.to_s.downcase}"
|
101
|
+
package_name = Google::Cloud::Dataflow
|
102
|
+
.constants
|
103
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
104
|
+
.first
|
105
|
+
return false unless package_name
|
106
|
+
service_module = Google::Cloud::Dataflow.const_get package_name
|
107
|
+
return false unless service_module.const_defined? :Snapshots
|
108
|
+
service_module = service_module.const_get :Snapshots
|
109
|
+
if transport == :rest
|
110
|
+
return false unless service_module.const_defined? :Rest
|
111
|
+
service_module = service_module.const_get :Rest
|
112
|
+
end
|
113
|
+
service_module.const_defined? :Client
|
114
|
+
rescue ::LoadError
|
115
|
+
false
|
116
|
+
end
|
117
|
+
|
82
118
|
##
|
83
119
|
# Create a new client object for Jobs.
|
84
120
|
#
|
@@ -92,6 +128,11 @@ module Google
|
|
92
128
|
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
93
129
|
# the `transport` parameter.
|
94
130
|
#
|
131
|
+
# Raises an exception if the currently installed versioned client gem for the
|
132
|
+
# given API version does not support the given transport of the Jobs service.
|
133
|
+
# You can determine whether the method will succeed by calling
|
134
|
+
# {Google::Cloud::Dataflow.jobs_available?}.
|
135
|
+
#
|
95
136
|
# ## About Jobs
|
96
137
|
#
|
97
138
|
# Provides a method to create and modify Google Cloud Dataflow jobs.
|
@@ -114,6 +155,37 @@ module Google
|
|
114
155
|
service_module.const_get(:Client).new(&block)
|
115
156
|
end
|
116
157
|
|
158
|
+
##
|
159
|
+
# Determines whether the Jobs service is supported by the current client.
|
160
|
+
# If true, you can retrieve a client object by calling {Google::Cloud::Dataflow.jobs}.
|
161
|
+
# If false, that method will raise an exception. This could happen if the given
|
162
|
+
# API version does not exist or does not support the Jobs service,
|
163
|
+
# or if the versioned client gem needs an update to support the Jobs service.
|
164
|
+
#
|
165
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
166
|
+
# Defaults to `:v1beta3`.
|
167
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
168
|
+
# @return [boolean] Whether the service is available.
|
169
|
+
#
|
170
|
+
def self.jobs_available? version: :v1beta3, transport: :grpc
|
171
|
+
require "google/cloud/dataflow/#{version.to_s.downcase}"
|
172
|
+
package_name = Google::Cloud::Dataflow
|
173
|
+
.constants
|
174
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
175
|
+
.first
|
176
|
+
return false unless package_name
|
177
|
+
service_module = Google::Cloud::Dataflow.const_get package_name
|
178
|
+
return false unless service_module.const_defined? :Jobs
|
179
|
+
service_module = service_module.const_get :Jobs
|
180
|
+
if transport == :rest
|
181
|
+
return false unless service_module.const_defined? :Rest
|
182
|
+
service_module = service_module.const_get :Rest
|
183
|
+
end
|
184
|
+
service_module.const_defined? :Client
|
185
|
+
rescue ::LoadError
|
186
|
+
false
|
187
|
+
end
|
188
|
+
|
117
189
|
##
|
118
190
|
# Create a new client object for Messages.
|
119
191
|
#
|
@@ -127,6 +199,11 @@ module Google
|
|
127
199
|
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
128
200
|
# the `transport` parameter.
|
129
201
|
#
|
202
|
+
# Raises an exception if the currently installed versioned client gem for the
|
203
|
+
# given API version does not support the given transport of the Messages service.
|
204
|
+
# You can determine whether the method will succeed by calling
|
205
|
+
# {Google::Cloud::Dataflow.messages_available?}.
|
206
|
+
#
|
130
207
|
# ## About Messages
|
131
208
|
#
|
132
209
|
# The Dataflow Messages API is used for monitoring the progress of
|
@@ -149,6 +226,37 @@ module Google
|
|
149
226
|
service_module.const_get(:Client).new(&block)
|
150
227
|
end
|
151
228
|
|
229
|
+
##
|
230
|
+
# Determines whether the Messages service is supported by the current client.
|
231
|
+
# If true, you can retrieve a client object by calling {Google::Cloud::Dataflow.messages}.
|
232
|
+
# If false, that method will raise an exception. This could happen if the given
|
233
|
+
# API version does not exist or does not support the Messages service,
|
234
|
+
# or if the versioned client gem needs an update to support the Messages service.
|
235
|
+
#
|
236
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
237
|
+
# Defaults to `:v1beta3`.
|
238
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
239
|
+
# @return [boolean] Whether the service is available.
|
240
|
+
#
|
241
|
+
def self.messages_available? version: :v1beta3, transport: :grpc
|
242
|
+
require "google/cloud/dataflow/#{version.to_s.downcase}"
|
243
|
+
package_name = Google::Cloud::Dataflow
|
244
|
+
.constants
|
245
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
246
|
+
.first
|
247
|
+
return false unless package_name
|
248
|
+
service_module = Google::Cloud::Dataflow.const_get package_name
|
249
|
+
return false unless service_module.const_defined? :Messages
|
250
|
+
service_module = service_module.const_get :Messages
|
251
|
+
if transport == :rest
|
252
|
+
return false unless service_module.const_defined? :Rest
|
253
|
+
service_module = service_module.const_get :Rest
|
254
|
+
end
|
255
|
+
service_module.const_defined? :Client
|
256
|
+
rescue ::LoadError
|
257
|
+
false
|
258
|
+
end
|
259
|
+
|
152
260
|
##
|
153
261
|
# Create a new client object for Metrics.
|
154
262
|
#
|
@@ -162,6 +270,11 @@ module Google
|
|
162
270
|
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
163
271
|
# the `transport` parameter.
|
164
272
|
#
|
273
|
+
# Raises an exception if the currently installed versioned client gem for the
|
274
|
+
# given API version does not support the given transport of the Metrics service.
|
275
|
+
# You can determine whether the method will succeed by calling
|
276
|
+
# {Google::Cloud::Dataflow.metrics_available?}.
|
277
|
+
#
|
165
278
|
# ## About Metrics
|
166
279
|
#
|
167
280
|
# The Dataflow Metrics API lets you monitor the progress of Dataflow
|
@@ -184,6 +297,37 @@ module Google
|
|
184
297
|
service_module.const_get(:Client).new(&block)
|
185
298
|
end
|
186
299
|
|
300
|
+
##
|
301
|
+
# Determines whether the Metrics service is supported by the current client.
|
302
|
+
# If true, you can retrieve a client object by calling {Google::Cloud::Dataflow.metrics}.
|
303
|
+
# If false, that method will raise an exception. This could happen if the given
|
304
|
+
# API version does not exist or does not support the Metrics service,
|
305
|
+
# or if the versioned client gem needs an update to support the Metrics service.
|
306
|
+
#
|
307
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
308
|
+
# Defaults to `:v1beta3`.
|
309
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
310
|
+
# @return [boolean] Whether the service is available.
|
311
|
+
#
|
312
|
+
def self.metrics_available? version: :v1beta3, transport: :grpc
|
313
|
+
require "google/cloud/dataflow/#{version.to_s.downcase}"
|
314
|
+
package_name = Google::Cloud::Dataflow
|
315
|
+
.constants
|
316
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
317
|
+
.first
|
318
|
+
return false unless package_name
|
319
|
+
service_module = Google::Cloud::Dataflow.const_get package_name
|
320
|
+
return false unless service_module.const_defined? :Metrics
|
321
|
+
service_module = service_module.const_get :Metrics
|
322
|
+
if transport == :rest
|
323
|
+
return false unless service_module.const_defined? :Rest
|
324
|
+
service_module = service_module.const_get :Rest
|
325
|
+
end
|
326
|
+
service_module.const_defined? :Client
|
327
|
+
rescue ::LoadError
|
328
|
+
false
|
329
|
+
end
|
330
|
+
|
187
331
|
##
|
188
332
|
# Create a new client object for TemplatesService.
|
189
333
|
#
|
@@ -197,6 +341,11 @@ module Google
|
|
197
341
|
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
198
342
|
# the `transport` parameter.
|
199
343
|
#
|
344
|
+
# Raises an exception if the currently installed versioned client gem for the
|
345
|
+
# given API version does not support the given transport of the TemplatesService service.
|
346
|
+
# You can determine whether the method will succeed by calling
|
347
|
+
# {Google::Cloud::Dataflow.templates_service_available?}.
|
348
|
+
#
|
200
349
|
# ## About TemplatesService
|
201
350
|
#
|
202
351
|
# Provides a method to create Cloud Dataflow jobs from templates.
|
@@ -218,6 +367,37 @@ module Google
|
|
218
367
|
service_module.const_get(:Client).new(&block)
|
219
368
|
end
|
220
369
|
|
370
|
+
##
|
371
|
+
# Determines whether the TemplatesService service is supported by the current client.
|
372
|
+
# If true, you can retrieve a client object by calling {Google::Cloud::Dataflow.templates_service}.
|
373
|
+
# If false, that method will raise an exception. This could happen if the given
|
374
|
+
# API version does not exist or does not support the TemplatesService service,
|
375
|
+
# or if the versioned client gem needs an update to support the TemplatesService service.
|
376
|
+
#
|
377
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
378
|
+
# Defaults to `:v1beta3`.
|
379
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
380
|
+
# @return [boolean] Whether the service is available.
|
381
|
+
#
|
382
|
+
def self.templates_service_available? version: :v1beta3, transport: :grpc
|
383
|
+
require "google/cloud/dataflow/#{version.to_s.downcase}"
|
384
|
+
package_name = Google::Cloud::Dataflow
|
385
|
+
.constants
|
386
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
387
|
+
.first
|
388
|
+
return false unless package_name
|
389
|
+
service_module = Google::Cloud::Dataflow.const_get package_name
|
390
|
+
return false unless service_module.const_defined? :TemplatesService
|
391
|
+
service_module = service_module.const_get :TemplatesService
|
392
|
+
if transport == :rest
|
393
|
+
return false unless service_module.const_defined? :Rest
|
394
|
+
service_module = service_module.const_get :Rest
|
395
|
+
end
|
396
|
+
service_module.const_defined? :Client
|
397
|
+
rescue ::LoadError
|
398
|
+
false
|
399
|
+
end
|
400
|
+
|
221
401
|
##
|
222
402
|
# Create a new client object for FlexTemplatesService.
|
223
403
|
#
|
@@ -231,6 +411,11 @@ module Google
|
|
231
411
|
# You can also specify a different transport by passing `:rest` or `:grpc` in
|
232
412
|
# the `transport` parameter.
|
233
413
|
#
|
414
|
+
# Raises an exception if the currently installed versioned client gem for the
|
415
|
+
# given API version does not support the given transport of the FlexTemplatesService service.
|
416
|
+
# You can determine whether the method will succeed by calling
|
417
|
+
# {Google::Cloud::Dataflow.flex_templates_service_available?}.
|
418
|
+
#
|
234
419
|
# ## About FlexTemplatesService
|
235
420
|
#
|
236
421
|
# Provides a service for Flex templates. This feature is not ready yet.
|
@@ -252,6 +437,37 @@ module Google
|
|
252
437
|
service_module.const_get(:Client).new(&block)
|
253
438
|
end
|
254
439
|
|
440
|
+
##
|
441
|
+
# Determines whether the FlexTemplatesService service is supported by the current client.
|
442
|
+
# If true, you can retrieve a client object by calling {Google::Cloud::Dataflow.flex_templates_service}.
|
443
|
+
# If false, that method will raise an exception. This could happen if the given
|
444
|
+
# API version does not exist or does not support the FlexTemplatesService service,
|
445
|
+
# or if the versioned client gem needs an update to support the FlexTemplatesService service.
|
446
|
+
#
|
447
|
+
# @param version [::String, ::Symbol] The API version to connect to. Optional.
|
448
|
+
# Defaults to `:v1beta3`.
|
449
|
+
# @param transport [:grpc, :rest] The transport to use. Defaults to `:grpc`.
|
450
|
+
# @return [boolean] Whether the service is available.
|
451
|
+
#
|
452
|
+
def self.flex_templates_service_available? version: :v1beta3, transport: :grpc
|
453
|
+
require "google/cloud/dataflow/#{version.to_s.downcase}"
|
454
|
+
package_name = Google::Cloud::Dataflow
|
455
|
+
.constants
|
456
|
+
.select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
|
457
|
+
.first
|
458
|
+
return false unless package_name
|
459
|
+
service_module = Google::Cloud::Dataflow.const_get package_name
|
460
|
+
return false unless service_module.const_defined? :FlexTemplatesService
|
461
|
+
service_module = service_module.const_get :FlexTemplatesService
|
462
|
+
if transport == :rest
|
463
|
+
return false unless service_module.const_defined? :Rest
|
464
|
+
service_module = service_module.const_get :Rest
|
465
|
+
end
|
466
|
+
service_module.const_defined? :Client
|
467
|
+
rescue ::LoadError
|
468
|
+
false
|
469
|
+
end
|
470
|
+
|
255
471
|
##
|
256
472
|
# Configure the google-cloud-dataflow library.
|
257
473
|
#
|
metadata
CHANGED
@@ -1,14 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: google-cloud-dataflow
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.6.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Google LLC
|
8
|
-
autorequire:
|
9
8
|
bindir: bin
|
10
9
|
cert_chain: []
|
11
|
-
date:
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
12
11
|
dependencies:
|
13
12
|
- !ruby/object:Gem::Dependency
|
14
13
|
name: google-cloud-core
|
@@ -62,7 +61,6 @@ homepage: https://github.com/googleapis/google-cloud-ruby
|
|
62
61
|
licenses:
|
63
62
|
- Apache-2.0
|
64
63
|
metadata: {}
|
65
|
-
post_install_message:
|
66
64
|
rdoc_options: []
|
67
65
|
require_paths:
|
68
66
|
- lib
|
@@ -70,15 +68,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
70
68
|
requirements:
|
71
69
|
- - ">="
|
72
70
|
- !ruby/object:Gem::Version
|
73
|
-
version: '
|
71
|
+
version: '3.0'
|
74
72
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
75
73
|
requirements:
|
76
74
|
- - ">="
|
77
75
|
- !ruby/object:Gem::Version
|
78
76
|
version: '0'
|
79
77
|
requirements: []
|
80
|
-
rubygems_version: 3.
|
81
|
-
signing_key:
|
78
|
+
rubygems_version: 3.6.8
|
82
79
|
specification_version: 4
|
83
80
|
summary: API Client library for the Dataflow API
|
84
81
|
test_files: []
|