google-cloud-dataflow 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 3224381599ad115bea964d20b2e021d3657222bf895e74fbaad99f1546234c27
4
+ data.tar.gz: 76983b27ae1395f3d3925fd1d7fc717e52024ac1f79a1b66fac2247301b87a14
5
+ SHA512:
6
+ metadata.gz: b940d788cfd943cfd7ef9d27b9aa8cabec9298e9689e90f782e2b4dc9f109b1eb350963c74c65ebd812270c049d9269667585b08af2217844d652e125804267d
7
+ data.tar.gz: 4dc628564e1caad5ffe5aadda75dc99375273298aa3989c30c737dd8249433e27111d52301a7c28a4b132f40c82e83affa25a0fec52cad4c40dd5add3c3b0212
data/.yardopts ADDED
@@ -0,0 +1,11 @@
1
+ --no-private
2
+ --title=Dataflow API
3
+ --exclude _pb\.rb$
4
+ --markup markdown
5
+ --markup-provider redcarpet
6
+
7
+ ./lib/**/*.rb
8
+ -
9
+ README.md
10
+ AUTHENTICATION.md
11
+ LICENSE.md
data/AUTHENTICATION.md ADDED
@@ -0,0 +1,169 @@
1
+ # Authentication
2
+
3
+ In general, the google-cloud-dataflow library uses
4
+ [Service Account](https://cloud.google.com/iam/docs/creating-managing-service-accounts)
5
+ credentials to connect to Google Cloud services. When running within
6
+ [Google Cloud Platform environments](#google-cloud-platform-environments) the
7
+ credentials will be discovered automatically. When running on other
8
+ environments, the Service Account credentials can be specified by providing the
9
+ path to the
10
+ [JSON keyfile](https://cloud.google.com/iam/docs/managing-service-account-keys)
11
+ for the account (or the JSON itself) in
12
+ [environment variables](#environment-variables). Additionally, Cloud SDK
13
+ credentials can also be discovered automatically, but this is only recommended
14
+ during development.
15
+
16
+ ## Quickstart
17
+
18
+ 1. [Create a service account and credentials](#creating-a-service-account).
19
+ 2. Set the [environment variable](#environment-variables).
20
+
21
+ ```sh
22
+ export DATAFLOW_CREDENTIALS=path/to/keyfile.json
23
+ ```
24
+
25
+ 3. Initialize the client.
26
+
27
+ ```ruby
28
+ require "google/cloud/dataflow"
29
+
30
+ client = Google::Cloud::Dataflow.snapshots
31
+ ```
32
+
33
+ ## Credential Lookup
34
+
35
+ The google-cloud-dataflow library aims to make authentication
36
+ as simple as possible, and provides several mechanisms to configure your system
37
+ without requiring **Service Account Credentials** directly in code.
38
+
39
+ **Credentials** are discovered in the following order:
40
+
41
+ 1. Specify credentials in method arguments
42
+ 2. Specify credentials in configuration
43
+ 3. Discover credentials path in environment variables
44
+ 4. Discover credentials JSON in environment variables
45
+ 5. Discover credentials file in the Cloud SDK's path
46
+ 6. Discover GCP credentials
47
+
48
+ ### Google Cloud Platform environments
49
+
50
+ When running on Google Cloud Platform (GCP), including Google Compute Engine
51
+ (GCE), Google Kubernetes Engine (GKE), Google App Engine (GAE), Google Cloud
52
+ Functions (GCF) and Cloud Run, **Credentials** are discovered automatically.
53
+ Code should be written as if already authenticated.
54
+
55
+ ### Environment Variables
56
+
57
+ The **Credentials JSON** can be placed in environment variables instead of
58
+ declaring them directly in code. Each service has its own environment variable,
59
+ allowing for different service accounts to be used for different services. (See
60
+ the READMEs for the individual service gems for details.) The path to the
61
+ **Credentials JSON** file can be stored in the environment variable, or the
62
+ **Credentials JSON** itself can be stored for environments such as Docker
63
+ containers where writing files is difficult or not encouraged.
64
+
65
+ The environment variables that google-cloud-dataflow
66
+ checks for credentials are configured on the service Credentials class (such as
67
+ `::Google::Cloud::Dataflow::V1beta3::Snapshots::Credentials`):
68
+
69
+ 1. `DATAFLOW_CREDENTIALS` - Path to JSON file, or JSON contents
70
+ 2. `DATAFLOW_KEYFILE` - Path to JSON file, or JSON contents
71
+ 3. `GOOGLE_CLOUD_CREDENTIALS` - Path to JSON file, or JSON contents
72
+ 4. `GOOGLE_CLOUD_KEYFILE` - Path to JSON file, or JSON contents
73
+ 5. `GOOGLE_APPLICATION_CREDENTIALS` - Path to JSON file
74
+
75
+ ```ruby
76
+ require "google/cloud/dataflow"
77
+
78
+ ENV["DATAFLOW_CREDENTIALS"] = "path/to/keyfile.json"
79
+
80
+ client = Google::Cloud::Dataflow.snapshots
81
+ ```
82
+
83
+ ### Configuration
84
+
85
+ The **Credentials JSON** can be configured instead of placing them in
86
+ environment variables. Either on an individual client initialization:
87
+
88
+ ```ruby
89
+ require "google/cloud/dataflow"
90
+
91
+ client = Google::Cloud::Dataflow.snapshots do |config|
92
+ config.credentials = "path/to/keyfile.json"
93
+ end
94
+ ```
95
+
96
+ Or configured globally for all clients:
97
+
98
+ ```ruby
99
+ require "google/cloud/dataflow"
100
+
101
+ Google::Cloud::Dataflow.configure do |config|
102
+ config.credentials = "path/to/keyfile.json"
103
+ end
104
+
105
+ client = Google::Cloud::Dataflow.snapshots
106
+ ```
107
+
108
+ ### Cloud SDK
109
+
110
+ This option allows for an easy way to authenticate during development. If
111
+ credentials are not provided in code or in environment variables, then Cloud SDK
112
+ credentials are discovered.
113
+
114
+ To configure your system for this, simply:
115
+
116
+ 1. [Download and install the Cloud SDK](https://cloud.google.com/sdk)
117
+ 2. Authenticate using OAuth 2.0 `$ gcloud auth login`
118
+ 3. Write code as if already authenticated.
119
+
120
+ **NOTE:** This is _not_ recommended for running in production. The Cloud SDK
121
+ *should* only be used during development.
122
+
123
+ [gce-how-to]: https://cloud.google.com/compute/docs/authentication#using
124
+ [dev-console]: https://console.cloud.google.com/project
125
+
126
+ [enable-apis]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/enable-apis.png
127
+
128
+ [create-new-service-account]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/create-new-service-account.png
129
+ [create-new-service-account-existing-keys]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/create-new-service-account-existing-keys.png
130
+ [reuse-service-account]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/reuse-service-account.png
131
+
132
+ ## Creating a Service Account
133
+
134
+ Google Cloud requires **Service Account Credentials** to
135
+ connect to the APIs. You will use the **JSON key file** to
136
+ connect to most services with google-cloud-dataflow.
137
+
138
+ If you are not running this client within
139
+ [Google Cloud Platform environments](#google-cloud-platform-environments), you
140
+ need a Google Developers service account.
141
+
142
+ 1. Visit the [Google Developers Console][dev-console].
143
+ 2. Create a new project or click on an existing project.
144
+ 3. Activate the slide-out navigation tray and select **API Manager**. From
145
+ here, you will enable the APIs that your application requires.
146
+
147
+ ![Enable the APIs that your application requires][enable-apis]
148
+
149
+ *Note: You may need to enable billing in order to use these services.*
150
+
151
+ 4. Select **Credentials** from the side navigation.
152
+
153
+ You should see a screen like one of the following.
154
+
155
+ ![Create a new service account][create-new-service-account]
156
+
157
+ ![Create a new service account With Existing Keys][create-new-service-account-existing-keys]
158
+
159
+ Find the "Add credentials" drop down and select "Service account" to be
160
+ guided through downloading a new JSON key file.
161
+
162
+ If you want to re-use an existing service account, you can easily generate a
163
+ new key file. Just select the account you wish to re-use, and click "Generate
164
+ new JSON key":
165
+
166
+ ![Re-use an existing service account][reuse-service-account]
167
+
168
+ The key file you download will be used by this library to authenticate API
169
+ requests and should be stored in a secure location.
data/LICENSE.md ADDED
@@ -0,0 +1,201 @@
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright [yyyy] [name of copyright owner]
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
data/README.md ADDED
@@ -0,0 +1,130 @@
1
+ # Ruby Client for the Dataflow API
2
+
3
+ API Client library for the Dataflow API
4
+
5
+ Dataflow is a managed service for executing a wide variety of data processing patterns.
6
+
7
+ Actual client classes for the various versions of this API are defined in
8
+ _versioned_ client gems, with names of the form `google-cloud-dataflow-v*`.
9
+ The gem `google-cloud-dataflow` is the main client library that brings the
10
+ verisoned gems in as dependencies, and provides high-level methods for
11
+ constructing clients. More information on versioned clients can be found below
12
+ in the section titled *Which client should I use?*.
13
+
14
+ View the [Client Library Documentation](https://googleapis.dev/ruby/google-cloud-dataflow/latest)
15
+ for this library, google-cloud-dataflow, to see the convenience methods for
16
+ constructing client objects. Reference documentation for the client objects
17
+ themselves can be found in the client library documentation for the versioned
18
+ client gems:
19
+ [google-cloud-dataflow-v1beta3](https://googleapis.dev/ruby/google-cloud-dataflow-v1beta3/latest).
20
+
21
+ See also the [Product Documentation](https://cloud.google.com/dataflow)
22
+ for more usage information.
23
+
24
+ ## Quick Start
25
+
26
+ ```
27
+ $ gem install google-cloud-dataflow
28
+ ```
29
+
30
+ In order to use this library, you first need to go through the following steps:
31
+
32
+ 1. [Select or create a Cloud Platform project.](https://console.cloud.google.com/project)
33
+ 1. [Enable billing for your project.](https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project)
34
+ 1. [Enable the API.](https://console.cloud.google.com/apis/library/dataflow.googleapis.com)
35
+ 1. {file:AUTHENTICATION.md Set up authentication.}
36
+
37
+ ## Enabling Logging
38
+
39
+ To enable logging for this library, set the logger for the underlying [gRPC](https://github.com/grpc/grpc/tree/master/src/ruby) library.
40
+ The logger that you set may be a Ruby stdlib [`Logger`](https://ruby-doc.org/stdlib/libdoc/logger/rdoc/Logger.html) as shown below,
41
+ or a [`Google::Cloud::Logging::Logger`](https://googleapis.dev/ruby/google-cloud-logging/latest)
42
+ that will write logs to [Cloud Logging](https://cloud.google.com/logging/). See [grpc/logconfig.rb](https://github.com/grpc/grpc/blob/master/src/ruby/lib/grpc/logconfig.rb)
43
+ and the gRPC [spec_helper.rb](https://github.com/grpc/grpc/blob/master/src/ruby/spec/spec_helper.rb) for additional information.
44
+
45
+ Configuring a Ruby stdlib logger:
46
+
47
+ ```ruby
48
+ require "logger"
49
+
50
+ module MyLogger
51
+ LOGGER = Logger.new $stderr, level: Logger::WARN
52
+ def logger
53
+ LOGGER
54
+ end
55
+ end
56
+
57
+ # Define a gRPC module-level logger method before grpc/logconfig.rb loads.
58
+ module GRPC
59
+ extend MyLogger
60
+ end
61
+ ```
62
+
63
+ ## Supported Ruby Versions
64
+
65
+ This library is supported on Ruby 2.5+.
66
+
67
+ Google provides official support for Ruby versions that are actively supported
68
+ by Ruby Core—that is, Ruby versions that are either in normal maintenance or
69
+ in security maintenance, and not end of life. Currently, this means Ruby 2.5
70
+ and later. Older versions of Ruby _may_ still work, but are unsupported and not
71
+ recommended. See https://www.ruby-lang.org/en/downloads/branches/ for details
72
+ about the Ruby support schedule.
73
+
74
+ ## Which client should I use?
75
+
76
+ Most modern Ruby client libraries for Google APIs come in two flavors: the main
77
+ client library with a name such as `google-cloud-dataflow`,
78
+ and lower-level _versioned_ client libraries with names such as
79
+ `google-cloud-dataflow-v1beta3`.
80
+ _In most cases, you should install the main client._
81
+
82
+ ### What's the difference between the main client and a versioned client?
83
+
84
+ A _versioned client_ provides a basic set of data types and client classes for
85
+ a _single version_ of a specific service. (That is, for a service with multiple
86
+ versions, there might be a separate versioned client for each service version.)
87
+ Most versioned clients are written and maintained by a code generator.
88
+
89
+ The _main client_ is designed to provide you with the _recommended_ client
90
+ interfaces for the service. There will be only one main client for any given
91
+ service, even a service with multiple versions. The main client includes
92
+ factory methods for constructing the client objects we recommend for most
93
+ users. In some cases, those will be classes provided by an underlying versioned
94
+ client; in other cases, they will be handwritten higher-level client objects
95
+ with additional capabilities, convenience methods, or best practices built in.
96
+ Generally, the main client will default to a recommended service version,
97
+ although in some cases you can override this if you need to talk to a specific
98
+ service version.
99
+
100
+ ### Why would I want to use the main client?
101
+
102
+ We recommend that most users install the main client gem for a service. You can
103
+ identify this gem as the one _without_ a version in its name, e.g.
104
+ `google-cloud-dataflow`.
105
+ The main client is recommended because it will embody the best practices for
106
+ accessing the service, and may also provide more convenient interfaces or
107
+ tighter integration into frameworks and third-party libraries. In addition, the
108
+ documentation and samples published by Google will generally demonstrate use of
109
+ the main client.
110
+
111
+ ### Why would I want to use a versioned client?
112
+
113
+ You can use a versioned client if you are content with a possibly lower-level
114
+ class interface, you explicitly want to avoid features provided by the main
115
+ client, or you want to access a specific service version not be covered by the
116
+ main client. You can identify versioned client gems because the service version
117
+ is part of the name, e.g. `google-cloud-dataflow-v1beta3`.
118
+
119
+ ### What about the google-apis-<name> clients?
120
+
121
+ Client library gems with names that begin with `google-apis-` are based on an
122
+ older code generation technology. They talk to a REST/JSON backend (whereas
123
+ most modern clients talk to a [gRPC](https://grpc.io/) backend) and they may
124
+ not offer the same performance, features, and ease of use provided by more
125
+ modern clients.
126
+
127
+ The `google-apis-` clients have wide coverage across Google services, so you
128
+ might need to use one if there is no modern client available for the service.
129
+ However, if a modern client is available, we generally recommend it over the
130
+ older `google-apis-` clients.
@@ -0,0 +1,19 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2021 Google LLC
4
+ #
5
+ # Licensed under the Apache License, Version 2.0 (the "License");
6
+ # you may not use this file except in compliance with the License.
7
+ # You may obtain a copy of the License at
8
+ #
9
+ # https://www.apache.org/licenses/LICENSE-2.0
10
+ #
11
+ # Unless required by applicable law or agreed to in writing, software
12
+ # distributed under the License is distributed on an "AS IS" BASIS,
13
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14
+ # See the License for the specific language governing permissions and
15
+ # limitations under the License.
16
+
17
+ # Auto-generated by gapic-generator-ruby. DO NOT EDIT!
18
+
19
+ require "google/cloud/dataflow" unless defined? Google::Cloud::Dataflow::VERSION
@@ -0,0 +1,268 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2021 Google LLC
4
+ #
5
+ # Licensed under the Apache License, Version 2.0 (the "License");
6
+ # you may not use this file except in compliance with the License.
7
+ # You may obtain a copy of the License at
8
+ #
9
+ # https://www.apache.org/licenses/LICENSE-2.0
10
+ #
11
+ # Unless required by applicable law or agreed to in writing, software
12
+ # distributed under the License is distributed on an "AS IS" BASIS,
13
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14
+ # See the License for the specific language governing permissions and
15
+ # limitations under the License.
16
+
17
+ # Auto-generated by gapic-generator-ruby. DO NOT EDIT!
18
+
19
+ # Require this file early so that the version constant gets defined before
20
+ # requiring "google/cloud". This is because google-cloud-core will load the
21
+ # entrypoint (gem name) file, which in turn re-requires this file (hence
22
+ # causing a require cycle) unless the version constant is already defined.
23
+ require "google/cloud/dataflow/version"
24
+
25
+ require "googleauth"
26
+ gem "google-cloud-core"
27
+ require "google/cloud" unless defined? ::Google::Cloud.new
28
+ require "google/cloud/config"
29
+
30
+ # Set the default configuration
31
+ ::Google::Cloud.configure.add_config! :dataflow do |config|
32
+ config.add_field! :endpoint, "dataflow.googleapis.com", match: ::String
33
+ config.add_field! :credentials, nil, match: [::String, ::Hash, ::Google::Auth::Credentials]
34
+ config.add_field! :scope, nil, match: [::Array, ::String]
35
+ config.add_field! :lib_name, nil, match: ::String
36
+ config.add_field! :lib_version, nil, match: ::String
37
+ config.add_field! :interceptors, nil, match: ::Array
38
+ config.add_field! :timeout, nil, match: ::Numeric
39
+ config.add_field! :metadata, nil, match: ::Hash
40
+ config.add_field! :retry_policy, nil, match: [::Hash, ::Proc]
41
+ config.add_field! :quota_project, nil, match: ::String
42
+ end
43
+
44
+ module Google
45
+ module Cloud
46
+ module Dataflow
47
+ ##
48
+ # Create a new client object for Snapshots.
49
+ #
50
+ # By default, this returns an instance of
51
+ # [Google::Cloud::Dataflow::V1beta3::Snapshots::Client](https://googleapis.dev/ruby/google-cloud-dataflow-v1beta3/latest/Google/Cloud/Dataflow/V1beta3/Snapshots/Client.html)
52
+ # for version V1beta3 of the API.
53
+ # However, you can specify specify a different API version by passing it in the
54
+ # `version` parameter. If the Snapshots service is
55
+ # supported by that API version, and the corresponding gem is available, the
56
+ # appropriate versioned client will be returned.
57
+ #
58
+ # ## About Snapshots
59
+ #
60
+ # Provides methods to manage snapshots of Google Cloud Dataflow jobs.
61
+ #
62
+ # @param version [::String, ::Symbol] The API version to connect to. Optional.
63
+ # Defaults to `:v1beta3`.
64
+ # @return [Snapshots::Client] A client object for the specified version.
65
+ #
66
+ def self.snapshots version: :v1beta3, &block
67
+ require "google/cloud/dataflow/#{version.to_s.downcase}"
68
+
69
+ package_name = Google::Cloud::Dataflow
70
+ .constants
71
+ .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
72
+ .first
73
+ package_module = Google::Cloud::Dataflow.const_get package_name
74
+ package_module.const_get(:Snapshots).const_get(:Client).new(&block)
75
+ end
76
+
77
+ ##
78
+ # Create a new client object for Jobs.
79
+ #
80
+ # By default, this returns an instance of
81
+ # [Google::Cloud::Dataflow::V1beta3::Jobs::Client](https://googleapis.dev/ruby/google-cloud-dataflow-v1beta3/latest/Google/Cloud/Dataflow/V1beta3/Jobs/Client.html)
82
+ # for version V1beta3 of the API.
83
+ # However, you can specify specify a different API version by passing it in the
84
+ # `version` parameter. If the Jobs service is
85
+ # supported by that API version, and the corresponding gem is available, the
86
+ # appropriate versioned client will be returned.
87
+ #
88
+ # ## About Jobs
89
+ #
90
+ # Provides a method to create and modify Google Cloud Dataflow jobs.
91
+ # A Job is a multi-stage computation graph run by the Cloud Dataflow service.
92
+ #
93
+ # @param version [::String, ::Symbol] The API version to connect to. Optional.
94
+ # Defaults to `:v1beta3`.
95
+ # @return [Jobs::Client] A client object for the specified version.
96
+ #
97
+ def self.jobs version: :v1beta3, &block
98
+ require "google/cloud/dataflow/#{version.to_s.downcase}"
99
+
100
+ package_name = Google::Cloud::Dataflow
101
+ .constants
102
+ .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
103
+ .first
104
+ package_module = Google::Cloud::Dataflow.const_get package_name
105
+ package_module.const_get(:Jobs).const_get(:Client).new(&block)
106
+ end
107
+
108
+ ##
109
+ # Create a new client object for Messages.
110
+ #
111
+ # By default, this returns an instance of
112
+ # [Google::Cloud::Dataflow::V1beta3::Messages::Client](https://googleapis.dev/ruby/google-cloud-dataflow-v1beta3/latest/Google/Cloud/Dataflow/V1beta3/Messages/Client.html)
113
+ # for version V1beta3 of the API.
114
+ # However, you can specify specify a different API version by passing it in the
115
+ # `version` parameter. If the Messages service is
116
+ # supported by that API version, and the corresponding gem is available, the
117
+ # appropriate versioned client will be returned.
118
+ #
119
+ # ## About Messages
120
+ #
121
+ # The Dataflow Messages API is used for monitoring the progress of
122
+ # Dataflow jobs.
123
+ #
124
+ # @param version [::String, ::Symbol] The API version to connect to. Optional.
125
+ # Defaults to `:v1beta3`.
126
+ # @return [Messages::Client] A client object for the specified version.
127
+ #
128
+ def self.messages version: :v1beta3, &block
129
+ require "google/cloud/dataflow/#{version.to_s.downcase}"
130
+
131
+ package_name = Google::Cloud::Dataflow
132
+ .constants
133
+ .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
134
+ .first
135
+ package_module = Google::Cloud::Dataflow.const_get package_name
136
+ package_module.const_get(:Messages).const_get(:Client).new(&block)
137
+ end
138
+
139
+ ##
140
+ # Create a new client object for Metrics.
141
+ #
142
+ # By default, this returns an instance of
143
+ # [Google::Cloud::Dataflow::V1beta3::Metrics::Client](https://googleapis.dev/ruby/google-cloud-dataflow-v1beta3/latest/Google/Cloud/Dataflow/V1beta3/Metrics/Client.html)
144
+ # for version V1beta3 of the API.
145
+ # However, you can specify specify a different API version by passing it in the
146
+ # `version` parameter. If the Metrics service is
147
+ # supported by that API version, and the corresponding gem is available, the
148
+ # appropriate versioned client will be returned.
149
+ #
150
+ # ## About Metrics
151
+ #
152
+ # The Dataflow Metrics API lets you monitor the progress of Dataflow
153
+ # jobs.
154
+ #
155
+ # @param version [::String, ::Symbol] The API version to connect to. Optional.
156
+ # Defaults to `:v1beta3`.
157
+ # @return [Metrics::Client] A client object for the specified version.
158
+ #
159
+ def self.metrics version: :v1beta3, &block
160
+ require "google/cloud/dataflow/#{version.to_s.downcase}"
161
+
162
+ package_name = Google::Cloud::Dataflow
163
+ .constants
164
+ .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
165
+ .first
166
+ package_module = Google::Cloud::Dataflow.const_get package_name
167
+ package_module.const_get(:Metrics).const_get(:Client).new(&block)
168
+ end
169
+
170
+ ##
171
+ # Create a new client object for TemplatesService.
172
+ #
173
+ # By default, this returns an instance of
174
+ # [Google::Cloud::Dataflow::V1beta3::TemplatesService::Client](https://googleapis.dev/ruby/google-cloud-dataflow-v1beta3/latest/Google/Cloud/Dataflow/V1beta3/TemplatesService/Client.html)
175
+ # for version V1beta3 of the API.
176
+ # However, you can specify specify a different API version by passing it in the
177
+ # `version` parameter. If the TemplatesService service is
178
+ # supported by that API version, and the corresponding gem is available, the
179
+ # appropriate versioned client will be returned.
180
+ #
181
+ # ## About TemplatesService
182
+ #
183
+ # Provides a method to create Cloud Dataflow jobs from templates.
184
+ #
185
+ # @param version [::String, ::Symbol] The API version to connect to. Optional.
186
+ # Defaults to `:v1beta3`.
187
+ # @return [TemplatesService::Client] A client object for the specified version.
188
+ #
189
+ def self.templates_service version: :v1beta3, &block
190
+ require "google/cloud/dataflow/#{version.to_s.downcase}"
191
+
192
+ package_name = Google::Cloud::Dataflow
193
+ .constants
194
+ .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
195
+ .first
196
+ package_module = Google::Cloud::Dataflow.const_get package_name
197
+ package_module.const_get(:TemplatesService).const_get(:Client).new(&block)
198
+ end
199
+
200
+ ##
201
+ # Create a new client object for FlexTemplatesService.
202
+ #
203
+ # By default, this returns an instance of
204
+ # [Google::Cloud::Dataflow::V1beta3::FlexTemplatesService::Client](https://googleapis.dev/ruby/google-cloud-dataflow-v1beta3/latest/Google/Cloud/Dataflow/V1beta3/FlexTemplatesService/Client.html)
205
+ # for version V1beta3 of the API.
206
+ # However, you can specify specify a different API version by passing it in the
207
+ # `version` parameter. If the FlexTemplatesService service is
208
+ # supported by that API version, and the corresponding gem is available, the
209
+ # appropriate versioned client will be returned.
210
+ #
211
+ # ## About FlexTemplatesService
212
+ #
213
+ # Provides a service for Flex templates. This feature is not ready yet.
214
+ #
215
+ # @param version [::String, ::Symbol] The API version to connect to. Optional.
216
+ # Defaults to `:v1beta3`.
217
+ # @return [FlexTemplatesService::Client] A client object for the specified version.
218
+ #
219
+ def self.flex_templates_service version: :v1beta3, &block
220
+ require "google/cloud/dataflow/#{version.to_s.downcase}"
221
+
222
+ package_name = Google::Cloud::Dataflow
223
+ .constants
224
+ .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
225
+ .first
226
+ package_module = Google::Cloud::Dataflow.const_get package_name
227
+ package_module.const_get(:FlexTemplatesService).const_get(:Client).new(&block)
228
+ end
229
+
230
+ ##
231
+ # Configure the google-cloud-dataflow library.
232
+ #
233
+ # The following configuration parameters are supported:
234
+ #
235
+ # * `credentials` (*type:* `String, Hash, Google::Auth::Credentials`) -
236
+ # The path to the keyfile as a String, the contents of the keyfile as a
237
+ # Hash, or a Google::Auth::Credentials object.
238
+ # * `lib_name` (*type:* `String`) -
239
+ # The library name as recorded in instrumentation and logging.
240
+ # * `lib_version` (*type:* `String`) -
241
+ # The library version as recorded in instrumentation and logging.
242
+ # * `interceptors` (*type:* `Array<GRPC::ClientInterceptor>`) -
243
+ # An array of interceptors that are run before calls are executed.
244
+ # * `timeout` (*type:* `Numeric`) -
245
+ # Default timeout in seconds.
246
+ # * `metadata` (*type:* `Hash{Symbol=>String}`) -
247
+ # Additional gRPC headers to be sent with the call.
248
+ # * `retry_policy` (*type:* `Hash`) -
249
+ # The retry policy. The value is a hash with the following keys:
250
+ # * `:initial_delay` (*type:* `Numeric`) - The initial delay in seconds.
251
+ # * `:max_delay` (*type:* `Numeric`) - The max delay in seconds.
252
+ # * `:multiplier` (*type:* `Numeric`) - The incremental backoff multiplier.
253
+ # * `:retry_codes` (*type:* `Array<String>`) -
254
+ # The error codes that should trigger a retry.
255
+ #
256
+ # @return [::Google::Cloud::Config] The default configuration used by this library
257
+ #
258
+ def self.configure
259
+ yield ::Google::Cloud.configure.dataflow if block_given?
260
+
261
+ ::Google::Cloud.configure.dataflow
262
+ end
263
+ end
264
+ end
265
+ end
266
+
267
+ helper_path = ::File.join __dir__, "dataflow", "helpers.rb"
268
+ require "google/cloud/dataflow/helpers" if ::File.file? helper_path
@@ -0,0 +1,26 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright 2021 Google LLC
4
+ #
5
+ # Licensed under the Apache License, Version 2.0 (the "License");
6
+ # you may not use this file except in compliance with the License.
7
+ # You may obtain a copy of the License at
8
+ #
9
+ # https://www.apache.org/licenses/LICENSE-2.0
10
+ #
11
+ # Unless required by applicable law or agreed to in writing, software
12
+ # distributed under the License is distributed on an "AS IS" BASIS,
13
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14
+ # See the License for the specific language governing permissions and
15
+ # limitations under the License.
16
+
17
+ # Auto-generated by gapic-generator-ruby. DO NOT EDIT!
18
+
19
+
20
+ module Google
21
+ module Cloud
22
+ module Dataflow
23
+ VERSION = "0.1.0"
24
+ end
25
+ end
26
+ end
metadata ADDED
@@ -0,0 +1,190 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: google-cloud-dataflow
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Google LLC
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2021-06-29 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: google-cloud-core
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '1.5'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '1.5'
27
+ - !ruby/object:Gem::Dependency
28
+ name: google-cloud-dataflow-v1beta3
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '0.0'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '0.0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: google-style
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: 1.25.1
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: 1.25.1
55
+ - !ruby/object:Gem::Dependency
56
+ name: minitest
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '5.14'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '5.14'
69
+ - !ruby/object:Gem::Dependency
70
+ name: minitest-focus
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - "~>"
74
+ - !ruby/object:Gem::Version
75
+ version: '1.1'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - "~>"
81
+ - !ruby/object:Gem::Version
82
+ version: '1.1'
83
+ - !ruby/object:Gem::Dependency
84
+ name: minitest-rg
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - "~>"
88
+ - !ruby/object:Gem::Version
89
+ version: '5.2'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - "~>"
95
+ - !ruby/object:Gem::Version
96
+ version: '5.2'
97
+ - !ruby/object:Gem::Dependency
98
+ name: rake
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '12.0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '12.0'
111
+ - !ruby/object:Gem::Dependency
112
+ name: redcarpet
113
+ requirement: !ruby/object:Gem::Requirement
114
+ requirements:
115
+ - - "~>"
116
+ - !ruby/object:Gem::Version
117
+ version: '3.0'
118
+ type: :development
119
+ prerelease: false
120
+ version_requirements: !ruby/object:Gem::Requirement
121
+ requirements:
122
+ - - "~>"
123
+ - !ruby/object:Gem::Version
124
+ version: '3.0'
125
+ - !ruby/object:Gem::Dependency
126
+ name: simplecov
127
+ requirement: !ruby/object:Gem::Requirement
128
+ requirements:
129
+ - - "~>"
130
+ - !ruby/object:Gem::Version
131
+ version: '0.9'
132
+ type: :development
133
+ prerelease: false
134
+ version_requirements: !ruby/object:Gem::Requirement
135
+ requirements:
136
+ - - "~>"
137
+ - !ruby/object:Gem::Version
138
+ version: '0.9'
139
+ - !ruby/object:Gem::Dependency
140
+ name: yard
141
+ requirement: !ruby/object:Gem::Requirement
142
+ requirements:
143
+ - - "~>"
144
+ - !ruby/object:Gem::Version
145
+ version: '0.9'
146
+ type: :development
147
+ prerelease: false
148
+ version_requirements: !ruby/object:Gem::Requirement
149
+ requirements:
150
+ - - "~>"
151
+ - !ruby/object:Gem::Version
152
+ version: '0.9'
153
+ description: Dataflow is a managed service for executing a wide variety of data processing
154
+ patterns.
155
+ email: googleapis-packages@google.com
156
+ executables: []
157
+ extensions: []
158
+ extra_rdoc_files: []
159
+ files:
160
+ - ".yardopts"
161
+ - AUTHENTICATION.md
162
+ - LICENSE.md
163
+ - README.md
164
+ - lib/google-cloud-dataflow.rb
165
+ - lib/google/cloud/dataflow.rb
166
+ - lib/google/cloud/dataflow/version.rb
167
+ homepage: https://github.com/googleapis/google-cloud-ruby
168
+ licenses:
169
+ - Apache-2.0
170
+ metadata: {}
171
+ post_install_message:
172
+ rdoc_options: []
173
+ require_paths:
174
+ - lib
175
+ required_ruby_version: !ruby/object:Gem::Requirement
176
+ requirements:
177
+ - - ">="
178
+ - !ruby/object:Gem::Version
179
+ version: '2.5'
180
+ required_rubygems_version: !ruby/object:Gem::Requirement
181
+ requirements:
182
+ - - ">="
183
+ - !ruby/object:Gem::Version
184
+ version: '0'
185
+ requirements: []
186
+ rubygems_version: 3.2.17
187
+ signing_key:
188
+ specification_version: 4
189
+ summary: API Client library for the Dataflow API
190
+ test_files: []