google-cloud-pubsub 0.26.0 → 2.6.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.yardopts +12 -2
- data/AUTHENTICATION.md +178 -0
- data/CHANGELOG.md +659 -0
- data/CODE_OF_CONDUCT.md +40 -0
- data/CONTRIBUTING.md +187 -0
- data/EMULATOR.md +37 -0
- data/LICENSE +2 -2
- data/LOGGING.md +32 -0
- data/OVERVIEW.md +528 -0
- data/TROUBLESHOOTING.md +31 -0
- data/lib/google/cloud/pubsub/async_publisher/batch.rb +310 -0
- data/lib/google/cloud/pubsub/async_publisher.rb +402 -0
- data/lib/google/cloud/pubsub/batch_publisher.rb +100 -0
- data/lib/google/cloud/pubsub/convert.rb +91 -0
- data/lib/google/cloud/pubsub/credentials.rb +26 -10
- data/lib/google/cloud/pubsub/errors.rb +85 -0
- data/lib/google/cloud/pubsub/message.rb +80 -17
- data/lib/google/cloud/pubsub/policy.rb +17 -14
- data/lib/google/cloud/pubsub/project.rb +364 -250
- data/lib/google/cloud/pubsub/publish_result.rb +103 -0
- data/lib/google/cloud/pubsub/received_message.rb +162 -24
- data/lib/google/cloud/pubsub/retry_policy.rb +88 -0
- data/lib/google/cloud/pubsub/schema/list.rb +180 -0
- data/lib/google/cloud/pubsub/schema.rb +310 -0
- data/lib/google/cloud/pubsub/service.rb +281 -265
- data/lib/google/cloud/pubsub/snapshot/list.rb +21 -21
- data/lib/google/cloud/pubsub/snapshot.rb +55 -15
- data/lib/google/cloud/pubsub/subscriber/enumerator_queue.rb +54 -0
- data/lib/google/cloud/pubsub/subscriber/inventory.rb +173 -0
- data/lib/google/cloud/pubsub/subscriber/sequencer.rb +115 -0
- data/lib/google/cloud/pubsub/subscriber/stream.rb +400 -0
- data/lib/google/cloud/pubsub/subscriber/timed_unary_buffer.rb +230 -0
- data/lib/google/cloud/pubsub/subscriber.rb +417 -0
- data/lib/google/cloud/pubsub/subscription/list.rb +28 -28
- data/lib/google/cloud/pubsub/subscription/push_config.rb +268 -0
- data/lib/google/cloud/pubsub/subscription.rb +900 -172
- data/lib/google/cloud/pubsub/topic/list.rb +21 -21
- data/lib/google/cloud/pubsub/topic.rb +674 -95
- data/lib/google/cloud/pubsub/version.rb +6 -4
- data/lib/google/cloud/pubsub.rb +104 -439
- data/lib/google-cloud-pubsub.rb +60 -29
- metadata +88 -50
- data/README.md +0 -69
- data/lib/google/cloud/pubsub/topic/publisher.rb +0 -86
- data/lib/google/cloud/pubsub/v1/doc/google/protobuf/duration.rb +0 -77
- data/lib/google/cloud/pubsub/v1/doc/google/protobuf/field_mask.rb +0 -223
- data/lib/google/cloud/pubsub/v1/doc/google/protobuf/timestamp.rb +0 -81
- data/lib/google/cloud/pubsub/v1/doc/google/pubsub/v1/pubsub.rb +0 -503
- data/lib/google/cloud/pubsub/v1/publisher_client.rb +0 -605
- data/lib/google/cloud/pubsub/v1/publisher_client_config.json +0 -96
- data/lib/google/cloud/pubsub/v1/subscriber_client.rb +0 -1104
- data/lib/google/cloud/pubsub/v1/subscriber_client_config.json +0 -127
- data/lib/google/cloud/pubsub/v1.rb +0 -17
- data/lib/google/pubsub/v1/pubsub_pb.rb +0 -187
- data/lib/google/pubsub/v1/pubsub_services_pb.rb +0 -159
data/CODE_OF_CONDUCT.md
ADDED
@@ -0,0 +1,40 @@
|
|
1
|
+
# Contributor Code of Conduct
|
2
|
+
|
3
|
+
As contributors and maintainers of this project, and in the interest of
|
4
|
+
fostering an open and welcoming community, we pledge to respect all people who
|
5
|
+
contribute through reporting issues, posting feature requests, updating
|
6
|
+
documentation, submitting pull requests or patches, and other activities.
|
7
|
+
|
8
|
+
We are committed to making participation in this project a harassment-free
|
9
|
+
experience for everyone, regardless of level of experience, gender, gender
|
10
|
+
identity and expression, sexual orientation, disability, personal appearance,
|
11
|
+
body size, race, ethnicity, age, religion, or nationality.
|
12
|
+
|
13
|
+
Examples of unacceptable behavior by participants include:
|
14
|
+
|
15
|
+
* The use of sexualized language or imagery
|
16
|
+
* Personal attacks
|
17
|
+
* Trolling or insulting/derogatory comments
|
18
|
+
* Public or private harassment
|
19
|
+
* Publishing other's private information, such as physical or electronic
|
20
|
+
addresses, without explicit permission
|
21
|
+
* Other unethical or unprofessional conduct.
|
22
|
+
|
23
|
+
Project maintainers have the right and responsibility to remove, edit, or reject
|
24
|
+
comments, commits, code, wiki edits, issues, and other contributions that are
|
25
|
+
not aligned to this Code of Conduct. By adopting this Code of Conduct, project
|
26
|
+
maintainers commit themselves to fairly and consistently applying these
|
27
|
+
principles to every aspect of managing this project. Project maintainers who do
|
28
|
+
not follow or enforce the Code of Conduct may be permanently removed from the
|
29
|
+
project team.
|
30
|
+
|
31
|
+
This code of conduct applies both within project spaces and in public spaces
|
32
|
+
when an individual is representing the project or its community.
|
33
|
+
|
34
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
35
|
+
reported by opening an issue or contacting one or more of the project
|
36
|
+
maintainers.
|
37
|
+
|
38
|
+
This Code of Conduct is adapted from the [Contributor
|
39
|
+
Covenant](http://contributor-covenant.org), version 1.2.0, available at
|
40
|
+
[http://contributor-covenant.org/version/1/2/0/](http://contributor-covenant.org/version/1/2/0/)
|
data/CONTRIBUTING.md
ADDED
@@ -0,0 +1,187 @@
|
|
1
|
+
# Contributing to Google Cloud Pub/Sub
|
2
|
+
|
3
|
+
1. **Sign one of the contributor license agreements below.**
|
4
|
+
2. Fork the repo, develop and test your code changes.
|
5
|
+
3. Send a pull request.
|
6
|
+
|
7
|
+
## Contributor License Agreements
|
8
|
+
|
9
|
+
Before we can accept your pull requests you'll need to sign a Contributor
|
10
|
+
License Agreement (CLA):
|
11
|
+
|
12
|
+
- **If you are an individual writing original source code** and **you own the
|
13
|
+
intellectual property**, then you'll need to sign an [individual
|
14
|
+
CLA](https://developers.google.com/open-source/cla/individual).
|
15
|
+
- **If you work for a company that wants to allow you to contribute your work**,
|
16
|
+
then you'll need to sign a [corporate
|
17
|
+
CLA](https://developers.google.com/open-source/cla/corporate).
|
18
|
+
|
19
|
+
You can sign these electronically (just scroll to the bottom). After that, we'll
|
20
|
+
be able to accept your pull requests.
|
21
|
+
|
22
|
+
## Setup
|
23
|
+
|
24
|
+
In order to use the google-cloud-pubsub console and run the project's tests,
|
25
|
+
there is a small amount of setup:
|
26
|
+
|
27
|
+
1. Install Ruby. google-cloud-pubsub requires Ruby 2.5+. You may choose to
|
28
|
+
manage your Ruby and gem installations with [RVM](https://rvm.io/),
|
29
|
+
[rbenv](https://github.com/rbenv/rbenv), or
|
30
|
+
[chruby](https://github.com/postmodern/chruby).
|
31
|
+
|
32
|
+
2. Install [Bundler](http://bundler.io/).
|
33
|
+
|
34
|
+
```sh
|
35
|
+
$ gem install bundler
|
36
|
+
```
|
37
|
+
|
38
|
+
3. Install the top-level project dependencies.
|
39
|
+
|
40
|
+
```sh
|
41
|
+
$ bundle install
|
42
|
+
```
|
43
|
+
|
44
|
+
4. Install the Pub/Sub dependencies.
|
45
|
+
|
46
|
+
```sh
|
47
|
+
$ cd google-cloud-pubsub/
|
48
|
+
$ bundle install
|
49
|
+
```
|
50
|
+
|
51
|
+
## Console
|
52
|
+
|
53
|
+
In order to run code interactively, you can automatically load
|
54
|
+
google-cloud-pubsub and its dependencies in IRB. This requires that your
|
55
|
+
developer environment has already been configured by following the steps
|
56
|
+
described in the {file:AUTHENTICATION.md Authentication Guide}. An IRB console
|
57
|
+
can be created with:
|
58
|
+
|
59
|
+
```sh
|
60
|
+
$ cd google-cloud-pubsub/
|
61
|
+
$ bundle exec rake console
|
62
|
+
```
|
63
|
+
|
64
|
+
## Pub/Sub Tests
|
65
|
+
|
66
|
+
Tests are very important part of google-cloud-pubsub. All contributions
|
67
|
+
should include tests that ensure the contributed code behaves as expected.
|
68
|
+
|
69
|
+
To run the unit tests, documentation tests, and code style checks together for a
|
70
|
+
package:
|
71
|
+
|
72
|
+
``` sh
|
73
|
+
$ cd google-cloud-pubsub/
|
74
|
+
$ bundle exec rake ci
|
75
|
+
```
|
76
|
+
|
77
|
+
To run the command above, plus all acceptance tests, use `rake ci:acceptance` or
|
78
|
+
its handy alias, `rake ci:a`.
|
79
|
+
|
80
|
+
### Pub/Sub Unit Tests
|
81
|
+
|
82
|
+
|
83
|
+
The project uses the [minitest](https://github.com/seattlerb/minitest) library,
|
84
|
+
including [specs](https://github.com/seattlerb/minitest#specs),
|
85
|
+
[mocks](https://github.com/seattlerb/minitest#mocks) and
|
86
|
+
[minitest-autotest](https://github.com/seattlerb/minitest-autotest).
|
87
|
+
|
88
|
+
To run the Pub/Sub unit tests:
|
89
|
+
|
90
|
+
``` sh
|
91
|
+
$ cd google-cloud-pubsub/
|
92
|
+
$ bundle exec rake test
|
93
|
+
```
|
94
|
+
|
95
|
+
### Pub/Sub Documentation Tests
|
96
|
+
|
97
|
+
The project tests the code examples in the gem's
|
98
|
+
[YARD](https://github.com/lsegal/yard)-based documentation.
|
99
|
+
|
100
|
+
The example testing functions in a way that is very similar to unit testing, and
|
101
|
+
in fact the library providing it,
|
102
|
+
[yard-doctest](https://github.com/p0deje/yard-doctest), is based on the
|
103
|
+
project's unit test library, [minitest](https://github.com/seattlerb/minitest).
|
104
|
+
|
105
|
+
To run the Pub/Sub documentation tests:
|
106
|
+
|
107
|
+
``` sh
|
108
|
+
$ cd google-cloud-pubsub/
|
109
|
+
$ bundle exec rake doctest
|
110
|
+
```
|
111
|
+
|
112
|
+
If you add, remove or modify documentation examples when working on a pull
|
113
|
+
request, you may need to update the setup for the tests. The stubs and mocks
|
114
|
+
required to run the tests are located in `support/doctest_helper.rb`. Please
|
115
|
+
note that much of the setup is matched by the title of the
|
116
|
+
[`@example`](http://www.rubydoc.info/gems/yard/file/docs/Tags.md#example) tag.
|
117
|
+
If you alter an example's title, you may encounter breaking tests.
|
118
|
+
|
119
|
+
### Pub/Sub Acceptance Tests
|
120
|
+
|
121
|
+
The Pub/Sub acceptance tests interact with the live service API. Follow the
|
122
|
+
instructions in the {file:AUTHENTICATION.md Authentication Guide} for enabling
|
123
|
+
the Pub/Sub API. Occasionally, some API features may not yet be generally
|
124
|
+
available, making it difficult for some contributors to successfully run the
|
125
|
+
entire acceptance test suite. However, please ensure that you do successfully
|
126
|
+
run acceptance tests for any code areas covered by your pull request.
|
127
|
+
|
128
|
+
To run the acceptance tests, first create and configure a project in the Google
|
129
|
+
Developers Console, as described in the {file:AUTHENTICATION.md Authentication Guide}. Be sure to download the JSON KEY file. Make note of the PROJECT_ID and
|
130
|
+
the KEYFILE location on your system.
|
131
|
+
|
132
|
+
Before you can run the Pub/Sub acceptance tests, you must first create indexes
|
133
|
+
used in the tests.
|
134
|
+
|
135
|
+
#### Running the Pub/Sub acceptance tests
|
136
|
+
|
137
|
+
To run the Pub/Sub acceptance tests:
|
138
|
+
|
139
|
+
``` sh
|
140
|
+
$ cd google-cloud-pubsub/
|
141
|
+
$ bundle exec rake acceptance[\\{my-project-id},\\{/path/to/keyfile.json}]
|
142
|
+
```
|
143
|
+
|
144
|
+
Or, if you prefer you can store the values in the `GCLOUD_TEST_PROJECT` and
|
145
|
+
`GCLOUD_TEST_KEYFILE` environment variables:
|
146
|
+
|
147
|
+
``` sh
|
148
|
+
$ cd google-cloud-pubsub/
|
149
|
+
$ export GCLOUD_TEST_PROJECT=\\{my-project-id}
|
150
|
+
$ export GCLOUD_TEST_KEYFILE=\\{/path/to/keyfile.json}
|
151
|
+
$ bundle exec rake acceptance
|
152
|
+
```
|
153
|
+
|
154
|
+
If you want to use a different project and credentials for acceptance tests, you
|
155
|
+
can use the more specific `PUBSUB_TEST_PROJECT` and `PUBSUB_TEST_KEYFILE`
|
156
|
+
environment variables:
|
157
|
+
|
158
|
+
``` sh
|
159
|
+
$ cd google-cloud-pubsub/
|
160
|
+
$ export PUBSUB_TEST_PROJECT=\\{my-project-id}
|
161
|
+
$ export PUBSUB_TEST_KEYFILE=\\{/path/to/keyfile.json}
|
162
|
+
$ bundle exec rake acceptance
|
163
|
+
```
|
164
|
+
|
165
|
+
## Coding Style
|
166
|
+
|
167
|
+
Please follow the established coding style in the library. The style is is
|
168
|
+
largely based on [The Ruby Style
|
169
|
+
Guide](https://github.com/bbatsov/ruby-style-guide) with a few exceptions based
|
170
|
+
on seattle-style:
|
171
|
+
|
172
|
+
* Avoid parenthesis when possible, including in method definitions.
|
173
|
+
* Always use double quotes strings. ([Option
|
174
|
+
B](https://github.com/bbatsov/ruby-style-guide#strings))
|
175
|
+
|
176
|
+
You can check your code against these rules by running Rubocop like so:
|
177
|
+
|
178
|
+
```sh
|
179
|
+
$ cd google-cloud-pubsub/
|
180
|
+
$ bundle exec rake rubocop
|
181
|
+
```
|
182
|
+
|
183
|
+
## Code of Conduct
|
184
|
+
|
185
|
+
Please note that this project is released with a Contributor Code of Conduct. By
|
186
|
+
participating in this project you agree to abide by its terms. See
|
187
|
+
{file:CODE_OF_CONDUCT.md Code of Conduct} for more information.
|
data/EMULATOR.md
ADDED
@@ -0,0 +1,37 @@
|
|
1
|
+
# Google Cloud Pub/Sub Emulator
|
2
|
+
|
3
|
+
To develop and test your application locally, you can use the [Google Cloud
|
4
|
+
Pub/Sub Emulator](https://cloud.google.com/pubsub/emulator), which provides
|
5
|
+
[local emulation](https://cloud.google.com/sdk/gcloud/reference/beta/emulators/)
|
6
|
+
of the production Google Cloud Pub/Sub environment. You can start the Google
|
7
|
+
Cloud Pub/Sub emulator using the `gcloud` command-line tool.
|
8
|
+
|
9
|
+
To configure your ruby code to use the emulator, set the `PUBSUB_EMULATOR_HOST`
|
10
|
+
environment variable to the host and port where the emulator is running. The
|
11
|
+
value can be set as an environment variable in the shell running the ruby code,
|
12
|
+
or can be set directly in the ruby code as shown below.
|
13
|
+
|
14
|
+
```ruby
|
15
|
+
require "google/cloud/pubsub"
|
16
|
+
|
17
|
+
# Make Pub/Sub use the emulator
|
18
|
+
ENV["PUBSUB_EMULATOR_HOST"] = "localhost:8918"
|
19
|
+
|
20
|
+
pubsub = Google::Cloud::PubSub.new project_id:"emulator-project-id"
|
21
|
+
|
22
|
+
# Get a topic in the current project
|
23
|
+
my_topic = pubsub.new_topic "my-topic"
|
24
|
+
my_topic.name #=> "projects/emulator-project-id/topics/my-topic"
|
25
|
+
```
|
26
|
+
|
27
|
+
Or by providing the `emulator_host` argument:
|
28
|
+
|
29
|
+
```ruby
|
30
|
+
require "google/cloud/pubsub"
|
31
|
+
|
32
|
+
pubsub = Google::Cloud::PubSub.new emulator_host: "localhost:8918"
|
33
|
+
|
34
|
+
# Get a topic in the current project
|
35
|
+
my_topic = pubsub.new_topic "my-topic"
|
36
|
+
my_topic.name #=> "projects/emulator-project-id/topics/my-topic"
|
37
|
+
```
|
data/LICENSE
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
Apache License
|
2
2
|
Version 2.0, January 2004
|
3
|
-
|
3
|
+
https://www.apache.org/licenses/
|
4
4
|
|
5
5
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
6
6
|
|
@@ -192,7 +192,7 @@
|
|
192
192
|
you may not use this file except in compliance with the License.
|
193
193
|
You may obtain a copy of the License at
|
194
194
|
|
195
|
-
|
195
|
+
https://www.apache.org/licenses/LICENSE-2.0
|
196
196
|
|
197
197
|
Unless required by applicable law or agreed to in writing, software
|
198
198
|
distributed under the License is distributed on an "AS IS" BASIS,
|
data/LOGGING.md
ADDED
@@ -0,0 +1,32 @@
|
|
1
|
+
# Enabling gRPC Logging
|
2
|
+
|
3
|
+
To enable logging for this library, set the logger for the underlying
|
4
|
+
[gRPC](https://github.com/grpc/grpc/tree/master/src/ruby) library. The logger
|
5
|
+
that you set may be a Ruby stdlib
|
6
|
+
[`Logger`](https://ruby-doc.org/stdlib/libdoc/logger/rdoc/Logger.html) as
|
7
|
+
shown below, or a
|
8
|
+
[`Google::Cloud::Logging::Logger`](https://googleapis.dev/ruby/google-cloud-logging/latest)
|
9
|
+
that will write logs to [Stackdriver
|
10
|
+
Logging](https://cloud.google.com/logging/). See
|
11
|
+
[grpc/logconfig.rb](https://github.com/grpc/grpc/blob/master/src/ruby/lib/grpc/logconfig.rb)
|
12
|
+
and the gRPC
|
13
|
+
[spec_helper.rb](https://github.com/grpc/grpc/blob/master/src/ruby/spec/spec_helper.rb)
|
14
|
+
for additional information.
|
15
|
+
|
16
|
+
Configuring a Ruby stdlib logger:
|
17
|
+
|
18
|
+
```ruby
|
19
|
+
require "logger"
|
20
|
+
|
21
|
+
module MyLogger
|
22
|
+
LOGGER = Logger.new $stderr, level: Logger::WARN
|
23
|
+
def logger
|
24
|
+
LOGGER
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
# Define a gRPC module-level logger method before grpc/logconfig.rb loads.
|
29
|
+
module GRPC
|
30
|
+
extend MyLogger
|
31
|
+
end
|
32
|
+
```
|
data/OVERVIEW.md
ADDED
@@ -0,0 +1,528 @@
|
|
1
|
+
# Google Cloud Pub/Sub
|
2
|
+
|
3
|
+
Google Cloud Pub/Sub is designed to provide reliable, many-to-many, asynchronous
|
4
|
+
messaging between applications. Publisher applications can send messages to a
|
5
|
+
"topic" and other applications can subscribe to that topic to receive the
|
6
|
+
messages. By decoupling senders and receivers, Google Cloud Pub/Sub allows
|
7
|
+
developers to communicate between independently written applications.
|
8
|
+
|
9
|
+
The goal of google-cloud is to provide an API that is comfortable to Rubyists.
|
10
|
+
Your authentication credentials are detected automatically in Google Cloud
|
11
|
+
Platform (GCP), including Google Compute Engine (GCE), Google Kubernetes Engine
|
12
|
+
(GKE), Google App Engine (GAE), Google Cloud Functions (GCF) and Cloud Run. In
|
13
|
+
other environments you can configure authentication easily, either directly in
|
14
|
+
your code or via environment variables. Read more about the options for
|
15
|
+
connecting in the {file:AUTHENTICATION.md Authentication Guide}.
|
16
|
+
|
17
|
+
```ruby
|
18
|
+
require "google/cloud/pubsub"
|
19
|
+
|
20
|
+
pubsub = Google::Cloud::PubSub.new
|
21
|
+
|
22
|
+
topic = pubsub.topic "my-topic"
|
23
|
+
topic.publish "task completed"
|
24
|
+
```
|
25
|
+
|
26
|
+
To learn more about Pub/Sub, read the [Google Cloud Pub/Sub Overview
|
27
|
+
](https://cloud.google.com/pubsub/overview).
|
28
|
+
|
29
|
+
## Retrieving Topics
|
30
|
+
|
31
|
+
A Topic is a named resource to which messages are sent by publishers. A Topic is
|
32
|
+
found by its name. (See {Google::Cloud::PubSub::Project#topic Project#topic})
|
33
|
+
|
34
|
+
```ruby
|
35
|
+
require "google/cloud/pubsub"
|
36
|
+
|
37
|
+
pubsub = Google::Cloud::PubSub.new
|
38
|
+
topic = pubsub.topic "my-topic"
|
39
|
+
```
|
40
|
+
|
41
|
+
## Creating a Topic
|
42
|
+
|
43
|
+
A Topic is created from a Project. (See
|
44
|
+
{Google::Cloud::PubSub::Project#create_topic Project#create_topic})
|
45
|
+
|
46
|
+
```ruby
|
47
|
+
require "google/cloud/pubsub"
|
48
|
+
|
49
|
+
pubsub = Google::Cloud::PubSub.new
|
50
|
+
topic = pubsub.create_topic "my-topic"
|
51
|
+
```
|
52
|
+
|
53
|
+
## Retrieving Subscriptions
|
54
|
+
|
55
|
+
A Subscription is a named resource representing the stream of messages from a
|
56
|
+
single, specific Topic, to be delivered to the subscribing application. A
|
57
|
+
Subscription is found by its name. (See
|
58
|
+
{Google::Cloud::PubSub::Topic#subscription Topic#subscription})
|
59
|
+
|
60
|
+
```ruby
|
61
|
+
require "google/cloud/pubsub"
|
62
|
+
|
63
|
+
pubsub = Google::Cloud::PubSub.new
|
64
|
+
|
65
|
+
topic = pubsub.topic "my-topic"
|
66
|
+
subscription = topic.subscription "my-topic-subscription"
|
67
|
+
puts subscription.name
|
68
|
+
```
|
69
|
+
|
70
|
+
## Creating a Subscription
|
71
|
+
|
72
|
+
A Subscription is created from a Topic. (See
|
73
|
+
{Google::Cloud::PubSub::Topic#subscribe Topic#subscribe})
|
74
|
+
|
75
|
+
```ruby
|
76
|
+
require "google/cloud/pubsub"
|
77
|
+
|
78
|
+
pubsub = Google::Cloud::PubSub.new
|
79
|
+
|
80
|
+
topic = pubsub.topic "my-topic"
|
81
|
+
sub = topic.subscribe "my-topic-sub"
|
82
|
+
puts sub.name # => "my-topic-sub"
|
83
|
+
```
|
84
|
+
|
85
|
+
The subscription can be created that specifies the number of seconds to wait to
|
86
|
+
be acknowledged as well as an endpoint URL to push the messages to:
|
87
|
+
|
88
|
+
```ruby
|
89
|
+
require "google/cloud/pubsub"
|
90
|
+
|
91
|
+
pubsub = Google::Cloud::PubSub.new
|
92
|
+
|
93
|
+
topic = pubsub.topic "my-topic"
|
94
|
+
sub = topic.subscribe "my-topic-sub",
|
95
|
+
deadline: 120,
|
96
|
+
endpoint: "https://example.com/push"
|
97
|
+
```
|
98
|
+
|
99
|
+
## Publishing Messages
|
100
|
+
|
101
|
+
Messages are published to a topic. Any message published to a topic without a
|
102
|
+
subscription will be lost. Ensure the topic has a subscription before
|
103
|
+
publishing. (See {Google::Cloud::PubSub::Topic#publish Topic#publish})
|
104
|
+
|
105
|
+
```ruby
|
106
|
+
require "google/cloud/pubsub"
|
107
|
+
|
108
|
+
pubsub = Google::Cloud::PubSub.new
|
109
|
+
|
110
|
+
topic = pubsub.topic "my-topic"
|
111
|
+
msg = topic.publish "task completed"
|
112
|
+
```
|
113
|
+
|
114
|
+
Messages can also be published with attributes:
|
115
|
+
|
116
|
+
```ruby
|
117
|
+
require "google/cloud/pubsub"
|
118
|
+
|
119
|
+
pubsub = Google::Cloud::PubSub.new
|
120
|
+
|
121
|
+
topic = pubsub.topic "my-topic"
|
122
|
+
msg = topic.publish "task completed",
|
123
|
+
foo: :bar,
|
124
|
+
this: :that
|
125
|
+
```
|
126
|
+
|
127
|
+
Messages can also be published in batches asynchronously using `publish_async`.
|
128
|
+
(See {Google::Cloud::PubSub::Topic#publish_async Topic#publish_async} and
|
129
|
+
{Google::Cloud::PubSub::AsyncPublisher AsyncPublisher})
|
130
|
+
|
131
|
+
```ruby
|
132
|
+
require "google/cloud/pubsub"
|
133
|
+
|
134
|
+
pubsub = Google::Cloud::PubSub.new
|
135
|
+
|
136
|
+
topic = pubsub.topic "my-topic"
|
137
|
+
topic.publish_async "task completed" do |result|
|
138
|
+
if result.succeeded?
|
139
|
+
log_publish_success result.data
|
140
|
+
else
|
141
|
+
log_publish_failure result.data, result.error
|
142
|
+
end
|
143
|
+
end
|
144
|
+
|
145
|
+
topic.async_publisher.stop!
|
146
|
+
```
|
147
|
+
|
148
|
+
Or multiple messages can be published in batches at the same time by passing a
|
149
|
+
block to `publish`. (See {Google::Cloud::PubSub::BatchPublisher BatchPublisher})
|
150
|
+
|
151
|
+
```ruby
|
152
|
+
require "google/cloud/pubsub"
|
153
|
+
|
154
|
+
pubsub = Google::Cloud::PubSub.new
|
155
|
+
|
156
|
+
topic = pubsub.topic "my-topic"
|
157
|
+
msgs = topic.publish do |batch|
|
158
|
+
batch.publish "task 1 completed", foo: :bar
|
159
|
+
batch.publish "task 2 completed", foo: :baz
|
160
|
+
batch.publish "task 3 completed", foo: :bif
|
161
|
+
end
|
162
|
+
```
|
163
|
+
|
164
|
+
## Receiving Messages
|
165
|
+
|
166
|
+
Messages can be streamed from a subscription with a subscriber object that is
|
167
|
+
created using `listen`. (See {Google::Cloud::PubSub::Subscription#listen
|
168
|
+
Subscription#listen} and {Google::Cloud::PubSub::Subscriber Subscriber})
|
169
|
+
|
170
|
+
```ruby
|
171
|
+
require "google/cloud/pubsub"
|
172
|
+
|
173
|
+
pubsub = Google::Cloud::PubSub.new
|
174
|
+
|
175
|
+
sub = pubsub.subscription "my-topic-sub"
|
176
|
+
|
177
|
+
# Create a subscriber to listen for available messages.
|
178
|
+
# By default, this block will be called on 8 concurrent threads
|
179
|
+
# but this can be tuned with the `threads` option.
|
180
|
+
# The `streams` and `inventory` parameters allow further tuning.
|
181
|
+
subscriber = sub.listen threads: { callback: 16 } do |received_message|
|
182
|
+
# process message
|
183
|
+
puts "Data: #{received_message.message.data}, published at #{received_message.message.published_at}"
|
184
|
+
received_message.acknowledge!
|
185
|
+
end
|
186
|
+
|
187
|
+
# Handle exceptions from listener
|
188
|
+
subscriber.on_error do |exception|
|
189
|
+
puts "Exception: #{exception.class} #{exception.message}"
|
190
|
+
end
|
191
|
+
|
192
|
+
# Gracefully shut down the subscriber on program exit, blocking until
|
193
|
+
# all received messages have been processed or 10 seconds have passed
|
194
|
+
at_exit do
|
195
|
+
subscriber.stop!(10)
|
196
|
+
end
|
197
|
+
|
198
|
+
# Start background threads that will call the block passed to listen.
|
199
|
+
subscriber.start
|
200
|
+
|
201
|
+
# Block, letting processing threads continue in the background
|
202
|
+
sleep
|
203
|
+
```
|
204
|
+
|
205
|
+
Messages also can be pulled directly in a one-time operation. (See
|
206
|
+
{Google::Cloud::PubSub::Subscription#pull Subscription#pull})
|
207
|
+
|
208
|
+
The `immediate: false` option is recommended to avoid adverse impacts on the
|
209
|
+
performance of pull operations.
|
210
|
+
|
211
|
+
```ruby
|
212
|
+
require "google/cloud/pubsub"
|
213
|
+
|
214
|
+
pubsub = Google::Cloud::PubSub.new
|
215
|
+
|
216
|
+
sub = pubsub.subscription "my-topic-sub"
|
217
|
+
received_messages = sub.pull immediate: false
|
218
|
+
```
|
219
|
+
|
220
|
+
A maximum number of messages to pull can be specified:
|
221
|
+
|
222
|
+
```ruby
|
223
|
+
require "google/cloud/pubsub"
|
224
|
+
|
225
|
+
pubsub = Google::Cloud::PubSub.new
|
226
|
+
|
227
|
+
sub = pubsub.subscription "my-topic-sub"
|
228
|
+
received_messages = sub.pull immediate: false, max: 10
|
229
|
+
```
|
230
|
+
|
231
|
+
## Acknowledging a Message
|
232
|
+
|
233
|
+
Messages that are received can be acknowledged in Pub/Sub, marking the message
|
234
|
+
to be removed so it cannot be pulled again.
|
235
|
+
|
236
|
+
A Message that can be acknowledged is called a ReceivedMessage. ReceivedMessages
|
237
|
+
can be acknowledged one at a time: (See
|
238
|
+
{Google::Cloud::PubSub::ReceivedMessage#acknowledge!
|
239
|
+
ReceivedMessage#acknowledge!})
|
240
|
+
|
241
|
+
```ruby
|
242
|
+
require "google/cloud/pubsub"
|
243
|
+
|
244
|
+
pubsub = Google::Cloud::PubSub.new
|
245
|
+
|
246
|
+
sub = pubsub.subscription "my-topic-sub"
|
247
|
+
|
248
|
+
subscriber = sub.listen do |received_message|
|
249
|
+
# process message
|
250
|
+
received_message.acknowledge!
|
251
|
+
end
|
252
|
+
|
253
|
+
# Start background threads that will call the block passed to listen.
|
254
|
+
subscriber.start
|
255
|
+
|
256
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
257
|
+
subscriber.stop!
|
258
|
+
```
|
259
|
+
|
260
|
+
Or, multiple messages can be acknowledged in a single API call: (See
|
261
|
+
{Google::Cloud::PubSub::Subscription#acknowledge Subscription#acknowledge})
|
262
|
+
|
263
|
+
```ruby
|
264
|
+
require "google/cloud/pubsub"
|
265
|
+
|
266
|
+
pubsub = Google::Cloud::PubSub.new
|
267
|
+
|
268
|
+
sub = pubsub.subscription "my-topic-sub"
|
269
|
+
received_messages = sub.pull immediate: false
|
270
|
+
sub.acknowledge received_messages
|
271
|
+
```
|
272
|
+
|
273
|
+
## Modifying a Deadline
|
274
|
+
|
275
|
+
A message must be acknowledged after it is pulled, or Pub/Sub will mark the
|
276
|
+
message for redelivery. The message acknowledgement deadline can delayed if more
|
277
|
+
time is needed. This will allow more time to process the message before the
|
278
|
+
message is marked for redelivery. (See
|
279
|
+
{Google::Cloud::PubSub::ReceivedMessage#modify_ack_deadline!
|
280
|
+
ReceivedMessage#modify_ack_deadline!})
|
281
|
+
|
282
|
+
```ruby
|
283
|
+
require "google/cloud/pubsub"
|
284
|
+
|
285
|
+
pubsub = Google::Cloud::PubSub.new
|
286
|
+
|
287
|
+
sub = pubsub.subscription "my-topic-sub"
|
288
|
+
subscriber = sub.listen do |received_message|
|
289
|
+
puts received_message.message.data
|
290
|
+
|
291
|
+
# Delay for 2 minutes
|
292
|
+
received_message.modify_ack_deadline! 120
|
293
|
+
end
|
294
|
+
|
295
|
+
# Start background threads that will call the block passed to listen.
|
296
|
+
subscriber.start
|
297
|
+
|
298
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
299
|
+
subscriber.stop!
|
300
|
+
```
|
301
|
+
|
302
|
+
The message can also be made available for immediate redelivery:
|
303
|
+
|
304
|
+
```ruby
|
305
|
+
require "google/cloud/pubsub"
|
306
|
+
|
307
|
+
pubsub = Google::Cloud::PubSub.new
|
308
|
+
|
309
|
+
sub = pubsub.subscription "my-topic-sub"
|
310
|
+
subscriber = sub.listen do |received_message|
|
311
|
+
puts received_message.message.data
|
312
|
+
|
313
|
+
# Mark for redelivery
|
314
|
+
received_message.reject!
|
315
|
+
end
|
316
|
+
|
317
|
+
# Start background threads that will call the block passed to listen.
|
318
|
+
subscriber.start
|
319
|
+
|
320
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
321
|
+
subscriber.stop!
|
322
|
+
```
|
323
|
+
|
324
|
+
Multiple messages can be delayed or made available for immediate redelivery:
|
325
|
+
(See {Google::Cloud::PubSub::Subscription#modify_ack_deadline
|
326
|
+
Subscription#modify_ack_deadline})
|
327
|
+
|
328
|
+
```ruby
|
329
|
+
require "google/cloud/pubsub"
|
330
|
+
|
331
|
+
pubsub = Google::Cloud::PubSub.new
|
332
|
+
|
333
|
+
sub = pubsub.subscription "my-topic-sub"
|
334
|
+
received_messages = sub.pull immediate: false
|
335
|
+
sub.modify_ack_deadline 120, received_messages
|
336
|
+
```
|
337
|
+
|
338
|
+
## Using Ordering Keys
|
339
|
+
|
340
|
+
Google Cloud Pub/Sub ordering keys provide the ability to ensure related
|
341
|
+
messages are sent to subscribers in the order in which they were published.
|
342
|
+
Messages can be tagged with an ordering key, a string that identifies related
|
343
|
+
messages for which publish order should be respected. The service guarantees
|
344
|
+
that, for a given ordering key and publisher, messages are sent to subscribers
|
345
|
+
in the order in which they were published. Ordering does not require sacrificing
|
346
|
+
high throughput or scalability, as the service automatically distributes
|
347
|
+
messages for different ordering keys across subscribers.
|
348
|
+
|
349
|
+
Note: At the time of this release, ordering keys are not yet publicly enabled
|
350
|
+
and requires special project enablements.
|
351
|
+
|
352
|
+
### Publishing Ordered Messages
|
353
|
+
|
354
|
+
To use ordering keys when publishing messages, a call to
|
355
|
+
{Google::Cloud::PubSub::Topic#enable_message_ordering!
|
356
|
+
Topic#enable_message_ordering!} must be made and the `ordering_key` argument
|
357
|
+
must be provided when calling {Google::Cloud::PubSub::Topic#publish_async
|
358
|
+
Topic#publish_async}.
|
359
|
+
|
360
|
+
```ruby
|
361
|
+
require "google/cloud/pubsub"
|
362
|
+
|
363
|
+
pubsub = Google::Cloud::PubSub.new
|
364
|
+
|
365
|
+
topic = pubsub.topic "my-ordered-topic"
|
366
|
+
|
367
|
+
# Ensure that message ordering is enabled.
|
368
|
+
topic.enable_message_ordering!
|
369
|
+
|
370
|
+
# Publish an ordered message with an ordering key.
|
371
|
+
topic.publish_async "task completed",
|
372
|
+
ordering_key: "task-key"
|
373
|
+
|
374
|
+
# Shut down the publisher when ready to stop publishing messages.
|
375
|
+
topic.async_publisher.stop!
|
376
|
+
```
|
377
|
+
|
378
|
+
### Handling errors with Ordered Keys
|
379
|
+
|
380
|
+
Ordered messages that fail to publish to the Pub/Sub API due to error will put
|
381
|
+
the `ordering_key` in a failed state, and future calls to
|
382
|
+
{Google::Cloud::PubSub::Topic#publish_async Topic#publish_async} with the
|
383
|
+
`ordering_key` will raise {Google::Cloud::PubSub::OrderingKeyError
|
384
|
+
OrderingKeyError}. To allow future messages with the `ordering_key` to be
|
385
|
+
published, the `ordering_key` must be passed to
|
386
|
+
{Google::Cloud::PubSub::Topic#resume_publish Topic#resume_publish}.
|
387
|
+
|
388
|
+
### Receiving Ordered Messages
|
389
|
+
|
390
|
+
To use ordering keys when subscribing to messages, the subscription must be
|
391
|
+
created with message ordering enabled (See
|
392
|
+
{Google::Cloud::PubSub::Topic#subscribe Topic#subscribe} and
|
393
|
+
{Google::Cloud::PubSub::Subscription#message_ordering?
|
394
|
+
Subscription#message_ordering?}) before calling
|
395
|
+
{Google::Cloud::PubSub::Subscription#listen Subscription#listen}. When enabled,
|
396
|
+
the subscriber will deliver messages with the same `ordering_key` in the order
|
397
|
+
they were published.
|
398
|
+
|
399
|
+
```ruby
|
400
|
+
require "google/cloud/pubsub"
|
401
|
+
|
402
|
+
pubsub = Google::Cloud::PubSub.new
|
403
|
+
|
404
|
+
sub = pubsub.subscription "my-ordered-topic-sub"
|
405
|
+
sub.message_ordering? #=> true
|
406
|
+
|
407
|
+
subscriber = sub.listen do |received_message|
|
408
|
+
# Messsages with the same ordering_key are received
|
409
|
+
# in the order in which they were published.
|
410
|
+
received_message.acknowledge!
|
411
|
+
end
|
412
|
+
|
413
|
+
# Start background threads that will call block passed to listen.
|
414
|
+
subscriber.start
|
415
|
+
|
416
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
417
|
+
subscriber.stop!
|
418
|
+
```
|
419
|
+
|
420
|
+
## Minimizing API calls before receiving and acknowledging messages
|
421
|
+
|
422
|
+
A subscription object can be created without making any API calls by providing
|
423
|
+
the `skip_lookup` argument to {Google::Cloud::PubSub::Project#subscription
|
424
|
+
Project#subscription} or {Google::Cloud::PubSub::Topic#subscription
|
425
|
+
Topic#subscription}. A subscriber object can also be created without an API call
|
426
|
+
by providing the `deadline` optional argument to
|
427
|
+
{Google::Cloud::PubSub::Subscription#listen Subscription#listen}:
|
428
|
+
|
429
|
+
```ruby
|
430
|
+
require "google/cloud/pubsub"
|
431
|
+
|
432
|
+
pubsub = Google::Cloud::PubSub.new
|
433
|
+
|
434
|
+
# No API call is made to retrieve the subscription resource.
|
435
|
+
sub = pubsub.subscription "my-topic-sub", skip_lookup: true
|
436
|
+
|
437
|
+
# No API call is made to retrieve the subscription deadline.
|
438
|
+
subscriber = sub.listen deadline: 60 do |received_message|
|
439
|
+
# process message
|
440
|
+
received_message.acknowledge!
|
441
|
+
end
|
442
|
+
|
443
|
+
# Start background threads that will call block passed to listen.
|
444
|
+
subscriber.start
|
445
|
+
|
446
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
447
|
+
subscriber.stop!
|
448
|
+
```
|
449
|
+
|
450
|
+
Skipping API calls may be used to avoid `Google::Cloud::PermissionDeniedError`
|
451
|
+
if your account has limited access to the Pub/Sub API. In particular, the role
|
452
|
+
`roles/pubsub.subscriber` does not have the permission
|
453
|
+
`pubsub.subscriptions.get`, which is required to retrieve a subscription
|
454
|
+
resource. See [Access Control -
|
455
|
+
Roles](https://cloud.google.com/pubsub/docs/access-control#tbl_roles) for the
|
456
|
+
complete list of Pub/Sub roles and permissions.
|
457
|
+
|
458
|
+
## Creating a snapshot and using seek
|
459
|
+
|
460
|
+
You can create a snapshot to retain the existing backlog on a subscription. The
|
461
|
+
snapshot will hold the messages in the subscription's backlog that are
|
462
|
+
unacknowledged upon the successful completion of the `create_snapshot`
|
463
|
+
operation.
|
464
|
+
|
465
|
+
Later, you can use `seek` to reset the subscription's backlog to the snapshot.
|
466
|
+
|
467
|
+
(See {Google::Cloud::PubSub::Subscription#create_snapshot
|
468
|
+
Subscription#create_snapshot} and {Google::Cloud::PubSub::Subscription#seek
|
469
|
+
Subscription#seek})
|
470
|
+
|
471
|
+
```ruby
|
472
|
+
require "google/cloud/pubsub"
|
473
|
+
|
474
|
+
pubsub = Google::Cloud::PubSub.new
|
475
|
+
|
476
|
+
sub = pubsub.subscription "my-topic-sub"
|
477
|
+
|
478
|
+
snapshot = sub.create_snapshot
|
479
|
+
|
480
|
+
received_messages = sub.pull immediate: false
|
481
|
+
sub.acknowledge received_messages
|
482
|
+
|
483
|
+
sub.seek snapshot
|
484
|
+
```
|
485
|
+
|
486
|
+
## Working Across Projects
|
487
|
+
|
488
|
+
All calls to the Pub/Sub service use the same project and credentials provided
|
489
|
+
to the {Google::Cloud::PubSub.new PubSub.new} method. However, it is common to
|
490
|
+
reference topics or subscriptions in other projects, which can be achieved by
|
491
|
+
using the `project` option. The main credentials must have permissions to the
|
492
|
+
topics and subscriptions in other projects.
|
493
|
+
|
494
|
+
```ruby
|
495
|
+
require "google/cloud/pubsub"
|
496
|
+
|
497
|
+
pubsub = Google::Cloud::PubSub.new # my-project
|
498
|
+
|
499
|
+
# Get a topic in the current project
|
500
|
+
my_topic = pubsub.topic "my-topic"
|
501
|
+
my_topic.name #=> "projects/my-project/topics/my-topic"
|
502
|
+
# Get a topic in another project
|
503
|
+
other_topic = pubsub.topic "other-topic", project: "other-project-id"
|
504
|
+
other_topic.name #=> "projects/other-project-id/topics/other-topic"
|
505
|
+
```
|
506
|
+
|
507
|
+
It is possible to create a subscription in the current project that pulls
|
508
|
+
from a topic in another project:
|
509
|
+
|
510
|
+
```ruby
|
511
|
+
require "google/cloud/pubsub"
|
512
|
+
|
513
|
+
pubsub = Google::Cloud::PubSub.new # my-project
|
514
|
+
|
515
|
+
# Get a topic in another project
|
516
|
+
topic = pubsub.topic "other-topic", project: "other-project-id"
|
517
|
+
# Create a subscription in the current project that pulls from
|
518
|
+
# the topic in another project
|
519
|
+
sub = topic.subscribe "my-sub"
|
520
|
+
sub.name #=> "projects/my-project/subscriptions/my-sub"
|
521
|
+
sub.topic.name #=> "projects/other-project-id/topics/other-topic"
|
522
|
+
```
|
523
|
+
|
524
|
+
## Additional information
|
525
|
+
|
526
|
+
Google Cloud Pub/Sub can be configured to use an emulator or to enable gRPC's
|
527
|
+
logging. To learn more, see the {file:EMULATOR.md Emulator guide}} and
|
528
|
+
{file:LOGGING.md Logging guide}.
|