google-cloud-pubsub 0.26.0 → 2.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/.yardopts +12 -2
- data/AUTHENTICATION.md +178 -0
- data/CHANGELOG.md +659 -0
- data/CODE_OF_CONDUCT.md +40 -0
- data/CONTRIBUTING.md +187 -0
- data/EMULATOR.md +37 -0
- data/LICENSE +2 -2
- data/LOGGING.md +32 -0
- data/OVERVIEW.md +528 -0
- data/TROUBLESHOOTING.md +31 -0
- data/lib/google/cloud/pubsub/async_publisher/batch.rb +310 -0
- data/lib/google/cloud/pubsub/async_publisher.rb +402 -0
- data/lib/google/cloud/pubsub/batch_publisher.rb +100 -0
- data/lib/google/cloud/pubsub/convert.rb +91 -0
- data/lib/google/cloud/pubsub/credentials.rb +26 -10
- data/lib/google/cloud/pubsub/errors.rb +85 -0
- data/lib/google/cloud/pubsub/message.rb +80 -17
- data/lib/google/cloud/pubsub/policy.rb +17 -14
- data/lib/google/cloud/pubsub/project.rb +364 -250
- data/lib/google/cloud/pubsub/publish_result.rb +103 -0
- data/lib/google/cloud/pubsub/received_message.rb +162 -24
- data/lib/google/cloud/pubsub/retry_policy.rb +88 -0
- data/lib/google/cloud/pubsub/schema/list.rb +180 -0
- data/lib/google/cloud/pubsub/schema.rb +310 -0
- data/lib/google/cloud/pubsub/service.rb +281 -265
- data/lib/google/cloud/pubsub/snapshot/list.rb +21 -21
- data/lib/google/cloud/pubsub/snapshot.rb +55 -15
- data/lib/google/cloud/pubsub/subscriber/enumerator_queue.rb +54 -0
- data/lib/google/cloud/pubsub/subscriber/inventory.rb +173 -0
- data/lib/google/cloud/pubsub/subscriber/sequencer.rb +115 -0
- data/lib/google/cloud/pubsub/subscriber/stream.rb +400 -0
- data/lib/google/cloud/pubsub/subscriber/timed_unary_buffer.rb +230 -0
- data/lib/google/cloud/pubsub/subscriber.rb +417 -0
- data/lib/google/cloud/pubsub/subscription/list.rb +28 -28
- data/lib/google/cloud/pubsub/subscription/push_config.rb +268 -0
- data/lib/google/cloud/pubsub/subscription.rb +900 -172
- data/lib/google/cloud/pubsub/topic/list.rb +21 -21
- data/lib/google/cloud/pubsub/topic.rb +674 -95
- data/lib/google/cloud/pubsub/version.rb +6 -4
- data/lib/google/cloud/pubsub.rb +104 -439
- data/lib/google-cloud-pubsub.rb +60 -29
- metadata +88 -50
- data/README.md +0 -69
- data/lib/google/cloud/pubsub/topic/publisher.rb +0 -86
- data/lib/google/cloud/pubsub/v1/doc/google/protobuf/duration.rb +0 -77
- data/lib/google/cloud/pubsub/v1/doc/google/protobuf/field_mask.rb +0 -223
- data/lib/google/cloud/pubsub/v1/doc/google/protobuf/timestamp.rb +0 -81
- data/lib/google/cloud/pubsub/v1/doc/google/pubsub/v1/pubsub.rb +0 -503
- data/lib/google/cloud/pubsub/v1/publisher_client.rb +0 -605
- data/lib/google/cloud/pubsub/v1/publisher_client_config.json +0 -96
- data/lib/google/cloud/pubsub/v1/subscriber_client.rb +0 -1104
- data/lib/google/cloud/pubsub/v1/subscriber_client_config.json +0 -127
- data/lib/google/cloud/pubsub/v1.rb +0 -17
- data/lib/google/pubsub/v1/pubsub_pb.rb +0 -187
- data/lib/google/pubsub/v1/pubsub_services_pb.rb +0 -159
data/CODE_OF_CONDUCT.md
ADDED
@@ -0,0 +1,40 @@
|
|
1
|
+
# Contributor Code of Conduct
|
2
|
+
|
3
|
+
As contributors and maintainers of this project, and in the interest of
|
4
|
+
fostering an open and welcoming community, we pledge to respect all people who
|
5
|
+
contribute through reporting issues, posting feature requests, updating
|
6
|
+
documentation, submitting pull requests or patches, and other activities.
|
7
|
+
|
8
|
+
We are committed to making participation in this project a harassment-free
|
9
|
+
experience for everyone, regardless of level of experience, gender, gender
|
10
|
+
identity and expression, sexual orientation, disability, personal appearance,
|
11
|
+
body size, race, ethnicity, age, religion, or nationality.
|
12
|
+
|
13
|
+
Examples of unacceptable behavior by participants include:
|
14
|
+
|
15
|
+
* The use of sexualized language or imagery
|
16
|
+
* Personal attacks
|
17
|
+
* Trolling or insulting/derogatory comments
|
18
|
+
* Public or private harassment
|
19
|
+
* Publishing other's private information, such as physical or electronic
|
20
|
+
addresses, without explicit permission
|
21
|
+
* Other unethical or unprofessional conduct.
|
22
|
+
|
23
|
+
Project maintainers have the right and responsibility to remove, edit, or reject
|
24
|
+
comments, commits, code, wiki edits, issues, and other contributions that are
|
25
|
+
not aligned to this Code of Conduct. By adopting this Code of Conduct, project
|
26
|
+
maintainers commit themselves to fairly and consistently applying these
|
27
|
+
principles to every aspect of managing this project. Project maintainers who do
|
28
|
+
not follow or enforce the Code of Conduct may be permanently removed from the
|
29
|
+
project team.
|
30
|
+
|
31
|
+
This code of conduct applies both within project spaces and in public spaces
|
32
|
+
when an individual is representing the project or its community.
|
33
|
+
|
34
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
35
|
+
reported by opening an issue or contacting one or more of the project
|
36
|
+
maintainers.
|
37
|
+
|
38
|
+
This Code of Conduct is adapted from the [Contributor
|
39
|
+
Covenant](http://contributor-covenant.org), version 1.2.0, available at
|
40
|
+
[http://contributor-covenant.org/version/1/2/0/](http://contributor-covenant.org/version/1/2/0/)
|
data/CONTRIBUTING.md
ADDED
@@ -0,0 +1,187 @@
|
|
1
|
+
# Contributing to Google Cloud Pub/Sub
|
2
|
+
|
3
|
+
1. **Sign one of the contributor license agreements below.**
|
4
|
+
2. Fork the repo, develop and test your code changes.
|
5
|
+
3. Send a pull request.
|
6
|
+
|
7
|
+
## Contributor License Agreements
|
8
|
+
|
9
|
+
Before we can accept your pull requests you'll need to sign a Contributor
|
10
|
+
License Agreement (CLA):
|
11
|
+
|
12
|
+
- **If you are an individual writing original source code** and **you own the
|
13
|
+
intellectual property**, then you'll need to sign an [individual
|
14
|
+
CLA](https://developers.google.com/open-source/cla/individual).
|
15
|
+
- **If you work for a company that wants to allow you to contribute your work**,
|
16
|
+
then you'll need to sign a [corporate
|
17
|
+
CLA](https://developers.google.com/open-source/cla/corporate).
|
18
|
+
|
19
|
+
You can sign these electronically (just scroll to the bottom). After that, we'll
|
20
|
+
be able to accept your pull requests.
|
21
|
+
|
22
|
+
## Setup
|
23
|
+
|
24
|
+
In order to use the google-cloud-pubsub console and run the project's tests,
|
25
|
+
there is a small amount of setup:
|
26
|
+
|
27
|
+
1. Install Ruby. google-cloud-pubsub requires Ruby 2.5+. You may choose to
|
28
|
+
manage your Ruby and gem installations with [RVM](https://rvm.io/),
|
29
|
+
[rbenv](https://github.com/rbenv/rbenv), or
|
30
|
+
[chruby](https://github.com/postmodern/chruby).
|
31
|
+
|
32
|
+
2. Install [Bundler](http://bundler.io/).
|
33
|
+
|
34
|
+
```sh
|
35
|
+
$ gem install bundler
|
36
|
+
```
|
37
|
+
|
38
|
+
3. Install the top-level project dependencies.
|
39
|
+
|
40
|
+
```sh
|
41
|
+
$ bundle install
|
42
|
+
```
|
43
|
+
|
44
|
+
4. Install the Pub/Sub dependencies.
|
45
|
+
|
46
|
+
```sh
|
47
|
+
$ cd google-cloud-pubsub/
|
48
|
+
$ bundle install
|
49
|
+
```
|
50
|
+
|
51
|
+
## Console
|
52
|
+
|
53
|
+
In order to run code interactively, you can automatically load
|
54
|
+
google-cloud-pubsub and its dependencies in IRB. This requires that your
|
55
|
+
developer environment has already been configured by following the steps
|
56
|
+
described in the {file:AUTHENTICATION.md Authentication Guide}. An IRB console
|
57
|
+
can be created with:
|
58
|
+
|
59
|
+
```sh
|
60
|
+
$ cd google-cloud-pubsub/
|
61
|
+
$ bundle exec rake console
|
62
|
+
```
|
63
|
+
|
64
|
+
## Pub/Sub Tests
|
65
|
+
|
66
|
+
Tests are very important part of google-cloud-pubsub. All contributions
|
67
|
+
should include tests that ensure the contributed code behaves as expected.
|
68
|
+
|
69
|
+
To run the unit tests, documentation tests, and code style checks together for a
|
70
|
+
package:
|
71
|
+
|
72
|
+
``` sh
|
73
|
+
$ cd google-cloud-pubsub/
|
74
|
+
$ bundle exec rake ci
|
75
|
+
```
|
76
|
+
|
77
|
+
To run the command above, plus all acceptance tests, use `rake ci:acceptance` or
|
78
|
+
its handy alias, `rake ci:a`.
|
79
|
+
|
80
|
+
### Pub/Sub Unit Tests
|
81
|
+
|
82
|
+
|
83
|
+
The project uses the [minitest](https://github.com/seattlerb/minitest) library,
|
84
|
+
including [specs](https://github.com/seattlerb/minitest#specs),
|
85
|
+
[mocks](https://github.com/seattlerb/minitest#mocks) and
|
86
|
+
[minitest-autotest](https://github.com/seattlerb/minitest-autotest).
|
87
|
+
|
88
|
+
To run the Pub/Sub unit tests:
|
89
|
+
|
90
|
+
``` sh
|
91
|
+
$ cd google-cloud-pubsub/
|
92
|
+
$ bundle exec rake test
|
93
|
+
```
|
94
|
+
|
95
|
+
### Pub/Sub Documentation Tests
|
96
|
+
|
97
|
+
The project tests the code examples in the gem's
|
98
|
+
[YARD](https://github.com/lsegal/yard)-based documentation.
|
99
|
+
|
100
|
+
The example testing functions in a way that is very similar to unit testing, and
|
101
|
+
in fact the library providing it,
|
102
|
+
[yard-doctest](https://github.com/p0deje/yard-doctest), is based on the
|
103
|
+
project's unit test library, [minitest](https://github.com/seattlerb/minitest).
|
104
|
+
|
105
|
+
To run the Pub/Sub documentation tests:
|
106
|
+
|
107
|
+
``` sh
|
108
|
+
$ cd google-cloud-pubsub/
|
109
|
+
$ bundle exec rake doctest
|
110
|
+
```
|
111
|
+
|
112
|
+
If you add, remove or modify documentation examples when working on a pull
|
113
|
+
request, you may need to update the setup for the tests. The stubs and mocks
|
114
|
+
required to run the tests are located in `support/doctest_helper.rb`. Please
|
115
|
+
note that much of the setup is matched by the title of the
|
116
|
+
[`@example`](http://www.rubydoc.info/gems/yard/file/docs/Tags.md#example) tag.
|
117
|
+
If you alter an example's title, you may encounter breaking tests.
|
118
|
+
|
119
|
+
### Pub/Sub Acceptance Tests
|
120
|
+
|
121
|
+
The Pub/Sub acceptance tests interact with the live service API. Follow the
|
122
|
+
instructions in the {file:AUTHENTICATION.md Authentication Guide} for enabling
|
123
|
+
the Pub/Sub API. Occasionally, some API features may not yet be generally
|
124
|
+
available, making it difficult for some contributors to successfully run the
|
125
|
+
entire acceptance test suite. However, please ensure that you do successfully
|
126
|
+
run acceptance tests for any code areas covered by your pull request.
|
127
|
+
|
128
|
+
To run the acceptance tests, first create and configure a project in the Google
|
129
|
+
Developers Console, as described in the {file:AUTHENTICATION.md Authentication Guide}. Be sure to download the JSON KEY file. Make note of the PROJECT_ID and
|
130
|
+
the KEYFILE location on your system.
|
131
|
+
|
132
|
+
Before you can run the Pub/Sub acceptance tests, you must first create indexes
|
133
|
+
used in the tests.
|
134
|
+
|
135
|
+
#### Running the Pub/Sub acceptance tests
|
136
|
+
|
137
|
+
To run the Pub/Sub acceptance tests:
|
138
|
+
|
139
|
+
``` sh
|
140
|
+
$ cd google-cloud-pubsub/
|
141
|
+
$ bundle exec rake acceptance[\\{my-project-id},\\{/path/to/keyfile.json}]
|
142
|
+
```
|
143
|
+
|
144
|
+
Or, if you prefer you can store the values in the `GCLOUD_TEST_PROJECT` and
|
145
|
+
`GCLOUD_TEST_KEYFILE` environment variables:
|
146
|
+
|
147
|
+
``` sh
|
148
|
+
$ cd google-cloud-pubsub/
|
149
|
+
$ export GCLOUD_TEST_PROJECT=\\{my-project-id}
|
150
|
+
$ export GCLOUD_TEST_KEYFILE=\\{/path/to/keyfile.json}
|
151
|
+
$ bundle exec rake acceptance
|
152
|
+
```
|
153
|
+
|
154
|
+
If you want to use a different project and credentials for acceptance tests, you
|
155
|
+
can use the more specific `PUBSUB_TEST_PROJECT` and `PUBSUB_TEST_KEYFILE`
|
156
|
+
environment variables:
|
157
|
+
|
158
|
+
``` sh
|
159
|
+
$ cd google-cloud-pubsub/
|
160
|
+
$ export PUBSUB_TEST_PROJECT=\\{my-project-id}
|
161
|
+
$ export PUBSUB_TEST_KEYFILE=\\{/path/to/keyfile.json}
|
162
|
+
$ bundle exec rake acceptance
|
163
|
+
```
|
164
|
+
|
165
|
+
## Coding Style
|
166
|
+
|
167
|
+
Please follow the established coding style in the library. The style is is
|
168
|
+
largely based on [The Ruby Style
|
169
|
+
Guide](https://github.com/bbatsov/ruby-style-guide) with a few exceptions based
|
170
|
+
on seattle-style:
|
171
|
+
|
172
|
+
* Avoid parenthesis when possible, including in method definitions.
|
173
|
+
* Always use double quotes strings. ([Option
|
174
|
+
B](https://github.com/bbatsov/ruby-style-guide#strings))
|
175
|
+
|
176
|
+
You can check your code against these rules by running Rubocop like so:
|
177
|
+
|
178
|
+
```sh
|
179
|
+
$ cd google-cloud-pubsub/
|
180
|
+
$ bundle exec rake rubocop
|
181
|
+
```
|
182
|
+
|
183
|
+
## Code of Conduct
|
184
|
+
|
185
|
+
Please note that this project is released with a Contributor Code of Conduct. By
|
186
|
+
participating in this project you agree to abide by its terms. See
|
187
|
+
{file:CODE_OF_CONDUCT.md Code of Conduct} for more information.
|
data/EMULATOR.md
ADDED
@@ -0,0 +1,37 @@
|
|
1
|
+
# Google Cloud Pub/Sub Emulator
|
2
|
+
|
3
|
+
To develop and test your application locally, you can use the [Google Cloud
|
4
|
+
Pub/Sub Emulator](https://cloud.google.com/pubsub/emulator), which provides
|
5
|
+
[local emulation](https://cloud.google.com/sdk/gcloud/reference/beta/emulators/)
|
6
|
+
of the production Google Cloud Pub/Sub environment. You can start the Google
|
7
|
+
Cloud Pub/Sub emulator using the `gcloud` command-line tool.
|
8
|
+
|
9
|
+
To configure your ruby code to use the emulator, set the `PUBSUB_EMULATOR_HOST`
|
10
|
+
environment variable to the host and port where the emulator is running. The
|
11
|
+
value can be set as an environment variable in the shell running the ruby code,
|
12
|
+
or can be set directly in the ruby code as shown below.
|
13
|
+
|
14
|
+
```ruby
|
15
|
+
require "google/cloud/pubsub"
|
16
|
+
|
17
|
+
# Make Pub/Sub use the emulator
|
18
|
+
ENV["PUBSUB_EMULATOR_HOST"] = "localhost:8918"
|
19
|
+
|
20
|
+
pubsub = Google::Cloud::PubSub.new project_id:"emulator-project-id"
|
21
|
+
|
22
|
+
# Get a topic in the current project
|
23
|
+
my_topic = pubsub.new_topic "my-topic"
|
24
|
+
my_topic.name #=> "projects/emulator-project-id/topics/my-topic"
|
25
|
+
```
|
26
|
+
|
27
|
+
Or by providing the `emulator_host` argument:
|
28
|
+
|
29
|
+
```ruby
|
30
|
+
require "google/cloud/pubsub"
|
31
|
+
|
32
|
+
pubsub = Google::Cloud::PubSub.new emulator_host: "localhost:8918"
|
33
|
+
|
34
|
+
# Get a topic in the current project
|
35
|
+
my_topic = pubsub.new_topic "my-topic"
|
36
|
+
my_topic.name #=> "projects/emulator-project-id/topics/my-topic"
|
37
|
+
```
|
data/LICENSE
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
Apache License
|
2
2
|
Version 2.0, January 2004
|
3
|
-
|
3
|
+
https://www.apache.org/licenses/
|
4
4
|
|
5
5
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
6
6
|
|
@@ -192,7 +192,7 @@
|
|
192
192
|
you may not use this file except in compliance with the License.
|
193
193
|
You may obtain a copy of the License at
|
194
194
|
|
195
|
-
|
195
|
+
https://www.apache.org/licenses/LICENSE-2.0
|
196
196
|
|
197
197
|
Unless required by applicable law or agreed to in writing, software
|
198
198
|
distributed under the License is distributed on an "AS IS" BASIS,
|
data/LOGGING.md
ADDED
@@ -0,0 +1,32 @@
|
|
1
|
+
# Enabling gRPC Logging
|
2
|
+
|
3
|
+
To enable logging for this library, set the logger for the underlying
|
4
|
+
[gRPC](https://github.com/grpc/grpc/tree/master/src/ruby) library. The logger
|
5
|
+
that you set may be a Ruby stdlib
|
6
|
+
[`Logger`](https://ruby-doc.org/stdlib/libdoc/logger/rdoc/Logger.html) as
|
7
|
+
shown below, or a
|
8
|
+
[`Google::Cloud::Logging::Logger`](https://googleapis.dev/ruby/google-cloud-logging/latest)
|
9
|
+
that will write logs to [Stackdriver
|
10
|
+
Logging](https://cloud.google.com/logging/). See
|
11
|
+
[grpc/logconfig.rb](https://github.com/grpc/grpc/blob/master/src/ruby/lib/grpc/logconfig.rb)
|
12
|
+
and the gRPC
|
13
|
+
[spec_helper.rb](https://github.com/grpc/grpc/blob/master/src/ruby/spec/spec_helper.rb)
|
14
|
+
for additional information.
|
15
|
+
|
16
|
+
Configuring a Ruby stdlib logger:
|
17
|
+
|
18
|
+
```ruby
|
19
|
+
require "logger"
|
20
|
+
|
21
|
+
module MyLogger
|
22
|
+
LOGGER = Logger.new $stderr, level: Logger::WARN
|
23
|
+
def logger
|
24
|
+
LOGGER
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
# Define a gRPC module-level logger method before grpc/logconfig.rb loads.
|
29
|
+
module GRPC
|
30
|
+
extend MyLogger
|
31
|
+
end
|
32
|
+
```
|
data/OVERVIEW.md
ADDED
@@ -0,0 +1,528 @@
|
|
1
|
+
# Google Cloud Pub/Sub
|
2
|
+
|
3
|
+
Google Cloud Pub/Sub is designed to provide reliable, many-to-many, asynchronous
|
4
|
+
messaging between applications. Publisher applications can send messages to a
|
5
|
+
"topic" and other applications can subscribe to that topic to receive the
|
6
|
+
messages. By decoupling senders and receivers, Google Cloud Pub/Sub allows
|
7
|
+
developers to communicate between independently written applications.
|
8
|
+
|
9
|
+
The goal of google-cloud is to provide an API that is comfortable to Rubyists.
|
10
|
+
Your authentication credentials are detected automatically in Google Cloud
|
11
|
+
Platform (GCP), including Google Compute Engine (GCE), Google Kubernetes Engine
|
12
|
+
(GKE), Google App Engine (GAE), Google Cloud Functions (GCF) and Cloud Run. In
|
13
|
+
other environments you can configure authentication easily, either directly in
|
14
|
+
your code or via environment variables. Read more about the options for
|
15
|
+
connecting in the {file:AUTHENTICATION.md Authentication Guide}.
|
16
|
+
|
17
|
+
```ruby
|
18
|
+
require "google/cloud/pubsub"
|
19
|
+
|
20
|
+
pubsub = Google::Cloud::PubSub.new
|
21
|
+
|
22
|
+
topic = pubsub.topic "my-topic"
|
23
|
+
topic.publish "task completed"
|
24
|
+
```
|
25
|
+
|
26
|
+
To learn more about Pub/Sub, read the [Google Cloud Pub/Sub Overview
|
27
|
+
](https://cloud.google.com/pubsub/overview).
|
28
|
+
|
29
|
+
## Retrieving Topics
|
30
|
+
|
31
|
+
A Topic is a named resource to which messages are sent by publishers. A Topic is
|
32
|
+
found by its name. (See {Google::Cloud::PubSub::Project#topic Project#topic})
|
33
|
+
|
34
|
+
```ruby
|
35
|
+
require "google/cloud/pubsub"
|
36
|
+
|
37
|
+
pubsub = Google::Cloud::PubSub.new
|
38
|
+
topic = pubsub.topic "my-topic"
|
39
|
+
```
|
40
|
+
|
41
|
+
## Creating a Topic
|
42
|
+
|
43
|
+
A Topic is created from a Project. (See
|
44
|
+
{Google::Cloud::PubSub::Project#create_topic Project#create_topic})
|
45
|
+
|
46
|
+
```ruby
|
47
|
+
require "google/cloud/pubsub"
|
48
|
+
|
49
|
+
pubsub = Google::Cloud::PubSub.new
|
50
|
+
topic = pubsub.create_topic "my-topic"
|
51
|
+
```
|
52
|
+
|
53
|
+
## Retrieving Subscriptions
|
54
|
+
|
55
|
+
A Subscription is a named resource representing the stream of messages from a
|
56
|
+
single, specific Topic, to be delivered to the subscribing application. A
|
57
|
+
Subscription is found by its name. (See
|
58
|
+
{Google::Cloud::PubSub::Topic#subscription Topic#subscription})
|
59
|
+
|
60
|
+
```ruby
|
61
|
+
require "google/cloud/pubsub"
|
62
|
+
|
63
|
+
pubsub = Google::Cloud::PubSub.new
|
64
|
+
|
65
|
+
topic = pubsub.topic "my-topic"
|
66
|
+
subscription = topic.subscription "my-topic-subscription"
|
67
|
+
puts subscription.name
|
68
|
+
```
|
69
|
+
|
70
|
+
## Creating a Subscription
|
71
|
+
|
72
|
+
A Subscription is created from a Topic. (See
|
73
|
+
{Google::Cloud::PubSub::Topic#subscribe Topic#subscribe})
|
74
|
+
|
75
|
+
```ruby
|
76
|
+
require "google/cloud/pubsub"
|
77
|
+
|
78
|
+
pubsub = Google::Cloud::PubSub.new
|
79
|
+
|
80
|
+
topic = pubsub.topic "my-topic"
|
81
|
+
sub = topic.subscribe "my-topic-sub"
|
82
|
+
puts sub.name # => "my-topic-sub"
|
83
|
+
```
|
84
|
+
|
85
|
+
The subscription can be created that specifies the number of seconds to wait to
|
86
|
+
be acknowledged as well as an endpoint URL to push the messages to:
|
87
|
+
|
88
|
+
```ruby
|
89
|
+
require "google/cloud/pubsub"
|
90
|
+
|
91
|
+
pubsub = Google::Cloud::PubSub.new
|
92
|
+
|
93
|
+
topic = pubsub.topic "my-topic"
|
94
|
+
sub = topic.subscribe "my-topic-sub",
|
95
|
+
deadline: 120,
|
96
|
+
endpoint: "https://example.com/push"
|
97
|
+
```
|
98
|
+
|
99
|
+
## Publishing Messages
|
100
|
+
|
101
|
+
Messages are published to a topic. Any message published to a topic without a
|
102
|
+
subscription will be lost. Ensure the topic has a subscription before
|
103
|
+
publishing. (See {Google::Cloud::PubSub::Topic#publish Topic#publish})
|
104
|
+
|
105
|
+
```ruby
|
106
|
+
require "google/cloud/pubsub"
|
107
|
+
|
108
|
+
pubsub = Google::Cloud::PubSub.new
|
109
|
+
|
110
|
+
topic = pubsub.topic "my-topic"
|
111
|
+
msg = topic.publish "task completed"
|
112
|
+
```
|
113
|
+
|
114
|
+
Messages can also be published with attributes:
|
115
|
+
|
116
|
+
```ruby
|
117
|
+
require "google/cloud/pubsub"
|
118
|
+
|
119
|
+
pubsub = Google::Cloud::PubSub.new
|
120
|
+
|
121
|
+
topic = pubsub.topic "my-topic"
|
122
|
+
msg = topic.publish "task completed",
|
123
|
+
foo: :bar,
|
124
|
+
this: :that
|
125
|
+
```
|
126
|
+
|
127
|
+
Messages can also be published in batches asynchronously using `publish_async`.
|
128
|
+
(See {Google::Cloud::PubSub::Topic#publish_async Topic#publish_async} and
|
129
|
+
{Google::Cloud::PubSub::AsyncPublisher AsyncPublisher})
|
130
|
+
|
131
|
+
```ruby
|
132
|
+
require "google/cloud/pubsub"
|
133
|
+
|
134
|
+
pubsub = Google::Cloud::PubSub.new
|
135
|
+
|
136
|
+
topic = pubsub.topic "my-topic"
|
137
|
+
topic.publish_async "task completed" do |result|
|
138
|
+
if result.succeeded?
|
139
|
+
log_publish_success result.data
|
140
|
+
else
|
141
|
+
log_publish_failure result.data, result.error
|
142
|
+
end
|
143
|
+
end
|
144
|
+
|
145
|
+
topic.async_publisher.stop!
|
146
|
+
```
|
147
|
+
|
148
|
+
Or multiple messages can be published in batches at the same time by passing a
|
149
|
+
block to `publish`. (See {Google::Cloud::PubSub::BatchPublisher BatchPublisher})
|
150
|
+
|
151
|
+
```ruby
|
152
|
+
require "google/cloud/pubsub"
|
153
|
+
|
154
|
+
pubsub = Google::Cloud::PubSub.new
|
155
|
+
|
156
|
+
topic = pubsub.topic "my-topic"
|
157
|
+
msgs = topic.publish do |batch|
|
158
|
+
batch.publish "task 1 completed", foo: :bar
|
159
|
+
batch.publish "task 2 completed", foo: :baz
|
160
|
+
batch.publish "task 3 completed", foo: :bif
|
161
|
+
end
|
162
|
+
```
|
163
|
+
|
164
|
+
## Receiving Messages
|
165
|
+
|
166
|
+
Messages can be streamed from a subscription with a subscriber object that is
|
167
|
+
created using `listen`. (See {Google::Cloud::PubSub::Subscription#listen
|
168
|
+
Subscription#listen} and {Google::Cloud::PubSub::Subscriber Subscriber})
|
169
|
+
|
170
|
+
```ruby
|
171
|
+
require "google/cloud/pubsub"
|
172
|
+
|
173
|
+
pubsub = Google::Cloud::PubSub.new
|
174
|
+
|
175
|
+
sub = pubsub.subscription "my-topic-sub"
|
176
|
+
|
177
|
+
# Create a subscriber to listen for available messages.
|
178
|
+
# By default, this block will be called on 8 concurrent threads
|
179
|
+
# but this can be tuned with the `threads` option.
|
180
|
+
# The `streams` and `inventory` parameters allow further tuning.
|
181
|
+
subscriber = sub.listen threads: { callback: 16 } do |received_message|
|
182
|
+
# process message
|
183
|
+
puts "Data: #{received_message.message.data}, published at #{received_message.message.published_at}"
|
184
|
+
received_message.acknowledge!
|
185
|
+
end
|
186
|
+
|
187
|
+
# Handle exceptions from listener
|
188
|
+
subscriber.on_error do |exception|
|
189
|
+
puts "Exception: #{exception.class} #{exception.message}"
|
190
|
+
end
|
191
|
+
|
192
|
+
# Gracefully shut down the subscriber on program exit, blocking until
|
193
|
+
# all received messages have been processed or 10 seconds have passed
|
194
|
+
at_exit do
|
195
|
+
subscriber.stop!(10)
|
196
|
+
end
|
197
|
+
|
198
|
+
# Start background threads that will call the block passed to listen.
|
199
|
+
subscriber.start
|
200
|
+
|
201
|
+
# Block, letting processing threads continue in the background
|
202
|
+
sleep
|
203
|
+
```
|
204
|
+
|
205
|
+
Messages also can be pulled directly in a one-time operation. (See
|
206
|
+
{Google::Cloud::PubSub::Subscription#pull Subscription#pull})
|
207
|
+
|
208
|
+
The `immediate: false` option is recommended to avoid adverse impacts on the
|
209
|
+
performance of pull operations.
|
210
|
+
|
211
|
+
```ruby
|
212
|
+
require "google/cloud/pubsub"
|
213
|
+
|
214
|
+
pubsub = Google::Cloud::PubSub.new
|
215
|
+
|
216
|
+
sub = pubsub.subscription "my-topic-sub"
|
217
|
+
received_messages = sub.pull immediate: false
|
218
|
+
```
|
219
|
+
|
220
|
+
A maximum number of messages to pull can be specified:
|
221
|
+
|
222
|
+
```ruby
|
223
|
+
require "google/cloud/pubsub"
|
224
|
+
|
225
|
+
pubsub = Google::Cloud::PubSub.new
|
226
|
+
|
227
|
+
sub = pubsub.subscription "my-topic-sub"
|
228
|
+
received_messages = sub.pull immediate: false, max: 10
|
229
|
+
```
|
230
|
+
|
231
|
+
## Acknowledging a Message
|
232
|
+
|
233
|
+
Messages that are received can be acknowledged in Pub/Sub, marking the message
|
234
|
+
to be removed so it cannot be pulled again.
|
235
|
+
|
236
|
+
A Message that can be acknowledged is called a ReceivedMessage. ReceivedMessages
|
237
|
+
can be acknowledged one at a time: (See
|
238
|
+
{Google::Cloud::PubSub::ReceivedMessage#acknowledge!
|
239
|
+
ReceivedMessage#acknowledge!})
|
240
|
+
|
241
|
+
```ruby
|
242
|
+
require "google/cloud/pubsub"
|
243
|
+
|
244
|
+
pubsub = Google::Cloud::PubSub.new
|
245
|
+
|
246
|
+
sub = pubsub.subscription "my-topic-sub"
|
247
|
+
|
248
|
+
subscriber = sub.listen do |received_message|
|
249
|
+
# process message
|
250
|
+
received_message.acknowledge!
|
251
|
+
end
|
252
|
+
|
253
|
+
# Start background threads that will call the block passed to listen.
|
254
|
+
subscriber.start
|
255
|
+
|
256
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
257
|
+
subscriber.stop!
|
258
|
+
```
|
259
|
+
|
260
|
+
Or, multiple messages can be acknowledged in a single API call: (See
|
261
|
+
{Google::Cloud::PubSub::Subscription#acknowledge Subscription#acknowledge})
|
262
|
+
|
263
|
+
```ruby
|
264
|
+
require "google/cloud/pubsub"
|
265
|
+
|
266
|
+
pubsub = Google::Cloud::PubSub.new
|
267
|
+
|
268
|
+
sub = pubsub.subscription "my-topic-sub"
|
269
|
+
received_messages = sub.pull immediate: false
|
270
|
+
sub.acknowledge received_messages
|
271
|
+
```
|
272
|
+
|
273
|
+
## Modifying a Deadline
|
274
|
+
|
275
|
+
A message must be acknowledged after it is pulled, or Pub/Sub will mark the
|
276
|
+
message for redelivery. The message acknowledgement deadline can delayed if more
|
277
|
+
time is needed. This will allow more time to process the message before the
|
278
|
+
message is marked for redelivery. (See
|
279
|
+
{Google::Cloud::PubSub::ReceivedMessage#modify_ack_deadline!
|
280
|
+
ReceivedMessage#modify_ack_deadline!})
|
281
|
+
|
282
|
+
```ruby
|
283
|
+
require "google/cloud/pubsub"
|
284
|
+
|
285
|
+
pubsub = Google::Cloud::PubSub.new
|
286
|
+
|
287
|
+
sub = pubsub.subscription "my-topic-sub"
|
288
|
+
subscriber = sub.listen do |received_message|
|
289
|
+
puts received_message.message.data
|
290
|
+
|
291
|
+
# Delay for 2 minutes
|
292
|
+
received_message.modify_ack_deadline! 120
|
293
|
+
end
|
294
|
+
|
295
|
+
# Start background threads that will call the block passed to listen.
|
296
|
+
subscriber.start
|
297
|
+
|
298
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
299
|
+
subscriber.stop!
|
300
|
+
```
|
301
|
+
|
302
|
+
The message can also be made available for immediate redelivery:
|
303
|
+
|
304
|
+
```ruby
|
305
|
+
require "google/cloud/pubsub"
|
306
|
+
|
307
|
+
pubsub = Google::Cloud::PubSub.new
|
308
|
+
|
309
|
+
sub = pubsub.subscription "my-topic-sub"
|
310
|
+
subscriber = sub.listen do |received_message|
|
311
|
+
puts received_message.message.data
|
312
|
+
|
313
|
+
# Mark for redelivery
|
314
|
+
received_message.reject!
|
315
|
+
end
|
316
|
+
|
317
|
+
# Start background threads that will call the block passed to listen.
|
318
|
+
subscriber.start
|
319
|
+
|
320
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
321
|
+
subscriber.stop!
|
322
|
+
```
|
323
|
+
|
324
|
+
Multiple messages can be delayed or made available for immediate redelivery:
|
325
|
+
(See {Google::Cloud::PubSub::Subscription#modify_ack_deadline
|
326
|
+
Subscription#modify_ack_deadline})
|
327
|
+
|
328
|
+
```ruby
|
329
|
+
require "google/cloud/pubsub"
|
330
|
+
|
331
|
+
pubsub = Google::Cloud::PubSub.new
|
332
|
+
|
333
|
+
sub = pubsub.subscription "my-topic-sub"
|
334
|
+
received_messages = sub.pull immediate: false
|
335
|
+
sub.modify_ack_deadline 120, received_messages
|
336
|
+
```
|
337
|
+
|
338
|
+
## Using Ordering Keys
|
339
|
+
|
340
|
+
Google Cloud Pub/Sub ordering keys provide the ability to ensure related
|
341
|
+
messages are sent to subscribers in the order in which they were published.
|
342
|
+
Messages can be tagged with an ordering key, a string that identifies related
|
343
|
+
messages for which publish order should be respected. The service guarantees
|
344
|
+
that, for a given ordering key and publisher, messages are sent to subscribers
|
345
|
+
in the order in which they were published. Ordering does not require sacrificing
|
346
|
+
high throughput or scalability, as the service automatically distributes
|
347
|
+
messages for different ordering keys across subscribers.
|
348
|
+
|
349
|
+
Note: At the time of this release, ordering keys are not yet publicly enabled
|
350
|
+
and requires special project enablements.
|
351
|
+
|
352
|
+
### Publishing Ordered Messages
|
353
|
+
|
354
|
+
To use ordering keys when publishing messages, a call to
|
355
|
+
{Google::Cloud::PubSub::Topic#enable_message_ordering!
|
356
|
+
Topic#enable_message_ordering!} must be made and the `ordering_key` argument
|
357
|
+
must be provided when calling {Google::Cloud::PubSub::Topic#publish_async
|
358
|
+
Topic#publish_async}.
|
359
|
+
|
360
|
+
```ruby
|
361
|
+
require "google/cloud/pubsub"
|
362
|
+
|
363
|
+
pubsub = Google::Cloud::PubSub.new
|
364
|
+
|
365
|
+
topic = pubsub.topic "my-ordered-topic"
|
366
|
+
|
367
|
+
# Ensure that message ordering is enabled.
|
368
|
+
topic.enable_message_ordering!
|
369
|
+
|
370
|
+
# Publish an ordered message with an ordering key.
|
371
|
+
topic.publish_async "task completed",
|
372
|
+
ordering_key: "task-key"
|
373
|
+
|
374
|
+
# Shut down the publisher when ready to stop publishing messages.
|
375
|
+
topic.async_publisher.stop!
|
376
|
+
```
|
377
|
+
|
378
|
+
### Handling errors with Ordered Keys
|
379
|
+
|
380
|
+
Ordered messages that fail to publish to the Pub/Sub API due to error will put
|
381
|
+
the `ordering_key` in a failed state, and future calls to
|
382
|
+
{Google::Cloud::PubSub::Topic#publish_async Topic#publish_async} with the
|
383
|
+
`ordering_key` will raise {Google::Cloud::PubSub::OrderingKeyError
|
384
|
+
OrderingKeyError}. To allow future messages with the `ordering_key` to be
|
385
|
+
published, the `ordering_key` must be passed to
|
386
|
+
{Google::Cloud::PubSub::Topic#resume_publish Topic#resume_publish}.
|
387
|
+
|
388
|
+
### Receiving Ordered Messages
|
389
|
+
|
390
|
+
To use ordering keys when subscribing to messages, the subscription must be
|
391
|
+
created with message ordering enabled (See
|
392
|
+
{Google::Cloud::PubSub::Topic#subscribe Topic#subscribe} and
|
393
|
+
{Google::Cloud::PubSub::Subscription#message_ordering?
|
394
|
+
Subscription#message_ordering?}) before calling
|
395
|
+
{Google::Cloud::PubSub::Subscription#listen Subscription#listen}. When enabled,
|
396
|
+
the subscriber will deliver messages with the same `ordering_key` in the order
|
397
|
+
they were published.
|
398
|
+
|
399
|
+
```ruby
|
400
|
+
require "google/cloud/pubsub"
|
401
|
+
|
402
|
+
pubsub = Google::Cloud::PubSub.new
|
403
|
+
|
404
|
+
sub = pubsub.subscription "my-ordered-topic-sub"
|
405
|
+
sub.message_ordering? #=> true
|
406
|
+
|
407
|
+
subscriber = sub.listen do |received_message|
|
408
|
+
# Messsages with the same ordering_key are received
|
409
|
+
# in the order in which they were published.
|
410
|
+
received_message.acknowledge!
|
411
|
+
end
|
412
|
+
|
413
|
+
# Start background threads that will call block passed to listen.
|
414
|
+
subscriber.start
|
415
|
+
|
416
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
417
|
+
subscriber.stop!
|
418
|
+
```
|
419
|
+
|
420
|
+
## Minimizing API calls before receiving and acknowledging messages
|
421
|
+
|
422
|
+
A subscription object can be created without making any API calls by providing
|
423
|
+
the `skip_lookup` argument to {Google::Cloud::PubSub::Project#subscription
|
424
|
+
Project#subscription} or {Google::Cloud::PubSub::Topic#subscription
|
425
|
+
Topic#subscription}. A subscriber object can also be created without an API call
|
426
|
+
by providing the `deadline` optional argument to
|
427
|
+
{Google::Cloud::PubSub::Subscription#listen Subscription#listen}:
|
428
|
+
|
429
|
+
```ruby
|
430
|
+
require "google/cloud/pubsub"
|
431
|
+
|
432
|
+
pubsub = Google::Cloud::PubSub.new
|
433
|
+
|
434
|
+
# No API call is made to retrieve the subscription resource.
|
435
|
+
sub = pubsub.subscription "my-topic-sub", skip_lookup: true
|
436
|
+
|
437
|
+
# No API call is made to retrieve the subscription deadline.
|
438
|
+
subscriber = sub.listen deadline: 60 do |received_message|
|
439
|
+
# process message
|
440
|
+
received_message.acknowledge!
|
441
|
+
end
|
442
|
+
|
443
|
+
# Start background threads that will call block passed to listen.
|
444
|
+
subscriber.start
|
445
|
+
|
446
|
+
# Shut down the subscriber when ready to stop receiving messages.
|
447
|
+
subscriber.stop!
|
448
|
+
```
|
449
|
+
|
450
|
+
Skipping API calls may be used to avoid `Google::Cloud::PermissionDeniedError`
|
451
|
+
if your account has limited access to the Pub/Sub API. In particular, the role
|
452
|
+
`roles/pubsub.subscriber` does not have the permission
|
453
|
+
`pubsub.subscriptions.get`, which is required to retrieve a subscription
|
454
|
+
resource. See [Access Control -
|
455
|
+
Roles](https://cloud.google.com/pubsub/docs/access-control#tbl_roles) for the
|
456
|
+
complete list of Pub/Sub roles and permissions.
|
457
|
+
|
458
|
+
## Creating a snapshot and using seek
|
459
|
+
|
460
|
+
You can create a snapshot to retain the existing backlog on a subscription. The
|
461
|
+
snapshot will hold the messages in the subscription's backlog that are
|
462
|
+
unacknowledged upon the successful completion of the `create_snapshot`
|
463
|
+
operation.
|
464
|
+
|
465
|
+
Later, you can use `seek` to reset the subscription's backlog to the snapshot.
|
466
|
+
|
467
|
+
(See {Google::Cloud::PubSub::Subscription#create_snapshot
|
468
|
+
Subscription#create_snapshot} and {Google::Cloud::PubSub::Subscription#seek
|
469
|
+
Subscription#seek})
|
470
|
+
|
471
|
+
```ruby
|
472
|
+
require "google/cloud/pubsub"
|
473
|
+
|
474
|
+
pubsub = Google::Cloud::PubSub.new
|
475
|
+
|
476
|
+
sub = pubsub.subscription "my-topic-sub"
|
477
|
+
|
478
|
+
snapshot = sub.create_snapshot
|
479
|
+
|
480
|
+
received_messages = sub.pull immediate: false
|
481
|
+
sub.acknowledge received_messages
|
482
|
+
|
483
|
+
sub.seek snapshot
|
484
|
+
```
|
485
|
+
|
486
|
+
## Working Across Projects
|
487
|
+
|
488
|
+
All calls to the Pub/Sub service use the same project and credentials provided
|
489
|
+
to the {Google::Cloud::PubSub.new PubSub.new} method. However, it is common to
|
490
|
+
reference topics or subscriptions in other projects, which can be achieved by
|
491
|
+
using the `project` option. The main credentials must have permissions to the
|
492
|
+
topics and subscriptions in other projects.
|
493
|
+
|
494
|
+
```ruby
|
495
|
+
require "google/cloud/pubsub"
|
496
|
+
|
497
|
+
pubsub = Google::Cloud::PubSub.new # my-project
|
498
|
+
|
499
|
+
# Get a topic in the current project
|
500
|
+
my_topic = pubsub.topic "my-topic"
|
501
|
+
my_topic.name #=> "projects/my-project/topics/my-topic"
|
502
|
+
# Get a topic in another project
|
503
|
+
other_topic = pubsub.topic "other-topic", project: "other-project-id"
|
504
|
+
other_topic.name #=> "projects/other-project-id/topics/other-topic"
|
505
|
+
```
|
506
|
+
|
507
|
+
It is possible to create a subscription in the current project that pulls
|
508
|
+
from a topic in another project:
|
509
|
+
|
510
|
+
```ruby
|
511
|
+
require "google/cloud/pubsub"
|
512
|
+
|
513
|
+
pubsub = Google::Cloud::PubSub.new # my-project
|
514
|
+
|
515
|
+
# Get a topic in another project
|
516
|
+
topic = pubsub.topic "other-topic", project: "other-project-id"
|
517
|
+
# Create a subscription in the current project that pulls from
|
518
|
+
# the topic in another project
|
519
|
+
sub = topic.subscribe "my-sub"
|
520
|
+
sub.name #=> "projects/my-project/subscriptions/my-sub"
|
521
|
+
sub.topic.name #=> "projects/other-project-id/topics/other-topic"
|
522
|
+
```
|
523
|
+
|
524
|
+
## Additional information
|
525
|
+
|
526
|
+
Google Cloud Pub/Sub can be configured to use an emulator or to enable gRPC's
|
527
|
+
logging. To learn more, see the {file:EMULATOR.md Emulator guide}} and
|
528
|
+
{file:LOGGING.md Logging guide}.
|