google-cloud-pubsub 0.26.0 → 2.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (56) hide show
  1. checksums.yaml +5 -5
  2. data/.yardopts +12 -2
  3. data/AUTHENTICATION.md +178 -0
  4. data/CHANGELOG.md +659 -0
  5. data/CODE_OF_CONDUCT.md +40 -0
  6. data/CONTRIBUTING.md +187 -0
  7. data/EMULATOR.md +37 -0
  8. data/LICENSE +2 -2
  9. data/LOGGING.md +32 -0
  10. data/OVERVIEW.md +528 -0
  11. data/TROUBLESHOOTING.md +31 -0
  12. data/lib/google/cloud/pubsub/async_publisher/batch.rb +310 -0
  13. data/lib/google/cloud/pubsub/async_publisher.rb +402 -0
  14. data/lib/google/cloud/pubsub/batch_publisher.rb +100 -0
  15. data/lib/google/cloud/pubsub/convert.rb +91 -0
  16. data/lib/google/cloud/pubsub/credentials.rb +26 -10
  17. data/lib/google/cloud/pubsub/errors.rb +85 -0
  18. data/lib/google/cloud/pubsub/message.rb +80 -17
  19. data/lib/google/cloud/pubsub/policy.rb +17 -14
  20. data/lib/google/cloud/pubsub/project.rb +364 -250
  21. data/lib/google/cloud/pubsub/publish_result.rb +103 -0
  22. data/lib/google/cloud/pubsub/received_message.rb +162 -24
  23. data/lib/google/cloud/pubsub/retry_policy.rb +88 -0
  24. data/lib/google/cloud/pubsub/schema/list.rb +180 -0
  25. data/lib/google/cloud/pubsub/schema.rb +310 -0
  26. data/lib/google/cloud/pubsub/service.rb +281 -265
  27. data/lib/google/cloud/pubsub/snapshot/list.rb +21 -21
  28. data/lib/google/cloud/pubsub/snapshot.rb +55 -15
  29. data/lib/google/cloud/pubsub/subscriber/enumerator_queue.rb +54 -0
  30. data/lib/google/cloud/pubsub/subscriber/inventory.rb +173 -0
  31. data/lib/google/cloud/pubsub/subscriber/sequencer.rb +115 -0
  32. data/lib/google/cloud/pubsub/subscriber/stream.rb +400 -0
  33. data/lib/google/cloud/pubsub/subscriber/timed_unary_buffer.rb +230 -0
  34. data/lib/google/cloud/pubsub/subscriber.rb +417 -0
  35. data/lib/google/cloud/pubsub/subscription/list.rb +28 -28
  36. data/lib/google/cloud/pubsub/subscription/push_config.rb +268 -0
  37. data/lib/google/cloud/pubsub/subscription.rb +900 -172
  38. data/lib/google/cloud/pubsub/topic/list.rb +21 -21
  39. data/lib/google/cloud/pubsub/topic.rb +674 -95
  40. data/lib/google/cloud/pubsub/version.rb +6 -4
  41. data/lib/google/cloud/pubsub.rb +104 -439
  42. data/lib/google-cloud-pubsub.rb +60 -29
  43. metadata +88 -50
  44. data/README.md +0 -69
  45. data/lib/google/cloud/pubsub/topic/publisher.rb +0 -86
  46. data/lib/google/cloud/pubsub/v1/doc/google/protobuf/duration.rb +0 -77
  47. data/lib/google/cloud/pubsub/v1/doc/google/protobuf/field_mask.rb +0 -223
  48. data/lib/google/cloud/pubsub/v1/doc/google/protobuf/timestamp.rb +0 -81
  49. data/lib/google/cloud/pubsub/v1/doc/google/pubsub/v1/pubsub.rb +0 -503
  50. data/lib/google/cloud/pubsub/v1/publisher_client.rb +0 -605
  51. data/lib/google/cloud/pubsub/v1/publisher_client_config.json +0 -96
  52. data/lib/google/cloud/pubsub/v1/subscriber_client.rb +0 -1104
  53. data/lib/google/cloud/pubsub/v1/subscriber_client_config.json +0 -127
  54. data/lib/google/cloud/pubsub/v1.rb +0 -17
  55. data/lib/google/pubsub/v1/pubsub_pb.rb +0 -187
  56. data/lib/google/pubsub/v1/pubsub_services_pb.rb +0 -159
@@ -0,0 +1,40 @@
1
+ # Contributor Code of Conduct
2
+
3
+ As contributors and maintainers of this project, and in the interest of
4
+ fostering an open and welcoming community, we pledge to respect all people who
5
+ contribute through reporting issues, posting feature requests, updating
6
+ documentation, submitting pull requests or patches, and other activities.
7
+
8
+ We are committed to making participation in this project a harassment-free
9
+ experience for everyone, regardless of level of experience, gender, gender
10
+ identity and expression, sexual orientation, disability, personal appearance,
11
+ body size, race, ethnicity, age, religion, or nationality.
12
+
13
+ Examples of unacceptable behavior by participants include:
14
+
15
+ * The use of sexualized language or imagery
16
+ * Personal attacks
17
+ * Trolling or insulting/derogatory comments
18
+ * Public or private harassment
19
+ * Publishing other's private information, such as physical or electronic
20
+ addresses, without explicit permission
21
+ * Other unethical or unprofessional conduct.
22
+
23
+ Project maintainers have the right and responsibility to remove, edit, or reject
24
+ comments, commits, code, wiki edits, issues, and other contributions that are
25
+ not aligned to this Code of Conduct. By adopting this Code of Conduct, project
26
+ maintainers commit themselves to fairly and consistently applying these
27
+ principles to every aspect of managing this project. Project maintainers who do
28
+ not follow or enforce the Code of Conduct may be permanently removed from the
29
+ project team.
30
+
31
+ This code of conduct applies both within project spaces and in public spaces
32
+ when an individual is representing the project or its community.
33
+
34
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
35
+ reported by opening an issue or contacting one or more of the project
36
+ maintainers.
37
+
38
+ This Code of Conduct is adapted from the [Contributor
39
+ Covenant](http://contributor-covenant.org), version 1.2.0, available at
40
+ [http://contributor-covenant.org/version/1/2/0/](http://contributor-covenant.org/version/1/2/0/)
data/CONTRIBUTING.md ADDED
@@ -0,0 +1,187 @@
1
+ # Contributing to Google Cloud Pub/Sub
2
+
3
+ 1. **Sign one of the contributor license agreements below.**
4
+ 2. Fork the repo, develop and test your code changes.
5
+ 3. Send a pull request.
6
+
7
+ ## Contributor License Agreements
8
+
9
+ Before we can accept your pull requests you'll need to sign a Contributor
10
+ License Agreement (CLA):
11
+
12
+ - **If you are an individual writing original source code** and **you own the
13
+ intellectual property**, then you'll need to sign an [individual
14
+ CLA](https://developers.google.com/open-source/cla/individual).
15
+ - **If you work for a company that wants to allow you to contribute your work**,
16
+ then you'll need to sign a [corporate
17
+ CLA](https://developers.google.com/open-source/cla/corporate).
18
+
19
+ You can sign these electronically (just scroll to the bottom). After that, we'll
20
+ be able to accept your pull requests.
21
+
22
+ ## Setup
23
+
24
+ In order to use the google-cloud-pubsub console and run the project's tests,
25
+ there is a small amount of setup:
26
+
27
+ 1. Install Ruby. google-cloud-pubsub requires Ruby 2.5+. You may choose to
28
+ manage your Ruby and gem installations with [RVM](https://rvm.io/),
29
+ [rbenv](https://github.com/rbenv/rbenv), or
30
+ [chruby](https://github.com/postmodern/chruby).
31
+
32
+ 2. Install [Bundler](http://bundler.io/).
33
+
34
+ ```sh
35
+ $ gem install bundler
36
+ ```
37
+
38
+ 3. Install the top-level project dependencies.
39
+
40
+ ```sh
41
+ $ bundle install
42
+ ```
43
+
44
+ 4. Install the Pub/Sub dependencies.
45
+
46
+ ```sh
47
+ $ cd google-cloud-pubsub/
48
+ $ bundle install
49
+ ```
50
+
51
+ ## Console
52
+
53
+ In order to run code interactively, you can automatically load
54
+ google-cloud-pubsub and its dependencies in IRB. This requires that your
55
+ developer environment has already been configured by following the steps
56
+ described in the {file:AUTHENTICATION.md Authentication Guide}. An IRB console
57
+ can be created with:
58
+
59
+ ```sh
60
+ $ cd google-cloud-pubsub/
61
+ $ bundle exec rake console
62
+ ```
63
+
64
+ ## Pub/Sub Tests
65
+
66
+ Tests are very important part of google-cloud-pubsub. All contributions
67
+ should include tests that ensure the contributed code behaves as expected.
68
+
69
+ To run the unit tests, documentation tests, and code style checks together for a
70
+ package:
71
+
72
+ ``` sh
73
+ $ cd google-cloud-pubsub/
74
+ $ bundle exec rake ci
75
+ ```
76
+
77
+ To run the command above, plus all acceptance tests, use `rake ci:acceptance` or
78
+ its handy alias, `rake ci:a`.
79
+
80
+ ### Pub/Sub Unit Tests
81
+
82
+
83
+ The project uses the [minitest](https://github.com/seattlerb/minitest) library,
84
+ including [specs](https://github.com/seattlerb/minitest#specs),
85
+ [mocks](https://github.com/seattlerb/minitest#mocks) and
86
+ [minitest-autotest](https://github.com/seattlerb/minitest-autotest).
87
+
88
+ To run the Pub/Sub unit tests:
89
+
90
+ ``` sh
91
+ $ cd google-cloud-pubsub/
92
+ $ bundle exec rake test
93
+ ```
94
+
95
+ ### Pub/Sub Documentation Tests
96
+
97
+ The project tests the code examples in the gem's
98
+ [YARD](https://github.com/lsegal/yard)-based documentation.
99
+
100
+ The example testing functions in a way that is very similar to unit testing, and
101
+ in fact the library providing it,
102
+ [yard-doctest](https://github.com/p0deje/yard-doctest), is based on the
103
+ project's unit test library, [minitest](https://github.com/seattlerb/minitest).
104
+
105
+ To run the Pub/Sub documentation tests:
106
+
107
+ ``` sh
108
+ $ cd google-cloud-pubsub/
109
+ $ bundle exec rake doctest
110
+ ```
111
+
112
+ If you add, remove or modify documentation examples when working on a pull
113
+ request, you may need to update the setup for the tests. The stubs and mocks
114
+ required to run the tests are located in `support/doctest_helper.rb`. Please
115
+ note that much of the setup is matched by the title of the
116
+ [`@example`](http://www.rubydoc.info/gems/yard/file/docs/Tags.md#example) tag.
117
+ If you alter an example's title, you may encounter breaking tests.
118
+
119
+ ### Pub/Sub Acceptance Tests
120
+
121
+ The Pub/Sub acceptance tests interact with the live service API. Follow the
122
+ instructions in the {file:AUTHENTICATION.md Authentication Guide} for enabling
123
+ the Pub/Sub API. Occasionally, some API features may not yet be generally
124
+ available, making it difficult for some contributors to successfully run the
125
+ entire acceptance test suite. However, please ensure that you do successfully
126
+ run acceptance tests for any code areas covered by your pull request.
127
+
128
+ To run the acceptance tests, first create and configure a project in the Google
129
+ Developers Console, as described in the {file:AUTHENTICATION.md Authentication Guide}. Be sure to download the JSON KEY file. Make note of the PROJECT_ID and
130
+ the KEYFILE location on your system.
131
+
132
+ Before you can run the Pub/Sub acceptance tests, you must first create indexes
133
+ used in the tests.
134
+
135
+ #### Running the Pub/Sub acceptance tests
136
+
137
+ To run the Pub/Sub acceptance tests:
138
+
139
+ ``` sh
140
+ $ cd google-cloud-pubsub/
141
+ $ bundle exec rake acceptance[\\{my-project-id},\\{/path/to/keyfile.json}]
142
+ ```
143
+
144
+ Or, if you prefer you can store the values in the `GCLOUD_TEST_PROJECT` and
145
+ `GCLOUD_TEST_KEYFILE` environment variables:
146
+
147
+ ``` sh
148
+ $ cd google-cloud-pubsub/
149
+ $ export GCLOUD_TEST_PROJECT=\\{my-project-id}
150
+ $ export GCLOUD_TEST_KEYFILE=\\{/path/to/keyfile.json}
151
+ $ bundle exec rake acceptance
152
+ ```
153
+
154
+ If you want to use a different project and credentials for acceptance tests, you
155
+ can use the more specific `PUBSUB_TEST_PROJECT` and `PUBSUB_TEST_KEYFILE`
156
+ environment variables:
157
+
158
+ ``` sh
159
+ $ cd google-cloud-pubsub/
160
+ $ export PUBSUB_TEST_PROJECT=\\{my-project-id}
161
+ $ export PUBSUB_TEST_KEYFILE=\\{/path/to/keyfile.json}
162
+ $ bundle exec rake acceptance
163
+ ```
164
+
165
+ ## Coding Style
166
+
167
+ Please follow the established coding style in the library. The style is is
168
+ largely based on [The Ruby Style
169
+ Guide](https://github.com/bbatsov/ruby-style-guide) with a few exceptions based
170
+ on seattle-style:
171
+
172
+ * Avoid parenthesis when possible, including in method definitions.
173
+ * Always use double quotes strings. ([Option
174
+ B](https://github.com/bbatsov/ruby-style-guide#strings))
175
+
176
+ You can check your code against these rules by running Rubocop like so:
177
+
178
+ ```sh
179
+ $ cd google-cloud-pubsub/
180
+ $ bundle exec rake rubocop
181
+ ```
182
+
183
+ ## Code of Conduct
184
+
185
+ Please note that this project is released with a Contributor Code of Conduct. By
186
+ participating in this project you agree to abide by its terms. See
187
+ {file:CODE_OF_CONDUCT.md Code of Conduct} for more information.
data/EMULATOR.md ADDED
@@ -0,0 +1,37 @@
1
+ # Google Cloud Pub/Sub Emulator
2
+
3
+ To develop and test your application locally, you can use the [Google Cloud
4
+ Pub/Sub Emulator](https://cloud.google.com/pubsub/emulator), which provides
5
+ [local emulation](https://cloud.google.com/sdk/gcloud/reference/beta/emulators/)
6
+ of the production Google Cloud Pub/Sub environment. You can start the Google
7
+ Cloud Pub/Sub emulator using the `gcloud` command-line tool.
8
+
9
+ To configure your ruby code to use the emulator, set the `PUBSUB_EMULATOR_HOST`
10
+ environment variable to the host and port where the emulator is running. The
11
+ value can be set as an environment variable in the shell running the ruby code,
12
+ or can be set directly in the ruby code as shown below.
13
+
14
+ ```ruby
15
+ require "google/cloud/pubsub"
16
+
17
+ # Make Pub/Sub use the emulator
18
+ ENV["PUBSUB_EMULATOR_HOST"] = "localhost:8918"
19
+
20
+ pubsub = Google::Cloud::PubSub.new project_id:"emulator-project-id"
21
+
22
+ # Get a topic in the current project
23
+ my_topic = pubsub.new_topic "my-topic"
24
+ my_topic.name #=> "projects/emulator-project-id/topics/my-topic"
25
+ ```
26
+
27
+ Or by providing the `emulator_host` argument:
28
+
29
+ ```ruby
30
+ require "google/cloud/pubsub"
31
+
32
+ pubsub = Google::Cloud::PubSub.new emulator_host: "localhost:8918"
33
+
34
+ # Get a topic in the current project
35
+ my_topic = pubsub.new_topic "my-topic"
36
+ my_topic.name #=> "projects/emulator-project-id/topics/my-topic"
37
+ ```
data/LICENSE CHANGED
@@ -1,6 +1,6 @@
1
1
  Apache License
2
2
  Version 2.0, January 2004
3
- http://www.apache.org/licenses/
3
+ https://www.apache.org/licenses/
4
4
 
5
5
  TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
6
 
@@ -192,7 +192,7 @@
192
192
  you may not use this file except in compliance with the License.
193
193
  You may obtain a copy of the License at
194
194
 
195
- http://www.apache.org/licenses/LICENSE-2.0
195
+ https://www.apache.org/licenses/LICENSE-2.0
196
196
 
197
197
  Unless required by applicable law or agreed to in writing, software
198
198
  distributed under the License is distributed on an "AS IS" BASIS,
data/LOGGING.md ADDED
@@ -0,0 +1,32 @@
1
+ # Enabling gRPC Logging
2
+
3
+ To enable logging for this library, set the logger for the underlying
4
+ [gRPC](https://github.com/grpc/grpc/tree/master/src/ruby) library. The logger
5
+ that you set may be a Ruby stdlib
6
+ [`Logger`](https://ruby-doc.org/stdlib/libdoc/logger/rdoc/Logger.html) as
7
+ shown below, or a
8
+ [`Google::Cloud::Logging::Logger`](https://googleapis.dev/ruby/google-cloud-logging/latest)
9
+ that will write logs to [Stackdriver
10
+ Logging](https://cloud.google.com/logging/). See
11
+ [grpc/logconfig.rb](https://github.com/grpc/grpc/blob/master/src/ruby/lib/grpc/logconfig.rb)
12
+ and the gRPC
13
+ [spec_helper.rb](https://github.com/grpc/grpc/blob/master/src/ruby/spec/spec_helper.rb)
14
+ for additional information.
15
+
16
+ Configuring a Ruby stdlib logger:
17
+
18
+ ```ruby
19
+ require "logger"
20
+
21
+ module MyLogger
22
+ LOGGER = Logger.new $stderr, level: Logger::WARN
23
+ def logger
24
+ LOGGER
25
+ end
26
+ end
27
+
28
+ # Define a gRPC module-level logger method before grpc/logconfig.rb loads.
29
+ module GRPC
30
+ extend MyLogger
31
+ end
32
+ ```
data/OVERVIEW.md ADDED
@@ -0,0 +1,528 @@
1
+ # Google Cloud Pub/Sub
2
+
3
+ Google Cloud Pub/Sub is designed to provide reliable, many-to-many, asynchronous
4
+ messaging between applications. Publisher applications can send messages to a
5
+ "topic" and other applications can subscribe to that topic to receive the
6
+ messages. By decoupling senders and receivers, Google Cloud Pub/Sub allows
7
+ developers to communicate between independently written applications.
8
+
9
+ The goal of google-cloud is to provide an API that is comfortable to Rubyists.
10
+ Your authentication credentials are detected automatically in Google Cloud
11
+ Platform (GCP), including Google Compute Engine (GCE), Google Kubernetes Engine
12
+ (GKE), Google App Engine (GAE), Google Cloud Functions (GCF) and Cloud Run. In
13
+ other environments you can configure authentication easily, either directly in
14
+ your code or via environment variables. Read more about the options for
15
+ connecting in the {file:AUTHENTICATION.md Authentication Guide}.
16
+
17
+ ```ruby
18
+ require "google/cloud/pubsub"
19
+
20
+ pubsub = Google::Cloud::PubSub.new
21
+
22
+ topic = pubsub.topic "my-topic"
23
+ topic.publish "task completed"
24
+ ```
25
+
26
+ To learn more about Pub/Sub, read the [Google Cloud Pub/Sub Overview
27
+ ](https://cloud.google.com/pubsub/overview).
28
+
29
+ ## Retrieving Topics
30
+
31
+ A Topic is a named resource to which messages are sent by publishers. A Topic is
32
+ found by its name. (See {Google::Cloud::PubSub::Project#topic Project#topic})
33
+
34
+ ```ruby
35
+ require "google/cloud/pubsub"
36
+
37
+ pubsub = Google::Cloud::PubSub.new
38
+ topic = pubsub.topic "my-topic"
39
+ ```
40
+
41
+ ## Creating a Topic
42
+
43
+ A Topic is created from a Project. (See
44
+ {Google::Cloud::PubSub::Project#create_topic Project#create_topic})
45
+
46
+ ```ruby
47
+ require "google/cloud/pubsub"
48
+
49
+ pubsub = Google::Cloud::PubSub.new
50
+ topic = pubsub.create_topic "my-topic"
51
+ ```
52
+
53
+ ## Retrieving Subscriptions
54
+
55
+ A Subscription is a named resource representing the stream of messages from a
56
+ single, specific Topic, to be delivered to the subscribing application. A
57
+ Subscription is found by its name. (See
58
+ {Google::Cloud::PubSub::Topic#subscription Topic#subscription})
59
+
60
+ ```ruby
61
+ require "google/cloud/pubsub"
62
+
63
+ pubsub = Google::Cloud::PubSub.new
64
+
65
+ topic = pubsub.topic "my-topic"
66
+ subscription = topic.subscription "my-topic-subscription"
67
+ puts subscription.name
68
+ ```
69
+
70
+ ## Creating a Subscription
71
+
72
+ A Subscription is created from a Topic. (See
73
+ {Google::Cloud::PubSub::Topic#subscribe Topic#subscribe})
74
+
75
+ ```ruby
76
+ require "google/cloud/pubsub"
77
+
78
+ pubsub = Google::Cloud::PubSub.new
79
+
80
+ topic = pubsub.topic "my-topic"
81
+ sub = topic.subscribe "my-topic-sub"
82
+ puts sub.name # => "my-topic-sub"
83
+ ```
84
+
85
+ The subscription can be created that specifies the number of seconds to wait to
86
+ be acknowledged as well as an endpoint URL to push the messages to:
87
+
88
+ ```ruby
89
+ require "google/cloud/pubsub"
90
+
91
+ pubsub = Google::Cloud::PubSub.new
92
+
93
+ topic = pubsub.topic "my-topic"
94
+ sub = topic.subscribe "my-topic-sub",
95
+ deadline: 120,
96
+ endpoint: "https://example.com/push"
97
+ ```
98
+
99
+ ## Publishing Messages
100
+
101
+ Messages are published to a topic. Any message published to a topic without a
102
+ subscription will be lost. Ensure the topic has a subscription before
103
+ publishing. (See {Google::Cloud::PubSub::Topic#publish Topic#publish})
104
+
105
+ ```ruby
106
+ require "google/cloud/pubsub"
107
+
108
+ pubsub = Google::Cloud::PubSub.new
109
+
110
+ topic = pubsub.topic "my-topic"
111
+ msg = topic.publish "task completed"
112
+ ```
113
+
114
+ Messages can also be published with attributes:
115
+
116
+ ```ruby
117
+ require "google/cloud/pubsub"
118
+
119
+ pubsub = Google::Cloud::PubSub.new
120
+
121
+ topic = pubsub.topic "my-topic"
122
+ msg = topic.publish "task completed",
123
+ foo: :bar,
124
+ this: :that
125
+ ```
126
+
127
+ Messages can also be published in batches asynchronously using `publish_async`.
128
+ (See {Google::Cloud::PubSub::Topic#publish_async Topic#publish_async} and
129
+ {Google::Cloud::PubSub::AsyncPublisher AsyncPublisher})
130
+
131
+ ```ruby
132
+ require "google/cloud/pubsub"
133
+
134
+ pubsub = Google::Cloud::PubSub.new
135
+
136
+ topic = pubsub.topic "my-topic"
137
+ topic.publish_async "task completed" do |result|
138
+ if result.succeeded?
139
+ log_publish_success result.data
140
+ else
141
+ log_publish_failure result.data, result.error
142
+ end
143
+ end
144
+
145
+ topic.async_publisher.stop!
146
+ ```
147
+
148
+ Or multiple messages can be published in batches at the same time by passing a
149
+ block to `publish`. (See {Google::Cloud::PubSub::BatchPublisher BatchPublisher})
150
+
151
+ ```ruby
152
+ require "google/cloud/pubsub"
153
+
154
+ pubsub = Google::Cloud::PubSub.new
155
+
156
+ topic = pubsub.topic "my-topic"
157
+ msgs = topic.publish do |batch|
158
+ batch.publish "task 1 completed", foo: :bar
159
+ batch.publish "task 2 completed", foo: :baz
160
+ batch.publish "task 3 completed", foo: :bif
161
+ end
162
+ ```
163
+
164
+ ## Receiving Messages
165
+
166
+ Messages can be streamed from a subscription with a subscriber object that is
167
+ created using `listen`. (See {Google::Cloud::PubSub::Subscription#listen
168
+ Subscription#listen} and {Google::Cloud::PubSub::Subscriber Subscriber})
169
+
170
+ ```ruby
171
+ require "google/cloud/pubsub"
172
+
173
+ pubsub = Google::Cloud::PubSub.new
174
+
175
+ sub = pubsub.subscription "my-topic-sub"
176
+
177
+ # Create a subscriber to listen for available messages.
178
+ # By default, this block will be called on 8 concurrent threads
179
+ # but this can be tuned with the `threads` option.
180
+ # The `streams` and `inventory` parameters allow further tuning.
181
+ subscriber = sub.listen threads: { callback: 16 } do |received_message|
182
+ # process message
183
+ puts "Data: #{received_message.message.data}, published at #{received_message.message.published_at}"
184
+ received_message.acknowledge!
185
+ end
186
+
187
+ # Handle exceptions from listener
188
+ subscriber.on_error do |exception|
189
+ puts "Exception: #{exception.class} #{exception.message}"
190
+ end
191
+
192
+ # Gracefully shut down the subscriber on program exit, blocking until
193
+ # all received messages have been processed or 10 seconds have passed
194
+ at_exit do
195
+ subscriber.stop!(10)
196
+ end
197
+
198
+ # Start background threads that will call the block passed to listen.
199
+ subscriber.start
200
+
201
+ # Block, letting processing threads continue in the background
202
+ sleep
203
+ ```
204
+
205
+ Messages also can be pulled directly in a one-time operation. (See
206
+ {Google::Cloud::PubSub::Subscription#pull Subscription#pull})
207
+
208
+ The `immediate: false` option is recommended to avoid adverse impacts on the
209
+ performance of pull operations.
210
+
211
+ ```ruby
212
+ require "google/cloud/pubsub"
213
+
214
+ pubsub = Google::Cloud::PubSub.new
215
+
216
+ sub = pubsub.subscription "my-topic-sub"
217
+ received_messages = sub.pull immediate: false
218
+ ```
219
+
220
+ A maximum number of messages to pull can be specified:
221
+
222
+ ```ruby
223
+ require "google/cloud/pubsub"
224
+
225
+ pubsub = Google::Cloud::PubSub.new
226
+
227
+ sub = pubsub.subscription "my-topic-sub"
228
+ received_messages = sub.pull immediate: false, max: 10
229
+ ```
230
+
231
+ ## Acknowledging a Message
232
+
233
+ Messages that are received can be acknowledged in Pub/Sub, marking the message
234
+ to be removed so it cannot be pulled again.
235
+
236
+ A Message that can be acknowledged is called a ReceivedMessage. ReceivedMessages
237
+ can be acknowledged one at a time: (See
238
+ {Google::Cloud::PubSub::ReceivedMessage#acknowledge!
239
+ ReceivedMessage#acknowledge!})
240
+
241
+ ```ruby
242
+ require "google/cloud/pubsub"
243
+
244
+ pubsub = Google::Cloud::PubSub.new
245
+
246
+ sub = pubsub.subscription "my-topic-sub"
247
+
248
+ subscriber = sub.listen do |received_message|
249
+ # process message
250
+ received_message.acknowledge!
251
+ end
252
+
253
+ # Start background threads that will call the block passed to listen.
254
+ subscriber.start
255
+
256
+ # Shut down the subscriber when ready to stop receiving messages.
257
+ subscriber.stop!
258
+ ```
259
+
260
+ Or, multiple messages can be acknowledged in a single API call: (See
261
+ {Google::Cloud::PubSub::Subscription#acknowledge Subscription#acknowledge})
262
+
263
+ ```ruby
264
+ require "google/cloud/pubsub"
265
+
266
+ pubsub = Google::Cloud::PubSub.new
267
+
268
+ sub = pubsub.subscription "my-topic-sub"
269
+ received_messages = sub.pull immediate: false
270
+ sub.acknowledge received_messages
271
+ ```
272
+
273
+ ## Modifying a Deadline
274
+
275
+ A message must be acknowledged after it is pulled, or Pub/Sub will mark the
276
+ message for redelivery. The message acknowledgement deadline can delayed if more
277
+ time is needed. This will allow more time to process the message before the
278
+ message is marked for redelivery. (See
279
+ {Google::Cloud::PubSub::ReceivedMessage#modify_ack_deadline!
280
+ ReceivedMessage#modify_ack_deadline!})
281
+
282
+ ```ruby
283
+ require "google/cloud/pubsub"
284
+
285
+ pubsub = Google::Cloud::PubSub.new
286
+
287
+ sub = pubsub.subscription "my-topic-sub"
288
+ subscriber = sub.listen do |received_message|
289
+ puts received_message.message.data
290
+
291
+ # Delay for 2 minutes
292
+ received_message.modify_ack_deadline! 120
293
+ end
294
+
295
+ # Start background threads that will call the block passed to listen.
296
+ subscriber.start
297
+
298
+ # Shut down the subscriber when ready to stop receiving messages.
299
+ subscriber.stop!
300
+ ```
301
+
302
+ The message can also be made available for immediate redelivery:
303
+
304
+ ```ruby
305
+ require "google/cloud/pubsub"
306
+
307
+ pubsub = Google::Cloud::PubSub.new
308
+
309
+ sub = pubsub.subscription "my-topic-sub"
310
+ subscriber = sub.listen do |received_message|
311
+ puts received_message.message.data
312
+
313
+ # Mark for redelivery
314
+ received_message.reject!
315
+ end
316
+
317
+ # Start background threads that will call the block passed to listen.
318
+ subscriber.start
319
+
320
+ # Shut down the subscriber when ready to stop receiving messages.
321
+ subscriber.stop!
322
+ ```
323
+
324
+ Multiple messages can be delayed or made available for immediate redelivery:
325
+ (See {Google::Cloud::PubSub::Subscription#modify_ack_deadline
326
+ Subscription#modify_ack_deadline})
327
+
328
+ ```ruby
329
+ require "google/cloud/pubsub"
330
+
331
+ pubsub = Google::Cloud::PubSub.new
332
+
333
+ sub = pubsub.subscription "my-topic-sub"
334
+ received_messages = sub.pull immediate: false
335
+ sub.modify_ack_deadline 120, received_messages
336
+ ```
337
+
338
+ ## Using Ordering Keys
339
+
340
+ Google Cloud Pub/Sub ordering keys provide the ability to ensure related
341
+ messages are sent to subscribers in the order in which they were published.
342
+ Messages can be tagged with an ordering key, a string that identifies related
343
+ messages for which publish order should be respected. The service guarantees
344
+ that, for a given ordering key and publisher, messages are sent to subscribers
345
+ in the order in which they were published. Ordering does not require sacrificing
346
+ high throughput or scalability, as the service automatically distributes
347
+ messages for different ordering keys across subscribers.
348
+
349
+ Note: At the time of this release, ordering keys are not yet publicly enabled
350
+ and requires special project enablements.
351
+
352
+ ### Publishing Ordered Messages
353
+
354
+ To use ordering keys when publishing messages, a call to
355
+ {Google::Cloud::PubSub::Topic#enable_message_ordering!
356
+ Topic#enable_message_ordering!} must be made and the `ordering_key` argument
357
+ must be provided when calling {Google::Cloud::PubSub::Topic#publish_async
358
+ Topic#publish_async}.
359
+
360
+ ```ruby
361
+ require "google/cloud/pubsub"
362
+
363
+ pubsub = Google::Cloud::PubSub.new
364
+
365
+ topic = pubsub.topic "my-ordered-topic"
366
+
367
+ # Ensure that message ordering is enabled.
368
+ topic.enable_message_ordering!
369
+
370
+ # Publish an ordered message with an ordering key.
371
+ topic.publish_async "task completed",
372
+ ordering_key: "task-key"
373
+
374
+ # Shut down the publisher when ready to stop publishing messages.
375
+ topic.async_publisher.stop!
376
+ ```
377
+
378
+ ### Handling errors with Ordered Keys
379
+
380
+ Ordered messages that fail to publish to the Pub/Sub API due to error will put
381
+ the `ordering_key` in a failed state, and future calls to
382
+ {Google::Cloud::PubSub::Topic#publish_async Topic#publish_async} with the
383
+ `ordering_key` will raise {Google::Cloud::PubSub::OrderingKeyError
384
+ OrderingKeyError}. To allow future messages with the `ordering_key` to be
385
+ published, the `ordering_key` must be passed to
386
+ {Google::Cloud::PubSub::Topic#resume_publish Topic#resume_publish}.
387
+
388
+ ### Receiving Ordered Messages
389
+
390
+ To use ordering keys when subscribing to messages, the subscription must be
391
+ created with message ordering enabled (See
392
+ {Google::Cloud::PubSub::Topic#subscribe Topic#subscribe} and
393
+ {Google::Cloud::PubSub::Subscription#message_ordering?
394
+ Subscription#message_ordering?}) before calling
395
+ {Google::Cloud::PubSub::Subscription#listen Subscription#listen}. When enabled,
396
+ the subscriber will deliver messages with the same `ordering_key` in the order
397
+ they were published.
398
+
399
+ ```ruby
400
+ require "google/cloud/pubsub"
401
+
402
+ pubsub = Google::Cloud::PubSub.new
403
+
404
+ sub = pubsub.subscription "my-ordered-topic-sub"
405
+ sub.message_ordering? #=> true
406
+
407
+ subscriber = sub.listen do |received_message|
408
+ # Messsages with the same ordering_key are received
409
+ # in the order in which they were published.
410
+ received_message.acknowledge!
411
+ end
412
+
413
+ # Start background threads that will call block passed to listen.
414
+ subscriber.start
415
+
416
+ # Shut down the subscriber when ready to stop receiving messages.
417
+ subscriber.stop!
418
+ ```
419
+
420
+ ## Minimizing API calls before receiving and acknowledging messages
421
+
422
+ A subscription object can be created without making any API calls by providing
423
+ the `skip_lookup` argument to {Google::Cloud::PubSub::Project#subscription
424
+ Project#subscription} or {Google::Cloud::PubSub::Topic#subscription
425
+ Topic#subscription}. A subscriber object can also be created without an API call
426
+ by providing the `deadline` optional argument to
427
+ {Google::Cloud::PubSub::Subscription#listen Subscription#listen}:
428
+
429
+ ```ruby
430
+ require "google/cloud/pubsub"
431
+
432
+ pubsub = Google::Cloud::PubSub.new
433
+
434
+ # No API call is made to retrieve the subscription resource.
435
+ sub = pubsub.subscription "my-topic-sub", skip_lookup: true
436
+
437
+ # No API call is made to retrieve the subscription deadline.
438
+ subscriber = sub.listen deadline: 60 do |received_message|
439
+ # process message
440
+ received_message.acknowledge!
441
+ end
442
+
443
+ # Start background threads that will call block passed to listen.
444
+ subscriber.start
445
+
446
+ # Shut down the subscriber when ready to stop receiving messages.
447
+ subscriber.stop!
448
+ ```
449
+
450
+ Skipping API calls may be used to avoid `Google::Cloud::PermissionDeniedError`
451
+ if your account has limited access to the Pub/Sub API. In particular, the role
452
+ `roles/pubsub.subscriber` does not have the permission
453
+ `pubsub.subscriptions.get`, which is required to retrieve a subscription
454
+ resource. See [Access Control -
455
+ Roles](https://cloud.google.com/pubsub/docs/access-control#tbl_roles) for the
456
+ complete list of Pub/Sub roles and permissions.
457
+
458
+ ## Creating a snapshot and using seek
459
+
460
+ You can create a snapshot to retain the existing backlog on a subscription. The
461
+ snapshot will hold the messages in the subscription's backlog that are
462
+ unacknowledged upon the successful completion of the `create_snapshot`
463
+ operation.
464
+
465
+ Later, you can use `seek` to reset the subscription's backlog to the snapshot.
466
+
467
+ (See {Google::Cloud::PubSub::Subscription#create_snapshot
468
+ Subscription#create_snapshot} and {Google::Cloud::PubSub::Subscription#seek
469
+ Subscription#seek})
470
+
471
+ ```ruby
472
+ require "google/cloud/pubsub"
473
+
474
+ pubsub = Google::Cloud::PubSub.new
475
+
476
+ sub = pubsub.subscription "my-topic-sub"
477
+
478
+ snapshot = sub.create_snapshot
479
+
480
+ received_messages = sub.pull immediate: false
481
+ sub.acknowledge received_messages
482
+
483
+ sub.seek snapshot
484
+ ```
485
+
486
+ ## Working Across Projects
487
+
488
+ All calls to the Pub/Sub service use the same project and credentials provided
489
+ to the {Google::Cloud::PubSub.new PubSub.new} method. However, it is common to
490
+ reference topics or subscriptions in other projects, which can be achieved by
491
+ using the `project` option. The main credentials must have permissions to the
492
+ topics and subscriptions in other projects.
493
+
494
+ ```ruby
495
+ require "google/cloud/pubsub"
496
+
497
+ pubsub = Google::Cloud::PubSub.new # my-project
498
+
499
+ # Get a topic in the current project
500
+ my_topic = pubsub.topic "my-topic"
501
+ my_topic.name #=> "projects/my-project/topics/my-topic"
502
+ # Get a topic in another project
503
+ other_topic = pubsub.topic "other-topic", project: "other-project-id"
504
+ other_topic.name #=> "projects/other-project-id/topics/other-topic"
505
+ ```
506
+
507
+ It is possible to create a subscription in the current project that pulls
508
+ from a topic in another project:
509
+
510
+ ```ruby
511
+ require "google/cloud/pubsub"
512
+
513
+ pubsub = Google::Cloud::PubSub.new # my-project
514
+
515
+ # Get a topic in another project
516
+ topic = pubsub.topic "other-topic", project: "other-project-id"
517
+ # Create a subscription in the current project that pulls from
518
+ # the topic in another project
519
+ sub = topic.subscribe "my-sub"
520
+ sub.name #=> "projects/my-project/subscriptions/my-sub"
521
+ sub.topic.name #=> "projects/other-project-id/topics/other-topic"
522
+ ```
523
+
524
+ ## Additional information
525
+
526
+ Google Cloud Pub/Sub can be configured to use an emulator or to enable gRPC's
527
+ logging. To learn more, see the {file:EMULATOR.md Emulator guide}} and
528
+ {file:LOGGING.md Logging guide}.