google-cloud-pubsub 1.7.1 → 2.1.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (36) hide show
  1. checksums.yaml +4 -4
  2. data/AUTHENTICATION.md +2 -1
  3. data/CHANGELOG.md +53 -0
  4. data/OVERVIEW.md +26 -58
  5. data/lib/google-cloud-pubsub.rb +14 -13
  6. data/lib/google/cloud/pubsub.rb +15 -18
  7. data/lib/google/cloud/pubsub/async_publisher.rb +2 -3
  8. data/lib/google/cloud/pubsub/credentials.rb +2 -2
  9. data/lib/google/cloud/pubsub/message.rb +1 -1
  10. data/lib/google/cloud/pubsub/project.rb +1 -1
  11. data/lib/google/cloud/pubsub/received_message.rb +4 -4
  12. data/lib/google/cloud/pubsub/service.rb +110 -260
  13. data/lib/google/cloud/pubsub/subscriber.rb +3 -3
  14. data/lib/google/cloud/pubsub/subscriber/stream.rb +3 -2
  15. data/lib/google/cloud/pubsub/subscription.rb +86 -11
  16. data/lib/google/cloud/pubsub/subscription/push_config.rb +55 -31
  17. data/lib/google/cloud/pubsub/topic.rb +52 -14
  18. data/lib/google/cloud/pubsub/version.rb +1 -1
  19. metadata +8 -79
  20. data/lib/google/cloud/pubsub/v1.rb +0 -17
  21. data/lib/google/cloud/pubsub/v1/credentials.rb +0 -41
  22. data/lib/google/cloud/pubsub/v1/doc/google/iam/v1/iam_policy.rb +0 -21
  23. data/lib/google/cloud/pubsub/v1/doc/google/iam/v1/options.rb +0 -21
  24. data/lib/google/cloud/pubsub/v1/doc/google/iam/v1/policy.rb +0 -21
  25. data/lib/google/cloud/pubsub/v1/doc/google/protobuf/duration.rb +0 -91
  26. data/lib/google/cloud/pubsub/v1/doc/google/protobuf/empty.rb +0 -29
  27. data/lib/google/cloud/pubsub/v1/doc/google/protobuf/field_mask.rb +0 -222
  28. data/lib/google/cloud/pubsub/v1/doc/google/protobuf/timestamp.rb +0 -113
  29. data/lib/google/cloud/pubsub/v1/doc/google/pubsub/v1/pubsub.rb +0 -807
  30. data/lib/google/cloud/pubsub/v1/doc/google/type/expr.rb +0 -19
  31. data/lib/google/cloud/pubsub/v1/publisher_client.rb +0 -928
  32. data/lib/google/cloud/pubsub/v1/publisher_client_config.json +0 -120
  33. data/lib/google/cloud/pubsub/v1/subscriber_client.rb +0 -1463
  34. data/lib/google/cloud/pubsub/v1/subscriber_client_config.json +0 -153
  35. data/lib/google/pubsub/v1/pubsub_pb.rb +0 -266
  36. data/lib/google/pubsub/v1/pubsub_services_pb.rb +0 -215
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 25d3602668077c7dee90a868dcc01353710ac3001ece8ebe92b7acc6c7bc1bb3
4
- data.tar.gz: 2119781ea548d6e87d201bdb759643c7ac78b7ffab0a4ee7505f731c3c812670
3
+ metadata.gz: 78dd56b9b8fa38d07a2d712afbf6b2d32f20ab11c292018ca15e23c838102e31
4
+ data.tar.gz: e60a9d91b0f0ac99ac8207f3cbfa14d10f48f3fa831461494c3ca05dda804bc6
5
5
  SHA512:
6
- metadata.gz: ed1de151bbd790feb886162024dbe20e75825c7313de7f0349468d8adbdf0400ce0616f53925315a1a2558b2683fd9ac8b478b842b2008dae67990676c163a4b
7
- data.tar.gz: 2247b98b5f04c936bd908c7a517d396e03af0f07030e7939282f750d8d2b9f569f428961c793f929d5595da60445a55948fe17864a1f25f1ab7d1746b06be9c0
6
+ metadata.gz: 823ff96d9886348199ce5e46335e8bd17f8f0fb316fde14336d9c464c49c13978daab8bd6b15e84c44422d3494b877207535bbc6b15f57c5caff1fb7d4c9f7f0
7
+ data.tar.gz: 04cbd6adba655294efefc38bbf18c48bada0ad7c7f61ab482c8fb9c460115e5a2acf1a936a3e20e39cdac83de004fb404f23032f5a1cc29d2ac9b85d94d6528d
@@ -76,7 +76,8 @@ The environment variables that google-cloud-pubsub checks for project ID are:
76
76
  1. `PUBSUB_PROJECT`
77
77
  2. `GOOGLE_CLOUD_PROJECT`
78
78
 
79
- The environment variables that google-cloud-pubsub checks for credentials are configured on {Google::Cloud::PubSub::V1::Credentials}:
79
+ The environment variables that google-cloud-pubsub checks for credentials
80
+ are configured on {Google::Cloud::PubSub::Credentials}:
80
81
 
81
82
  1. `PUBSUB_CREDENTIALS` - Path to JSON file, or JSON contents
82
83
  2. `PUBSUB_KEYFILE` - Path to JSON file, or JSON contents
@@ -1,5 +1,58 @@
1
1
  # Release History
2
2
 
3
+ ### 2.1.0 / 2020-09-17
4
+
5
+ #### Features
6
+
7
+ * quota_project can be set via library configuration ([#7630](https://www.github.com/googleapis/google-cloud-ruby/issues/7630))
8
+ * Add push_config (PushConfig) param to Topic#subscribe
9
+ * Make PushConfig constructor public.
10
+
11
+ #### Documentation
12
+
13
+ * Update sample code for on_error, at_exit, and concurrency tuning
14
+
15
+ ### 2.0.0 / 2020-08-06
16
+
17
+ This is a major update that removes the "low-level" client interface code, and
18
+ instead adds the new `google-cloud-pubsub-v1` gem as a dependency.
19
+ The new dependency is a rewritten low-level client, produced by a next-
20
+ generation client code generator, with improved performance and stability.
21
+
22
+ This change should have no effect on the high-level interface that most users
23
+ will use. The one exception is that the (mostly undocumented) `client_config`
24
+ argument, for adjusting low-level parameters such as RPC retry settings on
25
+ client objects, has been removed. If you need to adjust these parameters, use
26
+ the configuration interface in `google-cloud-pubsub-v1`.
27
+
28
+ Substantial changes have been made in the low-level interfaces, however. If you
29
+ are using the low-level classes under the `Google::Cloud::PubSub::V1` module,
30
+ please review the docs for the new `google-cloud-pubsub-v1` gem. In
31
+ particular:
32
+
33
+ * Some classes have been renamed, notably the client classes themselves.
34
+ * The client constructor takes a configuration block instead of configuration
35
+ keyword arguments.
36
+ * All RPC method arguments are now keyword arguments.
37
+
38
+ ### 1.10.0 / 2020-07-23
39
+
40
+ #### Features
41
+
42
+ * Add Subscription#detach and #detached?
43
+
44
+ ### 1.9.0 / 2020-07-21
45
+
46
+ #### Features
47
+
48
+ * Add support for server-side flow control
49
+
50
+ ### 1.8.0 / 2020-06-29
51
+
52
+ #### Features
53
+
54
+ * Add Subscription#filter
55
+
3
56
  ### 1.7.1 / 2020-05-28
4
57
 
5
58
  #### Documentation
@@ -142,7 +142,7 @@ topic.publish_async "task completed" do |result|
142
142
  end
143
143
  end
144
144
 
145
- topic.async_publisher.stop.wait!
145
+ topic.async_publisher.stop!
146
146
  ```
147
147
 
148
148
  Or multiple messages can be published in batches at the same time by passing a
@@ -174,16 +174,32 @@ pubsub = Google::Cloud::PubSub.new
174
174
 
175
175
  sub = pubsub.subscription "my-topic-sub"
176
176
 
177
- subscriber = sub.listen do |received_message|
177
+ # Create a subscriber to listen for available messages.
178
+ # By default, this block will be called on 8 concurrent threads
179
+ # but this can be tuned with the `threads` option.
180
+ # The `streams` and `inventory` parameters allow further tuning.
181
+ subscriber = sub.listen threads: { callback: 16 } do |received_message|
178
182
  # process message
183
+ puts "Data: #{received_message.message.data}, published at #{received_message.message.published_at}"
179
184
  received_message.acknowledge!
180
185
  end
181
186
 
187
+ # Handle exceptions from listener
188
+ subscriber.on_error do |exception|
189
+ puts "Exception: #{exception.class} #{exception.message}"
190
+ end
191
+
192
+ # Gracefully shut down the subscriber on program exit, blocking until
193
+ # all received messages have been processed or 10 seconds have passed
194
+ at_exit do
195
+ subscriber.stop!(10)
196
+ end
197
+
182
198
  # Start background threads that will call the block passed to listen.
183
199
  subscriber.start
184
200
 
185
- # Shut down the subscriber when ready to stop receiving messages.
186
- subscriber.stop.wait!
201
+ # Block, letting processing threads continue in the background
202
+ sleep
187
203
  ```
188
204
 
189
205
  Messages also can be pulled directly in a one-time operation. (See
@@ -235,7 +251,7 @@ end
235
251
  subscriber.start
236
252
 
237
253
  # Shut down the subscriber when ready to stop receiving messages.
238
- subscriber.stop.wait!
254
+ subscriber.stop!
239
255
  ```
240
256
 
241
257
  Or, multiple messages can be acknowledged in a single API call: (See
@@ -277,7 +293,7 @@ end
277
293
  subscriber.start
278
294
 
279
295
  # Shut down the subscriber when ready to stop receiving messages.
280
- subscriber.stop.wait!
296
+ subscriber.stop!
281
297
  ```
282
298
 
283
299
  The message can also be made available for immediate redelivery:
@@ -299,7 +315,7 @@ end
299
315
  subscriber.start
300
316
 
301
317
  # Shut down the subscriber when ready to stop receiving messages.
302
- subscriber.stop.wait!
318
+ subscriber.stop!
303
319
  ```
304
320
 
305
321
  Multiple messages can be delayed or made available for immediate redelivery:
@@ -353,7 +369,7 @@ topic.publish_async "task completed",
353
369
  ordering_key: "task-key"
354
370
 
355
371
  # Shut down the publisher when ready to stop publishing messages.
356
- topic.async_publisher.stop.wait!
372
+ topic.async_publisher.stop!
357
373
  ```
358
374
 
359
375
  ### Handling errors with Ordered Keys
@@ -395,7 +411,7 @@ end
395
411
  subscriber.start
396
412
 
397
413
  # Shut down the subscriber when ready to stop receiving messages.
398
- subscriber.stop.wait!
414
+ subscriber.stop!
399
415
  ```
400
416
 
401
417
  ## Minimizing API calls before receiving and acknowledging messages
@@ -425,7 +441,7 @@ end
425
441
  subscriber.start
426
442
 
427
443
  # Shut down the subscriber when ready to stop receiving messages.
428
- subscriber.stop.wait!
444
+ subscriber.stop!
429
445
  ```
430
446
 
431
447
  Skipping API calls may be used to avoid `Google::Cloud::PermissionDeniedError`
@@ -464,54 +480,6 @@ sub.acknowledge received_messages
464
480
  sub.seek snapshot
465
481
  ```
466
482
 
467
- ## Listening for Messages
468
-
469
- A subscriber object can be created using `listen`, which streams messages from
470
- the backend and processes them as they are received. (See
471
- {Google::Cloud::PubSub::Subscription#listen Subscription#listen} and
472
- {Google::Cloud::PubSub::Subscriber Subscriber})
473
-
474
- ```ruby
475
- require "google/cloud/pubsub"
476
-
477
- pubsub = Google::Cloud::PubSub.new
478
-
479
- sub = pubsub.subscription "my-topic-sub"
480
-
481
- subscriber = sub.listen do |received_message|
482
- # process message
483
- received_message.acknowledge!
484
- end
485
-
486
- # Start background threads that will call the block passed to listen.
487
- subscriber.start
488
-
489
- # Shut down the subscriber when ready to stop receiving messages.
490
- subscriber.stop.wait!
491
- ```
492
-
493
- The subscriber object can be configured to control the number of concurrent
494
- streams to open, the number of received messages to be collected, and the number
495
- of threads each stream opens for concurrent calls made to handle the received
496
- messages.
497
-
498
- ```ruby
499
- require "google/cloud/pubsub"
500
-
501
- pubsub = Google::Cloud::PubSub.new
502
-
503
- sub = pubsub.subscription "my-topic-sub"
504
-
505
- subscriber = sub.listen threads: { callback: 16 } do |received_message|
506
- # store the message somewhere before acknowledging
507
- store_in_backend received_message.data # takes a few seconds
508
- received_message.acknowledge!
509
- end
510
-
511
- # Start background threads that will call the block passed to listen.
512
- subscriber.start
513
- ```
514
-
515
483
  ## Working Across Projects
516
484
 
517
485
  All calls to the Pub/Sub service use the same project and credentials provided
@@ -42,8 +42,6 @@ module Google
42
42
  #
43
43
  # * `https://www.googleapis.com/auth/pubsub`
44
44
  # @param [Integer] timeout Default timeout to use in requests. Optional.
45
- # @param [Hash] client_config A hash of values to override the default
46
- # behavior of the API client. Optional.
47
45
  #
48
46
  # @return [Google::Cloud::PubSub::Project]
49
47
  #
@@ -62,9 +60,9 @@ module Google
62
60
  # platform_scope = "https://www.googleapis.com/auth/cloud-platform"
63
61
  # pubsub = gcloud.pubsub scope: platform_scope
64
62
  #
65
- def pubsub scope: nil, timeout: nil, client_config: nil
63
+ def pubsub scope: nil, timeout: nil
66
64
  timeout ||= @timeout
67
- Google::Cloud.pubsub @project, @keyfile, scope: scope, timeout: timeout, client_config: client_config
65
+ Google::Cloud.pubsub @project, @keyfile, scope: scope, timeout: timeout
68
66
  end
69
67
 
70
68
  ##
@@ -90,8 +88,6 @@ module Google
90
88
  #
91
89
  # * `https://www.googleapis.com/auth/pubsub`
92
90
  # @param [Integer] timeout Default timeout to use in requests. Optional.
93
- # @param [Hash] client_config A hash of values to override the default
94
- # behavior of the API client. Optional.
95
91
  #
96
92
  # @return [Google::Cloud::PubSub::Project]
97
93
  #
@@ -103,11 +99,12 @@ module Google
103
99
  # topic = pubsub.topic "my-topic"
104
100
  # topic.publish "task completed"
105
101
  #
106
- def self.pubsub project_id = nil, credentials = nil, scope: nil,
107
- timeout: nil, client_config: nil
102
+ def self.pubsub project_id = nil,
103
+ credentials = nil,
104
+ scope: nil,
105
+ timeout: nil
108
106
  require "google/cloud/pubsub"
109
- Google::Cloud::PubSub.new project_id: project_id, credentials: credentials,
110
- scope: scope, timeout: timeout, client_config: client_config
107
+ Google::Cloud::PubSub.new project_id: project_id, credentials: credentials, scope: scope, timeout: timeout
111
108
  end
112
109
  end
113
110
  end
@@ -125,15 +122,19 @@ Google::Cloud.configure.add_config! :pubsub do |config|
125
122
  default_emulator = Google::Cloud::Config.deferred do
126
123
  ENV["PUBSUB_EMULATOR_HOST"]
127
124
  end
125
+ default_scopes = [
126
+ "https://www.googleapis.com/auth/cloud-platform",
127
+ "https://www.googleapis.com/auth/pubsub"
128
+ ]
128
129
 
129
130
  config.add_field! :project_id, default_project, match: String, allow_nil: true
130
131
  config.add_alias! :project, :project_id
131
132
  config.add_field! :credentials, default_creds, match: [String, Hash, Google::Auth::Credentials], allow_nil: true
132
133
  config.add_alias! :keyfile, :credentials
133
- config.add_field! :scope, nil, match: [String, Array]
134
+ config.add_field! :scope, default_scopes, match: [String, Array]
135
+ config.add_field! :quota_project, nil, match: String
134
136
  config.add_field! :timeout, nil, match: Integer
135
- config.add_field! :client_config, nil, match: Hash
136
137
  config.add_field! :emulator_host, default_emulator, match: String, allow_nil: true
137
138
  config.add_field! :on_error, nil, match: Proc
138
- config.add_field! :endpoint, nil, match: String
139
+ config.add_field! :endpoint, "pubsub.googleapis.com", match: String
139
140
  end
@@ -57,8 +57,6 @@ module Google
57
57
  #
58
58
  # * `https://www.googleapis.com/auth/pubsub`
59
59
  # @param [Integer] timeout Default timeout to use in requests. Optional.
60
- # @param [Hash] client_config A hash of values to override the default
61
- # behavior of the API client. Optional.
62
60
  # @param [String] endpoint Override of the endpoint host name. Optional.
63
61
  # If the param is nil, uses the default endpoint.
64
62
  # @param [String] emulator_host Pub/Sub emulator host. Optional.
@@ -77,12 +75,17 @@ module Google
77
75
  # topic = pubsub.topic "my-topic"
78
76
  # topic.publish "task completed"
79
77
  #
80
- def self.new project_id: nil, credentials: nil, scope: nil, timeout: nil, client_config: nil, endpoint: nil,
81
- emulator_host: nil, project: nil, keyfile: nil
78
+ def self.new project_id: nil,
79
+ credentials: nil,
80
+ scope: nil,
81
+ timeout: nil,
82
+ endpoint: nil,
83
+ emulator_host: nil,
84
+ project: nil,
85
+ keyfile: nil
82
86
  project_id ||= (project || default_project_id)
83
87
  scope ||= configure.scope
84
88
  timeout ||= configure.timeout
85
- client_config ||= configure.client_config
86
89
  endpoint ||= configure.endpoint
87
90
  emulator_host ||= configure.emulator_host
88
91
 
@@ -90,11 +93,8 @@ module Google
90
93
  project_id = project_id.to_s # Always cast to a string
91
94
  raise ArgumentError, "project_id is missing" if project_id.empty?
92
95
 
93
- return PubSub::Project.new(
94
- PubSub::Service.new(
95
- project_id, :this_channel_is_insecure, host: emulator_host, timeout: timeout, client_config: client_config
96
- )
97
- )
96
+ service = PubSub::Service.new project_id, :this_channel_is_insecure, host: emulator_host, timeout: timeout
97
+ return PubSub::Project.new service
98
98
  end
99
99
 
100
100
  credentials ||= (keyfile || default_credentials(scope: scope))
@@ -106,11 +106,8 @@ module Google
106
106
  project_id = project_id.to_s # Always cast to a string
107
107
  raise ArgumentError, "project_id is missing" if project_id.empty?
108
108
 
109
- PubSub::Project.new(
110
- PubSub::Service.new(
111
- project_id, credentials, timeout: timeout, host: endpoint, client_config: client_config
112
- )
113
- )
109
+ service = PubSub::Service.new project_id, credentials, host: endpoint, timeout: timeout
110
+ PubSub::Project.new service
114
111
  end
115
112
 
116
113
  # rubocop:enable Metrics/AbcSize
@@ -131,8 +128,6 @@ module Google
131
128
  # * `retries` - (Integer) Number of times to retry requests on server
132
129
  # error.
133
130
  # * `timeout` - (Integer) Default timeout to use in requests.
134
- # * `client_config` - (Hash) A hash of values to override the default
135
- # behavior of the API client.
136
131
  # * `endpoint` - (String) Override of the endpoint host name, or `nil`
137
132
  # to use the default endpoint.
138
133
  # * `emulator_host` - (String) Host name of the emulator. Defaults to
@@ -167,7 +162,9 @@ module Google
167
162
  end
168
163
  end
169
164
 
170
- ## Legacy namespace
165
+ ## Legacy veneer namespace
171
166
  Pubsub = PubSub unless const_defined? :Pubsub
172
167
  end
168
+ ## Legacy generated client namespace
169
+ Pubsub = Cloud::PubSub unless const_defined? :Pubsub
173
170
  end
@@ -42,7 +42,7 @@ module Google
42
42
  # end
43
43
  # end
44
44
  #
45
- # topic.async_publisher.stop.wait!
45
+ # topic.async_publisher.stop!
46
46
  #
47
47
  # @attr_reader [String] topic_name The name of the topic the messages are published to. In the form of
48
48
  # "/projects/project-identifier/topics/topic-name".
@@ -374,8 +374,7 @@ module Google
374
374
 
375
375
  PUBLISH_RETRY_ERRORS = [
376
376
  GRPC::Cancelled, GRPC::DeadlineExceeded, GRPC::Internal,
377
- GRPC::ResourceExhausted, GRPC::Unauthenticated, GRPC::Unavailable,
378
- GRPC::Core::CallError
377
+ GRPC::ResourceExhausted, GRPC::Unauthenticated, GRPC::Unavailable
379
378
  ].freeze
380
379
 
381
380
  def publish_batch_error_retryable? error
@@ -14,7 +14,7 @@
14
14
 
15
15
 
16
16
  require "googleauth"
17
- require "google/cloud/pubsub/v1/credentials.rb"
17
+ require "google/cloud/pubsub/v1/publisher/credentials.rb"
18
18
 
19
19
  module Google
20
20
  module Cloud
@@ -38,7 +38,7 @@ module Google
38
38
  #
39
39
  # pubsub.project_id #=> "my-project"
40
40
  #
41
- class Credentials < Google::Cloud::PubSub::V1::Credentials
41
+ class Credentials < Google::Cloud::PubSub::V1::Publisher::Credentials
42
42
  end
43
43
  end
44
44
 
@@ -50,7 +50,7 @@ module Google
50
50
  # subscriber.start
51
51
  #
52
52
  # # Shut down the subscriber when ready to stop receiving messages.
53
- # subscriber.stop.wait!
53
+ # subscriber.stop!
54
54
  #
55
55
  class Message
56
56
  ##
@@ -141,7 +141,7 @@ module Google
141
141
  # end
142
142
  # end
143
143
  #
144
- # topic.async_publisher.stop.wait!
144
+ # topic.async_publisher.stop!
145
145
  #
146
146
  def topic topic_name, project: nil, skip_lookup: nil, async: nil
147
147
  ensure_service!
@@ -39,7 +39,7 @@ module Google
39
39
  # subscriber.start
40
40
  #
41
41
  # # Shut down the subscriber when ready to stop receiving messages.
42
- # subscriber.stop.wait!
42
+ # subscriber.stop!
43
43
  #
44
44
  class ReceivedMessage
45
45
  ##
@@ -177,7 +177,7 @@ module Google
177
177
  # subscriber.start
178
178
  #
179
179
  # # Shut down the subscriber when ready to stop receiving messages.
180
- # subscriber.stop.wait!
180
+ # subscriber.stop!
181
181
  #
182
182
  def acknowledge!
183
183
  ensure_subscription!
@@ -214,7 +214,7 @@ module Google
214
214
  # subscriber.start
215
215
  #
216
216
  # # Shut down the subscriber when ready to stop receiving messages.
217
- # subscriber.stop.wait!
217
+ # subscriber.stop!
218
218
  #
219
219
  def modify_ack_deadline! new_deadline
220
220
  ensure_subscription!
@@ -244,7 +244,7 @@ module Google
244
244
  # subscriber.start
245
245
  #
246
246
  # # Shut down the subscriber when ready to stop receiving messages.
247
- # subscriber.stop.wait!
247
+ # subscriber.stop!
248
248
  #
249
249
  def reject!
250
250
  modify_ack_deadline! 0