google-cloud-pubsub 1.1.3 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1feb85392aa3adbb0358718fbd2b3f9cfad766c2dd30fa03ffdd93a0ab9199f3
-  data.tar.gz: 726f51f0d1851d82de1ee42230da2480a39d1dc18e3ebe418afdb03ce28fa6ea
+  metadata.gz: f5faee5de29df83057422ad7029a047eb10e8408a9b4e87599d301bc1c907a06
+  data.tar.gz: bc9d9b75232d66b09b5d45310875fbbd734663cf226575e43656da47aa955204
 SHA512:
-  metadata.gz: 9c9b2ac59f9fac674bb1bfbddf8d5a23cf768986fbd13e0e41128bf86574884b3d46cd88c564f9b28a531dfebaf6ba05bced34d5f6ee50664442876eea601d6a
-  data.tar.gz: e3e93f611b142aad3e8323d3aeae2339c810924f31369370e668063da54af64806b0270db81c75b2e6361537ec596d238a5bad4c8cdf79a4d818af08790efc6b
+  metadata.gz: b59bc075932f2a592276a8e0472371a18a8cc6e835ccb6c36ece17f277528bf92813d978e06393cf384c54c282c8a04fa6ba1d2fcfab08214e3cb3a55d2724ac
+  data.tar.gz: 8a6fffc01ab630eaf960c3c87858f942fe6a0241cfdf024875d379f2a915be724ea209bcfdc07c7ab12a1299949785456c27f0725d6d4b4cac519749cbc6f1fc

data/AUTHENTICATION.md

@@ -76,7 +76,8 @@ The environment variables that google-cloud-pubsub checks for project ID are:
 1. `PUBSUB_PROJECT`
 2. `GOOGLE_CLOUD_PROJECT`
 
-The environment variables that google-cloud-pubsub checks for credentials are configured on {Google::Cloud::PubSub::V1::Credentials}:
+The environment variables that google-cloud-pubsub checks for credentials
+are configured on {Google::Cloud::PubSub::Credentials}:
 
 1. `PUBSUB_CREDENTIALS` - Path to JSON file, or JSON contents
 2. `PUBSUB_KEYFILE` - Path to JSON file, or JSON contents

data/CHANGELOG.md

@@ -1,5 +1,150 @@
 # Release History
 
+### 2.0.0 / 2020-08-06
+
+This is a major update that removes the "low-level" client interface code, and
+instead adds the new `google-cloud-pubsub-v1` gem as a dependency.
+The new dependency is a rewritten low-level client, produced by a next-
+generation client code generator, with improved performance and stability.
+
+This change should have no effect on the high-level interface that most users
+will use. The one exception is that the (mostly undocumented) `client_config`
+argument, for adjusting low-level parameters such as RPC retry settings on
+client objects, has been removed. If you need to adjust these parameters, use
+the configuration interface in `google-cloud-pubsub-v1`.
+
+Substantial changes have been made in the low-level interfaces, however. If you
+are using the low-level classes under the `Google::Cloud::PubSub::V1` module,
+please review the docs for the new `google-cloud-pubsub-v1` gem. In
+particular:
+
+* Some classes have been renamed, notably the client classes themselves.
+* The client constructor takes a configuration block instead of configuration
+  keyword arguments.
+* All RPC method arguments are now keyword arguments.
+
+### 1.10.0 / 2020-07-23
+
+#### Features
+
+* Add Subscription#detach and #detached?
+
+### 1.9.0 / 2020-07-21
+
+#### Features
+
+* Add support for server-side flow control
+
+### 1.8.0 / 2020-06-29
+
+#### Features
+
+* Add Subscription#filter
+
+### 1.7.1 / 2020-05-28
+
+#### Documentation
+
+* Fix a few broken links
+
+### 1.7.0 / 2020-05-21
+
+#### Features
+
+* Add Retry Policy support
+  * Add RetryPolicy
+  * Add retry_policy param to Topic#subscribe
+  * Add Subscription#retry_policy
+  * Add Subscription#retry_policy=
+* Set client-scoped UUID in initial StreamingPullRequest#client_id
+
+### 1.6.1 / 2020-05-06
+
+#### Documentation
+
+* Fix example in Emulator documentation
+* Remove experimental notice from ReceivedMessage#delivery_attempt
+* Wrap example URLs in backticks
+
+### 1.6.0 / 2020-04-06
+
+#### Features
+
+* Add list_topic_snapshots and get_snapshot
+  * Add PublisherClient#list_topic_snapshots
+  * Add SubscriberClient#get_snapshot
+
+#### Documentation
+
+* Remove a spurious link in the low-level interface documentation.
+
+### 1.5.0 / 2020-03-25
+
+#### Features
+
+* Add max_duration_per_lease_extension to Subscription#listen and Subscriber
+
+### 1.4.0 / 2020-03-11
+
+#### Features
+
+*  Rename Subscriber inventory methods and params
+  * Rename Subscriber#inventory_limit to #max_outstanding_messages
+  * Rename Subscriber#bytesize to #max_outstanding_bytes
+  * Rename Subscriber#extension to #max_total_lease_duration
+  * Add deprecated aliases for the original methods
+* Support separate project setting for quota/billing
+
+#### Documentation
+
+* Update documentation in the lower-level client
+
+### 1.3.1 / 2020-02-18
+
+#### Bug Fixes
+
+* Move Thread.new to end of AsyncPublisher#initialize
+
+### 1.3.0 / 2020-02-10
+
+#### Features
+
+* Add support for Dead Letter Topics
+  * Add `dead_letter_topic` and `dead_letter_max_delivery_attempts` to `Topic#subscribe`
+  * Add `Subscription#dead_letter_topic` and `Subscription#dead_letter_topic=`
+  * Add `Subscription#dead_letter_max_delivery_attempts` and `Subscription#dead_letter_max_delivery_attempts=`
+  * Add `ReceivedMessage#delivery_attempt`
+
+### 1.2.2 / 2020-02-04
+
+#### Performance Improvements
+
+* Add StreamingPullRequest#client_id to the lower-level API
+
+### 1.2.1 / 2020-01-23
+
+#### Documentation
+
+* Update copyright year
+
+### 1.2.0 / 2020-01-09
+
+#### Features
+
+* Add Subscriber inventory settings
+  * Add the following settings to Subscriber:
+    * Subscriber#inventory_limit
+    * Subscriber#inventory_bytesize
+    * Subscriber#extension
+  * Allow Subscription#listen inventory argument to be a hash.
+* Update AsyncPublisher configuration defaults
+  * Update AsyncPublisher defaults to the following:
+    * max_bytes to 1MB, was 10MB.
+    * max_messages to 100, was 1,000.
+    * interval to 10 milliseconds, was 250 milliseconds.
+    * publish thread count to 2, was 4
+    * callback thread count to 4, was 8.
+
 ### 1.1.3 / 2019-12-18
 
 #### Bug Fixes

data/EMULATOR.md

@@ -17,7 +17,7 @@ require "google/cloud/pubsub"
 # Make Pub/Sub use the emulator
 ENV["PUBSUB_EMULATOR_HOST"] = "localhost:8918"
 
-pubsub = Google::Cloud::PubSub.new "emulator-project-id"
+pubsub = Google::Cloud::PubSub.new project_id:"emulator-project-id"
 
 # Get a topic in the current project
 my_topic = pubsub.new_topic "my-topic"

data/TROUBLESHOOTING.md

@@ -24,14 +24,8 @@ improved, *please* create a new issue on GitHub so we can talk about it.
 
   - [New issue][gh-ruby]
 
-Or, you can ask questions on the [Google Cloud Platform Slack][slack-ruby]. You
-can use the "ruby" channel for general Ruby questions, or use the
-"google-cloud-ruby" channel if you have questions about this gem in particular.
-
 [so-ruby]: http://stackoverflow.com/questions/tagged/google-cloud-platform+ruby+pubsub
 
-[gh-search-ruby]: https://github.com/googlecloudplatform/google-cloud-ruby/issues?q=label%3A%22api%3A+pubsub%22
-
-[gh-ruby]: https://github.com/googlecloudplatform/google-cloud-ruby/issues/new
+[gh-search-ruby]: https://github.com/googleapis/google-cloud-ruby/issues?q=label%3A%22api%3A+pubsub%22
 
-[slack-ruby]: https://gcp-slack.appspot.com/
+[gh-ruby]: https://github.com/googleapis/google-cloud-ruby/issues/new

data/lib/google/cloud/pubsub/async_publisher.rb

@@ -44,20 +44,17 @@ module Google
       #
       #   topic.async_publisher.stop.wait!
       #
-      # @attr_reader [String] topic_name The name of the topic the messages
-      #   are published to. In the form of
+      # @attr_reader [String] topic_name The name of the topic the messages are published to. In the form of
       #   "/projects/project-identifier/topics/topic-name".
-      # @attr_reader [Integer] max_bytes The maximum size of messages to be
-      #   collected before the batch is published. Default is 10,000,000
-      #   (10MB).
-      # @attr_reader [Integer] max_messages The maximum number of messages to
-      #   be collected before the batch is published. Default is 1,000.
-      # @attr_reader [Numeric] interval The number of seconds to collect
-      #   messages before the batch is published. Default is 0.25.
-      # @attr_reader [Numeric] publish_threads The number of threads used to
-      #   publish messages. Default is 4.
-      # @attr_reader [Numeric] callback_threads The number of threads to
-      #   handle the published messages' callbacks. Default is 8.
+      # @attr_reader [Integer] max_bytes The maximum size of messages to be collected before the batch is published.
+      #   Default is 1,000,000 (1MB).
+      # @attr_reader [Integer] max_messages The maximum number of messages to be collected before the batch is
+      #   published. Default is 100.
+      # @attr_reader [Numeric] interval The number of seconds to collect messages before the batch is published. Default
+      #   is 0.01.
+      # @attr_reader [Numeric] publish_threads The number of threads used to publish messages. Default is 2.
+      # @attr_reader [Numeric] callback_threads The number of threads to handle the published messages' callbacks.
+      #   Default is 4.
       #
       class AsyncPublisher
         include MonitorMixin
@@ -71,28 +68,27 @@ module Google
 
         ##
         # @private Create a new instance of the object.
-        def initialize topic_name, service, max_bytes: 10_000_000, max_messages: 1000, interval: 0.25, threads: {}
+        def initialize topic_name, service, max_bytes: 1_000_000, max_messages: 100, interval: 0.01, threads: {}
           # init MonitorMixin
           super()
-
           @topic_name = service.topic_path topic_name
           @service    = service
 
           @max_bytes        = max_bytes
           @max_messages     = max_messages
           @interval         = interval
-          @publish_threads  = (threads[:publish] || 4).to_i
-          @callback_threads = (threads[:callback] || 8).to_i
+          @publish_threads  = (threads[:publish] || 2).to_i
+          @callback_threads = (threads[:callback] || 4).to_i
 
           @published_at = nil
           @publish_thread_pool = Concurrent::ThreadPoolExecutor.new max_threads: @publish_threads
           @callback_thread_pool = Concurrent::ThreadPoolExecutor.new max_threads: @callback_threads
-          @thread = Thread.new { run_background }
 
           @ordered = false
           @batches = {}
-
           @cond = new_cond
+
+          @thread = Thread.new { run_background }
         end
 
         ##
@@ -378,8 +374,7 @@ module Google
 
         PUBLISH_RETRY_ERRORS = [
           GRPC::Cancelled, GRPC::DeadlineExceeded, GRPC::Internal,
-          GRPC::ResourceExhausted, GRPC::Unauthenticated, GRPC::Unavailable,
-          GRPC::Core::CallError
+          GRPC::ResourceExhausted, GRPC::Unauthenticated, GRPC::Unavailable
         ].freeze
 
         def publish_batch_error_retryable? error

data/lib/google/cloud/pubsub/credentials.rb

@@ -14,7 +14,7 @@
 
 
 require "googleauth"
-require "google/cloud/pubsub/v1/credentials.rb"
+require "google/cloud/pubsub/v1/publisher/credentials.rb"
 
 module Google
   module Cloud
@@ -38,7 +38,7 @@ module Google
       #
       #   pubsub.project_id #=> "my-project"
       #
-      class Credentials < Google::Cloud::PubSub::V1::Credentials
+      class Credentials < Google::Cloud::PubSub::V1::Publisher::Credentials
       end
     end
 

data/lib/google/cloud/pubsub/project.rb

@@ -89,19 +89,15 @@ module Google
         #
         #   Hash keys and values may include the following:
         #
-        #   * `:max_bytes` (Integer) The maximum size of messages to be
-        #     collected before the batch is published. Default is 10,000,000
-        #     (10MB).
-        #   * `:max_messages` (Integer) The maximum number of messages to be
-        #     collected before the batch is published. Default is 1,000.
-        #   * `:interval` (Numeric) The number of seconds to collect messages
-        #     before the batch is published. Default is 0.25.
-        #   * `:threads` (Hash) The number of threads to create to handle
-        #     concurrent calls by the publisher:
-        #     * `:publish` (Integer) The number of threads used to publish
-        #       messages. Default is 4.
-        #     * `:callback` (Integer) The number of threads to handle the
-        #       published messages' callbacks. Default is 8.
+        #   * `:max_bytes` (Integer) The maximum size of messages to be collected before the batch is published. Default
+        #     is 1,000,000 (1MB).
+        #   * `:max_messages` (Integer) The maximum number of messages to be collected before the batch is published.
+        #     Default is 100.
+        #   * `:interval` (Numeric) The number of seconds to collect messages before the batch is published. Default is
+        #     0.01.
+        #   * `:threads` (Hash) The number of threads to create to handle concurrent calls by the publisher:
+        #     * `:publish` (Integer) The number of threads used to publish messages. Default is 2.
+        #     * `:callback` (Integer) The number of threads to handle the published messages' callbacks. Default is 4.
         #
         # @return [Google::Cloud::PubSub::Topic, nil] Returns `nil` if topic
         #   does not exist.
@@ -183,19 +179,15 @@ module Google
         #
         #   Hash keys and values may include the following:
         #
-        #   * `:max_bytes` (Integer) The maximum size of messages to be
-        #     collected before the batch is published. Default is 10,000,000
-        #     (10MB).
-        #   * `:max_messages` (Integer) The maximum number of messages to be
-        #     collected before the batch is published. Default is 1,000.
-        #   * `:interval` (Numeric) The number of seconds to collect messages
-        #     before the batch is published. Default is 0.25.
-        #   * `:threads` (Hash) The number of threads to create to handle
-        #     concurrent calls by the publisher:
-        #     * `:publish` (Integer) The number of threads used to publish
-        #       messages. Default is 4.
-        #     * `:callback` (Integer) The number of threads to handle the
-        #       published messages' callbacks. Default is 8.
+        #   * `:max_bytes` (Integer) The maximum size of messages to be collected before the batch is published. Default
+        #     is 1,000,000 (1MB).
+        #   * `:max_messages` (Integer) The maximum number of messages to be collected before the batch is published.
+        #     Default is 100.
+        #   * `:interval` (Numeric) The number of seconds to collect messages before the batch is published. Default is
+        #     0.01.
+        #   * `:threads` (Hash) The number of threads to create to handle concurrent calls by the publisher:
+        #     * `:publish` (Integer) The number of threads used to publish messages. Default is 2.
+        #     * `:callback` (Integer) The number of threads to handle the published messages' callbacks. Default is 4.
         #
         # @return [Google::Cloud::PubSub::Topic]
         #

data/lib/google/cloud/pubsub/received_message.rb

@@ -63,6 +63,44 @@ module Google
           @grpc.ack_id
         end
 
+        ##
+        # Returns the delivery attempt counter for the message. If a dead letter policy is not set on the subscription,
+        # this will be `nil`. See {Topic#subscribe}, {Subscription#dead_letter_topic=} and
+        # {Subscription#dead_letter_max_delivery_attempts=}.
+        #
+        # The delivery attempt counter is `1 + (the sum of number of NACKs and number of ack_deadline exceeds)` for the
+        # message.
+        #
+        # A NACK is any call to `ModifyAckDeadline` with a `0` deadline. An `ack_deadline` exceeds event is whenever a
+        # message is not acknowledged within `ack_deadline`. Note that `ack_deadline` is initially
+        # `Subscription.ackDeadlineSeconds`, but may get extended automatically by the client library.
+        #
+        # The first delivery of a given message will have this value as `1`. The value is calculated at best effort and
+        # is approximate.
+        #
+        # @return [Integer, nil] A delivery attempt value of `1` or greater, or `nil` if a dead letter policy is not set
+        #   on the subscription.
+        #
+        # @example
+        #   require "google/cloud/pubsub"
+        #
+        #   pubsub = Google::Cloud::PubSub.new
+        #
+        #   topic = pubsub.topic "my-topic"
+        #   dead_letter_topic = pubsub.topic "my-dead-letter-topic", skip_lookup: true
+        #   sub = topic.subscribe "my-topic-sub",
+        #                         dead_letter_topic: dead_letter_topic,
+        #                         dead_letter_max_delivery_attempts: 10
+        #
+        #   subscriber = sub.listen do |received_message|
+        #     puts received_message.message.delivery_attempt
+        #   end
+        #
+        def delivery_attempt
+          return nil if @grpc.delivery_attempt && @grpc.delivery_attempt < 1
+          @grpc.delivery_attempt
+        end
+
         ##
         # The received message.
         def message

data/lib/google/cloud/pubsub/retry_policy.rb

@@ -0,0 +1,90 @@
+# Copyright 2016 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+require "google/cloud/errors"
+
+module Google
+  module Cloud
+    module PubSub
+      ##
+      # # RetryPolicy
+      #
+      # An immutable Retry Policy value object that specifies how Cloud Pub/Sub retries message delivery.
+      #
+      # Retry delay will be exponential based on provided minimum and maximum backoffs. (See [Exponential
+      # backoff](https://en.wikipedia.org/wiki/Exponential_backoff).)
+      #
+      # Retry Policy will be triggered on NACKs or acknowledgement deadline exceeded events for a given message.
+      #
+      # Retry Policy is implemented on a best effort basis. At times, the delay between consecutive deliveries may not
+      # match the configuration. That is, delay can be more or less than configured backoff.
+      #
+      # **EXPERIMENTAL:** This API might be changed in backward-incompatible ways and is not recommended for production
+      # use. It is not subject to any SLA or deprecation policy.
+      #
+      # @attr [Numeric] minimum_backoff The minimum delay between consecutive deliveries of a given message. Value
+      #   should be between 0 and 600 seconds. The default value is 10 seconds.
+      # @attr [Numeric] maximum_backoff The maximum delay between consecutive deliveries of a given message. Value
+      #   should be between 0 and 600 seconds. The default value is 600 seconds.
+      #
+      # @example
+      #   require "google/cloud/pubsub"
+      #
+      #   pubsub = Google::Cloud::PubSub.new
+      #
+      #   sub = pubsub.subscription "my-topic-sub"
+      #
+      #   sub.retry_policy = Google::Cloud::PubSub::RetryPolicy.new minimum_backoff: 5, maximum_backoff: 300
+      #
+      #   sub.retry_policy.minimum_backoff #=> 5
+      #   sub.retry_policy.maximum_backoff #=> 300
+      #
+      class RetryPolicy
+        attr_reader :minimum_backoff, :maximum_backoff
+
+        ##
+        # Creates a new, immutable RetryPolicy value object.
+        #
+        # @attr [Numeric, nil] minimum_backoff The minimum delay between consecutive deliveries of a given message.
+        #   Value should be between 0 and 600 seconds. If `nil` is provided, the default value is 10 seconds.
+        # @attr [Numeric, nil] maximum_backoff The maximum delay between consecutive deliveries of a given message.
+        #   Value should be between 0 and 600 seconds. If `nil` is provided, the default value is 600 seconds.
+        #
+        def initialize minimum_backoff: nil, maximum_backoff: nil
+          @minimum_backoff = minimum_backoff
+          @maximum_backoff = maximum_backoff
+        end
+
+        ##
+        # @private Convert the RetryPolicy to a Google::Cloud::PubSub::V1::RetryPolicy object.
+        def to_grpc
+          Google::Cloud::PubSub::V1::RetryPolicy.new(
+            minimum_backoff: Convert.number_to_duration(minimum_backoff),
+            maximum_backoff: Convert.number_to_duration(maximum_backoff)
+          )
+        end
+
+        ##
+        # @private New RetryPolicy from a Google::Cloud::PubSub::V1::RetryPolicy object.
+        def self.from_grpc grpc
+          new(
+            minimum_backoff: Convert.duration_to_number(grpc.minimum_backoff),
+            maximum_backoff: Convert.duration_to_number(grpc.maximum_backoff)
+          )
+        end
+      end
+    end
+  end
+end