pubsub_client 1.1.0 → 1.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7da1c7755c4338934a7b0b9067b5dbc509cc11dbe9f22b046578048d23d3529b
-  data.tar.gz: cab1c8f5d426d36805931f0e75fd6976a65812f5c9ab86b607a61596c1a5d8a9
+  metadata.gz: 03ba6e12917979bfedb1b0d4aeade60f2ce675c561e99d57ace0b0850abb0f24
+  data.tar.gz: e0628ab5d8c8183c95a1a438be7e32964870d891d7ded108bf5ce4eb10f63c5d
 SHA512:
-  metadata.gz: 68b17c969910bb8534ac1d904ea65d78997bac2d20f5992cecbc1c35ef4bc96f7c544662d63591efefddde3038912fda18e86401bca2669f07348baab4da4c1e
-  data.tar.gz: 8529bb3c895d07054a2fa957c33fdb344a7fc2927a5a72448c37554066374afbd0de3c7223a8bcb2b9168accdad5d72a9e573e4331378829a901db8ad9b640b5
+  metadata.gz: 4dafd9b8a6da6055d4d10bba716eb54d72fb6202b84ed693c090a9a53aa16c43ec6306fc641943f207da575bab6922fade8780536392f2f6c5231c7f88dc1558
+  data.tar.gz: 2f975bcc3d71fb66d9031be80929b0e709d0605e7596039274e4591b7ab36f83c5f70b67a49d4aebb2adac1a3d75588595bf787a2c4f7e1fdec45078f708d176

data/CHANGELOG.txt

@@ -1,3 +1,31 @@
+Version 1.5.0 2020-11-05
+------------------------
+6cab366 Do not require GAC when stubbed
+dab08dd Update `Subscriber#on_error` documentation
+
+Version 1.4.1 2020-10-26
+------------------------
+d28d99f Pass published attributes all the way through the client
+
+Version 1.4.0 2020-10-26
+------------------------
+60b8403 Publish attributes
+
+Version 1.3.0 2020-10-23
+------------------------
+e9e87f7 This adds the ability to subscribe to a Pub/Sub topic. Of note, we expose two additional configuration params:
+- `concurrency`: the number of threads the subscriber will run to process messages (defaults to 8 threads)
+-  `auto_ack`: flag to auto ack messages (default is `true` and _will_ ack messages)
+
+These changes come with a handful of useful checks:
+- ensures credentials are configured prior to subscribing
+- raises an error if the target subscription does not exist
+- raises an error if attempting to subscribe to a topic that has already been subscribed to
+
+Version 1.2.0 2020-09-25
+------------------------
+aede3ab Raise custom error if GOOGLE_APPLICATION_CREDENTIALS not set
+
 Version 1.1.0 2020-09-25
 ------------------------
 d3f8ed1 Throw error if topic does not exist

data/README.md

@@ -32,9 +32,11 @@ end
 
 ## Usage
 
+### Publishing
+
 To publish a message to Pub/Sub, call `PubsubClient.publish(message, 'the-topic')`. This method takes any serializable object as an argument and yields a result object to a block. The `result` object has a method `#succeeded?` that returns `true` if the message was successfully published, otherwise `false`. In the latter case, there is a method `#error` that returns the error.
 
-### Example
+#### Example
 ```ruby
 PubsubClient.publish(message, 'some-topic') do |result|
   if result.succeeded?
@@ -45,14 +47,54 @@ PubsubClient.publish(message, 'some-topic') do |result|
 end
 ```
 
+### Subscribing
+
+To subscribe to a topic, a client must first get a handle to the subscriber object. After doing so, a call to `subscriber.listener` will yield two arguments: the data (most clients will only need this) and the full Pub/Sub message (for anything more robust). Documentation for the full message can be found [here](https://googleapis.dev/ruby/google-cloud-pubsub/latest/Google/Cloud/PubSub/ReceivedMessage.html).
+
+Optionally, a client can choose to handle exceptions raised by the subscriber. If a client chooses to do so, the listener **must** be configured before `on_error` since the latter needs a handler to the underlying listener. Additionally, exceptions will only be raised when the work inside the block is happening on the same thread. For instance, if the block enqueues a background worker and that worker raises an error, it will not be handled by the `on_error` block.
+
+#### Example
+```ruby
+subscriber = PubsubClient.subscriber('some-topic')
+
+subscriber.listener(concurrency: 4, auto_ack: false) do |data, received_message|
+  # Most clients will only need the first yielded arg.
+  # It is the same as calling received_message.data
+end
+
+# Optional
+subscriber.on_error do |ex|
+  # Do something with the exception.
+end
+
+subscriber.subscribe # This will sleep
+```
+
+By default, the underlying subscriber will use a concurrency of `8` threads and will acknowledge all messages. If these defaults are acceptable to the client, no arguments need to be passed in the call to `listener`.
+```ruby
+subscriber.listener do |data, received_message|
+  # Do something
+end
+```
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `bundle exec rake install`.
 
 ## Contributing
 
+To contribute, open a pull request against `main`. Note that once your changes have been made, you should _not_ manually modify the `version.rb` or `CHANGELOG` as these will get updated automatically as part of the release process.
+
+To release a new version, after you have merged your changes into `main`:
+1. Run the `gem-release` script. This can be found [here](https://github.com/apartmentlist/scripts/blob/main/bin/gem-release).
+    ```
+    path/to/gem-release [major/minor/patch] "Short message with changes"
+    ```
+    Note that the `Short message with changes` is what gets reflected in the releases of the repo.
+1. Run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org). This step will update the `version.rb` and `CHANGELOG` files.
+
 Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/pubsub_client. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/pubsub_client/blob/master/CODE_OF_CONDUCT.md).
 
 

data/lib/pubsub_client.rb

@@ -1,21 +1,48 @@
 require 'pubsub_client/version'
 require 'pubsub_client/null_publisher_factory'
+require 'pubsub_client/null_subscriber_factory'
 require 'pubsub_client/publisher_factory'
+require 'pubsub_client/subscriber_factory'
 
 module PubsubClient
   Error = Class.new(StandardError)
   ConfigurationError = Class.new(Error)
+  CredentialsError = Class.new(Error)
   InvalidTopicError = Class.new(Error)
+  InvalidSubscriptionError = Class.new(Error)
 
   class << self
     def stub!
-      raise ConfigurationError, 'PubsubClient is already configured' if @publisher_factory
+      raise ConfigurationError, 'PubsubClient is already configured' if @publisher_factory || @subscriber_factory
       @publisher_factory = NullPublisherFactory.new
+      @subscriber_factory = NullSubscriberFactory.new
+      @stubbed = true
     end
 
-    def publish(message, topic, &block)
+    # @param message [String] The message to publish.
+    # @param topic [String] The name of the topic to publish to.
+    def publish(message, topic, attributes = {}, &block)
+      ensure_credentials!
+
       @publisher_factory ||= PublisherFactory.new
-      @publisher_factory.build(topic).publish(message, &block)
+      @publisher_factory.build(topic).publish(message, attributes, &block)
+    end
+
+    # @param subscription [String] - The name of the topic to subscribe to.
+    def subscriber(subscription)
+      ensure_credentials!
+
+      @subscriber_factory ||= SubscriberFactory.new
+      @subscriber_factory.build(subscription)
+    end
+
+    private
+
+    def ensure_credentials!
+      return if defined?(@stubbed) && @stubbed
+      unless ENV['GOOGLE_APPLICATION_CREDENTIALS']
+        raise CredentialsError, 'GOOGLE_APPLICATION_CREDENTIALS must be set'
+      end
     end
   end
 end

data/lib/pubsub_client/null_subscriber.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module PubsubClient
+  # A null object to act as a subscriber when clients are in dev or test
+  class NullSubscriber
+    # This adds a subset of the available methods on the
+    # Google::Cloud::PubSub::ReceivedMessage, which is what
+    # gets yielded by the subscription when configuring the listener.
+    # For a list of methods, see the following link:
+    # https://googleapis.dev/ruby/google-cloud-pubsub/latest/Google/Cloud/PubSub/ReceivedMessage.html
+    NullResult = Struct.new(:acknowledge!) do
+      def data
+        '{"key":"value"}'
+      end
+
+      def published_at
+        Time.now
+      end
+    end
+
+    def listener(*, &block)
+      res = NullResult.new
+      yield res.data, res
+    end
+
+    def subscribe
+      # no-op
+    end
+
+    def on_error(&block)
+      # no-op
+    end
+  end
+end

data/lib/pubsub_client/null_subscriber_factory.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require_relative 'null_subscriber'
+
+module PubsubClient
+  # A null object to act as a subscriber factory when clients are in dev or test
+  class NullSubscriberFactory
+    def build(*)
+      NullSubscriber.new
+    end
+  end
+end

data/lib/pubsub_client/publisher.rb

@@ -4,12 +4,13 @@ require 'google/cloud/pubsub'
 
 module PubsubClient
   class Publisher
+    # @param topic [Google::Cloud::PubSub::Topic]
     def initialize(topic)
       @topic = topic
     end
 
-    def publish(message, &block)
-      topic.publish_async(message, &block)
+    def publish(message, attributes = {}, &block)
+      topic.publish_async(message, attributes, &block)
     end
 
     def flush

data/lib/pubsub_client/publisher_factory.rb

@@ -10,6 +10,8 @@ module PubsubClient
       @publishers = {}
     end
 
+    # @param topic_name [String]
+    # @return [Publisher]
     def build(topic_name)
       # GRPC fails when attempting to use a connection created in a process that gets
       # forked with the message

data/lib/pubsub_client/subscriber.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require 'google/cloud/pubsub'
+
+module PubsubClient
+  class Subscriber
+    DEFAULT_CONCURRENCY = 8
+
+    # @param subscription [Google::Cloud::PubSub::Subscription]
+    def initialize(subscription)
+      @subscription = subscription
+    end
+
+    # @param concurrency [Integer] - The number of threads to run the subscriber with. Default is 8.
+    # @param auto_ack [Boolean] - Flag to acknowledge the Pub/Sub message. A message must be acked
+    # to remove it from the topic. Default is `true`.
+    #
+    # @return [Google::Cloud::PubSub::Subscriber]
+    def listener(concurrency: DEFAULT_CONCURRENCY, auto_ack: true, &block)
+      @listener ||= begin
+        @subscription.listen(threads: { callback: concurrency }) do |received_message|
+          yield received_message.data, received_message
+          received_message.acknowledge! if auto_ack
+        end
+      end
+    end
+
+    def subscribe
+      raise ConfigurationError, 'A listener must be configured' unless @listener
+
+      begin
+        @listener.start
+
+        sleep
+      rescue SignalException
+        @listener.stop.wait!
+      end
+    end
+
+    def on_error(&block)
+      raise ConfigurationError, 'A listener must be configured' unless @listener
+
+      @listener.on_error do |exception|
+        yield exception
+      end
+    end
+  end
+end

data/lib/pubsub_client/subscriber_factory.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require_relative 'subscriber'
+
+module PubsubClient
+  class SubscriberFactory
+    def initialize
+      @subscribers = {}
+    end
+
+    # @param subscription_name [String]
+    # @retrun [Subscriber]
+    def build(subscription_name)
+      if @subscribers.key?(subscription_name)
+        raise ConfigurationError, "PubsubClient already subscribed to #{subscription_name}"
+      end
+
+      @subscribers[subscription_name] = build_subscriber(subscription_name)
+    end
+
+    private
+
+    def build_subscriber(subscription_name)
+      pubsub = Google::Cloud::PubSub.new
+      subscription = pubsub.subscription(subscription_name)
+      raise InvalidSubscriptionError, "The subscription #{subscription_name} does not exist" unless subscription
+      Subscriber.new(subscription)
+    end
+  end
+end

data/lib/pubsub_client/version.rb

@@ -1,3 +1,3 @@
 module PubsubClient
-  VERSION = '1.1.0'
+  VERSION = '1.5.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: pubsub_client
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.5.0
 platform: ruby
 authors:
 - Apartment List Platforms
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-09-29 00:00:00.000000000 Z
+date: 2020-11-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -88,8 +88,12 @@ files:
 - lib/pubsub_client.rb
 - lib/pubsub_client/null_publisher.rb
 - lib/pubsub_client/null_publisher_factory.rb
+- lib/pubsub_client/null_subscriber.rb
+- lib/pubsub_client/null_subscriber_factory.rb
 - lib/pubsub_client/publisher.rb
 - lib/pubsub_client/publisher_factory.rb
+- lib/pubsub_client/subscriber.rb
+- lib/pubsub_client/subscriber_factory.rb
 - lib/pubsub_client/version.rb
 - pubsub_client.gemspec
 homepage: https://github.com/apartmentlist/pubsub_client