pubsub_client 1.0.0 → 1.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 13c10d53daf5538c3aca168f2cee4ba8926bc6796bf77653fb9b9a87d2b26f72
-  data.tar.gz: eb7a89953c274694319285cb9350803d7b13f31a99e2bd1d365e074460ab06ce
+  metadata.gz: 82ac8b7b0bd3910b7e69b2f63bb08bffae388afe1b02a39bd6e3e8b05e35d76b
+  data.tar.gz: 98a3e736fb8e58cb13e9daf6e0ffd15e2507bef1156fe5da90a61be0979a8314
 SHA512:
-  metadata.gz: 81e5cbdad71787603ab5d5a6d2ac7802a7cc8e9bb40fe4414d5c87a4935ca266ad0afa0efb0fa48b03b2f8ccdaefe3963054614e6b25db359381758849e82e60
-  data.tar.gz: 289d015b198b809a9683b5d7594ffcbd98a70ce964a2ab4471ab42bba3f83e0de1dbe0a3bd29cbd19ea5498e6f4e85a171658a7a60f326393d019361125dd7f6
+  metadata.gz: e22163305a0f11e5f80326964df992ccd8b56235a434dd0ed7fe20ac81340bcb69cad77cafa4782b35017fec54b6897112cfa0feade5a891fef0aad88496ccad
+  data.tar.gz: 8e6bbeadb5b6c604fd984990a317ec8faf630daa0b73605aeb4873686b461c6f2fefea5debddbf51ba42269bcf9e3b9e166e9535d214ea33c7248742034d38ac

data/CHANGELOG.txt

@@ -1,6 +1,34 @@
+Version 1.4.1 2020-10-26
+------------------------
+d28d99f Pass published attributes all the way through the client
+
+Version 1.4.0 2020-10-26
+------------------------
+60b8403 Publish attributes
+
+Version 1.3.0 2020-10-23
+------------------------
+e9e87f7 This adds the ability to subscribe to a Pub/Sub topic. Of note, we expose two additional configuration params:
+- `concurrency`: the number of threads the subscriber will run to process messages (defaults to 8 threads)
+-  `auto_ack`: flag to auto ack messages (default is `true` and _will_ ack messages)
+
+These changes come with a handful of useful checks:
+- ensures credentials are configured prior to subscribing
+- raises an error if the target subscription does not exist
+- raises an error if attempting to subscribe to a topic that has already been subscribed to
+
+Version 1.2.0 2020-09-25
+------------------------
+aede3ab Raise custom error if GOOGLE_APPLICATION_CREDENTIALS not set
+
+Version 1.1.0 2020-09-25
+------------------------
+d3f8ed1 Throw error if topic does not exist
+a95430f Bump major version and update change log
+
 Version 1.0.0 2020-09-24
 ------------------------
-396f5f8 Allow publishing to any topic
+1d7dc1c Allow publishing to any topic
 
 Version 0.1.0 2020-08-25
 ------------------------

data/README.md

@@ -32,9 +32,11 @@ end
 
 ## Usage
 
+### Publishing
+
 To publish a message to Pub/Sub, call `PubsubClient.publish(message, 'the-topic')`. This method takes any serializable object as an argument and yields a result object to a block. The `result` object has a method `#succeeded?` that returns `true` if the message was successfully published, otherwise `false`. In the latter case, there is a method `#error` that returns the error.
 
-### Example
+#### Example
 ```ruby
 PubsubClient.publish(message, 'some-topic') do |result|
   if result.succeeded?
@@ -45,14 +47,54 @@ PubsubClient.publish(message, 'some-topic') do |result|
 end
 ```
 
+### Subscribing
+
+To subscribe to a topic, a client must first get a handle to the subscriber object. After doing so, a call to `subscriber.listener` will yield two arguments: the data (most clients will only need this) and the full Pub/Sub message (for anything more robust). Documentation for the full message can be found [here](https://googleapis.dev/ruby/google-cloud-pubsub/latest/Google/Cloud/PubSub/ReceivedMessage.html).
+
+Optionally, a client can choose to handle exceptions raised by the subscriber. If a client chooses to do so, the listener **must** be configured before `on_error` since the latter needs a handler to the underlying listener.
+
+#### Example
+```ruby
+subscriber = PubsubClient.subscriber('some-topic')
+
+subscriber.listener(concurrency: 4, auto_ack: false) do |data, received_message|
+  # Most clients will only need the first yielded arg.
+  # It is the same as calling received_message.data
+end
+
+# Optional
+subscriber.on_error do |ex|
+  # Do something with the exception.
+end
+
+subscriber.subscribe # This will sleep
+```
+
+By default, the underlying subscriber will use a concurrency of `8` threads and will acknowledge all messages. If these defaults are acceptable to the client, no arguments need to be passed in the call to `listener`.
+```ruby
+subscriber.listener do |data, received_message|
+  # Do something
+end
+```
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `bundle exec rake install`.
 
 ## Contributing
 
+To contribute, open a pull request against `main`. Note that once your changes have been made, you should _not_ manually modify the `version.rb` or `CHANGELOG` as these will get updated automatically as part of the release process.
+
+To release a new version, after you have merged your changes into `main`:
+1. Run the `gem-release` script. This can be found [here](https://github.com/apartmentlist/scripts/blob/main/bin/gem-release).
+    ```
+    path/to/gem-release [major/minor/patch] "Short message with changes"
+    ```
+    Note that the `Short message with changes` is what gets reflected in the releases of the repo.
+1. Run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org). This step will update the `version.rb` and `CHANGELOG` files.
+
 Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/pubsub_client. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/pubsub_client/blob/master/CODE_OF_CONDUCT.md).
 
 

data/lib/pubsub_client.rb

@@ -1,20 +1,46 @@
 require 'pubsub_client/version'
 require 'pubsub_client/null_publisher_factory'
+require 'pubsub_client/null_subscriber_factory'
 require 'pubsub_client/publisher_factory'
+require 'pubsub_client/subscriber_factory'
 
 module PubsubClient
   Error = Class.new(StandardError)
   ConfigurationError = Class.new(Error)
+  CredentialsError = Class.new(Error)
+  InvalidTopicError = Class.new(Error)
+  InvalidSubscriptionError = Class.new(Error)
 
   class << self
     def stub!
-      raise ConfigurationError, 'PubsubClient is already configured' if @publisher_factory
+      raise ConfigurationError, 'PubsubClient is already configured' if @publisher_factory || @subscriber_factory
       @publisher_factory = NullPublisherFactory.new
+      @subscriber_factory = NullSubscriberFactory.new
     end
 
-    def publish(message, topic, &block)
+    # @param message [String] The message to publish.
+    # @param topic [String] The name of the topic to publish to.
+    def publish(message, topic, attributes = {}, &block)
+      ensure_credentials!
+
       @publisher_factory ||= PublisherFactory.new
-      @publisher_factory.build(topic).publish(message, &block)
+      @publisher_factory.build(topic).publish(message, attributes, &block)
+    end
+
+    # @param subscription [String] - The name of the topic to subscribe to.
+    def subscriber(subscription)
+      ensure_credentials!
+
+      @subscriber_factory ||= SubscriberFactory.new
+      @subscriber_factory.build(subscription)
+    end
+
+    private
+
+    def ensure_credentials!
+      unless ENV['GOOGLE_APPLICATION_CREDENTIALS']
+        raise CredentialsError, 'GOOGLE_APPLICATION_CREDENTIALS must be set'
+      end
     end
   end
 end

data/lib/pubsub_client/null_subscriber.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module PubsubClient
+  # A null object to act as a subscriber when clients are in dev or test
+  class NullSubscriber
+    # This adds a subset of the available methods on the
+    # Google::Cloud::PubSub::ReceivedMessage, which is what
+    # gets yielded by the subscription when configuring the listener.
+    # For a list of methods, see the following link:
+    # https://googleapis.dev/ruby/google-cloud-pubsub/latest/Google/Cloud/PubSub/ReceivedMessage.html
+    NullResult = Struct.new(:acknowledge!) do
+      def data
+        '{"key":"value"}'
+      end
+
+      def published_at
+        Time.now
+      end
+    end
+
+    def listener(*, &block)
+      res = NullResult.new
+      yield res.data, res
+    end
+
+    def subscribe
+      # no-op
+    end
+
+    def on_error(&block)
+      # no-op
+    end
+  end
+end

data/lib/pubsub_client/null_subscriber_factory.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require_relative 'null_subscriber'
+
+module PubsubClient
+  # A null object to act as a subscriber factory when clients are in dev or test
+  class NullSubscriberFactory
+    def build(*)
+      NullSubscriber.new
+    end
+  end
+end

data/lib/pubsub_client/publisher.rb

@@ -4,12 +4,13 @@ require 'google/cloud/pubsub'
 
 module PubsubClient
   class Publisher
+    # @param topic [Google::Cloud::PubSub::Topic]
     def initialize(topic)
       @topic = topic
     end
 
-    def publish(message, &block)
-      topic.publish_async(message, &block)
+    def publish(message, attributes = {}, &block)
+      topic.publish_async(message, attributes, &block)
     end
 
     def flush

data/lib/pubsub_client/publisher_factory.rb

@@ -10,6 +10,8 @@ module PubsubClient
       @publishers = {}
     end
 
+    # @param topic_name [String]
+    # @return [Publisher]
     def build(topic_name)
       # GRPC fails when attempting to use a connection created in a process that gets
       # forked with the message
@@ -52,6 +54,7 @@ module PubsubClient
     def build_publisher(topic_name)
       pubsub = Google::Cloud::PubSub.new
       topic = pubsub.topic(topic_name)
+      raise InvalidTopicError, "The topic #{topic_name} does not exist" unless topic
       publisher = Publisher.new(topic)
 
       at_exit { publisher.flush }

data/lib/pubsub_client/subscriber.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require 'google/cloud/pubsub'
+
+module PubsubClient
+  class Subscriber
+    DEFAULT_CONCURRENCY = 8
+
+    # @param subscription [Google::Cloud::PubSub::Subscription]
+    def initialize(subscription)
+      @subscription = subscription
+    end
+
+    # @param concurrency [Integer] - The number of threads to run the subscriber with. Default is 8.
+    # @param auto_ack [Boolean] - Flag to acknowledge the Pub/Sub message. A message must be acked
+    # to remove it from the topic. Default is `true`.
+    #
+    # @return [Google::Cloud::PubSub::Subscriber]
+    def listener(concurrency: DEFAULT_CONCURRENCY, auto_ack: true, &block)
+      @listener ||= begin
+        @subscription.listen(threads: { callback: concurrency }) do |received_message|
+          yield received_message.data, received_message
+          received_message.acknowledge! if auto_ack
+        end
+      end
+    end
+
+    def subscribe
+      raise ConfigurationError, 'A listener must be configured' unless @listener
+
+      begin
+        @listener.start
+
+        sleep
+      rescue SignalException
+        @listener.stop.wait!
+      end
+    end
+
+    def on_error(&block)
+      raise ConfigurationError, 'A listener must be configured' unless @listener
+
+      @listener.on_error do |exception|
+        yield exception
+      end
+    end
+  end
+end

data/lib/pubsub_client/subscriber_factory.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require_relative 'subscriber'
+
+module PubsubClient
+  class SubscriberFactory
+    def initialize
+      @subscribers = {}
+    end
+
+    # @param subscription_name [String]
+    # @retrun [Subscriber]
+    def build(subscription_name)
+      if @subscribers.key?(subscription_name)
+        raise ConfigurationError, "PubsubClient already subscribed to #{subscription_name}"
+      end
+
+      @subscribers[subscription_name] = build_subscriber(subscription_name)
+    end
+
+    private
+
+    def build_subscriber(subscription_name)
+      pubsub = Google::Cloud::PubSub.new
+      subscription = pubsub.subscription(subscription_name)
+      raise InvalidSubscriptionError, "The subscription #{subscription_name} does not exist" unless subscription
+      Subscriber.new(subscription)
+    end
+  end
+end

data/lib/pubsub_client/version.rb

@@ -1,3 +1,3 @@
 module PubsubClient
-  VERSION = '1.0.0'
+  VERSION = '1.4.1'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: pubsub_client
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.4.1
 platform: ruby
 authors:
 - Apartment List Platforms
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-09-29 00:00:00.000000000 Z
+date: 2020-10-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -88,8 +88,12 @@ files:
 - lib/pubsub_client.rb
 - lib/pubsub_client/null_publisher.rb
 - lib/pubsub_client/null_publisher_factory.rb
+- lib/pubsub_client/null_subscriber.rb
+- lib/pubsub_client/null_subscriber_factory.rb
 - lib/pubsub_client/publisher.rb
 - lib/pubsub_client/publisher_factory.rb
+- lib/pubsub_client/subscriber.rb
+- lib/pubsub_client/subscriber_factory.rb
 - lib/pubsub_client/version.rb
 - pubsub_client.gemspec
 homepage: https://github.com/apartmentlist/pubsub_client
@@ -114,7 +118,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.8
+rubygems_version: 3.1.4
 signing_key:
 specification_version: 4
 summary: Google Pub/Sub Wrapper