pubsub_client 1.0.0 → 1.4.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 13c10d53daf5538c3aca168f2cee4ba8926bc6796bf77653fb9b9a87d2b26f72
4
- data.tar.gz: eb7a89953c274694319285cb9350803d7b13f31a99e2bd1d365e074460ab06ce
3
+ metadata.gz: 82ac8b7b0bd3910b7e69b2f63bb08bffae388afe1b02a39bd6e3e8b05e35d76b
4
+ data.tar.gz: 98a3e736fb8e58cb13e9daf6e0ffd15e2507bef1156fe5da90a61be0979a8314
5
5
  SHA512:
6
- metadata.gz: 81e5cbdad71787603ab5d5a6d2ac7802a7cc8e9bb40fe4414d5c87a4935ca266ad0afa0efb0fa48b03b2f8ccdaefe3963054614e6b25db359381758849e82e60
7
- data.tar.gz: 289d015b198b809a9683b5d7594ffcbd98a70ce964a2ab4471ab42bba3f83e0de1dbe0a3bd29cbd19ea5498e6f4e85a171658a7a60f326393d019361125dd7f6
6
+ metadata.gz: e22163305a0f11e5f80326964df992ccd8b56235a434dd0ed7fe20ac81340bcb69cad77cafa4782b35017fec54b6897112cfa0feade5a891fef0aad88496ccad
7
+ data.tar.gz: 8e6bbeadb5b6c604fd984990a317ec8faf630daa0b73605aeb4873686b461c6f2fefea5debddbf51ba42269bcf9e3b9e166e9535d214ea33c7248742034d38ac
@@ -1,6 +1,34 @@
1
+ Version 1.4.1 2020-10-26
2
+ ------------------------
3
+ d28d99f Pass published attributes all the way through the client
4
+
5
+ Version 1.4.0 2020-10-26
6
+ ------------------------
7
+ 60b8403 Publish attributes
8
+
9
+ Version 1.3.0 2020-10-23
10
+ ------------------------
11
+ e9e87f7 This adds the ability to subscribe to a Pub/Sub topic. Of note, we expose two additional configuration params:
12
+ - `concurrency`: the number of threads the subscriber will run to process messages (defaults to 8 threads)
13
+ - `auto_ack`: flag to auto ack messages (default is `true` and _will_ ack messages)
14
+
15
+ These changes come with a handful of useful checks:
16
+ - ensures credentials are configured prior to subscribing
17
+ - raises an error if the target subscription does not exist
18
+ - raises an error if attempting to subscribe to a topic that has already been subscribed to
19
+
20
+ Version 1.2.0 2020-09-25
21
+ ------------------------
22
+ aede3ab Raise custom error if GOOGLE_APPLICATION_CREDENTIALS not set
23
+
24
+ Version 1.1.0 2020-09-25
25
+ ------------------------
26
+ d3f8ed1 Throw error if topic does not exist
27
+ a95430f Bump major version and update change log
28
+
1
29
  Version 1.0.0 2020-09-24
2
30
  ------------------------
3
- 396f5f8 Allow publishing to any topic
31
+ 1d7dc1c Allow publishing to any topic
4
32
 
5
33
  Version 0.1.0 2020-08-25
6
34
  ------------------------
data/README.md CHANGED
@@ -32,9 +32,11 @@ end
32
32
 
33
33
  ## Usage
34
34
 
35
+ ### Publishing
36
+
35
37
  To publish a message to Pub/Sub, call `PubsubClient.publish(message, 'the-topic')`. This method takes any serializable object as an argument and yields a result object to a block. The `result` object has a method `#succeeded?` that returns `true` if the message was successfully published, otherwise `false`. In the latter case, there is a method `#error` that returns the error.
36
38
 
37
- ### Example
39
+ #### Example
38
40
  ```ruby
39
41
  PubsubClient.publish(message, 'some-topic') do |result|
40
42
  if result.succeeded?
@@ -45,14 +47,54 @@ PubsubClient.publish(message, 'some-topic') do |result|
45
47
  end
46
48
  ```
47
49
 
50
+ ### Subscribing
51
+
52
+ To subscribe to a topic, a client must first get a handle to the subscriber object. After doing so, a call to `subscriber.listener` will yield two arguments: the data (most clients will only need this) and the full Pub/Sub message (for anything more robust). Documentation for the full message can be found [here](https://googleapis.dev/ruby/google-cloud-pubsub/latest/Google/Cloud/PubSub/ReceivedMessage.html).
53
+
54
+ Optionally, a client can choose to handle exceptions raised by the subscriber. If a client chooses to do so, the listener **must** be configured before `on_error` since the latter needs a handler to the underlying listener.
55
+
56
+ #### Example
57
+ ```ruby
58
+ subscriber = PubsubClient.subscriber('some-topic')
59
+
60
+ subscriber.listener(concurrency: 4, auto_ack: false) do |data, received_message|
61
+ # Most clients will only need the first yielded arg.
62
+ # It is the same as calling received_message.data
63
+ end
64
+
65
+ # Optional
66
+ subscriber.on_error do |ex|
67
+ # Do something with the exception.
68
+ end
69
+
70
+ subscriber.subscribe # This will sleep
71
+ ```
72
+
73
+ By default, the underlying subscriber will use a concurrency of `8` threads and will acknowledge all messages. If these defaults are acceptable to the client, no arguments need to be passed in the call to `listener`.
74
+ ```ruby
75
+ subscriber.listener do |data, received_message|
76
+ # Do something
77
+ end
78
+ ```
79
+
48
80
  ## Development
49
81
 
50
82
  After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
51
83
 
52
- To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
84
+ To install this gem onto your local machine, run `bundle exec rake install`.
53
85
 
54
86
  ## Contributing
55
87
 
88
+ To contribute, open a pull request against `main`. Note that once your changes have been made, you should _not_ manually modify the `version.rb` or `CHANGELOG` as these will get updated automatically as part of the release process.
89
+
90
+ To release a new version, after you have merged your changes into `main`:
91
+ 1. Run the `gem-release` script. This can be found [here](https://github.com/apartmentlist/scripts/blob/main/bin/gem-release).
92
+ ```
93
+ path/to/gem-release [major/minor/patch] "Short message with changes"
94
+ ```
95
+ Note that the `Short message with changes` is what gets reflected in the releases of the repo.
96
+ 1. Run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org). This step will update the `version.rb` and `CHANGELOG` files.
97
+
56
98
  Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/pubsub_client. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/pubsub_client/blob/master/CODE_OF_CONDUCT.md).
57
99
 
58
100
 
@@ -1,20 +1,46 @@
1
1
  require 'pubsub_client/version'
2
2
  require 'pubsub_client/null_publisher_factory'
3
+ require 'pubsub_client/null_subscriber_factory'
3
4
  require 'pubsub_client/publisher_factory'
5
+ require 'pubsub_client/subscriber_factory'
4
6
 
5
7
  module PubsubClient
6
8
  Error = Class.new(StandardError)
7
9
  ConfigurationError = Class.new(Error)
10
+ CredentialsError = Class.new(Error)
11
+ InvalidTopicError = Class.new(Error)
12
+ InvalidSubscriptionError = Class.new(Error)
8
13
 
9
14
  class << self
10
15
  def stub!
11
- raise ConfigurationError, 'PubsubClient is already configured' if @publisher_factory
16
+ raise ConfigurationError, 'PubsubClient is already configured' if @publisher_factory || @subscriber_factory
12
17
  @publisher_factory = NullPublisherFactory.new
18
+ @subscriber_factory = NullSubscriberFactory.new
13
19
  end
14
20
 
15
- def publish(message, topic, &block)
21
+ # @param message [String] The message to publish.
22
+ # @param topic [String] The name of the topic to publish to.
23
+ def publish(message, topic, attributes = {}, &block)
24
+ ensure_credentials!
25
+
16
26
  @publisher_factory ||= PublisherFactory.new
17
- @publisher_factory.build(topic).publish(message, &block)
27
+ @publisher_factory.build(topic).publish(message, attributes, &block)
28
+ end
29
+
30
+ # @param subscription [String] - The name of the topic to subscribe to.
31
+ def subscriber(subscription)
32
+ ensure_credentials!
33
+
34
+ @subscriber_factory ||= SubscriberFactory.new
35
+ @subscriber_factory.build(subscription)
36
+ end
37
+
38
+ private
39
+
40
+ def ensure_credentials!
41
+ unless ENV['GOOGLE_APPLICATION_CREDENTIALS']
42
+ raise CredentialsError, 'GOOGLE_APPLICATION_CREDENTIALS must be set'
43
+ end
18
44
  end
19
45
  end
20
46
  end
@@ -0,0 +1,34 @@
1
+ # frozen_string_literal: true
2
+
3
+ module PubsubClient
4
+ # A null object to act as a subscriber when clients are in dev or test
5
+ class NullSubscriber
6
+ # This adds a subset of the available methods on the
7
+ # Google::Cloud::PubSub::ReceivedMessage, which is what
8
+ # gets yielded by the subscription when configuring the listener.
9
+ # For a list of methods, see the following link:
10
+ # https://googleapis.dev/ruby/google-cloud-pubsub/latest/Google/Cloud/PubSub/ReceivedMessage.html
11
+ NullResult = Struct.new(:acknowledge!) do
12
+ def data
13
+ '{"key":"value"}'
14
+ end
15
+
16
+ def published_at
17
+ Time.now
18
+ end
19
+ end
20
+
21
+ def listener(*, &block)
22
+ res = NullResult.new
23
+ yield res.data, res
24
+ end
25
+
26
+ def subscribe
27
+ # no-op
28
+ end
29
+
30
+ def on_error(&block)
31
+ # no-op
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,12 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'null_subscriber'
4
+
5
+ module PubsubClient
6
+ # A null object to act as a subscriber factory when clients are in dev or test
7
+ class NullSubscriberFactory
8
+ def build(*)
9
+ NullSubscriber.new
10
+ end
11
+ end
12
+ end
@@ -4,12 +4,13 @@ require 'google/cloud/pubsub'
4
4
 
5
5
  module PubsubClient
6
6
  class Publisher
7
+ # @param topic [Google::Cloud::PubSub::Topic]
7
8
  def initialize(topic)
8
9
  @topic = topic
9
10
  end
10
11
 
11
- def publish(message, &block)
12
- topic.publish_async(message, &block)
12
+ def publish(message, attributes = {}, &block)
13
+ topic.publish_async(message, attributes, &block)
13
14
  end
14
15
 
15
16
  def flush
@@ -10,6 +10,8 @@ module PubsubClient
10
10
  @publishers = {}
11
11
  end
12
12
 
13
+ # @param topic_name [String]
14
+ # @return [Publisher]
13
15
  def build(topic_name)
14
16
  # GRPC fails when attempting to use a connection created in a process that gets
15
17
  # forked with the message
@@ -52,6 +54,7 @@ module PubsubClient
52
54
  def build_publisher(topic_name)
53
55
  pubsub = Google::Cloud::PubSub.new
54
56
  topic = pubsub.topic(topic_name)
57
+ raise InvalidTopicError, "The topic #{topic_name} does not exist" unless topic
55
58
  publisher = Publisher.new(topic)
56
59
 
57
60
  at_exit { publisher.flush }
@@ -0,0 +1,48 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'google/cloud/pubsub'
4
+
5
+ module PubsubClient
6
+ class Subscriber
7
+ DEFAULT_CONCURRENCY = 8
8
+
9
+ # @param subscription [Google::Cloud::PubSub::Subscription]
10
+ def initialize(subscription)
11
+ @subscription = subscription
12
+ end
13
+
14
+ # @param concurrency [Integer] - The number of threads to run the subscriber with. Default is 8.
15
+ # @param auto_ack [Boolean] - Flag to acknowledge the Pub/Sub message. A message must be acked
16
+ # to remove it from the topic. Default is `true`.
17
+ #
18
+ # @return [Google::Cloud::PubSub::Subscriber]
19
+ def listener(concurrency: DEFAULT_CONCURRENCY, auto_ack: true, &block)
20
+ @listener ||= begin
21
+ @subscription.listen(threads: { callback: concurrency }) do |received_message|
22
+ yield received_message.data, received_message
23
+ received_message.acknowledge! if auto_ack
24
+ end
25
+ end
26
+ end
27
+
28
+ def subscribe
29
+ raise ConfigurationError, 'A listener must be configured' unless @listener
30
+
31
+ begin
32
+ @listener.start
33
+
34
+ sleep
35
+ rescue SignalException
36
+ @listener.stop.wait!
37
+ end
38
+ end
39
+
40
+ def on_error(&block)
41
+ raise ConfigurationError, 'A listener must be configured' unless @listener
42
+
43
+ @listener.on_error do |exception|
44
+ yield exception
45
+ end
46
+ end
47
+ end
48
+ end
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'subscriber'
4
+
5
+ module PubsubClient
6
+ class SubscriberFactory
7
+ def initialize
8
+ @subscribers = {}
9
+ end
10
+
11
+ # @param subscription_name [String]
12
+ # @retrun [Subscriber]
13
+ def build(subscription_name)
14
+ if @subscribers.key?(subscription_name)
15
+ raise ConfigurationError, "PubsubClient already subscribed to #{subscription_name}"
16
+ end
17
+
18
+ @subscribers[subscription_name] = build_subscriber(subscription_name)
19
+ end
20
+
21
+ private
22
+
23
+ def build_subscriber(subscription_name)
24
+ pubsub = Google::Cloud::PubSub.new
25
+ subscription = pubsub.subscription(subscription_name)
26
+ raise InvalidSubscriptionError, "The subscription #{subscription_name} does not exist" unless subscription
27
+ Subscriber.new(subscription)
28
+ end
29
+ end
30
+ end
@@ -1,3 +1,3 @@
1
1
  module PubsubClient
2
- VERSION = '1.0.0'
2
+ VERSION = '1.4.1'
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pubsub_client
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0
4
+ version: 1.4.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Apartment List Platforms
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-09-29 00:00:00.000000000 Z
11
+ date: 2020-10-26 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -88,8 +88,12 @@ files:
88
88
  - lib/pubsub_client.rb
89
89
  - lib/pubsub_client/null_publisher.rb
90
90
  - lib/pubsub_client/null_publisher_factory.rb
91
+ - lib/pubsub_client/null_subscriber.rb
92
+ - lib/pubsub_client/null_subscriber_factory.rb
91
93
  - lib/pubsub_client/publisher.rb
92
94
  - lib/pubsub_client/publisher_factory.rb
95
+ - lib/pubsub_client/subscriber.rb
96
+ - lib/pubsub_client/subscriber_factory.rb
93
97
  - lib/pubsub_client/version.rb
94
98
  - pubsub_client.gemspec
95
99
  homepage: https://github.com/apartmentlist/pubsub_client
@@ -114,7 +118,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
114
118
  - !ruby/object:Gem::Version
115
119
  version: '0'
116
120
  requirements: []
117
- rubygems_version: 3.0.8
121
+ rubygems_version: 3.1.4
118
122
  signing_key:
119
123
  specification_version: 4
120
124
  summary: Google Pub/Sub Wrapper