twingly-amqp 4.1.0 → 4.2.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +87 -13
- data/lib/twingly/amqp/connection.rb +9 -4
- data/lib/twingly/amqp/default_exchange_publisher.rb +48 -0
- data/lib/twingly/amqp/null_logger.rb +15 -0
- data/lib/twingly/amqp/ping_options.rb +1 -1
- data/lib/twingly/amqp/pinger.rb +15 -16
- data/lib/twingly/amqp/session.rb +11 -5
- data/lib/twingly/amqp/subscription.rb +4 -5
- data/lib/twingly/amqp/topic_exchange_publisher.rb +48 -0
- data/lib/twingly/amqp/version.rb +1 -1
- data/lib/twingly/amqp.rb +25 -0
- metadata +20 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 858c1c7b49f3bbd776a0aa0af9f070d1c40f31f9
|
4
|
+
data.tar.gz: ee5ee54beff37137f74fe60d0d4e60dc0db4e893
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ecc06c04e469767a9f9a521989f3ae8f85aec7b18da877f255bda9a12db90575e7d4b5f427624f6d5b7f3e7d94047a6e8ce30c007f14c82f0033341d00c2e12c
|
7
|
+
data.tar.gz: dcf7197d39bdff607daaa44957bb817a4a4f983a697949723051376ac4345595356b0c2b48d25081ae05942948ed8b408df0ee9645e00c7354301ad87af15c18
|
data/README.md
CHANGED
@@ -26,21 +26,26 @@ Environment variables:
|
|
26
26
|
* `AMQP_PASSWORD` - Defaults to `guest`
|
27
27
|
* `AMQP_TLS` - Use TLS connection if set
|
28
28
|
|
29
|
+
There are some usage examples in [examples/](examples/).
|
30
|
+
|
29
31
|
### Customize options
|
30
32
|
|
31
|
-
If you don't have the RabbitMQ hosts, user or password in your ENV you can set them
|
33
|
+
If you don't have the RabbitMQ hosts, user or password in your ENV you can set them via the configuration block `Twingly::AMQP.configure`, see example below, before you create an instance of `Subscription` or `Pinger`. All options are sent to `Bunny.new`, see the [documentation][ruby-bunny] for all available options.
|
32
34
|
|
33
|
-
*Options set
|
35
|
+
*Options set via `configure.connection_options` take precedence over the options defined in `ENV`.*
|
34
36
|
|
35
|
-
|
37
|
+
In addition to `connection_options` you may also configure an error logger via `logger`:
|
36
38
|
|
37
39
|
```ruby
|
38
|
-
Twingly::AMQP
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
|
40
|
+
Twingly::AMQP.configure do |config|
|
41
|
+
config.logger = Logger.new(STDOUT)
|
42
|
+
config.connection_options = {
|
43
|
+
hosts: %w(localhost),
|
44
|
+
user: "a-user",
|
45
|
+
pass: "1234",
|
46
|
+
# ...
|
47
|
+
}
|
48
|
+
end
|
44
49
|
```
|
45
50
|
|
46
51
|
[ruby-bunny]: http://rubybunny.info/articles/connecting.html
|
@@ -75,6 +80,45 @@ subscription.each_message do |message| # An instance of Twingly::AMQP::Message
|
|
75
80
|
end
|
76
81
|
```
|
77
82
|
|
83
|
+
### Publish to a queue on the default exchange
|
84
|
+
|
85
|
+
```ruby
|
86
|
+
publisher = Twingly::AMQP::DefaultExchangePublisher.new(queue_name: "my_queue")
|
87
|
+
|
88
|
+
publisher.configure_publish_options do |options|
|
89
|
+
options.content_type = "application/json" # Default
|
90
|
+
options.persistent = true # Default
|
91
|
+
options.expiration = 1000
|
92
|
+
options.priority = 1
|
93
|
+
end
|
94
|
+
|
95
|
+
publisher.publish({ my: "data" })
|
96
|
+
```
|
97
|
+
|
98
|
+
### Publish to a topic exchange
|
99
|
+
|
100
|
+
```ruby
|
101
|
+
# passed to Bunny::Channel#topic
|
102
|
+
exchange_options = {
|
103
|
+
durable: true,
|
104
|
+
}
|
105
|
+
|
106
|
+
publisher = Twingly::AMQP::TopicExchangePublisher.new(
|
107
|
+
exchange_name: "my_exchange",
|
108
|
+
routing_key: "my_key", # Optional
|
109
|
+
opts: exchange_options, # Optional
|
110
|
+
)
|
111
|
+
|
112
|
+
publisher.configure_publish_options do |options|
|
113
|
+
options.content_type = "application/json" # Default
|
114
|
+
options.persistent = true # Default
|
115
|
+
options.expiration = 1000
|
116
|
+
options.priority = 1
|
117
|
+
end
|
118
|
+
|
119
|
+
publisher.publish({ my: "data" })
|
120
|
+
```
|
121
|
+
|
78
122
|
### Ping urls
|
79
123
|
|
80
124
|
```ruby
|
@@ -124,9 +168,18 @@ class UrlCache
|
|
124
168
|
end
|
125
169
|
```
|
126
170
|
|
171
|
+
### Connection instance
|
172
|
+
|
173
|
+
It's possible to get ahold of the current connection, or create a new if no session has been established yet, which may be useful when needing to do things currently not abstracted by the gem.
|
174
|
+
|
175
|
+
```ruby
|
176
|
+
connection = Twingly::AMQP::Connection.instance
|
177
|
+
# connection is a Bunny::Session
|
178
|
+
```
|
179
|
+
|
127
180
|
## Tests
|
128
181
|
|
129
|
-
The tests require a local RabbitMQ server to run.
|
182
|
+
The integration tests run by default and require a local RabbitMQ server to run.
|
130
183
|
|
131
184
|
Run tests with
|
132
185
|
|
@@ -134,12 +187,33 @@ Run tests with
|
|
134
187
|
bundle exec rake
|
135
188
|
```
|
136
189
|
|
190
|
+
Run just the unit tests with
|
191
|
+
|
192
|
+
```shell
|
193
|
+
bundle exec rake spec:unit
|
194
|
+
```
|
195
|
+
|
196
|
+
### RuboCop
|
197
|
+
|
198
|
+
To run static code analysis:
|
199
|
+
|
200
|
+
bundle exec rubocop
|
201
|
+
|
202
|
+
# optionally on single file(s)
|
203
|
+
bundle exec rubocop lib/twingly/amqp/*
|
204
|
+
|
137
205
|
## Release workflow
|
138
206
|
|
139
|
-
|
207
|
+
* Bump the version in `lib/twingly/amqp/version.rb` in a commit, no need to push (the release task does that).
|
208
|
+
|
209
|
+
* Build and [publish](http://guides.rubygems.org/publishing/) the gem. This will create the proper tag in git, push the commit and tag and upload to RubyGems.
|
210
|
+
|
211
|
+
bundle exec rake release
|
212
|
+
|
213
|
+
* If you are not logged in as [twingly][twingly-rubygems] with ruby gems, the rake task will fail and tell you to set credentials via `gem push`, do that and run the `release` task again. It will be okay.
|
140
214
|
|
141
|
-
|
215
|
+
* Update the changelog with [GitHub Changelog Generator](https://github.com/skywinder/github-changelog-generator/) (`gem install github_changelog_generator` if you don't have it, set `CHANGELOG_GITHUB_TOKEN` to a personal access token to avoid rate limiting by GitHub). This command will update `CHANGELOG.md`, commit and push manually.
|
142
216
|
|
143
|
-
|
217
|
+
github_changelog_generator -u twingly -p twingly-amqp
|
144
218
|
|
145
219
|
[twingly-rubygems]: https://rubygems.org/profiles/twingly
|
@@ -1,4 +1,4 @@
|
|
1
|
-
require "twingly/amqp
|
1
|
+
require "twingly/amqp"
|
2
2
|
|
3
3
|
module Twingly
|
4
4
|
module AMQP
|
@@ -7,17 +7,22 @@ module Twingly
|
|
7
7
|
|
8
8
|
@@lock = Mutex.new
|
9
9
|
@@instance = nil
|
10
|
-
@@options = {}
|
11
10
|
|
12
11
|
def self.options=(options)
|
13
|
-
|
12
|
+
warn "[DEPRECATION] `options=` is deprecated. " \
|
13
|
+
"Please use configuration block: `Twingly::AMQP.configure` instead."
|
14
|
+
|
15
|
+
Twingly::AMQP.configure do |config|
|
16
|
+
config.connection_options = options
|
17
|
+
end
|
14
18
|
end
|
15
19
|
|
16
20
|
def self.instance
|
17
21
|
return @@instance if @@instance
|
18
22
|
@@lock.synchronize do
|
19
23
|
return @@instance if @@instance
|
20
|
-
|
24
|
+
options = Twingly::AMQP.configuration.connection_options
|
25
|
+
@@instance = Session.new(options).connection
|
21
26
|
end
|
22
27
|
end
|
23
28
|
end
|
@@ -0,0 +1,48 @@
|
|
1
|
+
require "twingly/amqp/connection"
|
2
|
+
require "json"
|
3
|
+
require "ostruct"
|
4
|
+
|
5
|
+
module Twingly
|
6
|
+
module AMQP
|
7
|
+
class DefaultExchangePublisher
|
8
|
+
def initialize(queue_name:, connection: nil)
|
9
|
+
options.routing_key = queue_name
|
10
|
+
|
11
|
+
connection ||= Connection.instance
|
12
|
+
@exchange = connection.create_channel.default_exchange
|
13
|
+
end
|
14
|
+
|
15
|
+
def publish(message)
|
16
|
+
raise ArgumentError unless message.respond_to?(:to_h)
|
17
|
+
|
18
|
+
payload = message.to_h.to_json
|
19
|
+
opts = options.to_h
|
20
|
+
@exchange.publish(payload, opts)
|
21
|
+
end
|
22
|
+
|
23
|
+
# only used by tests to avoid sleeping
|
24
|
+
def publish_with_confirm(message)
|
25
|
+
channel = @exchange.channel
|
26
|
+
channel.confirm_select unless channel.using_publisher_confirmations?
|
27
|
+
|
28
|
+
publish(message)
|
29
|
+
|
30
|
+
@exchange.wait_for_confirms
|
31
|
+
end
|
32
|
+
|
33
|
+
def configure_publish_options
|
34
|
+
yield options
|
35
|
+
end
|
36
|
+
|
37
|
+
private
|
38
|
+
|
39
|
+
def options
|
40
|
+
@options ||=
|
41
|
+
OpenStruct.new(
|
42
|
+
content_type: "application/json",
|
43
|
+
persistent: true,
|
44
|
+
)
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
@@ -23,7 +23,7 @@ module Twingly
|
|
23
23
|
def validate
|
24
24
|
missing_keys = to_h.select { |_, value| value.to_s.empty? }.keys
|
25
25
|
if missing_keys.any?
|
26
|
-
|
26
|
+
raise ArgumentError, "Required options not set: #{missing_keys}"
|
27
27
|
end
|
28
28
|
end
|
29
29
|
|
data/lib/twingly/amqp/pinger.rb
CHANGED
@@ -1,19 +1,22 @@
|
|
1
1
|
require "twingly/amqp/connection"
|
2
2
|
require "twingly/amqp/ping_options"
|
3
|
+
require "twingly/amqp/default_exchange_publisher"
|
3
4
|
require "json"
|
4
5
|
|
5
6
|
module Twingly
|
6
7
|
module AMQP
|
7
8
|
class Pinger
|
8
|
-
def initialize(queue_name:, ping_expiration: nil, url_cache: NullCache, connection: nil)
|
9
|
-
@
|
10
|
-
@url_cache = url_cache
|
11
|
-
|
9
|
+
def initialize(queue_name:, ping_expiration: nil, url_cache: NullCache, connection: nil, confirm_publish: false)
|
10
|
+
@url_cache = url_cache
|
12
11
|
connection ||= Connection.instance
|
13
|
-
@channel = connection.create_channel
|
14
12
|
|
15
|
-
@
|
13
|
+
@publisher = DefaultExchangePublisher.new(queue_name: queue_name, connection: connection)
|
14
|
+
@publisher.configure_publish_options do |options|
|
15
|
+
options.expiration = ping_expiration
|
16
|
+
end
|
17
|
+
|
16
18
|
@default_ping_options = PingOptions.new
|
19
|
+
@confirm_publish = confirm_publish
|
17
20
|
end
|
18
21
|
|
19
22
|
def ping(urls, options_hash = {})
|
@@ -39,17 +42,13 @@ module Twingly
|
|
39
42
|
private
|
40
43
|
|
41
44
|
def publish(url, options)
|
42
|
-
payload = message(url, options)
|
43
|
-
@channel.default_exchange.publish(payload, amqp_publish_options)
|
44
|
-
end
|
45
|
+
payload = message(url, options)
|
45
46
|
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
|
51
|
-
expiration: @ping_expiration,
|
52
|
-
}
|
47
|
+
if @confirm_publish
|
48
|
+
@publisher.publish_with_confirm(payload)
|
49
|
+
else
|
50
|
+
@publisher.publish(payload)
|
51
|
+
end
|
53
52
|
end
|
54
53
|
|
55
54
|
def message(url, options)
|
data/lib/twingly/amqp/session.rb
CHANGED
@@ -5,9 +5,9 @@ module Twingly
|
|
5
5
|
class Session
|
6
6
|
attr_reader :connection, :options
|
7
7
|
|
8
|
-
DEFAULT_USER = "guest"
|
9
|
-
DEFAULT_PASS = "guest"
|
10
|
-
DEFAULT_HOSTS = ["localhost"]
|
8
|
+
DEFAULT_USER = "guest".freeze
|
9
|
+
DEFAULT_PASS = "guest".freeze
|
10
|
+
DEFAULT_HOSTS = ["localhost"].freeze
|
11
11
|
|
12
12
|
def initialize(options = {})
|
13
13
|
@options = options
|
@@ -27,6 +27,10 @@ module Twingly
|
|
27
27
|
options[:pass] ||= password_from_env
|
28
28
|
options[:hosts] ||= hosts_from_env
|
29
29
|
|
30
|
+
# Used as a workaround for
|
31
|
+
# https://github.com/ruby-amqp/bunny/issues/446
|
32
|
+
options[:hosts] = options.fetch(:hosts).dup
|
33
|
+
|
30
34
|
default_connection_options.merge(options)
|
31
35
|
end
|
32
36
|
|
@@ -37,7 +41,7 @@ module Twingly
|
|
37
41
|
end
|
38
42
|
|
39
43
|
def tls?
|
40
|
-
ENV.
|
44
|
+
ENV.key?("AMQP_TLS")
|
41
45
|
end
|
42
46
|
|
43
47
|
def user_from_env
|
@@ -50,7 +54,9 @@ module Twingly
|
|
50
54
|
|
51
55
|
def hosts_from_env
|
52
56
|
# Matches env keys like `RABBITMQ_01_HOST`
|
53
|
-
environment_keys_with_host = ENV.keys.select
|
57
|
+
environment_keys_with_host = ENV.keys.select do |key|
|
58
|
+
key =~ /^rabbitmq_\d+_host$/i
|
59
|
+
end
|
54
60
|
hosts = environment_keys_with_host.map { |key| ENV[key] }
|
55
61
|
|
56
62
|
return DEFAULT_HOSTS if hosts.empty?
|
@@ -20,8 +20,8 @@ module Twingly
|
|
20
20
|
@queue.bind(exchange, routing_key: @routing_key)
|
21
21
|
end
|
22
22
|
|
23
|
-
@before_handle_message_callback =
|
24
|
-
@on_exception_callback =
|
23
|
+
@before_handle_message_callback = proc {}
|
24
|
+
@on_exception_callback = proc {}
|
25
25
|
end
|
26
26
|
|
27
27
|
def each_message(&block)
|
@@ -41,9 +41,7 @@ module Twingly
|
|
41
41
|
end
|
42
42
|
|
43
43
|
# The consumer isn't blocking, so we wait here
|
44
|
-
until cancel?
|
45
|
-
sleep 0.5
|
46
|
-
end
|
44
|
+
sleep 0.5 until cancel?
|
47
45
|
|
48
46
|
consumer.cancel
|
49
47
|
end
|
@@ -70,6 +68,7 @@ module Twingly
|
|
70
68
|
channel = connection.create_channel(nil, @consumer_threads)
|
71
69
|
channel.prefetch(@prefetch)
|
72
70
|
channel.on_uncaught_exception do |exception, _|
|
71
|
+
Twingly::AMQP.configuration.logger.error(exception)
|
73
72
|
@on_exception_callback.call(exception)
|
74
73
|
end
|
75
74
|
channel
|
@@ -0,0 +1,48 @@
|
|
1
|
+
require "twingly/amqp/connection"
|
2
|
+
require "json"
|
3
|
+
require "ostruct"
|
4
|
+
|
5
|
+
module Twingly
|
6
|
+
module AMQP
|
7
|
+
class TopicExchangePublisher
|
8
|
+
def initialize(exchange_name:, routing_key: nil, connection: nil, opts: {})
|
9
|
+
options.routing_key = routing_key
|
10
|
+
|
11
|
+
connection ||= Connection.instance
|
12
|
+
@exchange = connection.create_channel.topic(exchange_name, opts)
|
13
|
+
end
|
14
|
+
|
15
|
+
def publish(message)
|
16
|
+
raise ArgumentError unless message.respond_to?(:to_h)
|
17
|
+
|
18
|
+
payload = message.to_h.to_json
|
19
|
+
opts = options.to_h
|
20
|
+
@exchange.publish(payload, opts)
|
21
|
+
end
|
22
|
+
|
23
|
+
# only used by tests to avoid sleeping
|
24
|
+
def publish_with_confirm(message)
|
25
|
+
channel = @exchange.channel
|
26
|
+
channel.confirm_select unless channel.using_publisher_confirmations?
|
27
|
+
|
28
|
+
publish(message)
|
29
|
+
|
30
|
+
@exchange.wait_for_confirms
|
31
|
+
end
|
32
|
+
|
33
|
+
def configure_publish_options
|
34
|
+
yield options
|
35
|
+
end
|
36
|
+
|
37
|
+
private
|
38
|
+
|
39
|
+
def options
|
40
|
+
@options ||=
|
41
|
+
OpenStruct.new(
|
42
|
+
content_type: "application/json",
|
43
|
+
persistent: true,
|
44
|
+
)
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
data/lib/twingly/amqp/version.rb
CHANGED
data/lib/twingly/amqp.rb
CHANGED
@@ -4,3 +4,28 @@ require "twingly/amqp/connection"
|
|
4
4
|
require "twingly/amqp/subscription"
|
5
5
|
require "twingly/amqp/ping_options"
|
6
6
|
require "twingly/amqp/pinger"
|
7
|
+
require "twingly/amqp/default_exchange_publisher"
|
8
|
+
require "twingly/amqp/topic_exchange_publisher"
|
9
|
+
require "twingly/amqp/null_logger"
|
10
|
+
|
11
|
+
require "ostruct"
|
12
|
+
|
13
|
+
module Twingly
|
14
|
+
module AMQP
|
15
|
+
class << self
|
16
|
+
attr_writer :configuration
|
17
|
+
end
|
18
|
+
|
19
|
+
def self.configuration
|
20
|
+
@configuration ||=
|
21
|
+
OpenStruct.new(
|
22
|
+
logger: NullLogger.new,
|
23
|
+
connection_options: {},
|
24
|
+
)
|
25
|
+
end
|
26
|
+
|
27
|
+
def self.configure
|
28
|
+
yield configuration
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: twingly-amqp
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 4.1
|
4
|
+
version: 4.2.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Twingly AB
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2016-11-17 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bunny
|
@@ -52,6 +52,20 @@ dependencies:
|
|
52
52
|
- - "~>"
|
53
53
|
- !ruby/object:Gem::Version
|
54
54
|
version: '10.0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: rubocop
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '0'
|
55
69
|
description: Publish and subscribe to messages via RabbitMQ
|
56
70
|
email:
|
57
71
|
- support@twingly.com
|
@@ -62,11 +76,14 @@ files:
|
|
62
76
|
- README.md
|
63
77
|
- lib/twingly/amqp.rb
|
64
78
|
- lib/twingly/amqp/connection.rb
|
79
|
+
- lib/twingly/amqp/default_exchange_publisher.rb
|
65
80
|
- lib/twingly/amqp/message.rb
|
81
|
+
- lib/twingly/amqp/null_logger.rb
|
66
82
|
- lib/twingly/amqp/ping_options.rb
|
67
83
|
- lib/twingly/amqp/pinger.rb
|
68
84
|
- lib/twingly/amqp/session.rb
|
69
85
|
- lib/twingly/amqp/subscription.rb
|
86
|
+
- lib/twingly/amqp/topic_exchange_publisher.rb
|
70
87
|
- lib/twingly/amqp/version.rb
|
71
88
|
homepage: https://github.com/twingly/twingly-amqp
|
72
89
|
licenses: []
|
@@ -87,7 +104,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
87
104
|
version: '0'
|
88
105
|
requirements: []
|
89
106
|
rubyforge_project:
|
90
|
-
rubygems_version: 2.5.
|
107
|
+
rubygems_version: 2.4.5.1
|
91
108
|
signing_key:
|
92
109
|
specification_version: 4
|
93
110
|
summary: Ruby library for talking to RabbitMQ
|