pwwka 0.8.0 → 0.9.0.RC1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/.travis.yml +7 -3
- data/CONTRIBUTING.md +17 -2
- data/Gemfile.lock +29 -7
- data/README.md +144 -128
- data/docker-compose.yml +11 -0
- data/lib/pwwka/configuration.rb +6 -0
- data/lib/pwwka/receiver.rb +19 -2
- data/lib/pwwka/test_handler.rb +1 -0
- data/lib/pwwka/transmitter.rb +1 -0
- data/lib/pwwka/version.rb +1 -1
- data/pwwka.gemspec +3 -0
- data/spec/integration/interrupted_receivers_spec.rb +47 -0
- data/spec/integration/send_and_receive_spec.rb +98 -0
- data/spec/integration/support/integration_test_helpers.rb +5 -0
- data/spec/integration/support/integration_test_setup.rb +40 -0
- data/spec/integration/support/logging_receiver.rb +10 -0
- data/spec/integration/test_handler_spec.rb +78 -0
- data/spec/integration/unhandled_errors_in_receivers_spec.rb +115 -0
- data/spec/{handling_spec.rb → legacy/handling_spec.rb} +1 -1
- data/spec/{receiver_spec.rb → legacy/receiver_spec.rb} +2 -1
- data/spec/{send_message_async_job_spec.rb → legacy/send_message_async_job_spec.rb} +1 -1
- data/spec/{transmitter_spec.rb → legacy/transmitter_spec.rb} +1 -1
- data/spec/spec_helper.rb +32 -6
- data/spec/support/test_configuration.rb +10 -0
- data/spec/unit/channel_connector_spec.rb +40 -0
- data/spec/{logging_spec.rb → unit/logging_spec.rb} +0 -0
- data/spec/{message_queuer_spec.rb → unit/message_queuer_spec.rb} +4 -4
- data/spec/{queue_resque_job_handler_spec.rb → unit/queue_resque_job_handler_spec.rb} +0 -0
- data/spec/unit/test_handler_message_spec.rb +26 -0
- data/spec/unit/transmitter_spec.rb +229 -0
- metadata +83 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5e240642fe92b56915ca83419b9127de69b9f3b5
|
4
|
+
data.tar.gz: f88565f26342b93c80896adcc48564c83b033c0a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: a99d35a04c2b168a53c7ad26b3e891ed939deb7e511b0d1608f21291ccaeca78a82deec114a58dac1dc9ddf525ab8cc0e20d18d32437aabc4a7f40651f63b593
|
7
|
+
data.tar.gz: f77dbabed9e341659670e523829c05ad2a6cb46d6c1c5f3492d73ad5c4e3d15b64f1d55aff0116e3381c1afe0f4000c269de0923fc15dc0b27930aa94227b221
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
data/CONTRIBUTING.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
# Contributing
|
2
|
-
Thanks for using and improving *pwwka*! If you'd like to help out, check out [the project's issues list](https://github.com/stitchfix/pwwka/issues) for ideas on what could be improved.
|
2
|
+
Thanks for using and improving *pwwka*! If you'd like to help out, check out [the project's issues list](https://github.com/stitchfix/pwwka/issues) for ideas on what could be improved.
|
3
3
|
|
4
4
|
We're actively using Pwwka in production here at [Stitch Fix](http://technology.stitchfix.com/) and look forward to seeing Pwwka grow and improve with your help. Contributions are warmly welcomed.
|
5
5
|
|
@@ -18,6 +18,21 @@ If there's an idea you'd like to propose, or a design change, feel free to file
|
|
18
18
|
|
19
19
|
## General Guidelines
|
20
20
|
|
21
|
-
* You must be running RabbitMQ locally on the default port in order for the tests to work.
|
22
21
|
* When in doubt, test it. If you can't test it, re-think what you are doing.
|
23
22
|
* Code formatting and internal application architecture should be consistent.
|
23
|
+
|
24
|
+
## Testing
|
25
|
+
|
26
|
+
The tests assume that:
|
27
|
+
|
28
|
+
* Rabbit is running on port 10001
|
29
|
+
* Redis is running on port 10003
|
30
|
+
|
31
|
+
You can achieve this by using Docker and running `docker-compose up` in the root of this directory. If you don't want to use Docker,
|
32
|
+
that's fine. You'll need to set `PWWKA_RESQUE_REDIS_PORT` and `PWWKA_RABBIT_PORT` in your environment to the ports where those services
|
33
|
+
are running.
|
34
|
+
|
35
|
+
Tests in `spec/integration` are end-to-end tests that use Rabbita and attempt to assert behavior from the point of view of the application
|
36
|
+
owner. If you write tests here, depend on as few of Pwwka's internals as possible, and *no mocking of anything*.
|
37
|
+
|
38
|
+
Tests in `spec/unit` are more traditional unit tests and *should not require Rabbit or Redis* to be running.
|
data/Gemfile.lock
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
pwwka (0.
|
4
|
+
pwwka (0.9.0.RC1)
|
5
5
|
activemodel
|
6
6
|
activesupport
|
7
7
|
bunny
|
@@ -10,19 +10,21 @@ PATH
|
|
10
10
|
GEM
|
11
11
|
remote: https://www.rubygems.org/
|
12
12
|
specs:
|
13
|
-
activemodel (5.0.
|
14
|
-
activesupport (= 5.0.
|
15
|
-
activesupport (5.0.
|
13
|
+
activemodel (5.0.1)
|
14
|
+
activesupport (= 5.0.1)
|
15
|
+
activesupport (5.0.1)
|
16
16
|
concurrent-ruby (~> 1.0, >= 1.0.2)
|
17
17
|
i18n (~> 0.7)
|
18
18
|
minitest (~> 5.1)
|
19
19
|
tzinfo (~> 1.1)
|
20
20
|
amq-protocol (2.0.1)
|
21
|
-
bunny (2.6.
|
21
|
+
bunny (2.6.2)
|
22
22
|
amq-protocol (>= 2.0.1)
|
23
|
-
concurrent-ruby (1.0.
|
23
|
+
concurrent-ruby (1.0.4)
|
24
24
|
diff-lcs (1.2.5)
|
25
|
-
|
25
|
+
docile (1.1.5)
|
26
|
+
i18n (0.8.0)
|
27
|
+
json (1.8.3)
|
26
28
|
minitest (5.10.1)
|
27
29
|
mono_logger (1.1.0)
|
28
30
|
multi_json (1.11.2)
|
@@ -39,6 +41,16 @@ GEM
|
|
39
41
|
redis-namespace (~> 1.3)
|
40
42
|
sinatra (>= 0.9.2)
|
41
43
|
vegas (~> 0.1.2)
|
44
|
+
resque-retry (1.5.0)
|
45
|
+
resque (~> 1.25)
|
46
|
+
resque-scheduler (~> 4.0)
|
47
|
+
resque-scheduler (4.2.0)
|
48
|
+
mono_logger (~> 1.0)
|
49
|
+
redis (~> 3.0)
|
50
|
+
resque (~> 1.25)
|
51
|
+
rufus-scheduler (~> 3.2)
|
52
|
+
resqutils (1.2.0)
|
53
|
+
resque
|
42
54
|
rspec (3.4.0)
|
43
55
|
rspec-core (~> 3.4.0)
|
44
56
|
rspec-expectations (~> 3.4.0)
|
@@ -52,6 +64,13 @@ GEM
|
|
52
64
|
diff-lcs (>= 1.2.0, < 2.0)
|
53
65
|
rspec-support (~> 3.4.0)
|
54
66
|
rspec-support (3.4.1)
|
67
|
+
rufus-scheduler (3.3.1)
|
68
|
+
tzinfo
|
69
|
+
simplecov (0.12.0)
|
70
|
+
docile (~> 1.1.0)
|
71
|
+
json (>= 1.8, < 3)
|
72
|
+
simplecov-html (~> 0.10.0)
|
73
|
+
simplecov-html (0.10.0)
|
55
74
|
sinatra (1.4.7)
|
56
75
|
rack (~> 1.5)
|
57
76
|
rack-protection (~> 1.4)
|
@@ -70,7 +89,10 @@ DEPENDENCIES
|
|
70
89
|
pwwka!
|
71
90
|
rake
|
72
91
|
resque
|
92
|
+
resque-retry
|
93
|
+
resqutils
|
73
94
|
rspec
|
95
|
+
simplecov
|
74
96
|
|
75
97
|
BUNDLED WITH
|
76
98
|
1.13.6
|
data/README.md
CHANGED
@@ -8,31 +8,52 @@ Pronounced "Poo-ka" |ˈpo͞okə|
|
|
8
8
|
---
|
9
9
|
[![Build Status](https://travis-ci.org/stitchfix/pwwka.svg?branch=add_travis_yml)](https://travis-ci.org/stitchfix/pwwka)
|
10
10
|
|
11
|
-
|
11
|
+
Provides the means to both send and handle messages on an exchange of a RabbitMQ server. In a sense, this provides the RabbitMQ equivalent
|
12
|
+
of `Resque.enqueue` and `SomeResqueJob.perform`.
|
12
13
|
|
13
|
-
|
14
|
-
* Receive messages from the exchange and tell the exchange whether or not the message has been acknowledged.
|
14
|
+
## Set Up
|
15
15
|
|
16
|
-
|
16
|
+
In your `Gemfile`:
|
17
17
|
|
18
|
-
|
18
|
+
```ruby
|
19
|
+
gem 'pwwka'
|
20
|
+
```
|
19
21
|
|
20
|
-
|
22
|
+
(of course, you can always run `gem install pwwka` to install it without Bundler)
|
21
23
|
|
22
|
-
|
24
|
+
To run applications locally, you will need Rabbit installed. The [installation guide](https://www.rabbitmq.com/download.html) is a great
|
25
|
+
place to start. This repo includes a `docker-compose.yml` file which will run Rabbit inside a Docker container. It's used by the tests,
|
26
|
+
but you can use that, too.
|
23
27
|
|
24
|
-
|
25
|
-
* admin-app receives that message and updates its client index
|
26
|
-
* email-app receives that message and sends a welcome email to the client
|
28
|
+
### Configuration
|
27
29
|
|
28
|
-
|
30
|
+
Somewhere in your app, run the following code (in Rails, this would be `config/initializers/pwwka.rb`):
|
29
31
|
|
30
|
-
|
32
|
+
```ruby
|
33
|
+
require 'pwwka'
|
34
|
+
Pwwka.configure do |config|
|
35
|
+
config.rabbit_mq_host = ENV['RABBITMQ_URL']
|
36
|
+
config.topic_exchange_name = "mycompany-topics-#{Rails.env}"
|
37
|
+
config.delayed_exchange_name = "mycompany-topics-#{Rails.env}"
|
38
|
+
config.options = {allow_delayed: true}
|
39
|
+
config.requeue_on_error = true
|
40
|
+
end
|
41
|
+
```
|
31
42
|
|
32
|
-
|
33
|
-
|
34
|
-
|
43
|
+
Note that the absence of `RABBITMQ_URL` in your environment will cause the underlying RabbitMQ library to use the defaults. If you aren't
|
44
|
+
using the defaults, set that environment variable to something like this:
|
45
|
+
|
46
|
+
```
|
47
|
+
amqp://«user»:«password»@«host»:«port»/«vhost»
|
48
|
+
```
|
49
|
+
|
50
|
+
The defaults should be `amqp://guest:guest@localhost:5672/`, i.e.:
|
35
51
|
|
52
|
+
* user: guest
|
53
|
+
* password: guest
|
54
|
+
* host: localhost
|
55
|
+
* port: 5672
|
56
|
+
* vhost: `/`
|
36
57
|
|
37
58
|
## Setting it up
|
38
59
|
|
@@ -52,24 +73,13 @@ Add to your `Gemfile`:
|
|
52
73
|
gem 'pwwka'
|
53
74
|
```
|
54
75
|
|
76
|
+
## Using Pwwka
|
55
77
|
|
56
|
-
|
78
|
+
Pwwka provides the ability to send a message into Rabbit as well a the ability to receive/handle a message. Your app can do both of these
|
79
|
+
things if it needs to.
|
57
80
|
|
58
|
-
Connect to your RabbitMQ instance using the url and choose a name for your
|
59
|
-
topic exchange.
|
60
81
|
|
61
|
-
|
62
|
-
|
63
|
-
```ruby
|
64
|
-
require 'pwwka'
|
65
|
-
Pwwka.configure do |config|
|
66
|
-
config.rabbit_mq_host = ENV['RABBITMQ_URL']
|
67
|
-
config.topic_exchange_name = "mycompany-topics-#{Rails.env}"
|
68
|
-
config.options = {allow_delayed: true}
|
69
|
-
end
|
70
|
-
```
|
71
|
-
|
72
|
-
## Sending a message
|
82
|
+
### Sending a message
|
73
83
|
|
74
84
|
You can send any kind of message using `Pwwka::Transmitter.send_message!`:
|
75
85
|
|
@@ -78,34 +88,29 @@ payload = {client_id: '13452564'}
|
|
78
88
|
routing_key = 'sf.clients.client.created'
|
79
89
|
Pwwka::Transmitter.send_message!(payload, routing_key)
|
80
90
|
```
|
81
|
-
The payload should be a simple hash containing primitives. Don't send objects because the payload will be converted to JSON for sending. This will blow up if an exception is raised. If you want the exception to be rescued and logged, use this instead:
|
82
91
|
|
83
|
-
|
84
|
-
Pwwka::Transmitter.send_message_safely(payload, routing_key)
|
85
|
-
```
|
92
|
+
The payload should be a simple hash containing primitives. Don't send objects because the payload will be converted to JSON for sending.
|
86
93
|
|
87
|
-
You can also use the two convenience methods for sending a message. To include these methods
|
88
|
-
in your class use:
|
89
94
|
|
90
|
-
|
91
|
-
|
92
|
-
|
95
|
+
#### Error Handling
|
96
|
+
|
97
|
+
`Pwwka::Transmitter.send_message!` accepts several strategies for handling errors, passed in using the `on_error` parameter:
|
93
98
|
|
94
|
-
|
99
|
+
* `:raise` - Log the error and raise the exception received from Bunny. (default strategy)
|
100
|
+
* `:ignore` - Log the error and return false.
|
101
|
+
* `:resque` - Log the error and return false. Also, enqueue a job with Resque to send the message. See `send_message_async` below. **Note, this doesn't guarantee the message will actually be sent—it just guarantees an attempt is made to queue a Resque job [which could fail]**
|
102
|
+
|
103
|
+
Example usage:
|
95
104
|
|
96
105
|
```ruby
|
97
|
-
|
106
|
+
payload = {client_id: '13452564'}
|
107
|
+
routing_key = 'sf.clients.client.created'
|
108
|
+
Pwwka::Transmitter.send_message!(payload, routing_key, on_error: :ignore)
|
98
109
|
```
|
99
110
|
|
100
|
-
### Error Handling
|
101
111
|
|
102
|
-
|
112
|
+
#### Delayed Messages
|
103
113
|
|
104
|
-
* `:raise`: Log the error and raise the exception received from Bunny. (default strategy)
|
105
|
-
* `:ignore`: Log the error and return false.
|
106
|
-
* `:resque`: Log the error and return false. Also, enqueue a job with Resque to send the message. See `send_message_async` below. **Note, this doesn't guarantee the message will actually be sent—it just guarantees an attempt is made to queue a Resque job [which could fail]**
|
107
|
-
|
108
|
-
### Delayed Messages
|
109
114
|
You might want to delay sending a message (for example, if you have just created a database
|
110
115
|
record and a race condition keeps catching you out). In that case you can use delayed message
|
111
116
|
options:
|
@@ -121,7 +126,7 @@ Pwwka::Transmitter.send_message!(payload, routing_key, delayed: true, delay_by:
|
|
121
126
|
These extra arguments work for all message sending methods - the safe ones, the handling, and the `message_queuer` methods (see below).
|
122
127
|
|
123
128
|
|
124
|
-
|
129
|
+
#### Sending message Async with Resque
|
125
130
|
|
126
131
|
To enqueue a message in a background Resque job, use `Transmitter.send_message_async`
|
127
132
|
```ruby
|
@@ -132,14 +137,14 @@ If `Resque::Plugins::ExponentialBackoff` is available, the job will use it.
|
|
132
137
|
Customize the backoff intervals using the configuration `send_message_resque_backoff_strategy`.
|
133
138
|
The default backoff will retry quickly in case of an intermittent glitch, and then every ten
|
134
139
|
minutes for half an hour.
|
135
|
-
|
140
|
+
|
136
141
|
The name of the queue created is `pwwka_send_message_async`.
|
137
142
|
|
143
|
+
#### Message Queuer
|
138
144
|
|
139
|
-
### Message Queuer
|
140
145
|
You can queue up messages and send them in a batch. This is most useful when multiple messages
|
141
146
|
need to sent from within a transaction block.
|
142
|
-
|
147
|
+
|
143
148
|
For example:
|
144
149
|
|
145
150
|
```ruby
|
@@ -156,15 +161,17 @@ end
|
|
156
161
|
message_queuer.send_messages_safely
|
157
162
|
```
|
158
163
|
|
159
|
-
|
164
|
+
### Receiving messages
|
160
165
|
|
161
|
-
The message-handler comes with a rake task you can use in your Procfile to start up your message handler worker:
|
166
|
+
The message-handler comes with a rake task you can use (e.g. in your Procfile) to start up your message handler worker:
|
162
167
|
|
163
168
|
```ruby
|
164
169
|
message_handler: rake message_handler:receive HANDLER_KLASS=ClientIndexMessageHandler QUEUE_NAME=adminapp_style_index ROUTING_KEY='client.#.updated'
|
165
170
|
```
|
166
171
|
|
167
|
-
|
172
|
+
It requires some environment variables to work:
|
173
|
+
|
174
|
+
* `HANDLER_KLASS` (required) refers to the class you have to write in your app (equivalent to a `job` in Resque)
|
168
175
|
* `QUEUE_NAME` (required) we must use named queues - see below
|
169
176
|
* `ROUTING_KEY` (optional) defaults to `#.#` (all messages)
|
170
177
|
|
@@ -174,7 +181,7 @@ You'll also need to bring the Rake task into your app. For Rails, you'll need t
|
|
174
181
|
require 'pwwka/tasks'
|
175
182
|
```
|
176
183
|
|
177
|
-
|
184
|
+
#### Queues - what messages will your queue receive
|
178
185
|
|
179
186
|
It depends on your `routing_key`. If you set your routing key to `#.#` (the default) it will receive all the messages. The `#` is a wildcard so if you set it to `client.#` it will receive any message with `client.` at the beginning. The exchange registers the queue's name and routing key so it knows what messages the queue is supposed to receive. A named queue will receive each message it expects to get once and only once.
|
180
187
|
|
@@ -182,11 +189,11 @@ The available wildcards are as follows (and are not intuitive):
|
|
182
189
|
* `*` (star) can substitute for **exactly one word**.
|
183
190
|
* `#` (hash) can substitute for zero or more words.
|
184
191
|
|
185
|
-
__A note on re-queuing:__ At the moment messages that raise an error on receipt are marked 'not acknowledged, don't resend', and the failure message is logged.
|
192
|
+
__A note on re-queuing:__ At the moment messages that raise an error on receipt are marked 'not acknowledged, don't resend', and the failure message is logged. You can configure a single retry by setting the configuration option `requeue_on_error`. Note that all unacknowledged messages will be resent when the worker is restarted.
|
186
193
|
|
187
194
|
__Spinning up some more handlers to handle the load:__ Since each named queue will receive each message only once you can spin up multiple process using the *same named queue* and they will share the messages between them. If you spin up three processes each will receive roughly one third of the messages, but each message will still only be received once.
|
188
195
|
|
189
|
-
|
196
|
+
#### Handlers - The class that handles received messages
|
190
197
|
|
191
198
|
Handlers are simple classes that must respond to `self.handle!`. The receiver will send the handler three arguments:
|
192
199
|
|
@@ -199,17 +206,13 @@ Here is an example:
|
|
199
206
|
```ruby
|
200
207
|
class ClientIndexMessageHandler
|
201
208
|
|
202
|
-
def initialize(payload)
|
203
|
-
@payload = payload
|
204
|
-
end
|
205
|
-
|
206
209
|
def self.handle!(delivery_info, properties, payload)
|
207
|
-
|
208
|
-
handler = new(payload)
|
209
|
-
handler.do_a_thing
|
210
|
+
handler.do_a_thing(payload)
|
210
211
|
end
|
211
212
|
|
212
|
-
|
213
|
+
private
|
214
|
+
|
215
|
+
def self.do_a_thing(payload)
|
213
216
|
###
|
214
217
|
# some stuff that is being done
|
215
218
|
###
|
@@ -217,37 +220,34 @@ class ClientIndexMessageHandler
|
|
217
220
|
|
218
221
|
end
|
219
222
|
```
|
220
|
-
#### Handling Errors
|
221
223
|
|
222
|
-
|
223
|
-
it's dropped on the floor and likely won't have been completely processed.
|
224
|
+
#### Errors From Your Handler
|
224
225
|
|
225
|
-
|
226
|
+
By default, all unhandled errors will crash your handler. This is good, because it allows you to recover from most intermittent things. Just be aware of this when configuring your handler so that it gets
|
227
|
+
restarted after a crash.
|
226
228
|
|
227
|
-
|
228
|
-
require 'pwwka'
|
229
|
-
Pwwka.configure do |config|
|
230
|
-
|
231
|
-
# ...
|
232
|
-
|
233
|
-
config.requeue_on_error = true
|
234
|
-
end
|
235
|
-
```
|
229
|
+
What happens to the message you received during the error depends:
|
236
230
|
|
237
|
-
|
238
|
-
|
239
|
-
|
240
|
-
requeued
|
231
|
+
* If the error is not a `StandardError` or a subclass, the message will not be ack'ed and will be waiting on the queue for you when you next fetch a message
|
232
|
+
* If the errors *is* a `StandardError` or a subclass, the message will be ack'ed and removed from the queue.
|
233
|
+
- By default, the message is not re-queued and is essentially dropped on the floor. Its payload is logged, so you can recover that way.
|
234
|
+
- If you set `requeue_on_error = true` in your Pwwka configuration, a message gets requeued **exactly once** on failure. If the message involved in the failure has been redelivered before, it's dropped on the floor. This behavior allows you to recover from most intermittent failures, like so:
|
235
|
+
1. You receive message for the first time.
|
236
|
+
1. Intermittent failure (e.g. network problem) happens, and an exception is raised.
|
237
|
+
1. Pwwka catches this exception and requeues the message.
|
238
|
+
1. Pwwka then crashes your handler.
|
239
|
+
1. Your handler restarts.
|
240
|
+
1. The message is in the queue, waiting for you.
|
241
|
+
1. You handle it. (*if you error here, the message is not requeued*)
|
241
242
|
|
242
|
-
|
243
|
-
|
243
|
+
The reason we don't always requeue on error is that a hard failure would result in an infinite loop. The reason we don't use the dead letter exchange is that there is no way in the Rabbit console to deal with
|
244
|
+
these messages. Some day Pwwka might have code to allow that. Today is not that day.
|
244
245
|
|
245
|
-
**
|
246
|
+
**You should configure `requeue_on_error`**. It's not the default for backwards compatibility.
|
246
247
|
|
247
248
|
#### Handling Messages with Resque
|
248
249
|
|
249
|
-
If you use [Resque][resque], and you wish to handle messages in a resque job, you can use `Pwwka::QueueResqueJobHandler`, which is an adapter between the
|
250
|
-
standard `handle!` method provided by pwwka and your Resque job.
|
250
|
+
If you use [Resque][resque], and you wish to handle messages in a resque job, you can use `Pwwka::QueueResqueJobHandler`, which is an adapter between the standard `handle!` method provided by pwwka and your Resque job.
|
251
251
|
|
252
252
|
1. First, modify your `Gemfile` or otherwise arrange to include `pwwka/queue_resque_job_handler`:
|
253
253
|
|
@@ -262,7 +262,7 @@ standard `handle!` method provided by pwwka and your Resque job.
|
|
262
262
|
```
|
263
263
|
|
264
264
|
2. Now, configure your handler. For a `Procfile` setup:
|
265
|
-
|
265
|
+
|
266
266
|
```
|
267
267
|
my_handler: rake message_handler:receive HANDLER_KLASS=Pwwka::QueueResqueJobHandler JOB_KLASS=MyResqueJob QUEUE_NAME=my_queue ROUTING_KEY="my.key.completed"
|
268
268
|
```
|
@@ -288,42 +288,7 @@ standard `handle!` method provided by pwwka and your Resque job.
|
|
288
288
|
|
289
289
|
[resque]: https://github.com/resque/resque/tree/1-x-stable
|
290
290
|
|
291
|
-
|
292
|
-
## Monitoring
|
293
|
-
|
294
|
-
RabbitMQ has a good API that should make it easy to set up some simple monitoring. In the meantime there is logging and manual monitoring.
|
295
|
-
|
296
|
-
### Logging
|
297
|
-
|
298
|
-
The receiver logs details of any exception raised in message handling:
|
299
|
-
|
300
|
-
```ruby
|
301
|
-
error "Error Processing Message on #{queue_name} -> #{payload}, #{delivery_info.routing_key}: #{e}"
|
302
|
-
```
|
303
|
-
|
304
|
-
The transmitter will likewise log an error if you use the `_safely` methods:
|
305
|
-
|
306
|
-
```ruby
|
307
|
-
error "Error Transmitting Message on #{routing_key} -> #{payload}: #{e}"
|
308
|
-
```
|
309
|
-
|
310
|
-
If your payloads are large, you may not want to log them 2-3 times per message. In that case, you can adjust `payload_logging` in the configuration:
|
311
|
-
|
312
|
-
```ruby
|
313
|
-
Pwwka.configuration.payload_logging = :info # The default - payloads appear at INFO and above log levels
|
314
|
-
Pwwka.configuration.payload_logging = :error # Only log payloads for ERROR or FATAL messages
|
315
|
-
Pwwka.configuration.payload_logging = :fatal # Only log payloads for FATAL messages
|
316
|
-
```
|
317
|
-
|
318
|
-
### Manual monitoring
|
319
|
-
|
320
|
-
RabbitMQ has a web interface for checking out the health of connections, channels, exchanges and queues. Access it via the Heroku add-ons page for Enigma.
|
321
|
-
|
322
|
-
![RabbitMQ Management 1](docs/images/RabbitMQ_Management.png)
|
323
|
-
![RabbitMQ Management 2](docs/images/RabbitMQ_Management-2.png)
|
324
|
-
![RabbitMQ Management 3](docs/images/RabbitMQ_Management-3.png)
|
325
|
-
|
326
|
-
## Testing
|
291
|
+
### Testing
|
327
292
|
|
328
293
|
This gem has test coverage of interacting with RabbitMQ, so for unit tests, your best
|
329
294
|
strategy is to simply mock calls to `Pwwka::Transmitter`.
|
@@ -366,11 +331,62 @@ describe "my integration test" do
|
|
366
331
|
end
|
367
332
|
```
|
368
333
|
|
369
|
-
|
334
|
+
[See CONTRIBUTING.md for details on testing this gem](CONTRIBUTING.md#testing)
|
335
|
+
|
370
336
|
|
371
|
-
|
337
|
+
## Better Know a Message Bus
|
372
338
|
|
373
|
-
|
339
|
+
If you aren't familiar with Rabbit or Message Busses, the idea is that messages can be sent “into the ether” with no particular
|
340
|
+
destination. Subscribers can listen for those messages and choose to respond.
|
341
|
+
|
342
|
+
For example, suppose a customer purchases an order. The app serving our public website sends a message that this has happened. Another
|
343
|
+
app that sends emails will hear that message, and use it to trigger a receipt email to the customer. A yet other app that does financial
|
344
|
+
reporting might hear this same message and record the sale to the company's ledger. The app serving our public website doesn't know about
|
345
|
+
any of these things.
|
346
|
+
|
347
|
+
### How Pwwka Uses Rabbit
|
348
|
+
|
349
|
+
All transmitters and receivers share the same exchange. This means that all receivers can read all messages that any transmitter sends. To ensure that all messages are received by eveyone who wants them the Pwwka configures everything as follows:
|
350
|
+
|
351
|
+
* The exchange is named and durable. If the service goes down and restarts the named exchange will return with the same settings so everyone can reconnect.
|
352
|
+
* The receiver queues are all named and durable. If the service goes down and restarts the named queue will return with the same settings so everyone can reconnect, and with any unacknowledged messages waiting to be received.
|
353
|
+
* All messages are sent as persistent and require acknowledgement. They will stick around and wait to be received and acknowledged by every queue that wants them, regardless of service interruptions.
|
354
|
+
|
355
|
+
|
356
|
+
### Monitoring
|
357
|
+
|
358
|
+
RabbitMQ has a good API that should make it easy to set up some simple monitoring. In the meantime there is logging and manual monitoring.
|
359
|
+
|
360
|
+
#### Logging
|
361
|
+
|
362
|
+
The receiver logs details of any exception raised in message handling:
|
363
|
+
|
364
|
+
```ruby
|
365
|
+
error "Error Processing Message on #{queue_name} -> #{payload}, #{delivery_info.routing_key}: #{e}"
|
366
|
+
```
|
367
|
+
|
368
|
+
The transmitter will likewise log an error if you use the `_safely` methods:
|
369
|
+
|
370
|
+
```ruby
|
371
|
+
error "Error Transmitting Message on #{routing_key} -> #{payload}: #{e}"
|
372
|
+
```
|
373
|
+
|
374
|
+
If your payloads are large, you may not want to log them 2-3 times per message. In that case, you can adjust `payload_logging` in the configuration:
|
375
|
+
|
376
|
+
```ruby
|
377
|
+
Pwwka.configuration.payload_logging = :info # The default - payloads appear at INFO and above log levels
|
378
|
+
Pwwka.configuration.payload_logging = :error # Only log payloads for ERROR or FATAL messages
|
379
|
+
Pwwka.configuration.payload_logging = :fatal # Only log payloads for FATAL messages
|
380
|
+
```
|
381
|
+
|
382
|
+
#### Manual monitoring
|
383
|
+
|
384
|
+
RabbitMQ has a web interface for checking out the health of connections, channels, exchanges and queues. Your RabbitMQ provider should
|
385
|
+
provide a link. If you are running Rabbit locally, the management interface is on port 15672 by default (or port 10002 if using the included `docker-compose.yml`). The user is "guest" and the password is "guest".
|
386
|
+
|
387
|
+
![RabbitMQ Management 1](docs/images/RabbitMQ_Management.png)
|
388
|
+
![RabbitMQ Management 2](docs/images/RabbitMQ_Management-2.png)
|
389
|
+
![RabbitMQ Management 3](docs/images/RabbitMQ_Management-3.png)
|
374
390
|
|
375
391
|
## Contributing
|
376
392
|
|