shivam 0.0.0-java
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.github/workflows/test.yml +57 -0
- data/.gitignore +10 -0
- data/.rspec +1 -0
- data/.yardopts +5 -0
- data/CHANGELOG.md +899 -0
- data/Gemfile +35 -0
- data/Guardfile +14 -0
- data/LICENSE +23 -0
- data/README.md +679 -0
- data/Rakefile +21 -0
- data/bin/ci/before_build.sh +20 -0
- data/bin/ci/before_build_docker.sh +20 -0
- data/bin/ci/install_on_debian.sh +46 -0
- data/bin/hutch +8 -0
- data/examples/consumer.rb +13 -0
- data/examples/producer.rb +10 -0
- data/hutch.gemspec +27 -0
- data/lib/hutch/acknowledgements/base.rb +16 -0
- data/lib/hutch/acknowledgements/nack_on_all_failures.rb +19 -0
- data/lib/hutch/adapter.rb +11 -0
- data/lib/hutch/adapters/bunny.rb +37 -0
- data/lib/hutch/adapters/march_hare.rb +41 -0
- data/lib/hutch/broker.rb +384 -0
- data/lib/hutch/cli.rb +246 -0
- data/lib/hutch/config.rb +305 -0
- data/lib/hutch/consumer.rb +125 -0
- data/lib/hutch/error_handlers/airbrake.rb +54 -0
- data/lib/hutch/error_handlers/base.rb +15 -0
- data/lib/hutch/error_handlers/bugsnag.rb +30 -0
- data/lib/hutch/error_handlers/honeybadger.rb +43 -0
- data/lib/hutch/error_handlers/logger.rb +22 -0
- data/lib/hutch/error_handlers/rollbar.rb +28 -0
- data/lib/hutch/error_handlers/sentry.rb +26 -0
- data/lib/hutch/error_handlers/sentry_raven.rb +31 -0
- data/lib/hutch/error_handlers.rb +11 -0
- data/lib/hutch/exceptions.rb +14 -0
- data/lib/hutch/logging.rb +32 -0
- data/lib/hutch/message.rb +31 -0
- data/lib/hutch/publisher.rb +75 -0
- data/lib/hutch/serializers/identity.rb +19 -0
- data/lib/hutch/serializers/json.rb +22 -0
- data/lib/hutch/tracers/datadog.rb +18 -0
- data/lib/hutch/tracers/newrelic.rb +19 -0
- data/lib/hutch/tracers/null_tracer.rb +15 -0
- data/lib/hutch/tracers.rb +7 -0
- data/lib/hutch/version.rb +3 -0
- data/lib/hutch/waiter.rb +104 -0
- data/lib/hutch/worker.rb +145 -0
- data/lib/hutch.rb +69 -0
- data/lib/yard-settings/handler.rb +38 -0
- data/lib/yard-settings/yard-settings.rb +2 -0
- data/spec/hutch/broker_spec.rb +462 -0
- data/spec/hutch/cli_spec.rb +93 -0
- data/spec/hutch/config_spec.rb +259 -0
- data/spec/hutch/consumer_spec.rb +208 -0
- data/spec/hutch/error_handlers/airbrake_spec.rb +49 -0
- data/spec/hutch/error_handlers/bugsnag_spec.rb +55 -0
- data/spec/hutch/error_handlers/honeybadger_spec.rb +58 -0
- data/spec/hutch/error_handlers/logger_spec.rb +28 -0
- data/spec/hutch/error_handlers/rollbar_spec.rb +45 -0
- data/spec/hutch/error_handlers/sentry_raven_spec.rb +37 -0
- data/spec/hutch/error_handlers/sentry_spec.rb +47 -0
- data/spec/hutch/logger_spec.rb +34 -0
- data/spec/hutch/message_spec.rb +38 -0
- data/spec/hutch/serializers/json_spec.rb +17 -0
- data/spec/hutch/tracers/datadog_spec.rb +44 -0
- data/spec/hutch/waiter_spec.rb +51 -0
- data/spec/hutch/worker_spec.rb +184 -0
- data/spec/hutch_spec.rb +87 -0
- data/spec/spec_helper.rb +42 -0
- data/templates/default/class/html/settings.erb +0 -0
- data/templates/default/class/setup.rb +4 -0
- data/templates/default/fulldoc/html/css/hutch.css +13 -0
- data/templates/default/layout/html/setup.rb +7 -0
- data/templates/default/method_details/html/settings.erb +5 -0
- data/templates/default/method_details/setup.rb +4 -0
- data/templates/default/method_details/text/settings.erb +0 -0
- data/templates/default/module/html/settings.erb +40 -0
- data/templates/default/module/setup.rb +4 -0
- metadata +205 -0
data/README.md
ADDED
@@ -0,0 +1,679 @@
|
|
1
|
+
![](http://cl.ly/image/3h0q3F3G142K/hutch.png)
|
2
|
+
|
3
|
+
[![Gem Version](https://badge.fury.io/rb/hutch.svg)](http://badge.fury.io/rb/hutch)
|
4
|
+
[![Code Climate](https://codeclimate.com/github/gocardless/hutch.svg)](https://codeclimate.com/github/gocardless/hutch)
|
5
|
+
|
6
|
+
Hutch is a Ruby library for enabling asynchronous inter-service communication
|
7
|
+
in a service-oriented architecture, using RabbitMQ.
|
8
|
+
|
9
|
+
To install with RubyGems:
|
10
|
+
|
11
|
+
```
|
12
|
+
gem install hutch
|
13
|
+
```
|
14
|
+
|
15
|
+
<!-- Tocer[start]: Auto-generated, don't remove. -->
|
16
|
+
|
17
|
+
### Table of Contents
|
18
|
+
|
19
|
+
- [Requirements](#requirements)
|
20
|
+
- [Overview](#overview)
|
21
|
+
- [Project Maturity](#project-maturity)
|
22
|
+
- [Consumers](#consumers)
|
23
|
+
- [Message Processing Tracers](#message-processing-tracers)
|
24
|
+
- [Running Hutch](#running-hutch)
|
25
|
+
- [Loading Consumers](#loading-consumers)
|
26
|
+
- [Stopping Hutch](#stopping-hutch)
|
27
|
+
- [Producers](#producers)
|
28
|
+
- [Producer Configuration](#producer-configuration)
|
29
|
+
- [Publisher Confirms](#publisher-confirms)
|
30
|
+
- [Writing Well-Behaved Publishers](#writing-well-behaved-publishers)
|
31
|
+
- [Configuration](#configuration)
|
32
|
+
- [Config File](#config-file)
|
33
|
+
- [Environment variables](#environment-variables)
|
34
|
+
- [Configuration precedence](#configuration-precedence)
|
35
|
+
- [Generated list of configuration options](#generated-list-of-configuration-options)
|
36
|
+
|
37
|
+
<!-- Tocer[finish]: Auto-generated, don't remove. -->
|
38
|
+
|
39
|
+
## Requirements
|
40
|
+
|
41
|
+
- Hutch requires Ruby 2.4+ or JRuby 9K.
|
42
|
+
- Hutch requires RabbitMQ 3.3 or later.
|
43
|
+
|
44
|
+
## Overview
|
45
|
+
|
46
|
+
Hutch is a conventions-based framework for writing services that communicate
|
47
|
+
over RabbitMQ. Hutch is opinionated: it uses topic exchanges for message
|
48
|
+
distribution and makes some assumptions about how consumers and publishers
|
49
|
+
should work.
|
50
|
+
|
51
|
+
With Hutch, consumers are stored in separate files and include the `Hutch::Consumer` module.
|
52
|
+
They are then loaded by a command line runner which connects to RabbitMQ, sets up queues and bindings,
|
53
|
+
and so on. Publishers connect to RabbitMQ via `Hutch.connect` and publish using `Hutch.publish`.
|
54
|
+
|
55
|
+
Hutch uses [Bunny](http://rubybunny.info) or [March Hare](http://rubymarchhare.info)
|
56
|
+
(on JRuby) under the hood.
|
57
|
+
|
58
|
+
### Project Maturity
|
59
|
+
|
60
|
+
Hutch is a mature project that was originally extracted from production systems
|
61
|
+
at [GoCardless](https://gocardless.com) in 2013 and is now maintained by its contributors
|
62
|
+
and users.
|
63
|
+
|
64
|
+
## Consumers
|
65
|
+
|
66
|
+
Consumers receive messages from a RabbitMQ queue. That queue may be bound to
|
67
|
+
one or more topics (represented by routing keys).
|
68
|
+
|
69
|
+
To create a consumer, include the `Hutch::Consumer` module in a class that
|
70
|
+
defines a `#process` method. `#process` should take a single argument, which
|
71
|
+
will be a `Message` object. The `Message` object encapsulates the message data,
|
72
|
+
along with any associated metadata. To access properties of the message, use
|
73
|
+
Hash-style indexing syntax:
|
74
|
+
|
75
|
+
```ruby
|
76
|
+
message[:id] # => "02ABCXYZ"
|
77
|
+
```
|
78
|
+
|
79
|
+
To subscribe to a topic, pass a routing key to `consume` in the class
|
80
|
+
definition. To bind to multiple routing keys, simply pass extra routing keys
|
81
|
+
in as additional arguments. Refer to the [RabbitMQ docs on topic exchanges
|
82
|
+
](http://www.rabbitmq.com/tutorials/tutorial-five-ruby.html) for more information
|
83
|
+
about how to use routing keys. Here's an example consumer:
|
84
|
+
|
85
|
+
```ruby
|
86
|
+
class FailedPaymentConsumer
|
87
|
+
include Hutch::Consumer
|
88
|
+
consume 'gc.ps.payment.failed'
|
89
|
+
|
90
|
+
def process(message)
|
91
|
+
mark_payment_as_failed(message[:id])
|
92
|
+
end
|
93
|
+
end
|
94
|
+
```
|
95
|
+
|
96
|
+
By default, the queue name will be named using the consumer class. You can set
|
97
|
+
the queue name explicitly by using the `queue_name` method:
|
98
|
+
|
99
|
+
```ruby
|
100
|
+
class FailedPaymentConsumer
|
101
|
+
include Hutch::Consumer
|
102
|
+
consume 'gc.ps.payment.failed'
|
103
|
+
queue_name 'failed_payments'
|
104
|
+
|
105
|
+
def process(message)
|
106
|
+
mark_payment_as_failed(message[:id])
|
107
|
+
end
|
108
|
+
end
|
109
|
+
```
|
110
|
+
|
111
|
+
It is possible to set some custom options to consumer's queue explicitly.
|
112
|
+
This example sets the consumer's queue as a
|
113
|
+
[quorum queue](https://www.rabbitmq.com/quorum-queues.html)
|
114
|
+
and to operate in the [lazy mode](https://www.rabbitmq.com/lazy-queues.html).
|
115
|
+
The `initial_group_size`
|
116
|
+
[argument](https://www.rabbitmq.com/quorum-queues.html#replication-factor) is
|
117
|
+
optional.
|
118
|
+
|
119
|
+
```ruby
|
120
|
+
class FailedPaymentConsumer
|
121
|
+
include Hutch::Consumer
|
122
|
+
consume 'gc.ps.payment.failed'
|
123
|
+
queue_name 'failed_payments'
|
124
|
+
lazy_queue
|
125
|
+
quorum_queue initial_group_size: 3
|
126
|
+
|
127
|
+
def process(message)
|
128
|
+
mark_payment_as_failed(message[:id])
|
129
|
+
end
|
130
|
+
end
|
131
|
+
```
|
132
|
+
|
133
|
+
You can also set custom arguments per consumer. This example declares a consumer with
|
134
|
+
a maximum length of 10 messages:
|
135
|
+
|
136
|
+
```ruby
|
137
|
+
class FailedPaymentConsumer
|
138
|
+
include Hutch::Consumer
|
139
|
+
consume 'gc.ps.payment.failed'
|
140
|
+
arguments 'x-max-length' => 10
|
141
|
+
end
|
142
|
+
```
|
143
|
+
|
144
|
+
This sets the `x-max-length` header. For more details, see the [RabbitMQ
|
145
|
+
documentation on Queue Length Limit](https://www.rabbitmq.com/maxlength.html). To find out more
|
146
|
+
about custom queue arguments, consult the [RabbitMQ documentation on AMQP Protocol Extensions](https://www.rabbitmq.com/extensions.html).
|
147
|
+
|
148
|
+
Consumers can write to Hutch's log by calling the logger method. The logger method returns
|
149
|
+
a [Logger object](http://ruby-doc.org/stdlib-2.1.2/libdoc/logger/rdoc/Logger.html).
|
150
|
+
|
151
|
+
```ruby
|
152
|
+
class FailedPaymentConsumer
|
153
|
+
include Hutch::Consumer
|
154
|
+
consume 'gc.ps.payment.failed'
|
155
|
+
|
156
|
+
def process(message)
|
157
|
+
logger.info "Marking payment #{message[:id]} as failed"
|
158
|
+
mark_payment_as_failed(message[:id])
|
159
|
+
end
|
160
|
+
end
|
161
|
+
```
|
162
|
+
|
163
|
+
If you are using Hutch with Rails and want to make Hutch log to the Rails
|
164
|
+
logger rather than `stdout`, add this to `config/initializers/hutch.rb`
|
165
|
+
|
166
|
+
```ruby
|
167
|
+
Hutch::Logging.logger = Rails.logger
|
168
|
+
```
|
169
|
+
|
170
|
+
A logger can be set for the client by adding this config before calling `Hutch.connect`
|
171
|
+
|
172
|
+
```ruby
|
173
|
+
client_logger = Logger.new("/path/to/bunny.log")
|
174
|
+
Hutch::Config.set(:client_logger, client_logger)
|
175
|
+
```
|
176
|
+
|
177
|
+
See this [RabbitMQ tutorial on topic exchanges](http://www.rabbitmq.com/tutorials/tutorial-five-ruby.html)
|
178
|
+
to learn more.
|
179
|
+
|
180
|
+
### Message Processing Tracers
|
181
|
+
|
182
|
+
Tracers allow you to track message processing.
|
183
|
+
|
184
|
+
This will enable NewRelic custom instrumentation:
|
185
|
+
|
186
|
+
```ruby
|
187
|
+
Hutch::Config.set(:tracer, Hutch::Tracers::NewRelic)
|
188
|
+
```
|
189
|
+
|
190
|
+
And this will enable Datadog custom instrumentation:
|
191
|
+
|
192
|
+
```ruby
|
193
|
+
Hutch::Config.set(:tracer, Hutch::Tracers::Datadog)
|
194
|
+
```
|
195
|
+
|
196
|
+
Batteries included!
|
197
|
+
|
198
|
+
## Running Hutch
|
199
|
+
|
200
|
+
After installing the Hutch gem, you should be able to start it by simply
|
201
|
+
running `hutch` on the command line. `hutch` takes a number of options:
|
202
|
+
|
203
|
+
```console
|
204
|
+
$ hutch -h
|
205
|
+
usage: hutch [options]
|
206
|
+
--mq-host HOST Set the RabbitMQ host
|
207
|
+
--mq-port PORT Set the RabbitMQ port
|
208
|
+
-t, --[no-]mq-tls Use TLS for the AMQP connection
|
209
|
+
--mq-tls-cert FILE Certificate for TLS client verification
|
210
|
+
--mq-tls-key FILE Private key for TLS client verification
|
211
|
+
--mq-exchange EXCHANGE Set the RabbitMQ exchange
|
212
|
+
--mq-vhost VHOST Set the RabbitMQ vhost
|
213
|
+
--mq-username USERNAME Set the RabbitMQ username
|
214
|
+
--mq-password PASSWORD Set the RabbitMQ password
|
215
|
+
--mq-api-host HOST Set the RabbitMQ API host
|
216
|
+
--mq-api-port PORT Set the RabbitMQ API port
|
217
|
+
-s, --[no-]mq-api-ssl Use SSL for the RabbitMQ API
|
218
|
+
--config FILE Load Hutch configuration from a file
|
219
|
+
--require PATH Require a Rails app or path
|
220
|
+
--[no-]autoload-rails Require the current rails app directory
|
221
|
+
-q, --quiet Quiet logging
|
222
|
+
-v, --verbose Verbose logging
|
223
|
+
--version Print the version and exit
|
224
|
+
-h, --help Show this message and exit
|
225
|
+
```
|
226
|
+
|
227
|
+
The first three are for configuring which RabbitMQ instance to connect to.
|
228
|
+
`--require` is covered in the next section. Configurations can also be
|
229
|
+
specified in a YAML file for convenience by passing the file location
|
230
|
+
to the --config option. The file should look like:
|
231
|
+
|
232
|
+
```yaml
|
233
|
+
mq_username: peter
|
234
|
+
mq_password: rabbit
|
235
|
+
mq_host: broker.yourhost.com
|
236
|
+
```
|
237
|
+
|
238
|
+
Passing a setting as a command-line option will overwrite what's specified
|
239
|
+
in the config file, allowing for easy customization.
|
240
|
+
|
241
|
+
### Loading Consumers
|
242
|
+
|
243
|
+
Using Hutch with a Rails app is simple. Either start Hutch in the working
|
244
|
+
directory of a Rails app, or pass the path to a Rails app in with the
|
245
|
+
`--require` option. Consumers defined in Rails apps should be placed with in
|
246
|
+
the `app/consumers/` directory, to allow them to be auto-loaded when Rails
|
247
|
+
boots.
|
248
|
+
|
249
|
+
If you're using the new Zeitwerk autoloader (enabled by default in Rails 6)
|
250
|
+
and the consumers are not loaded in development environment you will need to
|
251
|
+
trigger the autoloading in an initializer with
|
252
|
+
|
253
|
+
```ruby
|
254
|
+
::Zeitwerk::Loader.eager_load_all
|
255
|
+
```
|
256
|
+
|
257
|
+
or with something more specific like
|
258
|
+
|
259
|
+
```ruby
|
260
|
+
autoloader = Rails.autoloaders.main
|
261
|
+
|
262
|
+
Dir.glob(File.join('app/consumers', '*_consumer.rb')).each do |consumer|
|
263
|
+
autoloader.preload(consumer)
|
264
|
+
end
|
265
|
+
```
|
266
|
+
|
267
|
+
### Consumer Groups
|
268
|
+
|
269
|
+
It is possible to load only a subset of consumers. This is done by defining a consumer
|
270
|
+
group under the `consumer_groups` configuration key:
|
271
|
+
|
272
|
+
``` yaml
|
273
|
+
consumer_groups:
|
274
|
+
payments:
|
275
|
+
- DepositConsumer
|
276
|
+
- CashoutConsumer
|
277
|
+
notification:
|
278
|
+
- EmailNotificationConsumer
|
279
|
+
```
|
280
|
+
|
281
|
+
To only load a group of consumers, use the `--only-group` option:
|
282
|
+
|
283
|
+
``` shell
|
284
|
+
hutch --only-group=payments --config=/path/to/hutch.yaml
|
285
|
+
```
|
286
|
+
|
287
|
+
### Loading Consumers Manually (One-by-One)
|
288
|
+
|
289
|
+
To require files that define consumers manually, simply pass each file as an
|
290
|
+
option to `--require`. Hutch will automatically detect whether you've provided
|
291
|
+
a Rails app or a standard file, and take the appropriate behaviour:
|
292
|
+
|
293
|
+
```bash
|
294
|
+
# loads a rails app
|
295
|
+
hutch --require path/to/rails-app
|
296
|
+
# loads a ruby file
|
297
|
+
hutch --require path/to/file.rb
|
298
|
+
```
|
299
|
+
|
300
|
+
### Stopping Hutch
|
301
|
+
|
302
|
+
Hutch supports graceful stops. That means that if done correctly, Hutch will wait for your consumer to finish processing before exiting.
|
303
|
+
|
304
|
+
To gracefully stop your workers, you may send the following signals to your Hutch processes: `INT`, `TERM`, or `QUIT`.
|
305
|
+
|
306
|
+
```bash
|
307
|
+
kill -SIGINT 123 # or kill -2 123
|
308
|
+
kill -SIGTERM 456 # or kill -15 456
|
309
|
+
kill -SIGQUIT 789 # or kill -3 789
|
310
|
+
```
|
311
|
+
|
312
|
+
![](http://g.recordit.co/wyCdzG9Kh3.gif)
|
313
|
+
|
314
|
+
## Producers
|
315
|
+
|
316
|
+
Hutch includes a `publish` method for sending messages to Hutch consumers. When
|
317
|
+
possible, this should be used, rather than directly interfacing with RabbitMQ
|
318
|
+
libraries.
|
319
|
+
|
320
|
+
```ruby
|
321
|
+
Hutch.connect
|
322
|
+
Hutch.publish('routing.key', subject: 'payment', action: 'received')
|
323
|
+
```
|
324
|
+
|
325
|
+
### Producer Configuration
|
326
|
+
|
327
|
+
Producers are not run with the 'hutch' command. You can specify configuration
|
328
|
+
options as follows:
|
329
|
+
|
330
|
+
```ruby
|
331
|
+
Hutch::Config.set(:mq_exchange, 'name')
|
332
|
+
```
|
333
|
+
|
334
|
+
### Publisher Confirms
|
335
|
+
|
336
|
+
For maximum message reliability when producing messages, you can force Hutch to use
|
337
|
+
[Publisher Confirms](https://www.rabbitmq.com/confirms.html) and wait for a confirmation
|
338
|
+
after every message published. This is the safest possible option for publishers
|
339
|
+
but also results in a **significant throughput drop**.
|
340
|
+
|
341
|
+
```ruby
|
342
|
+
Hutch::Config.set(:force_publisher_confirms, true)
|
343
|
+
```
|
344
|
+
|
345
|
+
### Writing Well-Behaved Publishers
|
346
|
+
|
347
|
+
You may need to send messages to Hutch from languages other than Ruby. This
|
348
|
+
prevents the use of `Hutch.publish`, requiring custom publication code to be
|
349
|
+
written. There are a few things to keep in mind when writing producers that
|
350
|
+
send messages to Hutch.
|
351
|
+
|
352
|
+
- Make sure that the producer exchange name matches the exchange name that
|
353
|
+
Hutch is using.
|
354
|
+
- Hutch works with topic exchanges, check the producer is also using topic
|
355
|
+
exchanges.
|
356
|
+
- Use message routing keys that match those used in your Hutch consumers.
|
357
|
+
- Be sure your exchanges are marked as durable. In the Ruby AMQP gem, this is
|
358
|
+
done by passing `durable: true` to the exchange creation method.
|
359
|
+
- Publish messages as persistent.
|
360
|
+
- Using publisher confirms is highly recommended.
|
361
|
+
|
362
|
+
Here's an example of a well-behaved publisher, minus publisher confirms:
|
363
|
+
|
364
|
+
```ruby
|
365
|
+
AMQP.connect(host: config[:host]) do |connection|
|
366
|
+
channel = AMQP::Channel.new(connection)
|
367
|
+
exchange = channel.topic(config[:exchange], durable: true)
|
368
|
+
|
369
|
+
message = JSON.dump({ subject: 'Test', id: 'abc' })
|
370
|
+
exchange.publish(message, routing_key: 'test', persistent: true)
|
371
|
+
end
|
372
|
+
```
|
373
|
+
|
374
|
+
If using publisher confirms with amqp gem, see [this issue](https://github.com/ruby-amqp/amqp/issues/92)
|
375
|
+
and [this gist](https://gist.github.com/3042381) for more info.
|
376
|
+
|
377
|
+
## Configuration
|
378
|
+
|
379
|
+
### Config File
|
380
|
+
|
381
|
+
It is recommended to use a separate config file, unless you use URIs for connection (see below).
|
382
|
+
|
383
|
+
Known configuration parameters are:
|
384
|
+
|
385
|
+
* `mq_host`: RabbitMQ hostname (default: `localhost`)
|
386
|
+
* `mq_port`: RabbitMQ port (default: `5672`)
|
387
|
+
* `mq_vhost`: vhost to use (default: `/`)
|
388
|
+
* `mq_username`: username to use (default: `guest`, only can connect from localhost as of RabbitMQ 3.3.0)
|
389
|
+
* `mq_password`: password to use (default: `guest`)
|
390
|
+
* `mq_tls`: should TLS be used? (default: `false`)
|
391
|
+
* `mq_tls_cert`: path to client TLS certificate (public key)
|
392
|
+
* `mq_tls_key`: path to client TLS private key
|
393
|
+
* `mq_tls_ca_certificates`: array of paths to CA keys (if not specified to Hutch, will default to Bunny defaults which are system-dependent)
|
394
|
+
* `mq_verify_peer`: should SSL certificate be verified? (default: `true`)
|
395
|
+
* `require_paths`: array of paths to require
|
396
|
+
* `autoload_rails`: should Hutch command line runner try to automatically load Rails environment files?
|
397
|
+
* `daemonise`: should Hutch runner process daemonise?
|
398
|
+
* `pidfile`: path to PID file the runner should use
|
399
|
+
* `channel_prefetch`: basic.qos prefetch value to use (default: `0`, no limit). See Bunny and RabbitMQ documentation.
|
400
|
+
* `publisher_confirms`: enables publisher confirms. Leaves it up to the app how they are
|
401
|
+
tracked (e.g. using `Hutch::Broker#confirm_select` callback or `Hutch::Broker#wait_for_confirms`)
|
402
|
+
* `force_publisher_confirms`: enables publisher confirms, forces `Hutch::Broker#wait_for_confirms` for every publish. **This is the safest option which also offers the lowest throughput**.
|
403
|
+
* `log_level`: log level used by the standard Ruby logger (default: `Logger::INFO`)
|
404
|
+
* `error_handlers`: a list of error handler objects, see classes in `Hutch::ErrorHandlers`. All configured
|
405
|
+
handlers will be invoked unconditionally in the order listed.
|
406
|
+
* `error_acknowledgements`: a chain of responsibility of objects that acknowledge/reject/requeue messages when an
|
407
|
+
exception happens, see classes in `Hutch::Acknowledgements`.
|
408
|
+
* `mq_exchange`: exchange to use for publishing (default: `hutch`)
|
409
|
+
* `heartbeat`: [RabbitMQ heartbeat timeout](http://rabbitmq.com/heartbeats.html) (default: `30`)
|
410
|
+
* `connection_timeout`: Bunny's socket open timeout (default: `11`)
|
411
|
+
* `read_timeout`: Bunny's socket read timeout (default: `11`)
|
412
|
+
* `write_timeout`: Bunny's socket write timeout (default: `11`)
|
413
|
+
* `automatically_recover`: Bunny's enable/disable network recovery (default: `true`)
|
414
|
+
* `network_recovery_interval`: Bunny's reconnect interval (default: `1`)
|
415
|
+
* `tracer`: tracer to use to track message processing
|
416
|
+
* `namespace`: A namespace string to help group your queues (default: `nil`)
|
417
|
+
|
418
|
+
### Environment variables
|
419
|
+
|
420
|
+
The file configuration options mentioned above can also be passed in via environment variables, using the `HUTCH_` prefix, eg.
|
421
|
+
|
422
|
+
* `connection_timeout` → `HUTCH_CONNECTION_TIMEOUT`.
|
423
|
+
|
424
|
+
### Configuration precedence
|
425
|
+
|
426
|
+
In order from lowest to highest precedence:
|
427
|
+
|
428
|
+
0. Default values
|
429
|
+
0. `HUTCH_*` environment variables
|
430
|
+
0. Configuration file
|
431
|
+
0. Explicit settings through `Hutch::Config.set`
|
432
|
+
|
433
|
+
### Generated list of configuration options
|
434
|
+
|
435
|
+
Generate with
|
436
|
+
|
437
|
+
0. `yard doc lib/hutch/config.rb`
|
438
|
+
0. Copy the _Configuration_ section from `doc/Hutch/Config.html` here, with the anchor tags stripped.
|
439
|
+
|
440
|
+
<table border="1" class="settings" style="overflow:visible;">
|
441
|
+
<thead>
|
442
|
+
<tr>
|
443
|
+
<th>
|
444
|
+
Setting name
|
445
|
+
</th>
|
446
|
+
<th>
|
447
|
+
Default value
|
448
|
+
</th>
|
449
|
+
<th>
|
450
|
+
Type
|
451
|
+
</th>
|
452
|
+
<th>
|
453
|
+
ENV variable
|
454
|
+
</th>
|
455
|
+
<th>
|
456
|
+
Description
|
457
|
+
</th>
|
458
|
+
</tr>
|
459
|
+
</thead>
|
460
|
+
<tbody>
|
461
|
+
<tr>
|
462
|
+
<td><tt>mq_host</tt></td>
|
463
|
+
<td>127.0.0.1</td>
|
464
|
+
<td>String</td>
|
465
|
+
<td><tt>HUTCH_MQ_HOST</tt></td>
|
466
|
+
<td><p>RabbitMQ hostname</p></td>
|
467
|
+
</tr>
|
468
|
+
<tr>
|
469
|
+
<td><tt>mq_exchange</tt></td>
|
470
|
+
<td>hutch</td>
|
471
|
+
<td>String</td>
|
472
|
+
<td><tt>HUTCH_MQ_EXCHANGE</tt></td>
|
473
|
+
<td><p>RabbitMQ Exchange to use for publishing</p></td>
|
474
|
+
</tr>
|
475
|
+
<tr>
|
476
|
+
<td><tt>mq_exchange_type</tt></td>
|
477
|
+
<td>topic</td>
|
478
|
+
<td>String</td>
|
479
|
+
<td><tt>HUTCH_MQ_EXCHANGE_TYPE</tt></td>
|
480
|
+
<td><p>RabbitMQ Exchange type to use for publishing</p></td>
|
481
|
+
</tr>
|
482
|
+
<tr>
|
483
|
+
<td><tt>mq_vhost</tt></td>
|
484
|
+
<td>/</td>
|
485
|
+
<td>String</td>
|
486
|
+
<td><tt>HUTCH_MQ_VHOST</tt></td>
|
487
|
+
<td><p>RabbitMQ vhost to use</p></td>
|
488
|
+
</tr>
|
489
|
+
<tr>
|
490
|
+
<td><tt>mq_username</tt></td>
|
491
|
+
<td>guest</td>
|
492
|
+
<td>String</td>
|
493
|
+
<td><tt>HUTCH_MQ_USERNAME</tt></td>
|
494
|
+
<td><p>RabbitMQ username to use.</p></td>
|
495
|
+
</tr>
|
496
|
+
<tr>
|
497
|
+
<td><tt>mq_password</tt></td>
|
498
|
+
<td>guest</td>
|
499
|
+
<td>String</td>
|
500
|
+
<td><tt>HUTCH_MQ_PASSWORD</tt></td>
|
501
|
+
<td><p>RabbitMQ password</p></td>
|
502
|
+
</tr>
|
503
|
+
<tr>
|
504
|
+
<td><tt>uri</tt></td>
|
505
|
+
<td>nil</td>
|
506
|
+
<td>String</td>
|
507
|
+
<td><tt>HUTCH_URI</tt></td>
|
508
|
+
<td><p>RabbitMQ URI (takes precedence over MQ username, password, host, port and vhost settings)</p></td>
|
509
|
+
</tr>
|
510
|
+
<tr>
|
511
|
+
<td><tt>mq_api_host</tt></td>
|
512
|
+
<td>127.0.0.1</td>
|
513
|
+
<td>String</td>
|
514
|
+
<td><tt>HUTCH_MQ_API_HOST</tt></td>
|
515
|
+
<td><p>RabbitMQ HTTP API hostname</p></td>
|
516
|
+
</tr>
|
517
|
+
<tr>
|
518
|
+
<td><tt>mq_port</tt></td>
|
519
|
+
<td>5672</td>
|
520
|
+
<td>Number</td>
|
521
|
+
<td><tt>HUTCH_MQ_PORT</tt></td>
|
522
|
+
<td><p>RabbitMQ port</p></td>
|
523
|
+
</tr>
|
524
|
+
<tr>
|
525
|
+
<td><tt>mq_api_port</tt></td>
|
526
|
+
<td>15672</td>
|
527
|
+
<td>Number</td>
|
528
|
+
<td><tt>HUTCH_MQ_API_PORT</tt></td>
|
529
|
+
<td><p>RabbitMQ HTTP API port</p></td>
|
530
|
+
</tr>
|
531
|
+
<tr>
|
532
|
+
<td><tt>heartbeat</tt></td>
|
533
|
+
<td>30</td>
|
534
|
+
<td>Number</td>
|
535
|
+
<td><tt>HUTCH_HEARTBEAT</tt></td>
|
536
|
+
<td><p><a href="http://rabbitmq.com/heartbeats.html">RabbitMQ heartbeat timeout</a></p></td>
|
537
|
+
</tr>
|
538
|
+
<tr>
|
539
|
+
<td><tt>channel_prefetch</tt></td>
|
540
|
+
<td>0</td>
|
541
|
+
<td>Number</td>
|
542
|
+
<td><tt>HUTCH_CHANNEL_PREFETCH</tt></td>
|
543
|
+
<td><p>The <tt>basic.qos</tt> prefetch value to use.</p></td>
|
544
|
+
</tr>
|
545
|
+
<tr>
|
546
|
+
<td><tt>connection_timeout</tt></td>
|
547
|
+
<td>11</td>
|
548
|
+
<td>Number</td>
|
549
|
+
<td><tt>HUTCH_CONNECTION_TIMEOUT</tt></td>
|
550
|
+
<td><p>Bunny's socket open timeout</p></td>
|
551
|
+
</tr>
|
552
|
+
<tr>
|
553
|
+
<td><tt>read_timeout</tt></td>
|
554
|
+
<td>11</td>
|
555
|
+
<td>Number</td>
|
556
|
+
<td><tt>HUTCH_READ_TIMEOUT</tt></td>
|
557
|
+
<td><p>Bunny's socket read timeout</p></td>
|
558
|
+
</tr>
|
559
|
+
<tr>
|
560
|
+
<td><tt>write_timeout</tt></td>
|
561
|
+
<td>11</td>
|
562
|
+
<td>Number</td>
|
563
|
+
<td><tt>HUTCH_WRITE_TIMEOUT</tt></td>
|
564
|
+
<td><p>Bunny's socket write timeout</p></td>
|
565
|
+
</tr>
|
566
|
+
<tr>
|
567
|
+
<td><tt>automatically_recover</tt></td>
|
568
|
+
<td>true</td>
|
569
|
+
<td>Boolean</td>
|
570
|
+
<td><tt>HUTCH_AUTOMATICALLY_RECOVER</tt></td>
|
571
|
+
<td><p>Bunny's enable/disable network recovery</p></td>
|
572
|
+
</tr>
|
573
|
+
<tr>
|
574
|
+
<td><tt>network_recovery_interval</tt></td>
|
575
|
+
<td>1</td>
|
576
|
+
<td>Number</td>
|
577
|
+
<td><tt>HUTCH_NETWORK_RECOVERY_INTERVAL</tt></td>
|
578
|
+
<td><p>Bunny's reconnect interval</p></td>
|
579
|
+
</tr>
|
580
|
+
<tr>
|
581
|
+
<td><tt>graceful_exit_timeout</tt></td>
|
582
|
+
<td>11</td>
|
583
|
+
<td>Number</td>
|
584
|
+
<td><tt>HUTCH_GRACEFUL_EXIT_TIMEOUT</tt></td>
|
585
|
+
<td><p>FIXME: DOCUMENT THIS</p></td>
|
586
|
+
</tr>
|
587
|
+
<tr>
|
588
|
+
<td><tt>consumer_pool_size</tt></td>
|
589
|
+
<td>1</td>
|
590
|
+
<td>Number</td>
|
591
|
+
<td><tt>HUTCH_CONSUMER_POOL_SIZE</tt></td>
|
592
|
+
<td><p>Bunny consumer work pool size</p></td>
|
593
|
+
</tr>
|
594
|
+
<tr>
|
595
|
+
<td><tt>mq_tls</tt></td>
|
596
|
+
<td>false</td>
|
597
|
+
<td>Boolean</td>
|
598
|
+
<td><tt>HUTCH_MQ_TLS</tt></td>
|
599
|
+
<td><p>Should TLS be used?</p></td>
|
600
|
+
</tr>
|
601
|
+
<tr>
|
602
|
+
<td><tt>mq_verify_peer</tt></td>
|
603
|
+
<td>true</td>
|
604
|
+
<td>Boolean</td>
|
605
|
+
<td><tt>HUTCH_MQ_VERIFY_PEER</tt></td>
|
606
|
+
<td><p>Should SSL certificate be verified?</p></td>
|
607
|
+
</tr>
|
608
|
+
<tr>
|
609
|
+
<td><tt>mq_api_ssl</tt></td>
|
610
|
+
<td>false</td>
|
611
|
+
<td>Boolean</td>
|
612
|
+
<td><tt>HUTCH_MQ_API_SSL</tt></td>
|
613
|
+
<td><p>Should SSL be used for the RabbitMQ API?</p></td>
|
614
|
+
</tr>
|
615
|
+
<tr>
|
616
|
+
<td><tt>autoload_rails</tt></td>
|
617
|
+
<td>true</td>
|
618
|
+
<td>Boolean</td>
|
619
|
+
<td><tt>HUTCH_AUTOLOAD_RAILS</tt></td>
|
620
|
+
<td><p>Should the current Rails app directory be required?</p></td>
|
621
|
+
</tr>
|
622
|
+
<tr>
|
623
|
+
<td><tt>daemonise</tt></td>
|
624
|
+
<td>false</td>
|
625
|
+
<td>Boolean</td>
|
626
|
+
<td><tt>HUTCH_DAEMONISE</tt></td>
|
627
|
+
<td><p>Should the Hutch runner process daemonise?</p></td>
|
628
|
+
</tr>
|
629
|
+
<tr>
|
630
|
+
<td><tt>publisher_confirms</tt></td>
|
631
|
+
<td>false</td>
|
632
|
+
<td>Boolean</td>
|
633
|
+
<td><tt>HUTCH_PUBLISHER_CONFIRMS</tt></td>
|
634
|
+
<td><p>Should RabbitMQ publisher confirms be enabled?</p></td>
|
635
|
+
</tr>
|
636
|
+
<tr>
|
637
|
+
<td><tt>force_publisher_confirms</tt></td>
|
638
|
+
<td>false</td>
|
639
|
+
<td>Boolean</td>
|
640
|
+
<td><tt>HUTCH_FORCE_PUBLISHER_CONFIRMS</tt></td>
|
641
|
+
<td><p>Enables publisher confirms, forces Hutch::Broker#wait_for_confirms for</p></td>
|
642
|
+
</tr>
|
643
|
+
<tr>
|
644
|
+
<td><tt>enable_http_api_use</tt></td>
|
645
|
+
<td>true</td>
|
646
|
+
<td>Boolean</td>
|
647
|
+
<td><tt>HUTCH_ENABLE_HTTP_API_USE</tt></td>
|
648
|
+
<td><p>Should the RabbitMQ HTTP API be used?</p></td>
|
649
|
+
</tr>
|
650
|
+
<tr>
|
651
|
+
<td><tt>consumer_pool_abort_on_exception</tt></td>
|
652
|
+
<td>false</td>
|
653
|
+
<td>Boolean</td>
|
654
|
+
<td><tt>HUTCH_CONSUMER_POOL_ABORT_ON_EXCEPTION</tt></td>
|
655
|
+
<td><p>Should Bunny's consumer work pool threads abort on exception.</p></td>
|
656
|
+
</tr>
|
657
|
+
<tr>
|
658
|
+
<td><tt>consumer_tag_prefix</tt></td>
|
659
|
+
<td>hutch</td>
|
660
|
+
<td>String</td>
|
661
|
+
<td><tt>HUTCH_CONSUMER_TAG_PREFIX</tt></td>
|
662
|
+
<td><p>Prefix displayed on the consumers tags.</p></td>
|
663
|
+
</tr>
|
664
|
+
<tr>
|
665
|
+
<td><tt>namespace</tt></td>
|
666
|
+
<td>nil</td>
|
667
|
+
<td>String</td>
|
668
|
+
<td><tt>HUTCH_NAMESPACE</tt></td>
|
669
|
+
<td><p>A namespace to help group your queues</p></td>
|
670
|
+
</tr>
|
671
|
+
<tr>
|
672
|
+
<td><tt>group</tt></td>
|
673
|
+
<td>''</td>
|
674
|
+
<td>String</td>
|
675
|
+
<td><tt>HUTCH_GROUP</tt></td>
|
676
|
+
<td></td>
|
677
|
+
</tr>
|
678
|
+
</tbody>
|
679
|
+
</table>
|
data/Rakefile
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
require 'rspec/core/rake_task'
|
2
|
+
|
3
|
+
desc "Run an IRB session with Hutch pre-loaded"
|
4
|
+
task :console do
|
5
|
+
exec "irb -I lib -r hutch"
|
6
|
+
end
|
7
|
+
|
8
|
+
desc "Run the test suite"
|
9
|
+
RSpec::Core::RakeTask.new(:spec) do |t|
|
10
|
+
t.pattern = FileList['spec/**/*_spec.rb']
|
11
|
+
t.rspec_opts = %w(--color --format doc)
|
12
|
+
end
|
13
|
+
|
14
|
+
task default: :spec
|
15
|
+
|
16
|
+
#
|
17
|
+
# Re-generate API docs
|
18
|
+
#
|
19
|
+
require 'yard'
|
20
|
+
require 'yard/rake/yardoc_task'
|
21
|
+
YARD::Rake::YardocTask.new
|