hutch 0.18.0 → 1.1.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (73) hide show
  1. checksums.yaml +5 -5
  2. data/.gitignore +3 -0
  3. data/.rspec +1 -0
  4. data/.travis.yml +20 -8
  5. data/.yardopts +5 -0
  6. data/CHANGELOG.md +466 -2
  7. data/Gemfile +18 -4
  8. data/Guardfile +13 -4
  9. data/LICENSE +2 -1
  10. data/README.md +397 -32
  11. data/Rakefile +8 -1
  12. data/bin/ci/before_build.sh +20 -0
  13. data/bin/ci/install_on_debian.sh +46 -0
  14. data/hutch.gemspec +6 -7
  15. data/lib/hutch/acknowledgements/base.rb +16 -0
  16. data/lib/hutch/acknowledgements/nack_on_all_failures.rb +19 -0
  17. data/lib/hutch/adapters/march_hare.rb +1 -1
  18. data/lib/hutch/broker.rb +127 -103
  19. data/lib/hutch/cli.rb +66 -25
  20. data/lib/hutch/config.rb +230 -55
  21. data/lib/hutch/consumer.rb +42 -3
  22. data/lib/hutch/error_handlers/airbrake.rb +44 -16
  23. data/lib/hutch/error_handlers/base.rb +15 -0
  24. data/lib/hutch/error_handlers/bugsnag.rb +30 -0
  25. data/lib/hutch/error_handlers/honeybadger.rb +33 -18
  26. data/lib/hutch/error_handlers/logger.rb +12 -6
  27. data/lib/hutch/error_handlers/rollbar.rb +28 -0
  28. data/lib/hutch/error_handlers/sentry.rb +15 -12
  29. data/lib/hutch/error_handlers/sentry_raven.rb +31 -0
  30. data/lib/hutch/error_handlers.rb +3 -0
  31. data/lib/hutch/exceptions.rb +8 -1
  32. data/lib/hutch/logging.rb +5 -5
  33. data/lib/hutch/message.rb +2 -4
  34. data/lib/hutch/publisher.rb +75 -0
  35. data/lib/hutch/serializers/identity.rb +19 -0
  36. data/lib/hutch/serializers/json.rb +22 -0
  37. data/lib/hutch/tracers/datadog.rb +17 -0
  38. data/lib/hutch/tracers.rb +1 -0
  39. data/lib/hutch/version.rb +1 -2
  40. data/lib/hutch/waiter.rb +104 -0
  41. data/lib/hutch/worker.rb +81 -75
  42. data/lib/hutch.rb +15 -6
  43. data/lib/yard-settings/handler.rb +38 -0
  44. data/lib/yard-settings/yard-settings.rb +2 -0
  45. data/spec/hutch/broker_spec.rb +162 -77
  46. data/spec/hutch/cli_spec.rb +16 -3
  47. data/spec/hutch/config_spec.rb +121 -22
  48. data/spec/hutch/consumer_spec.rb +82 -4
  49. data/spec/hutch/error_handlers/airbrake_spec.rb +25 -10
  50. data/spec/hutch/error_handlers/bugsnag_spec.rb +55 -0
  51. data/spec/hutch/error_handlers/honeybadger_spec.rb +24 -2
  52. data/spec/hutch/error_handlers/logger_spec.rb +14 -1
  53. data/spec/hutch/error_handlers/rollbar_spec.rb +45 -0
  54. data/spec/hutch/error_handlers/sentry_raven_spec.rb +37 -0
  55. data/spec/hutch/error_handlers/sentry_spec.rb +21 -2
  56. data/spec/hutch/logger_spec.rb +12 -6
  57. data/spec/hutch/message_spec.rb +2 -2
  58. data/spec/hutch/serializers/json_spec.rb +17 -0
  59. data/spec/hutch/tracers/datadog_spec.rb +44 -0
  60. data/spec/hutch/waiter_spec.rb +51 -0
  61. data/spec/hutch/worker_spec.rb +89 -5
  62. data/spec/spec_helper.rb +7 -5
  63. data/templates/default/class/html/settings.erb +0 -0
  64. data/templates/default/class/setup.rb +4 -0
  65. data/templates/default/fulldoc/html/css/hutch.css +13 -0
  66. data/templates/default/layout/html/setup.rb +7 -0
  67. data/templates/default/method_details/html/settings.erb +5 -0
  68. data/templates/default/method_details/setup.rb +4 -0
  69. data/templates/default/method_details/text/settings.erb +0 -0
  70. data/templates/default/module/html/settings.erb +40 -0
  71. data/templates/default/module/setup.rb +4 -0
  72. metadata +62 -43
  73. data/circle.yml +0 -3
data/README.md CHANGED
@@ -1,24 +1,46 @@
1
1
  ![](http://cl.ly/image/3h0q3F3G142K/hutch.png)
2
2
 
3
+ [![Gem Version](https://badge.fury.io/rb/hutch.svg)](http://badge.fury.io/rb/hutch)
4
+ [![Build Status](https://travis-ci.org/gocardless/hutch.svg?branch=master)](https://travis-ci.org/gocardless/hutch)
5
+ [![Code Climate](https://codeclimate.com/github/gocardless/hutch.svg)](https://codeclimate.com/github/gocardless/hutch)
6
+
3
7
  Hutch is a Ruby library for enabling asynchronous inter-service communication
4
8
  in a service-oriented architecture, using RabbitMQ.
5
9
 
6
- [![Gem Version](https://badge.fury.io/rb/hutch.png)](http://badge.fury.io/rb/hutch)
7
- [![Build Status](https://travis-ci.org/gocardless/hutch.png?branch=master)](https://travis-ci.org/gocardless/hutch)
8
- [![Dependency Status](https://gemnasium.com/gocardless/hutch.png)](https://gemnasium.com/gocardless/hutch)
9
- [![Code Climate](https://codeclimate.com/github/gocardless/hutch.png)](https://codeclimate.com/github/gocardless/hutch)
10
-
11
10
  To install with RubyGems:
12
11
 
13
12
  ```
14
- $ gem install hutch
13
+ gem install hutch
15
14
  ```
16
15
 
17
- ## Project Maturity
16
+ <!-- Tocer[start]: Auto-generated, don't remove. -->
17
+
18
+ ### Table of Contents
19
+
20
+ - [Requirements](#requirements)
21
+ - [Overview](#overview)
22
+ - [Project Maturity](#project-maturity)
23
+ - [Consumers](#consumers)
24
+ - [Message Processing Tracers](#message-processing-tracers)
25
+ - [Running Hutch](#running-hutch)
26
+ - [Loading Consumers](#loading-consumers)
27
+ - [Stopping Hutch](#stopping-hutch)
28
+ - [Producers](#producers)
29
+ - [Producer Configuration](#producer-configuration)
30
+ - [Publisher Confirms](#publisher-confirms)
31
+ - [Writing Well-Behaved Publishers](#writing-well-behaved-publishers)
32
+ - [Configuration](#configuration)
33
+ - [Config File](#config-file)
34
+ - [Environment variables](#environment-variables)
35
+ - [Configuration precedence](#configuration-precedence)
36
+ - [Generated list of configuration options](#generated-list-of-configuration-options)
18
37
 
19
- Hutch is a moderately mature project (started in early 2013)
20
- that was extracted from production systems.
38
+ <!-- Tocer[finish]: Auto-generated, don't remove. -->
21
39
 
40
+ ## Requirements
41
+
42
+ - Hutch requires Ruby 2.4+ or JRuby 9K.
43
+ - Hutch requires RabbitMQ 3.3 or later.
22
44
 
23
45
  ## Overview
24
46
 
@@ -31,10 +53,16 @@ With Hutch, consumers are stored in separate files and include the `Hutch::Consu
31
53
  They are then loaded by a command line runner which connects to RabbitMQ, sets up queues and bindings,
32
54
  and so on. Publishers connect to RabbitMQ via `Hutch.connect` and publish using `Hutch.publish`.
33
55
 
34
- Hutch uses [Bunny](http://rubybunny.info) under the hood.
56
+ Hutch uses [Bunny](http://rubybunny.info) or [March Hare](http://rubymarchhare.info)
57
+ (on JRuby) under the hood.
58
+
59
+ ### Project Maturity
35
60
 
61
+ Hutch is a mature project that was originally extracted from production systems
62
+ at [GoCardless](https://gocardless.com) in 2013 and is now maintained by its contributors
63
+ and users.
36
64
 
37
- ## Defining Consumers
65
+ ## Consumers
38
66
 
39
67
  Consumers receive messages from a RabbitMQ queue. That queue may be bound to
40
68
  one or more topics (represented by routing keys).
@@ -52,8 +80,8 @@ message[:id] # => "02ABCXYZ"
52
80
  To subscribe to a topic, pass a routing key to `consume` in the class
53
81
  definition. To bind to multiple routing keys, simply pass extra routing keys
54
82
  in as additional arguments. Refer to the [RabbitMQ docs on topic exchanges
55
- ][topic-docs] for more information about how to use routing keys. Here's an
56
- example consumer:
83
+ ](http://www.rabbitmq.com/tutorials/tutorial-five-ruby.html) for more information
84
+ about how to use routing keys. Here's an example consumer:
57
85
 
58
86
  ```ruby
59
87
  class FailedPaymentConsumer
@@ -81,6 +109,28 @@ class FailedPaymentConsumer
81
109
  end
82
110
  ```
83
111
 
112
+ It is possible to set some custom options to consumer's queue explicitly.
113
+ This example sets the consumer's queue as a
114
+ [quorum queue](https://www.rabbitmq.com/quorum-queues.html)
115
+ and to operate in the [lazy mode](https://www.rabbitmq.com/lazy-queues.html).
116
+ The `initial_group_size`
117
+ [argument](https://www.rabbitmq.com/quorum-queues.html#replication-factor) is
118
+ optional.
119
+
120
+ ```ruby
121
+ class FailedPaymentConsumer
122
+ include Hutch::Consumer
123
+ consume 'gc.ps.payment.failed'
124
+ queue_name 'failed_payments'
125
+ lazy_queue
126
+ quorum_queue initial_group_size: 3
127
+
128
+ def process(message)
129
+ mark_payment_as_failed(message[:id])
130
+ end
131
+ end
132
+ ```
133
+
84
134
  You can also set custom arguments per consumer. This example declares a consumer with
85
135
  a maximum length of 10 messages:
86
136
 
@@ -92,7 +142,9 @@ class FailedPaymentConsumer
92
142
  end
93
143
  ```
94
144
 
95
- Custom queue arguments can be found on [this page](https://www.rabbitmq.com/extensions.html).
145
+ This sets the `x-max-length` header. For more details, see the [RabbitMQ
146
+ documentation on Queue Length Limit](https://www.rabbitmq.com/maxlength.html). To find out more
147
+ about custom queue arguments, consult the [RabbitMQ documentation on AMQP Protocol Extensions](https://www.rabbitmq.com/extensions.html).
96
148
 
97
149
  Consumers can write to Hutch's log by calling the logger method. The logger method returns
98
150
  a [Logger object](http://ruby-doc.org/stdlib-2.1.2/libdoc/logger/rdoc/Logger.html).
@@ -101,7 +153,7 @@ a [Logger object](http://ruby-doc.org/stdlib-2.1.2/libdoc/logger/rdoc/Logger.htm
101
153
  class FailedPaymentConsumer
102
154
  include Hutch::Consumer
103
155
  consume 'gc.ps.payment.failed'
104
-
156
+
105
157
  def process(message)
106
158
  logger.info "Marking payment #{message[:id]} as failed"
107
159
  mark_payment_as_failed(message[:id])
@@ -130,11 +182,19 @@ to learn more.
130
182
 
131
183
  Tracers allow you to track message processing.
132
184
 
133
- #### NewRelic
185
+ This will enable NewRelic custom instrumentation:
186
+
134
187
  ```ruby
135
188
  Hutch::Config.set(:tracer, Hutch::Tracers::NewRelic)
136
189
  ```
137
- This will enable NewRelic custom instrumentation. Batteries included! Screenshoots available [here](https://monosnap.com/list/557020a000779174f23467e3).
190
+
191
+ And this will enable Datadog custom instrumentation:
192
+
193
+ ```ruby
194
+ Hutch::Config.set(:tracer, Hutch::Tracers::Datadog)
195
+ ```
196
+
197
+ Batteries included!
138
198
 
139
199
  ## Running Hutch
140
200
 
@@ -187,13 +247,55 @@ directory of a Rails app, or pass the path to a Rails app in with the
187
247
  the `app/consumers/` directory, to allow them to be auto-loaded when Rails
188
248
  boots.
189
249
 
250
+ If you're using the new Zeitwerk autoloader (enabled by default in Rails 6)
251
+ and the consumers are not loaded in development environment you will need to
252
+ trigger the autoloading in an initializer with
253
+
254
+ ```ruby
255
+ ::Zeitwerk::Loader.eager_load_all
256
+ ```
257
+
258
+ or with something more specific like
259
+
260
+ ```ruby
261
+ autoloader = Rails.autoloaders.main
262
+
263
+ Dir.glob(File.join('app/consumers', '*_consumer.rb')).each do |consumer|
264
+ autoloader.preload(consumer)
265
+ end
266
+ ```
267
+
268
+ ### Consumer Groups
269
+
270
+ It is possible to load only a subset of consumers. This is done by defining a consumer
271
+ group under the `consumer_groups` configuration key:
272
+
273
+ ``` yaml
274
+ consumer_groups:
275
+ payments:
276
+ - DepositConsumer
277
+ - CashoutConsumer
278
+ notification:
279
+ - EmailNotificationConsumer
280
+ ```
281
+
282
+ To only load a group of consumers, use the `--only-group` option:
283
+
284
+ ``` shell
285
+ hutch --only-group=payments --config=/path/to/hutch.yaml
286
+ ```
287
+
288
+ ### Loading Consumers Manually (One-by-One)
289
+
190
290
  To require files that define consumers manually, simply pass each file as an
191
291
  option to `--require`. Hutch will automatically detect whether you've provided
192
292
  a Rails app or a standard file, and take the appropriate behaviour:
193
293
 
194
294
  ```bash
195
- $ hutch --require path/to/rails-app # loads a rails app
196
- $ hutch --require path/to/file.rb # loads a ruby file
295
+ # loads a rails app
296
+ hutch --require path/to/rails-app
297
+ # loads a ruby file
298
+ hutch --require path/to/file.rb
197
299
  ```
198
300
 
199
301
  ### Stopping Hutch
@@ -270,10 +372,10 @@ AMQP.connect(host: config[:host]) do |connection|
270
372
  end
271
373
  ```
272
374
 
273
- If using publisher confirms with amqp gem, see [this issue][pc-issue]
274
- and [this gist][pc-gist] for more info.
375
+ If using publisher confirms with amqp gem, see [this issue](https://github.com/ruby-amqp/amqp/issues/92)
376
+ and [this gist](https://gist.github.com/3042381) for more info.
275
377
 
276
- ## Configuration Reference
378
+ ## Configuration
277
379
 
278
380
  ### Config File
279
381
 
@@ -289,6 +391,8 @@ Known configuration parameters are:
289
391
  * `mq_tls`: should TLS be used? (default: `false`)
290
392
  * `mq_tls_cert`: path to client TLS certificate (public key)
291
393
  * `mq_tls_key`: path to client TLS private key
394
+ * `mq_tls_ca_certificates`: array of paths to CA keys (if not specified to Hutch, will default to Bunny defaults which are system-dependent)
395
+ * `mq_verify_peer`: should SSL certificate be verified? (default: `true`)
292
396
  * `require_paths`: array of paths to require
293
397
  * `autoload_rails`: should Hutch command line runner try to automatically load Rails environment files?
294
398
  * `daemonise`: should Hutch runner process daemonise?
@@ -298,18 +402,279 @@ Known configuration parameters are:
298
402
  tracked (e.g. using `Hutch::Broker#confirm_select` callback or `Hutch::Broker#wait_for_confirms`)
299
403
  * `force_publisher_confirms`: enables publisher confirms, forces `Hutch::Broker#wait_for_confirms` for every publish. **This is the safest option which also offers the lowest throughput**.
300
404
  * `log_level`: log level used by the standard Ruby logger (default: `Logger::INFO`)
405
+ * `error_handlers`: a list of error handler objects, see classes in `Hutch::ErrorHandlers`. All configured
406
+ handlers will be invoked unconditionally in the order listed.
407
+ * `error_acknowledgements`: a chain of responsibility of objects that acknowledge/reject/requeue messages when an
408
+ exception happens, see classes in `Hutch::Acknowledgements`.
301
409
  * `mq_exchange`: exchange to use for publishing (default: `hutch`)
302
410
  * `heartbeat`: [RabbitMQ heartbeat timeout](http://rabbitmq.com/heartbeats.html) (default: `30`)
303
411
  * `connection_timeout`: Bunny's socket open timeout (default: `11`)
304
412
  * `read_timeout`: Bunny's socket read timeout (default: `11`)
305
- * `write_timemout`: Bunny's socket write timeout (default: `11`)
413
+ * `write_timeout`: Bunny's socket write timeout (default: `11`)
414
+ * `automatically_recover`: Bunny's enable/disable network recovery (default: `true`)
415
+ * `network_recovery_interval`: Bunny's reconnect interval (default: `1`)
306
416
  * `tracer`: tracer to use to track message processing
307
-
308
-
309
- ## Supported RabbitMQ Versions
310
-
311
- Hutch requires RabbitMQ 3.3 or later.
312
-
313
- ---
314
-
315
- GoCardless open source. If you do too, come [join us](https://gocardless.com/jobs/backend_developer).
417
+ * `namespace`: A namespace string to help group your queues (default: `nil`)
418
+
419
+ ### Environment variables
420
+
421
+ The file configuration options mentioned above can also be passed in via environment variables, using the `HUTCH_` prefix, eg.
422
+
423
+ * `connection_timeout` &rarr; `HUTCH_CONNECTION_TIMEOUT`.
424
+
425
+ ### Configuration precedence
426
+
427
+ In order from lowest to highest precedence:
428
+
429
+ 0. Default values
430
+ 0. `HUTCH_*` environment variables
431
+ 0. Configuration file
432
+ 0. Explicit settings through `Hutch::Config.set`
433
+
434
+ ### Generated list of configuration options
435
+
436
+ Generate with
437
+
438
+ 0. `yard doc lib/hutch/config.rb`
439
+ 0. Copy the _Configuration_ section from `doc/Hutch/Config.html` here, with the anchor tags stripped.
440
+
441
+ <table border="1" class="settings" style="overflow:visible;">
442
+ <thead>
443
+ <tr>
444
+ <th>
445
+ Setting name
446
+ </th>
447
+ <th>
448
+ Default value
449
+ </th>
450
+ <th>
451
+ Type
452
+ </th>
453
+ <th>
454
+ ENV variable
455
+ </th>
456
+ <th>
457
+ Description
458
+ </th>
459
+ </tr>
460
+ </thead>
461
+ <tbody>
462
+ <tr>
463
+ <td><tt>mq_host</tt></td>
464
+ <td>127.0.0.1</td>
465
+ <td>String</td>
466
+ <td><tt>HUTCH_MQ_HOST</tt></td>
467
+ <td><p>RabbitMQ hostname</p></td>
468
+ </tr>
469
+ <tr>
470
+ <td><tt>mq_exchange</tt></td>
471
+ <td>hutch</td>
472
+ <td>String</td>
473
+ <td><tt>HUTCH_MQ_EXCHANGE</tt></td>
474
+ <td><p>RabbitMQ Exchange to use for publishing</p></td>
475
+ </tr>
476
+ <tr>
477
+ <td><tt>mq_exchange_type</tt></td>
478
+ <td>topic</td>
479
+ <td>String</td>
480
+ <td><tt>HUTCH_MQ_EXCHANGE_TYPE</tt></td>
481
+ <td><p>RabbitMQ Exchange type to use for publishing</p></td>
482
+ </tr>
483
+ <tr>
484
+ <td><tt>mq_vhost</tt></td>
485
+ <td>/</td>
486
+ <td>String</td>
487
+ <td><tt>HUTCH_MQ_VHOST</tt></td>
488
+ <td><p>RabbitMQ vhost to use</p></td>
489
+ </tr>
490
+ <tr>
491
+ <td><tt>mq_username</tt></td>
492
+ <td>guest</td>
493
+ <td>String</td>
494
+ <td><tt>HUTCH_MQ_USERNAME</tt></td>
495
+ <td><p>RabbitMQ username to use.</p></td>
496
+ </tr>
497
+ <tr>
498
+ <td><tt>mq_password</tt></td>
499
+ <td>guest</td>
500
+ <td>String</td>
501
+ <td><tt>HUTCH_MQ_PASSWORD</tt></td>
502
+ <td><p>RabbitMQ password</p></td>
503
+ </tr>
504
+ <tr>
505
+ <td><tt>uri</tt></td>
506
+ <td>nil</td>
507
+ <td>String</td>
508
+ <td><tt>HUTCH_URI</tt></td>
509
+ <td><p>RabbitMQ URI (takes precedence over MQ username, password, host, port and vhost settings)</p></td>
510
+ </tr>
511
+ <tr>
512
+ <td><tt>mq_api_host</tt></td>
513
+ <td>127.0.0.1</td>
514
+ <td>String</td>
515
+ <td><tt>HUTCH_MQ_API_HOST</tt></td>
516
+ <td><p>RabbitMQ HTTP API hostname</p></td>
517
+ </tr>
518
+ <tr>
519
+ <td><tt>mq_port</tt></td>
520
+ <td>5672</td>
521
+ <td>Number</td>
522
+ <td><tt>HUTCH_MQ_PORT</tt></td>
523
+ <td><p>RabbitMQ port</p></td>
524
+ </tr>
525
+ <tr>
526
+ <td><tt>mq_api_port</tt></td>
527
+ <td>15672</td>
528
+ <td>Number</td>
529
+ <td><tt>HUTCH_MQ_API_PORT</tt></td>
530
+ <td><p>RabbitMQ HTTP API port</p></td>
531
+ </tr>
532
+ <tr>
533
+ <td><tt>heartbeat</tt></td>
534
+ <td>30</td>
535
+ <td>Number</td>
536
+ <td><tt>HUTCH_HEARTBEAT</tt></td>
537
+ <td><p><a href="http://rabbitmq.com/heartbeats.html">RabbitMQ heartbeat timeout</a></p></td>
538
+ </tr>
539
+ <tr>
540
+ <td><tt>channel_prefetch</tt></td>
541
+ <td>0</td>
542
+ <td>Number</td>
543
+ <td><tt>HUTCH_CHANNEL_PREFETCH</tt></td>
544
+ <td><p>The <tt>basic.qos</tt> prefetch value to use.</p></td>
545
+ </tr>
546
+ <tr>
547
+ <td><tt>connection_timeout</tt></td>
548
+ <td>11</td>
549
+ <td>Number</td>
550
+ <td><tt>HUTCH_CONNECTION_TIMEOUT</tt></td>
551
+ <td><p>Bunny's socket open timeout</p></td>
552
+ </tr>
553
+ <tr>
554
+ <td><tt>read_timeout</tt></td>
555
+ <td>11</td>
556
+ <td>Number</td>
557
+ <td><tt>HUTCH_READ_TIMEOUT</tt></td>
558
+ <td><p>Bunny's socket read timeout</p></td>
559
+ </tr>
560
+ <tr>
561
+ <td><tt>write_timeout</tt></td>
562
+ <td>11</td>
563
+ <td>Number</td>
564
+ <td><tt>HUTCH_WRITE_TIMEOUT</tt></td>
565
+ <td><p>Bunny's socket write timeout</p></td>
566
+ </tr>
567
+ <tr>
568
+ <td><tt>automatically_recover</tt></td>
569
+ <td>true</td>
570
+ <td>Boolean</td>
571
+ <td><tt>HUTCH_AUTOMATICALLY_RECOVER</tt></td>
572
+ <td><p>Bunny's enable/disable network recovery</p></td>
573
+ </tr>
574
+ <tr>
575
+ <td><tt>network_recovery_interval</tt></td>
576
+ <td>1</td>
577
+ <td>Number</td>
578
+ <td><tt>HUTCH_NETWORK_RECOVERY_INTERVAL</tt></td>
579
+ <td><p>Bunny's reconnect interval</p></td>
580
+ </tr>
581
+ <tr>
582
+ <td><tt>graceful_exit_timeout</tt></td>
583
+ <td>11</td>
584
+ <td>Number</td>
585
+ <td><tt>HUTCH_GRACEFUL_EXIT_TIMEOUT</tt></td>
586
+ <td><p>FIXME: DOCUMENT THIS</p></td>
587
+ </tr>
588
+ <tr>
589
+ <td><tt>consumer_pool_size</tt></td>
590
+ <td>1</td>
591
+ <td>Number</td>
592
+ <td><tt>HUTCH_CONSUMER_POOL_SIZE</tt></td>
593
+ <td><p>Bunny consumer work pool size</p></td>
594
+ </tr>
595
+ <tr>
596
+ <td><tt>mq_tls</tt></td>
597
+ <td>false</td>
598
+ <td>Boolean</td>
599
+ <td><tt>HUTCH_MQ_TLS</tt></td>
600
+ <td><p>Should TLS be used?</p></td>
601
+ </tr>
602
+ <tr>
603
+ <td><tt>mq_verify_peer</tt></td>
604
+ <td>true</td>
605
+ <td>Boolean</td>
606
+ <td><tt>HUTCH_MQ_VERIFY_PEER</tt></td>
607
+ <td><p>Should SSL certificate be verified?</p></td>
608
+ </tr>
609
+ <tr>
610
+ <td><tt>mq_api_ssl</tt></td>
611
+ <td>false</td>
612
+ <td>Boolean</td>
613
+ <td><tt>HUTCH_MQ_API_SSL</tt></td>
614
+ <td><p>Should SSL be used for the RabbitMQ API?</p></td>
615
+ </tr>
616
+ <tr>
617
+ <td><tt>autoload_rails</tt></td>
618
+ <td>true</td>
619
+ <td>Boolean</td>
620
+ <td><tt>HUTCH_AUTOLOAD_RAILS</tt></td>
621
+ <td><p>Should the current Rails app directory be required?</p></td>
622
+ </tr>
623
+ <tr>
624
+ <td><tt>daemonise</tt></td>
625
+ <td>false</td>
626
+ <td>Boolean</td>
627
+ <td><tt>HUTCH_DAEMONISE</tt></td>
628
+ <td><p>Should the Hutch runner process daemonise?</p></td>
629
+ </tr>
630
+ <tr>
631
+ <td><tt>publisher_confirms</tt></td>
632
+ <td>false</td>
633
+ <td>Boolean</td>
634
+ <td><tt>HUTCH_PUBLISHER_CONFIRMS</tt></td>
635
+ <td><p>Should RabbitMQ publisher confirms be enabled?</p></td>
636
+ </tr>
637
+ <tr>
638
+ <td><tt>force_publisher_confirms</tt></td>
639
+ <td>false</td>
640
+ <td>Boolean</td>
641
+ <td><tt>HUTCH_FORCE_PUBLISHER_CONFIRMS</tt></td>
642
+ <td><p>Enables publisher confirms, forces Hutch::Broker#wait_for_confirms for</p></td>
643
+ </tr>
644
+ <tr>
645
+ <td><tt>enable_http_api_use</tt></td>
646
+ <td>true</td>
647
+ <td>Boolean</td>
648
+ <td><tt>HUTCH_ENABLE_HTTP_API_USE</tt></td>
649
+ <td><p>Should the RabbitMQ HTTP API be used?</p></td>
650
+ </tr>
651
+ <tr>
652
+ <td><tt>consumer_pool_abort_on_exception</tt></td>
653
+ <td>false</td>
654
+ <td>Boolean</td>
655
+ <td><tt>HUTCH_CONSUMER_POOL_ABORT_ON_EXCEPTION</tt></td>
656
+ <td><p>Should Bunny's consumer work pool threads abort on exception.</p></td>
657
+ </tr>
658
+ <tr>
659
+ <td><tt>consumer_tag_prefix</tt></td>
660
+ <td>hutch</td>
661
+ <td>String</td>
662
+ <td><tt>HUTCH_CONSUMER_TAG_PREFIX</tt></td>
663
+ <td><p>Prefix displayed on the consumers tags.</p></td>
664
+ </tr>
665
+ <tr>
666
+ <td><tt>namespace</tt></td>
667
+ <td>nil</td>
668
+ <td>String</td>
669
+ <td><tt>HUTCH_NAMESPACE</tt></td>
670
+ <td><p>A namespace to help group your queues</p></td>
671
+ </tr>
672
+ <tr>
673
+ <td><tt>group</tt></td>
674
+ <td>''</td>
675
+ <td>String</td>
676
+ <td><tt>HUTCH_GROUP</tt></td>
677
+ <td></td>
678
+ </tr>
679
+ </tbody>
680
+ </table>
data/Rakefile CHANGED
@@ -11,4 +11,11 @@ RSpec::Core::RakeTask.new(:spec) do |t|
11
11
  t.rspec_opts = %w(--color --format doc)
12
12
  end
13
13
 
14
- task :default => :spec
14
+ task default: :spec
15
+
16
+ #
17
+ # Re-generate API docs
18
+ #
19
+ require 'yard'
20
+ require 'yard/rake/yardoc_task'
21
+ YARD::Rake::YardocTask.new
@@ -0,0 +1,20 @@
1
+ #!/usr/bin/env sh
2
+
3
+ CTL=${BUNNY_RABBITMQCTL:-"sudo rabbitmqctl"}
4
+ PLUGINS=${BUNNY_RABBITMQ_PLUGINS:-"sudo rabbitmq-plugins"}
5
+
6
+ echo "Will use rabbitmqctl at ${CTL}"
7
+ echo "Will use rabbitmq-plugins at ${PLUGINS}"
8
+
9
+ $PLUGINS enable rabbitmq_management
10
+
11
+ sleep 3
12
+
13
+ # guest:guest has full access to /
14
+ $CTL add_vhost /
15
+ $CTL add_user guest guest
16
+ $CTL set_permissions -p / guest ".*" ".*" ".*"
17
+
18
+ # Reduce retention policy for faster publishing of stats
19
+ $CTL eval 'supervisor2:terminate_child(rabbit_mgmt_sup_sup, rabbit_mgmt_sup), application:set_env(rabbitmq_management, sample_retention_policies, [{global, [{605, 1}]}, {basic, [{605, 1}]}, {detailed, [{10, 1}]}]), rabbit_mgmt_sup_sup:start_child().' || true
20
+ $CTL eval 'supervisor2:terminate_child(rabbit_mgmt_agent_sup_sup, rabbit_mgmt_agent_sup), application:set_env(rabbitmq_management_agent, sample_retention_policies, [{global, [{605, 1}]}, {basic, [{605, 1}]}, {detailed, [{10, 1}]}]), rabbit_mgmt_agent_sup_sup:start_child().' || true
@@ -0,0 +1,46 @@
1
+ #!/bin/sh
2
+
3
+ sudo apt-get install curl gnupg debian-keyring debian-archive-keyring apt-transport-https -y
4
+
5
+ ## Team RabbitMQ's main signing key
6
+ sudo apt-key adv --keyserver "hkps://keys.openpgp.org" --recv-keys "0x0A9AF2115F4687BD29803A206B73A36E6026DFCA"
7
+ ## Launchpad PPA that provides modern Erlang releases
8
+ sudo apt-key adv --keyserver "keyserver.ubuntu.com" --recv-keys "F77F1EDA57EBB1CC"
9
+ ## PackageCloud RabbitMQ repository
10
+ sudo apt-key adv --keyserver "keyserver.ubuntu.com" --recv-keys "F6609E60DC62814E"
11
+
12
+ ## Add apt repositories maintained by Team RabbitMQ
13
+ sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
14
+ ## Provides modern Erlang/OTP releases
15
+ ##
16
+ ## "bionic" as distribution name should work for any reasonably recent Ubuntu or Debian release.
17
+ ## See the release to distribution mapping table in RabbitMQ doc guides to learn more.
18
+ deb http://ppa.launchpad.net/rabbitmq/rabbitmq-erlang/ubuntu bionic main
19
+ deb-src http://ppa.launchpad.net/rabbitmq/rabbitmq-erlang/ubuntu bionic main
20
+
21
+ ## Provides RabbitMQ
22
+ ##
23
+ ## "bionic" as distribution name should work for any reasonably recent Ubuntu or Debian release.
24
+ ## See the release to distribution mapping table in RabbitMQ doc guides to learn more.
25
+ deb https://packagecloud.io/rabbitmq/rabbitmq-server/ubuntu/ bionic main
26
+ deb-src https://packagecloud.io/rabbitmq/rabbitmq-server/ubuntu/ bionic main
27
+ EOF
28
+
29
+ ## Update package indices
30
+ sudo apt-get update -y
31
+
32
+ ## Install Erlang packages
33
+ sudo apt-get install -y erlang-base \
34
+ erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
35
+ erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
36
+ erlang-runtime-tools erlang-snmp erlang-ssl \
37
+ erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl
38
+
39
+ ## Install rabbitmq-server and its dependencies
40
+ sudo apt-get install rabbitmq-server -y --fix-missing
41
+
42
+ sudo service rabbitmq-server start
43
+
44
+ sudo rabbitmqctl await_startup --timeout 120
45
+
46
+ until sudo lsof -i:5672; do echo "Waiting for RabbitMQ to start..."; sleep 1; done
data/hutch.gemspec CHANGED
@@ -3,22 +3,21 @@ require File.expand_path('../lib/hutch/version', __FILE__)
3
3
  Gem::Specification.new do |gem|
4
4
  if defined?(JRUBY_VERSION)
5
5
  gem.platform = 'java'
6
- gem.add_runtime_dependency 'march_hare', '>= 2.11.0'
6
+ gem.add_runtime_dependency 'march_hare', '>= 3.0.0'
7
7
  else
8
8
  gem.platform = Gem::Platform::RUBY
9
- gem.add_runtime_dependency 'bunny', '>= 1.7.0'
9
+ gem.add_runtime_dependency 'bunny', '>= 2.16', '< 3.0'
10
10
  end
11
11
  gem.add_runtime_dependency 'carrot-top', '~> 0.0.7'
12
- gem.add_runtime_dependency 'multi_json', '~> 1.5'
13
- gem.add_runtime_dependency 'activesupport', '>= 3.0'
14
- gem.add_development_dependency 'rspec', '~> 3.0'
15
- gem.add_development_dependency 'simplecov', '~> 0.7.1'
12
+ gem.add_runtime_dependency 'multi_json', '~> 1.14'
13
+ gem.add_runtime_dependency 'activesupport', '>= 4.2', '< 7'
16
14
 
17
15
  gem.name = 'hutch'
18
16
  gem.summary = 'Easy inter-service communication using RabbitMQ.'
19
- gem.description = 'Hutch is a Ruby library for enabling asynchronous ' +
17
+ gem.description = 'Hutch is a Ruby library for enabling asynchronous ' \
20
18
  'inter-service communication using RabbitMQ.'
21
19
  gem.version = Hutch::VERSION.dup
20
+ gem.required_ruby_version = '>= 2.2'
22
21
  gem.authors = ['Harry Marr']
23
22
  gem.email = ['developers@gocardless.com']
24
23
  gem.homepage = 'https://github.com/gocardless/hutch'
@@ -0,0 +1,16 @@
1
+ module Hutch
2
+ module Acknowledgements
3
+ # Defines acknowledgement handler interface.
4
+ class Base
5
+ # Implements negative acknowledgement/requeueing logic
6
+ # and returns a boolean to indicate whether acknowledgement
7
+ # was performed. If false is returned, next handler in the
8
+ # chain will be invoked.
9
+ #
10
+ # The chain always falls back to unconditional nacking.
11
+ def handle(delivery_info, properties, broker, ex)
12
+ raise NotImplementedError.new
13
+ end
14
+ end
15
+ end
16
+ end
@@ -0,0 +1,19 @@
1
+ require 'hutch/logging'
2
+ require 'hutch/acknowledgements/base'
3
+
4
+ module Hutch
5
+ module Acknowledgements
6
+ class NackOnAllFailures < Base
7
+ include Logging
8
+
9
+ def handle(delivery_info, properties, broker, ex)
10
+ prefix = "message(#{properties.message_id || '-'}): "
11
+ logger.debug "#{prefix} nacking message"
12
+
13
+ broker.nack(delivery_info.delivery_tag)
14
+
15
+ true
16
+ end
17
+ end
18
+ end
19
+ end