hutch 0.25.0.pre.rc1-java → 2.0.0.rc1-java

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (63) hide show
  1. checksums.yaml +5 -5
  2. data/CHANGELOG.md +402 -5
  3. data/LICENSE +1 -0
  4. data/README.md +157 -47
  5. data/bin/ci/before_build.sh +20 -0
  6. data/bin/ci/before_build_docker.sh +15 -0
  7. data/bin/ci/install_on_debian.sh +46 -0
  8. data/lib/hutch/acknowledgements/nack_on_all_failures.rb +1 -1
  9. data/lib/hutch/adapters/bunny.rb +29 -1
  10. data/lib/hutch/adapters/march_hare.rb +24 -0
  11. data/lib/hutch/broker.rb +122 -36
  12. data/lib/hutch/cli.rb +23 -12
  13. data/lib/hutch/config.rb +45 -13
  14. data/lib/hutch/consumer.rb +63 -4
  15. data/lib/hutch/error_handlers/airbrake.rb +20 -2
  16. data/lib/hutch/error_handlers/base.rb +15 -0
  17. data/lib/hutch/error_handlers/bugsnag.rb +30 -0
  18. data/lib/hutch/error_handlers/honeybadger.rb +11 -3
  19. data/lib/hutch/error_handlers/logger.rb +7 -2
  20. data/lib/hutch/error_handlers/rollbar.rb +28 -0
  21. data/lib/hutch/error_handlers/sentry.rb +13 -11
  22. data/lib/hutch/error_handlers.rb +2 -1
  23. data/lib/hutch/publisher.rb +1 -1
  24. data/lib/hutch/serializers/json.rb +3 -3
  25. data/lib/hutch/tracers/datadog.rb +18 -0
  26. data/lib/hutch/tracers.rb +1 -1
  27. data/lib/hutch/version.rb +1 -2
  28. data/lib/hutch/waiter.rb +1 -1
  29. data/lib/hutch/worker.rb +74 -8
  30. data/lib/hutch.rb +8 -4
  31. data/lib/yard-settings/yard-settings.rb +2 -2
  32. data/spec/hutch/broker_spec.rb +222 -10
  33. data/spec/hutch/cli_spec.rb +25 -9
  34. data/spec/hutch/config_spec.rb +84 -12
  35. data/spec/hutch/consumer_spec.rb +93 -4
  36. data/spec/hutch/error_handlers/airbrake_spec.rb +23 -1
  37. data/spec/hutch/error_handlers/bugsnag_spec.rb +56 -0
  38. data/spec/hutch/error_handlers/honeybadger_spec.rb +22 -1
  39. data/spec/hutch/error_handlers/logger_spec.rb +12 -1
  40. data/spec/hutch/error_handlers/rollbar_spec.rb +45 -0
  41. data/spec/hutch/error_handlers/sentry_spec.rb +27 -2
  42. data/spec/hutch/message_spec.rb +1 -1
  43. data/spec/hutch/tracers/datadog_spec.rb +46 -0
  44. data/spec/hutch/waiter_spec.rb +2 -2
  45. data/spec/hutch/worker_spec.rb +139 -25
  46. data/spec/integration/channel_recovery_spec.rb +187 -0
  47. data/spec/integration/publish_consume_spec.rb +47 -0
  48. metadata +35 -61
  49. data/.gitignore +0 -10
  50. data/.rspec +0 -1
  51. data/.travis.yml +0 -19
  52. data/.yardopts +0 -5
  53. data/Gemfile +0 -33
  54. data/Guardfile +0 -14
  55. data/Rakefile +0 -21
  56. data/examples/consumer.rb +0 -13
  57. data/examples/producer.rb +0 -10
  58. data/hutch.gemspec +0 -29
  59. data/lib/hutch/error_handlers/opbeat.rb +0 -24
  60. data/lib/hutch/tracers/opbeat.rb +0 -37
  61. data/spec/hutch/error_handlers/opbeat_spec.rb +0 -22
  62. data/spec/spec_helper.rb +0 -42
  63. data/spec/tracers/opbeat_spec.rb +0 -44
data/README.md CHANGED
@@ -1,9 +1,5 @@
1
- ![](http://cl.ly/image/3h0q3F3G142K/hutch.png)
2
-
3
- [![Gem Version](https://badge.fury.io/rb/hutch.svg)](http://badge.fury.io/rb/hutch)
4
- [![Build Status](https://travis-ci.org/gocardless/hutch.svg?branch=master)](https://travis-ci.org/gocardless/hutch)
5
- [![Dependency Status](https://gemnasium.com/gocardless/hutch.svg)](https://gemnasium.com/gocardless/hutch)
6
- [![Code Climate](https://codeclimate.com/github/gocardless/hutch.svg)](https://codeclimate.com/github/gocardless/hutch)
1
+ [![Gem Version](https://badge.fury.io/rb/hutch.svg)](https://badge.fury.io/rb/hutch)
2
+ [![CI](https://github.com/ruby-amqp/hutch/actions/workflows/test.yml/badge.svg)](https://github.com/ruby-amqp/hutch/actions/workflows/test.yml)
7
3
 
8
4
  Hutch is a Ruby library for enabling asynchronous inter-service communication
9
5
  in a service-oriented architecture, using RabbitMQ.
@@ -25,6 +21,8 @@ gem install hutch
25
21
  - [Message Processing Tracers](#message-processing-tracers)
26
22
  - [Running Hutch](#running-hutch)
27
23
  - [Loading Consumers](#loading-consumers)
24
+ - [Consumer Groups](#consumer-groups)
25
+ - [Loading Consumers Manually (One-by-One)](#loading-consumers-manually-one-by-one)
28
26
  - [Stopping Hutch](#stopping-hutch)
29
27
  - [Producers](#producers)
30
28
  - [Producer Configuration](#producer-configuration)
@@ -40,8 +38,8 @@ gem install hutch
40
38
 
41
39
  ## Requirements
42
40
 
43
- - Hutch requires Ruby 2.0+ or JRuby 9K.
44
- - Hutch requires RabbitMQ 3.3 or later.
41
+ - Hutch requires Ruby 3.2+ or JRuby 10+
42
+ - Hutch requires RabbitMQ 3.13 or later
45
43
 
46
44
  ## Overview
47
45
 
@@ -54,13 +52,14 @@ With Hutch, consumers are stored in separate files and include the `Hutch::Consu
54
52
  They are then loaded by a command line runner which connects to RabbitMQ, sets up queues and bindings,
55
53
  and so on. Publishers connect to RabbitMQ via `Hutch.connect` and publish using `Hutch.publish`.
56
54
 
57
- Hutch uses [Bunny](http://rubybunny.info) or [March Hare](http://rubymarchhare.info)
55
+ Hutch uses [Bunny](https://github.com/ruby-amqp/bunny) or [March Hare](https://github.com/ruby-amqp/march_hare)
58
56
  (on JRuby) under the hood.
59
57
 
60
58
  ### Project Maturity
61
59
 
62
- Hutch is a moderately mature project (started in early 2013)
63
- that was extracted from production systems.
60
+ Hutch is a mature project that was originally extracted from production systems
61
+ at [GoCardless](https://gocardless.com) in 2013 and is now maintained by its contributors
62
+ and users.
64
63
 
65
64
  ## Consumers
66
65
 
@@ -80,7 +79,7 @@ message[:id] # => "02ABCXYZ"
80
79
  To subscribe to a topic, pass a routing key to `consume` in the class
81
80
  definition. To bind to multiple routing keys, simply pass extra routing keys
82
81
  in as additional arguments. Refer to the [RabbitMQ docs on topic exchanges
83
- ](http://www.rabbitmq.com/tutorials/tutorial-five-ruby.html) for more information
82
+ ](https://www.rabbitmq.com/tutorials/tutorial-five-ruby) for more information
84
83
  about how to use routing keys. Here's an example consumer:
85
84
 
86
85
  ```ruby
@@ -109,6 +108,28 @@ class FailedPaymentConsumer
109
108
  end
110
109
  ```
111
110
 
111
+ It is possible to set some custom options to consumer's queue explicitly.
112
+ This example sets the consumer's queue as a
113
+ [quorum queue](https://www.rabbitmq.com/docs/quorum-queues)
114
+ and to operate in the [lazy mode](https://www.rabbitmq.com/docs/lazy-queues).
115
+ The `initial_group_size`
116
+ [argument](https://www.rabbitmq.com/docs/quorum-queues#replication-factor) is
117
+ optional.
118
+
119
+ ```ruby
120
+ class FailedPaymentConsumer
121
+ include Hutch::Consumer
122
+ consume 'gc.ps.payment.failed'
123
+ queue_name 'failed_payments'
124
+ lazy_queue
125
+ quorum_queue initial_group_size: 3
126
+
127
+ def process(message)
128
+ mark_payment_as_failed(message[:id])
129
+ end
130
+ end
131
+ ```
132
+
112
133
  You can also set custom arguments per consumer. This example declares a consumer with
113
134
  a maximum length of 10 messages:
114
135
 
@@ -121,11 +142,11 @@ end
121
142
  ```
122
143
 
123
144
  This sets the `x-max-length` header. For more details, see the [RabbitMQ
124
- documentation on Queue Length Limit](https://www.rabbitmq.com/maxlength.html). To find out more
125
- about custom queue arguments, consult the [RabbitMQ documentation on AMQP Protocol Extensions](https://www.rabbitmq.com/extensions.html).
145
+ documentation on Queue Length Limit](https://www.rabbitmq.com/docs/maxlength). To find out more
146
+ about custom queue arguments, consult the [RabbitMQ documentation on AMQP Protocol Extensions](https://www.rabbitmq.com/docs/extensions).
126
147
 
127
148
  Consumers can write to Hutch's log by calling the logger method. The logger method returns
128
- a [Logger object](http://ruby-doc.org/stdlib-2.1.2/libdoc/logger/rdoc/Logger.html).
149
+ a [Logger object](https://docs.ruby-lang.org/en/3.0/Logger.html).
129
150
 
130
151
  ```ruby
131
152
  class FailedPaymentConsumer
@@ -153,7 +174,7 @@ client_logger = Logger.new("/path/to/bunny.log")
153
174
  Hutch::Config.set(:client_logger, client_logger)
154
175
  ```
155
176
 
156
- See this [RabbitMQ tutorial on topic exchanges](http://www.rabbitmq.com/tutorials/tutorial-five-ruby.html)
177
+ See this [RabbitMQ tutorial on topic exchanges](https://www.rabbitmq.com/tutorials/tutorial-five-ruby)
157
178
  to learn more.
158
179
 
159
180
  ### Message Processing Tracers
@@ -166,7 +187,13 @@ This will enable NewRelic custom instrumentation:
166
187
  Hutch::Config.set(:tracer, Hutch::Tracers::NewRelic)
167
188
  ```
168
189
 
169
- Batteries included!
190
+ And this will enable Datadog custom instrumentation:
191
+
192
+ ```ruby
193
+ Hutch::Config.set(:tracer, Hutch::Tracers::Datadog)
194
+ ```
195
+
196
+ Batteries included!
170
197
 
171
198
  ## Running Hutch
172
199
 
@@ -179,7 +206,7 @@ usage: hutch [options]
179
206
  --mq-host HOST Set the RabbitMQ host
180
207
  --mq-port PORT Set the RabbitMQ port
181
208
  -t, --[no-]mq-tls Use TLS for the AMQP connection
182
- --mq-tls-cert FILE Certificate for TLS client verification
209
+ --mq-tls-cert FILE Certificate for TLS client verification
183
210
  --mq-tls-key FILE Private key for TLS client verification
184
211
  --mq-exchange EXCHANGE Set the RabbitMQ exchange
185
212
  --mq-vhost VHOST Set the RabbitMQ vhost
@@ -215,17 +242,59 @@ in the config file, allowing for easy customization.
215
242
 
216
243
  Using Hutch with a Rails app is simple. Either start Hutch in the working
217
244
  directory of a Rails app, or pass the path to a Rails app in with the
218
- `--require` option. Consumers defined in Rails apps should be placed with in
245
+ `--require` option. Consumers defined in Rails apps should be placed within
219
246
  the `app/consumers/` directory, to allow them to be auto-loaded when Rails
220
247
  boots.
221
248
 
249
+ If you're using the Zeitwerk autoloader (the default since Rails 6)
250
+ and the consumers are not loaded in the development environment, you will need to
251
+ trigger eager loading in an initializer with
252
+
253
+ ```ruby
254
+ ::Zeitwerk::Loader.eager_load_all
255
+ ```
256
+
257
+ or with something more specific like
258
+
259
+ ```ruby
260
+ autoloader = Rails.autoloaders.main
261
+
262
+ Dir.glob(File.join('app/consumers', '*_consumer.rb')).each do |consumer|
263
+ autoloader.preload(consumer)
264
+ end
265
+ ```
266
+
267
+ ### Consumer Groups
268
+
269
+ It is possible to load only a subset of consumers. This is done by defining a consumer
270
+ group under the `consumer_groups` configuration key:
271
+
272
+ ``` yaml
273
+ consumer_groups:
274
+ payments:
275
+ - DepositConsumer
276
+ - CashoutConsumer
277
+ notification:
278
+ - EmailNotificationConsumer
279
+ ```
280
+
281
+ To only load a group of consumers, use the `--only-group` option:
282
+
283
+ ``` shell
284
+ hutch --only-group=payments --config=/path/to/hutch.yaml
285
+ ```
286
+
287
+ ### Loading Consumers Manually (One-by-One)
288
+
222
289
  To require files that define consumers manually, simply pass each file as an
223
290
  option to `--require`. Hutch will automatically detect whether you've provided
224
291
  a Rails app or a standard file, and take the appropriate behaviour:
225
292
 
226
293
  ```bash
227
- $ hutch --require path/to/rails-app # loads a rails app
228
- $ hutch --require path/to/file.rb # loads a ruby file
294
+ # loads a rails app
295
+ hutch --require path/to/rails-app
296
+ # loads a ruby file
297
+ hutch --require path/to/file.rb
229
298
  ```
230
299
 
231
300
  ### Stopping Hutch
@@ -240,8 +309,6 @@ kill -SIGTERM 456 # or kill -15 456
240
309
  kill -SIGQUIT 789 # or kill -3 789
241
310
  ```
242
311
 
243
- ![](http://g.recordit.co/wyCdzG9Kh3.gif)
244
-
245
312
  ## Producers
246
313
 
247
314
  Hutch includes a `publish` method for sending messages to Hutch consumers. When
@@ -265,7 +332,7 @@ Hutch::Config.set(:mq_exchange, 'name')
265
332
  ### Publisher Confirms
266
333
 
267
334
  For maximum message reliability when producing messages, you can force Hutch to use
268
- [Publisher Confirms](https://www.rabbitmq.com/confirms.html) and wait for a confirmation
335
+ [Publisher Confirms](https://www.rabbitmq.com/docs/confirms) and wait for a confirmation
269
336
  after every message published. This is the safest possible option for publishers
270
337
  but also results in a **significant throughput drop**.
271
338
 
@@ -285,25 +352,27 @@ send messages to Hutch.
285
352
  - Hutch works with topic exchanges, check the producer is also using topic
286
353
  exchanges.
287
354
  - Use message routing keys that match those used in your Hutch consumers.
288
- - Be sure your exchanges are marked as durable. In the Ruby AMQP gem, this is
289
- done by passing `durable: true` to the exchange creation method.
290
- - Publish messages as persistent.
291
- - Using publisher confirms is highly recommended.
355
+ - Be sure your exchanges are marked as durable, by passing `durable: true`
356
+ to the exchange creation method
357
+ - Publish messages as persistent
358
+ - Using publisher confirms is highly recommended
292
359
 
293
360
  Here's an example of a well-behaved publisher, minus publisher confirms:
294
361
 
295
362
  ```ruby
296
- AMQP.connect(host: config[:host]) do |connection|
297
- channel = AMQP::Channel.new(connection)
298
- exchange = channel.topic(config[:exchange], durable: true)
363
+ require 'bunny'
299
364
 
300
- message = JSON.dump({ subject: 'Test', id: 'abc' })
301
- exchange.publish(message, routing_key: 'test', persistent: true)
302
- end
303
- ```
365
+ conn = Bunny.new(host: config[:host])
366
+ conn.start
304
367
 
305
- If using publisher confirms with amqp gem, see [this issue](https://github.com/ruby-amqp/amqp/issues/92)
306
- and [this gist](https://gist.github.com/3042381) for more info.
368
+ ch = conn.create_channel
369
+ exchange = ch.topic(config[:exchange], durable: true)
370
+
371
+ message = JSON.dump({ subject: 'Test', id: 'abc' })
372
+ exchange.publish(message, routing_key: 'test', persistent: true)
373
+
374
+ conn.close
375
+ ```
307
376
 
308
377
  ## Configuration
309
378
 
@@ -316,7 +385,7 @@ Known configuration parameters are:
316
385
  * `mq_host`: RabbitMQ hostname (default: `localhost`)
317
386
  * `mq_port`: RabbitMQ port (default: `5672`)
318
387
  * `mq_vhost`: vhost to use (default: `/`)
319
- * `mq_username`: username to use (default: `guest`, only can connect from localhost as of RabbitMQ 3.3.0)
388
+ * `mq_username`: username to use (default: `guest`, can only connect from localhost)
320
389
  * `mq_password`: password to use (default: `guest`)
321
390
  * `mq_tls`: should TLS be used? (default: `false`)
322
391
  * `mq_tls_cert`: path to client TLS certificate (public key)
@@ -337,11 +406,15 @@ Known configuration parameters are:
337
406
  * `error_acknowledgements`: a chain of responsibility of objects that acknowledge/reject/requeue messages when an
338
407
  exception happens, see classes in `Hutch::Acknowledgements`.
339
408
  * `mq_exchange`: exchange to use for publishing (default: `hutch`)
340
- * `heartbeat`: [RabbitMQ heartbeat timeout](http://rabbitmq.com/heartbeats.html) (default: `30`)
409
+ * `mq_client_properties`: Bunny's [client properties](https://www.rabbitmq.com/docs/connections#capabilities) (default: `{}`)
410
+ * `heartbeat`: [RabbitMQ heartbeat timeout](https://www.rabbitmq.com/docs/heartbeats) (default: `30`)
341
411
  * `connection_timeout`: Bunny's socket open timeout (default: `11`)
342
412
  * `read_timeout`: Bunny's socket read timeout (default: `11`)
343
413
  * `write_timeout`: Bunny's socket write timeout (default: `11`)
414
+ * `automatically_recover`: Bunny's enable/disable network recovery (default: `true`)
415
+ * `network_recovery_interval`: Bunny's reconnect interval (default: `1`)
344
416
  * `tracer`: tracer to use to track message processing
417
+ * `namespace`: A namespace string to help group your queues (default: `nil`)
345
418
 
346
419
  ### Environment variables
347
420
 
@@ -400,6 +473,13 @@ Generate with
400
473
  <td><tt>HUTCH_MQ_EXCHANGE</tt></td>
401
474
  <td><p>RabbitMQ Exchange to use for publishing</p></td>
402
475
  </tr>
476
+ <tr>
477
+ <td><tt>mq_exchange_type</tt></td>
478
+ <td>topic</td>
479
+ <td>String</td>
480
+ <td><tt>HUTCH_MQ_EXCHANGE_TYPE</tt></td>
481
+ <td><p>RabbitMQ Exchange type to use for publishing</p></td>
482
+ </tr>
403
483
  <tr>
404
484
  <td><tt>mq_vhost</tt></td>
405
485
  <td>/</td>
@@ -454,7 +534,7 @@ Generate with
454
534
  <td>30</td>
455
535
  <td>Number</td>
456
536
  <td><tt>HUTCH_HEARTBEAT</tt></td>
457
- <td><p><a href="http://rabbitmq.com/heartbeats.html">RabbitMQ heartbeat timeout</a></p></td>
537
+ <td><p><a href="https://www.rabbitmq.com/docs/heartbeats">RabbitMQ heartbeat timeout</a></p></td>
458
538
  </tr>
459
539
  <tr>
460
540
  <td><tt>channel_prefetch</tt></td>
@@ -463,6 +543,13 @@ Generate with
463
543
  <td><tt>HUTCH_CHANNEL_PREFETCH</tt></td>
464
544
  <td><p>The <tt>basic.qos</tt> prefetch value to use.</p></td>
465
545
  </tr>
546
+ <tr>
547
+ <td><tt>connection_name</tt></td>
548
+ <td>nil</td>
549
+ <td>String</td>
550
+ <td><tt>HUTCH_CONNECTION_NAME</tt></td>
551
+ <td><p><a href="https://www.rabbitmq.com/docs/connections#client-provided-names">Client-Provided Connection Name</a></p></td>
552
+ </tr>
466
553
  <tr>
467
554
  <td><tt>connection_timeout</tt></td>
468
555
  <td>11</td>
@@ -484,12 +571,26 @@ Generate with
484
571
  <td><tt>HUTCH_WRITE_TIMEOUT</tt></td>
485
572
  <td><p>Bunny's socket write timeout</p></td>
486
573
  </tr>
574
+ <tr>
575
+ <td><tt>automatically_recover</tt></td>
576
+ <td>true</td>
577
+ <td>Boolean</td>
578
+ <td><tt>HUTCH_AUTOMATICALLY_RECOVER</tt></td>
579
+ <td><p>Bunny's enable/disable network recovery</p></td>
580
+ </tr>
581
+ <tr>
582
+ <td><tt>network_recovery_interval</tt></td>
583
+ <td>1</td>
584
+ <td>Number</td>
585
+ <td><tt>HUTCH_NETWORK_RECOVERY_INTERVAL</tt></td>
586
+ <td><p>Bunny's reconnect interval</p></td>
587
+ </tr>
487
588
  <tr>
488
589
  <td><tt>graceful_exit_timeout</tt></td>
489
590
  <td>11</td>
490
591
  <td>Number</td>
491
592
  <td><tt>HUTCH_GRACEFUL_EXIT_TIMEOUT</tt></td>
492
- <td><p>FIXME: DOCUMENT THIS</p></td>
593
+ <td><p>Timeout (in seconds) for consumer threads to finish before being killed during graceful shutdown</p></td>
493
594
  </tr>
494
595
  <tr>
495
596
  <td><tt>consumer_pool_size</tt></td>
@@ -545,7 +646,7 @@ Generate with
545
646
  <td>false</td>
546
647
  <td>Boolean</td>
547
648
  <td><tt>HUTCH_FORCE_PUBLISHER_CONFIRMS</tt></td>
548
- <td><p>Enables publisher confirms, forces Hutch::Broker#wait_for_confirms for</p></td>
649
+ <td><p>Enables publisher confirms, forces Hutch::Broker#wait_for_confirms for every publish</p></td>
549
650
  </tr>
550
651
  <tr>
551
652
  <td><tt>enable_http_api_use</tt></td>
@@ -568,10 +669,19 @@ Generate with
568
669
  <td><tt>HUTCH_CONSUMER_TAG_PREFIX</tt></td>
569
670
  <td><p>Prefix displayed on the consumers tags.</p></td>
570
671
  </tr>
672
+ <tr>
673
+ <td><tt>namespace</tt></td>
674
+ <td>nil</td>
675
+ <td>String</td>
676
+ <td><tt>HUTCH_NAMESPACE</tt></td>
677
+ <td><p>A namespace to help group your queues</p></td>
678
+ </tr>
679
+ <tr>
680
+ <td><tt>group</tt></td>
681
+ <td>''</td>
682
+ <td>String</td>
683
+ <td><tt>HUTCH_GROUP</tt></td>
684
+ <td><p>Consumer group to load (see Consumer Groups)</p></td>
685
+ </tr>
571
686
  </tbody>
572
687
  </table>
573
-
574
-
575
- ---
576
-
577
- GoCardless ♥ open source. If you do too, come [join us](https://gocardless.com/jobs/backend_developer).
@@ -0,0 +1,20 @@
1
+ #!/usr/bin/env sh
2
+
3
+ CTL=${BUNNY_RABBITMQCTL:-"sudo rabbitmqctl"}
4
+ PLUGINS=${BUNNY_RABBITMQ_PLUGINS:-"sudo rabbitmq-plugins"}
5
+
6
+ echo "Will use rabbitmqctl at ${CTL}"
7
+ echo "Will use rabbitmq-plugins at ${PLUGINS}"
8
+
9
+ $PLUGINS enable rabbitmq_management
10
+
11
+ sleep 3
12
+
13
+ # guest:guest has full access to /
14
+ $CTL add_vhost /
15
+ $CTL add_user guest guest
16
+ $CTL set_permissions -p / guest ".*" ".*" ".*"
17
+
18
+ # Reduce retention policy for faster publishing of stats
19
+ $CTL eval 'supervisor2:terminate_child(rabbit_mgmt_sup_sup, rabbit_mgmt_sup), application:set_env(rabbitmq_management, sample_retention_policies, [{global, [{605, 1}]}, {basic, [{605, 1}]}, {detailed, [{10, 1}]}]), rabbit_mgmt_sup_sup:start_child().' || true
20
+ $CTL eval 'supervisor2:terminate_child(rabbit_mgmt_agent_sup_sup, rabbit_mgmt_agent_sup), application:set_env(rabbitmq_management_agent, sample_retention_policies, [{global, [{605, 1}]}, {basic, [{605, 1}]}, {detailed, [{10, 1}]}]), rabbit_mgmt_agent_sup_sup:start_child().' || true
@@ -0,0 +1,15 @@
1
+ #!/bin/sh
2
+
3
+ CTL=${MARCH_HARE_RABBITMQCTL:="docker exec rabbitmq rabbitmqctl"}
4
+
5
+ # guest:guest has full access to /
6
+
7
+ $CTL add_vhost /
8
+ # $CTL add_user guest guest # already exists
9
+ $CTL set_permissions -p / guest ".*" ".*" ".*"
10
+
11
+ # Reduce retention policy for faster publishing of stats
12
+ $CTL eval 'supervisor2:terminate_child(rabbit_mgmt_sup_sup, rabbit_mgmt_sup), application:set_env(rabbitmq_management, sample_retention_policies, [{global, [{605, 1}]}, {basic, [{605, 1}]}, {detailed, [{10, 1}]}]), rabbit_mgmt_sup_sup:start_child().' || true
13
+ $CTL eval 'supervisor2:terminate_child(rabbit_mgmt_agent_sup_sup, rabbit_mgmt_agent_sup), application:set_env(rabbitmq_management_agent, sample_retention_policies, [{global, [{605, 1}]}, {basic, [{605, 1}]}, {detailed, [{10, 1}]}]), rabbit_mgmt_agent_sup_sup:start_child().' || true
14
+
15
+ sleep 3
@@ -0,0 +1,46 @@
1
+ #!/bin/sh
2
+
3
+ sudo apt-get install curl gnupg debian-keyring debian-archive-keyring apt-transport-https -y
4
+
5
+ ## Team RabbitMQ's main signing key
6
+ sudo apt-key adv --keyserver "hkps://keys.openpgp.org" --recv-keys "0x0A9AF2115F4687BD29803A206B73A36E6026DFCA"
7
+ ## Launchpad PPA that provides modern Erlang releases
8
+ sudo apt-key adv --keyserver "keyserver.ubuntu.com" --recv-keys "F77F1EDA57EBB1CC"
9
+ ## PackageCloud RabbitMQ repository
10
+ sudo apt-key adv --keyserver "keyserver.ubuntu.com" --recv-keys "F6609E60DC62814E"
11
+
12
+ ## Add apt repositories maintained by Team RabbitMQ
13
+ sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
14
+ ## Provides modern Erlang/OTP releases
15
+ ##
16
+ ## "bionic" as distribution name should work for any reasonably recent Ubuntu or Debian release.
17
+ ## See the release to distribution mapping table in RabbitMQ doc guides to learn more.
18
+ deb http://ppa.launchpad.net/rabbitmq/rabbitmq-erlang/ubuntu bionic main
19
+ deb-src http://ppa.launchpad.net/rabbitmq/rabbitmq-erlang/ubuntu bionic main
20
+
21
+ ## Provides RabbitMQ
22
+ ##
23
+ ## "bionic" as distribution name should work for any reasonably recent Ubuntu or Debian release.
24
+ ## See the release to distribution mapping table in RabbitMQ doc guides to learn more.
25
+ deb https://packagecloud.io/rabbitmq/rabbitmq-server/ubuntu/ bionic main
26
+ deb-src https://packagecloud.io/rabbitmq/rabbitmq-server/ubuntu/ bionic main
27
+ EOF
28
+
29
+ ## Update package indices
30
+ sudo apt-get update -y
31
+
32
+ ## Install Erlang packages
33
+ sudo apt-get install -y erlang-base \
34
+ erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
35
+ erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
36
+ erlang-runtime-tools erlang-snmp erlang-ssl \
37
+ erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl
38
+
39
+ ## Install rabbitmq-server and its dependencies
40
+ sudo apt-get install rabbitmq-server -y --fix-missing
41
+
42
+ sudo service rabbitmq-server start
43
+
44
+ sudo rabbitmqctl await_startup --timeout 120
45
+
46
+ until sudo lsof -i:5672; do echo "Waiting for RabbitMQ to start..."; sleep 1; done
@@ -10,7 +10,7 @@ module Hutch
10
10
  prefix = "message(#{properties.message_id || '-'}): "
11
11
  logger.debug "#{prefix} nacking message"
12
12
 
13
- broker.nack(delivery_info.delivery_tag)
13
+ broker.nack(delivery_info.delivery_tag, channel: delivery_info.channel)
14
14
 
15
15
  true
16
16
  end
@@ -1,17 +1,21 @@
1
1
  require 'bunny'
2
2
  require 'forwardable'
3
3
 
4
+ require 'hutch/logging'
5
+
4
6
  module Hutch
5
7
  module Adapters
6
8
  class BunnyAdapter
7
9
  extend Forwardable
10
+ include Logging
8
11
 
9
12
  DEFAULT_VHOST = Bunny::Session::DEFAULT_VHOST
10
13
 
11
14
  ConnectionRefused = Bunny::TCPConnectionFailed
15
+ ChannelAlreadyClosed = Bunny::ChannelAlreadyClosed
12
16
  PreconditionFailed = Bunny::PreconditionFailed
13
17
 
14
- def_delegators :@connection, :start, :disconnect, :close, :create_channel, :open?
18
+ def_delegators :@connection, :start, :disconnect, :close, :create_channel, :open?, :recover_channel_topology
15
19
 
16
20
  def initialize(opts={})
17
21
  @connection = Bunny.new(opts)
@@ -25,9 +29,33 @@ module Hutch
25
29
  ch.prefetch(prefetch) if prefetch
26
30
  end
27
31
 
32
+ def queue_exists?(name)
33
+ @connection.queue_exists?(name)
34
+ end
35
+
36
+ def install_channel_recovery(ch)
37
+ # A consumer timeout invalidates the delivery tag, so a handler that
38
+ # was still running acknowledges an unknown tag and closes the channel.
39
+ ch.on_error do |channel, close|
40
+ next unless close.delivery_ack_timeout? || close.unknown_delivery_tag?
41
+
42
+ begin
43
+ channel.reopen
44
+ recover_channel_topology(channel)
45
+ logger.warn "recovered consumer channel closed with '#{close.reply_text}'"
46
+ rescue => ex
47
+ logger.error "channel recovery failed: #{ex.class}: #{ex.message}"
48
+ end
49
+ end
50
+ end
51
+
28
52
  def current_timestamp
29
53
  Time.now.to_i
30
54
  end
55
+
56
+ def self.new_exchange(ch, exchange_type, exchange_name, exchange_options)
57
+ Bunny::Exchange.new(ch, exchange_type, exchange_name, exchange_options)
58
+ end
31
59
  end
32
60
  end
33
61
  end
@@ -9,6 +9,7 @@ module Hutch
9
9
  DEFAULT_VHOST = "/"
10
10
 
11
11
  ConnectionRefused = MarchHare::ConnectionRefused
12
+ ChannelAlreadyClosed = MarchHare::ChannelAlreadyClosed
12
13
  PreconditionFailed = MarchHare::PreconditionFailed
13
14
 
14
15
  def_delegators :@connection, :start, :disconnect, :close, :open?
@@ -25,6 +26,25 @@ module Hutch
25
26
  ch.prefetch = prefetch if prefetch
26
27
  end
27
28
 
29
+ # `MarchHare::Session` has no `queue_exists?` counterpart, and a failed
30
+ # passive declare closes the channel, hence the throwaway one.
31
+ def queue_exists?(name)
32
+ ch = @connection.create_channel
33
+ begin
34
+ ch.queue(name, passive: true)
35
+ true
36
+ rescue MarchHare::NotFound
37
+ false
38
+ ensure
39
+ ch.close rescue nil
40
+ end
41
+ end
42
+
43
+ # MarchHare::Channel has no on_error callback, and neither Channel#reopen
44
+ # nor Session#recover_channel_topology has a `march_hare` counterpart.
45
+ def install_channel_recovery(ch)
46
+ end
47
+
28
48
  def create_channel(n = nil, consumer_pool_size = 1, consumer_pool_abort_on_exception = false)
29
49
  @connection.create_channel(n)
30
50
  end
@@ -32,6 +52,10 @@ module Hutch
32
52
  def current_timestamp
33
53
  Time.now
34
54
  end
55
+
56
+ def self.new_exchange(ch, exchange_type, exchange_name, exchange_options)
57
+ MarchHare::Exchange.new(ch, exchange_name, exchange_options.merge(type: exchange_type)).tap(&:declare!)
58
+ end
35
59
  end
36
60
  end
37
61
  end