hutch 0.19.0 → 0.20.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: c5b96423188a88e607bf694005618e1c13bc6ce1
4
- data.tar.gz: 97955f0060eea500cbd6cc49c3c393adccfaf8a6
3
+ metadata.gz: 9d93317d6750bca3658656b8b919dbdf9b277cf5
4
+ data.tar.gz: ce7f0b6bad0ea3764e13e634f764285f75927bdf
5
5
  SHA512:
6
- metadata.gz: d161d2cf60136e26356a18f7a6f552fd11361bce6258132ce9dc807d219290f1fa67fda7f801fb58cacd542c224a93b4941c473aaa4c5ee3b43446632ab03c96
7
- data.tar.gz: 7d2c9d3f0524a9583e76c853823a1527a9d516d7a2cf368aa061412442930e09d5a2b7855c915d4ab3e1ca8c39b2a82f0a864a6e51d558a420e1a9efd3d96cc1
6
+ metadata.gz: 6f39fa63c8d77ea71444ab25cfd599bfc89ad9e8c222323f69b322e66a179347fec7c1afb100b5575af08d1a6522af1d2e4adeef827c40f825313e76640482d9
7
+ data.tar.gz: d3c94ca6c2720fa3fd4197f408dc8832f9f904941dbd1cc556e13efae7b0768b2ab1be0c58159676cce4b78c0c5ccac9e15f8a55c780d6d251b0b42f54a55161
@@ -1,4 +1,68 @@
1
- ## 0.19.0 — (unreleased)
1
+ ## 0.20.0 — (unreleased)
2
+
3
+ ### Hutch::Exception includes Bunny::Exception
4
+
5
+ `Hutch::Exception` now inherits from `Bunny::Exception` which
6
+ inherits from `StandardError`.
7
+
8
+ GH issue: [#137](https://github.com/gocardless/hutch/issues/137).
9
+
10
+
11
+ ### Pluggable (Negative) Acknowledge Handlers
12
+
13
+ Hutch now can be configured to use a user-provided
14
+ object(s) to perform acknowledgement on consumer exceptions.
15
+
16
+ For example, this is what the default handler looks like:
17
+
18
+ ``` ruby
19
+ require 'hutch/logging'
20
+ require 'hutch/acknowledgements/base'
21
+
22
+ module Hutch
23
+ module Acknowledgements
24
+ class NackOnAllFailures < Base
25
+ include Logging
26
+
27
+ def handle(delivery_info, properties, broker, ex)
28
+ prefix = "message(#{properties.message_id || '-'}): "
29
+ logger.debug "#{prefix} nacking message"
30
+
31
+ broker.nack(delivery_info.delivery_tag)
32
+
33
+ # terminates further chain processing
34
+ true
35
+ end
36
+ end
37
+ end
38
+ end
39
+ ```
40
+
41
+ Handlers are configured similarly to exception notification handlers,
42
+ via `:error_acknowledgements` in Hutch config.
43
+
44
+ Contributed by Derek Kastner.
45
+
46
+ GH issue: [#177](https://github.com/gocardless/hutch/pull/177).
47
+
48
+
49
+ ### Configurable Exchange Properties
50
+
51
+ `:mq_exchange_options` is a new config option that can be used
52
+ to provide a hash of exchange attributes (durable, auto-delete).
53
+ The options will be passed directly to Bunny.
54
+
55
+ Contributed by Derek Kastner.
56
+
57
+ GH issue: [#170](https://github.com/gocardless/hutch/pull/170).
58
+
59
+
60
+ ### Bunny Update
61
+
62
+ Bunny is updated to 2.2.1.
63
+
64
+
65
+ ## 0.19.0 — September 7th, 2015
2
66
 
3
67
  ### Pluggable Serialisers
4
68
 
data/README.md CHANGED
@@ -20,6 +20,11 @@ Hutch is a moderately mature project (started in early 2013)
20
20
  that was extracted from production systems.
21
21
 
22
22
 
23
+ ## Supported Ruby Versions
24
+
25
+ Hutch requires CRuby 2.0+ or JRuby 9K.
26
+
27
+
23
28
  ## Overview
24
29
 
25
30
  Hutch is a conventions-based framework for writing services that communicate
@@ -31,7 +36,8 @@ With Hutch, consumers are stored in separate files and include the `Hutch::Consu
31
36
  They are then loaded by a command line runner which connects to RabbitMQ, sets up queues and bindings,
32
37
  and so on. Publishers connect to RabbitMQ via `Hutch.connect` and publish using `Hutch.publish`.
33
38
 
34
- Hutch uses [Bunny](http://rubybunny.info) under the hood.
39
+ Hutch uses [Bunny](http://rubybunny.info) or [March Hare](http://rubymarchhare.info)
40
+ (on JRuby) under the hood.
35
41
 
36
42
 
37
43
  ## Defining Consumers
@@ -300,6 +306,10 @@ Known configuration parameters are:
300
306
  tracked (e.g. using `Hutch::Broker#confirm_select` callback or `Hutch::Broker#wait_for_confirms`)
301
307
  * `force_publisher_confirms`: enables publisher confirms, forces `Hutch::Broker#wait_for_confirms` for every publish. **This is the safest option which also offers the lowest throughput**.
302
308
  * `log_level`: log level used by the standard Ruby logger (default: `Logger::INFO`)
309
+ * `error_handlers`: a list of error handler objects, see classes in `Hutch::ErrorHandlers`. All configured
310
+ handlers will be invoked unconditionally in the order listed.
311
+ * `error_acknowledgements`: a chain of responsibility of objects that acknowledge/reject/requeue messages when an
312
+ exception happens, see classes in `Hutch::Acknowledgements`.
303
313
  * `mq_exchange`: exchange to use for publishing (default: `hutch`)
304
314
  * `heartbeat`: [RabbitMQ heartbeat timeout](http://rabbitmq.com/heartbeats.html) (default: `30`)
305
315
  * `connection_timeout`: Bunny's socket open timeout (default: `11`)
@@ -6,7 +6,7 @@ Gem::Specification.new do |gem|
6
6
  gem.add_runtime_dependency 'march_hare', '>= 2.11.0'
7
7
  else
8
8
  gem.platform = Gem::Platform::RUBY
9
- gem.add_runtime_dependency 'bunny', '>= 2.2.0'
9
+ gem.add_runtime_dependency 'bunny', '>= 2.2.1'
10
10
  end
11
11
  gem.add_runtime_dependency 'carrot-top', '~> 0.0.7'
12
12
  gem.add_runtime_dependency 'multi_json', '~> 1.11.2'
@@ -0,0 +1,16 @@
1
+ module Hutch
2
+ module Acknowledgements
3
+ # Defines acknowledgement handler interface.
4
+ class Base
5
+ # Implements negative acknowledgement/requeueing logic
6
+ # and returns a boolean to indicate whether acknowledgement
7
+ # was performed. If false is returned, next handler in the
8
+ # chain will be invoked.
9
+ #
10
+ # The chain always falls back to unconditional nacking.
11
+ def handle(delivery_info, properties, broker, ex)
12
+ raise NotImplementedError.new
13
+ end
14
+ end
15
+ end
16
+ end
@@ -0,0 +1,19 @@
1
+ require 'hutch/logging'
2
+ require 'hutch/acknowledgements/base'
3
+
4
+ module Hutch
5
+ module Acknowledgements
6
+ class NackOnAllFailures < Base
7
+ include Logging
8
+
9
+ def handle(delivery_info, properties, broker, ex)
10
+ prefix = "message(#{properties.message_id || '-'}): "
11
+ logger.debug "#{prefix} nacking message"
12
+
13
+ broker.nack(delivery_info.delivery_tag)
14
+
15
+ true
16
+ end
17
+ end
18
+ end
19
+ end
@@ -52,10 +52,11 @@ module Hutch
52
52
  open_channel!
53
53
 
54
54
  exchange_name = @config[:mq_exchange]
55
+ exchange_options = { durable: true }.merge @config[:mq_exchange_options]
55
56
  logger.info "using topic exchange '#{exchange_name}'"
56
57
 
57
58
  with_bunny_precondition_handler('exchange') do
58
- @exchange = @channel.topic(exchange_name, durable: true)
59
+ @exchange = @channel.topic(exchange_name, exchange_options)
59
60
  end
60
61
  end
61
62
 
@@ -8,11 +8,12 @@ module Hutch
8
8
  module Config
9
9
  require 'yaml'
10
10
 
11
- def self.initialize(params={})
11
+ def self.initialize(params = {})
12
12
  @config = {
13
13
  mq_host: 'localhost',
14
14
  mq_port: 5672,
15
15
  mq_exchange: 'hutch', # TODO: should this be required?
16
+ mq_exchange_options: {},
16
17
  mq_vhost: '/',
17
18
  mq_tls: false,
18
19
  mq_tls_cert: nil,
@@ -32,6 +33,9 @@ module Hutch
32
33
  require_paths: [],
33
34
  autoload_rails: true,
34
35
  error_handlers: [Hutch::ErrorHandlers::Logger.new],
36
+ # note that this is not a list, it is a chain of responsibility
37
+ # that will fall back to "nack unconditionally"
38
+ error_acknowledgements: [],
35
39
  tracer: Hutch::Tracers::NullTracer,
36
40
  namespace: nil,
37
41
  daemonise: false,
@@ -1,5 +1,8 @@
1
+ require "bunny/exceptions"
2
+
1
3
  module Hutch
2
- class Exception < StandardError; end
4
+ # Bunny::Exception inherits from StandardError
5
+ class Exception < Bunny::Exception; end
3
6
  class ConnectionError < Exception; end
4
7
  class AuthenticationError < Exception; end
5
8
  class WorkerSetupError < Exception; end
@@ -1,4 +1,4 @@
1
1
  module Hutch
2
- VERSION = '0.19.0'.freeze
2
+ VERSION = '0.20.0'.freeze
3
3
  end
4
4
 
@@ -1,6 +1,7 @@
1
1
  require 'hutch/message'
2
2
  require 'hutch/logging'
3
3
  require 'hutch/broker'
4
+ require 'hutch/acknowledgements/nack_on_all_failures'
4
5
  require 'carrot-top'
5
6
 
6
7
  module Hutch
@@ -118,7 +119,7 @@ module Hutch
118
119
  with_tracing(consumer_instance).handle(message)
119
120
  broker.ack(delivery_info.delivery_tag)
120
121
  rescue StandardError => ex
121
- broker.nack(delivery_info.delivery_tag)
122
+ acknowledge_error(delivery_info, properties, broker, ex)
122
123
  handle_error(properties.message_id, payload, consumer, ex)
123
124
  end
124
125
  end
@@ -133,11 +134,23 @@ module Hutch
133
134
  end
134
135
  end
135
136
 
137
+ def acknowledge_error(delivery_info, properties, broker, ex)
138
+ acks = error_acknowledgements +
139
+ [Hutch::Acknowledgements::NackOnAllFailures.new]
140
+ acks.find do |backend|
141
+ backend.handle(delivery_info, properties, broker, ex)
142
+ end
143
+ end
144
+
136
145
  def consumers=(val)
137
146
  if val.empty?
138
147
  logger.warn "no consumer loaded, ensure there's no configuration issue"
139
148
  end
140
149
  @consumers = val
141
150
  end
151
+
152
+ def error_acknowledgements
153
+ Hutch::Config[:error_acknowledgements]
154
+ end
142
155
  end
143
156
  end
@@ -63,6 +63,25 @@ describe Hutch::Worker do
63
63
  worker.handle_message(consumer, delivery_info, properties, payload)
64
64
  end
65
65
 
66
+ context 'when the consumer fails and a requeue is configured' do
67
+
68
+ it 'requeues the message' do
69
+ allow(consumer_instance).to receive(:process).and_raise('failed')
70
+ requeuer = double
71
+ allow(requeuer).to receive(:handle).ordered { |delivery_info, properties, broker, e|
72
+ broker.requeue delivery_info.delivery_tag
73
+ true
74
+ }
75
+ allow(worker).to receive(:error_acknowledgements).and_return([requeuer])
76
+ expect(broker).to_not receive(:ack)
77
+ expect(broker).to_not receive(:nack)
78
+ expect(broker).to receive(:requeue)
79
+
80
+ worker.handle_message(consumer, delivery_info, properties, payload)
81
+ end
82
+ end
83
+
84
+
66
85
  context 'when the consumer raises an exception' do
67
86
  before { allow(consumer_instance).to receive(:process).and_raise('a consumer error') }
68
87
 
@@ -95,5 +114,40 @@ describe Hutch::Worker do
95
114
  end
96
115
  end
97
116
  end
117
+
118
+
119
+ describe '#acknowledge_error' do
120
+ let(:delivery_info) { double('Delivery Info', routing_key: '',
121
+ delivery_tag: 'dt') }
122
+ let(:properties) { double('Properties', message_id: 'abc123') }
123
+
124
+ subject { worker.acknowledge_error delivery_info, properties, broker, StandardError.new }
125
+
126
+ it 'stops when it runs a successful acknowledgement' do
127
+ skip_ack = double handle: false
128
+ always_ack = double handle: true
129
+ never_used = double handle: true
130
+
131
+ allow(worker).
132
+ to receive(:error_acknowledgements).
133
+ and_return([skip_ack, always_ack, never_used])
134
+
135
+ expect(never_used).to_not receive(:handle)
136
+
137
+ subject
138
+ end
139
+
140
+ it 'defaults to nacking' do
141
+ skip_ack = double handle: false
142
+
143
+ allow(worker).
144
+ to receive(:error_acknowledgements).
145
+ and_return([skip_ack, skip_ack])
146
+
147
+ expect(broker).to receive(:nack)
148
+
149
+ subject
150
+ end
151
+ end
98
152
  end
99
153
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: hutch
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.19.0
4
+ version: 0.20.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Harry Marr
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2015-09-06 00:00:00.000000000 Z
11
+ date: 2015-11-15 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bunny
@@ -16,14 +16,14 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 2.2.0
19
+ version: 2.2.1
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
24
  - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 2.2.0
26
+ version: 2.2.1
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: carrot-top
29
29
  requirement: !ruby/object:Gem::Requirement
@@ -117,6 +117,8 @@ files:
117
117
  - examples/producer.rb
118
118
  - hutch.gemspec
119
119
  - lib/hutch.rb
120
+ - lib/hutch/acknowledgements/base.rb
121
+ - lib/hutch/acknowledgements/nack_on_all_failures.rb
120
122
  - lib/hutch/adapter.rb
121
123
  - lib/hutch/adapters/bunny.rb
122
124
  - lib/hutch/adapters/march_hare.rb