dkastner-hutch 0.17.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (46) hide show
  1. checksums.yaml +7 -0
  2. data/.gitignore +7 -0
  3. data/.travis.yml +7 -0
  4. data/CHANGELOG.md +397 -0
  5. data/Gemfile +22 -0
  6. data/Guardfile +5 -0
  7. data/LICENSE +22 -0
  8. data/README.md +315 -0
  9. data/Rakefile +14 -0
  10. data/bin/hutch +8 -0
  11. data/circle.yml +3 -0
  12. data/examples/consumer.rb +13 -0
  13. data/examples/producer.rb +10 -0
  14. data/hutch.gemspec +24 -0
  15. data/lib/hutch/broker.rb +356 -0
  16. data/lib/hutch/cli.rb +205 -0
  17. data/lib/hutch/config.rb +121 -0
  18. data/lib/hutch/consumer.rb +66 -0
  19. data/lib/hutch/error_handlers/airbrake.rb +26 -0
  20. data/lib/hutch/error_handlers/honeybadger.rb +28 -0
  21. data/lib/hutch/error_handlers/logger.rb +16 -0
  22. data/lib/hutch/error_handlers/sentry.rb +23 -0
  23. data/lib/hutch/error_handlers.rb +8 -0
  24. data/lib/hutch/exceptions.rb +7 -0
  25. data/lib/hutch/logging.rb +32 -0
  26. data/lib/hutch/message.rb +33 -0
  27. data/lib/hutch/tracers/newrelic.rb +19 -0
  28. data/lib/hutch/tracers/null_tracer.rb +15 -0
  29. data/lib/hutch/tracers.rb +6 -0
  30. data/lib/hutch/version.rb +4 -0
  31. data/lib/hutch/worker.rb +110 -0
  32. data/lib/hutch.rb +60 -0
  33. data/spec/hutch/broker_spec.rb +325 -0
  34. data/spec/hutch/cli_spec.rb +80 -0
  35. data/spec/hutch/config_spec.rb +126 -0
  36. data/spec/hutch/consumer_spec.rb +130 -0
  37. data/spec/hutch/error_handlers/airbrake_spec.rb +34 -0
  38. data/spec/hutch/error_handlers/honeybadger_spec.rb +36 -0
  39. data/spec/hutch/error_handlers/logger_spec.rb +15 -0
  40. data/spec/hutch/error_handlers/sentry_spec.rb +20 -0
  41. data/spec/hutch/logger_spec.rb +28 -0
  42. data/spec/hutch/message_spec.rb +38 -0
  43. data/spec/hutch/worker_spec.rb +98 -0
  44. data/spec/hutch_spec.rb +87 -0
  45. data/spec/spec_helper.rb +35 -0
  46. metadata +187 -0
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 19db67e67c4be705a1941803cb05ac03357e32cf
4
+ data.tar.gz: 7e85becb93bb0c994b16f533a3cc7b7a34c4d0d6
5
+ SHA512:
6
+ metadata.gz: 6ccfebf8a648f1d33e7fc49bc7a03a0b7cac70a93cf3321ec1f1a5cb58018fd671f61ba76eef534aad60143b66eaf7edd1c2f726b3b217d5e2d5be680daca589
7
+ data.tar.gz: eca9fba818c940b01ca8969c8ad9794bc31935a22c980f6cedea833507aeeb8bf431b18e1079737cbfbe1f3ddf687b0b28aa0d54ef75095850d707ce52c92431
data/.gitignore ADDED
@@ -0,0 +1,7 @@
1
+ .bundle
2
+ hutch-*.gem
3
+ Gemfile.lock
4
+ .ruby-version
5
+ .ruby-gemset
6
+ .idea
7
+ coverage
data/.travis.yml ADDED
@@ -0,0 +1,7 @@
1
+ language: ruby
2
+ rvm:
3
+ - "2.2"
4
+ - "2.1"
5
+ - "2.0"
6
+ services:
7
+ - rabbitmq
data/CHANGELOG.md ADDED
@@ -0,0 +1,397 @@
1
+ ## 0.18.0 — (unreleased)
2
+
3
+ ### Bunny Logger Option
4
+
5
+ `:client_logger` is a new option that allows
6
+ for configuring loggerd used by Bunny, the underlying
7
+ RabbitMQ client library.
8
+
9
+ Contributed by Nate Salisbury.
10
+
11
+
12
+ ## 0.17.0 — July 19th, 2015
13
+
14
+ Fixes an issue with `NoMethodError` in `Hutch::Config`.
15
+
16
+
17
+ ## 0.16.0 — July 19th, 2015
18
+
19
+ ### Support amqps URIs
20
+
21
+ Hutch now automatically enables TLS and changes default port
22
+ when URI scheme is `amqps`.
23
+
24
+ Contributed by Carl Hörberg.
25
+
26
+ ### Hash With Indifferent Access
27
+
28
+ Hutch now uses `HashWithIndifferentAccess` internally
29
+ to reduce use of symbols (which are not garbage collected
30
+ by widely used Ruby versions).
31
+
32
+ Contributed by Teodor Pripoae.
33
+
34
+
35
+ ## 0.15.0 — May 5th, 2015
36
+
37
+ ### Airbrake Error Handler
38
+
39
+ Contributed by Nate Salisbury.
40
+
41
+ ### Ruby 1.9 Support Dropped
42
+
43
+ Ruby 1.9 is no longer supported by Hutch (and soon Bunny 2.0).
44
+ 1.9 is also no longer maintained by the Ruby core team.
45
+
46
+ ### Custom Arguments per Consumers
47
+
48
+ Allow to set custom arguments per consumers by using the `arguments` setter.
49
+ Arguments are usually used by rabbitmq plugins or to set queue policies. You can
50
+ find a list of supported arguments [here](https://www.rabbitmq.com/extensions.html).
51
+
52
+ Contributed by Pierre-Louis Gottfrois.
53
+
54
+ ### Message Processing Tracers
55
+
56
+ Allow to track message processing by using the `:tracer` config option,
57
+ the value should be a class (or fully-qualified string name of a class) that
58
+ implements the tracing interface.
59
+
60
+ A tracer that performs NewRelic instrumentation ships with Hutch
61
+ and requires New Relic gem to be loaded.
62
+
63
+ Contributed by Mirosław Nagaś.
64
+
65
+ ### Added Logger Method to Consumer Module
66
+
67
+ Consumers can now call a logger method to write to Hutch's log.
68
+
69
+ Contributed by Matty Courtney
70
+
71
+ ## 0.14.0 — Feb 23rd, 2015
72
+
73
+ ### Configurable Socket Timeouts
74
+
75
+ Socket read and write timeouts are now configurable using
76
+ the `read_timeout` and `write_timeout` options, respectively.
77
+
78
+ Contributed by Chris Barton.
79
+
80
+
81
+ ### Logged Messages as Serialised as JSON
82
+
83
+ ...as opposed to Ruby object printing.
84
+
85
+ Contributed by Andrew Morton.
86
+
87
+
88
+ ### Configurable Heartbeat
89
+
90
+ Config now supports a new option: `:heartbeat`, which is passed
91
+ on to Bunny.
92
+
93
+ Contributed by Simon Taranto.
94
+
95
+
96
+ ### HoneyBadger Error Handler
97
+
98
+ Contributed by Daniel Farrell.
99
+
100
+
101
+ ### Hutch.connected? Now Returns Up-to-Date Value
102
+
103
+ `Hutch.connected?` no longer relies on an ivar and always returns
104
+ an up-to-date value.
105
+
106
+ Contributed by Pierre-Louis Gottfrois.
107
+
108
+
109
+ ## 0.13.0 — Dec 5th, 2014
110
+
111
+ ### HTTP API Can Be Disabled for Consumers
112
+
113
+ HTTP API use can be disabled for consumers using the `:enable_http_api_use` config
114
+ option (defaults to true).
115
+
116
+
117
+
118
+ ## 0.12.0 — Nov 25th, 2014
119
+
120
+ ### Explicit Requires
121
+
122
+ Hutch no longer relies on `Kernel#autoload` to load its key
123
+ modules and classes.
124
+
125
+ Contributed by Pierre-Louis Gottfrois.
126
+
127
+
128
+ ### hutch --version No Longer Fails
129
+
130
+ ```
131
+ hutch --version
132
+ ```
133
+
134
+ no longer fails with an exception.
135
+
136
+ Contributed by Olle Jonsson.
137
+
138
+
139
+ ### Base Class for All Hutch Exceptions
140
+
141
+ All Hutch exceptions now inherit from `Hutch::Exception`.
142
+
143
+ Contributed by Pierre-Louis Gottfrois.
144
+
145
+
146
+ ## 0.11.0 — Nov 14th, 2014
147
+
148
+ ### Publisher Confirms Support
149
+
150
+ `:force_publisher_confirms` is a new configuration option that forces `Hutch.publish` to wait
151
+ for a confirm for every message published. Note that this **will cause a significant drop in throughput**:
152
+
153
+ ``` ruby
154
+ Hutch::Config.set(:force_publisher_confirms, true)
155
+ ```
156
+
157
+ `Hutch::Broker#confirm_select` and `Hutch::Broker#wait_for_confirms` are new public API methods
158
+ that delegate to their respective `Bunny::Channel` counterparts. `Hutch::Broker#confirm_select`
159
+ can be used to handle confirms with a callback instead of waiting:
160
+
161
+ ``` ruby
162
+ broker.confirm_select do |delivery_tag, multiple, nack|
163
+ # ...
164
+ end
165
+ ```
166
+
167
+
168
+ ### Bunny Update
169
+
170
+ Bunny is updated to [1.6.0](http://blog.rubyrabbitmq.info/blog/2014/10/31/bunny-1-dot-6-0-is-released/).
171
+
172
+
173
+ ## 0.10.0 — Oct 22, 2014
174
+
175
+ ### Configuration via URI
176
+
177
+ Hutch now supports a new configuration key, `:uri`, which allows
178
+ connection configuration via a URI.
179
+
180
+ Note that since the URI has to include credentials, this option
181
+ is not available on the command line.
182
+
183
+ ### Bunny Update
184
+
185
+ Bunny is updated to `1.5.1`, which mitigates the POODLE attack
186
+ by disabling SSL 3.0 where possible.
187
+
188
+ ### Payload in Error Handlers
189
+
190
+ Error handlers will now have access to message payload.
191
+
192
+ Contributed by Daniel Farrell.
193
+
194
+ ### Exceptions in Error Handlers Don't Prevent Nacks
195
+
196
+ Exceptions in error handlers no longer prevent messages from being
197
+ `basic.nack`-ed.
198
+
199
+ ### Pid File Support
200
+
201
+ `:pidfile` is a new configuration option that stores Hutch process
202
+ PID in a file at provided path.
203
+
204
+ Contributed by Rustam Sharshenov.
205
+
206
+ ### More Info on Message
207
+
208
+ Bunny's `delivery_info`, `properties` and payload are now accessible on `Hutch::Message`.
209
+
210
+ Contributed by gregory.
211
+
212
+
213
+ ### Optional Config Parameters
214
+
215
+ `Hutch::Config` constructor now accepts an extra hash of optional
216
+ configuration parameters.
217
+
218
+ Contributed by Ignazio Mostallino.
219
+
220
+
221
+ ## 0.9.0 — May 13, 2014
222
+
223
+ ### Platform-aware Signal Registration
224
+
225
+ Hutch will no longer attempt to register signal traps
226
+ for signals not supported by the environment (e.g. on by certain OSes).
227
+
228
+ Contributed by Tobias Matthies.
229
+
230
+ ### TLS Fixes
231
+
232
+ Hutch now properly passes client TLS key and certificate to
233
+ Bunny.
234
+
235
+ Contributed by Eric Nelson.
236
+
237
+ ### Bunny Update
238
+
239
+ Bunny is updated to 1.2.x which should offer
240
+ [much better latency](https://github.com/ruby-amqp/bunny/pull/187) for
241
+ workloads with lots of small messages published frequently.
242
+
243
+ ### More Unit Testing Friendly CLI/Runner
244
+
245
+ `Hutch::CLI#run` now accepts a parameter and is easier to use
246
+ in automated tests.
247
+
248
+
249
+ ## 0.8.0 — February 13, 2014
250
+
251
+ ### Uncaught Exceptions Result in Rejected Messages
252
+
253
+ Uncaught exceptions in consumers now result in Hutch rejecting
254
+ messages (deliveries) using `basic.nack`. This way they are [dead lettered](http://www.rabbitmq.com/dlx.html).
255
+
256
+ Contributed by Garrett Johnson.
257
+
258
+ ### Missing Require
259
+
260
+ `hutch/consumer.rb` no longer fails to load with the
261
+ apps that do not `require "set"`.
262
+
263
+ Contributed by Garrett Johnson.
264
+
265
+ ### Relaxed Queue Namespace Validation
266
+
267
+ Namespaces now can include any characters that are valid in RabbitMQ
268
+ queue names.
269
+
270
+ Contributed by Garrett Johnson.
271
+
272
+ ### basic.qos Configuration
273
+
274
+ It is now possible to configure `basic.qos` (aka channel prefetch) setting
275
+ used by Hutch using the `:channel_prefetch` config key.
276
+
277
+ ### Passwords No Longer Logged
278
+
279
+ Hutch now elides passwords from logs.
280
+
281
+
282
+ ## 0.7.0 — January 14, 2014
283
+
284
+ ### Optional HTTP API Use
285
+
286
+ It is now possible to make Hutch [not use RabbitMQ HTTP
287
+ API](https://github.com/gocardless/hutch/pull/69) (e.g. when the
288
+ RabbitMQ management plugin that provides it is not available).
289
+
290
+
291
+ ### Extra Arguments for Hutch::Broker#publish
292
+
293
+ Extra options [passed to `Hutch::Broker#publish` will now be propagated](https://github.com/gocardless/hutch/pull/61).
294
+
295
+
296
+ ### Content-Type for Messages
297
+
298
+ Messages published with Hutch now have content type set
299
+ to `application/json`.
300
+
301
+
302
+ ### Greater Heartbeat Interval
303
+
304
+ Hutch now uses heartbeat interval of 30, so heartbeats won't interfere with transfers
305
+ of large messages over high latency networks (e.g. between AWS availability regions).
306
+
307
+
308
+ ### Custom Queue Names
309
+
310
+ It is now possible to [specify an optional queue name](https://github.com/gocardless/hutch/pull/49):
311
+
312
+ ``` ruby
313
+ class FailedPaymentConsumer
314
+ include Hutch::Consumer
315
+ consume 'gc.ps.payment.failed'
316
+ queue_name 'failed_payments'
317
+
318
+ def process(message)
319
+ mark_payment_as_failed(message[:id])
320
+ end
321
+ end
322
+ ```
323
+
324
+ ### Global Properties for Publishers
325
+
326
+ [Global properties can now be specified](https://github.com/gocardless/hutch/pull/62) for publishing:
327
+
328
+ ``` ruby
329
+ Hutch.global_properties = proc {
330
+ { app_id: 'api', headers: { request_id: RequestId.request_id } }
331
+ }
332
+ ```
333
+
334
+ ## 0.6.0 - November 4, 2013
335
+
336
+ - Metadata can now be passed in to `#publish`
337
+
338
+ ## 0.5.1 - October 17, 2013
339
+
340
+ - Raise an exception when publishing fails
341
+
342
+ ## 0.5.0 - October 17, 2013
343
+
344
+ - Support for the `--mq-tls-key` and `--mq-tls-cert` configuration options.
345
+
346
+ ## 0.4.5 - October 15, 2013
347
+
348
+ - No exception raised when hutch is run with no consumers. Instead, a warning
349
+ is logged.
350
+ - Internal refactoring: use Bunny's shiny `ConsumerWorkPool#threads`
351
+ attr_reader.
352
+
353
+ ## 0.4.4 - October 12, 2013
354
+
355
+ - Friendlier Message#inspect, doesn't spew out detailed bunny info.
356
+
357
+ ## 0.4.3 - October 11, 2013
358
+
359
+ - More autoloading tweaks, all internal modules are now autoloaded.
360
+
361
+ ## 0.4.2 - October 11, 2013
362
+
363
+ - Autoload the Broker module, which was missed in the previous release.
364
+
365
+ ## 0.4.1 - October 11, 2013
366
+
367
+ - Autoload internal modules. Fixes regression where the `Config` module was
368
+ not available by simply `require`ing `hutch`.
369
+
370
+ ## 0.4.0 - October 9, 2013
371
+
372
+ - Support for loading configuration from a file, specified with the `--config`
373
+ command line option.
374
+
375
+ ## 0.3.0 - September 24, 2013
376
+
377
+ - Add `--[no-]autoload-rails` flag to optionally disable the autoloading of
378
+ Rails apps in the current directory
379
+
380
+ ## 0.2.1 - September 17, 2013
381
+
382
+ - Fix inconsistency with `mq-tls` option
383
+
384
+ ## 0.2.0 - September 16, 2013
385
+
386
+ - Support for connecting to RabbitMQ with TLS/SSL. There are two new
387
+ configuration options : `mq-tls` and `mq-api-ssl`.
388
+ - JSON message parsing errors are now handled properly.
389
+
390
+ ## 0.1.1 - September 9, 2013
391
+
392
+ - Relax Bunny dependency specification
393
+
394
+ ## 0.1.0 - September 9, 2013
395
+
396
+ - Initial release
397
+
data/Gemfile ADDED
@@ -0,0 +1,22 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec
4
+
5
+ group :development do
6
+ gem "rake"
7
+ gem "guard", "~> 0.8.8"
8
+ gem "guard-rspec", "~> 0.5.4"
9
+ end
10
+
11
+ group :development, :test do
12
+ gem "sentry-raven"
13
+ gem "honeybadger"
14
+ gem "coveralls", require: false
15
+ gem "newrelic_rpm"
16
+ gem "airbrake"
17
+ end
18
+
19
+ group :development, :darwin do
20
+ gem "rb-fsevent", "~> 0.9"
21
+ gem "growl", "~> 1.0.3"
22
+ end
data/Guardfile ADDED
@@ -0,0 +1,5 @@
1
+ guard 'rspec', :version => 2, :cli => '--color --format doc' do
2
+ watch(%r{^spec/.+_spec\.rb$})
3
+ watch(%r{^lib/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" }
4
+ watch('spec/spec_helper.rb') { "spec" }
5
+ end
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2013-2014 GoCardless
2
+
3
+ Permission is hereby granted, free of charge, to any person
4
+ obtaining a copy of this software and associated documentation
5
+ files (the "Software"), to deal in the Software without
6
+ restriction, including without limitation the rights to use,
7
+ copy, modify, merge, publish, distribute, sublicense, and/or sell
8
+ copies of the Software, and to permit persons to whom the
9
+ Software is furnished to do so, subject to the following
10
+ conditions:
11
+
12
+ The above copyright notice and this permission notice shall be
13
+ included in all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
16
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
17
+ OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
18
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
19
+ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
20
+ WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
21
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
22
+ OTHER DEALINGS IN THE SOFTWARE.