amqp 0.8.0.rc13 → 0.8.0.rc14

Sign up to get free protection for your applications and to get access to all the features.
Files changed (152) hide show
  1. data/.rspec +2 -1
  2. data/.travis.yml +8 -2
  3. data/.yardopts +1 -0
  4. data/CHANGELOG +9 -0
  5. data/Gemfile +17 -11
  6. data/README.md +26 -16
  7. data/amqp.gemspec +2 -2
  8. data/bin/ci/before_build.sh +21 -0
  9. data/docs/08Migration.textile +199 -5
  10. data/docs/AMQP091ModelExplained.textile +322 -0
  11. data/docs/Bindings.textile +24 -4
  12. data/docs/Clustering.textile +1 -1
  13. data/docs/ConnectingToTheBroker.textile +98 -82
  14. data/docs/ConnectionEncryptionWithTLS.textile +65 -5
  15. data/docs/DocumentationGuidesIndex.textile +93 -13
  16. data/docs/Durability.textile +1 -1
  17. data/docs/ErrorHandling.textile +458 -94
  18. data/docs/Exchanges.textile +901 -87
  19. data/docs/GettingStarted.textile +278 -143
  20. data/docs/PatternsAndUseCases.textile +420 -0
  21. data/docs/Queues.textile +730 -178
  22. data/docs/RabbitMQVersions.textile +18 -3
  23. data/docs/RunningTests.textile +1 -1
  24. data/docs/TestingWithEventedSpec.textile +121 -0
  25. data/docs/Troubleshooting.textile +15 -1
  26. data/docs/VendorSpecificExtensions.textile +1 -1
  27. data/docs/diagrams/001_hello_world_example_routing.png +0 -0
  28. data/docs/diagrams/002_blabbr_example_routing.png +0 -0
  29. data/docs/diagrams/003_weathr_example_routing.png +0 -0
  30. data/docs/diagrams/004_fanout_exchange.png +0 -0
  31. data/docs/diagrams/005_direct_exchange.png +0 -0
  32. data/docs/diagrams/redhat/direct_exchange.png +0 -0
  33. data/docs/diagrams/redhat/fanout_exchange.png +0 -0
  34. data/docs/diagrams/redhat/topic_exchange.png +0 -0
  35. data/examples/error_handling/automatic_recovery_of_channel_and_queues.rb +50 -0
  36. data/examples/error_handling/automatically_recovering_hello_world_consumer.rb +51 -0
  37. data/examples/error_handling/automatically_recovering_hello_world_consumer_that_uses_a_server_named_queue.rb +51 -0
  38. data/examples/error_handling/basic_connection_failover.rb +22 -0
  39. data/examples/error_handling/channel_level_exception.rb +9 -2
  40. data/examples/error_handling/connection_level_exception.rb +8 -1
  41. data/examples/error_handling/connection_level_exception_with_objects.rb +49 -0
  42. data/examples/error_handling/connection_loss_handler.rb +1 -5
  43. data/examples/error_handling/hello_world_producer.rb +43 -0
  44. data/examples/error_handling/insufficient_permissions.rb +54 -0
  45. data/examples/error_handling/manual_connection_and_channel_recovery.rb +71 -0
  46. data/examples/error_handling/queue_exclusivity_violation.rb +41 -0
  47. data/examples/error_handling/queue_name_violation.rb +31 -0
  48. data/examples/exchanges/autodeletion_of_exchanges.rb +1 -4
  49. data/examples/guides/queues/01a_declaring_a_server_named_queue_using_queue_constructor.rb +7 -8
  50. data/examples/guides/queues/01b_declaring_a_queue_using_queue_constructor.rb +7 -8
  51. data/examples/guides/queues/02a_declaring_a_durable_shared_queue.rb +5 -8
  52. data/examples/guides/queues/02b_declaring_a_durable_shared_queue.rb +5 -8
  53. data/examples/guides/queues/03a_declaring_a_temporary_exclusive_queue.rb +7 -8
  54. data/examples/guides/queues/04_bind_a_queue_using_exchange_instance.rb +9 -10
  55. data/examples/guides/queues/05_bind_a_queue_using_exchange_name.rb +8 -10
  56. data/examples/guides/queues/06_subscribe_to_receive_messages.rb +10 -12
  57. data/examples/guides/queues/07_fetch_a_message_from_the_queue.rb +14 -14
  58. data/examples/guides/queues/08_unsubscribing_a_consumer.rb +13 -16
  59. data/examples/guides/queues/09_unbinding_from_exchange.rb +16 -22
  60. data/examples/guides/queues/10_purge_a_queue.rb +13 -18
  61. data/examples/guides/queues/11_deleting_a_queue.rb +14 -19
  62. data/examples/guides/queues/12_objects_that_consume_messages.rb +69 -0
  63. data/examples/guides/queues/13_objects_that_consume_messages_take_two.rb +89 -0
  64. data/examples/hello_world.rb +1 -3
  65. data/examples/hello_world_with_an_empty_string.rb +5 -6
  66. data/examples/inspecting_server_information.rb +45 -0
  67. data/examples/issues/issue_93.rb +23 -0
  68. data/examples/issues/issue_94.rb +23 -0
  69. data/examples/patterns/command/consumer.rb +45 -0
  70. data/examples/patterns/command/producer.rb +26 -0
  71. data/examples/patterns/request_reply/client.rb +29 -0
  72. data/examples/patterns/request_reply/server.rb +26 -0
  73. data/examples/publishing/publishing_a_one_off_message.rb +6 -4
  74. data/examples/publishing/returned_messages.rb +2 -10
  75. data/examples/queues/accessing_message_metadata.rb +15 -13
  76. data/examples/queues/queue_status.rb +12 -15
  77. data/examples/routing/fanout_routing.rb +33 -0
  78. data/examples/routing/headers_routing.rb +17 -15
  79. data/examples/routing/round_robin_with_direct_exchange.rb +39 -0
  80. data/examples/routing/round_robin_with_the_default_exchange.rb +38 -0
  81. data/examples/routing/unroutable_mandatory_message_is_returned.rb +33 -0
  82. data/examples/routing/weather_updates.rb +15 -20
  83. data/examples/tls/using_tls.rb +41 -0
  84. data/lib/amqp/bit_set.rb +80 -0
  85. data/lib/amqp/broker.rb +72 -0
  86. data/lib/amqp/channel.rb +93 -13
  87. data/lib/amqp/client.rb +11 -22
  88. data/lib/amqp/compatibility/ruby187_patchlevel_check.rb +2 -0
  89. data/lib/amqp/connection.rb +2 -3
  90. data/lib/amqp/consumer.rb +208 -0
  91. data/lib/amqp/deprecated/fork.rb +2 -0
  92. data/lib/amqp/deprecated/mq.rb +2 -0
  93. data/lib/amqp/exchange.rb +6 -4
  94. data/lib/amqp/extensions/rabbitmq.rb +3 -1
  95. data/lib/amqp/header.rb +76 -14
  96. data/lib/amqp/int_allocator.rb +96 -0
  97. data/lib/amqp/logger.rb +2 -0
  98. data/lib/amqp/queue.rb +242 -86
  99. data/lib/amqp/rpc.rb +2 -0
  100. data/lib/amqp/session.rb +169 -9
  101. data/lib/amqp/utilities/event_loop_helper.rb +2 -0
  102. data/lib/amqp/utilities/server_type.rb +2 -0
  103. data/lib/amqp/version.rb +2 -2
  104. data/lib/mq.rb +4 -2
  105. data/lib/mq/logger.rb +3 -1
  106. data/lib/mq/rpc.rb +3 -1
  107. data/spec/integration/authentication_spec.rb +17 -10
  108. data/spec/integration/automatic_binding_for_default_direct_exchange_spec.rb +1 -1
  109. data/spec/integration/automatic_recovery_predicate_spec.rb +68 -0
  110. data/spec/integration/basic_get_spec.rb +2 -1
  111. data/spec/integration/{extensions/basic_return_spec.rb → basic_return_spec.rb} +2 -1
  112. data/spec/integration/channel_level_exception_handling_spec.rb +53 -0
  113. data/spec/integration/connection_level_exception_handling_spec.rb +49 -0
  114. data/spec/integration/declare_and_immediately_bind_a_server_named_queue_spec.rb +38 -17
  115. data/spec/integration/declare_one_hundred_server_named_queues_spec.rb +44 -0
  116. data/spec/integration/direct_exchange_routing_spec.rb +125 -0
  117. data/spec/integration/exchange_declaration_spec.rb +75 -46
  118. data/spec/integration/extensions/rabbitmq/publisher_confirmations_spec.rb +180 -0
  119. data/spec/integration/{workload_distribution_spec.rb → fanout_exchange_routing_spec.rb} +10 -9
  120. data/spec/integration/headers_exchange_routing_spec.rb +269 -0
  121. data/spec/integration/hello_world_spec.rb +77 -0
  122. data/spec/integration/immediate_messages_spec.rb +59 -0
  123. data/spec/integration/mandatory_messages_spec.rb +52 -0
  124. data/spec/integration/message_metadata_access_spec.rb +106 -0
  125. data/spec/integration/multiple_consumers_per_queue_spec.rb +319 -0
  126. data/spec/integration/ordering_of_published_messages_spec.rb +96 -0
  127. data/spec/integration/queue_declaration_spec.rb +8 -8
  128. data/spec/integration/queue_status_spec.rb +66 -0
  129. data/spec/integration/recovery/per_channel_automatic_recovery_on_graceful_broker_shutdown_spec.rb +76 -0
  130. data/spec/integration/recovery/per_channel_automatic_recovery_spec.rb +72 -0
  131. data/spec/integration/redelivery_of_unacknowledged_messages_spec.rb +96 -0
  132. data/spec/integration/regressions/concurrent_publishing_on_the_same_channel_spec.rb +91 -0
  133. data/spec/integration/regressions/empty_message_body_spec.rb +56 -0
  134. data/spec/integration/regressions/issue66_spec.rb +2 -1
  135. data/spec/integration/reply_queue_communication_spec.rb +2 -1
  136. data/spec/integration/store_and_forward_spec.rb +4 -3
  137. data/spec/integration/topic_subscription_spec.rb +2 -1
  138. data/spec/integration/tx_commit_spec.rb +124 -0
  139. data/spec/integration/tx_rollback_spec.rb +167 -0
  140. data/spec/spec_helper.rb +44 -71
  141. data/spec/unit/amqp/bit_set_spec.rb +127 -0
  142. data/spec/unit/amqp/channel_id_allocation_spec.rb +40 -0
  143. data/spec/unit/amqp/connection_spec.rb +4 -2
  144. data/spec/unit/amqp/int_allocator_spec.rb +116 -0
  145. metadata +92 -26
  146. data/CONTRIBUTORS +0 -29
  147. data/docs/Routing.textile +0 -30
  148. data/examples/real-world/task-queue/README.textile +0 -3
  149. data/examples/real-world/task-queue/consumer.rb +0 -27
  150. data/examples/real-world/task-queue/producer.rb +0 -22
  151. data/spec/unit/amqp/basic_spec.rb +0 -39
  152. data/tasks.rb +0 -4
@@ -0,0 +1,420 @@
1
+ # @title Ruby AMQP gem: Patterns and Use Cases
2
+
3
+
4
+ h1. Patterns and Use Cases
5
+
6
+
7
+ h2. About this guide
8
+
9
+ This guide explains typical messaging patterns and use cases. It only covers the most common scenarios. For comprehensive list
10
+ of messaging patterns, consult books on this subject, for example, "Enterprise Integration Patterns":http://www.eaipatterns.com.
11
+
12
+
13
+ h2. Covered versions
14
+
15
+ This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp v0.8.0 and later.
16
+
17
+
18
+
19
+ h2. Introduction
20
+
21
+ Messaging patterns are a lot like object-oriented design patterns: they are generalized reusable solutions to specific problems. They are not
22
+ recipes, however, and their exact implementation may and will vary from application to application. Just like OO design patterns,
23
+ they can classified:
24
+
25
+ * Message construction patterns describe form, content and purpose of messages.
26
+ * Message routing patterns outline how messages can be directed from producers to consumers.
27
+ * Message transformation patterns change message content or metadata.
28
+
29
+ There are other, more specialized group of messaging patterns that are out of scope of this guide.
30
+
31
+ This guide demonstrates implementation of several common routing patterns plus explains how built-in AMQP 0.9.1 features
32
+ can be used to implement message construction and message transformation patterns.
33
+
34
+ TBD
35
+
36
+
37
+
38
+ h2. Request/Reply pattern
39
+
40
+ h3. Description & Use cases
41
+
42
+ Request/Reply is a simple way of integration when one application issues a request and another application responds
43
+ to it. This pattern is often referred to as"Remote Procedure Call", even when it is not entirely correct. Request/Reply
44
+ pattern is a 1:1 communication pattern.
45
+
46
+ Some examples of Request/Reply pattern are:
47
+
48
+ * The 1st application requests a document that the 2nd application generates or loads and returns.
49
+ * End-user application issues a search request and another application returns results back.
50
+ * One application requests a progress report from another application.
51
+
52
+ h3. AMQP-based implementation
53
+
54
+ Implementation of Request/Reply pattern on top of AMQP 0.9.1 involves two messages: a request (Req) and a response (Res).
55
+ Client app generates a request identifier and sets :message_id attribute on Req. Client also uses a server-named
56
+ exclusive queue to receive replies and thus sets :reply_to Req attribute to the name of that queue.
57
+
58
+ Server app uses a well-known queue name to receive requests and sets :correlation_id to :message_id of the original
59
+ request message (Req) to make it possible for the client to identify what request this reply is for.
60
+
61
+ h4. Request message attributes
62
+
63
+ <dl>
64
+ <dt>:message_id</dt>
65
+ <dd>Unique message identifier</dd>
66
+
67
+ <dt>:reply_to</dt>
68
+ <dd>Queue name server should send the response to</dd>
69
+ </dl>
70
+
71
+ h4. Response message attributes
72
+
73
+ <dl>
74
+ <dt>:correlation_id</dt>
75
+ <dd>Identifier of the original request message (set to request's :correlation_id)</dd>
76
+
77
+ <dt>:routing_key</dt>
78
+ <dd>Client's replies queue name (set to request's :reply_to)</dd>
79
+ </dl>
80
+
81
+
82
+
83
+ h3. Code example
84
+
85
+ h4. Client code
86
+
87
+ <pre>
88
+ <code>
89
+ require "amqp"
90
+
91
+ EventMachine.run do
92
+ connection = AMQP.connect
93
+ channel = AMQP::Channel.new(connection)
94
+
95
+ replies_queue = channel.queue("", :exclusive => true, :auto_delete => true)
96
+ replies_queue.subscribe do |metadata, payload|
97
+ puts "[response] Response for #{metadata.correlation_id}: #{payload.inspect}"
98
+ end
99
+
100
+ # request time from a peer every 3 seconds
101
+ EventMachine.add_periodic_timer(3.0) do
102
+ puts "[request] Sending a request..."
103
+ channel.default_exchange.publish("get.time",
104
+ :routing_key => "amqpgem.examples.services.time",
105
+ :message_id => Kernel.rand(10101010).to_s,
106
+ :reply_to => replies_queue.name,
107
+ :immediate => true)
108
+ end
109
+
110
+
111
+ Signal.trap("INT") { connection.close { EventMachine.stop } }
112
+ end
113
+ </code>
114
+ </pre>
115
+
116
+
117
+ h4. Server code
118
+
119
+ <pre>
120
+ <code>
121
+ require "amqp"
122
+
123
+ EventMachine.run do
124
+ connection = AMQP.connect
125
+ channel = AMQP::Channel.new(connection)
126
+
127
+ requests_queue = channel.queue("amqpgem.examples.services.time", :exclusive => true, :auto_delete => true)
128
+ requests_queue.subscribe(:ack => true) do |metadata, payload|
129
+ puts "[requests] Got a request #{metadata.message_id}. Sending a reply..."
130
+ channel.default_exchange.publish(Time.now.to_s,
131
+ :routing_key => metadata.reply_to,
132
+ :correlation_id => metadata.message_id,
133
+ :immediate => true,
134
+ :mandatory => true)
135
+
136
+ metadata.ack
137
+ end
138
+
139
+
140
+ Signal.trap("INT") { connection.close { EventMachine.stop } }
141
+ end
142
+ </code>
143
+ </pre>
144
+
145
+ In the examples above messages are published with the :immediate attribute set. This is not necessary in all
146
+ cases: sometimes it is OK for requests to sit in the queue without active consumers. Replies, on the other hand,
147
+ assume an active consumer and existing replies queue, so if routing or immediate delivery do not succeed,
148
+ server application will log returned messages. More on this in the {file:docs/Exchanges.textile Working With Exchanges} guide.
149
+
150
+
151
+
152
+ h3. Related patterns
153
+
154
+ * Request/Reply
155
+ * Event
156
+ * Scatter/Gather
157
+ * Smart Proxy
158
+
159
+
160
+
161
+
162
+
163
+ h2. Command pattern
164
+
165
+ h3. Description & Use cases
166
+
167
+ Command pattern is very similar to Request/Reply, except that there is no reply and messages are typed. For example, most modern Web
168
+ applications have at least one "background task processor" that carries out a number of operations asynchronously,
169
+ without sending any responses back. Command pattern usually assumes 1:1 communication.
170
+
171
+ Some specific examples of Command pattern are:
172
+
173
+ * Account termination in a Web app triggers information archiving (or deletion) that is done by a separate app "in the background".
174
+ * After a document or profile update, a Web app sends out commands to a search indexer application.
175
+ * Virtual machines control dashboard app sends virtual machine controller application a command to reboot.
176
+
177
+
178
+ h3. AMQP-based implementation
179
+
180
+ Implementation of Command pattern on top of AMQP 0.9.1 involves well-known durable queues. Application that issues the command
181
+ then can use default exchange to publish messages to well-known services directly. Request message :type attribute then indicates
182
+ command type and message body (or body and headers) carry additional information consumer needs to carry it out.
183
+
184
+ h4. Request message attributes
185
+
186
+ <dl>
187
+ <dt>:type</dt>
188
+ <dd>Message type, as a string. For example: gems.install or commands.shutdown</dd>
189
+ </dl>
190
+
191
+ h3. Code example
192
+
193
+ h4. Producer (Sender)
194
+
195
+ <pre>
196
+ <code>
197
+ require "rubygems"
198
+ require "amqp"
199
+ require "yaml"
200
+
201
+ t = Thread.new { EventMachine.run }
202
+ sleep(0.5)
203
+
204
+
205
+ connection = AMQP.connect
206
+ channel = AMQP::Channel.new(connection, :auto_recovery => true)
207
+
208
+ channel.prefetch(1)
209
+
210
+ # Acknowledgements are good for letting the server know
211
+ # that the task is finished. If the consumer doesn't send
212
+ # the acknowledgement, then the task is considered to be unfinished
213
+ # and will be requeued when consumer closes AMQP connection (because of a crash, for example).
214
+ channel.queue("amqpgem.examples.patterns.command", :durable => true, :auto_delete => false).subscribe(:ack => true) do |metadata, payload|
215
+ case metadata.type
216
+ when "gems.install"
217
+ data = YAML.load(payload)
218
+ puts "[gems.install] Received a 'gems.install' request with #{data.inspect}"
219
+
220
+ # just to demonstrate a realistic example
221
+ shellout = "gem install #{data[:gem]} --version '#{data[:version]}'"
222
+ puts "[gems.install] Executing #{shellout}"; system(shellout)
223
+ puts "[gems.install] Done"
224
+ puts
225
+ else
226
+ puts "[commands] Unknown command: #{metadata.type}"
227
+ end
228
+
229
+ # message is processed, acknowledge it so that broker discards it
230
+ metadata.ack
231
+ end
232
+
233
+ puts "[boot] Ready. Will be publishing commands every 10 seconds."
234
+ Signal.trap("INT") { connection.close { EventMachine.stop } }
235
+ t.join
236
+ </code>
237
+ </pre>
238
+
239
+
240
+ h4. Consumer (Recipient)
241
+
242
+ <pre>
243
+ <code>
244
+ require "amqp"
245
+ require "yaml"
246
+
247
+ t = Thread.new { EventMachine.run }
248
+ sleep(0.5)
249
+
250
+ connection = AMQP.connect
251
+ channel = AMQP::Channel.new(connection)
252
+
253
+ # publish new commands every 3 seconds
254
+ EventMachine.add_periodic_timer(10.0) do
255
+ puts "Publishing a command (gems.install)"
256
+ payload = { :gem => "rack", :version => "~> 1.3.0" }.to_yaml
257
+
258
+ channel.default_exchange.publish(payload,
259
+ :type => "gems.install",
260
+ :routing_key => "amqpgem.examples.patterns.command")
261
+ end
262
+
263
+ puts "[boot] Ready"
264
+ Signal.trap("INT") { connection.close { EventMachine.stop } }
265
+ t.join
266
+ </code>
267
+ </pre>
268
+
269
+
270
+ h3. Related patterns
271
+
272
+ * Event
273
+ * Request/Reply
274
+
275
+
276
+
277
+
278
+
279
+ h2. Event pattern
280
+
281
+ h3. Description & Use cases
282
+
283
+ Event pattern is a version of the Command pattern, but with 1 or more receivers (1:N communication).
284
+ The world we live in is full of events, so applications of this pattern are endless.
285
+
286
+ Some specific use cases of Event pattern are
287
+
288
+ * Event logging (one application asks event collector to record certain event and possibly take action)
289
+ * Event propagation in MMO games
290
+ * Live sport score updates
291
+ * Various "push notifications" for mobile applications
292
+
293
+
294
+ h3. AMQP-based implementation
295
+
296
+ Because Event pattern is a 1:N communication pattern, it typically uses a fanout exchange. Event listeners
297
+ then use server-named exclusive queues and all bind to that exchange. Event messages use :type message
298
+ attribute to indicate event type and message body (plus, possibly, message headers) to pass event
299
+ context information.
300
+
301
+ More on fanout exchange type in the {file:docs/Exchanges.textile Working With Exchanges} guide.
302
+
303
+
304
+ h3. Code example
305
+
306
+ TBD
307
+
308
+
309
+ h3. Related patterns
310
+
311
+ * Command
312
+ * Publish/Subscribe
313
+
314
+
315
+
316
+ h2. Document Message pattern
317
+
318
+ h3. Description & Use cases
319
+
320
+ TBD
321
+
322
+ h3. AMQP-based implementation
323
+
324
+ TBD
325
+
326
+ h3. Code example
327
+
328
+ TBD
329
+
330
+
331
+ h2. Competing Consumers pattern
332
+
333
+ h3. Description & Use cases
334
+
335
+ "Competing Consumers":http://www.eaipatterns.com/CompetingConsumers.html are multiple consumers that process messages from a shared queue.
336
+
337
+ TBD
338
+
339
+ h3. AMQP-based implementation
340
+
341
+ TBD
342
+
343
+ h3. Code example
344
+
345
+ TBD
346
+
347
+
348
+
349
+ h2. Publish/Subscribe pattern
350
+
351
+ h3. Description & Use cases
352
+
353
+ TBD
354
+
355
+ h3. AMQP-based implementation
356
+
357
+ TBD
358
+
359
+ h3. Code example
360
+
361
+ TBD
362
+
363
+
364
+
365
+ h2. Scatter/Gather pattern
366
+
367
+ h3. Description & Use cases
368
+
369
+ TBD
370
+
371
+ h3. AMQP-based implementation
372
+
373
+ TBD
374
+
375
+ h3. Code example
376
+
377
+ TBD
378
+
379
+
380
+
381
+ h2. Smart Proxy pattern
382
+
383
+ h3. Description & Use cases
384
+
385
+ TBD
386
+
387
+ h3. AMQP-based implementation
388
+
389
+ TBD
390
+
391
+ h3. Code example
392
+
393
+ TBD
394
+
395
+
396
+
397
+
398
+ h2. Multistep Processing (Routing Slip) pattern
399
+
400
+ h3. Description & Use cases
401
+
402
+ TBD
403
+
404
+ h3. AMQP-based implementation
405
+
406
+ TBD
407
+
408
+ h3. Code example
409
+
410
+ TBD
411
+
412
+
413
+
414
+ h2. Tell us what you think!
415
+
416
+ Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
417
+ what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
418
+ key to making documentation better.
419
+
420
+ If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
data/docs/Queues.textile CHANGED
@@ -5,76 +5,138 @@ h1. Working with queues
5
5
 
6
6
  h2. About this guide
7
7
 
8
- This guide covers everything related to queues in AMQP 0.9.1, common usage scenarios and how to accomplish typical operations using
8
+ This guide covers everything related to queues in the AMQP v0.9.1 specification, common usage scenarios and how to accomplish typical operations using the
9
9
  amqp gem.
10
10
 
11
11
 
12
- h2. Covered versions
12
+ h2. Which versions of the amqp gem does this guide cover?
13
13
 
14
- This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp v0.8.0 and later.
14
+ This guide covers v0.8.0 and later of the "Ruby amqp gem":http://github.com/ruby-amqp/amqp.
15
15
 
16
16
 
17
-
18
- h2. Queues in AMQP 0.9.1, briefly
17
+ h2. Queues in AMQP v0.9.1 - overview
19
18
 
20
19
  h3. What are AMQP queues?
21
20
 
22
- Queues store and forward messages to consumers. They are similar to mailboxes in SMTP.
21
+ _Queues_ store and forward messages to consumers. They are similar to mailboxes in SMTP.
23
22
  Messages flow from producing applications to {file:docs/Exchanges.textile exchanges} that route them
24
- to queues and finally queues deliver them to consumer applications (or consumer applications fetch messages as needed).
23
+ to queues and finally queues deliver the messages to consumer applications (or consumer applications fetch messages as needed).
25
24
 
26
25
  Note that unlike some other messaging protocols/systems, messages are not delivered directly
27
26
  to queues. They are delivered to exchanges that route messages to queues using rules
28
- knows as *bindings*.
27
+ known as _bindings_.
29
28
 
30
29
  AMQP is a programmable protocol, so queues and bindings alike are declared by applications.
31
30
 
32
31
 
33
32
  h3. Concept of bindings
34
33
 
35
- Binding is an association between a queue and an exchange. Queues must be bound to at least one exchange in order to receive messages from publishers.
36
- Learn more about bindings in {file:docs/Bindings.textile Bindings guide}.
34
+ A _binding_ is an association between a queue and an exchange. Queues must be bound to at least one exchange in order to receive messages from publishers.
35
+ Learn more about bindings in the {file:docs/Bindings.textile Bindings guide}.
37
36
 
38
37
 
39
- h3. Attributes
38
+ h3. Queue attributes
40
39
 
41
40
  Queues have several attributes associated with them:
42
41
 
43
42
  * Name
44
43
  * Exclusivity
45
44
  * Durability
46
- * Whether queue is auto-deleted when no longer used
47
- * Other metadata (aka X-arguments)
45
+ * Whether the queue is auto-deleted when no longer used
46
+ * Other metadata (sometimes called _X-arguments_)
48
47
 
49
- These attributes define how queues can be used, what their lifecycle is like and other aspects of queue
48
+ These attributes define how queues can be used, what their life-cycle is like and other aspects of queue
50
49
  behavior.
51
50
 
52
- amqp gem represents queues as instances of {AMQP::Queue}.
51
+ The amqp gem represents queues as instances of {AMQP::Queue}.
53
52
 
54
- h2. Queue names. Server-named queues. Predefined queues.
55
53
 
56
- Every queue has a name that identifies it. Queue names often contain several segments separated by a dot (.), similarly to how URI
57
- path segments are separated by a slash (/), although it may be almost any string, with some limitations (see below).
58
- Applications may pick queue names or ask broker to generate a name for them. To do so, pass *empty string* as queue name argument.
54
+ h2. Queue names and declaring queues
59
55
 
60
- Here is an example:
56
+ Every AMQP queue has a name that identifies it. Queue names often contain several segments separated by a dot ".", in a similar fashion to URI
57
+ path segments being separated by a slash "/", although almost any string can represent a segment (with some limitations - see below).
61
58
 
62
- <script src="https://gist.github.com/998720.js"> </script>
59
+ Before a queue can be used, it has to be *declared*. Declaring a queue will cause it to be created if it does not already exist. The declaration will have no effect if the queue does already exist
60
+ and its attributes are the *same as those in the declaration*. When the existing queue attributes are not the same as those in the declaration a channel-level exception is raised. This case is explained later in this
61
+ guide.
62
+
63
+ h3. Explicitly named queues
64
+
65
+ Applications may pick queue names or ask the broker to generate a name for them.
66
+
67
+ To declare a queue with a particular name, for example, "images.resize", pass it to the Queue class constructor:
63
68
 
64
- If you want to declare a queue with a particular name, for example, "images.resize", pass it to Queue class constructor:
69
+ <pre>
70
+ <code>
71
+ queue = AMQP::Queue.new(channel, "images.resize", :auto_delete => true)
72
+ </code>
73
+ </pre>
65
74
 
75
+ Full example:
66
76
  <script src="https://gist.github.com/998721.js"> </script>
67
77
 
68
- Queue names starting with 'amq.' are reserved for internal use by the broker. Attempts to declare queue with a name that violates this
69
- rule will result in AMQP::IncompatibleOptionsError to be thrown (when queue is re-declared on the same channel object) or channel-level exception
70
- (when originally queue was declared on one channel and re-declaration with different attributes happens on another channel).
71
- Learn more in Error handling and recovery section below.
72
78
 
79
+ h3. Server-named queues
80
+
81
+ To ask an AMQP broker to generate a unique queue name for you, pass an *empty string* as the queue name argument:
82
+
83
+ <pre>
84
+ <code>
85
+ AMQP::Queue.new(channel, "", :auto_delete => true) do |queue, declare_ok|
86
+ puts "#{queue.name} is ready to go. AMQP method: #{declare_ok.inspect}"
87
+ end
88
+ </code>
89
+ </pre>
90
+
91
+ Full example:
92
+ <script src="https://gist.github.com/998720.js"> </script>
93
+
94
+ The amqp gem allows server-named queues to be declared without callbacks:
95
+
96
+ <pre>
97
+ <code>
98
+ queue = AMQP::Queue.new(channel, "", :auto_delete => true)
99
+ </code>
100
+ </pre>
101
+
102
+ In this case, as soon as the AMQP broker reply (`queue.declare-ok` AMQP method) arrives, the queue object name will
103
+ be assigned to the value that the broker generated. Many AMQP operations require a queue name, so before an
104
+ {AMQP::Queue} instance receives its name, those operations are delayed. This example demonstrates this:
105
+
106
+ <pre>
107
+ <code>
108
+ queue = channel.queue("")
109
+ queue.bind("builds").subscribe do |metadata, payload|
110
+ # message handling implementation...
111
+ end
112
+ </code>
113
+ </pre>
114
+
115
+ In this example, binding will be performed as soon as the queue has received its name generated by the broker.
116
+ If a particular piece of code relies on the queue name being available immediately a callback should be used.
117
+
118
+
119
+ h3. Reserved queue name prefix
120
+
121
+ Queue names starting with "amq." are reserved for internal use by the broker. Attempts to declare a queue with a name
122
+ that violates this rule will result in a channel-level exception with reply code 403 (ACCESS_REFUSED) and a reply
123
+ message similar to this:
73
124
 
125
+ <pre>ACCESS_REFUSED - queue name 'amq.queue' contains reserved prefix 'amq.*'</pre>
74
126
 
75
- h2. Queue life-cycle patterns.
76
127
 
77
- To quote AMQP 0.9.1 spec, there are two common message queue life-cycle patterns:
128
+ h3. Queue re-declaration with different attributes
129
+
130
+ When queue declaration attributes are different from those that the queue already has, a channel-level exception with
131
+ code 406 (PRECONDITION_FAILED) will be raised. The reply text will be similar to this:
132
+
133
+ <pre>PRECONDITION_FAILED - parameters for queue 'amqpgem.examples.channel_exception' in vhost '/' not equivalent</pre>
134
+
135
+
136
+
137
+ h2. Queue life-cycle patterns
138
+
139
+ According to the AMQP v0.9.1 specification, there are two common message queue life-cycle patterns:
78
140
 
79
141
  * Durable message queues that are shared by many consumers and have an independent existence: i.e. they
80
142
  will continue to exist and collect messages whether or not there are consumers to receive them.
@@ -84,40 +146,56 @@ To quote AMQP 0.9.1 spec, there are two common message queue life-cycle patterns
84
146
  There are some variations of these, such as shared message queues that are deleted when the last of
85
147
  many consumers disconnects.
86
148
 
87
- One example of durable message queues is well-known services like event collectors (event loggers).
88
- They are usually up whether there are services to log anything or not. Other applications know what
89
- queues they use and can rely on those queues being around all the time, survive broker restarts and
90
- in general be available should an application in the network need to use them. In this case,
91
- explicitly named durable queues are optimal and coupling it creates between applications is not
92
- an issue. Another scenario of a well-known long-lived service is distributed metadata/directory/locking server
93
- like Apache Zookeeper, Google's Chubby or DNS. Services like this benefit from using well-known, not generated
94
- queue names, and so do other applications that use them.
149
+ Let us examine the example of a well-known service like an event collector (event logger). A logger is
150
+ usually up and running regardless of the existence of services that want to log anything at a particular
151
+ point in time. Other applications know which queues to use in order to communicate with the logger and can
152
+ rely on those queues being available and able to survive broker restarts. In this case, explicitly named durable
153
+ queues are optimal and the coupling that is created between applications is not an issue.
154
+
155
+ Another example of a well-known long-lived service is a distributed metadata/directory/locking server like
156
+ "Apache Zookeeper":http://zookeeper.apache.org, "Google's Chubby":http://labs.google.com/papers/chubby.html or DNS. Services like this benefit from using well-known, not server-generated,
157
+ queue names and so do any other applications that use them.
95
158
 
96
- Different scenario is in "a cloud settings" when some kind of workers/instances may come online and
97
- go down basically any time and other applications cannot rely on them being available. Using well-known
98
- queue names in this case is possible but server-generated, short-lived queues that are bound to
99
- topic or fanout exchanges to receive relevant messages is a better idea.
159
+ A different sort of scenario is in "a cloud setting" when some kind of worker/instance might start and
160
+ stop at any time so that other applications cannot rely on it being available. In this case, it is possible
161
+ to use well-known queue names, but a much better solution is to use server-generated, short-lived queues
162
+ that are bound to topic or fanout exchanges in order to receive relevant messages.
100
163
 
101
- Imagine a service that processes an endless stream of events (Twitter is one example). When traffic goes
102
- up, development operations may spin up additional applications instances in the cloud to handle the load.
103
- Those new instances want to subscribe to receive messages to process but the rest of the system doesn't
104
- know anything about them, rely on them being online or try to address them directly: they process events
105
- from a shared stream and are not different from their peers. In a case like this, there is no reason for
106
- message consumers to not use queue names generated by the broker.
164
+ Imagine a service that processes an endless stream of events - Twitter is one example. When traffic
165
+ increases, development operations may start additional application instances in the cloud to handle the load.
166
+ Those new instances want to subscribe to receive messages to process, but the rest of the system does not
167
+ know anything about them and cannot rely on them being online or try to address them directly. The new instances
168
+ process events from a shared stream and are the same as their peers. In a case like this, there is no reason for
169
+ message consumers not to use queue names generated by the broker.
107
170
 
108
- In general, use of explicitly named or server-named queues depends on messaging pattern your application needs.
109
- {http://www.eaipatterns.com/ Enterprise Integration Patters} discusses many messaging patterns in depth.
110
- RabbitMQ FAQ also has a section on {http://www.rabbitmq.com/faq.html#scenarios use cases}.
171
+ In general, use of explicitly named or server-named queues depends on the messaging pattern that your application
172
+ needs. {http://www.eaipatterns.com/ Enterprise Integration Patterns} discusses many messaging patterns in depth and
173
+ the RabbitMQ FAQ also has a section on {http://www.rabbitmq.com/faq.html#scenarios use cases}.
111
174
 
112
175
 
113
176
 
114
177
  h2. Declaring a durable shared queue
115
178
 
116
- To declare a durable shared queue, you pass queue name that is a non-blank string and use :durable option:
179
+ To declare a durable shared queue, you pass a queue name that is a non-blank string and use the ":durable" option:
117
180
 
181
+ <pre>
182
+ <code>
183
+ queue = AMQP::Queue.new(channel, "images.resize", :durable => true)
184
+ </code>
185
+ </pre>
186
+
187
+ Full example:
118
188
  <script src="https://gist.github.com/998723.js"> </script>
119
189
 
120
- the same piece of code that uses {AMQP::Channel#queue} for convenience:
190
+ the same example rewritten to use {AMQP::Channel#queue}:
191
+
192
+ <pre>
193
+ <code>
194
+ channel.queue("images.resize", :durable => true) do |queue, declare_ok|
195
+ puts "#{queue.name} is ready to go."
196
+ end
197
+ </code>
198
+ </pre>
121
199
 
122
200
  <script src="https://gist.github.com/998724.js"> </script>
123
201
 
@@ -125,26 +203,72 @@ the same piece of code that uses {AMQP::Channel#queue} for convenience:
125
203
 
126
204
  h2. Declaring a temporary exclusive queue
127
205
 
128
- To declare a server-named, exclusive, auto-deleted queue, pass "" (empty string) as queue name and
129
- use :exclusive and :auto_delete options:
206
+ To declare a server-named, exclusive, auto-deleted queue, pass "" (empty string) as the queue name and
207
+ use the ":exclusive" and ":auto_delete" options:
208
+
209
+ <pre>
210
+ <code>
211
+ AMQP::Queue.new(channel, "", :auto_delete => true, :exclusive => true) do |queue, declare_ok|
212
+ puts "#{queue.name} is ready to go."
213
+ end
214
+ </code>
215
+ </pre>
216
+
217
+ Full example:
130
218
 
131
219
  <script src="https://gist.github.com/998725.js"> </script>
132
220
 
133
- the same piece of code that uses {AMQP::Channel#queue} for convenience:
221
+ The same example can be rewritten to use {AMQP::Channel#queue}:
222
+
223
+ <pre>
224
+ <code>
225
+ channel.queue("", :auto_delete => true, :exclusive => true) do |queue, declare_ok|
226
+ puts "#{queue.name} is ready to go."
227
+ end
228
+ </code>
229
+ </pre>
230
+
231
+ Full example:
134
232
 
135
233
  <script src="https://gist.github.com/998726.js"> </script>
136
234
 
235
+ Exclusive queues may only be accessed by the current connection and are deleted when that connection closes.
236
+ The declaration of an exclusive queue by other connections is not allowed and will result in a channel-level exception
237
+ with the code 405 (RESOURCE_LOCKED) and a reply message similar to
238
+
239
+ <pre>RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'amqpgem.examples.queue' in vhost '/'</pre>
240
+
241
+ The following example demonstrates this:
242
+ <script src="https://gist.github.com/1008529.js"> </script>
243
+
137
244
 
138
245
 
139
246
  h2. Binding queues to exchanges
140
247
 
141
- In order to receive messages, a queue needs to be bound to at least one exchange. Most of the time binding is explcit (done by applications).
142
- To bind a queue to an exchange, use {AMQP::Queue#bind}. Argument can be either an {AMQP::Exchange} instance, as demonstrated in this example
248
+ In order to receive messages, a queue needs to be bound to at least one exchange. Most of the time binding is explcit
249
+ (done by applications). To bind a queue to an exchange, use {AMQP::Queue#bind} where the argument passed can be
250
+ either an {AMQP::Exchange} instance or a string.
143
251
 
252
+ <pre>
253
+ <code>
254
+ queue.bind(exchange) do |bind_ok|
255
+ puts "Just bound #{queue.name} to #{exchange.name}"
256
+ end
257
+ </code>
258
+ </pre>
259
+
260
+ Full example:
144
261
  <script src="https://gist.github.com/998727.js"> </script>
145
262
 
146
- or an exchange name given as a string:
263
+ The same example using a string without callback:
147
264
 
265
+ <pre>
266
+ <code>
267
+ queue.bind("amq.fanout")
268
+ </code>
269
+ </pre>
270
+
271
+ Full example:
148
272
  <script src="https://gist.github.com/998729.js"> </script>
149
273
 
150
274
 
@@ -152,24 +276,35 @@ or an exchange name given as a string:
152
276
 
153
277
  h2. Subscribing to receive messages ("push API")
154
278
 
155
- To subscribe to receive messages when they arrive to the queue ("start a queue consumer"), one uses {AMQP::Queue#subscribe} method.
156
- Then when a message arrives, message header and body (aka payload) are passed to the handler:
279
+ To set up a queue subscription to enable an application to receive messages as they arrive in a queue, one uses the
280
+ {AMQP::Queue#subscribe} method. Then when a message arrives, the message header (metadata) and body (payload) are
281
+ passed to the handler:
282
+
283
+ <pre>
284
+ <code>
285
+ queue.subscribe do |metadata, payload|
286
+ puts "Received a message: #{payload.inspect}."
287
+ end
288
+ </code>
289
+ </pre>
157
290
 
291
+ Full example:
158
292
  <script src="https://gist.github.com/998731.js"> </script>
159
293
 
160
- Subscriptions for message delivery are usually referred to as "consumers" in the AMQP 0.9.1 spec, client libraries documentation and books.
161
- Consumers last as long as the channel they were declared on, or until the client cancels them (unsubscribes).
294
+ Subscriptions for message delivery are usually referred to as _consumers_ in the AMQP v0.9.1 specification, client
295
+ library documentation and books. Consumers last as long as the channel that they were declared on, or until the
296
+ client cancels them (unsubscribes).
162
297
 
163
- Consumers are identified by <i>consumer tags</i>. If you need to obtain consumer tag of a queue that is subscribed to receive messages,
164
- use {AMQP::Queue#consumer_tag}.
298
+ Consumers are identified by <i>consumer tags</i>. If you need to obtain the consumer tag of a subscribed
299
+ queue then use {AMQP::Queue#consumer_tag}.
165
300
 
166
301
 
167
302
  h3. Accessing message metadata
168
303
 
169
- `header` object in the example above provides access to message metadata and delivery information:
304
+ The `header` object in the example above provides access to message metadata and delivery information:
170
305
 
171
306
  * Message content type
172
- * Message content­encoding
307
+ * Message content encoding
173
308
  * Message routing key
174
309
  * Message delivery mode (persistent or not)
175
310
  * Consumer tag this delivery is for
@@ -180,16 +315,59 @@ h3. Accessing message metadata
180
315
 
181
316
  and so on. An example to demonstrate how to access some of those attributes:
182
317
 
318
+ <pre>
319
+ <code>
320
+ # producer
321
+ exchange.publish("Hello, world!",
322
+ :app_id => "amqpgem.example",
323
+ :priority => 8,
324
+ :type => "kinda.checkin",
325
+ # headers table keys can be anything
326
+ :headers => {
327
+ :coordinates => {
328
+ :latitude => 59.35,
329
+ :longitude => 18.066667
330
+ },
331
+ :participants => 11,
332
+ :venue => "Stockholm"
333
+ },
334
+ :timestamp => Time.now.to_i)
335
+ </code>
336
+ </pre>
337
+
338
+ <pre>
339
+ <code>
340
+ # consumer
341
+ queue.subscribe do |metadata, payload|
342
+ puts "metadata.routing_key : #{metadata.routing_key}"
343
+ puts "metadata.content_type: #{metadata.content_type}"
344
+ puts "metadata.priority : #{metadata.priority}"
345
+ puts "metadata.headers : #{metadata.headers.inspect}"
346
+ puts "metadata.timestamp : #{metadata.timestamp.inspect}"
347
+ puts "metadata.type : #{metadata.type}"
348
+ puts "metadata.delivery_tag: #{metadata.delivery_tag}"
349
+ puts "metadata.redelivered : #{metadata.redelivered?}"
350
+
351
+ puts "metadata.app_id : #{metadata.app_id}"
352
+ puts "metadata.exchange : #{metadata.exchange}"
353
+ puts
354
+ puts "Received a message: #{payload}."
355
+ end
356
+ </code>
357
+ </pre>
358
+
359
+ Full example:
183
360
  <script src="https://gist.github.com/998739.js"> </script>
184
361
 
185
362
 
186
363
  h3. Exclusive consumers
187
364
 
188
- Consumers can request exclusive access to the queue (meaning only this consumer can access the queue). This is useful when you want a long-lived shared
189
- queue to be temporarily accessible by just one application (or thread, or process). If application exclusive consumer is part of crashes or loses
190
- TCP connection to the broker, channel is closed and exclusive consumer is thus cancelled.
365
+ Consumers can request exclusive access to the queue (meaning only this consumer can access the queue). This is useful
366
+ when you want a long-lived shared queue to be temporarily accessible by just one application (or thread, or process).
367
+ If the application employing the exclusive consumer crashes or loses the TCP connection to the broker, then the
368
+ channel is closed and the exclusive consumer is cancelled.
191
369
 
192
- To exclusively receive messages from the queue, pass :exclusive option to {AMQP::Queue#subscribe}:
370
+ To exclusively receive messages from the queue, pass the ":exclusive" option to {AMQP::Queue#subscribe}:
193
371
 
194
372
  <pre>
195
373
  <code>
@@ -202,24 +380,90 @@ end
202
380
  TBD: describe what happens when exclusivity property is violated and how to handle it.
203
381
 
204
382
 
383
+ h3. Using multiple consumers per queue
384
+
385
+ Historically, amqp gem versions before v0.8.0.RC14 (current master branch in the repository) have had a "one consumer
386
+ per Queue instance" limitation. Previously, to work around this problem, application developers had to open multiple
387
+ channels and work with multiple queue instances on different channels. This is not very convenient and is surprising
388
+ for developers familiar with AMQP clients for other languages.
389
+
390
+ With more and more Ruby implementations dropping the "GIL":http://en.wikipedia.org/wiki/Global_Interpreter_Lock,
391
+ load balancing between multiple consumers in the same queue in the same OS process has become more and more common.
392
+ In certain cases, even applications that do not need any concurrency benefit from having multiple consumers on the
393
+ same queue in the same process.
394
+
395
+ Starting from amqp gem v0.8.0.RC14, it is possible to add any number of consumers by instantiating {AMQP::Consumer} directly:
396
+
397
+ <pre>
398
+ <code>
399
+ # non-exclusive consumer, consumer tag is generated
400
+ consumer1 = AMQP::Consumer.new(channel, queue)
401
+
402
+ # non-exclusive consumer, consumer tag is explicitly given
403
+ consumer2 = AMQP::Consumer.new(channel, queue, "#{queue.name}-consumer-#{rand}-#{Time.now}")
404
+
405
+ # exclusive consumer, consumer tag is generated
406
+ consumer3 = AMQP::Consumer.new(channel, queue, nil, true)
407
+ </code>
408
+ </pre>
409
+
410
+ Instantiated consumers do not begin consuming messages immediately. This is because in certain cases, it is useful to
411
+ add a consumer but make it active at a later time. To consume messages, use the {AMQP::Consumer#consume} method in
412
+ combination with {AMQP::Consumer#on_delivery}:
413
+
414
+ <pre>
415
+ <code>
416
+ consumer1.consume.on_delivery do |metadata, payload|
417
+ @consumer1_mailbox << payload
418
+ end
419
+ </code>
420
+ </pre>
421
+
422
+ {AMQP::Consumer#on_delivery} takes a block that is used exactly like the block passed to {AMQP::Queue#subscribe}.
423
+ In fact, {AMQP::Queue#subscribe} uses {AMQP::Consumer} under the hood, adding a _default consumer_ to the queue.
424
+
425
+ <span class="note">
426
+ Default consumers do not have any special properties, they just provide a convenient way for application developers
427
+ to register multiple consumers and a means of preserving backwards compatibility. Application developers are
428
+ always free to use AMQP::Consumer instances directly, or intermix them with AMQP::Queue#subscribe.
429
+ </span>
430
+
431
+ Most of the public API methods on {AMQP::Consumer} return self, so it is possible to use method chaining extensively.
432
+ An example from "amqp gem spec suite":https://github.com/ruby-amqp/amqp/tree/master/spec:
433
+
434
+ <pre>
435
+ <code>
436
+ consumer1 = AMQP::Consumer.new(@channel, @queue).consume.on_delivery { |metadata, payload| mailbox1 << payload }
437
+ consumer2 = AMQP::Consumer.new(@channel, @queue).consume.on_delivery { |metadata, payload| mailbox2 << payload }
438
+ </code>
439
+ </pre>
440
+
441
+ To cancel a particular consumer, use {AMQP::Consumer#cancel} method. To cancel a default queue consumer, use {AMQP::Queue#unsubscribe}.
442
+
443
+
205
444
  h3. Message acknowledgements
206
445
 
207
- Consumer applications (applications that receive and process messages) may (and will) occasionally fail to process individual messages, or will just
208
- crash. That's not to mention possible network issues. This raises a question: when should AMQP broker remove messages from queues? AMQP 0.9.1 lets
209
- you choose one of two answers:
446
+ Consumer applications - applications that receive and process messages - may occasionally fail to process individual
447
+ messages, or will just crash. There is also the possibility of network issues causing problems. This raises a
448
+ question - "When should the AMQP broker remove messages from queues?" The AMQP v0.9.1 specification proposes
449
+ two choices:
210
450
 
211
451
  * After broker sends a message to an application (using either basic.deliver or basic.get-ok methods).
212
452
  * After the application sends back an acknowledgement (using basic.ack AMQP method).
213
453
 
214
- The former model is called *automatic acknowledgement model* while the latter is *explicit acknowledgement model*. With the explicit model, application
215
- chooses when it's time to send an ack: it can be right after receiving it, or after persisting it to a data store before processing, or after fully
216
- processing the message (for example, successfully fetching a Web page, processing and storing it into some persistent data store).
454
+ The former choice is called the *automatic acknowledgement model*, while the latter is called the *explicit
455
+ acknowledgement model*. With the explicit model, the application chooses when it is time to send an acknowledgement.
456
+ It can be right after receiving a message, or after persisting it to a data store before processing, or after fully
457
+ processing the message (for example, successfully fetching a Web page, processing and storing it into some persistent
458
+ data store).
217
459
 
218
- If a consumer dies without sending an ack, AMQP broker will redeliver it to another consumer (or, if none are available at the time, it will wait
219
- until at least one consumer is registered for the same queue).
460
+ If a consumer dies without sending an acknowledgement, the AMQP broker will redeliver it to another consumer, or, if
461
+ none are available at the time, the broker will wait until at least one consumer is registered for the same queue
462
+ before attempting redelivery.
220
463
 
221
- Acknowledgement model is chosen when a new consumer is registered for a queue. By default, {AMQP::Queue#subscribe} will use the *automatic* model.
222
- To switch to the *explicit* model, :ack option should be used:
464
+ The acknowledgement model is chosen when a new consumer is registered for a queue. By default,
465
+ {AMQP::Queue#subscribe} will use the *automatic* model. To switch to the *explicit* model, the ":ack" option should
466
+ be used:
223
467
 
224
468
  <pre>
225
469
  <code>
@@ -229,16 +473,16 @@ end
229
473
  </code>
230
474
  </pre>
231
475
 
232
- To demonstrate how redelivery works, lets have a look at the following code example:
476
+ To demonstrate how redelivery works, let us have a look at the following code example:
233
477
 
234
478
  <script src="https://gist.github.com/999396.js"> </script>
235
479
 
236
- So what is going on here? This example uses 3 AMQP connections to imitate 3 applications, 1 producer and two consumers. Each AMQP connection opens a single
237
- channel:
480
+ So what is going on here? This example uses 3 AMQP connections to imitate 3 applications, 1 producer and two
481
+ consumers. Each AMQP connection opens a single channel:
238
482
 
239
483
  <pre>
240
484
  <code>
241
- # open two connections to imitate two apps
485
+ # open three connections to imitate three apps
242
486
  connection1 = AMQP.connect
243
487
  connection2 = AMQP.connect
244
488
  connection3 = AMQP.connect
@@ -260,9 +504,10 @@ channel3.on_error(&channel_exception_handler)
260
504
  </code>
261
505
  </pre>
262
506
 
263
- Consumers share a queue and producer publishes messages there periodically using `amq.direct` exchange. Both "applications" subscribe to receive messages
264
- using explicit acknowledgement model. AMQP broker by default will send each message to the next consumer, in sequence
265
- (this kind of load balancing is knownas *round-robin*). So some messages will be delivered to consumer #1 and some to consumer #2.
507
+ The consumers share a queue and the producer publishes messages to the queue periodically using an `amq.direct`
508
+ exchange. Both "applications" subscribe to receive messages using the explicit acknowledgement model. The AMQP broker
509
+ by default will send each message to the next consumer in sequence (this kind of load balancing is known as
510
+ *round-robin*). This means that some messages will be delivered to consumer #1 and some to consumer #2.
266
511
 
267
512
  <pre>
268
513
  <code>
@@ -271,7 +516,7 @@ exchange = channel3.direct("amq.direct")
271
516
  # ...
272
517
 
273
518
  queue1 = channel1.queue("amqpgem.examples.acknowledgements.explicit", :auto_delete => false)
274
- # purge the queue so that we don't get any redeliveries from previous runs
519
+ # purge the queue so that we do not get any redeliveries from previous runs
275
520
  queue1.purge
276
521
  queue1.bind(exchange).subscribe(:ack => true) do |metadata, payload|
277
522
  # do some work
@@ -285,7 +530,7 @@ queue1.bind(exchange).subscribe(:ack => true) do |metadata, payload|
285
530
  else
286
531
  # odd messages are not ack-ed and will remain in the queue for redelivery
287
532
  # when app #1 connection is closed (either properly or due to a crash)
288
- puts "[consumer1] Got message ##{metadata.headers['i']}, SKIPPPED"
533
+ puts "[consumer1] Got message ##{metadata.headers['i']}, SKIPPED"
289
534
  end
290
535
  end
291
536
 
@@ -298,21 +543,22 @@ end
298
543
  </code>
299
544
  </pre>
300
545
 
301
- To demonstrate message redelivery we make consumer #1 randomly select what messages to acknowledge. After 4 seconds we disconnect it (to imitate
302
- a crash). When that happens, AMQP broker redelivers unacknowledged messages to the consumer #2 which acknowledges them unconditionally. After 10 seconds,
303
- this example closes all outstanding connections and exits.
546
+ To demonstrate message redelivery we make consumer #1 randomly select which messages to acknowledge. After 4 seconds
547
+ we disconnect it (to imitate a crash). When that happens, the AMQP broker redelivers unacknowledged messages to
548
+ consumer #2 which acknowledges them unconditionally. After 10 seconds, this example closes all outstanding
549
+ connections and exits.
304
550
 
305
- An example output produced by this example:
551
+ An extract of output produced by this example:
306
552
 
307
553
  <pre>
308
554
  => Subscribing for messages using explicit acknowledgements model
309
555
 
310
556
  [consumer2] Received Message #0, redelivered = false, ack-ed
311
- [consumer1] Got message #1, SKIPPPED
312
- [consumer1] Got message #2, SKIPPPED
557
+ [consumer1] Got message #1, SKIPPED
558
+ [consumer1] Got message #2, SKIPPED
313
559
  [consumer1] Got message #3, ack-ed
314
560
  [consumer2] Received Message #4, redelivered = false, ack-ed
315
- [consumer1] Got message #5, SKIPPPED
561
+ [consumer1] Got message #5, SKIPPED
316
562
  [consumer2] Received Message #6, redelivered = false, ack-ed
317
563
  [consumer2] Received Message #7, redelivered = false, ack-ed
318
564
  [consumer2] Received Message #8, redelivered = false, ack-ed
@@ -352,10 +598,10 @@ An example output produced by this example:
352
598
  As we can see, consumer #1 did not acknowledge 3 messages (labelled 1, 2 and 5):
353
599
 
354
600
  <pre>
355
- [consumer1] Got message #1, SKIPPPED
356
- [consumer1] Got message #2, SKIPPPED
601
+ [consumer1] Got message #1, SKIPPED
602
+ [consumer1] Got message #2, SKIPPED
357
603
  ...
358
- [consumer1] Got message #5, SKIPPPED
604
+ [consumer1] Got message #5, SKIPPED
359
605
  </pre>
360
606
 
361
607
  and then, once consumer #1 had "crashed", those messages were immediately redelivered to the consumer #2:
@@ -367,7 +613,7 @@ Connection 1 is now closed (we pretend that it has crashed)
367
613
  [consumer2] Received Message #2, redelivered = true, ack-ed
368
614
  </pre>
369
615
 
370
- To acknowledge a message, use {AMQP::Channel#acknowledge}:
616
+ To acknowledge a message use {AMQP::Channel#acknowledge}:
371
617
 
372
618
  <pre>
373
619
  <code>
@@ -375,13 +621,14 @@ channel1.acknowledge(metadata.delivery_tag, false)
375
621
  </code>
376
622
  </pre>
377
623
 
378
- {AMQP::Channel#acknowledge} takes two arguments: message *delivery tag* and a flag that indicates whether or not we acknowledge
379
- multiple messages at once. Delivery tag is simply a channel-specific increasing number that server uses to identify deliveries.
624
+ {AMQP::Channel#acknowledge} takes two arguments: message *delivery tag* and a flag that indicates whether or not we
625
+ want to acknowledge multiple messages at once. Delivery tag is simply a channel-specific increasing number that
626
+ the server uses to identify deliveries.
380
627
 
381
- When acknowledging multiple messages at once, the delivery tag is treated as "up to and including". For example, for delivery
382
- tag = 5 that would mean "acknowledge messages 1, 2, 3, 4 and 5".
628
+ When acknowledging multiple messages at once, the delivery tag is treated as "up to and including". For example, if
629
+ delivery tag = 5 that would mean "acknowledge messages 1, 2, 3, 4 and 5".
383
630
 
384
- As a shortcut, it is possible to acknowledge messages using {AMQP::Header#ack} method:
631
+ As a shortcut, it is possible to acknowledge messages using the {AMQP::Header#ack} method:
385
632
 
386
633
  <pre>
387
634
  <code>
@@ -392,48 +639,49 @@ end
392
639
  </pre>
393
640
 
394
641
  <span class="note">
395
- Acknowledgements are channel-specific. Applications must not receive messages on one channel and acknowledge them on another channel.
642
+ Acknowledgements are channel-specific. Applications must not receive messages on one channel and acknowledge them on
643
+ another.
396
644
  </span>
397
645
 
398
646
  <span class="note">
399
- A message MUST not be acknowledged more than once. Doing so will result in a channel-level exception (PRECONDITION_FAILED)
400
- with an error message like this: «PRECONDITION_FAILED - unknown delivery tag»
647
+ A message MUST not be acknowledged more than once. Doing so will result in a channel-level exception
648
+ (PRECONDITION_FAILED) with an error message like this: «PRECONDITION_FAILED - unknown delivery tag»
401
649
  </span>
402
650
 
403
651
 
404
652
 
405
- h3. Rejecting messages.
653
+ h3. Rejecting messages
406
654
 
407
- When a consumer application receives a message, processing of that message may or may not succeed. Application can
408
- indicate it to the broker that message processing has failed (or cannot be accomplished at the time) by rejecting a message.
409
- While rejecting a message, application can ask broker to discard or requeue it.
655
+ When a consumer application receives a message, processing of that message may or may not succeed. An application can
656
+ indicate to the broker that message processing has failed (or cannot be accomplished at the time) by rejecting a
657
+ message. When rejecting a message, an application can ask the broker to discard or requeue it.
410
658
 
411
- To reject a message, use {AMQP::Channel#reject} method:
659
+ To reject a message use the {AMQP::Channel#reject} method:
412
660
 
413
661
  <pre>
414
662
  <code>
415
663
  queue.bind(exchange).subscribe do |metadata, payload|
416
- # reject but don't requeue (simply discard)
664
+ # reject but do not requeue (simply discard)
417
665
  channel.reject(metadata.delivery_tag)
418
666
  end
419
667
  </code>
420
668
  </pre>
421
669
 
422
- in the example above messages are rejected without requeueing (broker will simply discard them). To requeue rejected message,
423
- use the second argument {AMQP::Channel#reject} takes:
670
+ in the example above, messages are rejected without requeueing (broker will simply discard them). To requeue a
671
+ rejected message, use the second argument that {AMQP::Channel#reject} takes:
424
672
 
425
673
  <pre>
426
674
  <code>
427
675
  queue.bind(exchange).subscribe do |metadata, payload|
428
- # reject & requeue
676
+ # reject and requeue
429
677
  channel.reject(metadata.delivery_tag, true)
430
678
  end
431
679
  </code>
432
680
  </pre>
433
681
 
434
682
  <span class="note">
435
- When there is only one consumer on a queue, make sure you don't create infinite message delivery loops by rejecting & requeueing
436
- message from that consumer over and over.
683
+ When there is only one consumer on a queue, make sure you do not create infinite message delivery loops by rejecting
684
+ and requeueing a message from the same consumer over and over again.
437
685
  </span>
438
686
 
439
687
  Another way to reject a message is by using {AMQP::Header#reject}:
@@ -441,7 +689,7 @@ Another way to reject a message is by using {AMQP::Header#reject}:
441
689
  <pre>
442
690
  <code>
443
691
  queue.bind(exchange).subscribe do |metadata, payload|
444
- # reject but don't requeue (simply discard)
692
+ # reject but do not requeue (simply discard)
445
693
  metadata.reject
446
694
  end
447
695
  </code>
@@ -450,66 +698,73 @@ end
450
698
  <pre>
451
699
  <code>
452
700
  queue.bind(exchange).subscribe do |metadata, payload|
453
- # reject & requeue
701
+ # reject and requeue
454
702
  metadata.reject(true)
455
703
  end
456
704
  </code>
457
705
  </pre>
458
706
 
459
707
 
460
- h3. Negative acknowledgements.
708
+ h3. Negative acknowledgements
461
709
 
462
- Messages are rejected with `basic.reject` AMQP method. There is one limitation `basic.reject` has: there is no way to reject multiple
463
- messages, like you can do with acknowledgements. If you are using "RabbitMQ":http://rabbitmq.com, there is a solution: RabbitMQ provides
464
- an AMQP 0.9.1 extension known as "negative acknowledgements":http://www.rabbitmq.com/extensions.html#negative-acknowledgements (nacks) and
465
- amqp gem supports it. For more information, please refer to the {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}.
710
+ Messages are rejected with the `basic.reject` AMQP method. There is one limitation that `basic.reject` has: there
711
+ is no way to reject multiple messages, as you can do with acknowledgements. However, if you are using
712
+ "RabbitMQ":http://rabbitmq.com, then there is a solution. RabbitMQ provides an AMQP v0.9.1 extension known as
713
+ "negative acknowledgements":http://www.rabbitmq.com/extensions.html#negative-acknowledgements (nacks) and
714
+ the amqp gem supports this extension. For more information, please refer to the
715
+ {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}.
466
716
 
467
717
 
468
- h3. QoS. Prefetching messages.
718
+ h3. QoS - Prefetching messages
469
719
 
470
- For cases when multiple consumers share a queue, it is useful to be able to specify how many messages each consumer can be sent at once
471
- (before sending the next acknowledgement). This can be used as a simple load balancing technique or to improve throughput if messages tend
472
- to be published in batches (for example, if producing application sends them every minute because of the nature of the work it is doing).
720
+ For cases when multiple consumers share a queue, it is useful to be able to specify how many messages each consumer
721
+ can be sent at once before sending the next acknowledgement. This can be used as a simple load balancing technique or
722
+ to improve throughput if messages tend to be published in batches. For example, if a producing application sends
723
+ messages every minute because of the nature of the work it is doing.
473
724
 
474
- Imagine a Website that takes data from social media sources like Twitter or Facebook during the Champions League final (or the Superbowl), and then
475
- calculates how many tweets mentioned a particular team over the last minute. It can be structured as 3 applications:
725
+ Imagine a website that takes data from social media sources like Twitter or Facebook during the Champions League
726
+ final (or the Superbowl), and then calculates how many tweets mention a particular team during the last minute.
727
+ The site could be structured as 3 applications:
476
728
 
477
729
  * A crawler that uses streaming APIs to fetch tweets/statuses, normalizes them and sends them in JSON for processing by other applications ("app A").
478
730
  * A calculator that detects what team is mentioned in a message, updates statistics and pushes an update to the Web UI once a minute ("app B").
479
731
  * A Web UI that fans visit to see the stats ("app C").
480
732
 
481
- In this imaginary example, tweets per second rate will vary but to improve throughput of the system and decrease maximum number of messages AMQP broker
482
- has to hold in memory at once, applications can be designed in such a way that application "app B", the "calculator", receives 5000 messages and then
483
- acknowledges them all at once. Broker will not send message 5001 to it unless it receives an acknowledgement for #1, either individually or in bulk
484
- with other messages.
733
+ In this imaginary example, the "tweets per second" rate will vary, but to improve the throughput of the system and
734
+ to decrease the maximum number of messages that the AMQP broker has to hold in memory at once, applications can be
735
+ designed in such a way that application "app B", the "calculator", receives 5000 messages and then acknowledges them
736
+ all at once. The broker will not send message 5001 unless it receives an acknowledgement.
485
737
 
486
- In AMQP parlance this is know as *QoS* or *message prefetching*. Prefetching is configured on per-channel (typically) or per-connection (rarely used)
487
- basis. To configure prefetching per channel, use {AMQP::Channel#prefetch} method. Lets come back to the example we used in the "Message acknowledgements"
488
- section:
738
+ In AMQP parlance this is know as *QoS* or *message prefetching*. Prefetching is configured on a per-channel
739
+ (typically) or per-connection (rarely used) basis. To configure prefetching per channel, use
740
+ the {AMQP::Channel#prefetch} method. Let us return to the example we used in the "Message acknowledgements" section:
489
741
 
490
742
  <pre>
491
743
  <code>
492
- # first app will be given up to 3 messages at a time. If it doesn't
493
- # ack any messages after it was delivered 3, messages will be routed to
494
- # the app #2.
744
+ # app #1 will be given up to 3 messages at a time. If it does not
745
+ # send an ack after receiving the messages, then the messages will
746
+ # be routed to app #2.
495
747
  channel1.prefetch(3)
496
748
 
497
- # app #2 processes messages one-by-one and has to send and ack every time
749
+ # app #2 processes messages one-by-one and has to send an ack after receiving each message
498
750
  channel2.prefetch(1)
499
751
  </code>
500
752
  </pre>
501
753
 
502
- In that example, one consumer prefetches 3 messages and another consumer prefetches just 1. If we take a look at the output that example produces,
503
- we will see that `consumer1` fetched 4 messages and acknowledged 1. After that, all subsequent messages were delivered to the `consumer2`:
754
+ In that example, one consumer prefetches 3 messages and another consumer prefetches just 1. If we take a look at the output that the example produces, we will see that `consumer1` fetched 4 messages and acknowledged 1. After that,
755
+ all subsequent messages were delivered to `consumer2`:
504
756
 
505
757
  <pre>
506
758
  [consumer2] Received Message #0, redelivered = false, ack-ed
507
- [consumer1] Got message #1, SKIPPPED
508
- [consumer1] Got message #2, SKIPPPED
759
+ [consumer1] Got message #1, SKIPPED
760
+ [consumer1] Got message #2, SKIPPED
509
761
  [consumer1] Got message #3, ack-ed
510
762
  [consumer2] Received Message #4, redelivered = false, ack-ed
511
- [consumer1] Got message #5, SKIPPPED
512
- --- by now consumer 1 has received 3 messages it did not acknowledge. With prefetch = 3, AMQP broker won't send it any more messages until consumer 1 sends an ack ---
763
+ [consumer1] Got message #5, SKIPPED
764
+ ---
765
+ by now consumer 1 has received 3 messages it did not acknowledge.
766
+ With prefetch = 3, AMQP broker will not send it any more messages until consumer 1 sends an ack
767
+ ---
513
768
  [consumer2] Received Message #6, redelivered = false, ack-ed
514
769
  [consumer2] Received Message #7, redelivered = false, ack-ed
515
770
  [consumer2] Received Message #8, redelivered = false, ack-ed
@@ -519,68 +774,367 @@ we will see that `consumer1` fetched 4 messages and acknowledged 1. After that,
519
774
  </pre>
520
775
 
521
776
  <span class="note">
522
- Prefetching setting is ignored for consumers that do not use explicit acknowledgements
777
+ The prefetching setting is ignored for consumers that do not use explicit acknowledgements.
523
778
  </span>
524
779
 
525
780
 
781
+ h2. How message acknowledgements relate to transactions and Publisher Confirms
782
+
783
+ In cases where you cannot afford to lose a single message, AMQP v0.9.1 applications can use one or a combination of
784
+ the following protocol features:
785
+
786
+ * Publisher confirms (a RabbitMQ-specific extension to AMQP v0.9.1)
787
+ * Publishing messages as immediate
788
+ * Transactions (noticeable overhead)
789
+
790
+ This topic is covered in depth in the {file:docs/Exchanges.textile Working With Exchanges} guide.
791
+ In this guide, we will only mention how message acknowledgements are related to AMQP transactions and the Publisher
792
+ Confirms extension.
793
+
794
+ Let us consider a publisher application (P) that communications with a consumer (C) using AMQP v0.9.1. Their
795
+ communication can be graphically represented like this:
796
+
797
+ <pre>
798
+ ----- ----- -----
799
+ | | S1 | | S2 | |
800
+ | P | ====> | B | ====> | C |
801
+ | | | | | |
802
+ ----- ----- -----
803
+ </pre>
804
+
805
+ We have two network segments, S1 and S2. Each of them may fail. P is concerned with making sure that
806
+ messages cross S1, while broker (B) and C are concerned with ensuring that messages cross S2 and are only removed
807
+ from the queue when they are processed successfully.
808
+
809
+ Message acknowledgements cover reliable delivery over S2 as well as successful processing. For S1, P has to use
810
+ transactions (a heavyweight solution) or the more lightweight Publisher Confirms RabbitMQ extension.
811
+
526
812
 
527
813
  h2. Fetching messages when needed ("pull API")
528
814
 
529
- AMQP 0.9.1 also provides a way for applications to fetch (pull) messages from the queue only when necessary. For that, use
530
- {AMQP::Queue#pop}:
815
+ The AMQP v0.9.1 specification also provides a way for applications to fetch (pull) messages from the queue only
816
+ when necessary. For that, use {AMQP::Queue#pop}:
817
+
818
+ <pre>
819
+ <code>
820
+ queue.pop do |metadata, payload|
821
+ if payload
822
+ puts "Fetched a message: #{payload.inspect}, content_type: #{metadata.content_type}. Shutting down..."
823
+ else
824
+ puts "No messages in the queue"
825
+ end
826
+ end
827
+ </code>
828
+ </pre>
531
829
 
830
+ Full example:
532
831
  <script src="https://gist.github.com/998732.js"> </script>
533
832
 
534
- TBD
833
+ If the queue is empty, then the `payload` argument will be nil, otherwise arguments are identical to those of
834
+ the {AMQP::Queue#subscribe} callback.
535
835
 
536
836
 
537
837
  h2. Unsubscribing from messages
538
838
 
539
- Sometimes it is necessary to unsubscribe from messages without deleting a queue. To do that, use {AMQP::Queue#unsubscribe} method:
839
+ Sometimes it is necessary to unsubscribe from messages without deleting a queue. To do that, use
840
+ the {AMQP::Queue#unsubscribe} method:
841
+
842
+ <pre>
843
+ <code>
844
+ queue.unsubscribe
845
+ </code>
846
+ </pre>
847
+
848
+ By default {AMQP::Queue#unsubscribe} uses the ":noack" option to inform the broker that there is no need to send
849
+ a confirmation. In other words, it does not expect you to pass in a callback, because the consumer tag on the queue
850
+ instance and the registered callback for messages are cleared immediately.
851
+
852
+ If an application needs to execute a piece of code after the broker response arrives, {AMQP::Queue#unsubscribe} takes
853
+ an optional callback:
540
854
 
855
+ <pre>
856
+ <code>
857
+ queue.unsubscribe do |unbind_ok|
858
+ # server response arrived, handle it if necessary...
859
+ end
860
+ </code>
861
+ </pre>
862
+
863
+ Full example:
541
864
  <script src="https://gist.github.com/998734.js"> </script>
542
865
 
543
- By default {AMQP::Queue#unsubscribe} uses :noack option to inform broker that there is no need to send a
544
- confirmation. In other words, it does not expect you to pass in a callback, because consumer tag on the queue instance and registered
545
- callback for messages are cleared immediately.
866
+ In AMQP parlance, unsubscribing from messages is often referred to as "cancelling a consumer". Once a consumer is
867
+ cancelled, messages will no longer be delivered to it, however, due to the asynchronous nature of the protocol,
868
+ it is possible for "in flight" messages to be received after this call completes.
869
+
870
+ Fetching messages with {AMQP::Queue#pop} is still possible even after a consumer is cancelled.
546
871
 
547
872
 
548
873
  h2. Unbinding queues from exchanges
549
874
 
550
- To unbind queue from exchange, use {AMQP::Queue#unbind}:
875
+ To unbind a queue from an exchange use {AMQP::Queue#unbind}:
551
876
 
877
+ <pre>
878
+ <code>
879
+ queue.unbind(exchange)
880
+ </code>
881
+ </pre>
882
+
883
+ Full example:
552
884
  <script src="https://gist.github.com/998742.js"> </script>
553
885
 
554
- Note that unbinding an exchange queue was never bound to will result in a channel-level exception.
886
+ Note that trying to unbind a queue from an exchange that the queue was never bound to will result in a
887
+ channel-level exception.
888
+
889
+
890
+ h2. Querying the number of messages in a queue
891
+
892
+ It is possible to query the number of messages sitting in the queue by declaring the queue with the ":passive"
893
+ attribute set. The response (`queue.declare-ok` AMQP method) will include the number of messages along with other
894
+ attributes. However, the amqp gem provides a convenience method, {AMQP::Queue#status}:
895
+
896
+ <pre>
897
+ <code>
898
+ queue.status do |number_of_messages, number_of_consumers|
899
+ puts
900
+ puts "# of messages in the queue #{queue.name} = #{number_of_messages}"
901
+ puts
902
+ end
903
+ </code>
904
+ </pre>
905
+
906
+ Full example:
907
+ <script src="https://gist.github.com/1068363.js"> </script>
908
+
909
+
910
+ h2. Querying the number of consumers on a queue
911
+
912
+ It is possible to query the number of consumers on a queue by declaring the queue with the ":passive" attribute set.
913
+ The response (`queue.declare-ok` AMQP method) will include the number of consumers along with other attributes.
914
+ However, the amqp gem provides a convenience method, {AMQP::Queue#status}:
915
+
916
+ <pre>
917
+ <code>
918
+ queue.status do |number_of_messages, number_of_consumers|
919
+ puts
920
+ puts "# of consumers on the queue #{queue.name} = #{number_of_consumers}"
921
+ puts
922
+ end
923
+ </code>
924
+ </pre>
925
+
926
+ Full example:
927
+ <script src="https://gist.github.com/1068377.js"> </script>
555
928
 
556
929
 
557
930
  h2. Purging queues
558
931
 
559
- It is possible to purge a queue (remove all messages from it) using {AMQP::Queue#purge}:
932
+ It is possible to purge a queue (remove all of the messages from it) using {AMQP::Queue#purge}:
560
933
 
561
- <script src="https://gist.github.com/998743.js"> </script>
934
+ <pre>
935
+ <code>
936
+ queue.purge
937
+ </code>
938
+ </pre>
562
939
 
940
+ This method takes an optional callback. However, remember that this operation is performed asynchronously.
941
+ To run a piece of code when the AMQP broker confirms that a queue has been purged, use a callback that
942
+ {AMQP::Queue#purge} takes:
563
943
 
564
- This method takes a callback but it is optional. However, remember that this operation is performed asynchronously.
944
+ <pre>
945
+ <code>
946
+ queue.purge do |_|
947
+ puts "Purged #{queue.name}"
948
+ end
949
+ </code>
950
+ </pre>
565
951
 
952
+ Full example:
953
+ <script src="https://gist.github.com/998743.js"> </script>
954
+
955
+ Note that this example purges a newly declared queue with a unique server-generated name. When a queue is declared,
956
+ it is empty, so for server-named queues, there is no need to purge them before they are used.
566
957
 
567
958
 
568
959
  h2. Deleting queues
569
960
 
570
- To delete a queue, use {AMQP::Queue#delete}:
961
+ To delete a queue, use {AMQP::Queue#delete}. When a queue is deleted, all of the messages in it are deleted as well.
962
+
963
+ <pre>
964
+ <code>
965
+ queue.delete
966
+ </code>
967
+ </pre>
968
+
969
+ This method takes an optional callback. However, remember that this operation is performed asynchronously.
970
+ To run a piece of code when the AMQP broker confirms that a queue has been deleted, use a callback that
971
+ {AMQP::Queue#delete} takes:
972
+
973
+ <pre>
974
+ <code>
975
+ queue.delete do |_|
976
+ puts "Deleted #{queue.name}"
977
+ end
978
+ </code>
979
+ </pre>
571
980
 
981
+ Full example:
572
982
  <script src="https://gist.github.com/998744.js"> </script>
573
983
 
574
- This method takes a callback but it is optional. However, remember that this operation is performed asynchronously.
575
984
 
576
- When queue is deleted, all the messages in it are deleted as well.
985
+ h2. Objects as message consumers and unit testing consumers in isolation
577
986
 
987
+ Since Ruby is a genuine object-oriented language, it is important to demonstrate how the Ruby amqp gem can be
988
+ integrated into rich object-oriented code. This part of the guide focuses on queues and the problems/solutions
989
+ concerning consumer applications (applications that primarily receive and process messages, as opposed to producers
990
+ that publish them).
578
991
 
992
+ An {AMQP::Queue#subscribe} callback does not have to be a block. It can be any Ruby object that responds to the
993
+ `call` method. A common technique is to combine {http://rubydoc.info/stdlib/core/1.8.7/Object:method Object#method}
994
+ and {http://rubydoc.info/stdlib/core/1.8.7/Method:to_proc Method#to_proc} and use object methods as message handlers.
579
995
 
580
- h2. Queue durability vs Message durability
996
+ An example to demonstrate this technique:
997
+
998
+ <pre>
999
+ <code>
1000
+ class Consumer
1001
+
1002
+ #
1003
+ # API
1004
+ #
1005
+
1006
+ def initialize(channel, queue_name = AMQ::Protocol::EMPTY_STRING)
1007
+ @queue_name = queue_name
1008
+
1009
+ @channel = channel
1010
+ # Consumer#handle_channel_exception will handle channel
1011
+ # exceptions. Keep in mind that you can only register one error handler,
1012
+ # so the last one registered "wins".
1013
+ @channel.on_error(&method(:handle_channel_exception))
1014
+ end # initialize
1015
+
1016
+ def start
1017
+ @queue = @channel.queue(@queue_name, :exclusive => true)
1018
+ # #handle_message method will be handling messages routed to @queue
1019
+ @queue.subscribe(&method(:handle_message))
1020
+ end # start
1021
+
1022
+
1023
+
1024
+ #
1025
+ # Implementation
1026
+ #
1027
+
1028
+ def handle_message(metadata, payload)
1029
+ puts "Received a message: #{payload}, content_type = #{metadata.content_type}"
1030
+ end # handle_message(metadata, payload)
1031
+
1032
+ def handle_channel_exception(channel, channel_close)
1033
+ puts "Oops... a channel-level exception: code = #{channel_close.reply_code}, message = #{channel_close.reply_text}"
1034
+ end # handle_channel_exception(channel, channel_close)
1035
+ end
1036
+ </code>
1037
+ </pre>
1038
+
1039
+ Full example:
1040
+
1041
+ <script src="https://gist.github.com/1009425.js"> </script>
1042
+
1043
+
1044
+ In this example, `Consumer` instances have to be instantiated with an {AMQP::Channel} instance. If the message
1045
+ handling was done by an aggregated object, it would completely separate the handling logic and would be make it
1046
+ easy to unit test in isolation:
1047
+
1048
+ <pre>
1049
+ <code>
1050
+ class Consumer
1051
+
1052
+ #
1053
+ # API
1054
+ #
1055
+
1056
+ def handle_message(metadata, payload)
1057
+ puts "Received a message: #{payload}, content_type = #{metadata.content_type}"
1058
+ end # handle_message(metadata, payload)
1059
+ end
1060
+
1061
+
1062
+ class Worker
1063
+
1064
+ #
1065
+ # API
1066
+ #
581
1067
 
582
- See {file:docs/Durability.textile Durability guide}
583
1068
 
1069
+ def initialize(channel, queue_name = AMQ::Protocol::EMPTY_STRING, consumer = Consumer.new)
1070
+ @queue_name = queue_name
1071
+
1072
+ @channel = channel
1073
+ @channel.on_error(&method(:handle_channel_exception))
1074
+
1075
+ @consumer = consumer
1076
+ end # initialize
1077
+
1078
+ def start
1079
+ @queue = @channel.queue(@queue_name, :exclusive => true)
1080
+ @queue.subscribe(&@consumer.method(:handle_message))
1081
+ end # start
1082
+
1083
+
1084
+ #
1085
+ # Implementation
1086
+ #
1087
+
1088
+ def handle_channel_exception(channel, channel_close)
1089
+ puts "Oops... a channel-level exception: code = #{channel_close.reply_code}, message = #{channel_close.reply_text}"
1090
+ end # handle_channel_exception(channel, channel_close)
1091
+ end
1092
+ </code>
1093
+ </pre>
1094
+
1095
+ Full example:
1096
+ <script src="https://gist.github.com/1009447.js"> </script>
1097
+
1098
+
1099
+ Note that the `Consumer` class demonstrated above can be easily tested in isolation without spinning up any AMQP
1100
+ connections:
1101
+
1102
+ <pre>
1103
+ <code>
1104
+ require "ostruct"
1105
+ require "json"
1106
+
1107
+ # RSpec example
1108
+ describe Consumer do
1109
+ describe "when a new message arrives" do
1110
+ subject { described_class.new }
1111
+
1112
+ let(:metadata) do
1113
+ o = OpenStruct.new
1114
+
1115
+ o.content_type = "application/json"
1116
+ o
1117
+ end
1118
+ let(:payload) { JSON.encode({ :command => "reload_config" }) }
1119
+
1120
+ it "does some useful work" do
1121
+ # check preconditions here if necessary
1122
+
1123
+ subject.handle_message(metadata, payload)
1124
+
1125
+ # add your code expectations here
1126
+ end
1127
+ end
1128
+ end
1129
+ </code>
1130
+ </pre>
1131
+
1132
+ TBD
1133
+
1134
+
1135
+ h2. Queue durability vs message durability
1136
+
1137
+ See {file:docs/Durability.textile Durability guide}
584
1138
 
585
1139
 
586
1140
  h2. Error handling and recovery
@@ -588,34 +1142,32 @@ h2. Error handling and recovery
588
1142
  See {file:docs/ErrorHandling.textile Error handling and recovery guide}
589
1143
 
590
1144
 
591
-
592
1145
  h2. Vendor-specific extensions related to queues
593
1146
 
594
1147
  See {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}
595
1148
 
596
1149
 
597
-
598
1150
  h2. What to read next
599
1151
 
600
- Documentation is organized as several {file:docs/DocumentationGuidesIndex.textile documentation guides}, covering all kinds of
601
- topics. Guides related to this one are
1152
+ The documentation is organized as several {file:docs/DocumentationGuidesIndex.textile documentation guides},
1153
+ covering all kinds of topics. Guides related to this one are:
602
1154
 
603
- * {file:docs/Exchanges.textile Exchanges}
1155
+ * {file:docs/Exchanges.textile Working With Exchanges}
604
1156
  * {file:docs/Bindings.textile Bindings}
605
1157
  * {file:docs/ErrorHandling.textile Error handling and recovery}
606
1158
 
607
- RabbitMQ implements a number of extensions to AMQP 0.9.1 functionality, covered in the {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}.
608
- At least one extension, per-queue messages time-to-live (TTL), is related to this guide and can be used with amqp gem 0.8.0 and later.
609
-
1159
+ RabbitMQ implements a number of extensions to AMQP v0.9.1 functionality that are covered in the
1160
+ {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}. At least one extension,
1161
+ per-queue messages time-to-live (TTL), is related to this guide and can be used with the amqp gem v0.8.0 and later.
610
1162
 
611
1163
 
612
1164
  h2. Tell us what you think!
613
1165
 
614
- Please take a moment and tell us what you think about this guide on "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
615
- what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
616
- key to making documentation better.
1166
+ Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
1167
+ Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
1168
+ key to making the documentation better.
617
1169
 
618
- If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
1170
+ If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
619
1171
 
620
1172
 
621
1173
  <div id="disqus_thread"></div>