amqp 0.8.0.rc14 → 0.8.0.rc15
Sign up to get free protection for your applications and to get access to all the features.
- data/.travis.yml +3 -3
- data/Gemfile +9 -6
- data/README.md +18 -12
- data/amqp.gemspec +2 -2
- data/bin/docup +3 -0
- data/docs/08Migration.textile +67 -5
- data/docs/AMQP091ModelExplained.textile +138 -101
- data/docs/Bindings.textile +109 -8
- data/docs/ConnectingToTheBroker.textile +8 -0
- data/docs/ConnectionEncryptionWithTLS.textile +5 -0
- data/docs/DocumentationGuidesIndex.textile +21 -5
- data/docs/Durability.textile +3 -1
- data/docs/ErrorHandling.textile +20 -0
- data/docs/Exchanges.textile +7 -1
- data/docs/GettingStarted.textile +10 -0
- data/docs/PatternsAndUseCases.textile +6 -0
- data/docs/Queues.textile +7 -1
- data/docs/RabbitMQVersions.textile +6 -1
- data/docs/RunningTests.textile +8 -3
- data/docs/Troubleshooting.textile +31 -0
- data/docs/VendorSpecificExtensions.textile +137 -6
- data/examples/extensions/rabbitmq/per_queue_message_ttl.rb +24 -25
- data/examples/extensions/rabbitmq/publisher_confirmations_with_transient_messages.rb +11 -20
- data/examples/extensions/rabbitmq/using_alternate_exchanges.rb +28 -0
- data/examples/hello_world.rb +1 -1
- data/lib/amqp.rb +1 -0
- data/lib/amqp/compatibility/ruby187_patchlevel_check.rb +2 -0
- data/lib/amqp/integration/rails.rb +17 -0
- data/lib/amqp/version.rb +1 -1
- data/spec/integration/authentication_spec.rb +2 -2
- data/spec/integration/fanout_exchange_routing_spec.rb +43 -199
- data/spec/integration/multiple_consumers_per_queue_spec.rb +7 -7
- data/spec/integration/regressions/concurrent_publishing_on_the_same_channel_spec.rb +1 -1
- data/spec/integration/stress/publishing_of_messages_with_incrementing_sizes_spec.rb +50 -0
- metadata +13 -9
data/.travis.yml
CHANGED
data/Gemfile
CHANGED
@@ -18,16 +18,19 @@ custom_gem "eventmachine"
|
|
18
18
|
custom_gem "amq-client", :git => "git://github.com/ruby-amqp/amq-client.git", :branch => "master"
|
19
19
|
custom_gem "amq-protocol", :git => "git://github.com/ruby-amqp/amq-protocol.git", :branch => "master"
|
20
20
|
|
21
|
-
group
|
21
|
+
group :development do
|
22
22
|
gem "yard", ">= 0.7.2"
|
23
23
|
# yard tags this buddy along
|
24
24
|
gem "RedCloth", :platform => :mri
|
25
|
-
gem "rdiscount", :platform => :ruby
|
26
|
-
gem "yajl-ruby", :platform => :ruby
|
27
25
|
|
28
|
-
|
29
|
-
|
30
|
-
|
26
|
+
platform :ruby do
|
27
|
+
gem "rdiscount"
|
28
|
+
gem "yajl-ruby"
|
29
|
+
|
30
|
+
# To test event loop helper and various Rack apps
|
31
|
+
gem "thin"
|
32
|
+
gem "unicorn"
|
33
|
+
end
|
31
34
|
|
32
35
|
gem "changelog"
|
33
36
|
end
|
data/README.md
CHANGED
@@ -1,16 +1,16 @@
|
|
1
1
|
# About Ruby amqp gem #
|
2
2
|
|
3
3
|
Ruby amqp gem is a widely used, feature-rich, well-maintained asynchronous AMQP 0.9.1 client with batteries included.
|
4
|
-
This library works with Ruby 1.8.7 (
|
4
|
+
This library works with Ruby 1.8.7 (*except for p249*, see the FAQ), Ruby 1.9.2, Ruby 1.9.3-preview1, [JRuby](http://jruby.org), [Rubinius](http://rubini.us) as well as [REE](http://www.rubyenterpriseedition.com), and is licensed under the [Ruby License](http://www.ruby-lang.org/en/LICENSE.txt)
|
5
5
|
|
6
|
-
|
6
|
+
0.8.0.RCs and later versions of amqp gem implement [AMQP 0.9.1](http://bit.ly/amqp-model-explained) (see also [AMQP 0.9.1 spec document](http://bit.ly/hw2ELX)) and support [RabbitMQ extensions to AMQP 0.9.1](http://www.rabbitmq.com/extensions.html).
|
7
7
|
|
8
8
|
[](http://travis-ci.org/ruby-amqp/amqp)
|
9
9
|
|
10
10
|
|
11
11
|
## I know what AMQP is, how do I get started? ##
|
12
12
|
|
13
|
-
See [Getting started with amqp gem](http://bit.ly/getting-started-with-amqp-ruby-gem) and other [documentation guides](http://bit.ly/amqp-gem-docs).
|
13
|
+
See [Getting started with amqp gem](http://bit.ly/getting-started-with-amqp-ruby-gem) and other [documentation guides](http://bit.ly/amqp-gem-docs). We recommend that you read [AMQP 0.9.1 Model Explained](http://bit.ly/amqp-model-explained), too.
|
14
14
|
|
15
15
|
|
16
16
|
|
@@ -95,11 +95,11 @@ On other OSes or [JRuby](http://jruby.org):
|
|
95
95
|
|
96
96
|
EventMachine.run do
|
97
97
|
connection = AMQP.connect(:host => '127.0.0.1')
|
98
|
-
puts "
|
98
|
+
puts "Connecting to AMQP broker. Running #{AMQP::VERSION} version of the gem..."
|
99
99
|
|
100
100
|
channel = AMQP::Channel.new(connection)
|
101
101
|
queue = channel.queue("amqpgem.examples.hello_world", :auto_delete => true)
|
102
|
-
exchange = channel.
|
102
|
+
exchange = channel.default_exchange
|
103
103
|
|
104
104
|
queue.subscribe do |payload|
|
105
105
|
puts "Received a message: #{payload}. Disconnecting..."
|
@@ -136,8 +136,8 @@ should [file to us](http://github.com/ruby-amqp/amqp/issues). Or just complain t
|
|
136
136
|
* Blabbr, a Twitter-like example of broadcasting (1-to-many communication)
|
137
137
|
* Weathr, an example of sophisticated routing capabilities AMQP 0.9.1 has to offer (1-to-many or many-to-many communication)
|
138
138
|
|
139
|
-
all in under 20 minutes.
|
140
|
-
|
139
|
+
all in under 20 minutes. [AMQP 0.9.1 Protocol Tutorial](http://bit.ly/amqp-model-explained) will introduce you to protocol concepts
|
140
|
+
in less than 5 minutes.
|
141
141
|
|
142
142
|
|
143
143
|
### Examples ###
|
@@ -160,11 +160,16 @@ error handing & recovery, broker-specific extensions, TLS support, troubleshooti
|
|
160
160
|
|
161
161
|
|
162
162
|
|
163
|
-
|
164
163
|
## How to use AMQP gem with Ruby on Rails, Merb, Sinatra and other web frameworks ##
|
165
164
|
|
166
|
-
We cover
|
167
|
-
|
165
|
+
We cover Web application integration for multiple Ruby Web servers in [Connecting to the broker guide](http://bit.ly/kFCVQU).
|
166
|
+
|
167
|
+
|
168
|
+
|
169
|
+
## Migration from amqp gem 0.6.x and 0.7.x
|
170
|
+
|
171
|
+
Upgrading from amqp gem 0.6.x and 0.7.x to to 0.8.0.RCs is straightforward, please see [amqp gem 0.8.0 migration guide](http://bit.ly/amqp-gem-080-migration).
|
172
|
+
The same guide explains amqp gem versions history and why you would want to upgrade.
|
168
173
|
|
169
174
|
|
170
175
|
|
@@ -208,6 +213,7 @@ Special thanks to Dmitriy Samovskiy, Ben Hood and Tony Garnock-Jones.
|
|
208
213
|
|
209
214
|
### AMQP resources ###
|
210
215
|
|
216
|
+
* [AMQP 0.9.1 Model Explained](http://bit.ly/amqp-model-explained)
|
211
217
|
* [RabbitMQ tutorials](http://www.rabbitmq.com/getstarted.html) that demonstrate interoperability
|
212
218
|
* [Wikipedia page on AMQP](http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol)
|
213
219
|
* [AMQP quick reference](http://www.rabbitmq.com/amqp-0-9-1-quickref.html)
|
@@ -242,8 +248,8 @@ In order to make code like the following (pseudo-synchronous) work
|
|
242
248
|
ex.publish(some_data)
|
243
249
|
|
244
250
|
and not be affected by this [Ruby 1.8.7-p249-specific bug (super called outside of method)](http://bit.ly/iONBmH), we need to
|
245
|
-
avoid any inheritance for key amqp gem classes: Channel, Queue, Exchange. This will take a
|
246
|
-
|
251
|
+
avoid any inheritance for key amqp gem classes: Channel, Queue, Exchange, Consumer. This will take a significant refactoring effort and
|
252
|
+
we do not expect this to change at this time.
|
247
253
|
|
248
254
|
|
249
255
|
### How does amqp gem relate to amq-client gem, amq-protocol and libraries like bunny? ###
|
data/amqp.gemspec
CHANGED
@@ -23,8 +23,8 @@ Gem::Specification.new do |s|
|
|
23
23
|
|
24
24
|
# Dependencies
|
25
25
|
s.add_dependency "eventmachine"
|
26
|
-
s.add_dependency "amq-client", "
|
27
|
-
s.add_dependency "amq-protocol", "
|
26
|
+
s.add_dependency "amq-client", "~> 0.8.2"
|
27
|
+
s.add_dependency "amq-protocol", "~> 0.8.0"
|
28
28
|
|
29
29
|
begin
|
30
30
|
require "changelog"
|
data/bin/docup
ADDED
data/docs/08Migration.textile
CHANGED
@@ -8,6 +8,8 @@ h2. About this guide
|
|
8
8
|
This guide explains how (and why) applications that use amqp gem versions 0.6.x and 0.7.x should migrate
|
9
9
|
to 0.8.0.RCs and future versions. It also outlines deprecated features, when and why they will be removed.
|
10
10
|
|
11
|
+
This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/3.0/">Creative Commons Attribution 3.0 Unported License</a> (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
|
12
|
+
|
11
13
|
|
12
14
|
h2. Covered versions
|
13
15
|
|
@@ -129,24 +131,82 @@ MQ class and class methods that use implicit connection (MQ.queue and so on) wil
|
|
129
131
|
|
130
132
|
h2. MQ class is now AMQP::Channel
|
131
133
|
|
132
|
-
|
134
|
+
MQ class was renamed to AMQP::Channel to follow established {file:docs/AMQP091ModelExplained.textile AMQP 0.9.1 terminology}. Implicit per-thread
|
135
|
+
channels are deprecated so code like this:
|
136
|
+
|
137
|
+
<pre>
|
138
|
+
<code>
|
139
|
+
# amqp gem 0.6.x code style: connection is implicit, channel is implicit. Error handling & recovery are thus
|
140
|
+
# not possible.
|
141
|
+
# Deprecated, do not use.
|
142
|
+
MQ.queue("search.indexing")
|
143
|
+
|
144
|
+
# same for exchanges. Deprecated, do not use.
|
145
|
+
MQ.direct("services.imaging")
|
146
|
+
MQ.fanout("services.broadcast")
|
147
|
+
MQ.topic("services.weather_updates")
|
148
|
+
</code>
|
149
|
+
</pre>
|
150
|
+
|
151
|
+
<pre>
|
152
|
+
<code>
|
153
|
+
# connection object lets you define error handlers
|
154
|
+
connection = AMQP.connect(:vhost => "myapp/production", :host => "192.168.0.18")
|
155
|
+
# channek object lets you define error handlers and use multiple channels per application,
|
156
|
+
# for example, one per thread
|
157
|
+
channel = AMQP::Channel.new(connection)
|
158
|
+
# AMQP::Channel#queue has unchanged otherwise
|
159
|
+
channel.queue("search.indexing")
|
160
|
+
|
161
|
+
# exchanges examples
|
162
|
+
channel.direct("services.imaging")
|
163
|
+
channel.fanout("services.broadcast")
|
164
|
+
channel.topic("services.weather_updates")
|
165
|
+
</code>
|
166
|
+
</pre>
|
167
|
+
|
168
|
+
<span class="note">
|
169
|
+
MQ.queue, MQ.direct, MQ.fanout and MQ.topic methods will be removed before 1.0 release. Use
|
170
|
+
instance methods on AMQP::Channel (AMQP::Channel#queue, AMQP::Channel#direct, AMQP::Channel#default_exchange
|
171
|
+
and so on) instead.
|
172
|
+
</span>
|
133
173
|
|
134
174
|
|
135
175
|
h2. MQ::Queue is now AMQP::Queue
|
136
176
|
|
137
|
-
|
177
|
+
MQ::Queue is now AMQP::Queue. All the methods from 0.6.x series are still available.
|
178
|
+
|
179
|
+
<span class="note">
|
180
|
+
MQ::Queue alias will be removed before 1.0 release. Please switch to AMQP::Queue.
|
181
|
+
</span>
|
138
182
|
|
139
183
|
|
140
184
|
h2. MQ::Exchange is now AMQP::Exchange
|
141
185
|
|
142
|
-
|
186
|
+
MQ::Exchange is now AMQP::Exchange. All the methods from 0.6.x series are still available.
|
187
|
+
|
188
|
+
<span class="note">
|
189
|
+
MQ::Exchange alias will be removed before 1.0 release. Please switch to AMQP::Exchange.
|
190
|
+
</span>
|
191
|
+
|
143
192
|
|
144
193
|
|
145
194
|
h2. MQ::Header is now AMQP::Header
|
146
195
|
|
147
|
-
|
196
|
+
MQ::Header is now AMQP::Header. If you code has any type checks or case matches on
|
197
|
+
MQ::Header, it needs to change to AMQP::Header.
|
148
198
|
|
149
199
|
|
200
|
+
h2. AMQP.error is deprecated
|
201
|
+
|
202
|
+
Catch-all solutions for error handling are very difficult to use. Automatic recovery and fine-grained
|
203
|
+
event handling is also not possible. amqp gem 0.8.0.RC14 and later includes fine-grained {file:docs/ErrorHandling.textile Error Handling and Recovery API}
|
204
|
+
that is both significantly easier to use in real-world cases but also makes automatic recovery mode possible.
|
205
|
+
|
206
|
+
<span class="note">
|
207
|
+
AMQP.error method will be removed before 1.0 release.
|
208
|
+
</span>
|
209
|
+
|
150
210
|
|
151
211
|
h2. MQ::RPC is deprecated
|
152
212
|
|
@@ -210,7 +270,9 @@ AMQP::Protocol::* classes were removed in the the amqp gem 0.8.0 development cyc
|
|
210
270
|
|
211
271
|
|
212
272
|
|
213
|
-
h2.
|
273
|
+
h2. Authors
|
274
|
+
|
275
|
+
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
|
214
276
|
|
215
277
|
|
216
278
|
|
@@ -4,74 +4,92 @@ h1. AMQP 0.9.1 Model Explained
|
|
4
4
|
|
5
5
|
h2. About this guide
|
6
6
|
|
7
|
-
This guide explains AMQP 0.9.1 Model used by RabbitMQ. Understanding the AMQP Model will make a lot of other documentation, both for the Ruby amqp gem and
|
8
|
-
RabbitMQ itself, easier to follow.
|
7
|
+
This guide explains the AMQP 0.9.1 Model used by RabbitMQ. Understanding the AMQP Model will make a lot of other documentation, both for the Ruby amqp gem and
|
8
|
+
RabbitMQ itself, easier to follow. This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/3.0/">Creative Commons Attribution 3.0 Unported License</a> (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
|
9
9
|
|
10
|
-
This guide covers:
|
11
10
|
|
12
|
-
|
13
|
-
* Key differences of the AMQP model from some other messaging models.
|
14
|
-
* What are exchanges.
|
15
|
-
* What are queues.
|
16
|
-
* What are bindings.
|
17
|
-
* How AMQP protocol is structured. What are AMQP methods.
|
18
|
-
* What attributes AMQP 0.9.1 messages have.
|
19
|
-
* What are message acknowledgements.
|
20
|
-
* What are negative message acknowledgements.
|
21
|
-
* and a lot of other things.
|
11
|
+
h3. What this guide covers
|
22
12
|
|
13
|
+
This guide covers:
|
23
14
|
|
24
|
-
|
15
|
+
* High-level overview of the AMQP 0.9.1 Model
|
16
|
+
* Key differences of the AMQP model from some other messaging models
|
17
|
+
* What exchanges are
|
18
|
+
* What queues are
|
19
|
+
* What bindings are
|
20
|
+
* How AMQP protocol is structured and what AMQP methods are
|
21
|
+
* What attributes AMQP 0.9.1 messages have
|
22
|
+
* What message acknowledgements are
|
23
|
+
* What negative message acknowledgements are
|
24
|
+
* and a lot of other things
|
25
25
|
|
26
|
-
This guide covers v0.8.0 and later of the "Ruby amqp gem":https://github.com/ruby-amqp/amqp.
|
27
26
|
|
28
27
|
|
29
|
-
h2. High-level overview of
|
28
|
+
h2. High-level overview of AMQP 0.9.1 and the AMQP Model
|
30
29
|
|
31
|
-
h3.
|
30
|
+
h3. What is AMQP
|
32
31
|
|
33
32
|
AMQP (Advanced Message Queuing Protocol) is a networking protocol which enables conforming client applications to communicate with conforming
|
34
|
-
messaging middleware brokers.
|
33
|
+
messaging middleware brokers.
|
34
|
+
|
35
|
+
|
36
|
+
h3. Why AMQP was created
|
37
|
+
|
38
|
+
Messaging solutions have been around since the 1970s with a view to solving the problem of integrating incompatible products from diverse vendors. Without the use of messaging middleware, the integration of heterogenous systems has proved to be very expensive and complex. However, messaging solutions, such as IBM Websphere MQ and Tibco Enterprise Message Service, are also very costly and tend to be exclusively employed by large companies (who can afford them), especially those in the financial services industry.
|
39
|
+
|
40
|
+
There is also a problem with interoperability between messaging solutions. Vendors have created their own proprietary messaging protocols which do not interoperate with others, therefore resulting in 'vendor lock-in'.
|
41
|
+
|
42
|
+
AMQP has multiple design goals but two of the most important are:
|
43
|
+
|
44
|
+
* To produce an open standard for messaging middleware
|
45
|
+
* To enable interoperability between various technologies and platforms
|
46
|
+
|
47
|
+
There is a lot of software running on many operating systems built with multiple programming languages running on
|
48
|
+
various hardware architectures and virtual machines. AMQP not only makes it possible for these disparate systems to
|
49
|
+
communicate with one another, but also enables different products that implement AMQP to exchange information.
|
50
|
+
|
51
|
+
|
52
|
+
h3. Brokers and their role
|
53
|
+
|
54
|
+
Messaging brokers receive messages from _producers_ (applications that publish them) and route them to _consumers_ (applications
|
35
55
|
that process them).
|
36
56
|
|
37
|
-
|
57
|
+
If you imagine the human body, then brokers would be equivalent to centers of the nervous system and applications would be more like limbs.
|
38
58
|
|
39
59
|
|
40
|
-
h3. AMQP 0.9.1 Model
|
60
|
+
h3. AMQP 0.9.1 Model in brief
|
41
61
|
|
42
62
|
The AMQP 0.9.1 Model has the following view of the world: messages are published by producers to _exchanges_, often compared to post offices or mailboxes. Exchanges then
|
43
|
-
distribute message copies to _queues_ using rules called _bindings_. Then AMQP brokers either push messages to _consumers_ subscribed to
|
44
|
-
fetch/pull messages from queues on demand
|
63
|
+
distribute message copies to _queues_ using rules called _bindings_. Then AMQP brokers either push messages to _consumers_ subscribed to queues, or consumers
|
64
|
+
fetch/pull messages from queues on demand.
|
45
65
|
|
46
66
|
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/001_hello_world_example_routing.png!
|
47
67
|
|
48
|
-
When publishing a message, producers may specify various _message attributes_ (message metadata). Some of this metadata may be used by the broker,
|
49
|
-
is completely opaque and
|
68
|
+
When publishing a message, producers may specify various _message attributes_ (message metadata). Some of this metadata may be used by the broker, however, the rest of it
|
69
|
+
is completely opaque to the broker and is only used by applications that receive the message.
|
50
70
|
|
51
|
-
|
52
|
-
_message acknowledgements_: when a message is pushed down to a consumer, the consumer _notifies the broker_, either automatically or as soon as application
|
53
|
-
developer chooses to.
|
71
|
+
Networks are unreliable and applications may fail to process messages, therefore the AMQP Model has a notion of
|
72
|
+
_message acknowledgements_: when a message is pushed down to a consumer, the consumer _notifies the broker_, either automatically or as soon as the application
|
73
|
+
developer chooses to do so. When message acknowledgements are in use, a broker will only completely remove a message from a queue when it receives a notification for that message (or group of messages).
|
54
74
|
|
55
|
-
In certain situations, for example, when a message cannot be routed, messages may be _returned_ to producers
|
56
|
-
placed into a so-called "dead letter queue"
|
75
|
+
In certain situations, for example, when a message cannot be routed, messages may be _returned_ to producers, dropped, or, if the broker implements an extension,
|
76
|
+
placed into a so-called "dead letter queue". Producers choose how to handle situations like this by publishing messages using certain parameters.
|
57
77
|
|
58
78
|
Queues, exchanges and bindings are commonly referred to as _AMQP entities_.
|
59
79
|
|
60
80
|
|
61
81
|
h3. AMQP is a Programmable Protocol
|
62
82
|
|
63
|
-
AMQP 0.9.1 is a programmable protocol in the sense that AMQP entities
|
83
|
+
AMQP 0.9.1 is a programmable protocol in the sense that AMQP entities and routing schemes are defined by applications themselves, not a broker administrator. Accordingly, provision is made for protocol
|
64
84
|
operations that declare queues and exchanges, define bindings between them, subscribe to queues and so on.
|
65
85
|
|
66
|
-
This gives application developers a lot of freedom but also requires them to be aware of potential definition conflicts. In practice,
|
67
|
-
are rare and often indicate misconfigurations
|
68
|
-
|
69
|
-
Applications declare AMQP entities they need, define routing schemes necessary and may choose to delete AMQP entities when they are no longer used.
|
86
|
+
This gives application developers a lot of freedom but also requires them to be aware of potential definition conflicts. In practice, definition conflicts
|
87
|
+
are rare and often indicate misconfigurations. This can be very useful as it is a good thing if misconfigurations are caught early.
|
70
88
|
|
89
|
+
Applications declare the AMQP entities that they need, define necessary routing schemes and may choose to delete AMQP entities when they are no longer used.
|
71
90
|
|
72
91
|
|
73
|
-
|
74
|
-
h2. AMQP Exchanges and Exchange Types.
|
92
|
+
h2. AMQP Exchanges and Exchange Types
|
75
93
|
|
76
94
|
_Exchanges_ are AMQP entities where messages are sent. Exchanges then take a message and route it into one or more (or no) queues. The routing algorithm used
|
77
95
|
depends on _exchange type_ and rules called _bindings_. AMQP 0.9.1 brokers typically provide 4 exchange types out of the box:
|
@@ -96,44 +114,53 @@ Besides the type, exchanges have a number of attributes, most imprortant of whic
|
|
96
114
|
* Can have metadata associated with them on declaration
|
97
115
|
|
98
116
|
|
99
|
-
h2. AMQP Queues
|
117
|
+
h2. AMQP Queues
|
100
118
|
|
101
|
-
Queues in the AMQP Model are very similar to queues in other
|
102
|
-
applications. Like AMQP exchanges, AMQP
|
119
|
+
Queues in the AMQP Model are very similar to queues in other message and "task queueing" systems: they store messages that are consumed by
|
120
|
+
applications. Like AMQP exchanges, an AMQP queue has a name and a durability property but also
|
103
121
|
|
104
122
|
* Can be exclusive (used by only one connection)
|
105
123
|
* Can be automatically deleted when last consumer unsubscribes
|
106
|
-
* Can have metadata associated with them on declaration (some brokers use it to implement features like message TTL
|
107
|
-
|
124
|
+
* Can have metadata associated with them on declaration (some brokers use it to implement features like message TTL)
|
108
125
|
|
109
126
|
|
110
|
-
h2. AMQP Bindings
|
127
|
+
h2. AMQP Bindings
|
111
128
|
|
112
129
|
Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q,
|
113
130
|
Q has to _be bound_ to E. Bindings may have an optional _routing key_ attribute used by some exchange types. The purpose of the routing key is to
|
114
131
|
selectively match only specific (matching) messages published to an exchange to the bound queue. In other words, the routing key acts like a filter.
|
115
132
|
|
133
|
+
To draw an analogy:
|
116
134
|
|
135
|
+
* Queue is like your destination in New York city
|
136
|
+
* Exchange is like JFK airport
|
137
|
+
* Bindings are routes from JFK to your destination. There can be none or more than one way to reach it
|
117
138
|
|
118
|
-
|
139
|
+
Having this layer of indirection enables routing scenarios that are impossible of very hard to implement using publishing directly to queues and
|
140
|
+
also eliminates certain amount of duplicated work application developers have to do.
|
119
141
|
|
120
|
-
|
142
|
+
If AMQP message cannot be routed to any queue (for example, because there are no bindings for the exchange it was published to), it is either
|
143
|
+
dropped or returned to the publisher, depending on message attributes the publisher has set.
|
144
|
+
|
145
|
+
|
146
|
+
h2. AMQP Message Consumers
|
147
|
+
|
148
|
+
Storing messages in queues is useless unless applications can _consume_ them. In the AMQP 0.9.1 Model, there are two ways for applications to do this:
|
121
149
|
|
122
150
|
* Have messages pushed to them ("push API")
|
123
151
|
* Fetch messages as needed ("pull API")
|
124
152
|
|
125
153
|
With the "push API", applications have to indicate interest in consuming messages from a particular queue. When they do so, we say that they _register a consumer_
|
126
|
-
|
127
|
-
the queue).
|
128
|
-
|
129
|
-
Each consumer (subscription) has an identifier called _consumer tag_. It can be used to unsubscribe from messages. Consumer tags are just strings.
|
154
|
+
or, simply put, _subscribe to a queue_. It is possible to have more than one consumer per queue or to register an _exclusive consumer_ (excludes all other consumers from
|
155
|
+
the queue while it is consuming).
|
130
156
|
|
157
|
+
Each consumer (subscription) has an identifier called a _consumer tag_. It can be used to unsubscribe from messages. Consumer tags are just strings.
|
131
158
|
|
132
159
|
|
133
|
-
h2. AMQP Message Attributes and Payload
|
160
|
+
h2. AMQP Message Attributes and Payload
|
134
161
|
|
135
|
-
Messages in the AMQP Model have _attributes_. Some attributes are so common that AMQP
|
136
|
-
about exact attribute name. Some examples are
|
162
|
+
Messages in the AMQP Model have _attributes_. Some attributes are so common that the AMQP v0.9.1 specification defines them and application developers do not have to think
|
163
|
+
about the exact attribute name. Some examples are
|
137
164
|
|
138
165
|
* Content type
|
139
166
|
* Content encoding
|
@@ -144,36 +171,35 @@ about exact attribute name. Some examples are
|
|
144
171
|
* Expiration period
|
145
172
|
* Producer application id
|
146
173
|
|
147
|
-
Some attributes are used by AMQP brokers, but most are
|
148
|
-
as _headers_. They are similar to X-Headers in HTTP. Message attributes are set when message is published.
|
174
|
+
Some attributes are used by AMQP brokers, but most are open to interpretation by applications that receive them. Some attributes are optional and known
|
175
|
+
as _headers_. They are similar to X-Headers in HTTP. Message attributes are set when a message is published.
|
149
176
|
|
150
|
-
AMQP messages also have _payload_ (data they carry). Brokers treat this data as opaque (it is
|
151
|
-
and no
|
152
|
-
|
177
|
+
AMQP messages also have a _payload_ (the data that they carry). Brokers treat this data as opaque (it is neither modified nor used by them). It is possible for messages to contain only attributes
|
178
|
+
and no payload. It is common to use serialization formats like JSON, Thrift, Protocol Buffers and MessagePack
|
179
|
+
to serialize structured data
|
180
|
+
in order to publish it as AMQP message payload.
|
153
181
|
|
154
182
|
|
183
|
+
h2. AMQP Message Acknowledgements
|
155
184
|
|
156
|
-
|
185
|
+
Since networks are unreliable and applications fail, it is often necessary to have some kind of "processing acknowledgement". Sometimes it is only
|
186
|
+
necessary to acknowledge the fact that a message has been received. Sometimes acknowledgements mean that a message was validated and processed by a consumer,
|
187
|
+
for example, verified as having mandatory data and persisted to a data store or indexed.
|
157
188
|
|
158
|
-
|
159
|
-
|
160
|
-
|
161
|
-
|
162
|
-
Because this situation is so common, AMQP 0.9.1 has a built-in feature called _message acknowledgements_ (sometimes referred to as _acks_) that consumer
|
163
|
-
use to confirm message delivery and/or processing. If an application crashes (AMQP broker notice this when connection is closed), if acknowledgement for a
|
164
|
-
message was not received by AMQP broker, the message is re-queued (and possibly immediately delivered to another consumer, if any).
|
189
|
+
This situation is very common, so AMQP 0.9.1 has a built-in feature called _message acknowledgements_ (sometimes referred to as _acks_) that consumers
|
190
|
+
use to confirm message delivery and/or processing. If an application crashes (AMQP broker notices this when connection is closed), if an acknowledgement for a
|
191
|
+
message was expected but not received by the AMQP broker, the message is re-queued (and possibly immediately delivered to another consumer, if any exists).
|
165
192
|
|
166
193
|
Having acknowledgements built into the protocol helps developers to build more robust software.
|
167
194
|
|
168
195
|
|
169
|
-
|
170
196
|
h2. AMQP 0.9.1 Methods
|
171
197
|
|
172
198
|
AMQP 0.9.1 is structured as a number of _methods_. Methods are operations (like HTTP methods) and have nothing in common with methods in object-oriented programming
|
173
|
-
languages. AMQP methods are grouped into _classes_. Classes are just logical groupings of AMQP methods. "AMQP 0.9.1 reference":http://www.rabbitmq.com/amqp-0-9-1-reference.html can be found on
|
199
|
+
languages. AMQP methods are grouped into _classes_. Classes are just logical groupings of AMQP methods. The "AMQP 0.9.1 reference":http://www.rabbitmq.com/amqp-0-9-1-reference.html can be found on
|
174
200
|
the RabbitMQ website.
|
175
201
|
|
176
|
-
|
202
|
+
Let us take a look at the _exchange.*_ class, a group of methods related to operations on exchanges. It includes the following operations:
|
177
203
|
|
178
204
|
|
179
205
|
* exchange.declare
|
@@ -181,26 +207,26 @@ Lets take a look at the _exchange.*_ class, a group of methods related to operat
|
|
181
207
|
* exchange.delete
|
182
208
|
* exchange.delete-ok
|
183
209
|
|
184
|
-
(note that RabbitMQ site reference also includes RabbitMQ-specific extensions to the exchange.* class that we
|
210
|
+
(note that the RabbitMQ site reference also includes RabbitMQ-specific extensions to the exchange.* class that we will not discuss in this guide).
|
185
211
|
|
186
212
|
The operations above form logical pairs: *exchange.declare* and *exchange.declare-ok*,
|
187
213
|
*exchange.delete* and *exchange.delete-ok*. These operations are "requests" (sent by clients) and "responses" (sent by
|
188
|
-
brokers in response to aforementioned "requests").
|
214
|
+
brokers in response to the aforementioned "requests").
|
189
215
|
|
190
|
-
|
216
|
+
As an example, the client asks the broker to declare a new exchange using the *exchange.declare* method:
|
191
217
|
|
192
218
|
!https://img.skitch.com/20110720-c4qjdhmdrih9bn56npqnic4die.jpg!
|
193
219
|
|
194
|
-
As
|
220
|
+
As shown on the diagram above, *exchange.declare* carries several _parameters_. They enable the client to specify exchange name,
|
195
221
|
type, durability flag and so on.
|
196
222
|
|
197
|
-
|
223
|
+
If the operation succeeds, the broker responds with the *exchange.declare-ok* method:
|
198
224
|
|
199
225
|
!https://img.skitch.com/20110720-m4ptjbnex2sa52g6wdwj3e9ahm.jpg!
|
200
226
|
|
201
|
-
*exchange.declare-ok* does not carry any parameters except for the channel number (
|
227
|
+
*exchange.declare-ok* does not carry any parameters except for the channel number (channels will be described later in this guide).
|
202
228
|
|
203
|
-
|
229
|
+
The sequence of events is very similar for another method pair, *queue.declare* and *queue.declare-ok*:
|
204
230
|
|
205
231
|
!https://img.skitch.com/20110720-tmxswrie71ubb5m5nh8n17idhk.jpg!
|
206
232
|
|
@@ -210,38 +236,33 @@ Not all AMQP methods have counterparts. Some (*basic.publish* being the most wid
|
|
210
236
|
and some others (*basic.get*, for example) have more than one possible "response".
|
211
237
|
|
212
238
|
|
213
|
-
|
214
|
-
h2. AMQP Connections.
|
239
|
+
h2. AMQP Connections
|
215
240
|
|
216
241
|
AMQP connections are typically long-lived. AMQP is an application level protocol that uses TCP for reliable delivery. AMQP connections use
|
217
|
-
authentication and can be protected using TLS (SSL). When application no longer needs to be connected to AMQP broker, it should gracefully
|
218
|
-
close AMQP connection instead of abruptly closing the underlying TCP connection.
|
242
|
+
authentication and can be protected using TLS (SSL). When an application no longer needs to be connected to an AMQP broker, it should gracefully
|
243
|
+
close the AMQP connection instead of abruptly closing the underlying TCP connection.
|
219
244
|
|
220
245
|
|
246
|
+
h2. AMQP Channels
|
221
247
|
|
222
|
-
|
223
|
-
|
224
|
-
Some applications need multiple connections to AMQP broker. It is, however, undesirable to keep many TCP connections open at the same time
|
248
|
+
Some applications need multiple connections to an AMQP broker. However, it is undesirable to keep many TCP connections open at the same time
|
225
249
|
because doing so consumes system resources and makes it more difficult to configure firewalls. AMQP 0.9.1 connections are multiplexed with
|
226
250
|
_channels_ that can be thought of as "lightweight connections that share a single TCP connection".
|
227
251
|
|
228
|
-
For applications that use multiple threads/processes/etc for processing, it is very common to open a new channel per thread (process, etc)
|
252
|
+
For applications that use multiple threads/processes/etc for processing, it is very common to open a new channel per thread (process, etc.)
|
229
253
|
and *not share* channels between them.
|
230
254
|
|
231
|
-
|
232
|
-
use to figure out
|
233
|
-
|
255
|
+
Communication on a particular channel is completely separate from communication on another channel, therefore every AMQP method also carries a channel number that clients
|
256
|
+
use to figure out which channel the method is for (and thus, which event handler needs to be invoked, for example).
|
234
257
|
|
235
258
|
|
259
|
+
h2. AMQP Virtual Hosts (vhosts)
|
236
260
|
|
237
|
-
|
238
|
-
|
239
|
-
To make it possible for a single broker to host multiple isolated "environments" (groups of users, exchanges, queues and so on), AMQP includes concept
|
261
|
+
To make it possible for a single broker to host multiple isolated "environments" (groups of users, exchanges, queues and so on), AMQP includes the concept
|
240
262
|
of _virtual hosts_ (vhosts). They are similar to virtual hosts used by many popular Web servers and provide completely isolated environments
|
241
263
|
in which AMQP entities live. AMQP clients specify what vhosts they want to use during AMQP connection negotiation.
|
242
264
|
|
243
|
-
AMQP 0.9.1 vhost can be any non-blank string.
|
244
|
-
|
265
|
+
An AMQP 0.9.1 vhost name can be any non-blank string.
|
245
266
|
|
246
267
|
|
247
268
|
h2. AMQP is Extensible
|
@@ -249,32 +270,43 @@ h2. AMQP is Extensible
|
|
249
270
|
AMQP 0.9.1 has several extension points:
|
250
271
|
|
251
272
|
* Custom exchange types let developers implement routing schemes that exchange types provided out-of-the-box do not cover well, for example, geodata-based routing.
|
252
|
-
* Declaration of exchanges and queues can include additional attributes broker can use. For example, per-queue message TTL in RabbitMQ is implemented this way.
|
273
|
+
* Declaration of exchanges and queues can include additional attributes that the broker can use. For example, per-queue message TTL in RabbitMQ is implemented this way.
|
253
274
|
* Broker-specific extensions to the protocol. See, for example, "extensions RabbitMQ implements":http://www.rabbitmq.com/extensions.html.
|
254
275
|
* New AMQP 0.9.1 method classes can be introduced.
|
276
|
+
* Brokers can be extended with additional plugins, for example, RabbitMQ management frontend and HTTP API are implemented as a plugin.
|
255
277
|
|
256
|
-
These features make
|
257
|
-
|
258
|
-
|
278
|
+
These features make the AMQP 0.9.1 Model even more flexible and applicable to a very broad range of problems.
|
259
279
|
|
260
280
|
|
261
281
|
h2. Key differences from some other messaging models
|
262
282
|
|
263
|
-
|
283
|
+
One key difference to understand about the AMQP 0.9.1 Model is that *messages are not sent to queues. They are sent to exchanges that route them to
|
264
284
|
queues according to rules called "bindings"*. This means that routing is primarily handled by AMQP brokers and not applications themselves.
|
265
285
|
|
266
286
|
TBD
|
267
287
|
|
268
288
|
|
289
|
+
h2. AMQP 0.9.1 clients ecosystem
|
269
290
|
|
270
|
-
|
291
|
+
h3. Overview
|
271
292
|
|
272
|
-
|
273
|
-
|
293
|
+
There are many AMQP 0.9.1 clients for many popular programming languages and platforms. Some of them follow AMQP terminology closely
|
294
|
+
and only provide implementation of AMQP methods. Some others have additional features, convenience methods and abstractions. Some of the
|
295
|
+
clients are asynchronous (non-blocking), some are synchronous (blocking), some support both models. Some clients support vendor-specific
|
296
|
+
extensions (for example, RabbitMQ-specific extensions).
|
297
|
+
|
298
|
+
Because one of the main AMQP goals is interoperability, it is a good idea for developers to understand protocol operations
|
299
|
+
and not limit themselves to terminology of a particular client library. This way communicating with developers using different libraries will
|
300
|
+
be significantly easier.
|
274
301
|
|
275
|
-
To stay up to date with amqp gem development, "follow @rubyamqp on Twitter":http://twitter.com/rubyamqp and "join our mailing list":http://groups.google.com/group/ruby-amqp.
|
276
302
|
|
277
303
|
|
304
|
+
h2. Wrapping up
|
305
|
+
|
306
|
+
This is the end of the AMQP 0.9.1 Model tutorial. Congratulations! Armed with this knowledge, you will find it easier to follow the rest of
|
307
|
+
the amqp gem documentation as well as the rabbitmq.com documentation and the "rabbitmq-discuss mailing list":http://groups.google.com/group/rabbitmq-discuss
|
308
|
+
|
309
|
+
To stay up to date with amqp gem development, "follow @rubyamqp on Twitter":http://twitter.com/rubyamqp and "join our mailing list":http://groups.google.com/group/ruby-amqp.
|
278
310
|
|
279
311
|
|
280
312
|
h2. What to read next
|
@@ -295,13 +327,18 @@ If you are migrating your application from earlier versions of the amqp gem (0.6
|
|
295
327
|
{file:docs/08Migration.textile amqp gem 0.8 migration guide}.
|
296
328
|
|
297
329
|
|
330
|
+
h2. Authors
|
331
|
+
|
332
|
+
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
|
333
|
+
|
334
|
+
|
298
335
|
h2. Tell us what you think!
|
299
336
|
|
300
337
|
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
|
301
|
-
|
338
|
+
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
|
302
339
|
key to making the documentation better.
|
303
340
|
|
304
|
-
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:
|
341
|
+
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michaelklishin@me.com?subject=amqp%20gem%20documentation
|
305
342
|
|
306
343
|
|
307
344
|
<div id="disqus_thread"></div>
|