amqp 0.8.0.rc13 → 0.8.0.rc14
Sign up to get free protection for your applications and to get access to all the features.
- data/.rspec +2 -1
- data/.travis.yml +8 -2
- data/.yardopts +1 -0
- data/CHANGELOG +9 -0
- data/Gemfile +17 -11
- data/README.md +26 -16
- data/amqp.gemspec +2 -2
- data/bin/ci/before_build.sh +21 -0
- data/docs/08Migration.textile +199 -5
- data/docs/AMQP091ModelExplained.textile +322 -0
- data/docs/Bindings.textile +24 -4
- data/docs/Clustering.textile +1 -1
- data/docs/ConnectingToTheBroker.textile +98 -82
- data/docs/ConnectionEncryptionWithTLS.textile +65 -5
- data/docs/DocumentationGuidesIndex.textile +93 -13
- data/docs/Durability.textile +1 -1
- data/docs/ErrorHandling.textile +458 -94
- data/docs/Exchanges.textile +901 -87
- data/docs/GettingStarted.textile +278 -143
- data/docs/PatternsAndUseCases.textile +420 -0
- data/docs/Queues.textile +730 -178
- data/docs/RabbitMQVersions.textile +18 -3
- data/docs/RunningTests.textile +1 -1
- data/docs/TestingWithEventedSpec.textile +121 -0
- data/docs/Troubleshooting.textile +15 -1
- data/docs/VendorSpecificExtensions.textile +1 -1
- data/docs/diagrams/001_hello_world_example_routing.png +0 -0
- data/docs/diagrams/002_blabbr_example_routing.png +0 -0
- data/docs/diagrams/003_weathr_example_routing.png +0 -0
- data/docs/diagrams/004_fanout_exchange.png +0 -0
- data/docs/diagrams/005_direct_exchange.png +0 -0
- data/docs/diagrams/redhat/direct_exchange.png +0 -0
- data/docs/diagrams/redhat/fanout_exchange.png +0 -0
- data/docs/diagrams/redhat/topic_exchange.png +0 -0
- data/examples/error_handling/automatic_recovery_of_channel_and_queues.rb +50 -0
- data/examples/error_handling/automatically_recovering_hello_world_consumer.rb +51 -0
- data/examples/error_handling/automatically_recovering_hello_world_consumer_that_uses_a_server_named_queue.rb +51 -0
- data/examples/error_handling/basic_connection_failover.rb +22 -0
- data/examples/error_handling/channel_level_exception.rb +9 -2
- data/examples/error_handling/connection_level_exception.rb +8 -1
- data/examples/error_handling/connection_level_exception_with_objects.rb +49 -0
- data/examples/error_handling/connection_loss_handler.rb +1 -5
- data/examples/error_handling/hello_world_producer.rb +43 -0
- data/examples/error_handling/insufficient_permissions.rb +54 -0
- data/examples/error_handling/manual_connection_and_channel_recovery.rb +71 -0
- data/examples/error_handling/queue_exclusivity_violation.rb +41 -0
- data/examples/error_handling/queue_name_violation.rb +31 -0
- data/examples/exchanges/autodeletion_of_exchanges.rb +1 -4
- data/examples/guides/queues/01a_declaring_a_server_named_queue_using_queue_constructor.rb +7 -8
- data/examples/guides/queues/01b_declaring_a_queue_using_queue_constructor.rb +7 -8
- data/examples/guides/queues/02a_declaring_a_durable_shared_queue.rb +5 -8
- data/examples/guides/queues/02b_declaring_a_durable_shared_queue.rb +5 -8
- data/examples/guides/queues/03a_declaring_a_temporary_exclusive_queue.rb +7 -8
- data/examples/guides/queues/04_bind_a_queue_using_exchange_instance.rb +9 -10
- data/examples/guides/queues/05_bind_a_queue_using_exchange_name.rb +8 -10
- data/examples/guides/queues/06_subscribe_to_receive_messages.rb +10 -12
- data/examples/guides/queues/07_fetch_a_message_from_the_queue.rb +14 -14
- data/examples/guides/queues/08_unsubscribing_a_consumer.rb +13 -16
- data/examples/guides/queues/09_unbinding_from_exchange.rb +16 -22
- data/examples/guides/queues/10_purge_a_queue.rb +13 -18
- data/examples/guides/queues/11_deleting_a_queue.rb +14 -19
- data/examples/guides/queues/12_objects_that_consume_messages.rb +69 -0
- data/examples/guides/queues/13_objects_that_consume_messages_take_two.rb +89 -0
- data/examples/hello_world.rb +1 -3
- data/examples/hello_world_with_an_empty_string.rb +5 -6
- data/examples/inspecting_server_information.rb +45 -0
- data/examples/issues/issue_93.rb +23 -0
- data/examples/issues/issue_94.rb +23 -0
- data/examples/patterns/command/consumer.rb +45 -0
- data/examples/patterns/command/producer.rb +26 -0
- data/examples/patterns/request_reply/client.rb +29 -0
- data/examples/patterns/request_reply/server.rb +26 -0
- data/examples/publishing/publishing_a_one_off_message.rb +6 -4
- data/examples/publishing/returned_messages.rb +2 -10
- data/examples/queues/accessing_message_metadata.rb +15 -13
- data/examples/queues/queue_status.rb +12 -15
- data/examples/routing/fanout_routing.rb +33 -0
- data/examples/routing/headers_routing.rb +17 -15
- data/examples/routing/round_robin_with_direct_exchange.rb +39 -0
- data/examples/routing/round_robin_with_the_default_exchange.rb +38 -0
- data/examples/routing/unroutable_mandatory_message_is_returned.rb +33 -0
- data/examples/routing/weather_updates.rb +15 -20
- data/examples/tls/using_tls.rb +41 -0
- data/lib/amqp/bit_set.rb +80 -0
- data/lib/amqp/broker.rb +72 -0
- data/lib/amqp/channel.rb +93 -13
- data/lib/amqp/client.rb +11 -22
- data/lib/amqp/compatibility/ruby187_patchlevel_check.rb +2 -0
- data/lib/amqp/connection.rb +2 -3
- data/lib/amqp/consumer.rb +208 -0
- data/lib/amqp/deprecated/fork.rb +2 -0
- data/lib/amqp/deprecated/mq.rb +2 -0
- data/lib/amqp/exchange.rb +6 -4
- data/lib/amqp/extensions/rabbitmq.rb +3 -1
- data/lib/amqp/header.rb +76 -14
- data/lib/amqp/int_allocator.rb +96 -0
- data/lib/amqp/logger.rb +2 -0
- data/lib/amqp/queue.rb +242 -86
- data/lib/amqp/rpc.rb +2 -0
- data/lib/amqp/session.rb +169 -9
- data/lib/amqp/utilities/event_loop_helper.rb +2 -0
- data/lib/amqp/utilities/server_type.rb +2 -0
- data/lib/amqp/version.rb +2 -2
- data/lib/mq.rb +4 -2
- data/lib/mq/logger.rb +3 -1
- data/lib/mq/rpc.rb +3 -1
- data/spec/integration/authentication_spec.rb +17 -10
- data/spec/integration/automatic_binding_for_default_direct_exchange_spec.rb +1 -1
- data/spec/integration/automatic_recovery_predicate_spec.rb +68 -0
- data/spec/integration/basic_get_spec.rb +2 -1
- data/spec/integration/{extensions/basic_return_spec.rb → basic_return_spec.rb} +2 -1
- data/spec/integration/channel_level_exception_handling_spec.rb +53 -0
- data/spec/integration/connection_level_exception_handling_spec.rb +49 -0
- data/spec/integration/declare_and_immediately_bind_a_server_named_queue_spec.rb +38 -17
- data/spec/integration/declare_one_hundred_server_named_queues_spec.rb +44 -0
- data/spec/integration/direct_exchange_routing_spec.rb +125 -0
- data/spec/integration/exchange_declaration_spec.rb +75 -46
- data/spec/integration/extensions/rabbitmq/publisher_confirmations_spec.rb +180 -0
- data/spec/integration/{workload_distribution_spec.rb → fanout_exchange_routing_spec.rb} +10 -9
- data/spec/integration/headers_exchange_routing_spec.rb +269 -0
- data/spec/integration/hello_world_spec.rb +77 -0
- data/spec/integration/immediate_messages_spec.rb +59 -0
- data/spec/integration/mandatory_messages_spec.rb +52 -0
- data/spec/integration/message_metadata_access_spec.rb +106 -0
- data/spec/integration/multiple_consumers_per_queue_spec.rb +319 -0
- data/spec/integration/ordering_of_published_messages_spec.rb +96 -0
- data/spec/integration/queue_declaration_spec.rb +8 -8
- data/spec/integration/queue_status_spec.rb +66 -0
- data/spec/integration/recovery/per_channel_automatic_recovery_on_graceful_broker_shutdown_spec.rb +76 -0
- data/spec/integration/recovery/per_channel_automatic_recovery_spec.rb +72 -0
- data/spec/integration/redelivery_of_unacknowledged_messages_spec.rb +96 -0
- data/spec/integration/regressions/concurrent_publishing_on_the_same_channel_spec.rb +91 -0
- data/spec/integration/regressions/empty_message_body_spec.rb +56 -0
- data/spec/integration/regressions/issue66_spec.rb +2 -1
- data/spec/integration/reply_queue_communication_spec.rb +2 -1
- data/spec/integration/store_and_forward_spec.rb +4 -3
- data/spec/integration/topic_subscription_spec.rb +2 -1
- data/spec/integration/tx_commit_spec.rb +124 -0
- data/spec/integration/tx_rollback_spec.rb +167 -0
- data/spec/spec_helper.rb +44 -71
- data/spec/unit/amqp/bit_set_spec.rb +127 -0
- data/spec/unit/amqp/channel_id_allocation_spec.rb +40 -0
- data/spec/unit/amqp/connection_spec.rb +4 -2
- data/spec/unit/amqp/int_allocator_spec.rb +116 -0
- metadata +92 -26
- data/CONTRIBUTORS +0 -29
- data/docs/Routing.textile +0 -30
- data/examples/real-world/task-queue/README.textile +0 -3
- data/examples/real-world/task-queue/consumer.rb +0 -27
- data/examples/real-world/task-queue/producer.rb +0 -22
- data/spec/unit/amqp/basic_spec.rb +0 -39
- data/tasks.rb +0 -4
data/docs/Exchanges.textile
CHANGED
@@ -5,245 +5,1059 @@ h1. Working with exchanges
|
|
5
5
|
|
6
6
|
h2. About this guide
|
7
7
|
|
8
|
-
This guide covers
|
9
|
-
amqp gem.
|
8
|
+
This guide covers the use of exchanges according to the AMQP v0.9.1 specification including message publishing,
|
9
|
+
common usage scenarios and how to accomplish typical operations using the Ruby amqp gem.
|
10
10
|
|
11
11
|
|
12
|
-
h2.
|
12
|
+
h2. Which versions of the amqp gem does this guide cover?
|
13
13
|
|
14
|
-
This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp
|
14
|
+
This guide covers v0.8.0 and later of the "Ruby amqp gem":http://github.com/ruby-amqp/amqp.
|
15
15
|
|
16
16
|
|
17
|
-
h2. Exchanges in AMQP
|
17
|
+
h2. Exchanges in AMQP v0.9.1 - overview
|
18
18
|
|
19
19
|
h3. What are AMQP exchanges?
|
20
20
|
|
21
|
-
An
|
22
|
-
world. Unlike some other messaging middleware products and protocols, in AMQP
|
23
|
-
are published to exchanges that route them to queue(s)
|
21
|
+
An _exchange_ accepts messages from a producer application and routes them to message queues. They can be thought of
|
22
|
+
as the "mailboxes" of the AMQP world. Unlike some other messaging middleware products and protocols, in AMQP,
|
23
|
+
messages are *not* published directly to queues. Messages are published to exchanges that route them to queue(s)
|
24
|
+
using pre-arranged criteria called _bindings_.
|
24
25
|
|
25
|
-
There are multiple exchange types in AMQP
|
26
|
-
sophisticated routing scenarios (
|
26
|
+
There are multiple exchange types in the AMQP v0.9.1 specification, each with its own routing semantics. Custom
|
27
|
+
exchange types can be created to deal with sophisticated routing scenarios (e.g. routing based on geolocation data
|
28
|
+
or edge cases) or just for convenience.
|
27
29
|
|
28
30
|
|
29
31
|
h3. Concept of bindings
|
30
32
|
|
31
|
-
|
32
|
-
Learn more about bindings in {file:docs/Bindings.textile Bindings guide}.
|
33
|
+
A _binding_ is an association between a queue and an exchange. A queue must be bound to at least one exchange in
|
34
|
+
order to receive messages from publishers. Learn more about bindings in the {file:docs/Bindings.textile Bindings guide}.
|
33
35
|
|
34
36
|
|
35
|
-
h3.
|
37
|
+
h3. Exchange attributes
|
36
38
|
|
37
39
|
Exchanges have several attributes associated with them:
|
38
40
|
|
39
41
|
* Name
|
40
42
|
* Type (direct, fanout, topic, headers or some custom type)
|
41
43
|
* Durability
|
42
|
-
* Whether exchange is auto-deleted when no longer used
|
43
|
-
* Other metadata (
|
44
|
+
* Whether the exchange is auto-deleted when no longer used
|
45
|
+
* Other metadata (sometimes known as _X-arguments_)
|
44
46
|
|
45
47
|
|
46
48
|
h2. Exchange types
|
47
49
|
|
48
|
-
There are 4 built-in exchange types in AMQP
|
50
|
+
There are 4 built-in exchange types in AMQP v0.9.1:
|
49
51
|
|
50
52
|
* Direct
|
51
53
|
* Fanout
|
52
54
|
* Topic
|
53
55
|
* Headers
|
54
56
|
|
55
|
-
As stated previously, each exchange type has own routing semantics and new exchange types can be added by
|
56
|
-
Custom exchange types begin with x
|
57
|
+
As stated previously, each exchange type has its own routing semantics and new exchange types can be added by
|
58
|
+
extending brokers with plugins. Custom exchange types begin with "x-", much like custom HTTP headers, e.g.
|
59
|
+
"x-recent-history exchange":https://github.com/videlalvaro/rabbitmq-recent-history-exchange or
|
60
|
+
"x-random exchange":https://github.com/jbrisbin/random-exchange.
|
57
61
|
|
58
62
|
|
63
|
+
h2. Message attributes
|
59
64
|
|
65
|
+
Before we start looking at various exchange types and their routing semantics, we need to introduce message
|
66
|
+
attributes. Every AMQP message has a number of *attributes*. Some attributes are important and used very often,
|
67
|
+
others are rarely used. AMQP message attributes are metadata and are similar in purpose to HTTP request and
|
68
|
+
response headers.
|
60
69
|
|
61
|
-
|
70
|
+
Every AMQP v0.9.1 message has an attribute called *routing key*. The routing key is an "address" that the exchange
|
71
|
+
may use to decide how to route the message . This is similar to, but more generic than, a URL in HTTP. Most exchange
|
72
|
+
types use the routing key to implement routing logic, but some ignore it and use other criteria (e.g. message
|
73
|
+
content).
|
62
74
|
|
63
|
-
Before we start looking at various exchange types and their routing semantics, we need to introduce idea of message attributes.
|
64
|
-
Every AMQP message has a number of *attributes*. Some attributes are important and used very often, other are used rarely. AMQP message attributes are metadata:
|
65
|
-
they are similar in purpose to HTTP request and response headers.
|
66
75
|
|
67
|
-
|
68
|
-
the message (similar to, but more generic, than URL in HTTP). Most exchange types use routing key to implement routing logic, but some ignore
|
69
|
-
it and use other criteria (for example, message content).
|
76
|
+
h2. Fanout exchanges
|
70
77
|
|
78
|
+
h3. How fanout exchanges route messages
|
71
79
|
|
80
|
+
A fanout exchange routes messages to all of the queues that are bound to it and the routing key is ignored. If N
|
81
|
+
queues are bound to a fanout exchange, when a new message is published to that exchange a *copy of the message* is
|
82
|
+
delivered to all N queues. Fanout exchanges are ideal for the
|
83
|
+
"broadcast routing":http://en.wikipedia.org/wiki/Broadcasting_%28computing%29 of messages.
|
72
84
|
|
85
|
+
Graphically this can be represented as:
|
73
86
|
|
74
|
-
|
87
|
+
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/004_fanout_exchange.png!
|
88
|
+
|
89
|
+
|
90
|
+
h3. Declaring a fanout exchange
|
91
|
+
|
92
|
+
There are two ways to declare a fanout exchange:
|
93
|
+
|
94
|
+
* By instantiating an {AMQP::Exchange} and specifying the type as ":fanout"
|
95
|
+
* By using the {AMQP::Channel#fanout} method
|
96
|
+
|
97
|
+
Here are two examples to demonstrate:
|
98
|
+
|
99
|
+
<pre>
|
100
|
+
<code>
|
101
|
+
exchange = AMQP::Exchange.new(channel, :fanout, "nodes.metadata")
|
102
|
+
</code>
|
103
|
+
</pre>
|
104
|
+
|
105
|
+
<pre>
|
106
|
+
<code>
|
107
|
+
exchange = channel.fanout("nodes.metadata")
|
108
|
+
</code>
|
109
|
+
</pre>
|
110
|
+
|
111
|
+
Both methods asynchronously declare a queue. Because the declaration necessitates a network round-trip, publishing
|
112
|
+
operations on {AMQP::Exchange} instances are delayed until the broker reply (`exchange.declare-ok`) is received.
|
113
|
+
|
114
|
+
Also, both methods let you pass a block to run a piece of code when the broker responds with an `exchange.declare-ok`
|
115
|
+
(meaning that the exchange has been successfully declared).
|
116
|
+
|
117
|
+
<pre>
|
118
|
+
<code>
|
119
|
+
channel.fanout("nodes.metadata") do |exchange|
|
120
|
+
# exchange is declared and ready to be used.
|
121
|
+
end
|
122
|
+
</code>
|
123
|
+
</pre>
|
124
|
+
|
125
|
+
h3. Fanout routing example
|
126
|
+
|
127
|
+
To demonstrate fanout routing behavior we can declare 10 server-named exclusive queues, bind them all to one
|
128
|
+
fanout exchange and then publish a message to the exchange:
|
75
129
|
|
76
|
-
|
77
|
-
|
78
|
-
|
130
|
+
<pre>
|
131
|
+
<code>
|
132
|
+
exchange = channel.topic("amqpgem.examples.routing.fanout_routing", :auto_delete => true)
|
79
133
|
|
80
|
-
|
134
|
+
10.times do
|
135
|
+
q = channel.queue("", :exclusive => true, :auto_delete => true).bind(exchange)
|
136
|
+
q.subscribe do |payload|
|
137
|
+
puts "Queue #{q.name} received #{payload}"
|
138
|
+
end
|
139
|
+
end
|
140
|
+
</code>
|
141
|
+
</pre>
|
81
142
|
|
82
|
-
|
143
|
+
<pre>
|
144
|
+
<code>
|
145
|
+
# Publish some test data after all queues are declared and bound
|
146
|
+
EventMachine.add_timer(1.2) { exchange.publish "Hello, fanout exchanges world!" }
|
147
|
+
</code>
|
148
|
+
</pre>
|
83
149
|
|
84
|
-
|
85
|
-
This means that applications can rely on that exchange to always be available to them. Each vhost has a separate instance of that exchange,
|
86
|
-
it is *not shared across vhosts* for obvious reasons.
|
150
|
+
When run, this example produces the following output:
|
87
151
|
|
152
|
+
<pre>
|
153
|
+
Queue amq.gen-0p/BjxGNCue42RcJhpUrdg== received Hello, fanout exchanges world!
|
154
|
+
Queue amq.gen-3GXULvZuYh1KsOD83yvlNg== received Hello, fanout exchanges world!
|
155
|
+
Queue amq.gen-4EcyydTfoZzXjNSSLsh09Q== received Hello, fanout exchanges world!
|
156
|
+
Queue amq.gen-B1isyTpR5svB6ClQ2TQEBQ== received Hello, fanout exchanges world!
|
157
|
+
Queue amq.gen-FwLLioB7Mk4LGA4yJ1Mo7A== received Hello, fanout exchanges world!
|
158
|
+
Queue amq.gen-OtBQokiA/DmNkB5bPzaRig== received Hello, fanout exchanges world!
|
159
|
+
Queue amq.gen-RYHQUrj3yihb0DRF7KVpRg== received Hello, fanout exchanges world!
|
160
|
+
Queue amq.gen-SZJ40mGwbhdcbOGeHMhUkg== received Hello, fanout exchanges world!
|
161
|
+
Queue amq.gen-sDeVZg9Vx1knq+n9EMi8tA== received Hello, fanout exchanges world!
|
162
|
+
Queue amq.gen-uWOuVaosW4bWAHqKG6pZVw== received Hello, fanout exchanges world!
|
163
|
+
</pre>
|
164
|
+
|
165
|
+
Each of the queues bound to the exchange receives a *copy* of the message.
|
166
|
+
|
167
|
+
|
168
|
+
Full example:
|
169
|
+
<script src="https://gist.github.com/1020181.js"> </script>
|
170
|
+
|
171
|
+
|
172
|
+
h3. Fanout use cases
|
173
|
+
|
174
|
+
Because a fanout exchange delivers a copy of a message to every queue bound to it, its use cases are quite similar:
|
175
|
+
|
176
|
+
* Massively multiplayer online (MMO) games can use it for leaderboard updates or other global events
|
177
|
+
* Sport news sites can use fanout exchanges for distributing score updates to mobile clients in near real-time
|
178
|
+
* Distributed systems can broadcast various state and configuration updates
|
179
|
+
* Group chats can distribute messages between participants using a fanout exchange (although AMQP does not have a built-in concept of presence, so "XMPP":http://xmpp.org may be a better choice)
|
180
|
+
|
181
|
+
|
182
|
+
h3. Pre-declared fanout exchanges
|
183
|
+
|
184
|
+
AMQP v0.9.1 brokers must implement a fanout exchange type and pre-declare one instance with the name of "amq.fanout".
|
185
|
+
Applications can rely on that exchange always being available to them. Each vhost has a separate instance of that exchange, it is *not shared across vhosts* for obvious reasons.
|
88
186
|
|
89
187
|
|
90
188
|
h2. Direct exchanges
|
91
189
|
|
92
|
-
|
190
|
+
h3. How direct exchanges route messages
|
191
|
+
|
192
|
+
A direct exchange delivers messages to queues based on a *message routing key*, an attribute that every AMQP v0.9.1
|
193
|
+
message contains.
|
93
194
|
|
94
195
|
Here is how it works:
|
95
196
|
|
96
|
-
* A queue binds to the exchange with a routing key
|
97
|
-
* When a new message with routing key R arrives
|
197
|
+
* A queue binds to the exchange with a routing key K
|
198
|
+
* When a new message with routing key R arrives at the direct exchange, the exchange routes it to the queue if K = R
|
98
199
|
|
99
|
-
|
200
|
+
A direct exchange is ideal for the "unicast routing":http://en.wikipedia.org/wiki/Unicast of messages (although they
|
201
|
+
can be used for "multicast routing":http://en.wikipedia.org/wiki/Multicast as well).
|
100
202
|
|
101
|
-
|
203
|
+
Here is a graphical representation:
|
102
204
|
|
103
|
-
|
205
|
+
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/005_direct_exchange.png!
|
104
206
|
|
105
|
-
* *amq.direct*.
|
106
|
-
* *""* (unnamed, referred to as an empty string by many clients including amqp Ruby gem) exchange known as *default exchange*.
|
107
207
|
|
108
|
-
|
109
|
-
|
208
|
+
h3. Declaring a direct exchange
|
209
|
+
|
210
|
+
There are two ways to declare a direct exchange:
|
211
|
+
|
212
|
+
* By instantiating a {AMQP::Exchange} and specifying its type as ":direct"
|
213
|
+
* By using the {AMQP::Channel#direct} method
|
214
|
+
|
215
|
+
Here are two examples to demonstrate:
|
216
|
+
|
217
|
+
<pre>
|
218
|
+
<code>
|
219
|
+
exchange = AMQP::Exchange.new(channel, :direct, "nodes.metadata")
|
220
|
+
</code>
|
221
|
+
</pre>
|
222
|
+
|
223
|
+
<pre>
|
224
|
+
<code>
|
225
|
+
exchange = channel.direct("nodes.metadata")
|
226
|
+
</code>
|
227
|
+
</pre>
|
228
|
+
|
229
|
+
Both methods asynchronously declare a queue. Because the declaration necessitates a network round trip, publishing
|
230
|
+
operations on {AMQP::Exchange} instances are delayed until a broker reply (`exchange.declare-ok`) is received.
|
231
|
+
|
232
|
+
Also, both methods let you pass a block to run a piece of code when the broker responds with `exchange.declare-ok`
|
233
|
+
(meaning that the exchange has been successfully declared).
|
234
|
+
|
235
|
+
<pre>
|
236
|
+
<code>
|
237
|
+
channel.direct("pages.content.extraction") do |exchange|
|
238
|
+
# exchange is declared and ready to be used.
|
239
|
+
end
|
240
|
+
</code>
|
241
|
+
</pre>
|
242
|
+
|
243
|
+
|
244
|
+
h3. Direct routing example
|
245
|
+
|
246
|
+
Since direct exchanges use the *message routing key* for routing, message producers need to specify it:
|
247
|
+
|
248
|
+
<pre>
|
249
|
+
<code>
|
250
|
+
exchange.publish("Hello, direct exchanges world!", :routing_key => "amqpgem.examples.queues.shared")
|
251
|
+
</code>
|
252
|
+
</pre>
|
253
|
+
|
254
|
+
The routing key will then be compared for equality with routing keys on bindings, and consumers that subscribed with
|
255
|
+
the same routing key each get a copy of the message:
|
256
|
+
|
257
|
+
Full example:
|
258
|
+
<script src="https://gist.github.com/1041787.js"> </script>
|
259
|
+
|
260
|
+
|
261
|
+
h3. Direct exchanges and load balancing of messages
|
262
|
+
|
263
|
+
Direct exchanges are often used to distribute tasks between multiple workers (instances of the same application) in
|
264
|
+
a round robin manner. When doing so, it is important to understand that, in AMQP v0.9.1, *messages are load balanced
|
265
|
+
between consumers and not between queues*.
|
266
|
+
|
267
|
+
The Ruby amqp gem historically has a limitation that only one consumer (message handler) is allowed per {AMQP::Queue}
|
268
|
+
instance, however, this limitation will be addressed in the future. With the amqp gem v0.8.x, if you want to load
|
269
|
+
balance messages between multiple consumers in the same application/OS process, then you need to use a separate
|
270
|
+
channel for each of the consumers.
|
271
|
+
|
272
|
+
The {file:docs/Queues.textile Working With Queues} and {file:docs/PatternsAndUseCases.textile Patterns and Use Cases}
|
273
|
+
guides provide more information on this subject.
|
274
|
+
|
275
|
+
|
276
|
+
h3. Pre-declared direct exchanges
|
277
|
+
|
278
|
+
AMQP v0.9.1 brokers must implement a direct exchange type and pre-declare two instances:
|
279
|
+
|
280
|
+
* *amq.direct*
|
281
|
+
* *""* exchange known as *default exchange* (unnamed, referred to as an empty string by many clients including amqp Ruby gem)
|
282
|
+
|
283
|
+
Applications can rely on those exchanges always being available to them. Each vhost has separate instances of those
|
284
|
+
exchanges, they are *not shared across vhosts* for obvious reasons.
|
110
285
|
|
111
286
|
|
112
287
|
h3. Default exchange
|
113
288
|
|
114
|
-
|
115
|
-
property that makes it very useful for simple applications
|
289
|
+
The default exchange is a direct exchange with no name (the amqp gem refers to it using an empty string) pre-declared
|
290
|
+
by the broker. It has one special property that makes it very useful for simple applications, namely that *every
|
291
|
+
queue is automatically bound to it with a routing key which is the same as the queue name*.
|
292
|
+
|
293
|
+
For example, when you declare a queue with the name of "search.indexing.online", the AMQP broker will bind it to the
|
294
|
+
default exchange using "search.indexing.online" as the routing key. Therefore a message published to the default
|
295
|
+
exchange with routing key = "search.indexing.online" will be routed to the queue "search.indexing.online". In other
|
296
|
+
words, the default exchange makes it *seem like it is possible to deliver messages directly to queues*, even though
|
297
|
+
that is not technically what is happening.
|
298
|
+
|
299
|
+
|
300
|
+
The amqp gem offers two ways of obtaining a reference to the default exchange:
|
301
|
+
|
302
|
+
* Using the {AMQP::Channel#default_exchange} method
|
303
|
+
* Using the {AMQP::Channel#direct} method with an empty string as the exchange name
|
304
|
+
|
305
|
+
{AMQP::Exchange#initialize} can also be used, but requires more coding effort and it offers no benefits over instance
|
306
|
+
methods on {AMQP::Channel} in this particular case.
|
307
|
+
|
308
|
+
Some examples of usage:
|
309
|
+
|
310
|
+
<pre>
|
311
|
+
<code>
|
312
|
+
exchange = AMQP::Exchange.new(channel, :direct, "")
|
313
|
+
</code>
|
314
|
+
</pre>
|
315
|
+
|
316
|
+
<pre>
|
317
|
+
<code>
|
318
|
+
exchange = channel.default_exchange
|
319
|
+
</code>
|
320
|
+
</pre>
|
321
|
+
|
322
|
+
<pre>
|
323
|
+
<code>
|
324
|
+
exchange = channel.direct("")
|
325
|
+
</code>
|
326
|
+
</pre>
|
327
|
+
|
116
328
|
|
117
|
-
|
118
|
-
|
119
|
-
will be routed to the queue "search.indexing.online". In other words, default exchange makes it possible to "deliver messages directly to queues",
|
120
|
-
even though that is not technically correct.
|
329
|
+
The default exchange is used by the "Hello, World" example:
|
330
|
+
<script src="https://gist.github.com/998691.js"> </script>
|
121
331
|
|
332
|
+
Additionally, the routing example above can be rewritten to use the default exchange:
|
333
|
+
<script src="https://gist.github.com/1041778.js"> </script>
|
122
334
|
|
123
|
-
amqp gem offers two ways of obtaining of the default exchange: {AMQP::Channel#default_exchange} and {AMQP::Channel#direct} (use empty string as exchange name).
|
124
|
-
{AMQP::Exchange#initialize} can be used as well but requires more effort and for this particular case offers no benefits over instance methods on {AMQP::Channel}.
|
125
335
|
|
126
|
-
|
336
|
+
h3. Direct exchange use cases
|
127
337
|
|
128
|
-
|
338
|
+
Direct exchanges can be used in a wide variety of cases:
|
129
339
|
|
340
|
+
* Direct (near real-time) messages to individual players in an MMO game
|
341
|
+
* Delivering notifications to specific geographic locations (for example, points of sale)
|
342
|
+
* Distributing tasks between multiple instances of the same application all having the same function, for example, image processors
|
343
|
+
* Passing data between workflow steps, each having an identifier (also consider using headers exchange)
|
344
|
+
* Delivering notifications to individual software services in the network
|
130
345
|
|
131
346
|
|
132
347
|
h2. Topic exchanges
|
133
348
|
|
134
|
-
|
135
|
-
queue to the exchange.
|
349
|
+
h3. How topic exchanges route messages
|
136
350
|
|
351
|
+
Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern
|
352
|
+
that was used to bind a queue to an exchange. The topic exchange type is often used to implement various
|
353
|
+
"publish/subscribe pattern":http://en.wikipedia.org/wiki/Publish/subscribe variations.
|
137
354
|
|
138
|
-
Topic exchanges are commonly used for "multicast routing":http://en.wikipedia.org/wiki/Multicast of messages.
|
355
|
+
Topic exchanges are commonly used for the "multicast routing":http://en.wikipedia.org/wiki/Multicast of messages.
|
139
356
|
|
140
357
|
!http://upload.wikimedia.org/wikipedia/commons/thumb/3/30/Multicast.svg/500px-Multicast.svg.png!
|
141
358
|
|
142
|
-
Topic exchanges can be used for "broadcast routing":http://en.wikipedia.org/wiki/Broadcasting_%28computing%29 but
|
359
|
+
Topic exchanges can be used for "broadcast routing":http://en.wikipedia.org/wiki/Broadcasting_%28computing%29, but
|
360
|
+
fanout exchanges are usually more efficient for this use case.
|
143
361
|
|
144
362
|
|
363
|
+
h3. Topic exchange routing example
|
145
364
|
|
146
|
-
|
365
|
+
Two classic examples of topic-based routing are stock price updates and location-specific data (for instance, weather
|
366
|
+
broadcasts). Consumers indicate which topics they are interested in (think of it like subscribing to a feed for an
|
367
|
+
individual tag of your favourite blog as opposed to the full feed). The routing is enabled by specifying a _routing
|
368
|
+
pattern_ to the {AMQP::Queue#bind} method, for example:
|
147
369
|
|
148
|
-
|
370
|
+
<pre>
|
371
|
+
<code>
|
372
|
+
channel.queue("americas.south").bind(exchange, :routing_key => "americas.south.#").subscribe do |headers, payload|
|
373
|
+
puts "An update for South America: #{payload}, routing key is #{headers.routing_key}"
|
374
|
+
end
|
375
|
+
</code>
|
376
|
+
</pre>
|
149
377
|
|
150
|
-
|
378
|
+
In the example above we bind a queue with the name of "americas.south" to the topic exchange declared earlier using
|
379
|
+
the {AMQP::Queue#bind} method. This means that only messages with a routing key matching "americas.south.#" will be
|
380
|
+
routed to the "americas.south" queue.
|
151
381
|
|
382
|
+
A routing pattern consists of several words separated by dots, in a similar way to URI path segments being joined by
|
383
|
+
slash. A few of examples:
|
152
384
|
|
385
|
+
* asia.southeast.thailand.bangkok
|
386
|
+
* sports.basketball
|
387
|
+
* usa.nasdaq.aapl
|
388
|
+
* tasks.search.indexing.accounts
|
153
389
|
|
154
|
-
|
390
|
+
The following routing keys match the "americas.south.#" pattern:
|
155
391
|
|
392
|
+
* americas.south
|
393
|
+
* americas.south.*brazil*
|
394
|
+
* americas.south.*brazil.saopaolo*
|
395
|
+
* americas.south.*chile.santiago*
|
156
396
|
|
397
|
+
In other words, the "#" part of the pattern matches 0 or more words.
|
157
398
|
|
158
|
-
|
399
|
+
For the pattern "americas.south.*", some matching routing keys are:
|
159
400
|
|
401
|
+
* americas.south.*brazil*
|
402
|
+
* americas.south.*chile*
|
403
|
+
* americas.south.*peru*
|
160
404
|
|
405
|
+
but not
|
161
406
|
|
162
|
-
|
407
|
+
* americas.south
|
408
|
+
* americas.south.chile.santiago
|
163
409
|
|
164
|
-
|
410
|
+
As you can see, the "*" part of the pattern matches 1 word only.
|
165
411
|
|
166
412
|
|
413
|
+
Full example:
|
414
|
+
<script src="https://gist.github.com/1020308.js"> </script>
|
167
415
|
|
168
|
-
h2. Declaring/Instantiating exchanges
|
169
416
|
|
417
|
+
h3. Topic exchange use cases
|
170
418
|
|
419
|
+
Topic exchanges have a very broad set of use cases. Whenever a problem involves multiple consumers/applications that
|
420
|
+
selectively choose which type of messages they want to receive, the use of topic exchanges should be considered.
|
421
|
+
To name a few examples:
|
171
422
|
|
172
|
-
|
423
|
+
* Distributing data relevant to specific geographic location, for example, points of sale
|
424
|
+
* Background task processing done by multiple workers, each capable of handling specific set of tasks
|
425
|
+
* Stocks price updates (and updates on other kinds of financial data)
|
426
|
+
* News updates that involve categorization or tagging (for example, only for a particular sport or team)
|
427
|
+
* Orchestration of services of different kinds in the cloud
|
428
|
+
* Distributed architecture/OS-specific software builds or packaging where each builder can handle only one architecture or OS
|
173
429
|
|
174
|
-
{include:file:examples/exchanges/declare_an_exchange_without_assignment.rb}
|
175
430
|
|
431
|
+
h2. Declaring/Instantiating exchanges
|
176
432
|
|
177
|
-
|
433
|
+
With the Ruby amqp gem, exchanges can be declared in two ways:
|
178
434
|
|
435
|
+
* By using the {AMQP::Exchange#initialize} method that takes an optional callback
|
436
|
+
* By using a number of convenience methods on {AMQP::Channel} instances:
|
437
|
+
|
438
|
+
** {AMQP::Channel#direct}
|
439
|
+
** {AMQP::Channel#default_exchange}
|
440
|
+
** {AMQP::Channel#topic}
|
441
|
+
** {AMQP::Channel#fanout}
|
442
|
+
** {AMQP::Channel#headers}
|
443
|
+
|
444
|
+
The previous sections on specific exchange types (direct, fanout, headers, etc.) provide plenty of examples of how
|
445
|
+
these methods can be used.
|
179
446
|
|
180
447
|
|
181
448
|
h2. Publishing messages
|
182
449
|
|
450
|
+
To publish a message to an AMQP exchange, use {AMQP::Exchange#publish}:
|
451
|
+
|
452
|
+
<pre>
|
453
|
+
<code>
|
454
|
+
exchange.publish("Some payload")
|
455
|
+
</code>
|
456
|
+
</pre>
|
457
|
+
|
458
|
+
|
459
|
+
{AMQP::Exchange#publish} can accept any object that responds to the `to_s` method, not just string instances:
|
460
|
+
<script src="https://gist.github.com/1020336.js"> </script>
|
461
|
+
|
462
|
+
The message payload is completely opaque to the library and is not modified in any way.
|
463
|
+
|
464
|
+
|
465
|
+
h3. Data serialization
|
466
|
+
|
467
|
+
You are encouraged to take care of data serialization before publishing (i.e. by using JSON, Thrift, Protocol Buffers
|
468
|
+
or some other serialization library). Note that because AMQP is a binary protocol, text formats like JSON largely
|
469
|
+
lose their advantage of being easy to inspect as data travels across the network, so consider using
|
470
|
+
"BSON":http://bsonspec.org instead.
|
471
|
+
|
472
|
+
A few popular options for data serialization are:
|
473
|
+
|
474
|
+
* JSON: "json gem":https://rubygems.org/gems/json (part of standard Ruby library on Ruby 1.9) or "yajl-ruby":https://rubygems.org/gems/yajl-ruby (Ruby bindings to YAJL)
|
475
|
+
* BSON: "bson gem":https://rubygems.org/gems/bson for JRuby (implemented as a Java extension) and "bson_ext":https://rubygems.org/bson_ext (C extension) for C-based Rubies
|
476
|
+
* "Message Pack":http://msgpack.org has Ruby bindings but currently does not provide Java implementation for JRuby
|
477
|
+
* XML: "Nokogiri":https://nokogiri.org is a swiss army knife for XML processing with Ruby, built on top of libxml2
|
478
|
+
* Protocol Buffers: "beefcake":https://github.com/bmizerany/beefcake
|
479
|
+
* Thrift: "thrift-client":https://github.com/fauna/thrift_client
|
480
|
+
|
481
|
+
|
482
|
+
h3. Message metadata
|
483
|
+
|
484
|
+
AMQP messages have various metadata attributes that can be set when a message is published. Some of the attributes are
|
485
|
+
well-known and mentioned in the AMQP v0.9.1 specification, others are specific to a particular application.
|
486
|
+
Well-known attributes are listed here as options that {AMQP::Exchange#publish} takes:
|
487
|
+
|
488
|
+
* :routing_key
|
489
|
+
* :persistent
|
490
|
+
* :immediate
|
491
|
+
* :mandatory
|
492
|
+
* :content_type
|
493
|
+
* :content_encoding
|
494
|
+
* :priority
|
495
|
+
* :message_id
|
496
|
+
* :correlation_id
|
497
|
+
* :reply_to
|
498
|
+
* :type
|
499
|
+
* :user_id
|
500
|
+
* :app_id
|
501
|
+
* :timestamp
|
502
|
+
* :expiration
|
503
|
+
|
504
|
+
All other attributes can be added to a _headers table_ (in Ruby parlance, headers hash) that {AMQP::Exchange#publish}
|
505
|
+
accepts as the ":headers" argument.
|
506
|
+
|
507
|
+
An example to show how message metadata attributes are passed to {AMQP::Exchange#publish}:
|
508
|
+
<script src="https://gist.github.com/1020424.js"> </script>
|
509
|
+
|
510
|
+
<dl>
|
511
|
+
<dt>:routing_key</dt>
|
512
|
+
<dd>Used for routing messages depending on the exchange type and configuration.</dd>
|
513
|
+
|
514
|
+
<dt>:persistent</dt>
|
515
|
+
<dd>When set to true, AMQP broker will persist message to disk.</dd>
|
516
|
+
|
517
|
+
<dt>:immediate</dt>
|
518
|
+
<dd>
|
519
|
+
This flag tells the server how to react if the message cannot be routed to a queue consumer immediately. If this flag is set to true, the server will return
|
520
|
+
an undeliverable message to the producer with a basic.return AMQP method. If this flag is set to false, the server will queue the message, but with no guarantee
|
521
|
+
that it will ever be consumed.
|
522
|
+
</dd>
|
523
|
+
|
524
|
+
<dt>:mandatory</dt>
|
525
|
+
<dd>
|
526
|
+
This flag tells the server how to react if the message cannot be routed to a queue. If this flag is set to true, the server will return an unroutable message
|
527
|
+
to the producer with a basic.return AMQP method. If this flag is set to false, the server silently drops the message.
|
528
|
+
</dd>
|
529
|
+
|
530
|
+
<dt>:content_type</dt>
|
531
|
+
<dd>MIME content type of message payload. Has the same purpose/semantics as HTTP Content-Type header.</dd>
|
532
|
+
|
533
|
+
<dt>:content_encoding</dt>
|
534
|
+
<dd>MIME content encoding of message payload. Has the same purpose/semantics as HTTP Content-Encoding header.</dd>
|
535
|
+
|
536
|
+
<dt>:priority</dt>
|
537
|
+
<dd>Message priority, from 0 to 9.</dd>
|
538
|
+
|
539
|
+
<dt>:message_id</dt>
|
540
|
+
<dd>
|
541
|
+
Message identifier as a string. If applications need to identify messages, it is recommended that they use this attribute instead of putting it
|
542
|
+
into the message payload.
|
543
|
+
</dd>
|
544
|
+
|
545
|
+
<dt>:reply_to</dt>
|
546
|
+
<dd>
|
547
|
+
Commonly used to name a reply queue (or any other identifier that helps a consumer application to direct its response).
|
548
|
+
Applications are encouraged to use this attribute instead of putting this information into the message payload.
|
549
|
+
</dd>
|
550
|
+
|
551
|
+
<dt>:correlation_id</dt>
|
552
|
+
<dd>
|
553
|
+
ID of the message that this message is a reply to. Applications are encouraged to use this attribute instead of putting this information
|
554
|
+
into the message payload.
|
555
|
+
</dd>
|
556
|
+
|
557
|
+
<dt>:type</dt>
|
558
|
+
<dd>Message type as a string. Recommended to be used by applications instead of including this information into the message payload.</dd>
|
559
|
+
|
560
|
+
<dt>:user_id</dt>
|
561
|
+
<dd>
|
562
|
+
Sender's identifier. Note that RabbitMQ will check that the "value of this attribute is the same as username AMQP connection was authenticated with":http://www.rabbitmq.com/extensions.html#validated-user-id, it SHOULD NOT be used to transfer, for example, other application user ids or be used as a basis for some kind of Single Sign-On solution.
|
563
|
+
</dd>
|
564
|
+
|
565
|
+
<dt>:app_id</dt>
|
566
|
+
<dd>Application identifier string, for example, "eventoverse" or "webcrawler"</dd>
|
567
|
+
|
568
|
+
<dt>:timestamp</dt>
|
569
|
+
<dd>Timestamp of the moment when message was sent, in seconds since the Epoch</dd>
|
570
|
+
|
571
|
+
<dt>:expiration</dt>
|
572
|
+
<dd>Message expiration specification as a string</dd>
|
573
|
+
|
574
|
+
<dt>:headers</dt>
|
575
|
+
<dd>Ruby hash of any additional attributes that the application needs. Nested hashes are supported.</dd>
|
576
|
+
</dl>
|
577
|
+
|
578
|
+
It is recommended that application authors use well-known message attributes when applicable instead of relying on custom headers or placing
|
579
|
+
information into the message body. For example, if your application messages have priority, publishing timestamp, type and content type,
|
580
|
+
you should use the respective AMQP message attributes instead of reinventing the wheel.
|
581
|
+
|
582
|
+
|
583
|
+
h3. Validated user_id
|
584
|
+
|
585
|
+
In some scenarios it is useful for consumers to be able to know the identity of the user who published a message. RabbitMQ implements a feature known as
|
586
|
+
"validated User ID":http://www.rabbitmq.com/extensions.html#validated-user-id. If this property is set by a publisher, its value must be the same as the name of the user used to open the connection.
|
587
|
+
If the user-id property is not set, the publisher's identity is not validated and remains private.
|
588
|
+
|
589
|
+
|
590
|
+
h3. Publishing callback and reliable delivery in distributed environments
|
591
|
+
|
592
|
+
Sometimes it is convenient to execute an operation after publishing a message. For this, {AMQP::Exchange#publish} provides an optional
|
593
|
+
callback. It is important to clear up some expectations of when exactly it is run and how it is related to topics of message persistence,
|
594
|
+
delivery reliability and so on.
|
595
|
+
|
596
|
+
<pre>
|
597
|
+
<code>
|
598
|
+
exchange.publish(payload, :persistent => true, :type => "reports.done") do
|
599
|
+
# ...
|
600
|
+
end
|
601
|
+
</code>
|
602
|
+
</pre>
|
603
|
+
|
604
|
+
A common expectation of the code above is that it is run after the message "has been sent", or even "has been delivered". Unfortunately, neither of
|
605
|
+
these expectations can be met by the Ruby amqp gem alone. Message publishing happens in several steps:
|
606
|
+
|
607
|
+
* {AMQP::Exchange#publish} takes a message and various metadata attributes
|
608
|
+
* {AMQP::Exchange#publish} internally calls #to_s on the message argument to get message payload
|
609
|
+
* Resulting payload is staged for writing
|
610
|
+
* On the next event loop tick, data is transferred to the OS kernel using one of the underlying system calls ("epoll":http://en.wikipedia.org/wiki/Epoll, "kqueue":http://en.wikipedia.org/wiki/Kqueue and so on)
|
611
|
+
or NIO channels (in the case of JRuby)
|
612
|
+
* OS kernel buffers data before sending it
|
613
|
+
* Network driver may also employ buffering
|
614
|
+
|
615
|
+
<span class="note">
|
616
|
+
As you can see, "when data is sent" is a complicated issue and while methods to flush buffers certainly exist on various platforms, doing so in
|
617
|
+
a cross-platform way that *includes the JVM* (that EventMachine also runs on) is non-trivial. In addition, even flushing buffers does not guarantee that the
|
618
|
+
data was received by the broker because it might have crashed while data was travelling down the wire. The only way to reliably know whether data was
|
619
|
+
received by the broker or a peer application is to use message acknowledgements. This is how TCP works and this approach is proven to work at the
|
620
|
+
enormous scale of the modern Internet. AMQP (the protocol) fully embraces this fact and the amqp gem follows.
|
621
|
+
</span>
|
622
|
+
|
623
|
+
Given all of this, you may ask "when does the {AMQP::Exchange#publish} callback fire?" The answer is on the next event loop tick. By then the data is pushed down to
|
624
|
+
the OS kernel. As far as the Ruby library is concerned, it is reasonably safe behavior.
|
625
|
+
|
626
|
+
<span class="note">
|
627
|
+
The AMQP::Exchange#publish callback is fired on the next event loop tick. Data is staged for delivery immediately. Applications MUST NOT assume that
|
628
|
+
by the time the callback has fired, the data is guaranteed to leave the local machine networking stack, reach the AMQP broker or any peer applications that the
|
629
|
+
message needs to be routed to.
|
630
|
+
</span>
|
631
|
+
|
632
|
+
In cases when you cannot afford to lose a single message, AMQP v0.9.1 applications can use one (or a combination of) the following protocol features:
|
633
|
+
|
634
|
+
* Publisher confirms (a RabbitMQ-specific extension to AMQP v0.9.1)
|
635
|
+
* Publishing messages as immediate and/or mandatory
|
636
|
+
* Transactions (these introduce noticeable overhead and have a relatively narrow set of use cases)
|
637
|
+
|
638
|
+
A more detailed overview of the pros and cons of each option can be found in a "blog post that introduces Publisher Confirms extension":http://bit.ly/rabbitmq-publisher-confirms by the RabbitMQ team.
|
639
|
+
The next sections of this guide will describe how the features above can be used with the Ruby amqp gem.
|
640
|
+
|
641
|
+
|
642
|
+
h3. Publishing messages as immediate
|
643
|
+
|
644
|
+
When publishing messages, it is possible to use the ":immediate" option to publish a message as "immediate". When an immediate message cannot be
|
645
|
+
delivered to any consumer (meaning that one or more queues to which the message was routed have no active consumers), then the message is returned to the
|
646
|
+
producer.
|
647
|
+
|
648
|
+
An example of {AMQP::Exchange#publish} being used to publish an immediate message:
|
183
649
|
|
184
|
-
|
650
|
+
<pre>
|
651
|
+
<code>
|
652
|
+
exchange.publish("Message ##{i}", :immediate => true)
|
653
|
+
</code>
|
654
|
+
</pre>
|
185
655
|
|
186
|
-
The following example
|
656
|
+
The following code example demonstrates a message that is published as immediate but cannot be immediately consumed (no consumers) and thus is returned
|
657
|
+
back to the producer:
|
658
|
+
<script src="https://gist.github.com/1025700.js"></script>
|
659
|
+
|
660
|
+
|
661
|
+
h3. Publishing messages as mandatory
|
662
|
+
|
663
|
+
When publishing messages, it is possible to use the ":mandatory" option to publish a message as "mandatory". When a mandatory message cannot be
|
664
|
+
*routed* to any queue (for example, there are no bindings or none of the bindings match), the message is returned to the producer.
|
665
|
+
|
666
|
+
The following code example demonstrates a message that is published as mandatory but cannot be routed (no bindings) and thus is returned
|
667
|
+
back to the producer:
|
668
|
+
<script src="https://gist.github.com/1041835.js"> </script>
|
669
|
+
|
670
|
+
|
671
|
+
h3. Returned messages
|
672
|
+
|
673
|
+
When a message is returned, the application that produced it can handle that message in different ways:
|
674
|
+
|
675
|
+
* Store it for later redelivery in a persistent store
|
676
|
+
* Publish it to a different destination
|
677
|
+
* Log the event and discard the message
|
678
|
+
|
679
|
+
Returned messages contain information about the exchange they were published to. For convenience, the amqp gem associates returned message callbacks with {AMQP::Exchange}
|
680
|
+
instances. To handle returned messages, use {AMQP::Exchange#on_return}:
|
681
|
+
|
682
|
+
<pre>
|
683
|
+
<code>
|
684
|
+
exchange.on_return do |basic_return, metadata, payload|
|
685
|
+
puts "#{payload} was returned! reply_code = #{basic_return.reply_code}, reply_text = #{basic_return.reply_text}"
|
686
|
+
end
|
687
|
+
</code>
|
688
|
+
</pre>
|
689
|
+
|
690
|
+
A returned message handler has access to AMQP method (basic.return) information, message metadata and payload. The metadata and message body are returned without modifications so that the application can store the message for later redelivery.
|
691
|
+
|
692
|
+
|
693
|
+
h3. Publishing persistent messages
|
694
|
+
|
695
|
+
Messages potentially spend some time in the queues to which they were routed before they are consumed. During this period of time, the broker may crash or
|
696
|
+
experience a restart. To survive it, messages must be persisted to disk. This has a negative effect on performance, especially with network attached
|
697
|
+
storage like NAS devices and Amazon EBS. AMQP v0.9.1 lets applications trade off performance for durability, or vice versa, on a message-by-message basis.
|
698
|
+
|
699
|
+
To publish a persistent message, use the ":persistent" option that {AMQP::Exchange#publish} accepts:
|
700
|
+
|
701
|
+
<pre>
|
702
|
+
<code>
|
703
|
+
exchange.publish(payload, :persistent => true)
|
704
|
+
</code>
|
705
|
+
</pre>
|
706
|
+
|
707
|
+
<span class="note">
|
708
|
+
Note that in order to survive a broker crash, both the message and the queue that it was routed to must be persistent/durable.
|
709
|
+
</span>
|
710
|
+
|
711
|
+
{file:docs/Durability.textile Durability and Message Persistence} provides more information on the subject.
|
712
|
+
|
713
|
+
|
714
|
+
h3. Publishing In Multi-threaded Environments
|
715
|
+
|
716
|
+
When using amqp gem in multi-threaded environments, the rule of thumb is: avoid sharing {AMQP::Channel} instances
|
717
|
+
across threads.
|
718
|
+
|
719
|
+
Starting with 0.8.0.RC14, {AMQP::Exchange#publish} synchronizes data delivery on the channel object associated
|
720
|
+
with exchange. This protects application developers from the most common problems related to publishing messages on a
|
721
|
+
shared channel from multiple threads, however, by no means protects from every possible concurrency hazard.
|
722
|
+
|
723
|
+
<span class="note">
|
724
|
+
When using amqp gem in multi-threaded environments, the rule of thumb is: avoid sharing {AMQP::Channel} instances
|
725
|
+
across threads.
|
726
|
+
</span>
|
727
|
+
|
728
|
+
|
729
|
+
|
730
|
+
h3. Sending one-off messages
|
731
|
+
|
732
|
+
The following example publishes a message and *safely* closes the AMQP connection afterwards by passing a block
|
187
733
|
to {AMQP::Exchange#publish}:
|
188
734
|
|
189
|
-
|
735
|
+
<script src="https://gist.github.com/1020425.js"> </script>
|
736
|
+
|
737
|
+
|
738
|
+
h2. Headers exchanges
|
739
|
+
|
740
|
+
Now that message attributes and publishing have been introduced, it is time to take a look at one more core exchange type in AMQP v0.9.1. It is called the
|
741
|
+
_headers_ exchange type and is quite powerful.
|
742
|
+
|
743
|
+
h3. How headers exchanges route messages
|
744
|
+
|
745
|
+
h4. An example problem definition
|
746
|
+
|
747
|
+
The best way to explain headers-based routing is with an example. Imagine a distributed "continuous integration":http://martinfowler.com/articles/continuousIntegration.html system
|
748
|
+
that distributes builds across multiple machines with different hardware architectures (x86, IA-64, AMD64, ARM family and so on) and operating systems.
|
749
|
+
It strives to provide a way for a community to contribute machines to run tests on and a nice build matrix like "the one WebKit uses":http://build.webkit.org/waterfall?category=core.
|
750
|
+
One key problem such systems face is build distribution. It would be nice if a messaging broker could figure out
|
751
|
+
which machine has which OS, architecture or combination of the two and route build request messages accordingly.
|
752
|
+
|
753
|
+
A headers exchange is designed to help in situations like this by routing on multiple attributes that are more easily expressed as
|
754
|
+
message metadata attributes (headers) rather than a routing key string.
|
755
|
+
|
756
|
+
h4. Routing on multiple message attributes
|
757
|
+
|
758
|
+
Headers exchanges route messages based on message header matching. Headers exchanges ignore the routing key attribute. Instead, the attributes used for routing
|
759
|
+
are taken from the "headers" attribute. When a queue is bound to a headers exchange, the ":arguments" attribute is used to define matching rules:
|
760
|
+
|
761
|
+
<pre>
|
762
|
+
<code>
|
763
|
+
# when binding to a headers exchange, :arguments parameter is used to specify matching rules
|
764
|
+
@channel.queue("", :auto_delete => true).bind(exchange, :arguments => { :os => 'linux' })
|
765
|
+
</code>
|
766
|
+
</pre>
|
767
|
+
|
768
|
+
When matching on one header, a message is considered matching if the value of the header equals the value specified upon binding. Using the example above,
|
769
|
+
some messages that match would be:
|
770
|
+
|
771
|
+
<pre>
|
772
|
+
<code>
|
773
|
+
exchange.publish "For linux/IA64", :headers => { :arch => "IA64", :os => 'linux' }
|
774
|
+
exchange.publish "For linux/x86", :headers => { :arch => "x86", :os => 'linux' }
|
775
|
+
exchange.publish "For any linux", :headers => { :os => 'linux' }
|
776
|
+
</code>
|
777
|
+
</pre>
|
778
|
+
|
779
|
+
The following example demonstrates matching on integer values:
|
780
|
+
|
781
|
+
<pre>
|
782
|
+
<code>
|
783
|
+
# consumer part
|
784
|
+
@channel.queue("", :auto_delete => true).bind(exchange, :arguments => { :cores => 8 })
|
785
|
+
|
786
|
+
# ...
|
787
|
+
|
788
|
+
# producer part
|
789
|
+
exchange.publish "For ocotocore", :headers => { :cores => 8 }
|
790
|
+
</code>
|
791
|
+
</pre>
|
792
|
+
|
793
|
+
Matching on hashes (in AMQP v0.9.1 parlance - _attribute tables_) is also supported:
|
794
|
+
|
795
|
+
<pre>
|
796
|
+
<code>
|
797
|
+
# consumer part
|
798
|
+
channel.queue("", :auto_delete => true).bind(exchange, :arguments => { :package => { :name => 'riak', :version => '0.14.2' } })
|
799
|
+
|
800
|
+
# ...
|
801
|
+
|
802
|
+
# producer part
|
803
|
+
exchange.publish "For nodes with Riak 0.14.2", :headers => { :package => { :name => 'riak', :version => '0.14.2' } }
|
804
|
+
</code>
|
805
|
+
</pre>
|
806
|
+
|
807
|
+
h4. Matching all vs matching one
|
808
|
+
|
809
|
+
It is possible to bind a queue to a headers exchange using more than one header for matching. In this case, the broker needs one more piece of
|
810
|
+
information from the application developer, namely, should it consider messages with any of the headers matching, or all of them? This is what the "x-match"
|
811
|
+
binding argument is for:
|
812
|
+
|
813
|
+
<pre>
|
814
|
+
<code>
|
815
|
+
channel.queue("", :auto_delete => true).bind(exchange, :arguments => { 'x-match' => 'all', :arch => "ia64", :os => 'linux' })
|
816
|
+
</code>
|
817
|
+
</pre>
|
818
|
+
|
819
|
+
In the example above, only messages that have an "arch" header value equal to "ia64" and an "os" header value equal to "linux" will
|
820
|
+
be considered matching.
|
821
|
+
|
822
|
+
<pre>
|
823
|
+
<code>
|
824
|
+
channel.queue("", :auto_delete => true).bind(exchange, :arguments => { 'x-match' => 'any', :os => 'macosx', :cores => 8 })
|
825
|
+
</code>
|
826
|
+
</pre>
|
827
|
+
|
828
|
+
When the "x-match" argument is set to "any", just one matching header value is sufficient. So in the example above, any message with a
|
829
|
+
"cores" header value equal to 8 will be considered matching.
|
830
|
+
|
831
|
+
h4. More examples
|
190
832
|
|
191
833
|
TBD
|
192
834
|
|
193
835
|
|
836
|
+
h3. Declaring a headers exchange
|
194
837
|
|
195
|
-
|
838
|
+
There are two ways to declare a headers exchange:
|
839
|
+
|
840
|
+
* By instantiating {AMQP::Exchange} and specifying type as ":headers"
|
841
|
+
* By using the {AMQP::Channel#headers} method
|
842
|
+
|
843
|
+
Here are two examples to demonstrate:
|
844
|
+
|
845
|
+
<pre>
|
846
|
+
<code>
|
847
|
+
exchange = AMQP::Exchange.new(channel, :headers, "builds")
|
848
|
+
</code>
|
849
|
+
</pre>
|
850
|
+
|
851
|
+
<pre>
|
852
|
+
<code>
|
853
|
+
exchange = channel.headers("builds")
|
854
|
+
</code>
|
855
|
+
</pre>
|
856
|
+
|
857
|
+
Both methods asynchronously declare a queue. Because declaration necessitates a network round trip, publishing operations on {AMQP::Exchange} instances are
|
858
|
+
delayed until the broker reply (`exchange.declare-ok`) is received.
|
859
|
+
|
860
|
+
Both methods let you pass a block to run a piece of code when the broker responds with `exchange.declare-ok` (meaning that the exchange has been successfully declared).
|
861
|
+
|
862
|
+
<pre>
|
863
|
+
<code>
|
864
|
+
channel.headers("builds") do |exchange|
|
865
|
+
# exchange is declared and ready to be used.
|
866
|
+
end
|
867
|
+
</code>
|
868
|
+
</pre>
|
869
|
+
|
870
|
+
|
871
|
+
h3. Headers exchange routing example
|
872
|
+
|
873
|
+
When there is just one queue bound to a headers exchange, messages are routed to it if one or all headers of the message match those specified
|
874
|
+
upon binding. Whether it is "any header" or "all of them" depends on the "x-match" header value.
|
875
|
+
In the case of multiple queues, a headers exchange will deliver a copy of a message to each queue, just like direct exchanges do. Distribution rules between consumers
|
876
|
+
on a particular queue are the same as for a direct exchange.
|
877
|
+
|
878
|
+
Full example:
|
879
|
+
<script src="https://gist.github.com/1034875.js"> </script>
|
880
|
+
|
881
|
+
|
882
|
+
h3. Headers exchange use cases
|
883
|
+
|
884
|
+
Headers exchanges can be looked upon as "direct exchanges on steroids". Because they route based on header values, they can be used as
|
885
|
+
direct exchanges where the routing key does not have to be a string; it could be an integer or a hash (dictionary) for example.
|
886
|
+
|
887
|
+
Some specific use cases:
|
888
|
+
|
889
|
+
* Transfer of work between stages in a multi-step workflow ("routing slip pattern":http://eaipatterns.com/RoutingTable.html)
|
890
|
+
* Distributed build/continuous integration systems can distribute builds based on multiple parameters (OS, CPU architecture, availability
|
891
|
+
of a particular package).
|
892
|
+
|
893
|
+
|
894
|
+
h3. Pre-declared headers exchanges
|
895
|
+
|
896
|
+
AMQP v0.9.1 brokers should (as defined by "IETF RFC 2119":http://www.ietf.org/rfc/rfc2119.txt) implement a headers exchange type
|
897
|
+
and pre-declare one instance with the name of "amq.match". RabbitMQ also pre-declares one instance with the name of "amq.headers".
|
898
|
+
Applications can rely on that exchange always being available to them. Each vhost has a separate instance of those exchanges and
|
899
|
+
they are *not shared across vhosts* for obvious reasons.
|
900
|
+
|
901
|
+
|
902
|
+
h2. Custom exchange types
|
903
|
+
|
904
|
+
h3. x-random
|
905
|
+
|
906
|
+
The "x-random AMQP exchange type":https://github.com/jbrisbin/random-exchange is a custom exchange type developed as a
|
907
|
+
RabbitMQ plugin by Jon Brisbin. To quote from the project README:
|
908
|
+
|
909
|
+
bq. It is basically a direct exchange, with the exception that, instead of each consumer bound to that exchange with the
|
910
|
+
same routing key getting a copy of the message, the exchange type randomly selects a queue to route to.
|
911
|
+
|
912
|
+
This plugin is licensed under "Mozilla Public License 1.1":http://www.mozilla.org/MPL/MPL-1.1.html, same as RabbitMQ.
|
913
|
+
|
914
|
+
h3. x-recent-history
|
915
|
+
|
916
|
+
The "x-recent-history AMQP exchange type":https://github.com/videlalvaro/rabbitmq-recent-history-exchange is a customer exchange type
|
917
|
+
implemented as a RabbitMQ plugin by Alvaro Videla, one of the authors of "RabbitMQ in action":http://bit.ly/rabbitmq.
|
918
|
+
|
919
|
+
This plugin is licensed under the "MIT license":https://github.com/videlalvaro/rabbitmq-recent-history-exchange/blob/master/LICENSE.md.
|
920
|
+
|
921
|
+
|
922
|
+
h2. Using the Publisher Confirms extension to AMQP v0.9.1
|
923
|
+
|
924
|
+
Please refer to {file:docs/VendorSpecificExtensions.textile Vendor-specific extensions to AMQP 0.9.1 spec}
|
925
|
+
|
926
|
+
|
927
|
+
h3. Message acknowledgements and their relationship to transactions and publisher confirms
|
928
|
+
|
929
|
+
Consumer applications (applications that receive and process messages) may occasionally fail to process individual messages, or might just
|
930
|
+
crash. Additionally, network issues might be experienced. This raises a question - "when should the AMQP broker remove messages from queues?" This
|
931
|
+
topic is covered in depth in the {file:docs/Queues.textile Working With Queues} guide, including prefetching and examples.
|
932
|
+
|
933
|
+
In this guide, we will only mention how message acknowledgements are related to AMQP transactions and the Publisher Confirms extension.
|
934
|
+
Let us consider a publisher application (P) that communications with a consumer (C) using AMQP v0.9.1. Their communication can be graphically
|
935
|
+
represented like this:
|
936
|
+
|
937
|
+
<pre>
|
938
|
+
----- ----- -----
|
939
|
+
| | S1 | | S2 | |
|
940
|
+
| P | ====> | B | ====> | C |
|
941
|
+
| | | | | |
|
942
|
+
----- ----- -----
|
943
|
+
</pre>
|
944
|
+
|
945
|
+
We have two network segments, S1 and S2. Each of them may fail. P is concerned with making sure that messages cross S1,
|
946
|
+
while the broker (B) and C are concerned with ensuring that messages cross S2 and are only removed from the queue when
|
947
|
+
they are processed successfully.
|
948
|
+
|
949
|
+
Message acknowledgements cover reliable delivery over S2 as well as successful processing. For S1, P has to use transactions (a heavyweight solution) or the more
|
950
|
+
lightweight Publisher Confirms, a RabbitMQ-specific extension.
|
951
|
+
|
952
|
+
|
953
|
+
h2. Using AMQP transactions
|
196
954
|
|
197
955
|
TBD
|
198
956
|
|
199
957
|
|
958
|
+
h2. Binding queues to exchanges
|
959
|
+
|
960
|
+
Queues are bound to exchanges using the {AMQP::Queue#bind} method. This topic is described in detail in the {file:docs/Queues.textile Working with queues}
|
961
|
+
documentation guide.
|
962
|
+
|
200
963
|
|
201
964
|
h2. Unbinding queues from exchanges
|
202
965
|
|
203
|
-
|
966
|
+
Queues are unbound from exchanges using the {AMQP::Queue#unbind} method. This topic is described in detail in the {file:docs/Queues.textile Working with queues}
|
967
|
+
documentation guide.
|
204
968
|
|
205
969
|
|
206
|
-
h2. Deleting
|
970
|
+
h2. Deleting exchange
|
207
971
|
|
972
|
+
h3. Explicitly deleting an exchange
|
208
973
|
|
209
|
-
Exchanges are
|
974
|
+
Exchanges are deleted using the {AMQP::Exchange#delete} method:
|
210
975
|
|
211
|
-
|
976
|
+
<pre>
|
977
|
+
<code>
|
978
|
+
exchange.delete
|
979
|
+
</code>
|
980
|
+
</pre>
|
212
981
|
|
213
|
-
|
982
|
+
{AMQP::Exchange#delete} takes an optional callback that is run when a `exchange.delete-ok` reply arrives from the broker.
|
214
983
|
|
984
|
+
<pre>
|
985
|
+
<code>
|
986
|
+
exchange.delete do |delete_ok|
|
987
|
+
# by now exchange is guaranteed to be deleted
|
988
|
+
end
|
989
|
+
</code>
|
990
|
+
</pre>
|
215
991
|
|
992
|
+
h3. Auto-deleted exchanges
|
216
993
|
|
217
|
-
|
994
|
+
Exchanges can be *auto-deleted*. To declare an exchange as auto-deleted, use the ":auto_delete" option on declaration:
|
218
995
|
|
219
|
-
|
996
|
+
<pre>
|
997
|
+
<code>
|
998
|
+
exchange = AMQP::Exchange.new(channel, :direct, "nodes.metadata", :auto_delete => true)
|
999
|
+
</code>
|
1000
|
+
</pre>
|
220
1001
|
|
1002
|
+
<pre>
|
1003
|
+
<code>
|
1004
|
+
exchange = channel.direct("nodes.metadata", :auto_delete => true)
|
1005
|
+
</code>
|
1006
|
+
</pre>
|
221
1007
|
|
222
1008
|
|
223
|
-
|
1009
|
+
Full example:
|
1010
|
+
<script src="https://gist.github.com/1020226.js"> </script>
|
1011
|
+
|
1012
|
+
TBD: explain when exchange is considered to be "no longer in use"
|
1013
|
+
|
1014
|
+
|
1015
|
+
h2. Objects as message producers.
|
1016
|
+
|
1017
|
+
Since Ruby is a genuine object-oriented language, it is important to demonstrate how the Ruby amqp gem can be integrated
|
1018
|
+
into rich object-oriented code. This part of the guide focuses on exchanges and the problems/solutions concerning producer applications
|
1019
|
+
(applications that primarily generate and publish messages, as opposed to consumers that receive and process them).
|
1020
|
+
|
1021
|
+
Full example:
|
1022
|
+
<script src="https://gist.github.com/1009425.js"> </script>
|
224
1023
|
|
225
1024
|
TBD
|
226
1025
|
|
227
1026
|
|
1027
|
+
h2. Exchange durability vs Message durability
|
1028
|
+
|
1029
|
+
See {file:docs/Durability.textile Durability guide}
|
1030
|
+
|
1031
|
+
|
1032
|
+
h2. Error handling and recovery
|
1033
|
+
|
1034
|
+
See {file:docs/ErrorHandling.textile Error handling and recovery guide}
|
1035
|
+
|
228
1036
|
|
229
1037
|
h2. Vendor-specific extensions related to exchanges
|
230
1038
|
|
231
1039
|
See {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}
|
232
1040
|
|
233
1041
|
|
234
|
-
|
235
1042
|
h2. What to read next
|
236
1043
|
|
237
|
-
|
1044
|
+
Documentation is organized as several {file:docs/DocumentationGuidesIndex.textile documentation guides} that cover all kinds of
|
1045
|
+
topics. Guides related to this one are
|
1046
|
+
|
1047
|
+
* {file:docs/Durability.textile Durability and message persistence}
|
1048
|
+
* {file:docs/Bindings.textile Bindings}
|
1049
|
+
* {file:docs/PatternsAndUseCases.textile Patterns and Use Cases}
|
1050
|
+
* {file:docs/Queues.textile Working With Queues}
|
1051
|
+
* {file:docs/ErrorHandling.textile Error handling and recovery}
|
238
1052
|
|
239
1053
|
|
240
1054
|
h2. Tell us what you think!
|
241
1055
|
|
242
|
-
Please take a moment
|
243
|
-
what was unclear
|
244
|
-
key to making documentation better.
|
1056
|
+
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
|
1057
|
+
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
|
1058
|
+
key to making the documentation better.
|
245
1059
|
|
246
|
-
If
|
1060
|
+
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
|
247
1061
|
|
248
1062
|
|
249
1063
|
<div id="disqus_thread"></div>
|