amqp 0.8.0.rc13 → 0.8.0.rc14
Sign up to get free protection for your applications and to get access to all the features.
- data/.rspec +2 -1
- data/.travis.yml +8 -2
- data/.yardopts +1 -0
- data/CHANGELOG +9 -0
- data/Gemfile +17 -11
- data/README.md +26 -16
- data/amqp.gemspec +2 -2
- data/bin/ci/before_build.sh +21 -0
- data/docs/08Migration.textile +199 -5
- data/docs/AMQP091ModelExplained.textile +322 -0
- data/docs/Bindings.textile +24 -4
- data/docs/Clustering.textile +1 -1
- data/docs/ConnectingToTheBroker.textile +98 -82
- data/docs/ConnectionEncryptionWithTLS.textile +65 -5
- data/docs/DocumentationGuidesIndex.textile +93 -13
- data/docs/Durability.textile +1 -1
- data/docs/ErrorHandling.textile +458 -94
- data/docs/Exchanges.textile +901 -87
- data/docs/GettingStarted.textile +278 -143
- data/docs/PatternsAndUseCases.textile +420 -0
- data/docs/Queues.textile +730 -178
- data/docs/RabbitMQVersions.textile +18 -3
- data/docs/RunningTests.textile +1 -1
- data/docs/TestingWithEventedSpec.textile +121 -0
- data/docs/Troubleshooting.textile +15 -1
- data/docs/VendorSpecificExtensions.textile +1 -1
- data/docs/diagrams/001_hello_world_example_routing.png +0 -0
- data/docs/diagrams/002_blabbr_example_routing.png +0 -0
- data/docs/diagrams/003_weathr_example_routing.png +0 -0
- data/docs/diagrams/004_fanout_exchange.png +0 -0
- data/docs/diagrams/005_direct_exchange.png +0 -0
- data/docs/diagrams/redhat/direct_exchange.png +0 -0
- data/docs/diagrams/redhat/fanout_exchange.png +0 -0
- data/docs/diagrams/redhat/topic_exchange.png +0 -0
- data/examples/error_handling/automatic_recovery_of_channel_and_queues.rb +50 -0
- data/examples/error_handling/automatically_recovering_hello_world_consumer.rb +51 -0
- data/examples/error_handling/automatically_recovering_hello_world_consumer_that_uses_a_server_named_queue.rb +51 -0
- data/examples/error_handling/basic_connection_failover.rb +22 -0
- data/examples/error_handling/channel_level_exception.rb +9 -2
- data/examples/error_handling/connection_level_exception.rb +8 -1
- data/examples/error_handling/connection_level_exception_with_objects.rb +49 -0
- data/examples/error_handling/connection_loss_handler.rb +1 -5
- data/examples/error_handling/hello_world_producer.rb +43 -0
- data/examples/error_handling/insufficient_permissions.rb +54 -0
- data/examples/error_handling/manual_connection_and_channel_recovery.rb +71 -0
- data/examples/error_handling/queue_exclusivity_violation.rb +41 -0
- data/examples/error_handling/queue_name_violation.rb +31 -0
- data/examples/exchanges/autodeletion_of_exchanges.rb +1 -4
- data/examples/guides/queues/01a_declaring_a_server_named_queue_using_queue_constructor.rb +7 -8
- data/examples/guides/queues/01b_declaring_a_queue_using_queue_constructor.rb +7 -8
- data/examples/guides/queues/02a_declaring_a_durable_shared_queue.rb +5 -8
- data/examples/guides/queues/02b_declaring_a_durable_shared_queue.rb +5 -8
- data/examples/guides/queues/03a_declaring_a_temporary_exclusive_queue.rb +7 -8
- data/examples/guides/queues/04_bind_a_queue_using_exchange_instance.rb +9 -10
- data/examples/guides/queues/05_bind_a_queue_using_exchange_name.rb +8 -10
- data/examples/guides/queues/06_subscribe_to_receive_messages.rb +10 -12
- data/examples/guides/queues/07_fetch_a_message_from_the_queue.rb +14 -14
- data/examples/guides/queues/08_unsubscribing_a_consumer.rb +13 -16
- data/examples/guides/queues/09_unbinding_from_exchange.rb +16 -22
- data/examples/guides/queues/10_purge_a_queue.rb +13 -18
- data/examples/guides/queues/11_deleting_a_queue.rb +14 -19
- data/examples/guides/queues/12_objects_that_consume_messages.rb +69 -0
- data/examples/guides/queues/13_objects_that_consume_messages_take_two.rb +89 -0
- data/examples/hello_world.rb +1 -3
- data/examples/hello_world_with_an_empty_string.rb +5 -6
- data/examples/inspecting_server_information.rb +45 -0
- data/examples/issues/issue_93.rb +23 -0
- data/examples/issues/issue_94.rb +23 -0
- data/examples/patterns/command/consumer.rb +45 -0
- data/examples/patterns/command/producer.rb +26 -0
- data/examples/patterns/request_reply/client.rb +29 -0
- data/examples/patterns/request_reply/server.rb +26 -0
- data/examples/publishing/publishing_a_one_off_message.rb +6 -4
- data/examples/publishing/returned_messages.rb +2 -10
- data/examples/queues/accessing_message_metadata.rb +15 -13
- data/examples/queues/queue_status.rb +12 -15
- data/examples/routing/fanout_routing.rb +33 -0
- data/examples/routing/headers_routing.rb +17 -15
- data/examples/routing/round_robin_with_direct_exchange.rb +39 -0
- data/examples/routing/round_robin_with_the_default_exchange.rb +38 -0
- data/examples/routing/unroutable_mandatory_message_is_returned.rb +33 -0
- data/examples/routing/weather_updates.rb +15 -20
- data/examples/tls/using_tls.rb +41 -0
- data/lib/amqp/bit_set.rb +80 -0
- data/lib/amqp/broker.rb +72 -0
- data/lib/amqp/channel.rb +93 -13
- data/lib/amqp/client.rb +11 -22
- data/lib/amqp/compatibility/ruby187_patchlevel_check.rb +2 -0
- data/lib/amqp/connection.rb +2 -3
- data/lib/amqp/consumer.rb +208 -0
- data/lib/amqp/deprecated/fork.rb +2 -0
- data/lib/amqp/deprecated/mq.rb +2 -0
- data/lib/amqp/exchange.rb +6 -4
- data/lib/amqp/extensions/rabbitmq.rb +3 -1
- data/lib/amqp/header.rb +76 -14
- data/lib/amqp/int_allocator.rb +96 -0
- data/lib/amqp/logger.rb +2 -0
- data/lib/amqp/queue.rb +242 -86
- data/lib/amqp/rpc.rb +2 -0
- data/lib/amqp/session.rb +169 -9
- data/lib/amqp/utilities/event_loop_helper.rb +2 -0
- data/lib/amqp/utilities/server_type.rb +2 -0
- data/lib/amqp/version.rb +2 -2
- data/lib/mq.rb +4 -2
- data/lib/mq/logger.rb +3 -1
- data/lib/mq/rpc.rb +3 -1
- data/spec/integration/authentication_spec.rb +17 -10
- data/spec/integration/automatic_binding_for_default_direct_exchange_spec.rb +1 -1
- data/spec/integration/automatic_recovery_predicate_spec.rb +68 -0
- data/spec/integration/basic_get_spec.rb +2 -1
- data/spec/integration/{extensions/basic_return_spec.rb → basic_return_spec.rb} +2 -1
- data/spec/integration/channel_level_exception_handling_spec.rb +53 -0
- data/spec/integration/connection_level_exception_handling_spec.rb +49 -0
- data/spec/integration/declare_and_immediately_bind_a_server_named_queue_spec.rb +38 -17
- data/spec/integration/declare_one_hundred_server_named_queues_spec.rb +44 -0
- data/spec/integration/direct_exchange_routing_spec.rb +125 -0
- data/spec/integration/exchange_declaration_spec.rb +75 -46
- data/spec/integration/extensions/rabbitmq/publisher_confirmations_spec.rb +180 -0
- data/spec/integration/{workload_distribution_spec.rb → fanout_exchange_routing_spec.rb} +10 -9
- data/spec/integration/headers_exchange_routing_spec.rb +269 -0
- data/spec/integration/hello_world_spec.rb +77 -0
- data/spec/integration/immediate_messages_spec.rb +59 -0
- data/spec/integration/mandatory_messages_spec.rb +52 -0
- data/spec/integration/message_metadata_access_spec.rb +106 -0
- data/spec/integration/multiple_consumers_per_queue_spec.rb +319 -0
- data/spec/integration/ordering_of_published_messages_spec.rb +96 -0
- data/spec/integration/queue_declaration_spec.rb +8 -8
- data/spec/integration/queue_status_spec.rb +66 -0
- data/spec/integration/recovery/per_channel_automatic_recovery_on_graceful_broker_shutdown_spec.rb +76 -0
- data/spec/integration/recovery/per_channel_automatic_recovery_spec.rb +72 -0
- data/spec/integration/redelivery_of_unacknowledged_messages_spec.rb +96 -0
- data/spec/integration/regressions/concurrent_publishing_on_the_same_channel_spec.rb +91 -0
- data/spec/integration/regressions/empty_message_body_spec.rb +56 -0
- data/spec/integration/regressions/issue66_spec.rb +2 -1
- data/spec/integration/reply_queue_communication_spec.rb +2 -1
- data/spec/integration/store_and_forward_spec.rb +4 -3
- data/spec/integration/topic_subscription_spec.rb +2 -1
- data/spec/integration/tx_commit_spec.rb +124 -0
- data/spec/integration/tx_rollback_spec.rb +167 -0
- data/spec/spec_helper.rb +44 -71
- data/spec/unit/amqp/bit_set_spec.rb +127 -0
- data/spec/unit/amqp/channel_id_allocation_spec.rb +40 -0
- data/spec/unit/amqp/connection_spec.rb +4 -2
- data/spec/unit/amqp/int_allocator_spec.rb +116 -0
- metadata +92 -26
- data/CONTRIBUTORS +0 -29
- data/docs/Routing.textile +0 -30
- data/examples/real-world/task-queue/README.textile +0 -3
- data/examples/real-world/task-queue/consumer.rb +0 -27
- data/examples/real-world/task-queue/producer.rb +0 -22
- data/spec/unit/amqp/basic_spec.rb +0 -39
- data/tasks.rb +0 -4
@@ -0,0 +1,322 @@
|
|
1
|
+
# @title Ruby amqp gem: AMQP 0.9.1 Model Explained
|
2
|
+
|
3
|
+
h1. AMQP 0.9.1 Model Explained
|
4
|
+
|
5
|
+
h2. About this guide
|
6
|
+
|
7
|
+
This guide explains AMQP 0.9.1 Model used by RabbitMQ. Understanding the AMQP Model will make a lot of other documentation, both for the Ruby amqp gem and
|
8
|
+
RabbitMQ itself, easier to follow.
|
9
|
+
|
10
|
+
This guide covers:
|
11
|
+
|
12
|
+
* High-level overview of the AMQP 0.9.1 Model.
|
13
|
+
* Key differences of the AMQP model from some other messaging models.
|
14
|
+
* What are exchanges.
|
15
|
+
* What are queues.
|
16
|
+
* What are bindings.
|
17
|
+
* How AMQP protocol is structured. What are AMQP methods.
|
18
|
+
* What attributes AMQP 0.9.1 messages have.
|
19
|
+
* What are message acknowledgements.
|
20
|
+
* What are negative message acknowledgements.
|
21
|
+
* and a lot of other things.
|
22
|
+
|
23
|
+
|
24
|
+
h2. Which versions of the amqp gem does this guide cover?
|
25
|
+
|
26
|
+
This guide covers v0.8.0 and later of the "Ruby amqp gem":https://github.com/ruby-amqp/amqp.
|
27
|
+
|
28
|
+
|
29
|
+
h2. High-level overview of the AMQP 0.9.1 Model
|
30
|
+
|
31
|
+
h3. Brokers & Their Role
|
32
|
+
|
33
|
+
AMQP (Advanced Message Queuing Protocol) is a networking protocol which enables conforming client applications to communicate with conforming
|
34
|
+
messaging middleware brokers. Brokers receive messages from _producers_ (applications that publish them) and route them to _consumers_ (applications
|
35
|
+
that process them).
|
36
|
+
|
37
|
+
Brokers are centers of the nervous system, applications are more like limbs.
|
38
|
+
|
39
|
+
|
40
|
+
h3. AMQP 0.9.1 Model, briefly
|
41
|
+
|
42
|
+
The AMQP 0.9.1 Model has the following view of the world: messages are published by producers to _exchanges_, often compared to post offices or mailboxes. Exchanges then
|
43
|
+
distribute message copies to _queues_ using rules called _bindings_. Then AMQP brokers either push messages to _consumers_ subscribed to some queues, or consumers
|
44
|
+
fetch/pull messages from queues on demand, as needed.
|
45
|
+
|
46
|
+
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/001_hello_world_example_routing.png!
|
47
|
+
|
48
|
+
When publishing a message, producers may specify various _message attributes_ (message metadata). Some of this metadata may be used by the broker, some other
|
49
|
+
is completely opaque and will only be used by applications that receive the message.
|
50
|
+
|
51
|
+
Because networks are not reliable and applications may fail to process messages, the AMQP Model has a notion of
|
52
|
+
_message acknowledgements_: when a message is pushed down to a consumer, the consumer _notifies the broker_, either automatically or as soon as application
|
53
|
+
developer chooses to. Brokers then will only completely remove message from the queue when they receive a notification for that message (or a group of messages).
|
54
|
+
|
55
|
+
In certain situations, for example, when a message cannot be routed, messages may be _returned_ to producers or dropped (or, if broker implements an extension,
|
56
|
+
placed into a so-called "dead letter queue"). Producers choose how situations like this are handled by publishing messages with certain parameters.
|
57
|
+
|
58
|
+
Queues, exchanges and bindings are commonly referred to as _AMQP entities_.
|
59
|
+
|
60
|
+
|
61
|
+
h3. AMQP is a Programmable Protocol
|
62
|
+
|
63
|
+
AMQP 0.9.1 is a programmable protocol in the sense that AMQP entities & routing schemes are defined by applications themselves, not a broker administrator. So there are protocol
|
64
|
+
operations that declare queues and exchanges, define bindings between them, subscribe to queues and so on.
|
65
|
+
|
66
|
+
This gives application developers a lot of freedom but also requires them to be aware of potential definition conflicts. In practice, those definition conflicts
|
67
|
+
are rare and often indicate misconfigurations, and it is a good thing that misconfigurations are caught early.
|
68
|
+
|
69
|
+
Applications declare AMQP entities they need, define routing schemes necessary and may choose to delete AMQP entities when they are no longer used.
|
70
|
+
|
71
|
+
|
72
|
+
|
73
|
+
|
74
|
+
h2. AMQP Exchanges and Exchange Types.
|
75
|
+
|
76
|
+
_Exchanges_ are AMQP entities where messages are sent. Exchanges then take a message and route it into one or more (or no) queues. The routing algorithm used
|
77
|
+
depends on _exchange type_ and rules called _bindings_. AMQP 0.9.1 brokers typically provide 4 exchange types out of the box:
|
78
|
+
|
79
|
+
* Direct exchange (typically used for for 1-to-1 communication or unicasting)
|
80
|
+
* Fanout exchange (1-to-n communication or broadcasting)
|
81
|
+
* Topic exchange (1-to-n or n-to-m communication, multicasting)
|
82
|
+
* Headers exchange (message metadata-based routing)
|
83
|
+
|
84
|
+
but it is possible to extend AMQP 0.9.1 brokers with custom exchange types, for example:
|
85
|
+
|
86
|
+
* x-random exchange (randomly chooses a queue to route incoming messages to)
|
87
|
+
* x-recent-history (a fanout exchange that also keeps N recent messages in memory)
|
88
|
+
* regular expressions based variations of headers exchange
|
89
|
+
|
90
|
+
and so on.
|
91
|
+
|
92
|
+
Besides the type, exchanges have a number of attributes, most imprortant of which are:
|
93
|
+
|
94
|
+
* Name
|
95
|
+
* Can be durable (information about them is persisted to disk and thus survives broker restarts) or non-durable (information is only kept in RAM)
|
96
|
+
* Can have metadata associated with them on declaration
|
97
|
+
|
98
|
+
|
99
|
+
h2. AMQP Queues.
|
100
|
+
|
101
|
+
Queues in the AMQP Model are very similar to queues in other messages (and "task queueing") systems: they store messages that are consumed by
|
102
|
+
applications. Like AMQP exchanges, AMQP queues have names and durability property but also
|
103
|
+
|
104
|
+
* Can be exclusive (used by only one connection)
|
105
|
+
* Can be automatically deleted when last consumer unsubscribes
|
106
|
+
* Can have metadata associated with them on declaration (some brokers use it to implement features like message TTL and so on)
|
107
|
+
|
108
|
+
|
109
|
+
|
110
|
+
h2. AMQP Bindings.
|
111
|
+
|
112
|
+
Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q,
|
113
|
+
Q has to _be bound_ to E. Bindings may have an optional _routing key_ attribute used by some exchange types. The purpose of the routing key is to
|
114
|
+
selectively match only specific (matching) messages published to an exchange to the bound queue. In other words, the routing key acts like a filter.
|
115
|
+
|
116
|
+
|
117
|
+
|
118
|
+
h2. AMQP Message Consumers.
|
119
|
+
|
120
|
+
Storing messages in queues is useless unless applications can _consume_ them. In the AMQP 0.9.1 Model, there are two ways for applications to do it:
|
121
|
+
|
122
|
+
* Have messages pushed to them ("push API")
|
123
|
+
* Fetch messages as needed ("pull API")
|
124
|
+
|
125
|
+
With the "push API", applications have to indicate interest in consuming messages from a particular queue. When they do so, we say that they _register a consumer_
|
126
|
+
(or, simply put, _subscribe to a queue_). It is possible to have more than one consumer per queue or to register an _exclusive consumer_ (the only consumer on
|
127
|
+
the queue).
|
128
|
+
|
129
|
+
Each consumer (subscription) has an identifier called _consumer tag_. It can be used to unsubscribe from messages. Consumer tags are just strings.
|
130
|
+
|
131
|
+
|
132
|
+
|
133
|
+
h2. AMQP Message Attributes and Payload.
|
134
|
+
|
135
|
+
Messages in the AMQP Model have _attributes_. Some attributes are so common that AMQP 0.9.1 spec defines them and application developers don't have to think
|
136
|
+
about exact attribute name. Some examples are
|
137
|
+
|
138
|
+
* Content type
|
139
|
+
* Content encoding
|
140
|
+
* Routing key
|
141
|
+
* Delivery mode (persistent or not)
|
142
|
+
* Message priority
|
143
|
+
* Message publishing timestamp
|
144
|
+
* Expiration period
|
145
|
+
* Producer application id
|
146
|
+
|
147
|
+
Some attributes are used by AMQP brokers, but most are up to interpretation by applications that receive them. Some attributes are optional and known
|
148
|
+
as _headers_. They are similar to X-Headers in HTTP. Message attributes are set when message is published.
|
149
|
+
|
150
|
+
AMQP messages also have _payload_ (data they carry). Brokers treat this data as opaque (it is not modified nor used by them). It is possible for messages to only have attributes
|
151
|
+
and no body. It is common to use serialization formats like JSON, Thrift, Protocol Buffers, MessagePack and so on to serialize structured data
|
152
|
+
and publish it as AMQP message payload.
|
153
|
+
|
154
|
+
|
155
|
+
|
156
|
+
h2. AMQP Message Acknowledgements.
|
157
|
+
|
158
|
+
Since networks are unreliable and applications fail, it is often necessary to have some kind of "processing acknowledgements". Sometimes it is only
|
159
|
+
necessary to acknowledge the fact that message has been received, sometimes acknowledgements mean that message was validated & processed by consumer,
|
160
|
+
for example, verified to have mandatory data and persisted to a data store or indexed.
|
161
|
+
|
162
|
+
Because this situation is so common, AMQP 0.9.1 has a built-in feature called _message acknowledgements_ (sometimes referred to as _acks_) that consumer
|
163
|
+
use to confirm message delivery and/or processing. If an application crashes (AMQP broker notice this when connection is closed), if acknowledgement for a
|
164
|
+
message was not received by AMQP broker, the message is re-queued (and possibly immediately delivered to another consumer, if any).
|
165
|
+
|
166
|
+
Having acknowledgements built into the protocol helps developers to build more robust software.
|
167
|
+
|
168
|
+
|
169
|
+
|
170
|
+
h2. AMQP 0.9.1 Methods
|
171
|
+
|
172
|
+
AMQP 0.9.1 is structured as a number of _methods_. Methods are operations (like HTTP methods) and have nothing in common with methods in object-oriented programming
|
173
|
+
languages. AMQP methods are grouped into _classes_. Classes are just logical groupings of AMQP methods. "AMQP 0.9.1 reference":http://www.rabbitmq.com/amqp-0-9-1-reference.html can be found on
|
174
|
+
the RabbitMQ website.
|
175
|
+
|
176
|
+
Lets take a look at the _exchange.*_ class, a group of methods related to operations on exchanges. It includes the following operations:
|
177
|
+
|
178
|
+
|
179
|
+
* exchange.declare
|
180
|
+
* exchange.declare-ok
|
181
|
+
* exchange.delete
|
182
|
+
* exchange.delete-ok
|
183
|
+
|
184
|
+
(note that RabbitMQ site reference also includes RabbitMQ-specific extensions to the exchange.* class that we won't discuss in this guide).
|
185
|
+
|
186
|
+
The operations above form logical pairs: *exchange.declare* and *exchange.declare-ok*,
|
187
|
+
*exchange.delete* and *exchange.delete-ok*. These operations are "requests" (sent by clients) and "responses" (sent by
|
188
|
+
brokers in response to aforementioned "requests").
|
189
|
+
|
190
|
+
So first, client asks broker to declare a new exchange using *exchange.declare* method:
|
191
|
+
|
192
|
+
!https://img.skitch.com/20110720-c4qjdhmdrih9bn56npqnic4die.jpg!
|
193
|
+
|
194
|
+
As demonstrated on the diagram above, *exchange.declare* carries several _parameters_. They let client specify exchange name,
|
195
|
+
type, durability flag and so on.
|
196
|
+
|
197
|
+
And if the operation succeeds, broker responds with *exchange.declare-ok* method:
|
198
|
+
|
199
|
+
!https://img.skitch.com/20110720-m4ptjbnex2sa52g6wdwj3e9ahm.jpg!
|
200
|
+
|
201
|
+
*exchange.declare-ok* does not carry any parameters except for the channel number (channelswill be described later in this guide).
|
202
|
+
|
203
|
+
Event sequence is very similar for another methods pair, *queue.declare* and *queue.declare-ok*:
|
204
|
+
|
205
|
+
!https://img.skitch.com/20110720-tmxswrie71ubb5m5nh8n17idhk.jpg!
|
206
|
+
|
207
|
+
!https://img.skitch.com/20110720-g67urxg75c71qtwhwsjs684323.jpg!
|
208
|
+
|
209
|
+
Not all AMQP methods have counterparts. Some (*basic.publish* being the most widely used one) do not have corresponding "response" methods
|
210
|
+
and some others (*basic.get*, for example) have more than one possible "response".
|
211
|
+
|
212
|
+
|
213
|
+
|
214
|
+
h2. AMQP Connections.
|
215
|
+
|
216
|
+
AMQP connections are typically long-lived. AMQP is an application level protocol that uses TCP for reliable delivery. AMQP connections use
|
217
|
+
authentication and can be protected using TLS (SSL). When application no longer needs to be connected to AMQP broker, it should gracefully
|
218
|
+
close AMQP connection instead of abruptly closing the underlying TCP connection.
|
219
|
+
|
220
|
+
|
221
|
+
|
222
|
+
h2. AMQP Channels.
|
223
|
+
|
224
|
+
Some applications need multiple connections to AMQP broker. It is, however, undesirable to keep many TCP connections open at the same time
|
225
|
+
because doing so consumes system resources and makes it more difficult to configure firewalls. AMQP 0.9.1 connections are multiplexed with
|
226
|
+
_channels_ that can be thought of as "lightweight connections that share a single TCP connection".
|
227
|
+
|
228
|
+
For applications that use multiple threads/processes/etc for processing, it is very common to open a new channel per thread (process, etc)
|
229
|
+
and *not share* channels between them.
|
230
|
+
|
231
|
+
Because communication on a particular channel is completely separate from communication on a separate channel, every AMQP method also carries a channel number that clients
|
232
|
+
use to figure out what channel this method is for (and thus, what event handler needs to be invoked, for example).
|
233
|
+
|
234
|
+
|
235
|
+
|
236
|
+
|
237
|
+
h2. AMQP Virtual Hosts (vhosts).
|
238
|
+
|
239
|
+
To make it possible for a single broker to host multiple isolated "environments" (groups of users, exchanges, queues and so on), AMQP includes concept
|
240
|
+
of _virtual hosts_ (vhosts). They are similar to virtual hosts used by many popular Web servers and provide completely isolated environments
|
241
|
+
in which AMQP entities live. AMQP clients specify what vhosts they want to use during AMQP connection negotiation.
|
242
|
+
|
243
|
+
AMQP 0.9.1 vhost can be any non-blank string.
|
244
|
+
|
245
|
+
|
246
|
+
|
247
|
+
h2. AMQP is Extensible
|
248
|
+
|
249
|
+
AMQP 0.9.1 has several extension points:
|
250
|
+
|
251
|
+
* Custom exchange types let developers implement routing schemes that exchange types provided out-of-the-box do not cover well, for example, geodata-based routing.
|
252
|
+
* Declaration of exchanges and queues can include additional attributes broker can use. For example, per-queue message TTL in RabbitMQ is implemented this way.
|
253
|
+
* Broker-specific extensions to the protocol. See, for example, "extensions RabbitMQ implements":http://www.rabbitmq.com/extensions.html.
|
254
|
+
* New AMQP 0.9.1 method classes can be introduced.
|
255
|
+
|
256
|
+
These features make it even more flexible and applicable to a very broad range of problems.
|
257
|
+
|
258
|
+
|
259
|
+
|
260
|
+
|
261
|
+
h2. Key differences from some other messaging models
|
262
|
+
|
263
|
+
Key difference to understand about the AMQP 0.9.1 Model is that *messages are not sent to queues. They are sent to exchanges that route them to
|
264
|
+
queues according to rules called "bindings"*. This means that routing is primarily handled by AMQP brokers and not applications themselves.
|
265
|
+
|
266
|
+
TBD
|
267
|
+
|
268
|
+
|
269
|
+
|
270
|
+
h2. Wrapping up
|
271
|
+
|
272
|
+
This is the end of the AMQP 0.9.1 Model tutorial. Congratulations! Armed with this knowledge, you will find it easy to follow the rest of
|
273
|
+
the amqp gem documentation as well as rabbitmq.com documentation and "rabbitmq-discuss mailing list":http://groups.google.com/group/rabbitmq-discuss
|
274
|
+
|
275
|
+
To stay up to date with amqp gem development, "follow @rubyamqp on Twitter":http://twitter.com/rubyamqp and "join our mailing list":http://groups.google.com/group/ruby-amqp.
|
276
|
+
|
277
|
+
|
278
|
+
|
279
|
+
|
280
|
+
h2. What to read next
|
281
|
+
|
282
|
+
Documentation is organized as a number of {file:docs/DocumentationGuidesIndex.textile documentation guides}, covering all kinds of
|
283
|
+
topics from {file:docs/Exchanges.textile use cases for various exchange types} to {file:docs/ErrorHandling.textile error handling} and
|
284
|
+
{file:docs/VendorSpecificExchanges.textile Broker-specific AMQP 0.9.1 extensions}.
|
285
|
+
|
286
|
+
We recommend that you read the following guides next, if possible, in this order:
|
287
|
+
|
288
|
+
* {file:docs/ConnectingToTheBroker.textile Connection to the broker}. This guide explains how to connect to an AMQP broker and how to integrate the amqp gem into standalone and Web applications.
|
289
|
+
* {file:docs/Queues.textile Working With Queues}. This guide focuses on features that consumer applications use heavily.
|
290
|
+
* {file:docs/Exchanges.textile Working With Exchanges}. This guide focuses on features that producer applications use heavily.
|
291
|
+
* {file:docs/PatternsAndUseCases.textile Patterns & Use Cases}. This guide focuses implementation of "common messaging patterns":http://www.eaipatterns.com/ using AMQP Model features as building blocks.
|
292
|
+
* {file:docs/ErrorHandling.textile Error Handling & Recovery}. This guide explains how to handle protocol errors, network failures and other things that may go wrong in real world projects.
|
293
|
+
|
294
|
+
If you are migrating your application from earlier versions of the amqp gem (0.6.x and 0.7.x), to 0.8.x and later, there is the
|
295
|
+
{file:docs/08Migration.textile amqp gem 0.8 migration guide}.
|
296
|
+
|
297
|
+
|
298
|
+
h2. Tell us what you think!
|
299
|
+
|
300
|
+
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
|
301
|
+
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
|
302
|
+
key to making the documentation better.
|
303
|
+
|
304
|
+
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
|
305
|
+
|
306
|
+
|
307
|
+
<div id="disqus_thread"></div>
|
308
|
+
<script type="text/javascript">
|
309
|
+
/* * * CONFIGURATION VARIABLES * * */
|
310
|
+
var disqus_shortname = 'rubyamqpdocs'; // required: replace example with your forum shortname
|
311
|
+
|
312
|
+
var disqus_developer = 0; // set to 1 on local machine for testing comments
|
313
|
+
var disqus_identifier = 'amqp091_model_explained';
|
314
|
+
var disqus_url = 'http://rdoc.info/github/ruby-amqp/amqp/master/file/docs/AMQP091ModelExplained.textile';
|
315
|
+
|
316
|
+
/* * * DON'T EDIT BELOW THIS LINE * * */
|
317
|
+
(function() {
|
318
|
+
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
|
319
|
+
dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
|
320
|
+
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
|
321
|
+
})();
|
322
|
+
</script>
|
data/docs/Bindings.textile
CHANGED
@@ -2,10 +2,10 @@
|
|
2
2
|
|
3
3
|
h1. TBD
|
4
4
|
|
5
|
-
|
6
5
|
h2. About this guide
|
7
6
|
|
8
|
-
|
7
|
+
This guide covers bindings in AMQP 0.9.1, what they are, what role do they play and how to accomplish typical operations using
|
8
|
+
Ruby amqp gem.
|
9
9
|
|
10
10
|
|
11
11
|
h2. Covered versions
|
@@ -14,7 +14,27 @@ This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp v0.8.0 and la
|
|
14
14
|
|
15
15
|
|
16
16
|
|
17
|
-
h2.
|
17
|
+
h2. Bindings in AMQP 0.9.1
|
18
|
+
|
19
|
+
TBD
|
20
|
+
|
21
|
+
|
22
|
+
h2. One day in life of an AMQP 0.9.1 message
|
23
|
+
|
24
|
+
TBD
|
25
|
+
|
26
|
+
|
27
|
+
h2. Binding queues to exchanges
|
28
|
+
|
29
|
+
TBD
|
30
|
+
|
31
|
+
|
32
|
+
h2. Unbinding queues from exchanges
|
33
|
+
|
34
|
+
TBD
|
35
|
+
|
36
|
+
|
37
|
+
h2. Binding attributes
|
18
38
|
|
19
39
|
TBD
|
20
40
|
|
@@ -22,7 +42,7 @@ TBD
|
|
22
42
|
|
23
43
|
h2. Tell us what you think!
|
24
44
|
|
25
|
-
Please take a moment and tell us what you think about this guide on "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
|
45
|
+
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
|
26
46
|
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
|
27
47
|
key to making documentation better.
|
28
48
|
|
data/docs/Clustering.textile
CHANGED
@@ -22,7 +22,7 @@ TBD
|
|
22
22
|
|
23
23
|
h2. Tell us what you think!
|
24
24
|
|
25
|
-
Please take a moment and tell us what you think about this guide on "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
|
25
|
+
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
|
26
26
|
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
|
27
27
|
key to making documentation better.
|
28
28
|
|
@@ -5,23 +5,23 @@ h1. Connecting to the broker, integrating with Ruby on Rails, Merb and Sinatra
|
|
5
5
|
|
6
6
|
h2. About this guide
|
7
7
|
|
8
|
-
This guide covers connection to AMQP broker from standalone and Web applications,
|
8
|
+
This guide covers connection to an AMQP broker from standalone and Web applications,
|
9
9
|
connection error handling, authentication failure handling and related issues.
|
10
10
|
|
11
11
|
|
12
|
-
h2.
|
12
|
+
h2. Which versions of the amqp gem does this guide cover?
|
13
13
|
|
14
|
-
This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp
|
14
|
+
This guide covers v0.8.0 and later of the "Ruby amqp gem":http://github.com/ruby-amqp/amqp.
|
15
15
|
|
16
16
|
|
17
17
|
|
18
18
|
h2. Terminology
|
19
19
|
|
20
|
-
In this guide we define standalone application as application that does not run on
|
20
|
+
In this guide we define a standalone application as an application that does not run on
|
21
21
|
a Web server like Unicorn or Passenger. The key difference is that these applications
|
22
|
-
control main Ruby VM thread and often use it to run EventMachine event loop. When
|
23
|
-
amqp gem is used
|
24
|
-
server and code required to establish connection to AMQP broker needs to be a little
|
22
|
+
control the main Ruby VM thread and often use it to run the EventMachine event loop. When
|
23
|
+
the amqp gem is used in a Web application, the main thread is occupied by the Web application
|
24
|
+
server and the code required to establish a connection to an AMQP broker needs to be a little
|
25
25
|
bit different.
|
26
26
|
|
27
27
|
|
@@ -35,7 +35,7 @@ Connection parameters (host, port, username, vhost and so on) can be passed in t
|
|
35
35
|
|
36
36
|
h3. Using a hash
|
37
37
|
|
38
|
-
Hash options amqp gem will recognize are
|
38
|
+
Hash options that the amqp gem will recognize are
|
39
39
|
|
40
40
|
* :host
|
41
41
|
* :port
|
@@ -70,7 +70,7 @@ h3. Using connection strings
|
|
70
70
|
It is convenient to be able to specify the AMQP connection
|
71
71
|
parameters as a URI string, and various "amqp" URI schemes
|
72
72
|
exist. Unfortunately, there is no standard for these URIs, so
|
73
|
-
while the schemes share the basic idea, they differ in some
|
73
|
+
while the schemes share the same basic idea, they differ in some
|
74
74
|
details. This implementation aims to encourage URIs that work
|
75
75
|
as widely as possible.
|
76
76
|
|
@@ -99,23 +99,23 @@ connection URIs:
|
|
99
99
|
|
100
100
|
<pre>
|
101
101
|
<code>
|
102
|
-
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com") # => vhost is nil, so default (/) will be used
|
102
|
+
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com") # => vhost is nil, so default ("/") will be used
|
103
103
|
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/") # => vhost is an empty string
|
104
|
-
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/%2Fvault") # => vhost is /vault
|
105
|
-
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/production") # => vhost is production
|
106
|
-
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/a.b.c") # => vhost is a.b.c
|
104
|
+
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/%2Fvault") # => vhost is "/vault"
|
105
|
+
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/production") # => vhost is "production"
|
106
|
+
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/a.b.c") # => vhost is "a.b.c"
|
107
107
|
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/foo/bar") # => ArgumentError
|
108
108
|
</code>
|
109
109
|
</pre>
|
110
110
|
|
111
111
|
|
112
|
-
h2. Starting event loop
|
112
|
+
h2. Starting the event loop and connecting in standalone applications
|
113
113
|
|
114
114
|
h3. EventMachine event loop
|
115
115
|
|
116
|
-
amqp gem uses "EventMachine":http://rubyeventmachine.com under the hood and needs EventMachine
|
117
|
-
event loop to be running in order to connect to AMQP broker or send any data. This means that
|
118
|
-
before connecting to AMQP broker, we need to _start EventMachine reactor_ (get the event loop
|
116
|
+
The amqp gem uses "EventMachine":http://rubyeventmachine.com under the hood and needs an EventMachine
|
117
|
+
event loop to be running in order to connect to an AMQP broker or to send any data. This means that
|
118
|
+
before connecting to an AMQP broker, we need to _start the EventMachine reactor_ (get the event loop
|
119
119
|
going). Here is how to do it:
|
120
120
|
|
121
121
|
<pre>
|
@@ -128,14 +128,14 @@ end
|
|
128
128
|
</code>
|
129
129
|
</pre>
|
130
130
|
|
131
|
-
"EventMachine.run":http://eventmachine.rubyforge.org/EventMachine.html#M000461 will block current thread until event loop is stopped.
|
132
|
-
Standalone applications often can afford
|
133
|
-
recommended way.
|
131
|
+
"EventMachine.run":http://eventmachine.rubyforge.org/EventMachine.html#M000461 will block the current thread until the event loop is stopped.
|
132
|
+
Standalone applications often can afford to start the event loop on the main thread. If you have no experience with threading, this is a
|
133
|
+
recommended way to proceed.
|
134
134
|
|
135
135
|
|
136
136
|
h3. Using AMQP.connect with a block
|
137
137
|
|
138
|
-
Once event loop is running, {AMQP.connect} method will attempt to connect to the broker. It can be used in two ways. Here is the
|
138
|
+
Once the event loop is running, the {AMQP.connect} method will attempt to connect to the broker. It can be used in two ways. Here is the
|
139
139
|
first one:
|
140
140
|
|
141
141
|
<pre>
|
@@ -151,13 +151,13 @@ end
|
|
151
151
|
</code>
|
152
152
|
</pre>
|
153
153
|
|
154
|
-
{AMQP.connect} takes a block that will be executed as soon as AMQP connection is open
|
155
|
-
authentication
|
154
|
+
{AMQP.connect} takes a block that will be executed as soon as the AMQP connection is open. In order for a connection to be opened a TCP connection has to be set up,
|
155
|
+
authentication has to succeed, and the broker and client need to complete negotiation of connection parameters like max frame size.
|
156
156
|
|
157
157
|
|
158
158
|
h3. Using AMQP.connect without a callback
|
159
159
|
|
160
|
-
|
160
|
+
An alternative way of connecting is this:
|
161
161
|
|
162
162
|
<pre>
|
163
163
|
<code>
|
@@ -166,14 +166,14 @@ require "amqp"
|
|
166
166
|
EventMachine.run do
|
167
167
|
# using AMQP.connect with a block
|
168
168
|
client = AMQP.connect(:host => "hub.megacorp.internal", :username => "hedgehog", :password => "t0ps3kr3t")
|
169
|
-
# connection is not yet open, however, amqp gem will delay
|
170
|
-
#
|
169
|
+
# connection is not yet open, however, amqp gem will delay channel
|
170
|
+
# operations until after the connection is open. Bear in mind that
|
171
171
|
# amqp gem cannot solve every possible race condition so be careful
|
172
172
|
end
|
173
173
|
</code>
|
174
174
|
</pre>
|
175
175
|
|
176
|
-
If you do not need to assign returned value to a variable, "block version" is recommended because it eliminates issues that may
|
176
|
+
If you do not need to assign the returned value to a variable, then the "block version" is recommended because it eliminates issues that may
|
177
177
|
arise from attempts to use a connection object that is not fully opened yet. For example, handling of authentication failures is simpler
|
178
178
|
with the block version, as we will see in the following sections.
|
179
179
|
|
@@ -181,7 +181,7 @@ with the block version, as we will see in the following sections.
|
|
181
181
|
|
182
182
|
h3. Using AMQP.start
|
183
183
|
|
184
|
-
EventMachine.run and {AMQP.connect} with a block is such a common combination that amqp gem provides a shortcut:
|
184
|
+
EventMachine.run and {AMQP.connect} with a block is such a common combination that the amqp gem provides a shortcut:
|
185
185
|
|
186
186
|
<pre>
|
187
187
|
<code>
|
@@ -194,20 +194,20 @@ end
|
|
194
194
|
</pre>
|
195
195
|
|
196
196
|
As these examples demonstrate, {AMQP.connect} and {AMQP.start} accept either a Hash of connection options or a connection URI string.
|
197
|
-
See reference documentation for each method to learn all the options they accept and what the default values are.
|
197
|
+
See the reference documentation for each method to learn all of the options that they accept and what the default values are.
|
198
198
|
|
199
199
|
|
200
200
|
h3. On Thread#sleep use
|
201
201
|
|
202
|
-
When not passing a block to {AMQP.connect}, it is tempting to "give connection some time to
|
203
|
-
running event loop in a separate thread,
|
204
|
-
blocking
|
202
|
+
When not passing a block to {AMQP.connect}, it is tempting to "give the connection some time to become established" by using Thread#sleep. Unless you are
|
203
|
+
running the event loop in a separate thread, please do not do this. Thread#sleep blocks the current thread so that if the event loop is running in the current thread,
|
204
|
+
blocking the thread _will also block the event loop_. *When the event loop is blocked, no data is sent or received, so the connection does not proceed.*
|
205
205
|
|
206
206
|
|
207
207
|
h3. Detecting TCP connection failures
|
208
208
|
|
209
|
-
When applications connect to the broker, they need to handle connection failures. Networks are not 100% reliable
|
210
|
-
like Chef or Puppet misconfigurations happen
|
209
|
+
When applications connect to the broker, they need to handle connection failures. Networks are not 100% reliable and even with modern system configuration tools,
|
210
|
+
like "Chef":http://http://www.opscode.com/chef or "Puppet":http://http://www.puppetlabs.com, misconfigurations can happen. Also, the broker might be down for some reason. Ideally, error detection should happen as early as possible. There are two ways of detecting
|
211
211
|
TCP connection failure, the first one is to catch an exception:
|
212
212
|
|
213
213
|
<pre>
|
@@ -240,11 +240,11 @@ end
|
|
240
240
|
</code>
|
241
241
|
</pre>
|
242
242
|
|
243
|
-
{AMQP.connect} (and {AMQP.start}) will raise {AMQP::TCPConnectionFailed} if connection fails. Code that catches
|
244
|
-
about the issue or use retry to execute begin block one more time. Because initial connection failures are due to misconfiguration or network outage, reconnection
|
245
|
-
to the same endpoint (hostname, port, vhost combination) will result in the same
|
243
|
+
{AMQP.connect} (and {AMQP.start}) will raise {AMQP::TCPConnectionFailed} if the connection fails. Code that catches the error can write to a log
|
244
|
+
about the issue or use retry to execute the begin block one more time. Because initial connection failures are due to misconfiguration or network outage, reconnection
|
245
|
+
to the same endpoint (hostname, port, vhost combination) will result in the same error over and over again. TBD: failover, connection to the cluster.
|
246
246
|
|
247
|
-
|
247
|
+
An alternative way of handling connection failure is with an _errback_ (a callback for a specific kind of error):
|
248
248
|
|
249
249
|
<pre>
|
250
250
|
<code>
|
@@ -274,9 +274,9 @@ end
|
|
274
274
|
</code>
|
275
275
|
</pre>
|
276
276
|
|
277
|
-
:on_tcp_connection_failure option accepts any object that responds to #call.
|
277
|
+
the ":on_tcp_connection_failure" option accepts any object that responds to #call.
|
278
278
|
|
279
|
-
If you connect to the broker from
|
279
|
+
If you connect to the broker from code in a class (as opposed to top-level scope in a script), Object#method can be used to pass an object method as a handler
|
280
280
|
instead of a Proc.
|
281
281
|
|
282
282
|
TBD: provide an example
|
@@ -284,7 +284,7 @@ TBD: provide an example
|
|
284
284
|
|
285
285
|
h3. Detecting authentication failures
|
286
286
|
|
287
|
-
|
287
|
+
A connection may also fail due to authentication failure. Handling authentication failure is very similar to handling an initial TCP
|
288
288
|
connection failure:
|
289
289
|
|
290
290
|
<pre>
|
@@ -354,21 +354,21 @@ end
|
|
354
354
|
</code>
|
355
355
|
</pre>
|
356
356
|
|
357
|
-
In case you
|
358
|
-
simply close TCP connection without sending any more data when an exception
|
359
|
-
is open. In practice, however, when broker closes TCP connection
|
357
|
+
In case you are wondering why the callback name has "possible" in it, {http://bit.ly/mTr1YN AMQP 0.9.1 spec} requires broker implementations to
|
358
|
+
simply close the TCP connection without sending any more data when an exception, such as authentication failure, occurs before the AMQP connection
|
359
|
+
is open. In practice, however, when a broker closes a TCP connection after a successful TCP connection has been established but before an AMQP connection is open,
|
360
360
|
it means that authentication has failed.
|
361
361
|
|
362
362
|
|
363
363
|
|
364
|
-
h2. Starting event loop
|
364
|
+
h2. Starting the event loop and connecting in Web applications (Ruby on Rails, Sinatra, Merb, Rack)
|
365
365
|
|
366
|
-
Web applications are different from standalone applications in that main thread is occupied by Web/application server like Unicorn
|
367
|
-
or Thin, so you need to start EventMachine reactor before you attempt to use {AMQP.connect}.
|
368
|
-
In a Ruby on Rails
|
369
|
-
For Sinatra and pure Rack applications, place it next to other configuration code.
|
366
|
+
Web applications are different from standalone applications in that the main thread is occupied by a Web/application server like Unicorn
|
367
|
+
or Thin, so you need to start the EventMachine reactor before you attempt to use {AMQP.connect}.
|
368
|
+
In a Ruby on Rails application, probably the best place for this is in the initializer (like config/initializers/amqp.rb). For Merb applications it is config/init.rb.
|
369
|
+
For Sinatra and pure Rack applications, place it next to the other configuration code.
|
370
370
|
|
371
|
-
Next we are going to discuss issues specific to particular Web servers.
|
371
|
+
Next, we are going to discuss issues specific to particular Web servers.
|
372
372
|
|
373
373
|
|
374
374
|
|
@@ -377,18 +377,18 @@ h3. Using Ruby amqp gem with Unicorn
|
|
377
377
|
h4. Unicorn is a pre-forking server
|
378
378
|
|
379
379
|
"Unicorn":http://unicorn.bogomips.org is a pre-forking server. That means it forks worker processes that serve HTTP requests. The "fork(2)":http://en.wikipedia.org/wiki/Fork_(operating_system) system call
|
380
|
-
has several gotchas associated with it, two of which affect EventMachine and "Ruby amqp gem":http://github.com/ruby-amqp/amqp:
|
380
|
+
has several gotchas associated with it, two of which affect EventMachine and the "Ruby amqp gem":http://github.com/ruby-amqp/amqp:
|
381
381
|
|
382
382
|
* Unintentional file descriptor sharing
|
383
|
-
* The fact that "forked child process only inherits one thread":http://bit.ly/fork-and-threads
|
383
|
+
* The fact that a "forked child process only inherits one thread":http://bit.ly/fork-and-threads and therefore the EventMachine thread is not inherited
|
384
384
|
|
385
|
-
To avoid both problems, start EventMachine reactor and AMQP connection *after* master process forks workers.
|
386
|
-
|
385
|
+
To avoid both problems, start the EventMachine reactor and AMQP connection *after* the master process forks workers. The master Unicorn process never serves HTTP requests and usually
|
386
|
+
does not need to hold an AMQP connection. Next, let us see how to spin up the EventMachine reactor and connect to the broker after Unicorn forks a worker.
|
387
387
|
|
388
388
|
|
389
|
-
h4. Starting EventMachine reactor and connecting to the broker after Unicorn forks worker processes
|
389
|
+
h4. Starting the EventMachine reactor and connecting to the broker after Unicorn forks worker processes
|
390
390
|
|
391
|
-
Unicorn lets you specify a configuration file to use. In that file
|
391
|
+
Unicorn lets you specify a configuration file to use. In that file you define a callback that Unicorn runs after it forks worker process(es):
|
392
392
|
|
393
393
|
<pre>
|
394
394
|
<code>
|
@@ -427,38 +427,53 @@ end
|
|
427
427
|
</code>
|
428
428
|
</pre>
|
429
429
|
|
430
|
-
In the example above we start EventMachine reactor in a separate thread, block current thread for 1 second to let event loop spin up and then
|
431
|
-
connect to AMQP broker on the next event loop tick. Publishing several warmup messages on boot is a good idea because it
|
432
|
-
|
430
|
+
In the example above we start the EventMachine reactor in a separate thread, block the current thread for 1 second to let the event loop spin up and then
|
431
|
+
connect to the AMQP broker on the next event loop tick. Publishing several warmup messages on boot is a good idea because it
|
432
|
+
allows the early detection of issues that forking may cause.
|
433
433
|
|
434
|
-
Note that configuration file can easily be used in development environments
|
435
|
-
it gives you exactly the same application boot behavior as in QA and production environments
|
434
|
+
Note that a configuration file can easily be used in development environments because, other than the fact that Unicorn runs in the foreground,
|
435
|
+
it gives you exactly the same application boot behavior as in QA and production environments.
|
436
436
|
|
437
|
-
An "example Ruby on Rails application that uses Ruby amqp gem and Unicorn":http://bit.ly/ruby-amqp-gem-example-with-ruby-on-rails-and-unicorn is available on GitHub.
|
437
|
+
An "example Ruby on Rails application that uses the Ruby amqp gem and Unicorn":http://bit.ly/ruby-amqp-gem-example-with-ruby-on-rails-and-unicorn is available on GitHub.
|
438
438
|
|
439
439
|
|
440
440
|
|
441
441
|
|
442
|
-
h3. Using Ruby amqp gem with Passenger
|
442
|
+
h3. Using the Ruby amqp gem with Passenger
|
443
443
|
|
444
|
-
Phusion Passenger is a pre-forking server,
|
445
|
-
processes. Passenger documentation has "a section":http://bit.ly/passenger-forking-gotchas that explains how to avoid problems related to the behavior of fork(2) system
|
444
|
+
"Phusion Passenger":http://www.modrails.com is also a pre-forking server, and just as with Unicorn, the EventMachine reactor and AMQP connection should be started *after* it forks worker
|
445
|
+
processes. The Passenger documentation has "a section":http://bit.ly/passenger-forking-gotchas that explains how to avoid problems related to the behavior of the fork(2) system
|
446
446
|
call, namely:
|
447
447
|
|
448
448
|
* Unintentional file descriptor sharing
|
449
|
-
* The fact that "forked child process only inherits one thread":http://bit.ly/fork-and-threads
|
449
|
+
* The fact that a "forked child process only inherits one thread":http://bit.ly/fork-and-threads and therefore the EventMachine thread is not inherited
|
450
450
|
|
451
|
-
|
451
|
+
h4. Using an event handler to spawn one amqp connection per worker
|
452
452
|
|
453
|
+
Passenger provides a hook that you should use for spawning AMQP connections:
|
453
454
|
|
455
|
+
<pre>
|
456
|
+
<code>
|
457
|
+
if defined?(PhusionPassenger) # otherwise it breaks rake commands if you put this in an initializer
|
458
|
+
PhusionPassenger.on_event(:starting_worker_process) do |forked|
|
459
|
+
if forked
|
460
|
+
# We're in a smart spawning mode
|
461
|
+
# Now is a good place to connect to the broker
|
462
|
+
end
|
463
|
+
end
|
464
|
+
end
|
465
|
+
</code>
|
466
|
+
</pre>
|
467
|
+
|
468
|
+
Basically, the recommended default smart spawn mode works exactly the same as in Unicorn (with all of the same common pitfalls). An "example application":http://bit.ly/ruby-amqp-gem-example-with-ruby-on-rails-and-passenger is available on github.
|
454
469
|
|
455
|
-
h3. Using Ruby amqp gem with Thin and Goliath
|
470
|
+
h3. Using the Ruby amqp gem with Thin and Goliath
|
456
471
|
|
457
|
-
h4. Thin and Goliath start EventMachine reactor for you, but there is a little nuance
|
472
|
+
h4. Thin and Goliath start the EventMachine reactor for you, but there is a little nuance
|
458
473
|
|
459
|
-
If you use "Thin":http://code.macournoyer.com/thin/ or "Goliath":https://github.com/postrank-labs/goliath/, you are all set
|
460
|
-
There is no need to start EventMachine reactor. However, depending on
|
461
|
-
EventMachine reactor start may be slightly delayed. To
|
474
|
+
If you use "Thin":http://code.macournoyer.com/thin/ or "Goliath":https://github.com/postrank-labs/goliath/, you are all set because those two servers use EventMachine under the hood.
|
475
|
+
There is no need to start the EventMachine reactor. However, depending on the application server, its version, the version of the framework and Rack middleware being used,
|
476
|
+
EventMachine reactor start may be slightly delayed. To overcome this potential difficulty, use EventMachine.next_tick to delay connection until after the reactor is actually running:
|
462
477
|
|
463
478
|
<pre>
|
464
479
|
<code>
|
@@ -466,33 +481,34 @@ EventMachine.next_tick { AMQP.connect(...) }
|
|
466
481
|
</code>
|
467
482
|
</pre>
|
468
483
|
|
469
|
-
So in case EventMachine reactor
|
470
|
-
Thin and Goliath are not pre-forking servers so there is no need to re-establish connection
|
484
|
+
So, in case the EventMachine reactor is not yet running on server/application boot, the connection will not fail but will instead wait for the reactor to start.
|
485
|
+
Thin and Goliath are not pre-forking servers so there is no need to re-establish the connection as you do with Unicorn and Passenger.
|
471
486
|
|
472
487
|
|
473
488
|
|
474
489
|
|
475
|
-
h2. If it just
|
490
|
+
h2. If it just does not work: troubleshooting
|
476
491
|
|
477
|
-
If you read this guide
|
492
|
+
If you have read this guide and your issue is still unresolved, check our {file:docs/Troubleshooting.textile Troubleshooting guide} before asking on the mailing list.
|
478
493
|
|
479
494
|
|
480
495
|
|
481
496
|
|
482
497
|
h2. What to read next
|
483
498
|
|
484
|
-
* {file:docs/Queues.textile Queues}
|
499
|
+
* {file:docs/Queues.textile Working With Queues}. This guide focuses on features consumer applications use heavily.
|
500
|
+
* {file:docs/Exchanges.textile Working With Exchanges}. This guide focuses on features producer applications use heavily.
|
485
501
|
* {file:docs/ErrorHandling.textile Error handling}
|
486
|
-
* {file:docs/ConnectionEncryptionWithTLS.textile Using TLS (SSL)} (if you want to use SSL encrypted connection to the broker)
|
502
|
+
* {file:docs/ConnectionEncryptionWithTLS.textile Using TLS (SSL)} (if you want to use an SSL encrypted connection to the broker)
|
487
503
|
|
488
504
|
|
489
505
|
h2. Tell us what you think!
|
490
506
|
|
491
|
-
Please take a moment
|
492
|
-
what was unclear
|
493
|
-
key to making documentation better.
|
507
|
+
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
|
508
|
+
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
|
509
|
+
key to making the documentation better.
|
494
510
|
|
495
|
-
If
|
511
|
+
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
|
496
512
|
|
497
513
|
|
498
514
|
|