right_amqp 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2012 RightScale, Inc.
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ 'Software'), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
17
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
18
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
19
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
20
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,216 @@
1
+ = RightAMQP
2
+
3
+ = DESCRIPTION
4
+
5
+ == Synopsis
6
+
7
+ RightAMQP provides a high availability client for interfacing with the
8
+ RightScale RabbitMQ broker using the AMQP protocol. The AMQP version on which
9
+ this gem is based is 0.6.7 but beyond that it contains a number of bug fixes and
10
+ enhancements including reconnect, message return, heartbeat, and UTF-8 support.
11
+ The high availability is achieved by maintaining multiple broker connections
12
+ such that failed connections automatically reconnect and only connected
13
+ brokers are used when routing a message. Although the HABrokerClient class
14
+ is the intended primary means for accessing RabbitMQ services with this gem,
15
+ alternatively the underlying AMQP services may be used directly.
16
+
17
+ Refer to the wiki (https://github.com/rightscale/right_amqp/wiki) for up-to-date
18
+ documentation.
19
+
20
+ Also use the built-in issues tracker (https://github.com/rightscale/right_amqp/issues)
21
+ to report issues.
22
+
23
+ == Interface
24
+
25
+ The focus here is on the interface provided by the RightAMQP::HABrokerClient class
26
+ for connecting to one or more brokers in high availability fashion, subscribing
27
+ to queues or exchanges on each of the brokers, publishing messages using any of
28
+ the brokers, and monitoring the status of the broker connections.
29
+
30
+ The namespace for high availability access is RightAMQP. The namespace for low
31
+ level AMQP support remains AMQP. Both rely on eventmachine for task management.
32
+
33
+ === Connecting
34
+
35
+ Creating an HABrokerClient object causes connections to one or more brokers
36
+ to be created, e.g.,
37
+
38
+ b = RightAMQP::HABrokerClient.new(serializer, :user => 'user', :pass => 'secret',
39
+ :vhost => '/abc', :host => 'broker0.com,broker1.com')
40
+
41
+ would result in a connection to the brokers with domain names broker0.com and broker1.com
42
+ using the specified :user and :pass as RabbitMQ credentials and :vhost as the namespace
43
+ in which to operate. See the detailed code documentation for other configuration options.
44
+
45
+ To know when a connection has been established, #connection_status is used, e.g.,
46
+
47
+ b.connection_status(:one_off => 60) do |status|
48
+ if status == :connected
49
+ # Perform other application setup including subscribing to queues
50
+ elsif status == :failed
51
+ puts "Could not connect to any brokers"
52
+ EM.stop
53
+ end
54
+ end
55
+
56
+ with the :on_off option indicating that only seeking to be notified of the
57
+ first connection status change and in this case only willing to wait 60 seconds.
58
+
59
+ === Reconnecting, Heartbeat, and Status Updates
60
+
61
+ If a broker connection is lost, the HABrokerClient will automatically attempt
62
+ to reconnect on the interval specified with the :reconnect_interval option.
63
+ As further protection against lost connections the :heartbeat option may be
64
+ specified as the interval between AMQP keep alive messages being sent between
65
+ the application and the brokers. This is useful in firewall environments.
66
+
67
+ The #connection_status method used above to detect the initial connection may
68
+ also be used to monitor connectivity throughout the life of the application, e.g.,
69
+
70
+ b.connection_status do |status|
71
+ puts "Status changed to #{status}"
72
+ end
73
+
74
+ For this request the status is only reported for all the brokers as a whole,
75
+ i.e., a :disconnected status means that have lost connectivity to all
76
+ brokers, and then a subsequent :connected status means that have regained
77
+ connection to at least one broker. There are finer grain controls by
78
+ indicating the specific :brokers of interest
79
+
80
+ === Subscribing, Declaring, and Unsubscribing
81
+
82
+ To receive messages via this interface, one must subscribe to one or more
83
+ AMQP queues or exchanges. This is done with the #subscribe method, e.g.,
84
+
85
+ queue = {:name => 'my_queue', :options => {:durable => true}}
86
+ broker_ids = @broker.subscribe(queue, exchange = nil, :ack => true) do |id, msg|
87
+ puts "Received packet #{msg.inspect} via broker #{id}"
88
+ end
89
+
90
+ causes the queue named "my_queue" to be created on all currently connected
91
+ brokers in the AMQP durable fashion, meaning that it is preserved across
92
+ restarts of the broker. The #subscribe call returns the ids of the brokers
93
+ to which the subscribe request was submitted. The :ack option indicates to
94
+ explicitly acknowledge each message as soon as it is received, rather than use
95
+ implicit ack handling. The provided block is executed each time a message is
96
+ received on the queue.
97
+
98
+ If serialization is configured, the message delivered here is after applying
99
+ the serializers #load method; otherwise the message is exactly the bytes
100
+ that were placed in the queue.
101
+
102
+ To unsubscribe from a queue the #unsubscribe method is used.
103
+
104
+ To simply create a queue or exchange without subscribing to it to receive
105
+ messages, use #declare.
106
+
107
+ === Publishing
108
+
109
+ To publish a message to a queue or exchange, the #publish method is used, e.g.,
110
+
111
+ queue = {:type => :queue, :name => "request", :options => {:no_declare => true}}
112
+ broker_ids = b.publish(queue, message, :persistent => true, :mandatory => true)
113
+
114
+ causes the specified message to be published to the queue named "request".
115
+ The :no_declare option is specified to keep AMQP from attempting to create
116
+ the exchange before publishing to it. On the publish the :persistent option
117
+ indicates that all attempts are to be made to preserve the message if the broker
118
+ is stopped or crasheds, but this is not a guarantee. The :mandatory option
119
+ indicates that the broker is to return the message if the specified queue
120
+ does not exist or is not being consumed, i.e., subscribed to by another
121
+ application.
122
+
123
+ === Serialization
124
+
125
+ If a serializer is supplied when the HABrokerClient is constructed, its #load
126
+ method is applied to all messages received from a broker, and its #dump method
127
+ is applied to all messages that are published. Even if a serializer is specified
128
+ it is possible to specify :no_unserialize for a particular subscription or
129
+ :no_serialize for a message being published.
130
+
131
+ If no serializer is supplied, individual messages published or received are
132
+ not logged. Further a serialized message is probed for the existence of certain
133
+ properties, like :token, :type, and :from, and these are tracked so that if
134
+ a message is returned as undeliverable, this information can be provided on
135
+ the #non_delivery callback.
136
+
137
+ === Message Return and Non-Delivery Callback
138
+
139
+ If a broker returns a message because it cannot be delivered to the intended
140
+ recipient, e.g., because the option :mandatory was set and there are no consumers
141
+ or the broker is in the process of shutting down, the HABrokerClient attempts
142
+ to deliver the message using another broker in the configured set.
143
+
144
+ When it runs out of connected brokers to attempt the delivery, it declares
145
+ the message and undeliverable. In this case it also executes the block, if any,
146
+ supplied by the application via the #non_delivery method. The data supplied
147
+ includes the reason, type, token, from, and to, with the type, token, and from
148
+ values being nil unless they could be extracted from the message being published.
149
+
150
+ The possible reasons for non-delivery are:
151
+ "NO_ROUTE" - queue does not exist
152
+ "NO_CONSUMERS" - queue exists but it has no consumers
153
+ "ACCESS_REFUSED" - queue not usable because broker is in the process of stopping service
154
+
155
+ === Closing
156
+
157
+ When all connections to the brokers are to be closed, the #close method is used
158
+
159
+ === Identities
160
+
161
+ A broker is given an identity of the form "rs-broker-<hostname>-<port>" where
162
+ <hostname> is its host name, e.g., broker1.com, and <port> is the TCP port number
163
+ used (5672 by default). This is used in the interface when a specific broker
164
+ needs to be referred to, e.g., when communicating status.
165
+
166
+ === Error Callbacks
167
+
168
+ When constructing the HABrokerClient the :exception_callback option can be
169
+ specified to define the Proc to be activated on exception events. In addition
170
+ an :exception_on_receive_callback Proc can be specified for activation when
171
+ a message cannot be received.
172
+
173
+ === Logging
174
+
175
+ To enable logging in the HABrokerClient set RightSupport::Log::Mixin.default_logger
176
+ to the logger in use that supports the standard ruby Logger interface. Logging
177
+ can be disabled on individual #subscribe and #publish requests with :no_log.
178
+ By default each message received or published is logged unless no serializer
179
+ is supplied.
180
+
181
+ Detailed AMQP logging can be enabled by setting AMQP.logging = true.
182
+
183
+ === Status and Stats
184
+
185
+ The current status of all brokers can be obtained with the #status method.
186
+ The operation statistics are obtained with the #stats method. This method
187
+ has an option for resetting the statistics.
188
+
189
+ = ADDITIONAL RESOURCES
190
+
191
+ * [1] RabbitMQ is http://www.rabbitmq.com/documentation.html
192
+
193
+ = LICENSE
194
+
195
+ <b>RightAMQP</b>
196
+
197
+ Copyright:: Copyright (c) 2012 RightScale, Inc.
198
+
199
+ Permission is hereby granted, free of charge, to any person obtaining
200
+ a copy of this software and associated documentation files (the
201
+ 'Software'), to deal in the Software without restriction, including
202
+ without limitation the rights to use, copy, modify, merge, publish,
203
+ distribute, sublicense, and/or sell copies of the Software, and to
204
+ permit persons to whom the Software is furnished to do so, subject to
205
+ the following conditions:
206
+
207
+ The above copyright notice and this permission notice shall be
208
+ included in all copies or substantial portions of the Software.
209
+
210
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
211
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
212
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
213
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
214
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
215
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
216
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,70 @@
1
+ #-- -*-ruby-*-
2
+ # Copyright: Copyright (c) 2012 RightScale, Inc.
3
+ #
4
+ # Permission is hereby granted, free of charge, to any person obtaining
5
+ # a copy of this software and associated documentation files (the
6
+ # 'Software'), to deal in the Software without restriction, including
7
+ # without limitation the rights to use, copy, modify, merge, publish,
8
+ # distribute, sublicense, and/or sell copies of the Software, and to
9
+ # permit persons to whom the Software is furnished to do so, subject to
10
+ # the following conditions:
11
+ #
12
+ # The above copyright notice and this permission notice shall be
13
+ # included in all copies or substantial portions of the Software.
14
+ #
15
+ # THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
16
+ # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
17
+ # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
18
+ # IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
19
+ # CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
20
+ # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
21
+ # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
22
+ #++
23
+
24
+ require 'rubygems'
25
+ require 'bundler/setup'
26
+
27
+ require 'rake'
28
+ require 'rdoc/task'
29
+ require 'rubygems/package_task'
30
+ require 'rake/clean'
31
+ require 'rspec/core/rake_task'
32
+ require 'bacon'
33
+
34
+ desc "Run unit tests"
35
+ task :default => 'spec'
36
+
37
+ desc "Run unit tests"
38
+ RSpec::Core::RakeTask.new do |t|
39
+ t.rspec_opts = ['--options', "\"#{File.dirname(__FILE__)}/spec/spec.opts\""]
40
+ end
41
+
42
+ namespace :spec do
43
+ desc "Regenerate AMQP unit test code"
44
+ task :codegen do
45
+ sh 'ruby protocol/codegen.rb > lib/right_amqp/amqp/spec.rb'
46
+ sh 'ruby lib/right_amqp/amqp/spec.rb'
47
+ end
48
+
49
+ desc "Run AMQP unit tests"
50
+ task :amqp do
51
+ sh 'bacon lib/right_amqp/amqp.rb'
52
+ end
53
+ end
54
+
55
+ desc 'Generate documentation for the right_amqp gem'
56
+ Rake::RDocTask.new(:rdoc) do |rdoc|
57
+ rdoc.rdoc_dir = 'doc/rdocs'
58
+ rdoc.title = 'RightAMQP'
59
+ rdoc.options << '--line-numbers' << '--inline-source'
60
+ rdoc.rdoc_files.include('README.rdoc')
61
+ rdoc.rdoc_files.include('lib/**/*.rb')
62
+ end
63
+ CLEAN.include('doc/rdocs')
64
+
65
+ desc "Build right_amqp gem"
66
+ Gem::PackageTask.new(Gem::Specification.load("right_amqp.gemspec")) do |package|
67
+ package.need_zip = true
68
+ package.need_tar = true
69
+ end
70
+ CLEAN.include('pkg')
@@ -0,0 +1,28 @@
1
+ #
2
+ # Copyright (c) 2012 RightScale Inc
3
+ #
4
+ # Permission is hereby granted, free of charge, to any person obtaining
5
+ # a copy of this software and associated documentation files (the
6
+ # "Software"), to deal in the Software without restriction, including
7
+ # without limitation the rights to use, copy, modify, merge, publish,
8
+ # distribute, sublicense, and/or sell copies of the Software, and to
9
+ # permit persons to whom the Software is furnished to do so, subject to
10
+ # the following conditions:
11
+ #
12
+ # The above copyright notice and this permission notice shall be
13
+ # included in all copies or substantial portions of the Software.
14
+ #
15
+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
16
+ # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
17
+ # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
18
+ # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
19
+ # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
20
+ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
21
+ # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
22
+
23
+ require 'digest/md5'
24
+ require 'right_support'
25
+
26
+ require 'right_amqp/mq'
27
+ require 'right_amqp/amqp'
28
+ require 'right_amqp/ha_client'
@@ -0,0 +1,115 @@
1
+ require File.expand_path('../ext/em', __FILE__)
2
+ require File.expand_path('../ext/blankslate', __FILE__)
3
+
4
+ %w[ version buffer spec protocol frame client ].each do |file|
5
+ require File.expand_path("../amqp/#{file}", __FILE__)
6
+ end
7
+
8
+ module AMQP
9
+ class << self
10
+ @logging = false
11
+ attr_accessor :logging
12
+ attr_reader :conn, :closing
13
+ alias :closing? :closing
14
+ alias :connection :conn
15
+ end
16
+
17
+ def self.connect *args
18
+ Client.connect *args
19
+ end
20
+
21
+ def self.settings
22
+ @settings ||= {
23
+ # server address
24
+ :host => '127.0.0.1',
25
+ :port => PORT,
26
+
27
+ # login details
28
+ :user => 'guest',
29
+ :pass => 'guest',
30
+ :vhost => '/',
31
+
32
+ # connection timeout
33
+ :timeout => nil,
34
+
35
+ # logging
36
+ :logging => false,
37
+
38
+ # ssl
39
+ :ssl => false
40
+ }
41
+ end
42
+
43
+ # Must be called to startup the connection to the AMQP server.
44
+ #
45
+ # The method takes several arguments and an optional block.
46
+ #
47
+ # This takes any option that is also accepted by EventMachine::connect.
48
+ # Additionally, there are several AMQP-specific options.
49
+ #
50
+ # * :user => String (default 'guest')
51
+ # The username as defined by the AMQP server.
52
+ # * :pass => String (default 'guest')
53
+ # The password for the associated :user as defined by the AMQP server.
54
+ # * :vhost => String (default '/')
55
+ # The virtual host as defined by the AMQP server.
56
+ # * :timeout => Numeric (default nil)
57
+ # Measured in seconds.
58
+ # * :logging => true | false (default false)
59
+ # Toggle the extremely verbose logging of all protocol communications
60
+ # between the client and the server. Extremely useful for debugging.
61
+ #
62
+ # AMQP.start do
63
+ # # default is to connect to localhost:5672
64
+ #
65
+ # # define queues, exchanges and bindings here.
66
+ # # also define all subscriptions and/or publishers
67
+ # # here.
68
+ #
69
+ # # this block never exits unless EM.stop_event_loop
70
+ # # is called.
71
+ # end
72
+ #
73
+ # Most code will use the MQ api. Any calls to MQ.direct / MQ.fanout /
74
+ # MQ.topic / MQ.queue will implicitly call #start. In those cases,
75
+ # it is sufficient to put your code inside of an EventMachine.run
76
+ # block. See the code examples in MQ for details.
77
+ #
78
+ def self.start *args, &blk
79
+ begin
80
+ EM.run{
81
+ @conn ||= connect *args
82
+ @conn.callback(&blk) if blk
83
+ @conn
84
+ }
85
+ rescue Exception => e
86
+ @conn = nil
87
+ raise e
88
+ end
89
+ end
90
+
91
+ class << self
92
+ alias :run :start
93
+ end
94
+
95
+ def self.stop
96
+ if @conn and not @closing
97
+ @closing = true
98
+ @conn.close{
99
+ yield if block_given?
100
+ @conn = nil
101
+ @closing = false
102
+ }
103
+ end
104
+ end
105
+
106
+ def self.fork workers
107
+ EM.fork(workers) do
108
+ # clean up globals in the fork
109
+ Thread.current[:mq] = nil
110
+ AMQP.instance_variable_set('@conn', nil)
111
+
112
+ yield
113
+ end
114
+ end
115
+ end