qup 1.2.0 → 1.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,12 +1,45 @@
1
1
  = Changelog
2
2
 
3
+ == Version 1.4.0 - 2012-10-31
4
+
5
+ * Switch to 'kjess' as the Kestrel client.
6
+
7
+ == Version 1.3.6 - 2012-09-06
8
+
9
+ * Pass options to the Adapters (thanks rafer)
10
+
11
+ == Version 1.3.5 - 2012-09-06
12
+
13
+ * Rename 'Drainer' to 'BatchConsumer' and add documentation (issue #11, thanks rafer)
14
+
15
+ == Version 1.3.4 - 2012-08-23
16
+
17
+ * Add 'Drainer' class (issue #10, thanks rafer)
18
+
19
+ == Version 1.3.3 - 2012-08-06
20
+
21
+ * Fix newline bug (issue #9, thanks rafer)
22
+
23
+ == Version 1.3.2 - 2012-07-17
24
+
25
+ * Make Queue#consume non-blocking everywhere (issue #7, thanks rafer)
26
+
27
+ == Version 1.3.1 - 2012-06-21
28
+
29
+ * Add Consumer#depth
30
+ * Check if the data payload is marshalled data and unmarshal it if it is.
31
+
32
+ == Version 1.2.2 - 2012-05-17
33
+
34
+ * Use the thrift interface to kestrel instead of the memcache interface
35
+
3
36
  == Version 1.2.0 - 2012-03-17
4
37
 
5
- * Persistent subscriptions for the Redis Adapter (thanks aniero)
38
+ * Persistent subscriptions for the Redis Adapter (issue #2, thanks aniero)
6
39
 
7
40
  == Version 1.1.0 - 2012-03-12
8
41
 
9
- * Addition of a Redis Adapter (thanks aniero)
42
+ * Addition of a Redis Adapter (issue #1, thanks aniero)
10
43
 
11
44
  == Version 1.0.0 - 2012-03-10
12
45
 
@@ -18,6 +18,8 @@ lib/qup/adapter/redis.rb
18
18
  lib/qup/adapter/redis/connection.rb
19
19
  lib/qup/adapter/redis/queue.rb
20
20
  lib/qup/adapter/redis/topic.rb
21
+ lib/qup/backoff_sleeper.rb
22
+ lib/qup/batch_consumer.rb
21
23
  lib/qup/consumer.rb
22
24
  lib/qup/message.rb
23
25
  lib/qup/producer.rb
@@ -39,6 +41,8 @@ spec/qup/adapter/redis/topic_spec.rb
39
41
  spec/qup/adapter/redis_context.rb
40
42
  spec/qup/adapter/redis_spec.rb
41
43
  spec/qup/adapter_spec.rb
44
+ spec/qup/backoff_sleeper_sleeper_spec.rb
45
+ spec/qup/batch_consumer_spec.rb
42
46
  spec/qup/consumer_spec.rb
43
47
  spec/qup/message_spec.rb
44
48
  spec/qup/producer_spec.rb
@@ -99,7 +99,8 @@ install additional gems. At the current moment, these are the supported
99
99
  messaging backends.
100
100
 
101
101
  * Qup::Adapter::Maildir - built in and uses the 'maildir' gem
102
- * Qup::Adapter::Kestrel - uses the 'kestrel-client' gem
102
+ * Qup::Adapter::Kestrel - uses the 'kjess' gem
103
+ * Qup::Adapter::Redis - uses the 'redis' gem
103
104
 
104
105
  == INSTALL
105
106
 
@@ -123,13 +124,22 @@ Other tasks are viewable with
123
124
 
124
125
  To run the Kestrel tests you will need:
125
126
 
126
- * gem install kestrel-client
127
+ * gem install kjess
127
128
  * A Kestrel server running on <tt>localhost:22133</tt>
128
129
 
129
130
  You can download Kestrel from http://robey.github.com/kestrel/ and then run the
130
131
  +scripts/devel.sh+ command and you will have a default Kestrel server running on
131
132
  <tt>localhost:22133</tt>. This will be enough to run the kestrel tests.
132
133
 
134
+ === Redis
135
+
136
+ To run the Redis tests you will need:
137
+
138
+ * gem install redis
139
+ * A Redis server running on <tt>localhost:6479</tt>
140
+
141
+ You can download redis using brew, macports or your favorite linux package
142
+ manager.
133
143
 
134
144
  == LICENSE
135
145
 
data/Rakefile CHANGED
@@ -7,17 +7,28 @@ This.homepage = "http://github.com/copiousfreetime/#{ This.name }"
7
7
  This.version = Util.version
8
8
 
9
9
  #------------------------------------------------------------------------------
10
- # If you want to Develop on qup just run 'rake develop' and you'll have all you
11
- # need to get going. If you want to use bundler for development, then run
12
- # 'rake develop:using_bundler'
10
+ # If you want to Develop on this project just run 'rake develop' and you'll
11
+ # have all you need to get going. If you want to use bundler for development,
12
+ # then run 'rake develop:using_bundler'
13
13
  #------------------------------------------------------------------------------
14
14
  namespace :develop do
15
+
16
+ # Install all the development and runtime dependencies of this gem using the
17
+ # gemspec.
15
18
  task :default do
16
19
  require 'rubygems/dependency_installer'
17
20
  installer = Gem::DependencyInstaller.new
18
21
 
22
+ # list these here instead of gem dependencies since there is not a way to
23
+ # sepcify ruby version specific dependencies
24
+ if RUBY_VERSION < "1.9.2"
25
+ Util.platform_gemspec.add_development_dependency( 'rcov', '~> 1.0.0' )
26
+ else
27
+ Util.platform_gemspec.add_development_dependency( 'simplecov', '~> 0.6.4' )
28
+ end
29
+
19
30
  puts "Installing gem depedencies needed for development"
20
- This.gemspec.dependencies.each do |dep|
31
+ Util.platform_gemspec.dependencies.each do |dep|
21
32
  if dep.matching_specs.empty? then
22
33
  puts "Installing : #{dep}"
23
34
  installer.install dep
@@ -28,6 +39,7 @@ namespace :develop do
28
39
  puts "\n\nNow run 'rake test'"
29
40
  end
30
41
 
42
+ # Create a Gemfile that just references the gemspec
31
43
  file 'Gemfile' => :gemspec do
32
44
  File.open( "Gemfile", "w+" ) do |f|
33
45
  f.puts 'source :rubygems'
@@ -39,13 +51,15 @@ namespace :develop do
39
51
  task :using_bundler => 'Gemfile' do
40
52
  puts "Now you can 'bundle'"
41
53
  end
54
+
55
+ # Gemfiles are build artifacts
42
56
  CLOBBER << FileList['Gemfile*']
43
57
  end
44
58
  desc "Boostrap development"
45
59
  task :develop => "develop:default"
46
60
 
47
61
  #------------------------------------------------------------------------------
48
- # RSpec
62
+ # RSpec - standard RSpec rake task
49
63
  #------------------------------------------------------------------------------
50
64
  begin
51
65
  require 'rspec/core/rake_task'
@@ -59,7 +73,8 @@ rescue LoadError
59
73
  end
60
74
 
61
75
  #------------------------------------------------------------------------------
62
- # RDoc
76
+ # RDoc - standard rdoc rake task, although we must make sure to use a more
77
+ # recent version of rdoc since it is the one that has 'tomdoc' markup
63
78
  #------------------------------------------------------------------------------
64
79
  begin
65
80
  gem 'rdoc' # otherwise we get the wrong task from stdlib
@@ -76,23 +91,38 @@ rescue LoadError
76
91
  end
77
92
 
78
93
  #------------------------------------------------------------------------------
79
- # Coverage
94
+ # Coverage - optional code coverage, rcov for 1.8 and simplecov for 1.9, so
95
+ # for the moment only rcov is listed.
80
96
  #------------------------------------------------------------------------------
81
- begin
82
- require 'rcov/rcovtask'
83
- Rcov::RcovTask.new do |t|
84
- t.libs << 'spec'
85
- t.pattern = 'spec/**/*_spec.rb'
86
- t.verbose = true
87
- t.rcov_opts << "-x ^/" # remove all the global files
88
- t.rcov_opts << "--sort coverage" # so we see the worst files at the top
97
+ if RUBY_VERSION < "1.9.2"
98
+ begin
99
+ require 'rcov/rcovtask'
100
+ Rcov::RcovTask.new( :coverage ) do |t|
101
+ t.libs << 'spec'
102
+ t.pattern = 'spec/**/*_spec.rb'
103
+ t.verbose = true
104
+ t.rcov_opts << "-x ^/" # remove all the global files
105
+ t.rcov_opts << "--sort coverage" # so we see the worst files at the top
106
+ end
107
+ rescue LoadError
108
+ Util.task_warning( 'rcov' )
109
+ end
110
+ else
111
+ begin
112
+ require 'simplecov'
113
+ desc "Run tests with code coverage"
114
+ task :coverage do
115
+ ENV['COVERAGE'] = 'true'
116
+ Rake::Task[:test].execute
117
+ end
118
+ rescue LoadError
119
+ Util.task_warning( 'simplecov' )
89
120
  end
90
- rescue LoadError
91
- Util.task_warning( 'rcov' )
92
121
  end
93
122
 
94
123
  #------------------------------------------------------------------------------
95
- # Manifest - most of this is from Hoe
124
+ # Manifest - We want an explicit list of thos files that are to be packaged in
125
+ # the gem. Most of this is from Hoe.
96
126
  #------------------------------------------------------------------------------
97
127
  namespace 'manifest' do
98
128
  desc "Check the manifest"
@@ -126,7 +156,8 @@ end
126
156
  #------------------------------------------------------------------------------
127
157
  # Gem Specification
128
158
  #------------------------------------------------------------------------------
129
- This.gemspec = Gem::Specification.new do |spec|
159
+ This.gemspec = Hash.new
160
+ This.gemspec['ruby'] = Gem::Specification.new do |spec|
130
161
  spec.name = This.name
131
162
  spec.version = This.version
132
163
  spec.author = This.author
@@ -145,37 +176,52 @@ This.gemspec = Gem::Specification.new do |spec|
145
176
  "--markup", "tomdoc" ]
146
177
 
147
178
  # The Runtime Dependencies
148
- spec.add_runtime_dependency( 'maildir', '~> 2.0.0' )
179
+ spec.add_runtime_dependency( 'maildir', '~> 2.1.0' )
149
180
 
150
181
  # Additional functionality if used
151
- spec.add_development_dependency( 'kestrel-client' , '~> 0.7.1' )
152
- spec.add_development_dependency( 'redis' , '~> 2.2.2' )
153
- spec.add_development_dependency( 'SystemTimer' , '~> 1.2.3' )
182
+ spec.add_development_dependency( 'kjess' , '~> 1.0.0' )
183
+ spec.add_development_dependency( 'redis' , '~> 3.0.2' )
154
184
 
155
185
  # The Development Dependencies
156
186
  spec.add_development_dependency( 'rake' , '~> 0.9.2.2')
157
- spec.add_development_dependency( 'rcov' , '~> 1.0.0' )
158
- spec.add_development_dependency( 'rspec' , '~> 2.8.0' )
187
+ spec.add_development_dependency( 'rspec' , '~> 2.11.0' )
159
188
  spec.add_development_dependency( 'rdoc' , '~> 3.12' )
160
189
 
161
190
  end
191
+
192
+ # The name of the gemspec file on disk
162
193
  This.gemspec_file = "#{This.name}.gemspec"
163
194
 
195
+ # Really this is only here to support those who use bundler
164
196
  desc "Build the #{This.name}.gemspec file"
165
197
  task :gemspec do
166
198
  File.open( This.gemspec_file, "wb+" ) do |f|
167
- f.write This.gemspec.to_ruby
199
+ f.write Util.platform_gemspec.to_ruby
168
200
  end
169
201
  end
202
+
203
+ # the gemspec is also a dev artifact and should not be kept around.
170
204
  CLOBBER << This.gemspec_file
171
205
 
206
+ # The standard gem packaging task, everyone has it.
172
207
  require 'rubygems/package_task'
173
- Gem::PackageTask.new( This.gemspec ) do
208
+ Gem::PackageTask.new( Util.platform_gemspec ) do
174
209
  # nothing
175
210
  end
176
211
 
177
212
  #------------------------------------------------------------------------------
178
- # Release
213
+ # Release - the steps we go through to do a final release, this is pulled from
214
+ # a compbination of mojombo's rakegem, hoe and hoe-git
215
+ #
216
+ # 1) make sure we are on the master branch
217
+ # 2) make sure there are no uncommitted items
218
+ # 3) check the manifest and make sure all looks good
219
+ # 4) build the gem
220
+ # 5) do an empty commit to have the commit message of the version
221
+ # 6) tag that commit as the version
222
+ # 7) push master
223
+ # 8) push the tag
224
+ # 7) pus the gem
179
225
  #------------------------------------------------------------------------------
180
226
  task :release_check do
181
227
  unless `git branch` =~ /^\* master$/
@@ -186,17 +232,25 @@ task :release_check do
186
232
  end
187
233
  end
188
234
 
189
- desc "Create tag v#{This.version}, build and push #{This.gemspec.full_name} to rubygems.org"
235
+ desc "Create tag v#{This.version}, build and push #{Util.platform_gemspec.full_name} to rubygems.org"
190
236
  task :release => [ :release_check, 'manifest:check', :gem ] do
191
237
  sh "git commit --allow-empty -a -m 'Release #{This.version}'"
192
238
  sh "git tag -a -m 'v#{This.version}' v#{This.version}"
193
239
  sh "git push origin master"
194
240
  sh "git push origin v#{This.version}"
195
- sh "gem push pkg/#{This.gemspec.full_name}.gem"
241
+ sh "gem push pkg/#{Util.platform_gemspec.full_name}.gem"
196
242
  end
197
243
 
198
244
  #------------------------------------------------------------------------------
199
- # Rakefile Support
245
+ # Rakefile Support - This is all the guts and utility methods that are
246
+ # necessary to support the above tasks.
247
+ #
248
+ # Lots of Credit for this Rakefile goes to:
249
+ #
250
+ # Ara T. Howard - see the Rakefile in all of his projects -
251
+ # https://github.com/ahoward/
252
+ # Tom Preston Werner - his Rakegem project https://github.com/mojombo/rakegem
253
+ # Seattle.rb - Hoe - cuz it has relly good stuff in there
200
254
  #------------------------------------------------------------------------------
201
255
  BEGIN {
202
256
 
@@ -232,16 +286,22 @@ BEGIN {
232
286
  abort "You need a Manifest.txt" unless File.readable?( "Manifest.txt" )
233
287
  File.readlines( "Manifest.txt" ).map { |l| l.strip }
234
288
  end
289
+
290
+ def self.platform_gemspec
291
+ This.gemspec[This.platform]
292
+ end
235
293
  end
236
294
 
237
295
  # Hold all the metadata about this project
238
296
  This = OpenStruct.new
297
+ This.platform = (RUBY_PLATFORM == "java") ? "java" : Gem::Platform::RUBY
298
+
239
299
  desc = Util.section_of( 'README.rdoc', 'DESCRIPTION')
240
300
  This.summary = desc.first
241
301
  This.description = desc.join(" ").tr("\n", ' ').gsub(/[{}]/,'').gsub(/\[[^\]]+\]/,'') # strip rdoc
242
302
 
243
303
 
244
- This.exclude_from_manifest = %r/tmp$|\.(git|DS_Store)|^(doc|coverage|pkg)|\.gemspec$|\.swp$|\.jar|\.rvmrc$|~$/
304
+ This.exclude_from_manifest = %r/tmp$|\.(git|DS_Store)|^(doc|coverage|pkg)|\.gemspec$|\.swp$|\.jar|\.rvmrc$|^kestrel|~$/
245
305
  This.manifest = Util.read_manifest
246
306
 
247
307
  }
data/lib/qup.rb CHANGED
@@ -1,6 +1,6 @@
1
1
  module Qup
2
2
  # The Current Version of the library
3
- VERSION = '1.2.0'
3
+ VERSION = '1.4.0'
4
4
 
5
5
  class Error < StandardError; end
6
6
 
@@ -23,13 +23,14 @@ module Qup
23
23
  KNOWN_ADAPTERS = {
24
24
  # require => gem
25
25
  'maildir' => 'maildir',
26
- 'kestrel' => 'kestrel-client',
26
+ 'kestrel' => 'kjess',
27
27
  'redis' => 'redis'
28
28
  }
29
29
  end
30
30
 
31
31
  require 'qup/adapter'
32
32
  require 'qup/consumer'
33
+ require 'qup/batch_consumer'
33
34
  require 'qup/message'
34
35
  require 'qup/producer'
35
36
  require 'qup/publisher'
@@ -37,6 +38,7 @@ require 'qup/queue_api'
37
38
  require 'qup/session'
38
39
  require 'qup/subscriber'
39
40
  require 'qup/topic_api'
41
+ require 'qup/backoff_sleeper'
40
42
 
41
43
  # Load the known adapters, print a warning if $VERBOSE is set
42
44
  Qup::KNOWN_ADAPTERS.each do |adapter, gemname|
@@ -1,5 +1,5 @@
1
1
  require 'qup/adapter'
2
- require 'kestrel-client'
2
+ require 'kjess'
3
3
 
4
4
  class Qup::Adapter
5
5
  # Internal: The backing adapter for Qup that uses Kestrel as the messaging
@@ -14,8 +14,11 @@ class Qup::Adapter
14
14
  # uri - the URI instance for this adapter to use
15
15
  def initialize( uri, options = {} )
16
16
  @uri = uri
17
- @addr = "#{@uri.host}:#{@uri.port}"
18
- @options = options
17
+ @host = @uri.host
18
+ @port = @uri.port.to_i
19
+ @client_options = options.merge( :host => @host, :port => @port )
20
+ @client = KJess::Client.new( @client_options )
21
+ @client.ping
19
22
  @closed = false
20
23
  end
21
24
 
@@ -25,7 +28,7 @@ class Qup::Adapter
25
28
  #
26
29
  # Returns a Qup::Queue
27
30
  def queue( name )
28
- Qup::Adapter::Kestrel::Queue.new( @addr, name )
31
+ Qup::Adapter::Kestrel::Queue.new( @client, name )
29
32
  end
30
33
 
31
34
  # Internal: Create a new Topic from this Adapter
@@ -34,21 +37,21 @@ class Qup::Adapter
34
37
  #
35
38
  # Returns a Qup::Topic
36
39
  def topic( name )
37
- Qup::Adapter::Kestrel::Topic.new( @addr, name )
40
+ Qup::Adapter::Kestrel::Topic.new( @client, name )
38
41
  end
39
42
 
40
43
  # Internal: Close the Kestrel adapter
41
44
  #
42
45
  # Return nothing
43
46
  def close
44
- @closed = true
47
+ @client.disconnect
45
48
  end
46
49
 
47
50
  # Internal: Is the Kestrel Adapter closed
48
51
  #
49
52
  # Returns true or false
50
53
  def closed?
51
- @closed
54
+ not @client.connected?
52
55
  end
53
56
  end
54
57
  end
@@ -3,21 +3,18 @@ class Qup::Adapter::Kestrel
3
3
  # Internal: The Common base class for Kestrel Topic and Queue
4
4
  #
5
5
  class Destination
6
-
7
6
  # Internal: the name of the Queue or Topic
8
7
  attr_reader :name
9
8
 
10
9
  # Internal: Create a new Topic or Queue
11
10
  #
12
11
  # address - the Connection Address string for the Kestrel Client
13
- # name - the String name of the Topic
12
+ # name - the String name of the Topic or Queue
14
13
  #
15
- # Returns a new Topic.
16
- def initialize( address, name )
17
- @address = address
18
- @client = blocking_transactional_client( @address )
19
- @admin_client = regular_client( @address )
20
- @name = name
14
+ # Returns a new Topic or Queue.
15
+ def initialize( client, name )
16
+ @client = client
17
+ @name = name
21
18
  ping
22
19
  end
23
20
 
@@ -27,28 +24,15 @@ class Qup::Adapter::Kestrel
27
24
  #
28
25
  # Returns nothing.
29
26
  def destroy
30
- @admin_client.delete( name )
31
- @admin_client.delete( name+"_errors" )
27
+ @client.delete( name )
32
28
  end
33
29
 
34
30
  # Internal: Make sure the Topic or Queue exists
35
31
  #
36
32
  # Returns nothing
37
33
  def ping
38
- @admin_client.peek( name )
34
+ @client.peek( name )
39
35
  return true
40
36
  end
41
-
42
- #######
43
- private
44
- #######
45
-
46
- def regular_client( addr )
47
- Kestrel::Client.new( addr )
48
- end
49
-
50
- def blocking_transactional_client( addr )
51
- Kestrel::Client::Blocking.new( Kestrel::Client::Transactional.new( regular_client(addr) ) )
52
- end
53
37
  end
54
38
  end