shoryuken-later 0.0.3 → 0.0.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,15 +1,7 @@
1
1
  ---
2
- !binary "U0hBMQ==":
3
- metadata.gz: !binary |-
4
- ZmRmMzRlZjhlM2I4YTM3NjhjNDUxOTQxYzYwZTE1ZjZiNGNiMWQwNA==
5
- data.tar.gz: !binary |-
6
- ZTQwODFkZDI4YmE4NGM2MzZiZjFkM2NmNTViZTk4OWQ3NGI2NzVhYQ==
7
- !binary "U0hBNTEy":
8
- metadata.gz: !binary |-
9
- NjYxMDExNjQzYjhhNTJmZWE3ZmZmZWMxYjRmN2IwYjU1OTU0Nzg4NTIyOGQx
10
- YzM0NzUzNjZjYzAzOTBjNDZiZGEzYTk3ZDU0MjdmM2ZlNmI0OTg5ZWQ4YjM2
11
- NWU1Y2U0NDZmMTZkY2QwYjE5MTJlYzJjMmI1MzA5NzVkNGE2YTE=
12
- data.tar.gz: !binary |-
13
- MzRiYjI0NDUxMGExNzVmMjA3ZDRlNDlkM2E0NzgzZTc1ZGM5YWE2YzY0OGQy
14
- Y2Q0N2QwZjEwMzM2YmUxYzMxYjBkYzUxOWQ0ZjFkNjMwOTNhNjM2MzkxMzdi
15
- ZWRiOGRlYjk1MTIzM2IzYjU1ZWJkOWJlNWI2OWQxY2JlMWFiNzg=
2
+ SHA1:
3
+ metadata.gz: 452bf02360d9ab6eb4a73581d937476a0b49c2d1
4
+ data.tar.gz: 69348f8206e4c630cbc21b62a0dd9aac83051609
5
+ SHA512:
6
+ metadata.gz: b9392aa905bf2ea5be0211bf485cf157e34d4f8da7fef8a96ec3590d12fdd9b07ebf1ffd9a597871994c227aacd2e1eba9d91dc755f3976cd966bbf3f7f9814c
7
+ data.tar.gz: f0d5b96a457574ca36a352809f949255fa794ebecf73486e3789e86342e0e357aa6d859c6e56a2ea5416f0f314e097eeef959ca7e44bfd23850d43939ad90b8f
data/README.md CHANGED
@@ -1,15 +1,26 @@
1
1
  # shoryuken-later
2
2
 
3
3
  A scheduling plugin for [Shoryuken](https://github.com/phstc/shoryuken) that uses [Dynamo DB](https://aws.amazon.com/dynamodb/)
4
- to delay messages arbitrarily far into the future.
4
+ to schedule messages arbitrarily far into the future.
5
5
 
6
6
  ## Features
7
7
 
8
- ### Supports distributed architectures
8
+ ### Integration with Shoryuken::Worker
9
+
10
+ A new method named `perform_later` is added to `Shoryuken::Worker` allowing messages to be delayed arbitrarily far into the future. If the delay is 15 minutes or less, then the message is enqueued into the specified SQS `:queue` as usual. Otherwise, the message is inserted into the specified DynamoDB `:schedule_table`.
11
+
12
+ ```ruby
13
+ require 'shoryuken-later'
9
14
 
10
- An SQS message is *only* queued if a _conditional_ delete of the DDB item is successful. This eliminates any potential race condition, so if more than one `shoryuken-later` process is polling the same schedule table then no redundant SQS messages will be queued.
15
+ class MyWorker
16
+ include Shoryuken::Worker
17
+
18
+ shoryuken_options queue: 'default', schedule_table: 'default_schedule'
19
+ end
11
20
 
12
- NOTE: You shouldn't really _need_ to run more than one process, but if you do it will be safe.
21
+ # Schedules a message to be processed 30 minutes from now.
22
+ MyWorker.perform_later(Time.now + 30 * 60, 'Foobar')
23
+ ```
13
24
 
14
25
  ### One or more schedule tables
15
26
 
@@ -27,19 +38,21 @@ later:
27
38
  You can use the same configuration file for both `Shoryuken` and `Shoryuken::Later`, because the new configuration options are namespaced.
28
39
 
29
40
  ```yaml
30
- # These keys are used by both Shoryuken and Shoryuken::Later
41
+ # This key is used by both Shoryuken and Shoryuken::Later
31
42
  aws:
32
43
  access_key_id: ... # or <%= ENV['AWS_ACCESS_KEY_ID'] %>
33
44
  secret_access_key: ... # or <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
34
45
  region: us-east-1 # or <%= ENV['AWS_REGION'] %>
35
- logfile: some/path/to/file.log
36
46
 
37
47
  # This key is only used by Shoryuken::Later
38
48
  later:
39
49
  delay: 5 * 60 # How frequently to poll the schedule table, in seconds.
40
- pidfile: some/path/to/file.pid
41
50
  tables:
42
51
  - table1
52
+
53
+ # These keys are used by both Shoryuken and Shoryuken::Later
54
+ logfile: some/path/to/file.log
55
+ pidfile: some/path/to/file.pid
43
56
 
44
57
  # These keys are only used by Shoryuken
45
58
  concurrency: 3
@@ -47,102 +60,4 @@ delay: 0
47
60
  queues:
48
61
  - [queue1, 1]
49
62
  - [queue2, 2]
50
- ```
51
-
52
- ## Usage
53
-
54
- ### Starting the schedule poller
55
-
56
- Start the `shoryuken-later` schedule poller with a command like:
57
-
58
- ```shell
59
- bundle exec shoryuken-later --config shoryuken.yml
60
- ```
61
-
62
- Run it as a daemon inside of your Rails app with a command like:
63
-
64
- ```shell
65
- bundle exec shoryuken-later --config shoryuken.yml --rails --daemon
66
- ```
67
-
68
- [Command-line options](https://github.com/joekhoobyar/shoryuken-later/wiki/Command-line-options)
69
-
70
-
71
- ### Integration with ActiveJob
72
-
73
- A custom ActiveJob adapter can used to support delaying messages arbitrarily far into the future.
74
-
75
- ```ruby
76
- # config/application.rb
77
- config.active_job.queue_adapter = :shoryuken_later
78
- ```
79
-
80
- When you use the `:shoryuken_later` queue adapter, jobs to be performed farther than 15 minutes into the future (by setting the `wait` or `wait_until` ActiveJob options), will be inserted into the *default* schedule table. You can set the default schedule table in an initializer.
81
-
82
- ```ruby
83
- # config/initializers/shoryuken_later.rb
84
- Shoryuken::Later.default_table = "#{Rails.env}_myapp_later"
85
- ```
86
-
87
-
88
- ### Integration with Shoryuken::Worker
89
-
90
- A new method named `perform_later` is added to `Shoryuken::Worker` allowing messages to be delayed arbitrarily far into the future. If the delay is 15 minutes or less, then the message is enqueued into the specified SQS `:queue` as usual. Otherwise, the message is inserted into the specified DynamoDB `:schedule_table`.
91
-
92
- ```ruby
93
- require 'shoryuken-later'
94
-
95
- class MyWorker
96
- include Shoryuken::Worker
97
-
98
- shoryuken_options queue: 'default', schedule_table: 'default_schedule'
99
- end
100
-
101
- # Schedules a message to be processed 30 minutes from now.
102
- MyWorker.perform_later(Time.now + 30 * 60, 'Foobar')
103
- ```
104
-
105
-
106
- ## Requirements
107
-
108
- Ruby 1.9 or greater.
109
-
110
- ## Installation
111
-
112
- Add this line to your application's Gemfile:
113
-
114
- ```ruby
115
- gem 'shoryuken-later'
116
- ```
117
-
118
- Or to get the latest updates:
119
-
120
- ```ruby
121
- gem 'shoryuken-later', github: 'joekhoobyar/shoryuken-later', branch: 'master'
122
- ```
123
-
124
- And then execute:
125
-
126
- $ bundle
127
-
128
- Or install it yourself as:
129
-
130
- $ gem install shoryuken-later
131
-
132
- ## Documentation
133
-
134
- Learn about using Shoryuken::Later at the [Shoryuken::Later Wiki](https://github.com/joekhoobyar/shoryuken-later/wiki).
135
-
136
- Learn about using Shoryuken at the [Shoryuken Wiki](https://github.com/phstc/shoryuken/wiki).
137
-
138
- ## Credits
139
-
140
- [Pablo Cantero](https://github.com/phstc), creator of [Shoryuken](https://github.com/phstc/shoryuken), and [everybody who contributed to it](https://github.com/phstc/shoryuken/graphs/contributors). I borrowed a lot of code from Shoryuken itself as a shortcut to making this gem.
141
-
142
- ## Contributing
143
-
144
- 1. Fork it ( https://github.com/joekhoobyar/shoryuken-later/fork )
145
- 2. Create your feature branch (`git checkout -b my-new-feature`)
146
- 3. Commit your changes (`git commit -am 'Add some feature'`)
147
- 4. Push to the branch (`git push origin my-new-feature`)
148
- 5. Create a new Pull Request
63
+ ```
data/Rakefile CHANGED
File without changes
@@ -1,4 +1,4 @@
1
- # Practically all of this has been "borrowed" from Shoryuken.
1
+ # Build on top of Shoryuken's ActiveJob adapter.
2
2
 
3
3
  # @see ActiveJob::QueueAdapter::ShoryukenAdapter
4
4
 
@@ -20,20 +20,16 @@ module ActiveJob
20
20
  # To use Shoryuken::Later set the queue_adapter config to +:shoryuken_later+.
21
21
  #
22
22
  # Rails.application.config.active_job.queue_adapter = :shoryuken_later
23
- class ShoryukenLaterAdapter
23
+ class ShoryukenLaterAdapter < ShoryukenAdapter
24
+ JobWrapper = ShoryukenAdapter::JobWrapper
25
+
24
26
  class << self
25
- def enqueue(job) #:nodoc:
26
- register_worker!(job)
27
-
28
- Shoryuken::Client.send_message(job.queue_name, job.serialize, message_attributes: message_attributes)
29
- end
30
-
31
27
  def enqueue_at(job, timestamp) #:nodoc:
32
28
  register_worker!(job)
33
29
 
34
30
  delay = (timestamp - Time.current.to_f).round
35
31
  if delay > 15.minutes
36
- Shoryuken::Later::Client.put_item(Shoryuken::Later.default_table, perform_at: Time.current.to_i + delay.to_i,
32
+ Shoryuken::Later::Client.put_item(Shoryuken::Later.default_table, perform_at: delay.to_i,
37
33
  shoryuken_queue: job.queue_name, shoryuken_class: JobWrapper.to_s,
38
34
  shoryuken_args: JSON.dump(body: job.serialize, options: {}))
39
35
  else
@@ -41,32 +37,6 @@ module ActiveJob
41
37
  message_attributes: message_attributes)
42
38
  end
43
39
  end
44
-
45
-
46
- private
47
-
48
- def register_worker!(job)
49
- Shoryuken.register_worker(job.queue_name, JobWrapper)
50
- end
51
-
52
- def message_attributes
53
- @message_attributes ||= {
54
- 'shoryuken_class' => {
55
- string_value: JobWrapper.to_s,
56
- data_type: 'String'
57
- }
58
- }
59
- end
60
- end
61
-
62
- class JobWrapper #:nodoc:
63
- include Shoryuken::Worker
64
-
65
- shoryuken_options body_parser: :json, auto_delete: true
66
-
67
- def perform(sqs_msg, hash)
68
- Base.execute hash
69
- end
70
40
  end
71
41
  end
72
42
  end
@@ -7,7 +7,6 @@ require 'singleton'
7
7
  require 'optparse'
8
8
  require 'erb'
9
9
  require 'shoryuken/later'
10
- require 'timers'
11
10
 
12
11
  module Shoryuken
13
12
  module Later
@@ -15,10 +14,12 @@ module Shoryuken
15
14
  include Shoryuken::Util
16
15
  include Singleton
17
16
 
17
+ attr_accessor :launcher
18
+
18
19
  def run(args)
19
20
  self_read, self_write = IO.pipe
20
21
 
21
- %w[INT TERM USR1 USR2].each do |sig|
22
+ %w[INT TERM USR1 USR2 TTIN].each do |sig|
22
23
  trap sig do
23
24
  self_write.puts(sig)
24
25
  end
@@ -33,49 +34,38 @@ module Shoryuken
33
34
  validate!
34
35
  daemonize
35
36
  write_pid
36
-
37
- Shoryuken::Logging.with_context '[later]' do
38
- logger.info 'Starting'
39
-
40
- # Initialize the timers and poller.
41
- @timers = Timers.new
42
- require 'shoryuken/later/poller'
43
- @pollers = Shoryuken::Later.tables.map{|tbl| Poller.new(tbl) }
44
-
45
- begin
46
- # Poll for items on startup, and every :poll_delay
47
- poll_tables
48
- @timers.every(Shoryuken::Later.poll_delay){ poll_tables }
49
-
50
- # Loop watching for signals and firing off of timers
51
- while @timers
52
- interval = @timers.wait_interval
53
- readable, writable = IO.select([self_read], nil, nil, interval)
54
- if readable
55
- handle_signal readable.first.gets.strip
56
- else
57
- @timers.fire
58
- end
59
- end
60
- rescue Interrupt
61
- @timers.cancel
62
- exit 0
37
+ load_celluloid
38
+
39
+ require 'shoryuken/later/launcher'
40
+ @launcher = Shoryuken::Later::Launcher.new
41
+
42
+ begin
43
+ launcher.run
44
+
45
+ while readable_io = IO.select([self_read])
46
+ signal = readable_io.first[0].gets.strip
47
+ handle_signal(signal)
63
48
  end
49
+ rescue Interrupt
50
+ launcher.stop(shutdown: true)
51
+ exit 0
64
52
  end
65
53
  end
66
-
67
- protected
68
-
69
- def poll_tables
70
- logger.debug "Polling schedule tables"
71
- @pollers.each do |poller|
72
- poller.poll
73
- end
74
- logger.debug "Polling done"
75
- end
76
54
 
77
55
  private
78
56
 
57
+ def load_celluloid
58
+ raise "Celluloid cannot be required until here, or it will break Shoryuken::Later's daemonization" if defined?(::Celluloid) && Shoryuken::Later.options[:daemon]
59
+
60
+ # Celluloid can't be loaded until after we've daemonized
61
+ # because it spins up threads and creates locks which get
62
+ # into a very bad state if forked.
63
+ require 'celluloid/autostart'
64
+ Celluloid.logger = (Shoryuken::Later.options[:verbose] ? Shoryuken::Later.logger : nil)
65
+
66
+ require 'shoryuken/later/manager'
67
+ end
68
+
79
69
  def load_rails
80
70
  # Adapted from: https://github.com/mperham/sidekiq/blob/master/lib/sidekiq/cli.rb
81
71
 
@@ -89,7 +79,6 @@ module Shoryuken
89
79
  ::Rails::Application.initializer "shoryuken-later.eager_load" do
90
80
  ::Rails.application.config.eager_load = true
91
81
  end
92
- require 'shoryuken/later/active_job_adapter' if defined?(::ActiveJob)
93
82
  require File.expand_path("config/environment.rb")
94
83
  end
95
84
 
@@ -128,7 +117,7 @@ module Shoryuken
128
117
  end
129
118
 
130
119
  def write_pid
131
- if path = Shoryuken::Later.options[:later][:pidfile]
120
+ if path = Shoryuken::Later.options[:pidfile]
132
121
  File.open(path, 'w') do |f|
133
122
  f.puts Process.pid
134
123
  end
@@ -136,7 +125,7 @@ module Shoryuken
136
125
  end
137
126
 
138
127
  def parse_options(argv)
139
- opts = {later: {}}
128
+ opts = {}
140
129
 
141
130
  @parser = OptionParser.new do |o|
142
131
  o.on '-d', '--daemon', 'Daemonize process' do |arg|
@@ -164,7 +153,7 @@ module Shoryuken
164
153
  end
165
154
 
166
155
  o.on '-P', '--pidfile PATH', 'Path to pidfile' do |arg|
167
- opts[:later][:pidfile] = arg
156
+ opts[:pidfile] = arg
168
157
  end
169
158
 
170
159
  o.on '-v', '--verbose', 'Print more verbose output' do |arg|
@@ -192,10 +181,28 @@ module Shoryuken
192
181
  case sig
193
182
  when 'USR1'
194
183
  logger.info "Received USR1, will soft shutdown down"
195
- @timers.cancel
196
- @timers = nil
184
+
185
+ launcher.stop
186
+
187
+ exit 0
188
+ when 'TTIN'
189
+ Thread.list.each do |thread|
190
+ logger.info "Thread TID-#{thread.object_id.to_s(36)} #{thread['label']}"
191
+ if thread.backtrace
192
+ logger.info thread.backtrace.join("\n")
193
+ else
194
+ logger.info "<no backtrace available>"
195
+ end
196
+ end
197
+
198
+ idle = launcher.manager.instance_variable_get(:@idle).size
199
+ busy = launcher.manager.instance_variable_get(:@busy).size
200
+ tables = launcher.manager.instance_variable_get(:@tables)
201
+
202
+ logger.info "Idle: #{idle}, Busy: #{busy}, Polled Tables: #{tables.join(', ')}"
197
203
  else
198
204
  logger.info "Received #{sig}, will shutdown down"
205
+
199
206
  raise Interrupt
200
207
  end
201
208
  end
@@ -208,10 +215,8 @@ module Shoryuken
208
215
 
209
216
  config = options[:config_file] ? parse_config(options[:config_file]).deep_symbolize_keys : {}
210
217
 
211
- Shoryuken::Later.options[:later].merge!(config.delete(:later) || {})
212
218
  Shoryuken::Later.options.merge!(config)
213
219
 
214
- Shoryuken::Later.options[:later].merge!(options.delete(:later) || {})
215
220
  Shoryuken::Later.options.merge!(options)
216
221
 
217
222
  # Tables from command line options take precedence...
File without changes
@@ -0,0 +1,42 @@
1
+ # All of this has been "borrowed" from Shoryuken.
2
+
3
+ # @see Shoryuken::Launcher
4
+ module Shoryuken
5
+ module Later
6
+ class Launcher
7
+ include Celluloid
8
+ include Shoryuken::Util
9
+
10
+ trap_exit :actor_died
11
+
12
+ attr_accessor :manager
13
+
14
+ def initialize
15
+ @manager = Shoryuken::Later::Manager.new_link
16
+
17
+ @done = false
18
+ end
19
+
20
+ def stop(options = {})
21
+ watchdog('Later::Launcher#stop') do
22
+ @done = true
23
+
24
+ manager.async.stop(shutdown: !!options[:shutdown], timeout: Shoryuken::Later.options[:timeout])
25
+ manager.wait(:shutdown)
26
+ end
27
+ end
28
+
29
+ def run
30
+ watchdog('Later::Launcher#run') do
31
+ manager.async.start
32
+ end
33
+ end
34
+
35
+ def actor_died(actor, reason)
36
+ return if @done
37
+ logger.warn 'Shoryuken::Later died due to the following error, cannot recover, process exiting'
38
+ exit 1
39
+ end
40
+ end
41
+ end
42
+ end
@@ -0,0 +1,138 @@
1
+ # Most of this has been "borrowed" from Shoryuken, but then repurposed for periodic polling.
2
+
3
+ # @see Shoryuken::Manager
4
+ require 'set'
5
+ require 'shoryuken/later/poller'
6
+
7
+ module Shoryuken
8
+ module Later
9
+ class Manager
10
+ include Celluloid
11
+ include Shoryuken::Util
12
+
13
+ def initialize
14
+ @tables = Shoryuken::Later.tables.dup.uniq
15
+
16
+ @done = false
17
+
18
+ @idle = Set.new([])
19
+ @busy = Set.new([])
20
+ @timers = {}
21
+
22
+ @tables.each{|table| Poller.supervise_as :"poller-#{table}", current_actor, table }
23
+ end
24
+
25
+ def start
26
+ logger.info 'Starting'
27
+
28
+ # Start a poller for every table being polled.
29
+ @tables.each do |table|
30
+ dispatch table
31
+
32
+ # Save the timer so it can be cancelled at shutdown.
33
+ @timers[table] = every(Shoryuken::Later.poll_delay) { dispatch table }
34
+ end
35
+ end
36
+
37
+ def stop(options = {})
38
+ watchdog('Later::Manager#stop died') do
39
+ @done = true
40
+
41
+ @timers.each_value{|timer| timer.cancel if timer }
42
+ @timers.clear
43
+
44
+ logger.info { "Shutting down #{@idle.size} idle poller(s)" }
45
+
46
+ @idle.each do |name|
47
+ poller = Actor[name] and poller.alive? and poller.terminate
48
+ end
49
+ @idle.clear
50
+
51
+ if @busy.empty?
52
+ return after(0) { signal(:shutdown) }
53
+ end
54
+
55
+ if options[:shutdown]
56
+ hard_shutdown_in(options[:timeout])
57
+ else
58
+ soft_shutdown(options[:timeout])
59
+ end
60
+ end
61
+ end
62
+
63
+ def poller_done(table, poller)
64
+ watchdog('Later::Manager#poller_done died') do
65
+ logger.debug { "Poller done for '#{table}'" }
66
+
67
+ name = :"poller-#{table}"
68
+ @busy.delete name
69
+
70
+ if stopped?
71
+ poller.terminate if poller.alive?
72
+ else
73
+ @idle << name
74
+ end
75
+ end
76
+ end
77
+
78
+ def poller_ready(table, poller)
79
+ watchdog('Later::Manager#poller_ready died') do
80
+ logger.debug { "Poller for '#{table}' ready" }
81
+
82
+ name = :"poller-#{table}"
83
+ @busy.delete name
84
+ @idle << name
85
+ end
86
+ end
87
+
88
+ def stopped?
89
+ @done
90
+ end
91
+
92
+ private
93
+
94
+ def dispatch(table)
95
+ name = :"poller-#{table}"
96
+
97
+ # Only start polling if the poller is idle.
98
+ if ! stopped? && @idle.include?(name)
99
+ @idle.delete(name)
100
+ @busy << name
101
+
102
+ Actor[name].async.poll
103
+ end
104
+ end
105
+
106
+ def soft_shutdown(delay)
107
+ logger.info { "Waiting for #{@busy.size} busy pollers" }
108
+
109
+ if @busy.size > 0
110
+ after(delay) { soft_shutdown(delay) }
111
+ else
112
+ after(0) { signal(:shutdown) }
113
+ end
114
+ end
115
+
116
+ def hard_shutdown_in(delay)
117
+ logger.info { "Waiting for #{@busy.size} busy pollers" }
118
+ logger.info { "Pausing up to #{delay} seconds to allow pollers to finish..." }
119
+
120
+ after(delay) do
121
+ watchdog("Later::Manager#hard_shutdown_in died") do
122
+ if @busy.size > 0
123
+ logger.info { "Hard shutting down #{@busy.size} busy pollers" }
124
+
125
+ @busy.each do |busy|
126
+ if poller = Actor[busy]
127
+ t = poller.bare_object.actual_work_thread
128
+ t.raise Shutdown if poller.alive?
129
+ end
130
+ end
131
+ end
132
+ after(0) { signal(:shutdown) }
133
+ end
134
+ end
135
+ end
136
+ end
137
+ end
138
+ end
@@ -3,12 +3,16 @@ require 'json'
3
3
  module Shoryuken
4
4
  module Later
5
5
  class Poller
6
+ include Celluloid
6
7
  include Shoryuken::Util
7
8
 
8
9
  attr_reader :table_name
9
10
 
10
- def initialize(table_name)
11
+ def initialize(manager, table_name)
12
+ @manager = manager
11
13
  @table_name = table_name
14
+
15
+ @manager.async.poller_ready(@table_name, self)
12
16
  end
13
17
 
14
18
  def poll
@@ -21,10 +25,12 @@ module Shoryuken
21
25
  while item = next_item
22
26
  id = item.attributes['id']
23
27
  logger.info "Found message #{id} from '#{@table_name}'"
24
- if sent_msg = process_item(item)
25
- logger.debug { "Enqueued message #{id} from '#{@table_name}' as #{sent_msg.id}" }
26
- else
27
- logger.debug { "Skipping already queued message #{id} from '#{@table_name}'" }
28
+ defer do
29
+ if sent_msg = process_item(item)
30
+ logger.debug { "Enqueued message #{id} from '#{@table_name}' as #{sent_msg.id}" }
31
+ else
32
+ logger.debug { "Skipping already queued message #{id} from '#{@table_name}'" }
33
+ end
28
34
  end
29
35
  end
30
36
 
@@ -33,6 +39,8 @@ module Shoryuken
33
39
  logger.error "Error fetching message: #{ex}"
34
40
  logger.error ex.backtrace.first
35
41
  end
42
+
43
+ @manager.async.poller_done(@table_name, self)
36
44
  end
37
45
  end
38
46
 
@@ -1,5 +1,5 @@
1
1
  module Shoryuken
2
2
  module Later
3
- VERSION = '0.0.3'
3
+ VERSION = '0.0.5'
4
4
  end
5
5
  end
File without changes
@@ -20,13 +20,13 @@ Gem::Specification.new do |spec|
20
20
  spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
21
21
  spec.require_paths = ["lib"]
22
22
 
23
- spec.required_ruby_version = '>= 1.9.2'
23
+ spec.required_ruby_version = '>= 2.0.0'
24
24
 
25
25
  spec.add_development_dependency "bundler", '>= 1.3.5'
26
26
  spec.add_development_dependency "rake", '~> 10.0'
27
27
  spec.add_development_dependency "rspec", '~> 3.0', '< 3.1'
28
28
 
29
29
  spec.add_dependency "aws-sdk-v1"
30
- spec.add_dependency "timers", "~> 1.1.0"
31
- spec.add_dependency "shoryuken", "= 0.0.4"
30
+ spec.add_dependency "celluloid", "~> 0.16.0"
31
+ spec.add_dependency "shoryuken", "~> 0.0.5"
32
32
  end
File without changes
@@ -1,7 +1,9 @@
1
1
  require 'spec_helper'
2
2
  require 'shoryuken/later/poller'
3
+ require 'shoryuken/later/manager'
3
4
 
4
5
  describe Shoryuken::Later::Poller do
6
+ let(:manager) { double Shoryuken::Later::Manager, poller_ready: nil, poller_done: nil }
5
7
  let(:ddb_table) { double 'DynamoDb Table' }
6
8
  let(:ddb_items) { double 'Table Items' }
7
9
  let(:table) { 'shoryuken_later' }
@@ -15,26 +17,38 @@ describe Shoryuken::Later::Poller do
15
17
  end
16
18
 
17
19
  before do
20
+ allow(manager).to receive(:async).and_return(manager)
18
21
  allow(Shoryuken::Later::Client).to receive(:tables).with(table).and_return(ddb_table)
19
22
  end
20
23
 
21
24
  subject do
22
- described_class.new(table)
25
+ described_class.new(manager, table)
23
26
  end
24
27
 
28
+ describe '#initialize' do
29
+ it 'informs the manager that the poller is ready' do
30
+ expect(manager).to receive(:poller_ready).once
31
+
32
+ subject.inspect
33
+ subject.inspect
34
+ end
35
+ end
36
+
25
37
  describe '#poll' do
26
38
  it 'pulls items from #next_item, and processes with #process_item' do
27
39
  items = [ddb_item]
28
40
  expect_any_instance_of(described_class).to receive(:next_item).twice { items.pop }
29
41
  expect_any_instance_of(described_class).to receive(:process_item).once.with(ddb_item)
42
+ expect(manager).to receive(:poller_done).once
30
43
 
31
44
  subject.poll
32
45
  end
33
46
 
34
- it 'does not call #process_item when there are no items' do
47
+ it 'informs the manager after polling is done' do
35
48
  items = []
36
49
  expect_any_instance_of(described_class).to receive(:next_item).once { items.pop }
37
50
  expect_any_instance_of(described_class).not_to receive(:process_item)
51
+ expect(manager).to receive(:poller_done).once
38
52
 
39
53
  subject.poll
40
54
  end
File without changes
data/spec/spec_helper.rb CHANGED
@@ -1,6 +1,7 @@
1
1
  require 'bundler/setup'
2
2
  Bundler.setup
3
3
 
4
+ require 'celluloid'
4
5
  require 'shoryuken-later'
5
6
  require 'json'
6
7
 
@@ -15,6 +16,7 @@ if File.exists? options_file
15
16
  end
16
17
 
17
18
  Shoryuken.logger.level = Logger::UNKNOWN
19
+ Celluloid.logger.level = Logger::UNKNOWN
18
20
 
19
21
  # For Ruby 1.9
20
22
  module Kernel
@@ -66,7 +68,7 @@ RSpec.configure do |config|
66
68
  TestWorker.get_shoryuken_options['queue'] = 'shoryuken_later'
67
69
  TestWorker.get_shoryuken_options['schedule_table'] = 'shoryuken_later'
68
70
 
69
- Shoryuken.workers.clear
71
+ Shoryuken.worker_registry.clear
70
72
  Shoryuken.register_worker('shoryuken_later', TestWorker)
71
73
  end
72
74
  end
metadata CHANGED
@@ -1,27 +1,27 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: shoryuken-later
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.3
4
+ version: 0.0.5
5
5
  platform: ruby
6
6
  authors:
7
7
  - Joe Khoobyar
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2015-03-12 00:00:00.000000000 Z
11
+ date: 2015-03-06 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
15
15
  requirement: !ruby/object:Gem::Requirement
16
16
  requirements:
17
- - - ! '>='
17
+ - - '>='
18
18
  - !ruby/object:Gem::Version
19
19
  version: 1.3.5
20
20
  type: :development
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
- - - ! '>='
24
+ - - '>='
25
25
  - !ruby/object:Gem::Version
26
26
  version: 1.3.5
27
27
  - !ruby/object:Gem::Dependency
@@ -62,46 +62,46 @@ dependencies:
62
62
  name: aws-sdk-v1
63
63
  requirement: !ruby/object:Gem::Requirement
64
64
  requirements:
65
- - - ! '>='
65
+ - - '>='
66
66
  - !ruby/object:Gem::Version
67
67
  version: '0'
68
68
  type: :runtime
69
69
  prerelease: false
70
70
  version_requirements: !ruby/object:Gem::Requirement
71
71
  requirements:
72
- - - ! '>='
72
+ - - '>='
73
73
  - !ruby/object:Gem::Version
74
74
  version: '0'
75
75
  - !ruby/object:Gem::Dependency
76
- name: timers
76
+ name: celluloid
77
77
  requirement: !ruby/object:Gem::Requirement
78
78
  requirements:
79
79
  - - ~>
80
80
  - !ruby/object:Gem::Version
81
- version: 1.1.0
81
+ version: 0.16.0
82
82
  type: :runtime
83
83
  prerelease: false
84
84
  version_requirements: !ruby/object:Gem::Requirement
85
85
  requirements:
86
86
  - - ~>
87
87
  - !ruby/object:Gem::Version
88
- version: 1.1.0
88
+ version: 0.16.0
89
89
  - !ruby/object:Gem::Dependency
90
90
  name: shoryuken
91
91
  requirement: !ruby/object:Gem::Requirement
92
92
  requirements:
93
- - - '='
93
+ - - ~>
94
94
  - !ruby/object:Gem::Version
95
- version: 0.0.4
95
+ version: 0.0.5
96
96
  type: :runtime
97
97
  prerelease: false
98
98
  version_requirements: !ruby/object:Gem::Requirement
99
99
  requirements:
100
- - - '='
100
+ - - ~>
101
101
  - !ruby/object:Gem::Version
102
- version: 0.0.4
103
- description: ! "\n This gem provides a scheduling plugin (using Dynamo DB) for
104
- Shoryuken, as well as an ActiveJob adapter\n "
102
+ version: 0.0.5
103
+ description: "\n This gem provides a scheduling plugin (using Dynamo DB) for Shoryuken,
104
+ as well as an ActiveJob adapter\n "
105
105
  email:
106
106
  - joe@khoobyar.name
107
107
  executables:
@@ -120,6 +120,8 @@ files:
120
120
  - lib/shoryuken/later/active_job_adapter.rb
121
121
  - lib/shoryuken/later/cli.rb
122
122
  - lib/shoryuken/later/client.rb
123
+ - lib/shoryuken/later/launcher.rb
124
+ - lib/shoryuken/later/manager.rb
123
125
  - lib/shoryuken/later/poller.rb
124
126
  - lib/shoryuken/later/version.rb
125
127
  - lib/shoryuken/later/worker.rb
@@ -138,17 +140,17 @@ require_paths:
138
140
  - lib
139
141
  required_ruby_version: !ruby/object:Gem::Requirement
140
142
  requirements:
141
- - - ! '>='
143
+ - - '>='
142
144
  - !ruby/object:Gem::Version
143
- version: 1.9.2
145
+ version: 2.0.0
144
146
  required_rubygems_version: !ruby/object:Gem::Requirement
145
147
  requirements:
146
- - - ! '>='
148
+ - - '>='
147
149
  - !ruby/object:Gem::Version
148
150
  version: '0'
149
151
  requirements: []
150
152
  rubyforge_project:
151
- rubygems_version: 2.0.3
153
+ rubygems_version: 2.4.5
152
154
  signing_key:
153
155
  specification_version: 4
154
156
  summary: A scheduling plugin (using Dynamo DB) for Shoryuken
@@ -157,3 +159,4 @@ test_files:
157
159
  - spec/shoryuken/later/poller_spec.rb
158
160
  - spec/shoryuken/worker_spec.rb
159
161
  - spec/spec_helper.rb
162
+ has_rdoc: