resque 1.27.4 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of resque might be problematic. Click here for more details.

checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: acae072e3d9a8528a3dc20919ea2c0bd06c90d45
4
- data.tar.gz: 3de81881552589630db942941bb3ff6b8353ff24
2
+ SHA256:
3
+ metadata.gz: 845d352e5d2e155c7a3c0c94b825abe83cc0daa617e70ac85fb6f2ec0fbad452
4
+ data.tar.gz: 887666edf6aaa2b4bbc372001ece5b1fb354801be30786160ee0cde6b16828c0
5
5
  SHA512:
6
- metadata.gz: 20fa32261ff9fd4744a7c2bb32265fa545d2cc4dda3433f1513291e6cec91d3503cfc4120b990b2ed96507c7848dfbfe474682e10b0ed68a7188798b88819e08
7
- data.tar.gz: b623868af21a1ccb630ef02f0f9ed1892845041f01926564a01559a6bdac783c6353822705b7c82f1d750c167a743972b7d54d859c0fed66a57826c3ea498cf8
6
+ metadata.gz: e99e247f115ce75ad125e7f4aa0bf1587e67426c424e3767ab856c4a03c9868ebb922a958aaff689f316beb9b2b07ca0d1818b592a300d6a2358af3b1312932a
7
+ data.tar.gz: '081fe527bb158cee4c9e8c31d552904b20e1fa39984466a2f7d1851facdbb2f0eb1f3ec865ae6d9cafee79278a392316822d0cdc4aff7a5c3076f96f8907d26a'
data/HISTORY.md CHANGED
@@ -1,8 +1,25 @@
1
- ## Unreleased
1
+ ## 2.0.0 (2018-11-06)
2
2
 
3
- Nothing yet!
3
+ ### Fixed
4
+ * Fix Airbrake failure backend
5
+ * Fix failed jobs page "argument out of range" error
6
+
7
+ ### Changed
8
+ * Remove support for Rubies < 2.3
9
+ * Remove support to Rails < 4
10
+ * Reduce the number of redis calls when trying to get the list of queues
11
+ * Only run `eager_load!` if `Rails.application.config.eager_load` is true
12
+ * Don't display log message if running without hooks
13
+ * Add Support to Redis 4.0
14
+ * Drop complex Redis identifier logic in favor of simple inspect
15
+ * When a child process is killed, report the signal it was terminated with
16
+ * Report a job that pruned worker was processing
17
+
18
+ ### Added
19
+
20
+ * Allow to configure statistic data store
4
21
 
5
- ## 1.27.4 (2017-4-15)
22
+ ## 1.27.4 (2017-04-15)
6
23
 
7
24
  ### Fixed
8
25
  * Fix issue where removing a failure from Resque web didn't work when using `RedisMultiQueue` backend.
@@ -3,7 +3,6 @@ Resque
3
3
 
4
4
  [![Gem Version](https://badge.fury.io/rb/resque.svg)](https://rubygems.org/gems/resque)
5
5
  [![Build Status](https://travis-ci.org/resque/resque.svg)](https://travis-ci.org/resque/resque)
6
- [![Coverage Status](https://coveralls.io/repos/github/resque/resque/badge.svg?branch=1-x-stable)](https://coveralls.io/r/resque/resque?branch=1-x-stable)
7
6
 
8
7
  Resque (pronounced like "rescue") is a Redis-backed library for creating
9
8
  background jobs, placing those jobs on multiple queues, and processing
@@ -34,6 +33,9 @@ The Resque frontend tells you what workers are doing, what workers are
34
33
  not doing, what queues you're using, what's in those queues, provides
35
34
  general usage stats, and helps you track failures.
36
35
 
36
+ Resque now supports Ruby 2.3.0 and above.
37
+ We will also only be supporting Redis 3.0 and above going forward.
38
+
37
39
 
38
40
  The Blog Post
39
41
  -------------
@@ -196,7 +198,7 @@ We plan to provide first class `async` support in a future release.
196
198
  If a job raises an exception, it is logged and handed off to the
197
199
  `Resque::Failure` module. Failures are logged either locally in Redis
198
200
  or using some different backend. To see exceptions while developing,
199
- use VERBOSE env variable, see details below under Logging.
201
+ see details below under Logging.
200
202
 
201
203
  For example, Resque ships with Airbrake support. To configure it, put
202
204
  the following into an initialisation file or into your rake job:
@@ -267,12 +269,14 @@ but in the Resque workers it's fine.
267
269
 
268
270
  ### Logging
269
271
 
270
- Workers support basic logging to STDOUT. If you start them with the
271
- `VERBOSE` env variable set, they will print basic debugging
272
- information. You can also set the `VVERBOSE` (very verbose) env
273
- variable.
272
+ Workers support basic logging to STDOUT.
273
+
274
+ You can control the logging threshold using `Resque.logger.level`:
274
275
 
275
- $ VVERBOSE=1 QUEUE=file_serve rake environment resque:work
276
+ ```ruby
277
+ # config/initializers/resque.rb
278
+ Resque.logger.level = Logger::DEBUG
279
+ ```
276
280
 
277
281
  If you want Resque to log to a file, in Rails do:
278
282
 
@@ -281,6 +285,33 @@ If you want Resque to log to a file, in Rails do:
281
285
  Resque.logger = Logger.new(Rails.root.join('log', "#{Rails.env}_resque.log"))
282
286
  ```
283
287
 
288
+ ### Storing Statistics
289
+ Resque allows to store count of processed and failed jobs.
290
+
291
+ By default it will store it in Redis using the keys `stats:processed` and `stats:failed`.
292
+
293
+ Some apps would want another stats store, or even a null store:
294
+
295
+ ```ruby
296
+ # config/initializers/resque.rb
297
+ class NullDataStore
298
+ def stat(stat)
299
+ 0
300
+ end
301
+
302
+ def increment_stat(stat, by)
303
+ end
304
+
305
+ def decrement_stat(stat, by)
306
+ end
307
+
308
+ def clear_stat(stat)
309
+ end
310
+ end
311
+
312
+ Resque.stat_data_store = NullDataStore.new
313
+ ```
314
+
284
315
  ### Process IDs (PIDs)
285
316
 
286
317
  There are scenarios where it's helpful to record the PID of a resque
@@ -290,7 +321,7 @@ worker process. Use the PIDFILE option for easy access to the PID:
290
321
 
291
322
  ### Running in the background
292
323
 
293
- (Only supported with ruby >= 1.9). There are scenarios where it's helpful for
324
+ There are scenarios where it's helpful for
294
325
  the resque worker to run itself in the background (usually in combination with
295
326
  PIDFILE). Use the BACKGROUND option so that rake will return as soon as the
296
327
  worker is started.
@@ -458,6 +489,11 @@ leading to frequent `Resque::TermException` errors. For short running jobs, a s
458
489
  solution is to give a small amount of time for the job to finish
459
490
  before killing it.
460
491
 
492
+ Resque doesn't handle this out of the box (for both cedar-14 and heroku-16), you need to
493
+ install the [`resque-heroku-signals`](https://github.com/iloveitaly/resque-heroku-signals)
494
+ addon which adds the required signal handling to make the behavior described above work.
495
+ Related issue: https://github.com/resque/resque/issues/1559
496
+
461
497
  To accomplish this set the following environment variables:
462
498
 
463
499
  * `RESQUE_PRE_SHUTDOWN_TIMEOUT` - The time between the parent receiving a shutdown signal (TERM by default) and it sending that signal on to the child process. Designed to give the child process
@@ -651,7 +687,7 @@ require 'your/app'
651
687
  require 'resque/tasks'
652
688
  ```
653
689
 
654
- If you're using Rails 5.x, include the following in lib/tasks/resque.rb:
690
+ If you're using Rails 5.x, include the following in lib/tasks/resque.rake:
655
691
 
656
692
  ```ruby
657
693
  require 'resque/tasks'
@@ -770,15 +806,16 @@ Here's our `config/resque.yml`:
770
806
  test: localhost:6379
771
807
  staging: redis1.se.github.com:6379
772
808
  fi: localhost:6379
773
- production: redis1.ae.github.com:6379
809
+ production: <%= ENV['REDIS_URL'] %>
774
810
 
775
811
  And our initializer:
776
812
 
777
813
  ``` ruby
778
814
  rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
779
815
  rails_env = ENV['RAILS_ENV'] || 'development'
816
+ config_file = rails_root + '/config/resque.yml'
780
817
 
781
- resque_config = YAML.load_file(rails_root + '/config/resque.yml')
818
+ resque_config = YAML::load(ERB.new(IO.read(config_file)).result)
782
819
  Resque.redis = resque_config[rails_env]
783
820
  ```
784
821
 
@@ -112,8 +112,8 @@ module Resque
112
112
  def redis=(server)
113
113
  case server
114
114
  when String
115
- if server =~ /redis\:\/\//
116
- redis = Redis.connect(:url => server, :thread_safe => true)
115
+ if server =~ /rediss?\:\/\//
116
+ redis = Redis.new(:url => server, :thread_safe => true)
117
117
  else
118
118
  server, namespace = server.split('/', 2)
119
119
  host, port, db = server.split(':')
@@ -147,26 +147,76 @@ module Resque
147
147
  data_store.identifier
148
148
  end
149
149
 
150
+ # Set the data store for the processed and failed statistics.
151
+ #
152
+ # By default it uses the same as `Resque.redis`, but different stores can be used.
153
+ #
154
+ # A custom store needs to obey the following API to work correctly
155
+ #
156
+ # class NullDataStore
157
+ # # Returns the current value for the given stat.
158
+ # def stat(stat)
159
+ # end
160
+ #
161
+ # # Increments the stat by the given value.
162
+ # def increment_stat(stat, by)
163
+ # end
164
+ #
165
+ # # Decrements the stat by the given value.
166
+ # def decrement_stat(stat, by)
167
+ # end
168
+ #
169
+ # # Clear the values for the given stat.
170
+ # def clear_stat(stat)
171
+ # end
172
+ # end
173
+ def stat_data_store=(stat_data_store)
174
+ Resque::Stat.data_store = stat_data_store
175
+ end
176
+
177
+ # Returns the data store for the statistics module.
178
+ def stat_data_store
179
+ Resque::Stat.data_store
180
+ end
181
+
150
182
  # Set or retrieve the current logger object
151
183
  attr_accessor :logger
152
184
 
153
185
  DEFAULT_HEARTBEAT_INTERVAL = 60
154
186
  DEFAULT_PRUNE_INTERVAL = DEFAULT_HEARTBEAT_INTERVAL * 5
155
187
 
188
+ # Defines how often a Resque worker updates the heartbeat key. Must be less
189
+ # than the prune interval.
156
190
  attr_writer :heartbeat_interval
157
191
  def heartbeat_interval
158
- @heartbeat_interval || DEFAULT_HEARTBEAT_INTERVAL
192
+ if defined? @heartbeat_interval
193
+ @heartbeat_interval
194
+ else
195
+ DEFAULT_HEARTBEAT_INTERVAL
196
+ end
159
197
  end
160
198
 
199
+ # Defines how often Resque checks for dead workers.
161
200
  attr_writer :prune_interval
162
201
  def prune_interval
163
- @prune_interval || DEFAULT_PRUNE_INTERVAL
202
+ if defined? @prune_interval
203
+ @prune_interval
204
+ else
205
+ DEFAULT_PRUNE_INTERVAL
206
+ end
164
207
  end
165
208
 
209
+ # By default, jobs are pushed to the back of the queue and popped from
210
+ # the front, resulting in "first in, first out" (FIFO) execution order.
211
+ # Set to true to push jobs to the front of the queue instead, resulting
212
+ # in "last in, first out" (LIFO) execution order.
166
213
  attr_writer :enqueue_front
167
214
  def enqueue_front
168
- return @enqueue_front unless @enqueue_front.nil?
169
- @enqueue_front = false
215
+ if defined? @enqueue_front
216
+ @enqueue_front
217
+ else
218
+ @enqueue_front = false
219
+ end
170
220
  end
171
221
 
172
222
  # The `before_first_fork` hook will be run in the **parent** process
@@ -424,7 +474,7 @@ module Resque
424
474
  # Given a class, try to extrapolate an appropriate queue based on a
425
475
  # class instance variable or `queue` method.
426
476
  def queue_from_class(klass)
427
- klass.instance_variable_get(:@queue) ||
477
+ (klass.instance_variable_defined?(:@queue) && klass.instance_variable_get(:@queue)) ||
428
478
  (klass.respond_to?(:queue) and klass.queue)
429
479
  end
430
480
 
@@ -483,7 +533,7 @@ module Resque
483
533
  # Returns a hash, similar to redis-rb's #info, of interesting stats.
484
534
  def info
485
535
  return {
486
- :pending => queue_sizes.inject(0) { |sum, (queue_name, queue_size)| sum + queue_size },
536
+ :pending => queue_sizes.inject(0) { |sum, (_queue_name, queue_size)| sum + queue_size },
487
537
  :processed => Stat[:processed],
488
538
  :queues => queues.size,
489
539
  :workers => workers.size.to_i,
@@ -543,6 +593,8 @@ module Resque
543
593
 
544
594
  private
545
595
 
596
+ @hooks = Hash.new { |h, k| h[k] = [] }
597
+
546
598
  # Register a new proc as a hook. If the block is nil this is the
547
599
  # equivalent of removing all hooks of the given name.
548
600
  #
@@ -550,26 +602,18 @@ module Resque
550
602
  def register_hook(name, block)
551
603
  return clear_hooks(name) if block.nil?
552
604
 
553
- @hooks ||= {}
554
- @hooks[name] ||= []
555
-
556
605
  block = Array(block)
557
606
  @hooks[name].concat(block)
558
607
  end
559
608
 
560
609
  # Clear all hooks given a hook name.
561
610
  def clear_hooks(name)
562
- @hooks && @hooks[name] = []
563
- end
564
-
565
- # Retrieve all hooks
566
- def hooks
567
- @hooks || {}
611
+ @hooks[name] = []
568
612
  end
569
613
 
570
614
  # Retrieve all hooks of a given name.
571
615
  def hooks(name)
572
- (@hooks && @hooks[name]) || []
616
+ @hooks[name]
573
617
  end
574
618
  end
575
619
 
@@ -44,14 +44,20 @@ module Resque
44
44
  :heartbeat!,
45
45
  :remove_heartbeat,
46
46
  :all_heartbeats,
47
+ :acquire_pruning_dead_worker_lock,
47
48
  :set_worker_payload,
48
49
  :worker_start_time,
49
50
  :worker_done_working
50
51
 
51
- def_delegators :@stats_access, :clear_stat,
52
- :decremet_stat,
53
- :increment_stat,
54
- :stat
52
+ def_delegators :@stats_access, :clear_stat,
53
+ :decrement_stat,
54
+ :increment_stat,
55
+ :stat
56
+
57
+ def decremet_stat(*args)
58
+ warn '[Resque] [Deprecation] Resque::DataStore #decremet_stat method is deprecated (please use #decrement_stat)'
59
+ decrement_stat(*args)
60
+ end
55
61
 
56
62
  # Compatibility with any non-Resque classes that were using Resque.redis as a way to access Redis
57
63
  def method_missing(sym,*args,&block)
@@ -67,19 +73,12 @@ module Resque
67
73
  # Get a string identifying the underlying server.
68
74
  # Probably should be private, but was public so must stay public
69
75
  def identifier
70
- # support 1.x versions of redis-rb
71
- if @redis.respond_to?(:server)
72
- @redis.server
73
- elsif @redis.respond_to?(:nodes) # distributed
74
- @redis.nodes.map { |n| n.id }.join(', ')
75
- else
76
- @redis.client.id
77
- end
76
+ @redis.inspect
78
77
  end
79
78
 
80
79
  # Force a reconnect to Redis.
81
80
  def reconnect
82
- @redis.client.reconnect
81
+ @redis._client.reconnect
83
82
  end
84
83
 
85
84
  # Returns an array of all known Resque keys in Redis. Redis' KEYS operation
@@ -91,26 +90,10 @@ module Resque
91
90
  end
92
91
 
93
92
  def server_time
94
- time, _ = redis_time_available? ? @redis.time : Time.now
93
+ time, _ = @redis.time
95
94
  Time.at(time)
96
95
  end
97
96
 
98
- # only needed until support for Redis 2.4 is removed
99
- def redis_time_available?
100
- if @redis_time_available.nil? && !Resque.inline
101
- begin
102
- @redis_time_available = @redis.time
103
- rescue Redis::CommandError
104
- @redis_time_available = false
105
- end
106
- elsif Resque.inline
107
- false
108
- else
109
- @redis_time_available
110
- end
111
- end
112
- private :redis_time_available?
113
-
114
97
  class QueueAccess
115
98
  def initialize(redis)
116
99
  @redis = redis
@@ -291,6 +274,10 @@ module Resque
291
274
  @redis.hgetall(HEARTBEAT_KEY)
292
275
  end
293
276
 
277
+ def acquire_pruning_dead_worker_lock(worker, expiry)
278
+ @redis.set(redis_key_for_worker_pruning, worker.to_s, :ex => expiry, :nx => true)
279
+ end
280
+
294
281
  def set_worker_payload(worker, data)
295
282
  @redis.set(redis_key_for_worker(worker), data)
296
283
  end
@@ -315,6 +302,10 @@ module Resque
315
302
  def redis_key_for_worker_start_time(worker)
316
303
  "#{redis_key_for_worker(worker)}:started"
317
304
  end
305
+
306
+ def redis_key_for_worker_pruning
307
+ "pruning_dead_workers_in_progress"
308
+ end
318
309
  end
319
310
 
320
311
  class StatsAccess
@@ -14,7 +14,13 @@ module Resque
14
14
  super message
15
15
  end
16
16
  end
17
- class PruneDeadWorkerDirtyExit < DirtyExit; end
17
+
18
+ class PruneDeadWorkerDirtyExit < DirtyExit
19
+ def initialize(hostname, job)
20
+ job ||= "<Unknown Job>"
21
+ super("Worker #{hostname} did not gracefully exit while processing #{job}")
22
+ end
23
+ end
18
24
 
19
25
  # Raised when child process is TERM'd so job can rescue this to do shutdown work.
20
26
  class TermException < SignalException; end
@@ -27,6 +27,7 @@ module Resque
27
27
  def self.backend=(backend)
28
28
  @backend = backend
29
29
  end
30
+ self.backend = nil
30
31
 
31
32
  # Returns the current backend class. If none has been set, falls
32
33
  # back to `Resque::Failure::Redis`
@@ -9,7 +9,7 @@ module Resque
9
9
  class Airbrake < Base
10
10
  def self.configure(&block)
11
11
  Resque.logger.warn "This actually sets global Airbrake configuration, " \
12
- "which is probably not what you want. This will be gone in 2.0."
12
+ "which is probably not what you want."
13
13
  Resque::Failure.backend = self
14
14
  ::Airbrake.configure(&block)
15
15
  end
@@ -21,12 +21,24 @@ module Resque
21
21
  end
22
22
 
23
23
  def save
24
- ::Airbrake.notify(exception,
25
- :parameters => {
26
- :payload_class => payload['class'].to_s,
27
- :payload_args => payload['args'].inspect
28
- }
29
- )
24
+ notify(
25
+ exception,
26
+ parameters: {
27
+ payload_class: payload['class'].to_s,
28
+ payload_args: payload['args'].inspect
29
+ }
30
+ )
31
+ end
32
+
33
+ private
34
+
35
+ def notify(exception, options)
36
+ if ::Airbrake.respond_to?(:notify_sync)
37
+ ::Airbrake.notify_sync(exception, options)
38
+ else
39
+ # Older versions of Airbrake (< 5)
40
+ ::Airbrake.notify(exception, options)
41
+ end
30
42
  end
31
43
  end
32
44
  end
@@ -60,7 +60,7 @@ module Resque
60
60
  end
61
61
  items.each_with_index do |item, i|
62
62
  if !class_name || (item['payload'] && item['payload']['class'] == class_name)
63
- id = reversed ? (items.length - 1) - (offset + i) : offset + i
63
+ id = reversed ? (items.length - 1) + (offset - i) : offset + i
64
64
  yield id, item
65
65
  end
66
66
  end
@@ -5,7 +5,7 @@ require 'resque/version'
5
5
  require 'time'
6
6
  require 'yaml'
7
7
 
8
- if defined? Encoding
8
+ if defined?(Encoding) && Encoding.default_external != Encoding::UTF_8
9
9
  Encoding.default_external = Encoding::UTF_8
10
10
  end
11
11
 
@@ -42,10 +42,6 @@ module Resque
42
42
  end
43
43
  alias_method :u, :url_path
44
44
 
45
- def redirect_url_path(*path_parts)
46
- [ path_prefix, path_parts ].join("/").squeeze('/')
47
- end
48
-
49
45
  def path_prefix
50
46
  request.env['SCRIPT_NAME']
51
47
  end
@@ -102,6 +98,8 @@ module Resque
102
98
  Array(args).map do |a|
103
99
  a.to_yaml
104
100
  end.join("\n")
101
+ rescue
102
+ args.to_s
105
103
  end
106
104
 
107
105
  def worker_hosts
@@ -131,7 +129,7 @@ module Resque
131
129
  end
132
130
 
133
131
  def poll
134
- if @polling
132
+ if defined?(@polling) && @polling
135
133
  text = "Last Updated: #{Time.now.strftime("%H:%M:%S")}"
136
134
  else
137
135
  text = "<a href='#{u(request.path_info)}.poll' rel='poll'>Live Poll</a>"
@@ -158,7 +156,7 @@ module Resque
158
156
 
159
157
  # to make things easier on ourselves
160
158
  get "/?" do
161
- redirect redirect_url_path(:overview)
159
+ redirect url_path(:overview)
162
160
  end
163
161
 
164
162
  %w( overview workers ).each do |page|
@@ -219,7 +217,7 @@ module Resque
219
217
 
220
218
  post "/failed/:queue/requeue/all" do
221
219
  Resque::Failure.requeue_queue Resque::Failure.job_queue_name(params[:queue])
222
- redirect redirect_url_path("/failed/#{params[:queue]}")
220
+ redirect url_path("/failed/#{params[:queue]}")
223
221
  end
224
222
 
225
223
  get "/failed/requeue/:index/?" do
@@ -251,7 +249,7 @@ module Resque
251
249
  end
252
250
 
253
251
  get "/stats/?" do
254
- redirect redirect_url_path("/stats/resque")
252
+ redirect url_path("/stats/resque")
255
253
  end
256
254
 
257
255
  get "/stats/:id/?" do
@@ -6,10 +6,10 @@
6
6
  <% else %>
7
7
  <dt>Worker</dt>
8
8
  <dd>
9
- <a href="<%= u(:workers, job['worker']) %>"><%= job['worker'].split(':')[0...2].join(':') %></a> on <b class='queue-tag'><%= job['queue'] %></b > at <b><span class="time"><%= Time.parse(job['failed_at']).strftime(failed_date_format) %></span></b>
9
+ <a href="<%= u(:workers, job['worker']) %>"><%= job['worker'].split(':')[0...2].join(':') %></a> on <b class='queue-tag'><%= job['queue'] %></b > at <b><span class="time"><%= DateTime.parse(job['failed_at']).strftime(failed_date_format) %></span></b>
10
10
  <% if job['retried_at'] %>
11
11
  <div class='retried'>
12
- Retried <b><span class="time"><%= Time.parse(job['retried_at']).strftime(failed_date_format) %></span></b>
12
+ Retried <b><span class="time"><%= DateTime.parse(job['retried_at']).strftime(failed_date_format) %></span></b>
13
13
  <a href="<%= u "#{queue}/remove/#{id}" %>" class="remove" rel="remove">Remove</a>
14
14
  </div>
15
15
  <% else %>
@@ -23,7 +23,7 @@
23
23
  <% end %>
24
24
  </div>
25
25
 
26
- <% if @subtabs %>
26
+ <% if defined?(@subtabs) && @subtabs %>
27
27
  <ul class='subnav'>
28
28
  <% for subtab in @subtabs %>
29
29
  <li <%= class_if_current "#{current_section}/#{subtab}" %>><a href="<%= current_section %>/<%= subtab %>"><span><%= subtab %></span></a></li>
@@ -1,18 +1,18 @@
1
1
  <% @subtabs = resque.queues unless partial? || params[:id].nil? %>
2
2
 
3
- <% if queue = params[:id] %>
3
+ <% if current_queue = params[:id] %>
4
4
 
5
- <h1>Pending jobs on <span class='hl'><%= queue %></span></h1>
6
- <form method="POST" action="<%=u "/queues/#{queue}/remove" %>" class='remove-queue'>
5
+ <h1>Pending jobs on <span class='hl'><%= current_queue %></span></h1>
6
+ <form method="POST" action="<%=u "/queues/#{current_queue}/remove" %>" class='remove-queue'>
7
7
  <input type='submit' name='' value='Remove Queue' onclick='return confirm("Are you absolutely sure? This cannot be undone.");' />
8
8
  </form>
9
- <p class='sub'><%= page_entries_info start = params[:start].to_i, start + 19, size = resque.size(queue), 'job' %></p>
9
+ <p class='sub'><%= page_entries_info start = params[:start].to_i, start + 19, size = resque.size(current_queue), 'job' %></p>
10
10
  <table class='jobs'>
11
11
  <tr>
12
12
  <th>Class</th>
13
13
  <th>Args</th>
14
14
  </tr>
15
- <% for job in (jobs = resque.peek(queue, start, 20)) %>
15
+ <% for job in (jobs = resque.peek(current_queue, start, 20)) %>
16
16
  <tr>
17
17
  <td class='class'><%= partial :job_class, :job => job %></td>
18
18
  <td class='args'><%=h job['args'].inspect %></td>
@@ -24,7 +24,7 @@
24
24
 
25
25
  <h1><%= resque.redis_id %></h1>
26
26
  <table class='stats'>
27
- <% for key, value in resque.redis.info.to_a.sort_by { |i| i[0].to_s } %>
27
+ <% for key, value in resque.redis.redis.info.to_a.sort_by { |i| i[0].to_s } %>
28
28
  <tr>
29
29
  <th>
30
30
  <%= key %>
@@ -1,5 +1,5 @@
1
- <% if params[:id] && (worker = Resque::Worker.find(params[:id])) && (data = worker.job) %>
2
- <h1><%= worker %>'s job</h1>
1
+ <% if params[:id] && (current_worker = Resque::Worker.find(params[:id])) && (data = current_worker.job) %>
2
+ <h1><%= current_worker %>'s job</h1>
3
3
 
4
4
  <table>
5
5
  <tr>
@@ -12,8 +12,8 @@
12
12
  </tr>
13
13
  <tr>
14
14
  <td><img src="<%=u 'working.png' %>" alt="working" title="working"></td>
15
- <% host, pid, _ = worker.to_s.split(':') %>
16
- <td><a href="<%=u "/workers/#{worker}" %>"><%= host %>:<%= pid %></a></td>
15
+ <% host, pid, _ = current_worker.to_s.split(':') %>
16
+ <td><a href="<%=u "/workers/#{current_worker}" %>"><%= host %>:<%= pid %></a></td>
17
17
  <% queue = data['queue'] %>
18
18
  <td><a class="queue" href="<%=u "/queues/#{queue}" %>"><%= queue %></a></td>
19
19
  <td><span class="time"><%= data['run_at'] %></span></td>
@@ -49,10 +49,10 @@
49
49
  </tr>
50
50
  <% end %>
51
51
 
52
- <% worker_jobs.sort_by {|w, j| j['run_at'] ? j['run_at'].to_s() : '' }.each do |worker, job| %>
52
+ <% worker_jobs.sort_by { |_w, j| j['run_at'] ? j['run_at'].to_s() : '' }.each do |worker, job| %>
53
53
  <tr>
54
54
  <td class='icon'><img src="<%=u state = worker.state %>.png" alt="<%= state %>" title="<%= state %>"></td>
55
- <% host, pid, queues = worker.to_s.split(':') %>
55
+ <% host, pid, _queues = worker.to_s.split(':') %>
56
56
  <td class='where'><a href="<%=u "/workers/#{worker}" %>"><%= host %>:<%= pid %></a></td>
57
57
  <td class='queues queue'>
58
58
  <a class="queue-tag" href="<%=u "/queues/#{job['queue']}" %>"><%= job['queue'] %></a>
@@ -7,12 +7,19 @@ module Resque
7
7
  # Kill a stat: Stat.clear(name)
8
8
  module Stat
9
9
  extend self
10
-
11
- # Direct access to the Redis instance.
10
+
12
11
  def redis
13
- Resque.redis
12
+ warn '[Resque] [Deprecation] Resque::Stat #redis method is deprecated (please use #data_strore)'
13
+ data_store
14
+ end
15
+
16
+ def data_store
17
+ @data_store ||= Resque.redis
18
+ end
19
+
20
+ def data_store=(data_store)
21
+ @data_store = data_store
14
22
  end
15
- alias :data_store :redis
16
23
 
17
24
  # Returns the int value of a stat, given a string stat name.
18
25
  def get(stat)
@@ -42,7 +49,7 @@ module Resque
42
49
  # Can optionally accept a second int parameter. The stat is then
43
50
  # decremented by that amount.
44
51
  def decr(stat, by = 1)
45
- data_store.decremet_stat(stat,by)
52
+ data_store.decrement_stat(stat,by)
46
53
  end
47
54
 
48
55
  # Decrements a stat by one.
@@ -40,17 +40,9 @@ namespace :resque do
40
40
  # Preload app files if this is Rails
41
41
  task :preload => :setup do
42
42
  if defined?(Rails)
43
- if Rails::VERSION::MAJOR > 3
43
+ if Rails.application.config.eager_load
44
44
  ActiveSupport.run_load_hooks(:before_eager_load, Rails.application)
45
45
  Rails.application.config.eager_load_namespaces.each(&:eager_load!)
46
-
47
- elsif Rails::VERSION::MAJOR == 3
48
- ActiveSupport.run_load_hooks(:before_eager_load, Rails.application)
49
- Rails.application.eager_load!
50
-
51
- elsif defined?(Rails::Initializer)
52
- $rails_rake_task = false
53
- Rails::Initializer.run :load_application_classes
54
46
  end
55
47
  end
56
48
  end
@@ -1,45 +1,24 @@
1
1
  class Resque::ThreadSignal
2
- if RUBY_VERSION <= "1.9"
3
- def initialize
4
- @signaled = false
5
- end
2
+ def initialize
3
+ @mutex = Mutex.new
4
+ @signaled = false
5
+ @received = ConditionVariable.new
6
+ end
6
7
 
7
- def signal
8
+ def signal
9
+ @mutex.synchronize do
8
10
  @signaled = true
11
+ @received.signal
9
12
  end
13
+ end
10
14
 
11
- def wait_for_signal(timeout)
12
- (10 * timeout).times do
13
- sleep(0.1)
14
- return true if @signaled
15
+ def wait_for_signal(timeout)
16
+ @mutex.synchronize do
17
+ unless @signaled
18
+ @received.wait(@mutex, timeout)
15
19
  end
16
20
 
17
21
  @signaled
18
22
  end
19
-
20
- else
21
- def initialize
22
- @mutex = Mutex.new
23
- @signaled = false
24
- @received = ConditionVariable.new
25
- end
26
-
27
- def signal
28
- @mutex.synchronize do
29
- @signaled = true
30
- @received.signal
31
- end
32
- end
33
-
34
- def wait_for_signal(timeout)
35
- @mutex.synchronize do
36
- unless @signaled
37
- @received.wait(@mutex, timeout)
38
- end
39
-
40
- @signaled
41
- end
42
- end
43
-
44
23
  end
45
24
  end
@@ -7,7 +7,8 @@ module UTF8Util
7
7
  #
8
8
  # Returns self as valid UTF-8.
9
9
  def self.clean!(str)
10
- raise NotImplementedError
10
+ return str if str.encoding.to_s == "UTF-8"
11
+ str.force_encoding("binary").encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => REPLACEMENT_CHAR)
11
12
  end
12
13
 
13
14
  # Replace invalid UTF-8 character sequences with a replacement character
@@ -16,11 +17,4 @@ module UTF8Util
16
17
  def self.clean(str)
17
18
  clean!(str.dup)
18
19
  end
19
-
20
- end
21
-
22
- if RUBY_VERSION <= '1.9'
23
- require 'resque/vendor/utf8_util/utf8_util_18'
24
- else
25
- require 'resque/vendor/utf8_util/utf8_util_19'
26
20
  end
@@ -1,3 +1,3 @@
1
1
  module Resque
2
- Version = VERSION = '1.27.4'
2
+ Version = VERSION = '2.0.0'
3
3
  end
@@ -145,6 +145,9 @@ module Resque
145
145
  @paused = nil
146
146
  @before_first_fork_hook_ran = false
147
147
 
148
+ @heartbeat_thread = nil
149
+ @heartbeat_thread_signal = nil
150
+
148
151
  verbose_value = ENV['LOGGING'] || ENV['VERBOSE']
149
152
  self.verbose = verbose_value if verbose_value
150
153
  self.very_verbose = ENV['VVERBOSE'] if ENV['VVERBOSE']
@@ -162,9 +165,6 @@ module Resque
162
165
  # once per worker.
163
166
  def prepare
164
167
  if ENV['BACKGROUND']
165
- unless Process.respond_to?('daemon')
166
- abort "env var BACKGROUND is set, which requires ruby >= 1.9"
167
- end
168
168
  Process.daemon(true)
169
169
  end
170
170
 
@@ -175,12 +175,12 @@ module Resque
175
175
  self.reconnect if ENV['BACKGROUND']
176
176
  end
177
177
 
178
+ WILDCARDS = ['*', '?', '{', '}', '[', ']'].freeze
179
+
178
180
  def queues=(queues)
179
181
  queues = queues.empty? ? (ENV["QUEUES"] || ENV['QUEUE']).to_s.split(',') : queues
180
182
  @queues = queues.map { |queue| queue.to_s.strip }
181
- unless ['*', '?', '{', '}', '[', ']'].any? {|char| @queues.join.include?(char) }
182
- @static_queues = @queues.flatten.uniq
183
- end
183
+ @has_dynamic_queues = WILDCARDS.any? {|char| @queues.join.include?(char) }
184
184
  validate_queues
185
185
  end
186
186
 
@@ -198,12 +198,16 @@ module Resque
198
198
  # A splat ("*") means you want every queue (in alpha order) - this
199
199
  # can be useful for dynamically adding new queues.
200
200
  def queues
201
- return @static_queues if @static_queues
202
- @queues.map { |queue| glob_match(queue) }.flatten.uniq
201
+ if @has_dynamic_queues
202
+ current_queues = Resque.queues
203
+ @queues.map { |queue| glob_match(current_queues, queue) }.flatten.uniq
204
+ else
205
+ @queues
206
+ end
203
207
  end
204
208
 
205
- def glob_match(pattern)
206
- Resque.queues.select do |queue|
209
+ def glob_match(list, pattern)
210
+ list.select do |queue|
207
211
  File.fnmatch?(pattern, queue)
208
212
  end.sort
209
213
  end
@@ -588,6 +592,8 @@ module Resque
588
592
  # By checking the current Redis state against the actual
589
593
  # environment, we can determine if Redis is old and clean it up a bit.
590
594
  def prune_dead_workers
595
+ return unless data_store.acquire_pruning_dead_worker_lock(self, Resque.heartbeat_interval)
596
+
591
597
  all_workers = Worker.all
592
598
 
593
599
  unless all_workers.empty?
@@ -604,7 +610,9 @@ module Resque
604
610
  # client library or an older version of Resque. We won't touch these.
605
611
  if all_workers_with_expired_heartbeats.include?(worker)
606
612
  log_with_severity :info, "Pruning dead worker: #{worker}"
607
- worker.unregister_worker(PruneDeadWorkerDirtyExit.new(worker.to_s))
613
+
614
+ job_class = worker.job(false)['payload']['class'] rescue nil
615
+ worker.unregister_worker(PruneDeadWorkerDirtyExit.new(worker.to_s, job_class))
608
616
  next
609
617
  end
610
618
 
@@ -635,7 +643,8 @@ module Resque
635
643
 
636
644
  # Runs a named hook, passing along any arguments.
637
645
  def run_hook(name, *args)
638
- return unless hooks = Resque.send(name)
646
+ hooks = Resque.send(name)
647
+ return if hooks.empty?
639
648
  return if name == :before_first_fork && @before_first_fork_hook_ran
640
649
  msg = "Running #{name} hooks"
641
650
  msg << " with #{args.inspect}" if args.any?
@@ -808,7 +817,7 @@ module Resque
808
817
  # machine. Useful when pruning dead workers on startup.
809
818
  def windows_worker_pids
810
819
  tasklist_output = `tasklist /FI "IMAGENAME eq ruby.exe" /FO list`.encode("UTF-8", Encoding.locale_charmap)
811
- tasklist_output.split($/).select { |line| line =~ /^PID:/}.collect{ |line| line.gsub /PID:\s+/, '' }
820
+ tasklist_output.split($/).select { |line| line =~ /^PID:/ }.collect { |line| line.gsub(/PID:\s+/, '') }
812
821
  end
813
822
 
814
823
  # Find Resque worker pids on Linux and OS X.
@@ -850,13 +859,7 @@ module Resque
850
859
  end
851
860
 
852
861
 
853
- def verbose
854
- @verbose
855
- end
856
-
857
- def very_verbose
858
- @very_verbose
859
- end
862
+ attr_reader :verbose, :very_verbose
860
863
 
861
864
  def verbose=(value);
862
865
  if value && !very_verbose
@@ -909,7 +912,7 @@ module Resque
909
912
  nil
910
913
  end
911
914
 
912
- job.fail(DirtyExit.new("Child process received unhandled signal #{$?.stopsig}", $?)) if $?.signaled?
915
+ job.fail(DirtyExit.new("Child process received unhandled signal #{$?}", $?)) if $?.signaled?
913
916
  @child = nil
914
917
  end
915
918
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: resque
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.27.4
4
+ version: 2.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Chris Wanstrath
@@ -10,7 +10,7 @@ authors:
10
10
  autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
- date: 2017-04-15 00:00:00.000000000 Z
13
+ date: 2018-11-06 00:00:00.000000000 Z
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency
16
16
  name: redis-namespace
@@ -18,14 +18,14 @@ dependencies:
18
18
  requirements:
19
19
  - - "~>"
20
20
  - !ruby/object:Gem::Version
21
- version: '1.3'
21
+ version: '1.6'
22
22
  type: :runtime
23
23
  prerelease: false
24
24
  version_requirements: !ruby/object:Gem::Requirement
25
25
  requirements:
26
26
  - - "~>"
27
27
  - !ruby/object:Gem::Version
28
- version: '1.3'
28
+ version: '1.6'
29
29
  - !ruby/object:Gem::Dependency
30
30
  name: vegas
31
31
  requirement: !ruby/object:Gem::Requirement
@@ -159,8 +159,6 @@ files:
159
159
  - lib/resque/tasks.rb
160
160
  - lib/resque/thread_signal.rb
161
161
  - lib/resque/vendor/utf8_util.rb
162
- - lib/resque/vendor/utf8_util/utf8_util_18.rb
163
- - lib/resque/vendor/utf8_util/utf8_util_19.rb
164
162
  - lib/resque/version.rb
165
163
  - lib/resque/worker.rb
166
164
  - lib/tasks/redis.rake
@@ -178,7 +176,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
178
176
  requirements:
179
177
  - - ">="
180
178
  - !ruby/object:Gem::Version
181
- version: '0'
179
+ version: 2.3.0
182
180
  required_rubygems_version: !ruby/object:Gem::Requirement
183
181
  requirements:
184
182
  - - ">="
@@ -186,7 +184,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
186
184
  version: '0'
187
185
  requirements: []
188
186
  rubyforge_project:
189
- rubygems_version: 2.6.11
187
+ rubygems_version: 2.7.6
190
188
  signing_key:
191
189
  specification_version: 4
192
190
  summary: Resque is a Redis-backed queueing system.
@@ -1,91 +0,0 @@
1
- require 'strscan'
2
-
3
- module UTF8Util
4
- HIGH_BIT_RANGE = /[\x80-\xff]/
5
-
6
- # Check if this String is valid UTF-8
7
- #
8
- # Returns true or false.
9
- def self.valid?(str)
10
- sc = StringScanner.new(str)
11
-
12
- while sc.skip_until(HIGH_BIT_RANGE)
13
- sc.pos -= 1
14
-
15
- if !sequence_length(sc)
16
- return false
17
- end
18
- end
19
-
20
- true
21
- end
22
-
23
- # Replace invalid UTF-8 character sequences with a replacement character
24
- #
25
- # Returns self as valid UTF-8.
26
- def self.clean!(str)
27
- sc = StringScanner.new(str)
28
- while sc.skip_until(HIGH_BIT_RANGE)
29
- pos = sc.pos = sc.pos-1
30
-
31
- if !sequence_length(sc)
32
- str[pos] = REPLACEMENT_CHAR
33
- end
34
- end
35
-
36
- str
37
- end
38
-
39
- # Validate the UTF-8 sequence at the current scanner position.
40
- #
41
- # scanner - StringScanner instance so we can advance the pointer as we verify.
42
- #
43
- # Returns The length in bytes of this UTF-8 sequence, false if invalid.
44
- def self.sequence_length(scanner)
45
- leader = scanner.get_byte[0]
46
-
47
- if (leader >> 5) == 0x6
48
- if check_next_sequence(scanner)
49
- return 2
50
- else
51
- scanner.pos -= 1
52
- end
53
- elsif (leader >> 4) == 0x0e
54
- if check_next_sequence(scanner)
55
- if check_next_sequence(scanner)
56
- return 3
57
- else
58
- scanner.pos -= 2
59
- end
60
- else
61
- scanner.pos -= 1
62
- end
63
- elsif (leader >> 3) == 0x1e
64
- if check_next_sequence(scanner)
65
- if check_next_sequence(scanner)
66
- if check_next_sequence(scanner)
67
- return 4
68
- else
69
- scanner.pos -= 3
70
- end
71
- else
72
- scanner.pos -= 2
73
- end
74
- else
75
- scanner.pos -= 1
76
- end
77
- end
78
-
79
- false
80
- end
81
-
82
- private
83
-
84
- # Read another byte off the scanner oving the scan position forward one place
85
- #
86
- # Returns nothing.
87
- def self.check_next_sequence(scanner)
88
- byte = scanner.get_byte[0]
89
- (byte >> 6) == 0x2
90
- end
91
- end
@@ -1,6 +0,0 @@
1
- module UTF8Util
2
- def self.clean!(str)
3
- return str if str.encoding.to_s == "UTF-8"
4
- str.force_encoding("binary").encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => REPLACEMENT_CHAR)
5
- end
6
- end