resque 2.2.1 → 2.3.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of resque might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/HISTORY.md +9 -1
- data/README.markdown +24 -42
- data/lib/resque/data_store.rb +26 -26
- data/lib/resque/server/views/next_more.erb +2 -2
- data/lib/resque/stat.rb +4 -4
- data/lib/resque/tasks.rb +1 -1
- data/lib/resque/version.rb +1 -1
- data/lib/resque/web_runner.rb +4 -4
- data/lib/resque/worker.rb +9 -9
- data/lib/resque.rb +5 -5
- metadata +7 -6
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 26c77950669242c68e75f64480a798486a84ac3aa2c4bd22c5ae212caa6bf57c
|
4
|
+
data.tar.gz: bea29587e61e5b2a41c675ff26afb69cca7891613212e0d052a41a43ce5404b4
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 81f58c37a382d84e6f168ad9ada384ac160c223fa5cc1070c251ad01c8840389af78736af263aa3cd594dcd77970a2539a7f44082dca07a25f236c6d147625b4
|
7
|
+
data.tar.gz: f28ffa6efb6e8037cf3ec7d233f824b7190f97d91c4d3d6ba00399e6533f2958581e98422a31068a9145dd789f63d1caff58ca8744905a1cf747b02eaeed9d02
|
data/HISTORY.md
CHANGED
@@ -1,9 +1,17 @@
|
|
1
|
-
##
|
1
|
+
## 2.3.0
|
2
2
|
|
3
3
|
### Fixed
|
4
4
|
|
5
|
+
* Fix NoMethodError: undefined method 'application' for Rails:Module when Rails module is defined but not a full Rails app (#1799)
|
6
|
+
* Fix deprecation warnings in Redis#pipelined for redis 4.6 (#1806)
|
7
|
+
* Add Ruby syntax highlighting to remaining markdown docs (#1802)
|
8
|
+
* Fix pagination section (#1809)
|
9
|
+
* Fix `before_run` undefined local variable or method `runner` (#1811)
|
10
|
+
|
5
11
|
### Added
|
6
12
|
|
13
|
+
* Added support for pausing all workers by setting the Redis key `pause-all-workers` to string value "true" (#1803)
|
14
|
+
|
7
15
|
## 2.2.1
|
8
16
|
|
9
17
|
### Fixed
|
data/README.markdown
CHANGED
@@ -4,47 +4,6 @@ Resque
|
|
4
4
|
[![Gem Version](https://badge.fury.io/rb/resque.svg)](https://rubygems.org/gems/resque)
|
5
5
|
[![Build Status](https://github.com/resque/resque/actions/workflows/ci.yml/badge.svg)](https://github.com/resque/resque/actions/workflows/ci.yml)
|
6
6
|
|
7
|
-
Resque is a Redis-backed queueing system.
|
8
|
-
|
9
|
-
---------
|
10
|
-
### Note on the future of this repo
|
11
|
-
|
12
|
-
Would you like to be involved in Resque? Do you have thoughts about what
|
13
|
-
Resque should be and do going forward? There's currently an [open discussion here](https://github.com/resque/resque/issues/1759)
|
14
|
-
on just that topic, so please feel free to join in. We'd love to hear your thoughts
|
15
|
-
and/or have people volunteer to be a part of the project!
|
16
|
-
|
17
|
-
---------
|
18
|
-
|
19
|
-
Table of Contents
|
20
|
-
-----------------
|
21
|
-
|
22
|
-
* [Introduction](#introduction)
|
23
|
-
* [Example](#example)
|
24
|
-
* [Installation](#installation)
|
25
|
-
* [Running Workers](#running-workers)
|
26
|
-
* [The Front End](#the-front-end)
|
27
|
-
* [Jobs](#jobs)
|
28
|
-
* [Configuration](#configuration)
|
29
|
-
* * [Redis](#redis)
|
30
|
-
* * [Logging](#logging)
|
31
|
-
* * [Namespaces](#namespaces)
|
32
|
-
* * [Storing Statistics](#storing-statistics)
|
33
|
-
* [Plugins and Hooks](#plugins-and-hooks)
|
34
|
-
* [Additional Information](#additional-information)
|
35
|
-
* * [Resque vs DelayedJob](#resque-vs-delayedjob)
|
36
|
-
* * [Forking](#forking)
|
37
|
-
* * [Signals](#signals)
|
38
|
-
* * [Heroku](#heroku)
|
39
|
-
* * [Monitoring](#monitoring)
|
40
|
-
* * [Mysql::Error](#mysqlerror-mysql-server-has-gone-away)
|
41
|
-
* [Development](#development)
|
42
|
-
* * [Demo](#demo)
|
43
|
-
* * [Contributing](#contributing)
|
44
|
-
* [Questions](#questions)
|
45
|
-
* [Meta](#meta)
|
46
|
-
* [Author](#author)
|
47
|
-
|
48
7
|
Introduction
|
49
8
|
------------
|
50
9
|
|
@@ -83,6 +42,13 @@ general usage stats, and helps you track failures.
|
|
83
42
|
Resque now supports Ruby 2.3.0 and above.
|
84
43
|
We will also only be supporting Redis 3.0 and above going forward.
|
85
44
|
|
45
|
+
### Note on the future of Resque
|
46
|
+
|
47
|
+
Would you like to be involved in Resque? Do you have thoughts about what
|
48
|
+
Resque should be and do going forward? There's currently an [open discussion here](https://github.com/resque/resque/issues/1759)
|
49
|
+
on just that topic, so please feel free to join in. We'd love to hear your thoughts
|
50
|
+
and/or have people volunteer to be a part of the project!
|
51
|
+
|
86
52
|
Example
|
87
53
|
-------
|
88
54
|
|
@@ -750,7 +716,7 @@ If you want to kill a stale or stuck child and shutdown, use `TERM`
|
|
750
716
|
|
751
717
|
If you want to stop processing jobs, but want to leave the worker running
|
752
718
|
(for example, to temporarily alleviate load), use `USR2` to stop processing,
|
753
|
-
then `CONT` to start it again.
|
719
|
+
then `CONT` to start it again. It's also possible to [pause all workers](#pausing-all-workers).
|
754
720
|
|
755
721
|
#### Heroku
|
756
722
|
|
@@ -774,6 +740,22 @@ time to complete before being forced to die.
|
|
774
740
|
|
775
741
|
* `RESQUE_TERM_TIMEOUT` - By default you have a few seconds to handle `Resque::TermException` in your job. `RESQUE_TERM_TIMEOUT` and `RESQUE_PRE_SHUTDOWN_TIMEOUT` must be lower than the [heroku dyno timeout](https://devcenter.heroku.com/articles/limits#exit-timeout).
|
776
742
|
|
743
|
+
#### Pausing all workers
|
744
|
+
|
745
|
+
Workers will not process pending jobs if the Redis key `pause-all-workers` is set with the string value "true".
|
746
|
+
|
747
|
+
``` ruby
|
748
|
+
Resque.redis.set('pause-all-workers', 'true')
|
749
|
+
```
|
750
|
+
|
751
|
+
Nothing happens to jobs that are already being processed by workers.
|
752
|
+
|
753
|
+
Unpause by removing the Redis key `pause-all-workers`.
|
754
|
+
|
755
|
+
``` ruby
|
756
|
+
Resque.redis.del('pause-all-workers')
|
757
|
+
```
|
758
|
+
|
777
759
|
#### Monitoring
|
778
760
|
|
779
761
|
##### god
|
data/lib/resque/data_store.rb
CHANGED
@@ -100,9 +100,9 @@ module Resque
|
|
100
100
|
@redis = redis
|
101
101
|
end
|
102
102
|
def push_to_queue(queue,encoded_item)
|
103
|
-
@redis.pipelined do
|
104
|
-
watch_queue(queue)
|
105
|
-
|
103
|
+
@redis.pipelined do |piped|
|
104
|
+
watch_queue(queue, redis: piped)
|
105
|
+
piped.rpush redis_key_for_queue(queue), encoded_item
|
106
106
|
end
|
107
107
|
end
|
108
108
|
|
@@ -129,9 +129,9 @@ module Resque
|
|
129
129
|
end
|
130
130
|
|
131
131
|
def remove_queue(queue)
|
132
|
-
@redis.pipelined do
|
133
|
-
|
134
|
-
|
132
|
+
@redis.pipelined do |piped|
|
133
|
+
piped.srem(:queues, queue.to_s)
|
134
|
+
piped.del(redis_key_for_queue(queue))
|
135
135
|
end
|
136
136
|
end
|
137
137
|
|
@@ -145,8 +145,8 @@ module Resque
|
|
145
145
|
end
|
146
146
|
|
147
147
|
# Private: do not call
|
148
|
-
def watch_queue(queue)
|
149
|
-
|
148
|
+
def watch_queue(queue, redis: @redis)
|
149
|
+
redis.sadd(:queues, queue.to_s)
|
150
150
|
end
|
151
151
|
|
152
152
|
# Private: do not call
|
@@ -237,24 +237,24 @@ module Resque
|
|
237
237
|
end
|
238
238
|
|
239
239
|
def register_worker(worker)
|
240
|
-
@redis.pipelined do
|
241
|
-
|
242
|
-
worker_started(worker)
|
240
|
+
@redis.pipelined do |piped|
|
241
|
+
piped.sadd(:workers, worker)
|
242
|
+
worker_started(worker, redis: piped)
|
243
243
|
end
|
244
244
|
end
|
245
245
|
|
246
|
-
def worker_started(worker)
|
247
|
-
|
246
|
+
def worker_started(worker, redis: @redis)
|
247
|
+
redis.set(redis_key_for_worker_start_time(worker), Time.now.to_s)
|
248
248
|
end
|
249
249
|
|
250
250
|
def unregister_worker(worker, &block)
|
251
|
-
@redis.pipelined do
|
252
|
-
|
253
|
-
|
254
|
-
|
255
|
-
|
251
|
+
@redis.pipelined do |piped|
|
252
|
+
piped.srem(:workers, worker)
|
253
|
+
piped.del(redis_key_for_worker(worker))
|
254
|
+
piped.del(redis_key_for_worker_start_time(worker))
|
255
|
+
piped.hdel(HEARTBEAT_KEY, worker.to_s)
|
256
256
|
|
257
|
-
block.call
|
257
|
+
block.call redis: piped
|
258
258
|
end
|
259
259
|
end
|
260
260
|
|
@@ -288,9 +288,9 @@ module Resque
|
|
288
288
|
end
|
289
289
|
|
290
290
|
def worker_done_working(worker, &block)
|
291
|
-
@redis.pipelined do
|
292
|
-
|
293
|
-
block.call
|
291
|
+
@redis.pipelined do |piped|
|
292
|
+
piped.del(redis_key_for_worker(worker))
|
293
|
+
block.call redis: piped
|
294
294
|
end
|
295
295
|
end
|
296
296
|
|
@@ -317,16 +317,16 @@ module Resque
|
|
317
317
|
@redis.get("stat:#{stat}").to_i
|
318
318
|
end
|
319
319
|
|
320
|
-
def increment_stat(stat, by = 1)
|
321
|
-
|
320
|
+
def increment_stat(stat, by = 1, redis: @redis)
|
321
|
+
redis.incrby("stat:#{stat}", by)
|
322
322
|
end
|
323
323
|
|
324
324
|
def decremet_stat(stat, by = 1)
|
325
325
|
@redis.decrby("stat:#{stat}", by)
|
326
326
|
end
|
327
327
|
|
328
|
-
def clear_stat(stat)
|
329
|
-
|
328
|
+
def clear_stat(stat, redis: @redis)
|
329
|
+
redis.del("stat:#{stat}")
|
330
330
|
end
|
331
331
|
end
|
332
332
|
end
|
@@ -7,7 +7,7 @@
|
|
7
7
|
<a href="<%= current_page %>?start=<%= start - per_page %>" class='less'>« Previous</a>
|
8
8
|
<% end %>
|
9
9
|
|
10
|
-
<% (size / per_page.to_f
|
10
|
+
<% (size / per_page.to_f).ceil.times do |page_num| %>
|
11
11
|
<% if start == page_num * per_page %>
|
12
12
|
<span><%= page_num + 1 %></span>
|
13
13
|
<% else %>
|
@@ -15,7 +15,7 @@
|
|
15
15
|
<% end %>
|
16
16
|
<% end %>
|
17
17
|
|
18
|
-
<% if start + per_page
|
18
|
+
<% if start + per_page < size %>
|
19
19
|
<a href="<%= current_page %>?start=<%= start + per_page %>" class='more'>Next »</a>
|
20
20
|
<% end %>
|
21
21
|
</div>
|
data/lib/resque/stat.rb
CHANGED
@@ -35,8 +35,8 @@ module Resque
|
|
35
35
|
#
|
36
36
|
# Can optionally accept a second int parameter. The stat is then
|
37
37
|
# incremented by that amount.
|
38
|
-
def incr(stat, by = 1)
|
39
|
-
data_store.increment_stat(stat,by)
|
38
|
+
def incr(stat, by = 1, **opts)
|
39
|
+
data_store.increment_stat(stat, by, **opts)
|
40
40
|
end
|
41
41
|
|
42
42
|
# Increments a stat by one.
|
@@ -58,8 +58,8 @@ module Resque
|
|
58
58
|
end
|
59
59
|
|
60
60
|
# Removes a stat from Redis, effectively setting it to 0.
|
61
|
-
def clear(stat)
|
62
|
-
data_store.clear_stat(stat)
|
61
|
+
def clear(stat, **opts)
|
62
|
+
data_store.clear_stat(stat, **opts)
|
63
63
|
end
|
64
64
|
end
|
65
65
|
end
|
data/lib/resque/tasks.rb
CHANGED
@@ -39,7 +39,7 @@ namespace :resque do
|
|
39
39
|
|
40
40
|
# Preload app files if this is Rails
|
41
41
|
task :preload => :setup do
|
42
|
-
if defined?(Rails)
|
42
|
+
if defined?(Rails) && Rails.respond_to?(:application)
|
43
43
|
if Rails.application.config.eager_load
|
44
44
|
ActiveSupport.run_load_hooks(:before_eager_load, Rails.application)
|
45
45
|
Rails.application.config.eager_load_namespaces.each(&:eager_load!)
|
data/lib/resque/version.rb
CHANGED
data/lib/resque/web_runner.rb
CHANGED
@@ -85,14 +85,14 @@ module Resque
|
|
85
85
|
end
|
86
86
|
|
87
87
|
def before_run
|
88
|
-
if (namespace = options[:redis_namespace])
|
89
|
-
runner.logger.info "Using Redis namespace '#{namespace}'"
|
90
|
-
Resque.redis.namespace = namespace
|
91
|
-
end
|
92
88
|
if (redis_conf = options[:redis_conf])
|
93
89
|
logger.info "Using Redis connection '#{redis_conf}'"
|
94
90
|
Resque.redis = redis_conf
|
95
91
|
end
|
92
|
+
if (namespace = options[:redis_namespace])
|
93
|
+
logger.info "Using Redis namespace '#{namespace}'"
|
94
|
+
Resque.redis.namespace = namespace
|
95
|
+
end
|
96
96
|
if (url_prefix = options[:url_prefix])
|
97
97
|
logger.info "Using URL Prefix '#{url_prefix}'"
|
98
98
|
Resque::Server.url_prefix = url_prefix
|
data/lib/resque/worker.rb
CHANGED
@@ -579,7 +579,7 @@ module Resque
|
|
579
579
|
|
580
580
|
# are we paused?
|
581
581
|
def paused?
|
582
|
-
@paused
|
582
|
+
@paused || redis.get('pause-all-workers').to_s.strip.downcase == 'true'
|
583
583
|
end
|
584
584
|
|
585
585
|
# Stop processing jobs after the current one has completed (if we're
|
@@ -697,9 +697,9 @@ module Resque
|
|
697
697
|
|
698
698
|
kill_background_threads
|
699
699
|
|
700
|
-
data_store.unregister_worker(self) do
|
701
|
-
Stat.clear("processed:#{self}")
|
702
|
-
Stat.clear("failed:#{self}")
|
700
|
+
data_store.unregister_worker(self) do |**opts|
|
701
|
+
Stat.clear("processed:#{self}", **opts)
|
702
|
+
Stat.clear("failed:#{self}", **opts)
|
703
703
|
end
|
704
704
|
rescue Exception => exception_while_unregistering
|
705
705
|
message = exception_while_unregistering.message
|
@@ -726,8 +726,8 @@ module Resque
|
|
726
726
|
# Called when we are done working - clears our `working_on` state
|
727
727
|
# and tells Redis we processed a job.
|
728
728
|
def done_working
|
729
|
-
data_store.worker_done_working(self) do
|
730
|
-
processed!
|
729
|
+
data_store.worker_done_working(self) do |**opts|
|
730
|
+
processed!(**opts)
|
731
731
|
end
|
732
732
|
end
|
733
733
|
|
@@ -745,9 +745,9 @@ module Resque
|
|
745
745
|
end
|
746
746
|
|
747
747
|
# Tell Redis we've processed a job.
|
748
|
-
def processed!
|
749
|
-
Stat
|
750
|
-
Stat
|
748
|
+
def processed!(**opts)
|
749
|
+
Stat.incr("processed", 1, **opts)
|
750
|
+
Stat.incr("processed:#{self}", 1, **opts)
|
751
751
|
end
|
752
752
|
|
753
753
|
# How many failed jobs has this worker seen? Returns an int.
|
data/lib/resque.rb
CHANGED
@@ -584,9 +584,9 @@ module Resque
|
|
584
584
|
def queue_sizes
|
585
585
|
queue_names = queues
|
586
586
|
|
587
|
-
sizes = redis.pipelined do
|
587
|
+
sizes = redis.pipelined do |piped|
|
588
588
|
queue_names.each do |name|
|
589
|
-
|
589
|
+
piped.llen("queue:#{name}")
|
590
590
|
end
|
591
591
|
end
|
592
592
|
|
@@ -597,11 +597,11 @@ module Resque
|
|
597
597
|
def sample_queues(sample_size = 1000)
|
598
598
|
queue_names = queues
|
599
599
|
|
600
|
-
samples = redis.pipelined do
|
600
|
+
samples = redis.pipelined do |piped|
|
601
601
|
queue_names.each do |name|
|
602
602
|
key = "queue:#{name}"
|
603
|
-
|
604
|
-
|
603
|
+
piped.llen(key)
|
604
|
+
piped.lrange(key, 0, sample_size - 1)
|
605
605
|
end
|
606
606
|
end
|
607
607
|
|
metadata
CHANGED
@@ -1,17 +1,17 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: resque
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.
|
4
|
+
version: 2.3.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Chris Wanstrath
|
8
8
|
- Steve Klabnik
|
9
9
|
- Terence Lee
|
10
10
|
- Michael Bianco
|
11
|
-
autorequire:
|
11
|
+
autorequire:
|
12
12
|
bindir: bin
|
13
13
|
cert_chain: []
|
14
|
-
date: 2022-
|
14
|
+
date: 2022-08-20 00:00:00.000000000 Z
|
15
15
|
dependencies:
|
16
16
|
- !ruby/object:Gem::Dependency
|
17
17
|
name: redis-namespace
|
@@ -186,7 +186,8 @@ licenses:
|
|
186
186
|
- MIT
|
187
187
|
metadata:
|
188
188
|
changelog_uri: https://github.com/resque/resque/blob/master/HISTORY.md
|
189
|
-
|
189
|
+
rubygems_mfa_required: 'true'
|
190
|
+
post_install_message:
|
190
191
|
rdoc_options:
|
191
192
|
- "--charset=UTF-8"
|
192
193
|
require_paths:
|
@@ -202,8 +203,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
202
203
|
- !ruby/object:Gem::Version
|
203
204
|
version: '0'
|
204
205
|
requirements: []
|
205
|
-
rubygems_version: 3.
|
206
|
-
signing_key:
|
206
|
+
rubygems_version: 3.3.5
|
207
|
+
signing_key:
|
207
208
|
specification_version: 4
|
208
209
|
summary: Resque is a Redis-backed queueing system.
|
209
210
|
test_files: []
|