resque-stagger 0.2.1 → 1.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/.gitignore +2 -0
- data/CHANGELOG.md +26 -0
- data/README.md +31 -21
- data/bin/console +1 -1
- data/lib/resque/{stagger → staggered}/version.rb +1 -1
- data/lib/resque/staggered.rb +80 -0
- data/lib/resque-staggered.rb +2 -0
- data/resque-stagger.gemspec +3 -2
- metadata +22 -12
- data/.idea/.gitignore +0 -8
- data/.idea/misc.xml +0 -4
- data/.idea/modules.xml +0 -8
- data/.idea/resque-stagger.iml +0 -16
- data/.idea/vcs.xml +0 -6
- data/lib/resque/stagger.rb +0 -58
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
|
-
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 159e7ada89f6f00107b1445850c12eedfa73ad6dd5e45cbc806e286467b5a964
|
|
4
|
+
data.tar.gz: 8f0bb3379c3f431729907b1c652b62fbb6ed55319bef75569b683d7b87d55e27
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 50e12311484c9454639c035ef8ac8165ace0ccec1841454ea47305245083563678361ffd8e94c25ef5cca1f31f46c2d39328b0c2cc2db8b5946d27670eb3de07
|
|
7
|
+
data.tar.gz: ae8b9148e21084f60462a8e668900d205b58dd02c9624e16025c9805e62560b84cd72c6a5ea17db01e51f66bd11c627bdc9daa027a84e4d1fe849c49695cf225
|
data/.gitignore
CHANGED
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
## 0.2.1
|
|
2
|
+
|
|
3
|
+
### Added
|
|
4
|
+
|
|
5
|
+
* Staggering functionality
|
|
6
|
+
|
|
7
|
+
## 0.3.0
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
* Queue name can be given on runtime
|
|
12
|
+
|
|
13
|
+
## 1.0.2
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
|
|
17
|
+
* Improper way of prepending module
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
* Functionality to enqueue single job in a non-unit time range.
|
|
22
|
+
For example, earlier "n" jobs could have been scheduled per unit time,
|
|
23
|
+
i.e. in 1 second, but now it can be done both ways i.e. 1 job can be
|
|
24
|
+
scheduled to be enqueued per 5 seconds as well as 5 jobs can be
|
|
25
|
+
scheduled to be enqueued in 1 second.
|
|
26
|
+
|
data/README.md
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
|
-
# Resque
|
|
1
|
+
# Resque-Stagger
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
resque-stagger is a simple Ruby gem that allows you to stagger the enqueuing of jobs
|
|
4
|
+
in [Resque](https://github.com/resque/resque).
|
|
4
5
|
|
|
5
6
|
## Installation
|
|
6
7
|
|
|
@@ -12,39 +13,48 @@ gem 'resque-stagger'
|
|
|
12
13
|
|
|
13
14
|
And then execute:
|
|
14
15
|
|
|
15
|
-
|
|
16
|
+
```shell
|
|
17
|
+
bundle install
|
|
18
|
+
```
|
|
16
19
|
|
|
17
20
|
Or install it yourself as:
|
|
18
21
|
|
|
19
|
-
|
|
22
|
+
```shell
|
|
23
|
+
gem install resque-stagger
|
|
24
|
+
```
|
|
20
25
|
|
|
21
26
|
## Usage
|
|
22
27
|
|
|
23
|
-
You can use `Resque::Staggered` to enqueue jobs with a stagger effect.
|
|
24
|
-
|
|
25
28
|
```ruby
|
|
26
|
-
require
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
29
|
+
require 'resque/staggered'
|
|
30
|
+
|
|
31
|
+
# Set up a new staggered queue
|
|
32
|
+
queue = Resque::Staggered.new(
|
|
33
|
+
start_from: Time.now + 60, # Start queuing jobs 60 seconds from now
|
|
34
|
+
number_of_jobs: 10, # Enqueue 10 jobs per time interval
|
|
35
|
+
unit_time_in_seconds: 300, # Interval of 5 minutes between each set of jobs
|
|
36
|
+
queue: 'low' # Enqueue jobs in the 'low' queue
|
|
37
|
+
)
|
|
38
|
+
|
|
39
|
+
# Enqueue a job with arguments
|
|
40
|
+
queue.enqueue(MyJobClass, arg1, arg2, arg3)
|
|
31
41
|
```
|
|
32
42
|
|
|
33
|
-
The
|
|
43
|
+
- The code sets up a new staggered queue using Resque::Staggered and then enqueues a job with arguments using the
|
|
44
|
+
enqueue method of the staggered queue.
|
|
34
45
|
|
|
35
|
-
|
|
46
|
+
- Based on the options passed to the Resque::Staggered constructor, the staggered queue will enqueue 10 jobs every 5
|
|
47
|
+
minutes (300 seconds) starting 60 seconds from the current time, and enqueue the jobs in the "low" queue.
|
|
36
48
|
|
|
37
|
-
|
|
49
|
+
- So, if this code is run at 12:00 PM, the first set of 10 jobs will be enqueued at 12:01 PM (60 seconds later), the
|
|
50
|
+
second set of 10 jobs will be enqueued at 12:06 PM (5 minutes later), the third set of 10 jobs will be enqueued at 12:
|
|
51
|
+
11 PM (5 minutes later), and so on.
|
|
38
52
|
|
|
39
53
|
## Contributing
|
|
40
54
|
|
|
41
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/
|
|
55
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/parikshit223933/resque-stagger.
|
|
42
56
|
|
|
43
57
|
## License
|
|
44
58
|
|
|
45
|
-
The gem is available as open source under the terms of
|
|
46
|
-
|
|
47
|
-
## Code of Conduct
|
|
48
|
-
|
|
49
|
-
Everyone interacting in the Resque Stagger project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/resque-stagger/blob/master/CODE_OF_CONDUCT.md).
|
|
50
|
-
|
|
59
|
+
The gem is available as open source under the terms of
|
|
60
|
+
the [MIT License](https://github.com/parikshit223933/resque-stagger/blob/master/LICENSE.txt).
|
data/bin/console
CHANGED
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
require "resque/staggered/version"
|
|
2
|
+
|
|
3
|
+
module Resque
|
|
4
|
+
class Staggered
|
|
5
|
+
# Initializes a new Staggered object.
|
|
6
|
+
#
|
|
7
|
+
# @param options [Hash] The options to create the Staggered object with.
|
|
8
|
+
# @option options [Time] :start_from The time to start enqueueing jobs from (defaults to Time.current).
|
|
9
|
+
# @option options [Integer] :number_of_jobs The number of jobs to enqueue per unit time.
|
|
10
|
+
# @option options [Integer] :unit_time_in_seconds The time interval between each set of jobs.
|
|
11
|
+
# @option options [String] :queue The name of the queue to enqueue the jobs in (optional).
|
|
12
|
+
#
|
|
13
|
+
# @return [Staggered] A new instance of the Staggered class.
|
|
14
|
+
def initialize(**options)
|
|
15
|
+
@options = options
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
# Enqueues a job with the given arguments at a staggered time.
|
|
19
|
+
#
|
|
20
|
+
# @param klass [Object] The job class to enqueue.
|
|
21
|
+
# @param args [Array] The arguments to pass to the job.
|
|
22
|
+
#
|
|
23
|
+
# @return [void]
|
|
24
|
+
def enqueue(klass, *args)
|
|
25
|
+
if queue.present?
|
|
26
|
+
::Resque.enqueue_at_with_queue(queue, current_enqueue_at, klass, *args)
|
|
27
|
+
else
|
|
28
|
+
::Resque.enqueue_at(current_enqueue_at, klass, *args)
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
private
|
|
33
|
+
|
|
34
|
+
# Calculates the staggered time for the next job and returns it.
|
|
35
|
+
#
|
|
36
|
+
# @return [Time] The staggered time for the next job.
|
|
37
|
+
def current_enqueue_at
|
|
38
|
+
if number_of_jobs > @enqueues_in_last_unit_time.to_i
|
|
39
|
+
@enqueues_in_last_unit_time = @enqueues_in_last_unit_time.to_i + 1
|
|
40
|
+
if @unit_time_timestamp.nil?
|
|
41
|
+
return @unit_time_timestamp = starting_from
|
|
42
|
+
else
|
|
43
|
+
return @unit_time_timestamp
|
|
44
|
+
end
|
|
45
|
+
else
|
|
46
|
+
@enqueues_in_last_unit_time = 0
|
|
47
|
+
@unit_time_timestamp = @unit_time_timestamp + unit_time_in_seconds.second
|
|
48
|
+
current_enqueue_at
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# Returns the time to start from.
|
|
53
|
+
#
|
|
54
|
+
# @return [Time] The time to start from.
|
|
55
|
+
def starting_from
|
|
56
|
+
@starting_from ||= @options[:start_from] || Time.current
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
# Returns the number of jobs to enqueue.
|
|
60
|
+
#
|
|
61
|
+
# @return [Integer] The number of jobs to enqueue.
|
|
62
|
+
def number_of_jobs
|
|
63
|
+
@options[:number_of_jobs] || nil
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
# Returns the time interval between each set of jobs.
|
|
67
|
+
#
|
|
68
|
+
# @return [Integer] The time interval between each set of jobs.
|
|
69
|
+
def unit_time_in_seconds
|
|
70
|
+
@options[:unit_time_in_seconds] || nil
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
# Returns the name of the queue to enqueue the jobs in.
|
|
74
|
+
#
|
|
75
|
+
# @return [String] The name of the queue to enqueue the jobs in.
|
|
76
|
+
def queue
|
|
77
|
+
@options[:queue] || nil
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
end
|
data/resque-stagger.gemspec
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
lib = File.expand_path("lib", __dir__)
|
|
2
2
|
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
|
3
|
-
require "resque/
|
|
3
|
+
require "resque/staggered/version"
|
|
4
4
|
|
|
5
5
|
Gem::Specification.new do |spec|
|
|
6
6
|
spec.name = "resque-stagger"
|
|
@@ -17,7 +17,7 @@ Gem::Specification.new do |spec|
|
|
|
17
17
|
|
|
18
18
|
spec.metadata["homepage_uri"] = spec.homepage
|
|
19
19
|
spec.metadata["source_code_uri"] = "https://github.com/parikshit223933/resque-stagger"
|
|
20
|
-
spec.metadata["changelog_uri"] = "
|
|
20
|
+
spec.metadata["changelog_uri"] = "https://github.com/parikshit223933/resque-stagger/blob/master/CHANGELOG.md"
|
|
21
21
|
|
|
22
22
|
# Specify which files should be added to the gem when it is released.
|
|
23
23
|
# The `git ls-files -z` loads the files in the RubyGem that have been added into git.
|
|
@@ -33,5 +33,6 @@ Gem::Specification.new do |spec|
|
|
|
33
33
|
spec.add_development_dependency "rspec", "~> 3.0"
|
|
34
34
|
|
|
35
35
|
spec.add_runtime_dependency 'resque', '>= 1.25'
|
|
36
|
+
spec.add_runtime_dependency 'activesupport', '>= 5.1.7'
|
|
36
37
|
spec.add_runtime_dependency 'resque-scheduler', '>= 4.4.0'
|
|
37
38
|
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: resque-stagger
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 1.0.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Parikshit Singh
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2023-
|
|
11
|
+
date: 2023-03-03 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: bundler
|
|
@@ -66,6 +66,20 @@ dependencies:
|
|
|
66
66
|
- - ">="
|
|
67
67
|
- !ruby/object:Gem::Version
|
|
68
68
|
version: '1.25'
|
|
69
|
+
- !ruby/object:Gem::Dependency
|
|
70
|
+
name: activesupport
|
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
|
72
|
+
requirements:
|
|
73
|
+
- - ">="
|
|
74
|
+
- !ruby/object:Gem::Version
|
|
75
|
+
version: 5.1.7
|
|
76
|
+
type: :runtime
|
|
77
|
+
prerelease: false
|
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
79
|
+
requirements:
|
|
80
|
+
- - ">="
|
|
81
|
+
- !ruby/object:Gem::Version
|
|
82
|
+
version: 5.1.7
|
|
69
83
|
- !ruby/object:Gem::Dependency
|
|
70
84
|
name: resque-scheduler
|
|
71
85
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -88,13 +102,9 @@ extensions: []
|
|
|
88
102
|
extra_rdoc_files: []
|
|
89
103
|
files:
|
|
90
104
|
- ".gitignore"
|
|
91
|
-
- ".idea/.gitignore"
|
|
92
|
-
- ".idea/misc.xml"
|
|
93
|
-
- ".idea/modules.xml"
|
|
94
|
-
- ".idea/resque-stagger.iml"
|
|
95
|
-
- ".idea/vcs.xml"
|
|
96
105
|
- ".rspec"
|
|
97
106
|
- ".travis.yml"
|
|
107
|
+
- CHANGELOG.md
|
|
98
108
|
- CODE_OF_CONDUCT.md
|
|
99
109
|
- Gemfile
|
|
100
110
|
- LICENSE.txt
|
|
@@ -102,8 +112,9 @@ files:
|
|
|
102
112
|
- Rakefile
|
|
103
113
|
- bin/console
|
|
104
114
|
- bin/setup
|
|
105
|
-
- lib/resque
|
|
106
|
-
- lib/resque/
|
|
115
|
+
- lib/resque-staggered.rb
|
|
116
|
+
- lib/resque/staggered.rb
|
|
117
|
+
- lib/resque/staggered/version.rb
|
|
107
118
|
- resque-stagger.gemspec
|
|
108
119
|
homepage: https://github.com/parikshit223933/resque-stagger
|
|
109
120
|
licenses:
|
|
@@ -112,7 +123,7 @@ metadata:
|
|
|
112
123
|
allowed_push_host: https://rubygems.org
|
|
113
124
|
homepage_uri: https://github.com/parikshit223933/resque-stagger
|
|
114
125
|
source_code_uri: https://github.com/parikshit223933/resque-stagger
|
|
115
|
-
changelog_uri:
|
|
126
|
+
changelog_uri: https://github.com/parikshit223933/resque-stagger/blob/master/CHANGELOG.md
|
|
116
127
|
post_install_message:
|
|
117
128
|
rdoc_options: []
|
|
118
129
|
require_paths:
|
|
@@ -128,8 +139,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
128
139
|
- !ruby/object:Gem::Version
|
|
129
140
|
version: '0'
|
|
130
141
|
requirements: []
|
|
131
|
-
|
|
132
|
-
rubygems_version: 2.5.1
|
|
142
|
+
rubygems_version: 3.0.3.1
|
|
133
143
|
signing_key:
|
|
134
144
|
specification_version: 4
|
|
135
145
|
summary: This will stagger your resque jobs over a configurable duration rather then
|
data/.idea/.gitignore
DELETED
data/.idea/misc.xml
DELETED
data/.idea/modules.xml
DELETED
|
@@ -1,8 +0,0 @@
|
|
|
1
|
-
<?xml version="1.0" encoding="UTF-8"?>
|
|
2
|
-
<project version="4">
|
|
3
|
-
<component name="ProjectModuleManager">
|
|
4
|
-
<modules>
|
|
5
|
-
<module fileurl="file://$PROJECT_DIR$/.idea/resque-stagger.iml" filepath="$PROJECT_DIR$/.idea/resque-stagger.iml" />
|
|
6
|
-
</modules>
|
|
7
|
-
</component>
|
|
8
|
-
</project>
|
data/.idea/resque-stagger.iml
DELETED
|
@@ -1,16 +0,0 @@
|
|
|
1
|
-
<?xml version="1.0" encoding="UTF-8"?>
|
|
2
|
-
<module type="RUBY_MODULE" version="4">
|
|
3
|
-
<component name="ModuleRunConfigurationManager">
|
|
4
|
-
<shared />
|
|
5
|
-
</component>
|
|
6
|
-
<component name="NewModuleRootManager">
|
|
7
|
-
<content url="file://$MODULE_DIR$">
|
|
8
|
-
<sourceFolder url="file://$MODULE_DIR$/features" isTestSource="true" />
|
|
9
|
-
<sourceFolder url="file://$MODULE_DIR$/spec" isTestSource="true" />
|
|
10
|
-
<sourceFolder url="file://$MODULE_DIR$/test" isTestSource="true" />
|
|
11
|
-
</content>
|
|
12
|
-
<orderEntry type="inheritedJdk" />
|
|
13
|
-
<orderEntry type="sourceFolder" forTests="false" />
|
|
14
|
-
<orderEntry type="library" scope="PROVIDED" name="bundler (v2.2.32, ruby-2.6.3-p62) [gem]" level="application" />
|
|
15
|
-
</component>
|
|
16
|
-
</module>
|
data/.idea/vcs.xml
DELETED
data/lib/resque/stagger.rb
DELETED
|
@@ -1,58 +0,0 @@
|
|
|
1
|
-
require "resque/stagger/version"
|
|
2
|
-
|
|
3
|
-
# The Resque module contains the Stagger module.
|
|
4
|
-
module Resque
|
|
5
|
-
module Stagger
|
|
6
|
-
# The Staggered class is used to enqueue jobs with a stagger effect.
|
|
7
|
-
class Staggered
|
|
8
|
-
# Initializes the Staggered class with given options.
|
|
9
|
-
#
|
|
10
|
-
# @param options [Hash] the options for the stagger effect.
|
|
11
|
-
# @option options [Time] :start_from starting time for enqueuing jobs.
|
|
12
|
-
# @option options [Integer] :per_second number of jobs to enqueue per second.
|
|
13
|
-
def initialize(**options)
|
|
14
|
-
@options = options
|
|
15
|
-
end
|
|
16
|
-
|
|
17
|
-
# Enqueues the given job with a stagger effect.
|
|
18
|
-
#
|
|
19
|
-
# @param klass [Class] the job class to be enqueued.
|
|
20
|
-
# @param args [Array] the arguments for the job.
|
|
21
|
-
def enqueue(klass, *args)
|
|
22
|
-
::Resque.enqueue_at(current_enqueue_at, klass, *args)
|
|
23
|
-
end
|
|
24
|
-
|
|
25
|
-
private
|
|
26
|
-
|
|
27
|
-
# Calculates the time at which the job should be enqueued.
|
|
28
|
-
#
|
|
29
|
-
# @return [Time] the enqueue time for the job.
|
|
30
|
-
def current_enqueue_at
|
|
31
|
-
return starting_from if per_second.to_i.zero?
|
|
32
|
-
return @last_enqueued_at = starting_from if @last_enqueued_at.nil?
|
|
33
|
-
|
|
34
|
-
@enqueued_last_second = @enqueued_last_second.to_i + 1
|
|
35
|
-
return @last_enqueued_at if @enqueued_last_second < per_second.to_i
|
|
36
|
-
|
|
37
|
-
@enqueued_last_second = 0
|
|
38
|
-
@last_enqueued_at += 1.second
|
|
39
|
-
end
|
|
40
|
-
|
|
41
|
-
# Gets the starting time for enqueuing jobs.
|
|
42
|
-
#
|
|
43
|
-
# @return [Time] the starting time for enqueuing jobs.
|
|
44
|
-
def starting_from
|
|
45
|
-
@starting_from ||= @options[:start_from] || Time.current
|
|
46
|
-
end
|
|
47
|
-
|
|
48
|
-
# Gets the number of jobs to enqueue per second.
|
|
49
|
-
#
|
|
50
|
-
# @return [Integer, nil] the number of jobs to enqueue per second.
|
|
51
|
-
def per_second
|
|
52
|
-
@options[:per_second] || nil
|
|
53
|
-
end
|
|
54
|
-
end
|
|
55
|
-
end
|
|
56
|
-
end
|
|
57
|
-
|
|
58
|
-
Resque.prepend(Resque::Stagger)
|