unique_delayed_job 0.0.1 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README +47 -11
- data/lib/delayed/unique_delayed_job.rb +157 -0
- data/lib/unique_delayed_job.rb +1 -116
- data/unique_delayed_job.gemspec +5 -4
- metadata +4 -3
data/README
CHANGED
@@ -12,19 +12,55 @@ duplicate key is raised on insert, then the insert will just be ignored. There
|
|
12
12
|
are factory methods for creating a delayed job in the following ways:
|
13
13
|
* with a delayed job handler class (one that responds to perform())
|
14
14
|
* with an object, method and method arguments
|
15
|
-
* with a
|
15
|
+
* with a string to be evaled see the Delayed::Job.enqueue() method that accepts
|
16
|
+
a block. (looks like the idea may have been added to delayed job here:
|
17
|
+
http://github.com/dhh/delayed_job/commit/89c3a0b77470c0f8510d3c7a51a36cd07747d9a9)
|
18
|
+
the block is yielded to, and the result (assumed to be a string) is stored as
|
19
|
+
the code to be evaled when the job is deserialized.
|
20
|
+
|
21
|
+
== Configuration
|
22
|
+
|
23
|
+
You can add whatever columns you need to the delayed_jobs table, and put any
|
24
|
+
uniqueness constraints (or not) on them as you wish (unique columns, unique
|
25
|
+
composite keys etc.).
|
26
|
+
|
27
|
+
Note that by default, UniqueDelayedJob will modify any jobs that are currently locked
|
28
|
+
(locked_by is not null)...it will set all of the additional columns specified
|
29
|
+
on the job to NULL. (This allows for inserting a new job if a matching job is
|
30
|
+
already in process--and thus might be using out-of-date data.) If you do NOT
|
31
|
+
want this behavior, and do not want to insert a matching job if another job is
|
32
|
+
currently locked/being run, then make the following call. If you like, you can
|
33
|
+
change this before/after every job you insert if you want different jobs
|
34
|
+
to use different policies.
|
35
|
+
|
36
|
+
UniqueDelayedJob.do_not_mark_locked_jobs_with_null()
|
16
37
|
|
17
38
|
== Examples
|
18
39
|
|
19
|
-
|
20
|
-
|
21
|
-
|
40
|
+
In each example below, the :user_id => 123 is just to illustrate that you can add
|
41
|
+
arbitrary additional column(s) to the delayed job row being inserted. Presumably,
|
42
|
+
the user_id column has uniqueness constraints on it...
|
43
|
+
|
44
|
+
(More than one column could be specified; uniqueness is enforced by the db schema,
|
45
|
+
and unique delayed job just catches and swallows any duplicate key exceptions.)
|
46
|
+
|
47
|
+
# use a custom handler
|
48
|
+
job = Delayed::UniqueDelayedJob.use_handler(MyHandlerClass.new( ...), :user_id => 123)
|
49
|
+
job.enqueue # use default priority and run_at
|
22
50
|
|
23
|
-
# use a method call (similar to using send_later on the object)
|
24
|
-
record = MyActiveRecord.find(1)
|
25
|
-
job = Delayed::UniqueDelayedJob.call_method(record,
|
26
|
-
|
51
|
+
# use a method call (similar to using send_later on the object)
|
52
|
+
record = MyActiveRecord.find(1)
|
53
|
+
job = Delayed::UniqueDelayedJob.call_method(record,
|
54
|
+
:a_method,
|
55
|
+
[arg1, arg2],
|
56
|
+
:user_id => 123)
|
57
|
+
job.enqueue(1) # use priority of 1
|
27
58
|
|
28
|
-
#
|
29
|
-
|
30
|
-
|
59
|
+
# specify a block of code to execute with string (inside a block)
|
60
|
+
# note that the block could be complex logic that results in a string that is
|
61
|
+
# arbitrary ruby code to be evaled
|
62
|
+
# see Delayed::Job::enqueue(args, &block)
|
63
|
+
job = Delayed::UniqueDelayedJob.run_eval(:user_id => 123) do
|
64
|
+
'SomeClass.perform_some_slow_action'
|
65
|
+
end
|
66
|
+
job.enqueue(2, 1.hour.from_now) # run with priority 2 and run at 1 hour from now
|
@@ -0,0 +1,157 @@
|
|
1
|
+
module Delayed
|
2
|
+
|
3
|
+
# allows for specifying additional columns on the delayed_jobs table to help
|
4
|
+
# prevent duplicate delayed jobs from being entered into the queue...but still
|
5
|
+
# keep an easy interface to enqueuing delayed jobs
|
6
|
+
#-----------------------------------------------------------------------------
|
7
|
+
class UniqueDelayedJob
|
8
|
+
|
9
|
+
cattr_reader :mark_all_locked_jobs_with_null
|
10
|
+
|
11
|
+
@@mark_all_locked_jobs_with_null = true
|
12
|
+
|
13
|
+
# enable the polciy that all delayed job rows that are marked with non-null
|
14
|
+
# locked_by should have their columns set to null...ensuring that if a
|
15
|
+
# job is currently executing then we CAN insert another delayed job with
|
16
|
+
# the same unique keys.
|
17
|
+
# This is useful if you want to run a job every time some event occurs, but
|
18
|
+
# you're using this gem just to prevent having the same job twice in the
|
19
|
+
# queue. But if a job is currently running, it may miss the change in state
|
20
|
+
# due to the latest event, so a new job does need to be added to the queue.
|
21
|
+
# setting the columns to null on any jobs currently locked should
|
22
|
+
# ensure this behavior.
|
23
|
+
# This is the default behavior.
|
24
|
+
#-----------------------------------------------------------------------------
|
25
|
+
def self.mark_all_locked_jobs_with_null
|
26
|
+
@mark_all_locked_jobs_with_null = true
|
27
|
+
end
|
28
|
+
|
29
|
+
# override the default behavior, and DO NOT mark the columns on a delayed
|
30
|
+
# job that is currently locked to null. this prevents inserting a duplicate
|
31
|
+
# job with a job that is currently running (e.g. if you only want to run
|
32
|
+
# the job once). Default is mark_all_locked_jobs_with_null (see that method)
|
33
|
+
#-----------------------------------------------------------------------------
|
34
|
+
def self.do_not_mark_locked_jobs_with_null
|
35
|
+
@mark_all_locked_jobs_with_null = false
|
36
|
+
end
|
37
|
+
|
38
|
+
|
39
|
+
# factory method to create a new UniqueDelayedJob from a delayed job handler
|
40
|
+
# object (see delayed job documentation for requirements)
|
41
|
+
#
|
42
|
+
# arguments:
|
43
|
+
# - handler: the delayed jobs handler object you're using
|
44
|
+
# - columns: hash of column names and values to insert into the delayed
|
45
|
+
# jobs table in addition to the handler
|
46
|
+
#---------------------------------------------------------------------------
|
47
|
+
def self.use_handler(handler, columns = {})
|
48
|
+
job = self.new(handler, columns)
|
49
|
+
end
|
50
|
+
|
51
|
+
|
52
|
+
# factory method to create a new UniqueDelayedJob by specifying a method ton
|
53
|
+
# call. will use delayed jobs' PerformableMethod class to enqueue the job
|
54
|
+
#
|
55
|
+
# arguments:
|
56
|
+
# - object: the object (or class or module) on which to call the method
|
57
|
+
# - method: the method to call (specify a symbol or string)
|
58
|
+
# - args_arr: an array of arguments to pass in the method call)
|
59
|
+
# - columns: hash of column names and values to insert into the delayed
|
60
|
+
# jobs table in addition to the handler
|
61
|
+
#---------------------------------------------------------------------------
|
62
|
+
def self.call_method(object, method, args_arr, columns = {})
|
63
|
+
job = self.new(Delayed::PerformableMethod.new(object, method, args_arr),
|
64
|
+
columns)
|
65
|
+
end
|
66
|
+
|
67
|
+
|
68
|
+
# factory method to create a UniqueDelayedJob by specifying a block that
|
69
|
+
# is executed and whose result is stored as a string to be evaled by
|
70
|
+
# delayed_job
|
71
|
+
# arguments:
|
72
|
+
# - columns: hash of column names and values to insert into the delayed
|
73
|
+
# jobs table in addition to the handler
|
74
|
+
# NOTE: a block is expected
|
75
|
+
#---------------------------------------------------------------------------
|
76
|
+
def self.run_eval(columns = {}, &block)
|
77
|
+
raise "missing a block in call to run_block" if !block_given?
|
78
|
+
job = self.new(Delayed::EvaledJob.new(&block), columns)
|
79
|
+
end
|
80
|
+
|
81
|
+
|
82
|
+
# specify some additional columns to set in the delayed jobs table for this
|
83
|
+
# row. be sure that you've migrated to add these columns to the delayed jobs
|
84
|
+
# table. it is up to you to specify uniqueness constraints on any of the
|
85
|
+
# columns you'd like to use to prevent duplicate entries in the delayed jobs
|
86
|
+
# table. (it's also fine for some of these columns to not have unique
|
87
|
+
# constraints, though this class will not prevent duplicate values for those
|
88
|
+
# and they'll be for your use for other purposes.)
|
89
|
+
#---------------------------------------------------------------------------
|
90
|
+
def add_delayed_jobs_columns(new_columns)
|
91
|
+
columns.merge! new_columns
|
92
|
+
end
|
93
|
+
|
94
|
+
|
95
|
+
# put the job on the delayed jobs queue. if there already is a row in the
|
96
|
+
# delayed jobs table with the same value in any of the unique columns
|
97
|
+
# (enforced in the db), then the row will not be inserted
|
98
|
+
#
|
99
|
+
# return value:
|
100
|
+
# - if a new delayed job is inserted, the job object is returned
|
101
|
+
# - otherwise, returns nil
|
102
|
+
#
|
103
|
+
# arguments:
|
104
|
+
# priority:
|
105
|
+
# run_at:
|
106
|
+
#---------------------------------------------------------------------------
|
107
|
+
def enqueue(priority = nil, run_at = nil)
|
108
|
+
if mark_all_locked_jobs_with_null && !columns.blank?
|
109
|
+
null_setting_strings = []
|
110
|
+
|
111
|
+
columns.each_key do |c|
|
112
|
+
null_setting_strings << "#{c.to_s} = NULL"
|
113
|
+
end
|
114
|
+
|
115
|
+
Delayed::Job.update_all(null_setting_strings.join("\n ,"),
|
116
|
+
"locked_by IS NOT NULL")
|
117
|
+
end
|
118
|
+
|
119
|
+
cols_to_insert = columns
|
120
|
+
cols_to_insert.merge! :priority => priority if priority
|
121
|
+
cols_to_insert.merge! :run_at => run_at if run_at
|
122
|
+
cols_to_insert.merge! :handler => handler
|
123
|
+
|
124
|
+
job = nil
|
125
|
+
|
126
|
+
# try to catch if this raises an exception because of a duplicate key
|
127
|
+
# error this should work for mysql and postgresql which both have the
|
128
|
+
# word 'duplicate' followed (not necessarily immediately) by 'key'.
|
129
|
+
# ignoring case cause case differs between the two cases
|
130
|
+
# if doesn't look like a dupe key error, then reraise the exception
|
131
|
+
begin
|
132
|
+
job = Delayed::Job.create(cols_to_insert)
|
133
|
+
rescue => e
|
134
|
+
if /(duplicate).*(key)/i !~ e.message
|
135
|
+
raise e
|
136
|
+
end
|
137
|
+
end
|
138
|
+
|
139
|
+
return job
|
140
|
+
end
|
141
|
+
|
142
|
+
attr_accessor :columns
|
143
|
+
|
144
|
+
protected
|
145
|
+
|
146
|
+
attr_accessor :handler
|
147
|
+
|
148
|
+
# constructor used by the factory methods
|
149
|
+
#---------------------------------------------------------------------------
|
150
|
+
def initialize(handler, columns = {})
|
151
|
+
@handler = handler
|
152
|
+
@columns = columns
|
153
|
+
end
|
154
|
+
|
155
|
+
end
|
156
|
+
|
157
|
+
end
|
data/lib/unique_delayed_job.rb
CHANGED
@@ -1,116 +1 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
# allows for specifying additional columns on the delayed_jobs table to help
|
4
|
-
# prevent duplicate delayed jobs from being entered into the queue...but still
|
5
|
-
# keep an easy interface to enqueuing delayed jobs
|
6
|
-
#-----------------------------------------------------------------------------
|
7
|
-
class UniqueDelayedJob
|
8
|
-
|
9
|
-
# factory method to create a new UniqueDelayedJob from a delayed job handler
|
10
|
-
# object (see delayed job documentation for requirements)
|
11
|
-
#
|
12
|
-
# arguments:
|
13
|
-
# - handler: the delayed jobs handler object you're using
|
14
|
-
# - columns: hash of column names and values to insert into the delayed
|
15
|
-
# jobs table in addition to the handler
|
16
|
-
#---------------------------------------------------------------------------
|
17
|
-
def self.use_handler(handler, columns = {})
|
18
|
-
job = self.new
|
19
|
-
job.handler = handler
|
20
|
-
|
21
|
-
job
|
22
|
-
end
|
23
|
-
|
24
|
-
# factory method to create a new UniqueDelayedJob by specifying a method ton
|
25
|
-
# call. will use delayed jobs' PerformableMethod class to enqueue the job
|
26
|
-
#
|
27
|
-
# arguments:
|
28
|
-
# - object: the object (or class or module) on which to call the method
|
29
|
-
# - method: the method to call (specify a symbol or string)
|
30
|
-
# - args_arr: an array of arguments to pass in the method call)
|
31
|
-
# - columns: hash of column names and values to insert into the delayed
|
32
|
-
# jobs table in addition to the handler
|
33
|
-
#---------------------------------------------------------------------------
|
34
|
-
def self.call_method(object, method, args_arr, columns = {})
|
35
|
-
job = self.new
|
36
|
-
job.handler = Delayed::PerformableMethod.new(object, method, args_arr)
|
37
|
-
|
38
|
-
job
|
39
|
-
end
|
40
|
-
|
41
|
-
|
42
|
-
# factory method to create a UniqueDelayedJob by specifying a block to
|
43
|
-
# execute asynchronously
|
44
|
-
# arguments:
|
45
|
-
# - columns: hash of column names and values to insert into the delayed
|
46
|
-
# jobs table in addition to the handler
|
47
|
-
# NOTE: a block is expected
|
48
|
-
#---------------------------------------------------------------------------
|
49
|
-
def self.run_block(columns = {}, &block)
|
50
|
-
raise "missing a block in call to run_block" if !block_given?
|
51
|
-
job = self.new
|
52
|
-
job.handler = Delayed::EvaledJob.new(&block)
|
53
|
-
|
54
|
-
job
|
55
|
-
end
|
56
|
-
|
57
|
-
|
58
|
-
# specify some additional columns to set in the delayed jobs table for this
|
59
|
-
# row. be sure that you've migrated to add these columns to the delayed jobs
|
60
|
-
# table. it is up to you to specify uniqueness constraints on any of the
|
61
|
-
# columns you'd like to use to prevent duplicate entries in the delayed jobs
|
62
|
-
# table. (it's also fine for some of these columns to not have unique
|
63
|
-
# constraints, though this class will not prevent duplicate values for those
|
64
|
-
# and they'll be for your use for other purposes.)
|
65
|
-
#---------------------------------------------------------------------------
|
66
|
-
def add_delayed_jobs_columns(new_columns)
|
67
|
-
columns.merge! new_columns
|
68
|
-
end
|
69
|
-
|
70
|
-
# put the job on the delayed jobs queue. if there already is a row in the
|
71
|
-
# delayed jobs table with the same value in any of the unique columns
|
72
|
-
# (enforced in the db), then the row will not be inserted
|
73
|
-
#
|
74
|
-
# arguments:
|
75
|
-
# priority:
|
76
|
-
# run_at:
|
77
|
-
#---------------------------------------------------------------------------
|
78
|
-
def enqueue(priority = nil, run_at = nil)
|
79
|
-
cols_to_insert = columns
|
80
|
-
cols_to_insert.merge! :priority => priority if priority
|
81
|
-
cols_to_insert.merge! :run_at => run_at if run_at
|
82
|
-
cols_to_insert.merge! :handler => handler
|
83
|
-
|
84
|
-
# try to catch if this raises an exception because of a duplicate key
|
85
|
-
# error this should work for mysql and postgresql which both have the
|
86
|
-
# word 'duplicate' followed (not necessarily immediately) by 'key'.
|
87
|
-
# ignoring case cause case differs between the two cases
|
88
|
-
# if doesn't look like a dupe key error, then reraise the exception
|
89
|
-
begin
|
90
|
-
Delayed::Job.create(cols_to_insert)
|
91
|
-
rescue => e
|
92
|
-
if /(duplicate).*(key)/i !~ e.message
|
93
|
-
raise e
|
94
|
-
end
|
95
|
-
end
|
96
|
-
|
97
|
-
end
|
98
|
-
|
99
|
-
attr_accessor :columns
|
100
|
-
|
101
|
-
protected
|
102
|
-
|
103
|
-
attr_accessor :handler
|
104
|
-
|
105
|
-
|
106
|
-
# constructor used by the factory methods
|
107
|
-
#---------------------------------------------------------------------------
|
108
|
-
def initialize(handler, columns = {})
|
109
|
-
@handler = nil
|
110
|
-
@call_method = { :method => nil, args => nil }
|
111
|
-
@columns = {}
|
112
|
-
end
|
113
|
-
|
114
|
-
end
|
115
|
-
|
116
|
-
end
|
1
|
+
require 'delayed/unique_delayed_job'
|
data/unique_delayed_job.gemspec
CHANGED
@@ -5,11 +5,11 @@
|
|
5
5
|
|
6
6
|
Gem::Specification.new do |s|
|
7
7
|
s.name = %q{unique_delayed_job}
|
8
|
-
s.version = "0.0
|
8
|
+
s.version = "0.1.0"
|
9
9
|
|
10
10
|
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
11
11
|
s.authors = ["Brian Percival"]
|
12
|
-
s.date = %q{2009-11-
|
12
|
+
s.date = %q{2009-11-19}
|
13
13
|
s.description = %q{Class for creating delayed jobs that can be de-duped with existing delayed jobs
|
14
14
|
already in the delayed jobs table. You just specify some additional columns on
|
15
15
|
your delayed_jobs table and set them to have uniqueness constraints. Then
|
@@ -18,7 +18,7 @@ duplicate key is raised on insert, then the insert will just be ignored. There
|
|
18
18
|
are factory methods for creating a delayed job in the following ways:
|
19
19
|
* with a delayed job handler class (one that responds to perform())
|
20
20
|
* with an object, method and method arguments
|
21
|
-
* with a code
|
21
|
+
* with a code string to be evaled
|
22
22
|
|
23
23
|
NOTE: you must have delayed_job installed as a gem or plugin
|
24
24
|
}
|
@@ -27,7 +27,8 @@ NOTE: you must have delayed_job installed as a gem or plugin
|
|
27
27
|
"README"
|
28
28
|
]
|
29
29
|
s.files = [
|
30
|
-
"lib/unique_delayed_job.rb",
|
30
|
+
"lib/delayed/unique_delayed_job.rb",
|
31
|
+
"lib/unique_delayed_job.rb",
|
31
32
|
"unique_delayed_job.gemspec"
|
32
33
|
]
|
33
34
|
s.homepage = %q{http://github.com/bmpercy/unique_delayed_job}
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: unique_delayed_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0
|
4
|
+
version: 0.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Brian Percival
|
@@ -9,7 +9,7 @@ autorequire:
|
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
11
|
|
12
|
-
date: 2009-11-
|
12
|
+
date: 2009-11-19 00:00:00 -08:00
|
13
13
|
default_executable:
|
14
14
|
dependencies: []
|
15
15
|
|
@@ -22,7 +22,7 @@ description: |
|
|
22
22
|
are factory methods for creating a delayed job in the following ways:
|
23
23
|
* with a delayed job handler class (one that responds to perform())
|
24
24
|
* with an object, method and method arguments
|
25
|
-
* with a code
|
25
|
+
* with a code string to be evaled
|
26
26
|
|
27
27
|
NOTE: you must have delayed_job installed as a gem or plugin
|
28
28
|
|
@@ -34,6 +34,7 @@ extensions: []
|
|
34
34
|
extra_rdoc_files:
|
35
35
|
- README
|
36
36
|
files:
|
37
|
+
- lib/delayed/unique_delayed_job.rb
|
37
38
|
- lib/unique_delayed_job.rb
|
38
39
|
- unique_delayed_job.gemspec
|
39
40
|
- README
|