resque-unique_by_arity 2.0.2 → 3.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 2832d3ab2ddbac6a32f3402a0a24dea7eb8ca6b253d0eff6ea6f8a924e343fb1
4
- data.tar.gz: a03a10a39c8fe8b1bfbec33299b4de7a72f7d8da3e8d1131c693e97fb61cc580
3
+ metadata.gz: bfe4371869efe4deb3aed04f46d2417ab32c82e4fadc7954d6d79a0e9cbbe8d2
4
+ data.tar.gz: 7b8392ab635f8acc02c6e0d9fa09fe79fd287099ede2f75fd654c43661ab161d
5
5
  SHA512:
6
- metadata.gz: bdd1c1ff4d5ba5aca5ba17f3e6ae290c349f7064084c0876054200a28951352b410163287db536d0930109d655c7a43c51b61271bdc972d4dc65436fc85bc772
7
- data.tar.gz: dc859b3415561ec698bb82381c713cef59f298c8ae591ff281eef103b0b545b5dec7e30de45c58c41d5e9415b2e289b046312ec8335b887ff1fb7baae274c027
6
+ metadata.gz: 8e93042522fa854d08126b163847f5fb670321da013f1e7c20a179311aaf7e0c621855d8998cb465e96df38057a7aaa45357cba36caea1ebc035972a65f17727
7
+ data.tar.gz: 8a9033f210f024ef2cf99805a97e87137b585dbc6ad82ba58c410b2b6dd7d6021cf2f33af84647a85f4aa9c182ef76830674afa81628c5bbd046625efcb3558e
data/.gitignore CHANGED
@@ -15,3 +15,5 @@ spec/reports
15
15
  test/tmp
16
16
  test/version_tmp
17
17
  tmp
18
+
19
+ .rspec_status
data/Gemfile CHANGED
@@ -2,8 +2,8 @@ source 'https://rubygems.org'
2
2
 
3
3
  git_source(:github) { |repo_name| "https://github.com/#{repo_name}" }
4
4
 
5
- gem 'resque-unique_in_queue'
6
- gem 'resque-unique_at_runtime'
5
+ gem 'resque-unique_in_queue', path: '/Users/pboling/Documents/code/intricately/resque-unique_in_queue'
6
+ gem 'resque-unique_at_runtime', path: '/Users/pboling/Documents/code/intricately/resque-unique_at_runtime'
7
7
 
8
8
  group :test do
9
9
  unless ENV['TRAVIS']
data/README.md CHANGED
@@ -1,11 +1,18 @@
1
1
  # Resque::UniqueByArity
2
2
 
3
- Because some jobs have parameters that you do not want to consider for determination of uniqueness.
3
+ Because some jobs have parameters that you do not want to consider for
4
+ determination of uniqueness.
4
5
 
5
6
  NOTE:
6
7
 
7
- Requires `resque_solo` gem, and `resque-unique_by_arity` gem; the latter is a fork of `resque-lonely_job`.
8
- Why? `resque-lonely_job` and `resque_solo` can't be used together, because their `redis_key` methods conflict.
8
+ I rewrote, and renamed, both `resque_solo` and `resque-lonely_job`, becuase they
9
+ can't be used together. Why? Their `redis_key` methods directly conflict,
10
+ among other more subtle issues.
11
+
12
+ This gem requires use of my rewritten gems for uniqueness enforcement:
13
+
14
+ - [`resque-unique_at_runtime`](https://github.com/pboling/resque-unique_at_runtime)
15
+ - [`resque-unique_in_queue`](https://github.com/pboling/resque-unique_in_queue)
9
16
 
10
17
  | Project | Resque::UniqueByArity |
11
18
  |------------------------ | ----------------------- |
@@ -24,11 +31,13 @@ Why? `resque-lonely_job` and `resque_solo` can't be used together, because their
24
31
 
25
32
  ## Important Note
26
33
 
27
- See `lib/resque/unique_by_arity/configuration.rb` for all config options. Only a smattering of what is available is documented in this README.
34
+ See `lib/resque/unique_by_arity/configuration.rb` for all config options. Only
35
+ a smattering of what is available is documented in this README.
28
36
 
29
37
  ## Most Important Note
30
38
 
31
- You must configure this gem *after* you define the perform class method in your job or an error will be raised thanks to `perform` not having been defined yet.
39
+ You must configure this gem *after* you define the perform class method in your
40
+ job or an error will be raised thanks to `perform` not having been defined yet.
32
41
 
33
42
  Example:
34
43
 
@@ -65,9 +74,44 @@ Or install it yourself as:
65
74
 
66
75
  ## Usage
67
76
 
77
+ This gem will take care to set the class instance variables (similar to the
78
+ familiar `@queue` class instance variable) that are utilized by
79
+ `resque-unique_in_queue` and `resque-unique_at_runtime` (default values shown):
80
+
81
+ ```ruby
82
+ # For resque-unique_at_runtime
83
+ @runtime_lock_timeout = 60 * 60 * 24 * 5
84
+ @runtime_requeue_interval = 1
85
+ @unique_at_runtime_key_base = 'r-uar'.freeze
86
+
87
+ # For resque-unique_in_queue
88
+ @lock_after_execution_period = 0
89
+ @ttl = -1
90
+ @unique_in_queue_key_base = 'r-uiq'.freeze
91
+ ```
92
+
93
+ All you need to do is configure this gem accordingly:
94
+ ```ruby
95
+ include Resque::Plugins::UniqueByArity.new(
96
+ arity_for_uniqueness: 3,
97
+ unique_at_runtime: true,
98
+ unique_in_queue: true,
99
+ # No need to do the following if keeping default values
100
+ runtime_lock_timeout: 60 * 60 * 24 * 5,
101
+ runtime_requeue_interval: 1,
102
+ unique_at_runtime_key_base: 'r-uar'.freeze,
103
+ lock_after_execution_period: 0,
104
+ ttl: 0,
105
+ unique_in_queue_key_base: 'r-uiq'.freeze
106
+ )
107
+ ```
108
+
68
109
  ### Arity For Uniqueness
69
110
 
70
- Some jobs have parameters that you do not want to consider for determination of uniqueness. Resque jobs should use simple parameters, **not named parameters**, so you can just specify the number of parameters, counting from the left, you want to be considered for uniqueness.
111
+ Some jobs have parameters that you do not want to consider for determination of
112
+ uniqueness. Resque jobs should use simple parameters, **not named parameters**,
113
+ so you can just specify the number of parameters, counting from the left, you
114
+ want to be considered for uniqueness.
71
115
 
72
116
  ```ruby
73
117
  class MyJob
@@ -105,7 +149,8 @@ end
105
149
 
106
150
  ### Lock After Execution
107
151
 
108
- Give the job a break after it finishes running, and don't allow another of the same, with matching args @ configured arity, to start within X seconds.
152
+ Give the job a break after it finishes running, and don't allow another of the
153
+ same, with matching args @ configured arity, to start within X seconds.
109
154
 
110
155
  ```ruby
111
156
  class MyJob
@@ -122,7 +167,8 @@ end
122
167
 
123
168
  ### Runtime Lock Timeout
124
169
 
125
- If runtime lock keys get stale, they will expire on their own after some period. You can set the expiration period on a per class basis.
170
+ If runtime lock keys get stale, they will expire on their own after some period.
171
+ You can set the expiration period on a per class basis.
126
172
 
127
173
  ```ruby
128
174
  class MyJob
@@ -263,7 +309,8 @@ end
263
309
 
264
310
  ### Debugging
265
311
 
266
- Run your worker with `RESQUE_DEBUG=true` to see payloads printed before they are used to determine uniqueness, as well as a lot of other debugging output.
312
+ Run your worker with `RESQUE_DEBUG=true` to see payloads printed before they are
313
+ used to determine uniqueness, as well as a lot of other debugging output.
267
314
 
268
315
  ### Customize Unique Keys Per Job
269
316
 
@@ -0,0 +1,17 @@
1
+ require 'resque/unique_by_arity/version'
2
+
3
+ # External Gems
4
+ require 'colorized_string'
5
+ require 'resque'
6
+
7
+ # External Resque Plugins
8
+ require 'resque-unique_in_queue'
9
+ require 'resque-unique_at_runtime'
10
+
11
+ require 'resque/unique_by_arity/configuration'
12
+ require 'resque/unique_by_arity/global_configuration'
13
+ require 'resque/unique_by_arity'
14
+ require 'resque/unique_by_arity/modulizer'
15
+ require 'resque/unique_by_arity/unique_job'
16
+ require 'resque/unique_by_arity/validation'
17
+ require 'resque/plugins/unique_by_arity'
@@ -35,17 +35,48 @@ module Resque
35
35
  # As a result we can configure per class.
36
36
  @configuration.base_klass_name = base.to_s
37
37
  @configuration.validate
38
- base.send(:extend, Resque::UniqueByArity)
38
+ base.send(:extend, Resque::UniqueByArity::UniqueJob)
39
39
  base.uniqueness_config_reset(@configuration.dup)
40
40
 
41
- # gem is unique_in_queue, which is a rewrite of resque-loner
42
- # see: https://github.com/neighborland/resque_solo
41
+ # gem is resque-unique_in_queue, which is a rewrite of resque-solo / resque-loner
42
+ # see: https://github.com/pboling/resque-unique_in_queue
43
43
  # defines a redis_key method, which we have to override.
44
44
  base.send(:include, Resque::Plugins::UniqueInQueue) if @configuration.unique_in_queue || @configuration.unique_across_queues
45
45
 
46
46
  # gem is resque-unique_at_runtime, which is a rewrite of resque-lonely_job
47
47
  # see: https://github.com/pboling/resque-unique_at_runtime
48
- base.send(:extend, Resque::Plugins::UniqueAtRuntime) if @configuration.unique_at_runtime
48
+ base.send(:include, Resque::Plugins::UniqueAtRuntime) if @configuration.unique_at_runtime
49
+
50
+ # For resque-unique_at_runtime
51
+ #
52
+ if @configuration.runtime_lock_timeout
53
+ base.instance_variable_set(:@runtime_lock_timeout, @configuration.runtime_lock_timeout)
54
+ end
55
+
56
+ if @configuration.runtime_requeue_interval
57
+ base.instance_variable_set(:@runtime_requeue_interval, @configuration.runtime_requeue_interval)
58
+ end
59
+
60
+ if @configuration.unique_at_runtime_key_base
61
+ base.instance_variable_set(:@unique_at_runtime_key_base, @configuration.unique_at_runtime_key_base)
62
+ end
63
+
64
+ # For resque-unique_in_queue
65
+ #
66
+ if @configuration.lock_after_execution_period
67
+ base.instance_variable_set(:@lock_after_execution_period, @configuration.lock_after_execution_period)
68
+ end
69
+
70
+ if @configuration.ttl
71
+ base.instance_variable_set(:@ttl, @configuration.ttl)
72
+ end
73
+
74
+ # Normally doesn't make sense to override per each class because
75
+ # it wouldn't be able to determine or enforce uniqueness across queues,
76
+ # and general cleanup of stray keys would be nearly impossible.
77
+ if @configuration.unique_in_queue_key_base
78
+ base.instance_variable_set(:@unique_in_queue_key_base, @configuration.unique_in_queue_key_base)
79
+ end
49
80
 
50
81
  uniqueness_cop_module = Resque::UniqueByArity::Modulizer.to_mod(@configuration)
51
82
  # This will override methods from both plugins above, if configured for both
@@ -1,132 +1,34 @@
1
- require 'resque/unique_by_arity/version'
1
+ # Just in case some code still does require 'resque/unique_by_arity'
2
+ require 'resque-unique_by_arity'
2
3
 
3
- # External Gems
4
- require 'colorized_string'
5
- require 'resque'
6
-
7
- # External Resque Plugins
8
- require 'resque-unique_in_queue'
9
- require 'resque-unique_at_runtime'
10
-
11
- require 'resque/plugins/unique_by_arity'
12
- require 'resque/unique_by_arity/configuration'
13
- require 'resque/unique_by_arity/modulizer'
14
- require 'resque/unique_by_arity/validation'
15
-
16
- # Usage:
17
- #
18
- # class MyJob
19
- # def self.perform(arg1, arg2)
20
- # end
21
- # include Resque::Plugins::UniqueByArity.new(
22
- # arity_for_uniqueness: 1,
23
- # arity_validation: :warning, # or nil, false, or :error
24
- # unique_at_runtime: true,
25
- # unique_in_queue: true
26
- # )
27
- # end
28
- #
29
- # NOTE: DO NOT include this module directly.
30
- # Use the Resque::Plugins::UniqueByArity approach as above.
31
- # This module is ultimately extended into the job class.
32
4
  module Resque
33
5
  module UniqueByArity
34
6
  PLUGIN_TAG = (ColorizedString['[R-UBA] '].green).freeze
35
7
 
36
- def unique_log(message, config_proxy = nil)
37
- config_proxy ||= uniqueness_configuration
38
- config_proxy.unique_logger&.send(config_proxy.unique_log_level, message) if config_proxy.unique_logger
39
- end
40
-
41
- def unique_debug(message, config_proxy = nil)
42
- config_proxy ||= uniqueness_configuration
43
- config_proxy.unique_logger&.debug("#{Resque::UniqueByArity::PLUGIN_TAG}#{message}") if config_proxy.debug_mode
8
+ def log(message, config_proxy = nil)
9
+ config_proxy ||= configuration
10
+ config_proxy.logger&.send(config_proxy.log_level, message) if config_proxy.logger
44
11
  end
45
- module_function(:unique_log, :unique_debug)
12
+ module_function(:log)
46
13
 
47
- # There are times when the class will need access to the configuration object,
48
- # such as to override it per instance method
49
- def uniq_config
50
- @uniqueness_configuration
14
+ def debug(message, config_proxy = nil)
15
+ config_proxy ||= configuration
16
+ config_proxy.logger&.debug("#{Resque::UniqueByArity::PLUGIN_TAG}#{message}") if config_proxy.debug_mode
51
17
  end
18
+ module_function(:debug)
52
19
 
53
- # For per-class config with a block
54
- def uniqueness_configure
55
- @uniqueness_configuration ||= Configuration.new
56
- yield(@uniqueness_configuration)
20
+ # For config with a block
21
+ def configure
22
+ yield(@configuration)
57
23
  end
58
24
 
59
25
  #### CONFIG ####
26
+ # Access globally configured settings:
27
+ # >> Resque::UniqueByArity.configuration.logger
28
+ # => the Logger instance
60
29
  class << self
61
- attr_accessor :uniqueness_configuration
62
- end
63
- def uniqueness_config_reset(config = Configuration.new)
64
- @uniqueness_configuration = config
65
- end
66
-
67
- def uniqueness_log_level
68
- @uniqueness_configuration.log_level
69
- end
70
-
71
- def uniqueness_log_level=(log_level)
72
- @uniqueness_configuration.log_level = log_level
73
- end
74
-
75
- def uniqueness_arity_for_uniqueness
76
- @uniqueness_configuration.arity_for_uniqueness
77
- end
78
-
79
- def uniqueness_arity_for_uniqueness=(arity_for_uniqueness)
80
- @uniqueness_configuration.arity_for_uniqueness = arity_for_uniqueness
81
- end
82
-
83
- def uniqueness_arity_validation
84
- @uniqueness_configuration.arity_validation
85
- end
86
-
87
- def uniqueness_arity_validation=(arity_validation)
88
- @uniqueness_configuration.arity_validation = arity_validation
89
- end
90
-
91
- def uniqueness_lock_after_execution_period
92
- @uniqueness_configuration.lock_after_execution_period
93
- end
94
-
95
- def uniqueness_lock_after_execution_period=(lock_after_execution_period)
96
- @uniqueness_configuration.lock_after_execution_period = lock_after_execution_period
97
- end
98
-
99
- def uniqueness_runtime_lock_timeout
100
- @uniqueness_configuration.runtime_lock_timeout
101
- end
102
-
103
- def uniqueness_runtime_lock_timeout=(runtime_lock_timeout)
104
- @uniqueness_configuration.runtime_lock_timeout = runtime_lock_timeout
105
- end
106
-
107
- def uniqueness_unique_at_runtime
108
- @uniqueness_configuration.unique_at_runtime
109
- end
110
-
111
- def uniqueness_unique_at_runtime=(unique_at_runtime)
112
- @uniqueness_configuration.unique_at_runtime = unique_at_runtime
113
- end
114
-
115
- def uniqueness_unique_in_queue
116
- @uniqueness_configuration.unique_in_queue
117
- end
118
-
119
- def uniqueness_unique_in_queue=(unique_in_queue)
120
- @uniqueness_configuration.unique_in_queue = unique_in_queue
121
- end
122
-
123
- def uniqueness_unique_across_queues
124
- @uniqueness_configuration.unique_across_queues
125
- end
126
-
127
- def uniqueness_unique_across_queues=(unique_across_queues)
128
- @uniqueness_configuration.unique_across_queues = unique_across_queues
30
+ attr_accessor :configuration
129
31
  end
130
- self.uniqueness_configuration = Configuration.new # setup defaults
32
+ self.configuration = GlobalConfiguration.instance # setup defaults
131
33
  end
132
34
  end
@@ -1,11 +1,13 @@
1
1
  require 'logger'
2
2
  module Resque
3
3
  module UniqueByArity
4
+ # This class is for configurations that are per job class, *not* app-wide.
5
+ # Each setting will default to the global config values.
4
6
  class Configuration
5
7
  VALID_ARITY_VALIDATION_LEVELS = [:warning, :error, :skip, nil, false].freeze
6
8
  SKIPPED_ARITY_VALIDATION_LEVELS = [:skip, nil, false].freeze
7
- DEFAULT_AT_RUNTIME_KEY_BASE = 'r-uar'.freeze
8
- DEFAULT_IN_QUEUE_KEY_BASE = 'r-uiq'.freeze
9
+ DEFAULT_LOG_LEVEL = :debug
10
+
9
11
  attr_accessor :logger
10
12
  attr_accessor :log_level
11
13
  attr_accessor :arity_for_uniqueness
@@ -18,27 +20,35 @@ module Resque
18
20
  attr_accessor :unique_in_queue
19
21
  attr_accessor :unique_in_queue_key_base
20
22
  attr_accessor :unique_across_queues
23
+ attr_accessor :ttl
21
24
  attr_accessor :base_klass_name
22
25
  attr_accessor :debug_mode
26
+
23
27
  def initialize(**options)
24
- @logger = options.key?(:logger) ? options[:logger] : Logger.new(STDOUT)
25
- @log_level = options.key?(:log_level) ? options[:log_level] : :debug
26
- @arity_for_uniqueness = options.key?(:arity_for_uniqueness) ? options[:arity_for_uniqueness] : 1
27
- @arity_validation = options.key?(:arity_validation) ? options[:arity_validation] : :warning
28
+ @logger = options.key?(:logger) ? options[:logger] : defcon(:logger) || Logger.new(STDOUT)
29
+ @log_level = options.key?(:log_level) ? options[:log_level] : defcon(:log_level) || :debug
30
+ @arity_for_uniqueness = options.key?(:arity_for_uniqueness) ? options[:arity_for_uniqueness] : defcon(:arity_for_uniqueness) || 1
31
+ @arity_validation = options.key?(:arity_validation) ? options[:arity_validation] : defcon(:arity_validation) || :warning
28
32
  raise ArgumentError, "Resque::Plugins::UniqueByArity.new requires arity_validation values of #{arity_validation.inspect}, or a class inheriting from Exception, but the value is #{@arity_validation} (#{@arity_validation.class})" unless VALID_ARITY_VALIDATION_LEVELS.include?(@arity_validation) || !@arity_validation.respond_to?(:ancestors) || @arity_validation.ancestors.include?(Exception)
29
33
 
30
- @lock_after_execution_period = options.key?(:lock_after_execution_period) ? options[:lock_after_execution_period] : nil
31
- @runtime_lock_timeout = options.key?(:runtime_lock_timeout) ? options[:runtime_lock_timeout] : nil
32
- @runtime_requeue_interval = options.key?(:runtime_requeue_interval) ? options[:runtime_requeue_interval] : nil
33
- @unique_at_runtime = options.key?(:unique_at_runtime) ? options[:unique_at_runtime] : false
34
- @unique_at_runtime_key_base = options.key?(:unique_at_runtime_key_base) ? options[:unique_at_runtime_key_base] : DEFAULT_AT_RUNTIME_KEY_BASE
35
- @unique_in_queue_key_base = options.key?(:unique_in_queue_key_base) ? options[:unique_in_queue_key_base] : DEFAULT_IN_QUEUE_KEY_BASE
36
- @unique_in_queue = options.key?(:unique_in_queue) ? options[:unique_in_queue] : false
37
- @unique_across_queues = options.key?(:unique_across_queues) ? options[:unique_across_queues] : false
38
- # Can't be both unique in queue and unique across queues.
34
+ @ttl = options.key?(:ttl) ? options[:ttl] : defcon(:ttl) || nil
35
+ @lock_after_execution_period = options.key?(:lock_after_execution_period) ? options[:lock_after_execution_period] : defcon(:lock_after_execution_period) || nil
36
+ @runtime_lock_timeout = options.key?(:runtime_lock_timeout) ? options[:runtime_lock_timeout] : defcon(:runtime_lock_timeout) || nil
37
+ @runtime_requeue_interval = options.key?(:runtime_requeue_interval) ? options[:runtime_requeue_interval] : defcon(:runtime_requeue_interval) || nil
38
+ @unique_at_runtime = options.key?(:unique_at_runtime) ? options[:unique_at_runtime] : defcon(:unique_at_runtime) || false
39
+ @unique_at_runtime_key_base = options.key?(:unique_at_runtime_key_base) ? options[:unique_at_runtime_key_base] : defcon(:unique_at_runtime_key_base) || nil
40
+ @unique_in_queue_key_base = options.key?(:unique_in_queue_key_base) ? options[:unique_in_queue_key_base] : defcon(:unique_in_queue_key_base) || nil
41
+ @unique_in_queue = options.key?(:unique_in_queue) ? options[:unique_in_queue] : defcon(:unique_in_queue) || false
42
+ @unique_across_queues = options.key?(:unique_across_queues) ? options[:unique_across_queues] : defcon(:unique_across_queues) || false
43
+ # Can't be both unique in queue and unique across queues, since they
44
+ # must necessarily use different locking key structures, and therefore
45
+ # can't both be active.
39
46
  raise ArgumentError, "Resque::Plugins::UniqueByArity.new requires either one or none of @unique_across_queues and @unique_in_queue to be true. Having both set to true is non-sensical." if @unique_in_queue && @unique_across_queues
40
- env_debug = ENV['RESQUE_DEBUG']
41
- @debug_mode = !!(options.key?(:debug_mode) ? options[:debug_mode] : env_debug == 'true' || (env_debug.is_a?(String) && env_debug.match?(/arity/)))
47
+ @debug_mode = !!(options.key?(:debug_mode) ? options[:debug_mode] : defcon(:debug_mode))
48
+ if @debug_mode
49
+ # Make sure there is a logger when in debug_mode
50
+ @logger ||= Logger.new(STDOUT)
51
+ end
42
52
  end
43
53
 
44
54
  def validate
@@ -52,16 +62,8 @@ module Resque
52
62
  end
53
63
  end
54
64
 
55
- def unique_logger
56
- logger
57
- end
58
-
59
- def unique_log_level
60
- log_level
61
- end
62
-
63
65
  def log(msg)
64
- Resque::UniqueByArity.unique_log(msg, self)
66
+ Resque::UniqueByArity.log(msg, self)
65
67
  end
66
68
 
67
69
  def to_hash
@@ -74,6 +76,7 @@ module Resque
74
76
  debug_mode: debug_mode,
75
77
  lock_after_execution_period: lock_after_execution_period,
76
78
  runtime_lock_timeout: runtime_lock_timeout,
79
+ ttl: ttl,
77
80
  unique_at_runtime: unique_at_runtime,
78
81
  unique_in_queue: unique_in_queue,
79
82
  unique_across_queues: unique_across_queues
@@ -121,6 +124,21 @@ module Resque
121
124
  end
122
125
  end
123
126
  end
127
+
128
+ def defcon(sym)
129
+ Resque::UniqueByArity.configuration.send(sym)
130
+ end
131
+
132
+ def debug_mode=(val)
133
+ @debug_mode = !!val
134
+ end
135
+
136
+ private
137
+
138
+ def debug_mode_from_env
139
+ env_debug = ENV['RESQUE_DEBUG']
140
+ @debug_mode = !!(env_debug == 'true' || (env_debug.is_a?(String) && env_debug.match?(/queue/)))
141
+ end
124
142
  end
125
143
  end
126
144
  end
@@ -0,0 +1,52 @@
1
+ require 'logger'
2
+ module Resque
3
+ module UniqueByArity
4
+ # This class is for configurations that are app-wide, *not per job class.
5
+ # For this reason it is a Singleton.
6
+ # Will be used as the default settings for the per-job configs.
7
+ class GlobalConfiguration < Configuration
8
+ DEFAULT_LOG_LEVEL = :debug
9
+ DEFAULT_AT_RUNTIME_KEY_BASE = 'r-uar'.freeze
10
+ DEFAULT_IN_QUEUE_KEY_BASE = 'r-uiq'.freeze
11
+
12
+ # For resque-unique_at_runtime
13
+ DEFAULT_LOCK_TIMEOUT = 60 * 60 * 24 * 5
14
+ DEFAULT_REQUEUE_INTERVAL = 1
15
+ DEFAULT_UNIQUE_AT_RUNTIME_KEY_BASE = 'r-uar'.freeze
16
+
17
+ # For resque-unique_in_queue
18
+ DEFAULT_LOCK_AFTER_EXECUTION_PERIOD = 0
19
+ DEFAULT_TTL = -1
20
+ DEFAULT_UNIQUE_IN_QUEUE_KEY_BASE = 'r-uiq'.freeze
21
+
22
+ include Singleton
23
+
24
+ def initialize
25
+ debug_mode_from_env
26
+ @logger = nil
27
+ @log_level = DEFAULT_LOG_LEVEL
28
+ @arity_for_uniqueness = nil
29
+ @arity_validation = nil
30
+ @lock_after_execution_period = DEFAULT_LOCK_AFTER_EXECUTION_PERIOD
31
+ @runtime_lock_timeout = DEFAULT_LOCK_TIMEOUT
32
+ @runtime_requeue_interval = DEFAULT_REQUEUE_INTERVAL
33
+ @unique_at_runtime_key_base = DEFAULT_AT_RUNTIME_KEY_BASE
34
+ @unique_in_queue_key_base = DEFAULT_IN_QUEUE_KEY_BASE
35
+ @unique_at_runtime = false
36
+ @unique_at_runtime_key_base = DEFAULT_UNIQUE_AT_RUNTIME_KEY_BASE
37
+ @unique_in_queue_key_base = DEFAULT_UNIQUE_IN_QUEUE_KEY_BASE
38
+ @unique_in_queue = false
39
+ @unique_across_queues = false
40
+ @ttl = DEFAULT_TTL
41
+ if @debug_mode
42
+ # Make sure there is a logger when in debug_mode
43
+ @logger ||= Logger.new(STDOUT)
44
+ end
45
+ end
46
+
47
+ def defcon(sym)
48
+ self.send(sym)
49
+ end
50
+ end
51
+ end
52
+ end
@@ -7,7 +7,7 @@ module Resque
7
7
  # @return [Array<String, arguments>] the key base hash used to enforce uniqueness, and the arguments from the payload used to calculate it
8
8
  define_method(:redis_unique_hash) do |payload|
9
9
  payload = Resque.decode(Resque.encode(payload))
10
- Resque::UniqueByArity.unique_debug("payload is #{payload.inspect}")
10
+ Resque::UniqueByArity.debug("payload is #{payload.inspect}")
11
11
  job = payload['class']
12
12
  # It seems possible that some jobs may not have an "args" key in the payload.
13
13
  args = payload['args'] || []
@@ -28,36 +28,13 @@ module Resque
28
28
  end
29
29
  end
30
30
 
31
- if configuration.lock_after_execution_period
32
- instance_variable_set(:@lock_after_execution_period, configuration.lock_after_execution_period)
33
- end
34
-
35
- if configuration.runtime_lock_timeout
36
- instance_variable_set(:@runtime_lock_timeout, configuration.runtime_lock_timeout)
37
- end
38
-
39
- if configuration.runtime_requeue_interval
40
- instance_variable_set(:@runtime_requeue_interval, configuration.runtime_requeue_interval)
41
- end
42
-
43
- if configuration.unique_at_runtime_key_base
44
- instance_variable_set(:@unique_at_runtime_key_base, configuration.unique_at_runtime_key_base)
45
- end
46
-
47
- if configuration.unique_in_queue_key_base
48
- # Can't be overridden per each class because it wouldn't make sense.
49
- # It wouldn't be able to determine or enforce uniqueness across queues,
50
- # and general cleanup of stray keys would be nearly impossible.
51
- Resque::UniqueInQueue.uniq_config&.unique_in_queue_key_base = configuration.unique_in_queue_key_base
52
- end
53
-
54
31
  if configuration.unique_in_queue || configuration.unique_across_queues
55
- ### Gem: unique_in_queue
32
+ ### Gem: resque-unique_in_queue
56
33
  ### Plugin Name: Resque::Plugins::UniqueJob
57
34
  ### Provides: Queue-time uniqueness for a single queue, or across queues
58
35
  #
59
36
  # Returns a string, used by Resque::Plugins::UniqueJob, that will be used as the prefix to the redis key
60
- define_method(:unique_at_runtime_redis_key_prefix) do
37
+ define_method(:unique_in_queue_redis_key_prefix) do
61
38
  "unique_job:#{self}"
62
39
  end
63
40
  #
@@ -69,28 +46,29 @@ module Resque
69
46
  # @return [String] the key used to enforce uniqueness (at queue-time)
70
47
  define_method(:unique_in_queue_redis_key) do |queue, payload|
71
48
  unique_hash, args_for_uniqueness = redis_unique_hash(payload)
72
- key = "#{unique_at_runtime_key_namespace(queue)}:#{unique_at_runtime_redis_key_prefix}:#{unique_hash}"
73
- Resque::UniqueByArity.unique_debug("#{self}.unique_in_queue_redis_key for #{args_for_uniqueness} is: #{ColorizedString[key].green}")
49
+ key = "#{unique_in_queue_key_namespace(queue)}:#{unique_in_queue_redis_key_prefix}:#{unique_hash}"
50
+ Resque::UniqueByArity.debug("#{self}.unique_in_queue_redis_key for #{args_for_uniqueness} is: #{ColorizedString[key].green}")
74
51
  key
75
52
  end
76
53
  #
77
54
  # @return [Fixnum] number of keys that were deleted
78
55
  define_method(:purge_unique_queued_redis_keys) do
79
56
  # unique_at_runtime_key_namespace may or may not ignore the queue passed in, depending on config.
80
- key_match = "#{unique_at_runtime_key_namespace(instance_variable_get(:@queue))}:#{unique_at_runtime_redis_key_prefix}:*"
57
+ key_match = "#{unique_in_queue_key_namespace(instance_variable_get(:@queue))}:#{unique_in_queue_redis_key_prefix}:*"
81
58
  keys = Resque.redis.keys(key_match)
82
- Resque::UniqueByArity.unique_log("#{Resque::UniqueByArity::PLUGIN_TAG}#{Resque::UniqueInQueue::PLUGIN_TAG} Purging #{keys.length} keys from #{ColorizedString[key_match].red}")
59
+ Resque::UniqueByArity.log("#{Resque::UniqueByArity::PLUGIN_TAG}#{Resque::UniqueInQueue::PLUGIN_TAG} #{ColorizedString['Purging'].red} #{keys.length} keys from #{ColorizedString[key_match].red}")
83
60
  Resque.redis.del keys unless keys.empty?
84
61
  end
62
+
85
63
  if configuration.unique_in_queue
86
64
  # @return [String] the Redis namespace of the key used to enforce uniqueness (at queue-time)
87
- define_method(:unique_at_runtime_key_namespace) do |queue = nil|
88
- "#{configuration.unique_in_queue_key_base}:queue:#{queue}:job"
65
+ define_method(:unique_in_queue_key_namespace) do |queue = nil|
66
+ "#{unique_in_queue_key_base}:queue:#{queue}:job"
89
67
  end
90
68
  elsif configuration.unique_across_queues
91
69
  # @return [String] the Redis namespace of the key used to enforce uniqueness (at queue-time)
92
- define_method(:unique_at_runtime_key_namespace) do |_queue = nil|
93
- "#{configuration.unique_in_queue_key_base}:across_queues:job"
70
+ define_method(:unique_in_queue_key_namespace) do |_queue = nil|
71
+ "#{unique_in_queue_key_base}:across_queues:job"
94
72
  end
95
73
  end
96
74
  end
@@ -101,27 +79,26 @@ module Resque
101
79
  if configuration.unique_at_runtime
102
80
  # @return [String] the Redis namespace of the key used to enforce uniqueness (at runtime)
103
81
  define_method(:runtime_key_namespace) do
104
- "#{configuration.unique_at_runtime_key_base}:#{self}"
82
+ "#{unique_at_runtime_key_base}:#{self}"
105
83
  end
106
84
  # Returns a string, used by Resque::Plugins::UniqueAtRuntime, that will be used as the redis key
107
- # The versions of redis_key from unique_in_queue and resque-lonely_job are incompatible.
108
- # So we forked resque-lonely_job, change the name of the method so it would not conflict,
109
- # and now we can override it, and fix the params to be compatible with the redis_key
110
- # from unique_in_queue
111
- # Does not need any customization for arity, because it funnels down to redis_key,
112
- # and we handle the arity option there
85
+ # The versions of redis_key from resque-solo and resque-lonely_job are incompatible.
86
+ # So I forked and namespaced the methods according to the gem handling the key.
87
+ # Now I can override it, and fix the params to be compatible.
88
+ # Does not need any customization for arity, because ultimately it
89
+ # still funnels down to redis_key, and we handle the arity option there
113
90
  # @return [String] the key used to enforce loneliness (uniqueness at runtime)
114
91
  define_method(:unique_at_runtime_redis_key) do |*args|
115
92
  unique_hash, args_for_uniqueness = redis_unique_hash('class' => to_s, 'args' => args)
116
93
  key = "#{runtime_key_namespace}:#{unique_hash}"
117
- Resque::UniqueByArity.unique_debug("#{ColorizedString['[R-UAR]'].yellow} #{self}.unique_at_runtime_redis_key for #{args_for_uniqueness} is: #{ColorizedString[key].yellow}")
94
+ Resque::UniqueByArity.debug("#{Resque::UniqueAtRuntime::PLUGIN_TAG} #{self}.unique_at_runtime_redis_key for #{args_for_uniqueness} is: #{ColorizedString[key].yellow}")
118
95
  key
119
96
  end
120
97
  # @return [Fixnum] number of keys that were deleted
121
98
  define_method(:purge_unique_at_runtime_redis_keys) do
122
99
  key_match = "#{runtime_key_namespace}:*"
123
100
  keys = Resque.redis.keys(key_match)
124
- Resque::UniqueByArity.unique_log("#{ColorizedString['[R-UBA][R-UAR]'].red} Purging #{keys.length} keys from #{ColorizedString[key_match].red}")
101
+ Resque::UniqueByArity.log("#{Resque::UniqueByArity::PLUGIN_TAG}#{Resque::UniqueAtRuntime::PLUGIN_TAG} #{ColorizedString['Purging'].red} #{keys.length} keys from #{ColorizedString[key_match].red}")
125
102
  Resque.redis.del keys unless keys.empty?
126
103
  end
127
104
  end
@@ -0,0 +1,53 @@
1
+ # Usage:
2
+ #
3
+ # class MyJob
4
+ # def self.perform(arg1, arg2)
5
+ # end
6
+ # include Resque::Plugins::UniqueByArity.new(
7
+ # arity_for_uniqueness: 1,
8
+ # arity_validation: :warning, # or nil, false, or :error
9
+ # unique_at_runtime: true,
10
+ # unique_in_queue: true
11
+ # )
12
+ # end
13
+ #
14
+ # NOTE: DO NOT include this module directly.
15
+ # Use the Resque::Plugins::UniqueByArity approach as above.
16
+ # This module is ultimately extended into the job class.
17
+ module Resque
18
+ module UniqueByArity
19
+ module UniqueJob
20
+ PLUGIN_TAG = (ColorizedString['[R-UBA] '].green).freeze
21
+
22
+ def uniq_log(message, config_proxy = nil)
23
+ config_proxy ||= uniq_config
24
+ config_proxy.logger&.send(config_proxy.log_level, message) if config_proxy.logger
25
+ end
26
+
27
+ def uniq_debug(message, config_proxy = nil)
28
+ config_proxy ||= uniq_config
29
+ config_proxy.logger&.debug("#{Resque::UniqueByArity::PLUGIN_TAG}#{message}") if config_proxy.debug_mode
30
+ end
31
+
32
+ # For per-class config with a block
33
+ def uniqueness_configure
34
+ @uniqueness_configuration ||= Configuration.new
35
+ yield(@uniqueness_configuration)
36
+ end
37
+
38
+ #### CONFIG ####
39
+ class << self
40
+ attr_accessor :uniqueness_configuration
41
+ end
42
+ self.uniqueness_configuration = Configuration.new # setup defaults
43
+
44
+ def uniqueness_config_reset(config = Configuration.new)
45
+ @uniqueness_configuration = config
46
+ end
47
+
48
+ def uniq_config
49
+ @uniqueness_configuration
50
+ end
51
+ end
52
+ end
53
+ end
@@ -1,5 +1,5 @@
1
1
  module Resque
2
2
  module UniqueByArity
3
- VERSION = '2.0.2'.freeze
3
+ VERSION = '3.0.0'.freeze
4
4
  end
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: resque-unique_by_arity
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.2
4
+ version: 3.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Peter H. Boling
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2018-11-10 00:00:00.000000000 Z
11
+ date: 2018-11-15 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: colorize
@@ -216,7 +216,6 @@ extra_rdoc_files: []
216
216
  files:
217
217
  - ".gitignore"
218
218
  - ".rspec"
219
- - ".rspec_status"
220
219
  - ".rubocop.yml"
221
220
  - ".rubocop_todo.yml"
222
221
  - ".ruby-version"
@@ -227,10 +226,13 @@ files:
227
226
  - Rakefile
228
227
  - bin/console
229
228
  - bin/setup
229
+ - lib/resque-unique_by_arity.rb
230
230
  - lib/resque/plugins/unique_by_arity.rb
231
231
  - lib/resque/unique_by_arity.rb
232
232
  - lib/resque/unique_by_arity/configuration.rb
233
+ - lib/resque/unique_by_arity/global_configuration.rb
233
234
  - lib/resque/unique_by_arity/modulizer.rb
235
+ - lib/resque/unique_by_arity/unique_job.rb
234
236
  - lib/resque/unique_by_arity/validation.rb
235
237
  - lib/resque/unique_by_arity/version.rb
236
238
  - resque-unique_by_arity.gemspec
@@ -1,55 +0,0 @@
1
- example_id | status | run_time |
2
- ----------------------------------------------------------------- | ------ | --------------- |
3
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:1] | passed | 0.00313 seconds |
4
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:2:1] | passed | 0.00099 seconds |
5
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:3:1] | passed | 0.00038 seconds |
6
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:4:1] | passed | 0.00016 seconds |
7
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:5:1] | passed | 0.00017 seconds |
8
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:6:1] | passed | 0.00018 seconds |
9
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:7:1] | passed | 0.0002 seconds |
10
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:8:1] | passed | 0.00016 seconds |
11
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:9:1] | passed | 0.00014 seconds |
12
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:10:1] | passed | 0.00014 seconds |
13
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:11:1] | passed | 0.00017 seconds |
14
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:12:1] | passed | 0.00015 seconds |
15
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:13:1] | passed | 0.00017 seconds |
16
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:14:1] | passed | 0.00096 seconds |
17
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:14:2:1] | passed | 0.00014 seconds |
18
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:1:14:3:1] | passed | 0.00013 seconds |
19
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:2:1] | passed | 0.01014 seconds |
20
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:3:1] | passed | 0.00031 seconds |
21
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:3:2] | passed | 0.00024 seconds |
22
- ./spec/resque/unique_by_arity/configuration_spec.rb[1:1:4:1] | passed | 0.00017 seconds |
23
- ./spec/resque/unique_by_arity_spec.rb[1:1] | passed | 0.00006 seconds |
24
- ./spec/resque/unique_by_arity_spec.rb[1:2:1:1] | passed | 0.00094 seconds |
25
- ./spec/resque/unique_by_arity_spec.rb[1:3:1] | passed | 0.00062 seconds |
26
- ./spec/resque/unique_by_arity_spec.rb[1:4:1:1] | passed | 0.00022 seconds |
27
- ./spec/resque/unique_by_arity_spec.rb[1:5:1:1] | passed | 0.00086 seconds |
28
- ./spec/resque/unique_by_arity_spec.rb[1:5:2:1] | passed | 0.00026 seconds |
29
- ./spec/resque/unique_by_arity_spec.rb[1:5:3:1] | passed | 0.00027 seconds |
30
- ./spec/resque/unique_by_arity_spec.rb[1:6:1] | passed | 0.00047 seconds |
31
- ./spec/resque/unique_by_arity_spec.rb[1:7:1] | passed | 0.00021 seconds |
32
- ./spec/resque/unique_by_arity_spec.rb[1:7:2:1] | passed | 0.00057 seconds |
33
- ./spec/resque/unique_by_arity_spec.rb[1:7:3:1] | passed | 0.00062 seconds |
34
- ./spec/resque/unique_by_arity_spec.rb[1:7:4:1] | passed | 0.00021 seconds |
35
- ./spec/resque/unique_by_arity_spec.rb[1:8:1:1:1:1] | passed | 0.00028 seconds |
36
- ./spec/resque/unique_by_arity_spec.rb[1:8:1:2:1:1] | passed | 0.00025 seconds |
37
- ./spec/resque/unique_by_arity_spec.rb[1:8:1:3:1:1] | passed | 0.00061 seconds |
38
- ./spec/resque/unique_by_arity_spec.rb[1:8:2:1:1:1] | passed | 0.00022 seconds |
39
- ./spec/resque/unique_by_arity_spec.rb[1:8:2:2:1:1] | passed | 0.00029 seconds |
40
- ./spec/resque/unique_by_arity_spec.rb[1:8:2:3:1:1] | passed | 0.00057 seconds |
41
- ./spec/resque/unique_by_arity_spec.rb[1:8:3:1:1:1] | passed | 0.00076 seconds |
42
- ./spec/resque/unique_by_arity_spec.rb[1:8:3:2:1:1] | passed | 0.00075 seconds |
43
- ./spec/resque/unique_by_arity_spec.rb[1:8:3:3:1:1] | passed | 0.00062 seconds |
44
- ./spec/resque/unique_by_arity_spec.rb[1:8:4:1:1:1] | passed | 0.0004 seconds |
45
- ./spec/resque/unique_by_arity_spec.rb[1:8:4:2:1:1] | passed | 0.00032 seconds |
46
- ./spec/resque/unique_by_arity_spec.rb[1:8:4:3:1:1] | passed | 0.00042 seconds |
47
- ./spec/resque/unique_by_arity_spec.rb[1:8:5:1:1:1] | passed | 0.00023 seconds |
48
- ./spec/resque/unique_by_arity_spec.rb[1:8:5:2:1:1] | passed | 0.00029 seconds |
49
- ./spec/resque/unique_by_arity_spec.rb[1:8:5:3:1] | passed | 0.0002 seconds |
50
- ./spec/resque/unique_by_arity_spec.rb[1:9:1] | passed | 0.0025 seconds |
51
- ./spec/resque/unique_by_arity_spec.rb[1:10:1:1] | passed | 0.00012 seconds |
52
- ./spec/resque/unique_by_arity_spec.rb[1:11:1:1] | passed | 0.00029 seconds |
53
- ./spec/resque/unique_by_arity_spec.rb[1:11:2:1:1] | passed | 0.00025 seconds |
54
- ./spec/resque/unique_by_arity_spec.rb[1:11:2:2:1] | passed | 0.0006 seconds |
55
- ./spec/resque/unique_by_arity_spec.rb[1:11:2:3:1] | passed | 0.0005 seconds |