sidekiq-debouncer 1.0.1 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9f505c753200cb5a607d4a0f847372a72f5f5571631d6b50e12348e99b407f30
4
- data.tar.gz: b72d1d31b9b1b13a7ab8ec88a56a27e2a84e11a8dca721e296ec4a25da361dd7
3
+ metadata.gz: 939e5397d600c7f5f24b3757f46f0c613369079ef327c8e88bf5adcdf807733f
4
+ data.tar.gz: 6ee4f636b9297a232a82bcf6c27c1418408cd2ad66ad8e473945fd1fdbc12cb0
5
5
  SHA512:
6
- metadata.gz: 400505251465ceca2630d226a8f1017518a5bb068e755a05428e8f00be81519c2ddcda91be840c24f3eaf382ff7302e1c1941fe790eec95a5cb9f9d9f72dd797
7
- data.tar.gz: 6d01faab4b9f4b98512dadb598100674d241ab3fd99a324e1abb89773a9a4c92ba8782fb1564e68a97d4c171646ec9f6472a25cdb507eba218d739f368f1226f
6
+ metadata.gz: 3b84978f7ea580d81a052719860ef38fc10339d35064c768457ffcccf0effb7dd148a2bd7a6620489a68c5a03705aea400f2ce8f493e2d0e27fa48edcb806082
7
+ data.tar.gz: f0e2d36120e11883f9b2f29b4de2f4fcbfbae075c72592742b9b6a3f323458a0bf7222fb4853604df45c1755e15ead8b600024630bd976631d3df4b5679f996a
data/CHANGELOG.md ADDED
@@ -0,0 +1,12 @@
1
+ ## [2.0.0] - 2023-02-28
2
+ Complete rewrite of the library:
3
+ - Instead of iterating through whole schedule set, sidekiq-debouncer will now cache debounce key in redis with a reference to the job.
4
+ Thanks to that there is a huge performance boost compared to V1. With 1k jobs in schedule set it's over 100x faster.
5
+ The difference is even bigger with larger amount of jobs.
6
+ - Debouncing is now handled by Lua script instead of pure ruby so it's process safe.
7
+
8
+ Breaking changes:
9
+ - Including `Sidekiq::Debouncer` in the workers and using `debounce` method is now deprecated. Use `perform_async` instead.
10
+ - Setup requires middlewares to be added in sidekiq configuration.
11
+ - `by` attribute is now required
12
+ - dropped support for Ruby < 2.7 and Sidekiq < 6.5
data/LICENSE.txt CHANGED
@@ -1,4 +1,4 @@
1
- Copyright (c) 2012 Gabriel Evans
1
+ Copyright (c) 2023 Karol Bąk
2
2
 
3
3
  MIT License
4
4
 
@@ -19,4 +19,4 @@ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
19
  NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
20
  LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
21
  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
- WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md CHANGED
@@ -2,7 +2,9 @@
2
2
 
3
3
  Sidekiq extension that adds the ability to debounce job execution.
4
4
 
5
- Worker will postpone its execution after `wait time` have elapsed since the last time it was invoked. Useful for implementing behavior that should only happen after the input has stopped arriving. For example: sending group email to the user after he stopped interacting with the application.
5
+ Worker will postpone its execution after `wait time` have elapsed since the last time it was invoked.
6
+ Useful for implementing behavior that should only happen after the input has stopped arriving.
7
+ For example: sending one email with multiple notifications.
6
8
 
7
9
  ## Installation
8
10
 
@@ -18,39 +20,35 @@ And then execute:
18
20
 
19
21
  ## Basic usage
20
22
 
21
- In a worker, include `Sidekiq::Debouncer` module, specify debounce wait time (in seconds):
23
+ Add middlewares to sidekiq:
22
24
 
23
25
  ```ruby
24
- class MyWorker
25
- include Sidekiq::Worker
26
- include Sidekiq::Debouncer
27
-
28
- sidekiq_options(
29
- debounce: {
30
- time: 5 * 60
31
- }
32
- )
26
+ Sidekiq.configure_client do |config|
27
+ config.client_middleware do |chain|
28
+ chain.add Sidekiq::Debouncer::Middleware::Client
29
+ end
30
+ end
33
31
 
34
- def perform(group)
35
- group.each do
36
- # do some work with group
37
- end
32
+ Sidekiq.configure_server do |config|
33
+ config.client_middleware do |chain|
34
+ chain.add Sidekiq::Debouncer::Middleware::Client
35
+ end
36
+
37
+ config.server_middleware do |chain|
38
+ chain.add Sidekiq::Debouncer::Middleware::Server
38
39
  end
39
40
  end
40
41
  ```
41
42
 
42
- You can specify your own debounce method. In this case worker will be debounced if first argument matches.
43
+ Add debounce option to worker with `by` and `time` keys:
43
44
  ```ruby
44
45
  class MyWorker
45
46
  include Sidekiq::Worker
46
- include Sidekiq::Debouncer
47
47
 
48
48
  sidekiq_options(
49
49
  debounce: {
50
- time: 5 * 60,
51
- by: -> (job_args) {
52
- job_args[0]
53
- }
50
+ by: -> (args) { args[0] }, # debounce by first argument only
51
+ time: 5 * 60
54
52
  }
55
53
  )
56
54
 
@@ -62,11 +60,10 @@ class MyWorker
62
60
  end
63
61
  ```
64
62
 
65
- You can also pass symbol as `debounce_by` matching class method.
63
+ You can also pass symbol as `debounce.by` matching class method.
66
64
  ```ruby
67
65
  class MyWorker
68
66
  include Sidekiq::Worker
69
- include Sidekiq::Debouncer
70
67
 
71
68
  sidekiq_options(
72
69
  debounce: {
@@ -87,9 +84,29 @@ class MyWorker
87
84
  end
88
85
  ```
89
86
 
87
+ Keep in mind that the result of the debounce method will be converted to string, so make sure it doesn't return any objects that don't implement `to_s` method.
88
+
89
+ In the application, call `MyWorker.perform_async(...)` as usual. Everytime you call this function, `MyWorker`'s execution will be postponed by 5 minutes. After that time `MyWorker` will receive a method call `perform` with an array of arguments that were provided to the `MyWorker.perform_async(...)` calls.
90
+
91
+ To avoid keeping leftover keys in redis (for example, when job was manually removed from schedule set), all additional keys are created with TTL.
92
+ It's 7 days by default and should be ok in most of the cases. If you are debouncing your jobs in higher interval than that, you can overwrite this setting:
93
+
94
+ ```ruby
95
+ Sidekiq.configure_client do |config|
96
+ config.client_middleware do |chain|
97
+ chain.add Sidekiq::Debouncer::Middleware::Client, ttl: 60 * 60 * 24 * 30 # 30 days
98
+ end
99
+ end
100
+ ```
101
+
102
+ ## Testing
90
103
 
91
- In the application, call `MyWorker.debounce(...)`. Everytime you call this function, `MyWorker`'s execution will be postponed by 5 minutes. After that time `MyWorker` will receive a method call `perform` with an array of arguments that were provided to the `MyWorker.debounce(...)`.
104
+ In order to test the behavior of `sidekiq-debouncer` it is necessary to disable testing mode. It is the limitation of internal implementation.
92
105
 
93
106
  ## License
94
107
 
95
108
  MIT Licensed. See LICENSE.txt for details.
109
+
110
+ ## Notes
111
+
112
+ This gem was renamed from `sidekiq-debouce` due to name conflict on rubygems.
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Sidekiq
4
+ module Debouncer
5
+ Error = Class.new(StandardError)
6
+ NotSupportedError = Class.new(Error)
7
+ MissingArgumentError = Class.new(Error)
8
+ end
9
+ end
@@ -0,0 +1,22 @@
1
+ local set = KEYS[1]
2
+ local debounce_key = KEYS[2]
3
+
4
+ local job = ARGV[1]
5
+ local time = ARGV[2]
6
+ local ttl = ARGV[3]
7
+
8
+ local existing_debounce = redis.call("GET", debounce_key)
9
+
10
+ if existing_debounce then
11
+ redis.call("DEL", debounce_key)
12
+ -- skip if job wasn't found in schedule set
13
+ if redis.call("ZREM", set, existing_debounce) > 0 then
14
+ local new_args = cjson.decode(job)['args'][1]
15
+ local new_job = cjson.decode(existing_debounce)
16
+ table.insert(new_job['args'], new_args)
17
+ job = cjson.encode(new_job)
18
+ end
19
+ end
20
+
21
+ redis.call("SET", debounce_key, job, "EX", ttl)
22
+ redis.call("ZADD", set, time, job)
@@ -0,0 +1,85 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "digest/sha1"
4
+
5
+ module Sidekiq
6
+ module Debouncer
7
+ module Middleware
8
+ # Middleware used to debounce jobs. If a job has a debounce option it skips normal sidekiq flow and debounces
9
+ # job using lua script (thanks to that it's process safe). Script merges new job with existing one and creates
10
+ # debounce key in redis with a reference to the job placed in schedule set. Reference is used to remove existing
11
+ # job from schedule set when another debounce occurs.
12
+ class Client
13
+ include Sidekiq::ClientMiddleware
14
+
15
+ LUA_DEBOUNCE = File.read(File.expand_path("../../lua/debounce.lua", __FILE__))
16
+ LUA_DEBOUNCE_SHA = Digest::SHA1.hexdigest(LUA_DEBOUNCE)
17
+ REDIS_ERROR_CLASS = defined?(RedisClient::CommandError) ? RedisClient::CommandError : Redis::CommandError
18
+
19
+ def initialize(options = {})
20
+ @debounce_key_ttl = options.fetch(:ttl, 60 * 60 * 24 * 7) # 7 days by default
21
+ end
22
+
23
+ def call(worker_class, job, _queue, _redis_pool)
24
+ klass = worker_class.is_a?(String) ? Object.const_get(worker_class) : worker_class
25
+
26
+ yield # call the rest of middleware stack
27
+ # skip if debounce options not set or client middleware is included in server
28
+ return job if !klass.get_sidekiq_options["debounce"] || job["debounce_key"]
29
+
30
+ debounce(klass, job)
31
+
32
+ # prevent normal sidekiq flow
33
+ false
34
+ end
35
+
36
+ private
37
+
38
+ def debounce(klass, job)
39
+ raise NotSupportedError, "perform_at is not supported with debounce" if job.key?("at")
40
+
41
+ options = debounce_options(klass)
42
+ key = debounce_key(klass, job, options)
43
+ time = (options[:time].to_f + Time.now.to_f).to_s
44
+
45
+ job["debounce_key"] = key
46
+ job["args"] = [job["args"]]
47
+ job.delete("debounce")
48
+
49
+ redis do |connection|
50
+ redis_debounce(connection, keys: ["schedule", key], argv: [Sidekiq.dump_json(job), time, @debounce_key_ttl])
51
+ end
52
+ end
53
+
54
+ def debounce_key(klass, job, options)
55
+ method = options[:by]
56
+ result = method.is_a?(Symbol) ? klass.send(method, job["args"]) : method.call(job["args"])
57
+ "debounce/#{klass.name}/#{result}"
58
+ end
59
+
60
+ def debounce_options(klass)
61
+ options = klass.get_sidekiq_options["debounce"].transform_keys(&:to_sym)
62
+
63
+ raise MissingArgumentError, "'by' attribute not provided" unless options[:by]
64
+ raise MissingArgumentError, "'time' attribute not provided" unless options[:time]
65
+
66
+ options
67
+ end
68
+
69
+ def redis_debounce(connection, keys:, argv:)
70
+ retryable = true
71
+ begin
72
+ connection.call("EVALSHA", LUA_DEBOUNCE_SHA, keys.size, *keys, *argv)
73
+ rescue REDIS_ERROR_CLASS => e
74
+ raise if !e.message.start_with?("NOSCRIPT") || !retryable
75
+
76
+ # upload script to redis cache and retry
77
+ connection.call("SCRIPT", "LOAD", LUA_DEBOUNCE)
78
+ retryable = false
79
+ retry
80
+ end
81
+ end
82
+ end
83
+ end
84
+ end
85
+ end
@@ -0,0 +1,27 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Sidekiq
4
+ module Debouncer
5
+ module Middleware
6
+ # Server middleware removes debounce key from redis before executing the job
7
+ class Server
8
+ include Sidekiq::ServerMiddleware
9
+
10
+ def call(_worker, job, _queue)
11
+ if job.key?("debounce_key")
12
+ # skip if job comes from dead or retry set
13
+ unless job.key?("failed_at")
14
+ redis do |connection|
15
+ connection.call("DEL", job["debounce_key"])
16
+ end
17
+ end
18
+
19
+ job["args"] = [job["args"]] # wrap args into array because sidekiq uses splat while calling perform
20
+ end
21
+
22
+ yield
23
+ end
24
+ end
25
+ end
26
+ end
27
+ end
@@ -1,5 +1,7 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Sidekiq
2
4
  module Debouncer
3
- VERSION = '1.0.1'.freeze
5
+ VERSION = "2.0.0"
4
6
  end
5
7
  end
@@ -1,7 +1,9 @@
1
- require 'sidekiq'
2
- require 'sidekiq/api'
1
+ # frozen_string_literal: true
3
2
 
4
- require 'sidekiq/debouncer/version'
3
+ require "sidekiq/debouncer/version"
4
+ require "sidekiq/debouncer/errors"
5
+ require "sidekiq/debouncer/middleware/client"
6
+ require "sidekiq/debouncer/middleware/server"
5
7
 
6
8
  module Sidekiq
7
9
  module Debouncer
@@ -10,38 +12,9 @@ module Sidekiq
10
12
  end
11
13
 
12
14
  module ClassMethods
13
- DEFAULT_DEBOUNCE_FOR = 5 * 60 # 5.minutes
14
- DEFAULT_DEBOUNCE_BY = -> (job_args) { 0 }
15
-
16
- def debounce(*args)
17
- sidekiq_options["debounce"] ||= {}
18
-
19
- debounce_for = sidekiq_options["debounce"][:time] || DEFAULT_DEBOUNCE_FOR
20
- debounce_by = sidekiq_options["debounce"][:by] || DEFAULT_DEBOUNCE_BY
21
- debounce_by_value = debounce_by.is_a?(Symbol) ? send(debounce_by, args) : debounce_by.call(args)
22
-
23
- ss = Sidekiq::ScheduledSet.new
24
- jobs = ss.select do |job|
25
- next false unless job.klass == self.to_s
26
-
27
- debounce_by_value_job = debounce_by.is_a?(Symbol) ? send(debounce_by, job.args[0][0]) : debounce_by.call(job.args[0][0])
28
-
29
- debounce_by_value_job == debounce_by_value
30
- end
31
-
32
- time_from_now = Time.now + debounce_for
33
- jobs_to_group = []
34
-
35
- jobs.each do |job|
36
- if job.at > Time.now && job.at < time_from_now
37
- jobs_to_group += job.args[0]
38
- job.delete
39
- end
40
- end
41
-
42
- jobs_to_group << args
43
-
44
- perform_in(debounce_for, jobs_to_group)
15
+ def debounce(...)
16
+ warn "WARNING: debounce method is deprecated, use perform_async instead"
17
+ perform_async(...)
45
18
  end
46
19
  end
47
20
  end
@@ -0,0 +1,4 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "sidekiq"
4
+ require "sidekiq/debouncer"
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ lib = File.expand_path("../lib", __FILE__)
4
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
5
+
6
+ require "sidekiq/debouncer/version"
7
+
8
+ Gem::Specification.new do |gem|
9
+ gem.name = "sidekiq-debouncer"
10
+ gem.version = Sidekiq::Debouncer::VERSION
11
+ gem.authors = ["Sebastian Zuchmański", "Karol Bąk"]
12
+ gem.email = ["sebcioz@gmail.com", "kukicola@gmail.com"]
13
+ gem.summary = "Sidekiq extension that adds the ability to debounce job execution"
14
+ gem.description = <<~DESCRIPTION
15
+ Worker will postpone its execution after `wait time` have elapsed since the last time it was invoked.
16
+ Useful for implementing behavior that should only happen after the input has stopped arriving.
17
+ DESCRIPTION
18
+ gem.homepage = "https://github.com/paladinsoftware/sidekiq-debouncer"
19
+ gem.license = "MIT"
20
+ gem.required_ruby_version = ">= 2.7.0"
21
+
22
+ gem.files = Dir.glob("lib/**/*") + [
23
+ "CHANGELOG.md",
24
+ "LICENSE.txt",
25
+ "README.md",
26
+ "sidekiq-debouncer.gemspec"
27
+ ]
28
+
29
+ gem.add_dependency "sidekiq", ">= 6.5", "< 8.0"
30
+
31
+ gem.add_development_dependency "rspec", "~> 3.12.0"
32
+ gem.add_development_dependency "timecop", "~> 0.9.6"
33
+ gem.add_development_dependency "simplecov", "~> 0.22.0"
34
+ gem.add_development_dependency "parallel", "~> 1.22.1"
35
+ gem.add_development_dependency "standard", "~> 1.24.3"
36
+ end
metadata CHANGED
@@ -1,141 +1,132 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sidekiq-debouncer
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.1
4
+ version: 2.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Sebastian Zuchmański
8
8
  - Karol Bąk
9
- autorequire:
9
+ autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2019-10-25 00:00:00.000000000 Z
12
+ date: 2023-02-28 00:00:00.000000000 Z
13
13
  dependencies:
14
- - !ruby/object:Gem::Dependency
15
- name: activesupport
16
- requirement: !ruby/object:Gem::Requirement
17
- requirements:
18
- - - ">="
19
- - !ruby/object:Gem::Version
20
- version: '0'
21
- type: :runtime
22
- prerelease: false
23
- version_requirements: !ruby/object:Gem::Requirement
24
- requirements:
25
- - - ">="
26
- - !ruby/object:Gem::Version
27
- version: '0'
28
14
  - !ruby/object:Gem::Dependency
29
15
  name: sidekiq
30
16
  requirement: !ruby/object:Gem::Requirement
31
17
  requirements:
32
18
  - - ">="
33
19
  - !ruby/object:Gem::Version
34
- version: '5.0'
20
+ version: '6.5'
21
+ - - "<"
22
+ - !ruby/object:Gem::Version
23
+ version: '8.0'
35
24
  type: :runtime
36
25
  prerelease: false
37
26
  version_requirements: !ruby/object:Gem::Requirement
38
27
  requirements:
39
28
  - - ">="
40
29
  - !ruby/object:Gem::Version
41
- version: '5.0'
30
+ version: '6.5'
31
+ - - "<"
32
+ - !ruby/object:Gem::Version
33
+ version: '8.0'
42
34
  - !ruby/object:Gem::Dependency
43
35
  name: rspec
44
36
  requirement: !ruby/object:Gem::Requirement
45
37
  requirements:
46
- - - ">="
38
+ - - "~>"
47
39
  - !ruby/object:Gem::Version
48
- version: '0'
40
+ version: 3.12.0
49
41
  type: :development
50
42
  prerelease: false
51
43
  version_requirements: !ruby/object:Gem::Requirement
52
44
  requirements:
53
- - - ">="
45
+ - - "~>"
54
46
  - !ruby/object:Gem::Version
55
- version: '0'
47
+ version: 3.12.0
56
48
  - !ruby/object:Gem::Dependency
57
- name: rspec-sidekiq
49
+ name: timecop
58
50
  requirement: !ruby/object:Gem::Requirement
59
51
  requirements:
60
- - - ">="
52
+ - - "~>"
61
53
  - !ruby/object:Gem::Version
62
- version: '0'
54
+ version: 0.9.6
63
55
  type: :development
64
56
  prerelease: false
65
57
  version_requirements: !ruby/object:Gem::Requirement
66
58
  requirements:
67
- - - ">="
59
+ - - "~>"
68
60
  - !ruby/object:Gem::Version
69
- version: '0'
61
+ version: 0.9.6
70
62
  - !ruby/object:Gem::Dependency
71
- name: timecop
63
+ name: simplecov
72
64
  requirement: !ruby/object:Gem::Requirement
73
65
  requirements:
74
- - - ">="
66
+ - - "~>"
75
67
  - !ruby/object:Gem::Version
76
- version: '0'
68
+ version: 0.22.0
77
69
  type: :development
78
70
  prerelease: false
79
71
  version_requirements: !ruby/object:Gem::Requirement
80
72
  requirements:
81
- - - ">="
73
+ - - "~>"
82
74
  - !ruby/object:Gem::Version
83
- version: '0'
75
+ version: 0.22.0
84
76
  - !ruby/object:Gem::Dependency
85
- name: rspec-redis_helper
77
+ name: parallel
86
78
  requirement: !ruby/object:Gem::Requirement
87
79
  requirements:
88
- - - ">="
80
+ - - "~>"
89
81
  - !ruby/object:Gem::Version
90
- version: '0'
82
+ version: 1.22.1
91
83
  type: :development
92
84
  prerelease: false
93
85
  version_requirements: !ruby/object:Gem::Requirement
94
86
  requirements:
95
- - - ">="
87
+ - - "~>"
96
88
  - !ruby/object:Gem::Version
97
- version: '0'
89
+ version: 1.22.1
98
90
  - !ruby/object:Gem::Dependency
99
- name: redis-namespace
91
+ name: standard
100
92
  requirement: !ruby/object:Gem::Requirement
101
93
  requirements:
102
- - - ">="
94
+ - - "~>"
103
95
  - !ruby/object:Gem::Version
104
- version: '0'
96
+ version: 1.24.3
105
97
  type: :development
106
98
  prerelease: false
107
99
  version_requirements: !ruby/object:Gem::Requirement
108
100
  requirements:
109
- - - ">="
101
+ - - "~>"
110
102
  - !ruby/object:Gem::Version
111
- version: '0'
112
- description: Sidekiq extension that adds the ability to debounce job execution.
103
+ version: 1.24.3
104
+ description: |
105
+ Worker will postpone its execution after `wait time` have elapsed since the last time it was invoked.
106
+ Useful for implementing behavior that should only happen after the input has stopped arriving.
113
107
  email:
114
108
  - sebcioz@gmail.com
115
- - karol.bak@paladinsoftware.com
109
+ - kukicola@gmail.com
116
110
  executables: []
117
111
  extensions: []
118
112
  extra_rdoc_files: []
119
113
  files:
120
- - ".gitignore"
121
- - ".semaphore/semaphore.yml"
122
- - Gemfile
114
+ - CHANGELOG.md
123
115
  - LICENSE.txt
124
116
  - README.md
125
- - gemfiles/sidekiq-5.0.5.gemfile
126
- - gemfiles/sidekiq-5.1.3.gemfile
127
- - gemfiles/sidekiq-5.2.7.gemfile
128
- - gemfiles/sidekiq-6.0.2.gemfile
117
+ - lib/sidekiq-debouncer.rb
129
118
  - lib/sidekiq/debouncer.rb
119
+ - lib/sidekiq/debouncer/errors.rb
120
+ - lib/sidekiq/debouncer/lua/debounce.lua
121
+ - lib/sidekiq/debouncer/middleware/client.rb
122
+ - lib/sidekiq/debouncer/middleware/server.rb
130
123
  - lib/sidekiq/debouncer/version.rb
131
- - sidekiq-debounce.gemspec
132
- - spec/sidekiq/debouncer_spec.rb
133
- - spec/spec_helper.rb
124
+ - sidekiq-debouncer.gemspec
134
125
  homepage: https://github.com/paladinsoftware/sidekiq-debouncer
135
126
  licenses:
136
127
  - MIT
137
128
  metadata: {}
138
- post_install_message:
129
+ post_install_message:
139
130
  rdoc_options: []
140
131
  require_paths:
141
132
  - lib
@@ -143,17 +134,15 @@ required_ruby_version: !ruby/object:Gem::Requirement
143
134
  requirements:
144
135
  - - ">="
145
136
  - !ruby/object:Gem::Version
146
- version: '0'
137
+ version: 2.7.0
147
138
  required_rubygems_version: !ruby/object:Gem::Requirement
148
139
  requirements:
149
140
  - - ">="
150
141
  - !ruby/object:Gem::Version
151
142
  version: '0'
152
143
  requirements: []
153
- rubygems_version: 3.0.6
154
- signing_key:
144
+ rubygems_version: 3.3.26
145
+ signing_key:
155
146
  specification_version: 4
156
- summary: ''
157
- test_files:
158
- - spec/sidekiq/debouncer_spec.rb
159
- - spec/spec_helper.rb
147
+ summary: Sidekiq extension that adds the ability to debounce job execution
148
+ test_files: []
data/.gitignore DELETED
@@ -1,19 +0,0 @@
1
- *.gem
2
- *.rbc
3
- .bundle
4
- .config
5
- .rspec-local
6
- .yardoc
7
- Gemfile.lock
8
- InstalledFiles
9
- _yardoc
10
- coverage
11
- doc/
12
- gemfiles/*.gemfile.lock
13
- lib/bundler/man
14
- pkg
15
- rdoc
16
- spec/reports
17
- test/tmp
18
- test/version_tmp
19
- tmp
@@ -1,27 +0,0 @@
1
- version: v1.0
2
- name: Sidekiq-debouncer CI pipeline
3
- agent:
4
- machine:
5
- type: e1-standard-2
6
- os_image: ubuntu1804
7
-
8
- blocks:
9
- - name: Run specs
10
- task:
11
- env_vars:
12
- - name: RAILS_ENV
13
- value: "test"
14
-
15
- jobs:
16
- - name: Run RSpec
17
- matrix:
18
- - env_var: SIDEKIQ_VERSION
19
- values: [ "5.0.5", "5.1.3", "5.2.7", "6.0.2" ]
20
- commands:
21
- - checkout
22
- - export RUBY_VERSION=$(cat gemfiles/sidekiq-$SIDEKIQ_VERSION.gemfile | grep "ruby \".*\"" | grep -o "[0-9]\.[0-9]\.[0-9]")
23
- - sem-version ruby $RUBY_VERSION
24
- - sem-service start redis
25
- - gem install bundler -v 1.17.3 -N
26
- - bundle install --gemfile gemfiles/sidekiq-$SIDEKIQ_VERSION.gemfile
27
- - RAILS_ENV=test bundle exec --gemfile gemfiles/sidekiq-$SIDEKIQ_VERSION.gemfile rspec
data/Gemfile DELETED
@@ -1,3 +0,0 @@
1
- source 'https://rubygems.org'
2
-
3
- gemspec
@@ -1,7 +0,0 @@
1
- source 'https://rubygems.org'
2
-
3
- ruby "2.3.8"
4
-
5
- gem "sidekiq", "= 5.0.5"
6
-
7
- gemspec path: '../'
@@ -1,7 +0,0 @@
1
- source 'https://rubygems.org'
2
-
3
- ruby "2.4.6"
4
-
5
- gem "sidekiq", "= 5.1.3"
6
-
7
- gemspec path: '../'
@@ -1,7 +0,0 @@
1
- source 'https://rubygems.org'
2
-
3
- ruby "2.5.5"
4
-
5
- gem "sidekiq", "= 5.2.7"
6
-
7
- gemspec path: '../'
@@ -1,7 +0,0 @@
1
- source 'https://rubygems.org'
2
-
3
- ruby "2.6.5"
4
-
5
- gem "sidekiq", "= 6.0.2"
6
-
7
- gemspec path: '../'
@@ -1,28 +0,0 @@
1
- # -*- encoding: utf-8 -*-
2
- lib = File.expand_path('../lib', __FILE__)
3
- $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
- require 'sidekiq/debouncer/version'
5
-
6
- Gem::Specification.new do |gem|
7
- gem.name = 'sidekiq-debouncer'
8
- gem.version = Sidekiq::Debouncer::VERSION
9
- gem.authors = ['Sebastian Zuchmański', 'Karol Bąk']
10
- gem.email = ['sebcioz@gmail.com', 'karol.bak@paladinsoftware.com']
11
- gem.description = %q{Sidekiq extension that adds the ability to debounce job execution.}
12
- gem.summary = %q{}
13
- gem.homepage = 'https://github.com/paladinsoftware/sidekiq-debouncer'
14
- gem.license = 'MIT'
15
-
16
- gem.files = `git ls-files`.split($/)
17
- gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
18
- gem.require_paths = %w(lib)
19
-
20
- gem.add_dependency 'activesupport'
21
- gem.add_dependency 'sidekiq', '>= 5.0'
22
-
23
- gem.add_development_dependency 'rspec'
24
- gem.add_development_dependency 'rspec-sidekiq'
25
- gem.add_development_dependency 'timecop'
26
- gem.add_development_dependency 'rspec-redis_helper'
27
- gem.add_development_dependency 'redis-namespace'
28
- end
@@ -1,202 +0,0 @@
1
- require 'spec_helper'
2
-
3
- class TestWorker
4
- include Sidekiq::Worker
5
- include Sidekiq::Debouncer
6
-
7
- sidekiq_options(
8
- debounce: {
9
- time: 5 * 60,
10
- by: -> (job_args) {
11
- job_args[0]
12
- }
13
- }
14
- )
15
-
16
- # group - array of arguments for single job
17
- def perform(group)
18
- group.each do
19
- # do some work with group
20
- end
21
- end
22
- end
23
-
24
- class TestWorkerWithMultipleArguments
25
- include Sidekiq::Worker
26
- include Sidekiq::Debouncer
27
-
28
- sidekiq_options(
29
- debounce: {
30
- time: 5 * 60,
31
- by: -> (job_args) {
32
- job_args[0] + job_args[1]
33
- }
34
- }
35
- )
36
-
37
- # group - array of arguments for single job
38
- def perform(group)
39
- group.each do
40
- # do some work with group
41
- end
42
- end
43
- end
44
-
45
- class TestWorkerWithSymbolAsDebounce
46
- include Sidekiq::Worker
47
- include Sidekiq::Debouncer
48
-
49
- sidekiq_options(
50
- debounce: {
51
- time: 5 * 60,
52
- by: :debounce_method
53
- }
54
- )
55
-
56
- def self.debounce_method(args)
57
- args[0]
58
- end
59
-
60
- # group - array of arguments for single job
61
- def perform(group)
62
- group.each do
63
- # do some work with group
64
- end
65
- end
66
- end
67
-
68
- describe Sidekiq::Debouncer do
69
- let(:time_start) { Time.new(2016, 1, 1, 12, 0, 0) }
70
-
71
- before do
72
- Timecop.freeze(time_start)
73
- Sidekiq::ScheduledSet.new.clear
74
- Sidekiq::Testing.disable!
75
- end
76
-
77
- context "1 type of tasks" do
78
- context "1 task" do
79
- it "executes it after 5 minutes" do
80
- TestWorker.debounce("A", "job 1")
81
-
82
- expect(Sidekiq::ScheduledSet.new.size).to eq(1)
83
-
84
- group = Sidekiq::ScheduledSet.new.first
85
- expect(group.args[0]).to eq([["A", "job 1"]])
86
- expect(group.at.to_i).to eq((time_start + 5 * 60).to_i)
87
- end
88
-
89
- it "executes it after 5 minutes for symbol debounce" do
90
- expect(TestWorkerWithSymbolAsDebounce).to receive(:debounce_method).with(["A", "job 1"]).once.and_call_original
91
- TestWorkerWithSymbolAsDebounce.debounce("A", "job 1")
92
-
93
- expect(Sidekiq::ScheduledSet.new.size).to eq(1)
94
-
95
- group = Sidekiq::ScheduledSet.new.first
96
- expect(group.args[0]).to eq([["A", "job 1"]])
97
- expect(group.at.to_i).to eq((time_start + 5 * 60).to_i)
98
- end
99
- end
100
-
101
- context "1 task, 3 minutes break, 1 task" do
102
- it "executes both tasks after 8 minutes for multiple arguments" do
103
- TestWorkerWithMultipleArguments.debounce(1, 5)
104
- Timecop.freeze(time_start + 3 * 60)
105
- TestWorkerWithMultipleArguments.debounce(3, 3)
106
-
107
- expect(Sidekiq::ScheduledSet.new.size).to eq(1)
108
- group = Sidekiq::ScheduledSet.new.first
109
- expect(group.args[0]).to eq([[1, 5], [3, 3]])
110
- expect(group.at.to_i).to be((time_start + 8 * 60).to_i)
111
- end
112
-
113
- it "executes both tasks after 8 minutes" do
114
- TestWorker.debounce("A", "job 1")
115
- Timecop.freeze(time_start + 3 * 60)
116
- TestWorker.debounce("A", "job 2")
117
-
118
- expect(Sidekiq::ScheduledSet.new.size).to eq(1)
119
- group = Sidekiq::ScheduledSet.new.first
120
- expect(group.args[0]).to eq([["A", "job 1"], ["A", "job 2"]])
121
- expect(group.at.to_i).to be((time_start + 8 * 60).to_i)
122
- end
123
-
124
- it "executes both tasks after 8 minutes for symbol debounce" do
125
- TestWorkerWithSymbolAsDebounce.debounce("A", "job 1")
126
- Timecop.freeze(time_start + 3 * 60)
127
- TestWorkerWithSymbolAsDebounce.debounce("A", "job 2")
128
-
129
- expect(Sidekiq::ScheduledSet.new.size).to eq(1)
130
- group = Sidekiq::ScheduledSet.new.first
131
- expect(group.args[0]).to eq([["A", "job 1"], ["A", "job 2"]])
132
- expect(group.at.to_i).to be((time_start + 8 * 60).to_i)
133
- end
134
- end
135
-
136
- context "1 task, 3 minutes break, 1 task, 6 minutes break, 1 task" do
137
- it "executes two tasks after 8 minutes, the last one in 14 minutes" do
138
- TestWorker.debounce("A", "job 1")
139
- Timecop.freeze(time_start + 3 * 60)
140
- TestWorker.debounce("A", "job 2")
141
- Timecop.freeze(time_start + 3 * 60 + 6 * 60)
142
- TestWorker.debounce("A", "job 3")
143
-
144
- expect(Sidekiq::ScheduledSet.new.size).to eq(2)
145
-
146
- groups = Sidekiq::ScheduledSet.new.to_a.sort_by { |g| g.at.to_i }
147
-
148
- expect(groups[0].args[0]).to eq([["A", "job 1"], ["A", "job 2"]])
149
- expect(groups[0].at.to_i).to be((time_start + 8 * 60).to_i)
150
-
151
- expect(groups[1].args[0]).to eq([["A", "job 3"]])
152
- expect(groups[1].at.to_i).to be((time_start + 14 * 60).to_i)
153
- end
154
- end
155
-
156
- context "1 task, 6 minutes break, 1 task" do
157
- it "executes first task, the second one in 11 minutes" do
158
- TestWorker.debounce("A", "job 1")
159
- Timecop.freeze(time_start + 6 * 60)
160
- TestWorker.debounce("A", "job 2")
161
-
162
- expect(Sidekiq::ScheduledSet.new.size).to eq(2)
163
-
164
- groups = Sidekiq::ScheduledSet.new.to_a.sort_by { |g| g.at.to_i }
165
-
166
- expect(groups[0].args[0]).to eq([["A", "job 1"]])
167
- expect(groups[0].at.to_i).to be((time_start + 5 * 60).to_i)
168
-
169
- expect(groups[1].args[0]).to eq([["A", "job 2"]])
170
- expect(groups[1].at.to_i).to be((time_start + 11 * 60).to_i)
171
- end
172
- end
173
- end
174
-
175
- context "two types of tasks" do
176
- context "A task, 3 minutes break, A and B tasks, 3 minutes break, B task, 6 minutes break, B task" do
177
- it "executes two A tasks after 8 minuts, two B tasks after 11 minutes, last B task after 17 minutes" do
178
- TestWorker.debounce("A", "job 1")
179
- Timecop.freeze(time_start + 3 * 60)
180
- TestWorker.debounce("A", "job 2")
181
- TestWorker.debounce("B", "job 3")
182
- Timecop.freeze(time_start + 3 * 60 + 3 * 60)
183
- TestWorker.debounce("B", "job 4")
184
- Timecop.freeze(time_start + 3 * 60 + 3 * 60 + 6 * 60)
185
- TestWorker.debounce("B", "job 5")
186
-
187
- expect(Sidekiq::ScheduledSet.new.size).to eq(3)
188
-
189
- groups = Sidekiq::ScheduledSet.new.to_a.sort_by { |g| g.at.to_i }
190
-
191
- expect(groups[0].args[0]).to eq([["A", "job 1"], ["A", "job 2"]])
192
- expect(groups[0].at.to_i).to be((time_start + 8 * 60).to_i)
193
-
194
- expect(groups[1].args[0]).to eq([["B", "job 3"], ["B", "job 4"]])
195
- expect(groups[1].at.to_i).to be((time_start + 11 * 60).to_i)
196
-
197
- expect(groups[2].args[0]).to eq([["B", "job 5"]])
198
- expect(groups[2].at.to_i).to be((time_start + 17 * 60).to_i)
199
- end
200
- end
201
- end
202
- end
data/spec/spec_helper.rb DELETED
@@ -1,18 +0,0 @@
1
- $LOAD_PATH.unshift(File.dirname(__FILE__))
2
- $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
3
-
4
- require 'rspec'
5
- require 'redis/namespace'
6
- require 'rspec-sidekiq'
7
- require 'timecop'
8
-
9
- require 'sidekiq/debouncer'
10
-
11
- RSpec.configure do |config|
12
- config.order = 'random'
13
- config.color = true
14
-
15
- Sidekiq.configure_server do |config|
16
- config.redis[:namespace] = { namespace: 'sidekiq-debouncer' }
17
- end
18
- end