freekiqs 4.1.0 → 6.5.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 0a7ee677e8d100c7405dc91f9d4c341d7f7a8986
4
- data.tar.gz: 0281092a960af50ca75ed3b03370bd7a86b1ae17
2
+ SHA256:
3
+ metadata.gz: d6b06dcd42bf69b36943cb98073500edaa179251c807284ac1520788f4f9b868
4
+ data.tar.gz: 7d273e8393c7f4ec61b2bc3488daaa3a8e7c4d2c3c4e323d0787f3217a1ec772
5
5
  SHA512:
6
- metadata.gz: 35984aad9ab4d0fec1d33f07469ba0f82ff83a5b3cdeaecb39f1bbdb42f56ec9566df45c8735970034f99294919790481983047141a89750118845d46d7a3ad9
7
- data.tar.gz: d932c98cefbdcc672d00c2e85ae16a86b6b4b70cb719403a462b8981dacd22fca4f0cbaa7cf90735879cbd8cbc8efe3a5f221eaa9d8cd033962e0efd841dbaf4
6
+ metadata.gz: beeb49dbe17a46885ab99ff7c2128645fb739496a7636a2a0b8f041c7b1099dfd42e291f23d15bf49b0ff4b1da09d01da8a8f8d2cb9ed482db09982143d64d65
7
+ data.tar.gz: 1ce3a30d1d569967f3c0b3fe6941baf00cfaa6c99da09436e87481b29e8dfa1412cef2ede6853ae4c9e6099aa547149b01c943ac613891cb7b39d0020c7b44e1
data/.gitignore CHANGED
@@ -3,3 +3,8 @@ gemfiles/*.lock
3
3
  Gemfile.lock
4
4
  .ruby-version
5
5
  pkg
6
+ /.bundle
7
+ /.vagrant
8
+ /ubuntu-*-console.log
9
+ /vendor
10
+ dump.rdb
data/.rspec ADDED
@@ -0,0 +1 @@
1
+ --require spec_helper
data/Gemfile CHANGED
@@ -5,5 +5,5 @@ gemspec
5
5
  gem 'rake'
6
6
 
7
7
  group :test do
8
- gem 'rspec', '~> 3.1'
8
+ gem 'rspec', '~> 3.6'
9
9
  end
data/Procfile ADDED
@@ -0,0 +1 @@
1
+ redis: redis-server development/redis/redis.conf --bind $HOST
data/README.md CHANGED
@@ -3,13 +3,28 @@
3
3
  Sidekiq middleware that allows capturing exceptions thrown
4
4
  by failed jobs and wrapping them with a `FreekiqException` exception class
5
5
  that can be filtered by monitoring tools such as New Relic and
6
- Honeybadger.
6
+ Rollbar.
7
+
8
+ #### Implementation Details
9
+
10
+ This relies on Sidekiq's built-in retry handling. Specifically, its
11
+ `retry_count` value. When a job first fails, its `retry_count` value
12
+ is nil (because it hasn't actually been retried yet). That exception,
13
+ along with subsequent exceptions from retries, are caught and wrapped
14
+ with the `FreekiqException` exception.
15
+ Cases where Freekiqs does NOT wrap the exception:
16
+ - The `retry` option is false
17
+ - The `freekiqs` option is not set on the worker nor globally
18
+ - The `freekiqs` option is not set on worker and set to `false` globally
19
+ - The `freekiqs` option is set to `false` on the worker
20
+ - The job threw an exception that is not a StandardError (nor a subclass)
21
+ - The number of thrown exceptions is more than specified freekiqs
7
22
 
8
23
  Configuration example (in config/initializers/sidekiq.rb):
9
24
  ``` ruby
10
25
  Sidekiq.configure_server do |config|
11
26
  config.server_middleware do |chain|
12
- chain.insert_after Sidekiq::Middleware::Server::RetryJobs, Sidekiq::Middleware::Server::Freekiqs
27
+ chain.add Sidekiq::Middleware::Server::Freekiqs
13
28
  end
14
29
  end
15
30
  ```
@@ -19,7 +34,7 @@ Worker example:
19
34
  class MyWorker
20
35
  include Sidekiq::Worker
21
36
  sidekiq_options freekiqs: 3
22
- #
37
+
23
38
  def perform(param)
24
39
  ...
25
40
  end
@@ -35,7 +50,7 @@ Example:
35
50
  ``` ruby
36
51
  Sidekiq.configure_server do |config|
37
52
  config.server_middleware do |chain|
38
- chain.insert_after Sidekiq::Middleware::Server::RetryJobs, Sidekiq::Middleware::Server::Freekiqs, freekiqs: 3
53
+ chain.add Sidekiq::Middleware::Server::Freekiqs, freekiqs: 3
39
54
  end
40
55
  end
41
56
  ```
@@ -44,6 +59,17 @@ A callback can be fired when a freekiq happens.
44
59
  This can be useful for tracking or logging freekiqs separately from the sidekiq logs.
45
60
 
46
61
  Example:
62
+ ``` ruby
63
+ Sidekiq.configure_server do |config|
64
+ config.server_middleware do |chain|
65
+ chain.add Sidekiq::Middleware::Server::Freekiqs, callback: ->(worker, msg, queue) do
66
+ Librato::Metrics.submit freekiqs: { value: 1, source: worker.class.name }
67
+ end
68
+ end
69
+ end
70
+ ```
71
+
72
+ Or callback can be set outside middleware configuration:
47
73
  ``` ruby
48
74
  Sidekiq::Middleware::Server::Freekiqs::callback = ->(worker, msg, queue) do
49
75
  Librato::Metrics.submit freekiqs: { value: 1, source: worker.class.name }
@@ -57,7 +83,7 @@ Example:
57
83
  class MyWorker
58
84
  include Sidekiq::Worker
59
85
  sidekiq_options freekiqs: 1, freekiq_for: ['MyError']
60
- #
86
+
61
87
  def perform(param)
62
88
  ...
63
89
  end
@@ -67,8 +93,8 @@ In this case, if MyWorker fails with a MyError it will get 1 freekiq.
67
93
  All other errors thrown by this worker will get no freekiqs.
68
94
 
69
95
 
70
- If a `freekiq_for` contains a class name as a constant, any exception is that class
71
- *or a subclass* of that class will get freekiq'd.
96
+ If a `freekiq_for` contains a class name as a constant, any exception of that class
97
+ type *or a subclass* of that class will get freekiq'd.
72
98
 
73
99
  Example:
74
100
  ``` ruby
@@ -77,7 +103,7 @@ Example:
77
103
  class MyWorker
78
104
  include Sidekiq::Worker
79
105
  sidekiq_options freekiqs: 1, freekiq_for: [MyError]
80
- #
106
+
81
107
  def perform(param)
82
108
  ...
83
109
  end
@@ -85,26 +111,9 @@ Example:
85
111
  ```
86
112
  If MyWorker throws a SubMyError or MyError, it will get freekiq'd.
87
113
 
114
+ ## Sidekiq Versions
88
115
 
89
- ## Overview
116
+ Version 5 of this gem only works with Sidekiq 5 and higher. If you are using
117
+ an older version of Sidekiq, you'll need to use version [4.1.0](https://github.com/BookBub/freekiqs/tree/v4.1.0).
90
118
 
91
- Up to the configured number of "freekiqs", catch exceptions thrown
92
- from job and wrap them with the `FreekiqException` exception (which is a
93
- `RuntimeError`). That exception type can be ignored by monitoring
94
- tools. If job fails more times than configured "freekiqs", thrown
95
- exception will not be wrapped. That should result in normal operation
96
- and the exception showing up in monitoring tools.
97
-
98
- Implementation Details:
99
- This relies on Sidekiq's built-in retry handling. Specifically, its
100
- `retry_count` value. When a job first fails, its `retry_count` value
101
- is nil (because it hasn't actually been retried yet). That exception,
102
- along with subsequent exceptions from retries, are caught and wrapped
103
- with the `FreekiqException` exception.
104
- Cases where Freekiqs does NOT wrap the exception:
105
- - The `retry` option is false
106
- - The `freekiqs` option is not set on the worker nor globally
107
- - The `freekiqs` option is not set on worker and set to `false` globally
108
- - The `freekiqs` option is set to `false` on the worker
109
- - The job threw an exception that is not a StandardError (nor a subclass)
110
- - The number of thrown exceptions is more than specified freekiqs
119
+ Version 6.5.0 of this gem works with Sidekiq 6.5 and higher.
data/Vagrantfile ADDED
@@ -0,0 +1,31 @@
1
+ Vagrant.configure(2) do |config|
2
+ config.vm.box = 'ubuntu/xenial64'
3
+
4
+ config.vm.network :private_network, ip: '192.168.33.69'
5
+ config.vm.network :forwarded_port, host: 2202, guest: 22, id: 'ssh', auto_correct: true
6
+
7
+ config.vm.synced_folder '.', '/vagrant', type: 'nfs'
8
+
9
+ config.vm.provider :virtualbox do |vb|
10
+ vb.name = 'freekiqs'
11
+ vb.memory = 2048
12
+ vb.cpus = 2
13
+ end
14
+
15
+ config.vm.provision :shell, inline: <<-SCRIPT
16
+ apt-get update
17
+ apt-get upgrade
18
+ apt-get install -y git gnupg2 redis-server
19
+ SCRIPT
20
+
21
+ config.vm.provision :shell, privileged: false, inline: <<-SCRIPT
22
+ gpg --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
23
+ curl -sSL https://get.rvm.io | bash -s stable --quiet-curl --with-gems=bundler
24
+ source "$HOME/.rvm/scripts/rvm"
25
+ rvm --quiet-curl install 2.3.4
26
+ rvm use 2.3.4 --default
27
+ cd /vagrant
28
+ bundle install --full-index -j4
29
+ echo 'cd /vagrant' >> ~/.bash_profile
30
+ SCRIPT
31
+ end
data/circle.yml CHANGED
@@ -1,5 +1,5 @@
1
1
  machine:
2
2
  ruby:
3
- version: 2.3.0
3
+ version: 2.3.4
4
4
  services:
5
5
  - redis
@@ -0,0 +1,49 @@
1
+ protected-mode yes
2
+ port 6379
3
+ tcp-backlog 511
4
+ timeout 0
5
+ tcp-keepalive 300
6
+ daemonize no
7
+ supervised no
8
+ pidfile development/redis/redis_6379.pid
9
+ loglevel warning
10
+ logfile ""
11
+ databases 16
12
+ save 900 1
13
+ save 300 10
14
+ save 60 10000
15
+ stop-writes-on-bgsave-error yes
16
+ rdbcompression yes
17
+ rdbchecksum yes
18
+ dbfilename dump.rdb
19
+ dir development/redis
20
+ slave-serve-stale-data yes
21
+ slave-read-only yes
22
+ repl-diskless-sync no
23
+ repl-diskless-sync-delay 5
24
+ repl-disable-tcp-nodelay no
25
+ slave-priority 100
26
+ appendonly no
27
+ appendfilename "appendonly.aof"
28
+ appendfsync no
29
+ no-appendfsync-on-rewrite no
30
+ auto-aof-rewrite-percentage 100
31
+ auto-aof-rewrite-min-size 64mb
32
+ aof-load-truncated yes
33
+ lua-time-limit 5000
34
+ latency-monitor-threshold 0
35
+ notify-keyspace-events ""
36
+ hash-max-ziplist-entries 512
37
+ hash-max-ziplist-value 64
38
+ list-max-ziplist-size -2
39
+ list-compress-depth 0
40
+ set-max-intset-entries 512
41
+ zset-max-ziplist-entries 128
42
+ zset-max-ziplist-value 64
43
+ hll-sparse-max-bytes 3000
44
+ activerehashing yes
45
+ client-output-buffer-limit normal 0 0 0
46
+ client-output-buffer-limit slave 256mb 64mb 60
47
+ client-output-buffer-limit pubsub 32mb 8mb 60
48
+ hz 10
49
+ aof-rewrite-incremental-fsync yes
data/freekiqs.gemspec CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  Gem::Specification.new do |gem|
4
4
  gem.name = 'freekiqs'
5
- gem.version = '4.1.0'
5
+ gem.version = '6.5.0'
6
6
  gem.authors = ['Rob Lewis']
7
7
  gem.email = ['rob@bookbub.com']
8
8
  gem.summary = 'Sidekiq middleware extending RetryJobs to allow silient errors.'
@@ -14,8 +14,8 @@ Gem::Specification.new do |gem|
14
14
  gem.files = `git ls-files`.split("\n")
15
15
  gem.test_files = `git ls-files -- spec/*`.split("\n")
16
16
  gem.require_paths = ['lib']
17
- gem.required_ruby_version = '>= 1.9.3'
17
+ gem.required_ruby_version = '>= 2.2.2'
18
18
 
19
- gem.add_dependency 'sidekiq', '>= 4.0.0', '< 5.0.0'
20
- gem.add_development_dependency 'rspec', '>= 2.14.1'
19
+ gem.add_dependency 'sidekiq', '>= 6.5.0'
20
+ gem.add_development_dependency 'rspec', '>= 3.6.0'
21
21
  end
@@ -1,5 +1,5 @@
1
1
  require 'sidekiq'
2
- require 'sidekiq/util'
2
+ require 'sidekiq/middleware/modules'
3
3
 
4
4
  module Sidekiq
5
5
  class FreekiqException < RuntimeError; end
@@ -7,37 +7,38 @@ module Sidekiq
7
7
  module Middleware
8
8
  module Server
9
9
  class Freekiqs
10
- include Sidekiq::Util
10
+ include Sidekiq::ServerMiddleware
11
11
  @@callback = nil
12
12
 
13
13
  def initialize(opts={})
14
14
  @default_freekiqs = opts[:freekiqs]
15
15
  @default_freekiq_for = opts[:freekiq_for]
16
+ @@callback = opts[:callback]
16
17
  end
17
18
 
18
19
  def call(worker, msg, queue)
19
20
  yield
20
21
  rescue => ex
21
- freekiqs = get_freekiqs_if_enabled(worker, msg, ex)
22
+ freekiqs = get_freekiqs_if_enabled(worker, ex)
22
23
  if freekiqs
23
24
  if msg['retry_count'].nil? || msg['retry_count'] < freekiqs-1
24
25
  begin
25
26
  @@callback.call(worker, msg, queue) if @@callback
26
27
  rescue => callback_exception
27
- Sidekiq.logger.info { "Freekiq callback failed for #{msg['class']} job #{msg['jid']}" }
28
+ logger.info { "Freekiq callback failed for #{msg['class']} job #{msg['jid']}" }
28
29
  ensure
29
30
  raise FreekiqException, ex.message
30
31
  end
31
32
  else
32
- Sidekiq.logger.info { "Out of free kiqs for #{msg['class']} job #{msg['jid']}" }
33
+ logger.info { "Out of freekiqs for #{msg['class']} job #{msg['jid']}" }
33
34
  end
34
35
  end
35
36
  raise ex
36
37
  end
37
38
 
38
- def get_freekiqs_if_enabled(worker, msg, ex)
39
+ def get_freekiqs_if_enabled(worker, ex)
39
40
  freekiqs = nil
40
- if msg['retry']
41
+ if worker.class.get_sidekiq_options['retry']
41
42
  if worker.class.get_sidekiq_options['freekiqs'] != false
42
43
  errors = get_freekiq_errors(worker)
43
44
  if worker.class.get_sidekiq_options['freekiqs']
data/shell.nix ADDED
@@ -0,0 +1,42 @@
1
+ { system ? builtins.currentSystem }:
2
+
3
+ with (import (fetchTarball {
4
+ name = "nixpkgs-21.11";
5
+ url = "https://github.com/nixos/nixpkgs/archive/21.11.tar.gz";
6
+ sha256 = "162dywda2dvfj1248afxc45kcrg83appjd0nmdb541hl7rnncf02";
7
+ }) { inherit system; });
8
+
9
+
10
+ let
11
+ requiredNixVersion = "2.3";
12
+
13
+ pwd = builtins.getEnv "PWD";
14
+ in
15
+
16
+ if lib.versionOlder builtins.nixVersion requiredNixVersion == true then
17
+ abort "This project requires Nix >= ${requiredNixVersion}, please run 'nix-channel --update && nix-env -i nix'."
18
+ else
19
+
20
+ mkShell {
21
+ buildInputs = [
22
+ stdenv
23
+ git
24
+ cacert
25
+
26
+ # Ruby and Rails dependencies
27
+ ruby_2_7.devEnv
28
+ bundler
29
+
30
+ # Services
31
+ overmind
32
+ redis
33
+
34
+ ] ++ lib.optional (!stdenv.isDarwin) [
35
+ # linux-only packages
36
+ glibcLocales
37
+ ];
38
+
39
+ HOST = "127.0.0.24";
40
+ REDIS_URL="redis://127.0.0.24:6379/1";
41
+ BUNDLE_PATH = "vendor/bundle";
42
+ }
@@ -1,329 +1,249 @@
1
- require 'spec_helper'
2
- require 'sidekiq/cli'
3
- require 'sidekiq/middleware/server/retry_jobs'
1
+ require 'sidekiq'
2
+ require 'sidekiq/api'
3
+ require 'sidekiq/processor'
4
4
 
5
- shared_examples_for 'it should have 2 freekiqs for an ArgumentError' do
6
- it 'throws Freekiq exception for specified number of free kiqs' do
7
- expect {
8
- handler.invoke(worker, job, 'default') do
9
- raise ArgumentError, 'overlooked'
10
- end
11
- }.to raise_error(Sidekiq::FreekiqException, 'overlooked')
12
- expect(job['retry_count']).to eq(0)
13
- expect(job['error_class']).to eq('Sidekiq::FreekiqException')
14
- expect(job['error_message']).to eq('overlooked')
15
- expect {
16
- handler.invoke(worker, job, 'default') do
17
- raise ArgumentError, 'overlooked'
18
- end
19
- }.to raise_error(Sidekiq::FreekiqException, 'overlooked')
20
- expect(job['retry_count']).to eq(1)
21
- expect(job['error_class']).to eq('Sidekiq::FreekiqException')
22
- expect(job['error_message']).to eq('overlooked')
23
- expect {
24
- handler.invoke(worker, job, 'default') do
25
- raise ArgumentError, 'not overlooked'
26
- end
27
- }.to raise_error(ArgumentError, 'not overlooked')
28
- expect(job['retry_count']).to eq(2)
29
- expect(job['error_class']).to eq('ArgumentError')
30
- expect(job['error_message']).to eq('not overlooked')
31
- expect(Sidekiq::RetrySet.new.size).to eq(3)
32
- expect(Sidekiq::DeadSet.new.size).to eq(0)
33
- end
34
- end
5
+ RSpec.describe Sidekiq::Middleware::Server::Freekiqs do
6
+ class NonArgumentError < StandardError; end
35
7
 
36
- shared_examples_for 'it should have 0 freekiqs for an ArgumentError' do
37
- it 'raises the original error' do
38
- expect {
39
- handler.invoke(worker, job, 'default') do
40
- raise ArgumentError, 'not overlooked'
41
- end
42
- }.to raise_error(ArgumentError, 'not overlooked')
43
- expect(job['retry_count']).to eq(0)
44
- expect(job['error_class']).to eq('ArgumentError')
45
- expect(job['error_message']).to eq('not overlooked')
46
- expect(Sidekiq::RetrySet.new.size).to eq(1)
47
- expect(Sidekiq::DeadSet.new.size).to eq(0)
8
+ def build_job_hash(worker_class, args=[])
9
+ {'class' => worker_class, 'args' => args}
48
10
  end
49
- end
50
11
 
51
- describe Sidekiq::Middleware::Server::Freekiqs do
52
- class DummyWorkerPlain
53
- include Sidekiq::Worker
12
+ def fetch_retry_job
13
+ retry_set = Sidekiq::RetrySet.new
14
+ retry_job = retry_set.first
15
+ retry_set.clear
16
+ retry_job
54
17
  end
55
- class DummyWorkerWithFreekiqsEnabled
56
- include Sidekiq::Worker
57
- sidekiq_options freekiqs: 2
18
+
19
+ def process_job(job_hash)
20
+ processor = Sidekiq::Processor.new(Sidekiq)
21
+ job_msg = Sidekiq.dump_json(job_hash)
22
+ processor.process(Sidekiq::BasicFetch::UnitOfWork.new('queue:default', job_msg))
58
23
  end
59
24
 
60
- def build_handler_chain(freekiq_options={}, retry_options={})
61
- Sidekiq::Middleware::Chain.new do |chain|
62
- chain.add Sidekiq::Middleware::Server::RetryJobs, retry_options
63
- chain.add Sidekiq::Middleware::Server::Freekiqs, freekiq_options
25
+ def initialize_middleware(middleware_opts={})
26
+ Sidekiq.server_middleware do |chain|
27
+ chain.add Sidekiq::Middleware::Server::Freekiqs, middleware_opts
64
28
  end
65
29
  end
66
30
 
67
- def build_job(options={})
68
- {'class' => 'FreekiqDummyWorker', 'args' => [], 'retry' => true}.merge(options)
31
+ def initialize_worker_class(sidekiq_opts=nil)
32
+ worker_class_name = :TestDummyWorker
33
+ Object.send(:remove_const, worker_class_name) if Object.const_defined?(worker_class_name)
34
+ klass = Class.new do
35
+ include Sidekiq::Worker
36
+ sidekiq_options sidekiq_opts if sidekiq_opts
37
+ def perform
38
+ raise ArgumentError, 'Oops'
39
+ end
40
+ end
41
+ Object.const_set(worker_class_name, klass)
69
42
  end
70
43
 
71
44
  def cleanup_redis
72
- Sidekiq::RetrySet.new.select{|job| job.klass == 'FreekiqDummyWorker'}.each(&:delete)
73
- Sidekiq::DeadSet.new.clear
45
+ Sidekiq.redis {|c| c.flushdb }
46
+ end
47
+
48
+ shared_examples_for 'it should have 2 freekiqs for an ArgumentError' do
49
+ it 'throws Freekiq exception for specified number of freekiqs' do
50
+ args ||= []
51
+ expect {
52
+ process_job(build_job_hash(worker_class, args))
53
+ }.to raise_error(Sidekiq::FreekiqException, 'Oops')
54
+ expect(Sidekiq::RetrySet.new.size).to eq(1)
55
+ retry_job = fetch_retry_job
56
+ expect(retry_job['retry_count']).to eq(0)
57
+ expect(retry_job['error_class']).to eq('Sidekiq::FreekiqException')
58
+ expect(retry_job['error_message']).to eq('Oops')
59
+
60
+ expect {
61
+ process_job(retry_job.item)
62
+ }.to raise_error(Sidekiq::FreekiqException, 'Oops')
63
+ expect(Sidekiq::RetrySet.new.size).to eq(1)
64
+ retry_job = fetch_retry_job
65
+ expect(retry_job['retry_count']).to eq(1)
66
+ expect(retry_job['error_class']).to eq('Sidekiq::FreekiqException')
67
+ expect(retry_job['error_message']).to eq('Oops')
68
+
69
+ expect {
70
+ process_job(retry_job.item)
71
+ }.to raise_error(ArgumentError, 'Oops')
72
+ expect(Sidekiq::RetrySet.new.size).to eq(1)
73
+ retry_job = fetch_retry_job
74
+ expect(retry_job['retry_count']).to eq(2)
75
+ expect(retry_job['error_class']).to eq('ArgumentError')
76
+ expect(retry_job['error_message']).to eq('Oops')
77
+ end
78
+ end
79
+
80
+ shared_examples_for 'it should have 0 freekiqs for an ArgumentError' do
81
+ it 'raises the original error' do
82
+ args ||= []
83
+ expect {
84
+ process_job(build_job_hash(worker_class, args))
85
+ }.to raise_error(ArgumentError, 'Oops')
86
+ expect(Sidekiq::RetrySet.new.size).to eq(1)
87
+ retry_job = fetch_retry_job
88
+ expect(retry_job['retry_count']).to eq(0)
89
+ expect(retry_job['error_class']).to eq('ArgumentError')
90
+ expect(retry_job['error_message']).to eq('Oops')
91
+ end
92
+ end
93
+
94
+ shared_examples_for 'it should only raise exception for an ArgumentError' do
95
+ it 'raises the original error' do
96
+ args ||= []
97
+ expect {
98
+ process_job(build_job_hash(worker_class, args))
99
+ }.to raise_error(ArgumentError, 'Oops')
100
+ expect(Sidekiq::RetrySet.new.size).to eq(0)
101
+ # Note: Sidekiq doesn't send job to morgue when retries are diabled
102
+ expect(Sidekiq::DeadSet.new.size).to eq(0)
103
+ end
74
104
  end
75
105
 
76
106
  before(:each) do
77
107
  cleanup_redis
78
108
  end
79
109
 
80
- let(:worker_plain) { DummyWorkerPlain.new }
81
- let(:worker_with_freekiqs_enabled) { DummyWorkerWithFreekiqsEnabled.new }
110
+ context 'with default middleware config' do
111
+ before(:each) do
112
+ initialize_middleware
113
+ end
82
114
 
83
- it 'requires RetryJobs to update retry_count' do
84
- handler = Sidekiq::Middleware::Server::RetryJobs.new
85
- worker = worker_plain
86
- job = build_job
87
- expect {
88
- handler.call(worker, job, 'default') do
89
- raise 'Oops'
115
+ describe 'with nothing explicitly enabled' do
116
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
117
+ let!(:worker_class) { initialize_worker_class }
90
118
  end
91
- }.to raise_error(RuntimeError)
92
- expect(job['retry_count']).to eq(0)
93
- expect {
94
- handler.call(worker, job, 'default') do
95
- raise 'Oops'
96
- end
97
- }.to raise_error(RuntimeError)
98
- expect(job['retry_count']).to eq(1)
99
- end
100
-
101
- it 'should execute a defined callback' do
102
- Sidekiq::Middleware::Server::Freekiqs::callback = ->(worker, msg, queue) do
103
- return true
104
119
  end
105
120
 
106
- handler = build_handler_chain
107
- worker = worker_with_freekiqs_enabled
108
- job = build_job
109
-
110
- expect(Sidekiq::Middleware::Server::Freekiqs::callback).to receive(:call)
111
- expect {
112
- handler.invoke(worker, job, 'default') do
113
- raise ArgumentError, 'overlooked'
121
+ describe 'with freekiqs explicitly disabled' do
122
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
123
+ let!(:worker_class) { initialize_worker_class(freekiqs: false) }
114
124
  end
115
- }.to raise_error(Sidekiq::FreekiqException, 'overlooked')
116
- end
117
-
118
- it 'should still raise FreekiqException if the callback fails' do
119
- Sidekiq::Middleware::Server::Freekiqs::callback = ->(worker, msg, queue) do
120
- raise 'callback error'
121
125
  end
122
126
 
123
- handler = build_handler_chain
124
- worker = worker_with_freekiqs_enabled
125
- job = build_job
126
-
127
- expect {
128
- handler.invoke(worker, job, 'default') do
129
- raise ArgumentError, 'overlooked'
127
+ describe 'with 2 freekiqs in the worker' do
128
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
129
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2) }
130
130
  end
131
- }.to raise_error(Sidekiq::FreekiqException, 'overlooked')
132
- end
131
+ end
133
132
 
134
- describe 'with nothing explicitly enabled' do
135
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
136
- let (:handler) { build_handler_chain }
137
- let (:job) { build_job }
138
- let (:worker) do
139
- Class.new do
140
- include Sidekiq::Worker
141
- end.new
133
+ describe 'with freekiq_for ArgumentError and 2 freekiqs in worker' do
134
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
135
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [ArgumentError]) }
142
136
  end
143
137
  end
144
- end
145
138
 
146
- describe 'with freekiqs explicitly disabled' do
147
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
148
- let (:handler) { build_handler_chain(freekiqs: 3) }
149
- let (:job) { build_job }
150
- let (:worker) do
151
- Class.new do
152
- include Sidekiq::Worker
153
- sidekiq_options freekiqs: false
154
- end.new
139
+ describe 'with freekiq_for ArgumentError in worker and no freekiqs' do
140
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
141
+ let!(:worker_class) { initialize_worker_class(freekiqs: false, freekiq_for: [ArgumentError]) }
155
142
  end
156
143
  end
157
- end
158
144
 
159
- describe 'with 2 freekiqs in the initializer' do
160
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
161
- let (:handler) { build_handler_chain(freekiqs: 2) }
162
- let (:job) { build_job }
163
- let (:worker) do
164
- Class.new do
165
- include Sidekiq::Worker
166
- end.new
145
+ describe 'with freekiq_for NonArgumentError in worker and 2 freekiqs in worker' do
146
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
147
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [NonArgumentError]) }
167
148
  end
168
149
  end
169
- end
170
150
 
171
- describe 'with 2 freekiqs in the worker' do
172
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
173
- let (:handler) { build_handler_chain }
174
- let (:job) { build_job }
175
- let (:worker) do
176
- Class.new do
177
- include Sidekiq::Worker
178
- sidekiq_options freekiqs: 2
179
- end.new
151
+ describe 'with freekiq_for ArgumentError as a string' do
152
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
153
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: ['ArgumentError']) }
180
154
  end
181
155
  end
182
- end
183
156
 
184
- describe 'with freekiq_for ArgumentError and 2 freekiqs in worker' do
185
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
186
- let (:handler) { build_handler_chain }
187
- let (:job) { build_job }
188
- let (:worker) do
189
- Class.new do
190
- include Sidekiq::Worker
191
- sidekiq_options freekiqs: 2, freekiq_for: [ArgumentError]
192
- end.new
157
+ describe 'with freekiq_for the super class of ArgumentError' do
158
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
159
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [ArgumentError.superclass]) }
193
160
  end
194
161
  end
195
- end
196
162
 
197
- describe 'with freekiq_for ArgumentError in worker and no freekiqs' do
198
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
199
- let (:handler) { build_handler_chain }
200
- let (:job) { build_job }
201
- let (:worker) do
202
- Class.new do
203
- include Sidekiq::Worker
204
- class NonArgumentError < StandardError; end
205
- sidekiq_options freekiqs: 2, freekiq_for: [NonArgumentError]
206
- end.new
163
+ describe 'with 2 freekiqs in the worker and retries disabled' do
164
+ it_behaves_like 'it should only raise exception for an ArgumentError' do
165
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, retry: false) }
207
166
  end
208
167
  end
209
168
  end
210
169
 
211
- describe 'with freekiq_for ArgumentError in worker and 2 freekiqs in initializer' do
212
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
213
- let (:handler) { build_handler_chain(freekiqs: 2) }
214
- let (:job) { build_job }
215
- let (:worker) do
216
- Class.new do
217
- include Sidekiq::Worker
218
- sidekiq_options freekiq_for: [ArgumentError]
219
- end.new
220
- end
170
+ context 'with middleware configured with 2 freekiqs' do
171
+ before(:each) do
172
+ initialize_middleware(freekiqs: 2)
221
173
  end
222
- end
223
174
 
224
- describe 'with freekiq_for ArgumentError in initializer and 2 freekiqs in initializer' do
225
175
  it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
226
- let (:handler) { build_handler_chain(freekiqs: 2, freekiq_for: [ArgumentError]) }
227
- let (:job) { build_job }
228
- let (:worker) do
229
- Class.new do
230
- include Sidekiq::Worker
231
- end.new
232
- end
176
+ let!(:worker_class) { initialize_worker_class }
233
177
  end
234
- end
235
178
 
236
- describe 'with freekiq_for ArgumentError in initializer and 2 freekiqs in worker' do
237
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
238
- let (:handler) { build_handler_chain(freekiq_for: [ArgumentError]) }
239
- let (:job) { build_job }
240
- let (:worker) do
241
- Class.new do
242
- include Sidekiq::Worker
243
- sidekiq_options freekiqs: 2
244
- end.new
179
+ describe 'with freekiq_for ArgumentError in worker' do
180
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
181
+ let!(:worker_class) { initialize_worker_class(freekiq_for: [ArgumentError]) }
245
182
  end
246
183
  end
247
184
  end
248
185
 
249
- describe 'with freekiq_for NonArgumentError in worker and 2 freekiqs in worker' do
250
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
251
- let (:handler) { build_handler_chain }
252
- let (:job) { build_job }
253
- let (:worker) do
254
- Class.new do
255
- include Sidekiq::Worker
256
- class NonArgumentError < StandardError; end
257
- sidekiq_options freekiqs: 2, freekiq_for: [NonArgumentError]
258
- end.new
259
- end
186
+ context 'with middleware configured with freekiq_for ArgumentError' do
187
+ before(:each) do
188
+ initialize_middleware(freekiq_for: [ArgumentError])
260
189
  end
261
- end
262
190
 
263
- describe 'with freekiq_for NonArgumentError in initializer and 2 freekiqs in worker' do
264
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
265
- let (:handler) { build_handler_chain(freekiq_for: [NonArgumentError]) }
266
- let (:job) { build_job }
267
- let (:worker) do
268
- Class.new do
269
- include Sidekiq::Worker
270
- class NonArgumentError < StandardError; end
271
- sidekiq_options freekiqs: 2
272
- end.new
191
+ describe 'with 2 freekiqs in worker' do
192
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
193
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2) }
273
194
  end
274
195
  end
275
- end
276
196
 
277
- describe 'with freekiq_for NonArgumentError in initializer and freekiq for ArgumentError in worker' do
278
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
279
- let (:handler) { build_handler_chain(freekiq_for: [NonArgumentError]) }
280
- let (:job) { build_job }
281
- let (:worker) do
282
- Class.new do
283
- include Sidekiq::Worker
284
- sidekiq_options freekiqs: 2, freekiq_for: [ArgumentError]
285
- end.new
197
+ describe 'with freekiq_for NonArgumentError in worker' do
198
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
199
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [NonArgumentError]) }
286
200
  end
287
201
  end
288
202
  end
289
203
 
290
- describe 'with freekiq_for ArgumentError in initializer and freekiq for NonArgumentError in worker' do
291
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
292
- let (:handler) { build_handler_chain(freekiq_for: [ArgumentError]) }
293
- let (:job) { build_job }
294
- let (:worker) do
295
- Class.new do
296
- include Sidekiq::Worker
297
- class NonArgumentError < StandardError; end
298
- sidekiq_options freekiqs: 2, freekiq_for: [NonArgumentError]
299
- end.new
204
+ context 'with middleware configured with freekiq_for NonArgumentError' do
205
+ before(:each) do
206
+ initialize_middleware(freekiq_for: [NonArgumentError])
207
+ end
208
+
209
+ describe 'with 2 freekiqs in worker' do
210
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
211
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2) }
300
212
  end
301
213
  end
302
- end
303
214
 
304
- describe 'with freekiq_for ArgumentError as a string' do
305
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
306
- let (:handler) { build_handler_chain }
307
- let (:job) { build_job }
308
- let (:worker) do
309
- Class.new do
310
- include Sidekiq::Worker
311
- sidekiq_options freekiqs: 2, freekiq_for: ['ArgumentError']
312
- end.new
215
+ describe 'with freekiq_for ArgumentError in worker' do
216
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
217
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [ArgumentError]) }
313
218
  end
314
219
  end
315
220
  end
316
221
 
317
- describe 'with freekiq_for the super class of ArgumentError' do
222
+ context 'with middleware configured with 2 freekiqs and freekiq_for ArgumentError' do
223
+ before(:each) do
224
+ initialize_middleware(freekiqs: 2, freekiq_for: [ArgumentError])
225
+ end
226
+
318
227
  it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
319
- let (:handler) { build_handler_chain }
320
- let (:job) { build_job }
321
- let (:worker) do
322
- Class.new do
323
- include Sidekiq::Worker
324
- sidekiq_options freekiqs: 2, freekiq_for: [ArgumentError.superclass]
325
- end.new
326
- end
228
+ let!(:worker_class) { initialize_worker_class }
327
229
  end
328
230
  end
231
+
232
+ it 'should execute a defined callback' do
233
+ called = false
234
+ initialize_middleware(callback: ->(worker, msg, queue){called = true})
235
+
236
+ expect {
237
+ process_job(build_job_hash(initialize_worker_class(freekiqs: 2)))
238
+ }.to raise_error(Sidekiq::FreekiqException, 'Oops')
239
+ expect(called).to eq(true)
240
+ end
241
+
242
+ it 'should still raise FreekiqException if the callback fails' do
243
+ initialize_middleware(callback: ->(worker, msg, queue){raise 'callback error'})
244
+
245
+ expect {
246
+ process_job(build_job_hash(initialize_worker_class(freekiqs: 2)))
247
+ }.to raise_error(Sidekiq::FreekiqException, 'Oops')
248
+ end
329
249
  end
data/spec/spec_helper.rb CHANGED
@@ -1,11 +1,102 @@
1
1
  require 'freekiqs'
2
2
 
3
+ $TESTING = true
4
+
5
+ Sidekiq.logger = nil
6
+
7
+ # This file was generated by the `rspec --init` command. Conventionally, all
8
+ # specs live under a `spec` directory, which RSpec adds to the `$LOAD_PATH`.
9
+ # The generated `.rspec` file contains `--require spec_helper` which will cause
10
+ # this file to always be loaded, without a need to explicitly require it in any
11
+ # files.
12
+ #
13
+ # Given that it is always loaded, you are encouraged to keep this file as
14
+ # light-weight as possible. Requiring heavyweight dependencies from this file
15
+ # will add to the boot time of your test suite on EVERY test run, even for an
16
+ # individual file that may not need all of that loaded. Instead, consider making
17
+ # a separate helper file that requires the additional dependencies and performs
18
+ # the additional setup, and require it from the spec files that actually need
19
+ # it.
20
+ #
21
+ # See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
3
22
  RSpec.configure do |config|
23
+ # rspec-expectations config goes here. You can use an alternate
24
+ # assertion/expectation library such as wrong or the stdlib/minitest
25
+ # assertions if you prefer.
4
26
  config.expect_with :rspec do |expectations|
27
+ # This option will default to `true` in RSpec 4. It makes the `description`
28
+ # and `failure_message` of custom matchers include text for helper methods
29
+ # defined using `chain`, e.g.:
30
+ # be_bigger_than(2).and_smaller_than(4).description
31
+ # # => "be bigger than 2 and smaller than 4"
32
+ # ...rather than:
33
+ # # => "be bigger than 2"
5
34
  expectations.include_chain_clauses_in_custom_matcher_descriptions = true
6
35
  end
7
36
 
37
+ # rspec-mocks config goes here. You can use an alternate test double
38
+ # library (such as bogus or mocha) by changing the `mock_with` option here.
8
39
  config.mock_with :rspec do |mocks|
40
+ # Prevents you from mocking or stubbing a method that does not exist on
41
+ # a real object. This is generally recommended, and will default to
42
+ # `true` in RSpec 4.
9
43
  mocks.verify_partial_doubles = true
10
44
  end
45
+
46
+ # This option will default to `:apply_to_host_groups` in RSpec 4 (and will
47
+ # have no way to turn it off -- the option exists only for backwards
48
+ # compatibility in RSpec 3). It causes shared context metadata to be
49
+ # inherited by the metadata hash of host groups and examples, rather than
50
+ # triggering implicit auto-inclusion in groups with matching metadata.
51
+ config.shared_context_metadata_behavior = :apply_to_host_groups
52
+
53
+ # This allows you to limit a spec run to individual examples or groups
54
+ # you care about by tagging them with `:focus` metadata. When nothing
55
+ # is tagged with `:focus`, all examples get run. RSpec also provides
56
+ # aliases for `it`, `describe`, and `context` that include `:focus`
57
+ # metadata: `fit`, `fdescribe` and `fcontext`, respectively.
58
+ config.filter_run_when_matching :focus
59
+
60
+ # Allows RSpec to persist some state between runs in order to support
61
+ # the `--only-failures` and `--next-failure` CLI options. We recommend
62
+ # you configure your source control system to ignore this file.
63
+ # config.example_status_persistence_file_path = "spec/examples.txt"
64
+
65
+ # Limits the available syntax to the non-monkey patched syntax that is
66
+ # recommended. For more details, see:
67
+ # - http://rspec.info/blog/2012/06/rspecs-new-expectation-syntax/
68
+ # - http://www.teaisaweso.me/blog/2013/05/27/rspecs-new-message-expectation-syntax/
69
+ # - http://rspec.info/blog/2014/05/notable-changes-in-rspec-3/#zero-monkey-patching-mode
70
+ config.disable_monkey_patching!
71
+
72
+ # This setting enables warnings. It's recommended, but in some cases may
73
+ # be too noisy due to issues in dependencies.
74
+ config.warnings = true
75
+
76
+ # Many RSpec users commonly either run the entire suite or an individual
77
+ # file, and it's useful to allow more verbose output when running an
78
+ # individual spec file.
79
+ if config.files_to_run.one?
80
+ # Use the documentation formatter for detailed output,
81
+ # unless a formatter has already been configured
82
+ # (e.g. via a command-line flag).
83
+ config.default_formatter = "doc"
84
+ end
85
+
86
+ # Print the 10 slowest examples and example groups at the
87
+ # end of the spec run, to help surface which specs are running
88
+ # particularly slow.
89
+ config.profile_examples = 10
90
+
91
+ # Run specs in random order to surface order dependencies. If you find an
92
+ # order dependency and want to debug it, you can fix the order by providing
93
+ # the seed, which is printed after each run.
94
+ # --seed 1234
95
+ config.order = :random
96
+
97
+ # Seed global randomization in this process using the `--seed` CLI option.
98
+ # Setting this allows you to use `--seed` to deterministically reproduce
99
+ # test failures related to randomization by passing the same `--seed` value
100
+ # as the one that triggered the failure.
101
+ Kernel.srand config.seed
11
102
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: freekiqs
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.1.0
4
+ version: 6.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Rob Lewis
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-06-24 00:00:00.000000000 Z
11
+ date: 2022-10-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: sidekiq
@@ -16,34 +16,28 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 4.0.0
20
- - - "<"
21
- - !ruby/object:Gem::Version
22
- version: 5.0.0
19
+ version: 6.5.0
23
20
  type: :runtime
24
21
  prerelease: false
25
22
  version_requirements: !ruby/object:Gem::Requirement
26
23
  requirements:
27
24
  - - ">="
28
25
  - !ruby/object:Gem::Version
29
- version: 4.0.0
30
- - - "<"
31
- - !ruby/object:Gem::Version
32
- version: 5.0.0
26
+ version: 6.5.0
33
27
  - !ruby/object:Gem::Dependency
34
28
  name: rspec
35
29
  requirement: !ruby/object:Gem::Requirement
36
30
  requirements:
37
31
  - - ">="
38
32
  - !ruby/object:Gem::Version
39
- version: 2.14.1
33
+ version: 3.6.0
40
34
  type: :development
41
35
  prerelease: false
42
36
  version_requirements: !ruby/object:Gem::Requirement
43
37
  requirements:
44
38
  - - ">="
45
39
  - !ruby/object:Gem::Version
46
- version: 2.14.1
40
+ version: 3.6.0
47
41
  description: Sidekiq middleware extending RetryJobs to allow configuring how many
48
42
  exceptions a job can throw and be wrapped by a silenceable exception.
49
43
  email:
@@ -53,14 +47,19 @@ extensions: []
53
47
  extra_rdoc_files: []
54
48
  files:
55
49
  - ".gitignore"
50
+ - ".rspec"
56
51
  - Gemfile
57
52
  - MIT-LICENSE
53
+ - Procfile
58
54
  - README.md
59
55
  - Rakefile
56
+ - Vagrantfile
60
57
  - circle.yml
58
+ - development/redis/redis.conf
61
59
  - freekiqs.gemspec
62
60
  - lib/freekiqs.rb
63
61
  - lib/sidekiq/middleware/server/freekiqs.rb
62
+ - shell.nix
64
63
  - spec/freekiqs_spec.rb
65
64
  - spec/spec_helper.rb
66
65
  homepage: https://github.com/BookBub/freekiqs
@@ -75,15 +74,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
75
74
  requirements:
76
75
  - - ">="
77
76
  - !ruby/object:Gem::Version
78
- version: 1.9.3
77
+ version: 2.2.2
79
78
  required_rubygems_version: !ruby/object:Gem::Requirement
80
79
  requirements:
81
80
  - - ">="
82
81
  - !ruby/object:Gem::Version
83
82
  version: '0'
84
83
  requirements: []
85
- rubyforge_project:
86
- rubygems_version: 2.5.1
84
+ rubygems_version: 3.0.3.1
87
85
  signing_key:
88
86
  specification_version: 4
89
87
  summary: Sidekiq middleware extending RetryJobs to allow silient errors.