freekiqs 4.1.0 → 6.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 0a7ee677e8d100c7405dc91f9d4c341d7f7a8986
4
- data.tar.gz: 0281092a960af50ca75ed3b03370bd7a86b1ae17
2
+ SHA256:
3
+ metadata.gz: d6b06dcd42bf69b36943cb98073500edaa179251c807284ac1520788f4f9b868
4
+ data.tar.gz: 7d273e8393c7f4ec61b2bc3488daaa3a8e7c4d2c3c4e323d0787f3217a1ec772
5
5
  SHA512:
6
- metadata.gz: 35984aad9ab4d0fec1d33f07469ba0f82ff83a5b3cdeaecb39f1bbdb42f56ec9566df45c8735970034f99294919790481983047141a89750118845d46d7a3ad9
7
- data.tar.gz: d932c98cefbdcc672d00c2e85ae16a86b6b4b70cb719403a462b8981dacd22fca4f0cbaa7cf90735879cbd8cbc8efe3a5f221eaa9d8cd033962e0efd841dbaf4
6
+ metadata.gz: beeb49dbe17a46885ab99ff7c2128645fb739496a7636a2a0b8f041c7b1099dfd42e291f23d15bf49b0ff4b1da09d01da8a8f8d2cb9ed482db09982143d64d65
7
+ data.tar.gz: 1ce3a30d1d569967f3c0b3fe6941baf00cfaa6c99da09436e87481b29e8dfa1412cef2ede6853ae4c9e6099aa547149b01c943ac613891cb7b39d0020c7b44e1
data/.gitignore CHANGED
@@ -3,3 +3,8 @@ gemfiles/*.lock
3
3
  Gemfile.lock
4
4
  .ruby-version
5
5
  pkg
6
+ /.bundle
7
+ /.vagrant
8
+ /ubuntu-*-console.log
9
+ /vendor
10
+ dump.rdb
data/.rspec ADDED
@@ -0,0 +1 @@
1
+ --require spec_helper
data/Gemfile CHANGED
@@ -5,5 +5,5 @@ gemspec
5
5
  gem 'rake'
6
6
 
7
7
  group :test do
8
- gem 'rspec', '~> 3.1'
8
+ gem 'rspec', '~> 3.6'
9
9
  end
data/Procfile ADDED
@@ -0,0 +1 @@
1
+ redis: redis-server development/redis/redis.conf --bind $HOST
data/README.md CHANGED
@@ -3,13 +3,28 @@
3
3
  Sidekiq middleware that allows capturing exceptions thrown
4
4
  by failed jobs and wrapping them with a `FreekiqException` exception class
5
5
  that can be filtered by monitoring tools such as New Relic and
6
- Honeybadger.
6
+ Rollbar.
7
+
8
+ #### Implementation Details
9
+
10
+ This relies on Sidekiq's built-in retry handling. Specifically, its
11
+ `retry_count` value. When a job first fails, its `retry_count` value
12
+ is nil (because it hasn't actually been retried yet). That exception,
13
+ along with subsequent exceptions from retries, are caught and wrapped
14
+ with the `FreekiqException` exception.
15
+ Cases where Freekiqs does NOT wrap the exception:
16
+ - The `retry` option is false
17
+ - The `freekiqs` option is not set on the worker nor globally
18
+ - The `freekiqs` option is not set on worker and set to `false` globally
19
+ - The `freekiqs` option is set to `false` on the worker
20
+ - The job threw an exception that is not a StandardError (nor a subclass)
21
+ - The number of thrown exceptions is more than specified freekiqs
7
22
 
8
23
  Configuration example (in config/initializers/sidekiq.rb):
9
24
  ``` ruby
10
25
  Sidekiq.configure_server do |config|
11
26
  config.server_middleware do |chain|
12
- chain.insert_after Sidekiq::Middleware::Server::RetryJobs, Sidekiq::Middleware::Server::Freekiqs
27
+ chain.add Sidekiq::Middleware::Server::Freekiqs
13
28
  end
14
29
  end
15
30
  ```
@@ -19,7 +34,7 @@ Worker example:
19
34
  class MyWorker
20
35
  include Sidekiq::Worker
21
36
  sidekiq_options freekiqs: 3
22
- #
37
+
23
38
  def perform(param)
24
39
  ...
25
40
  end
@@ -35,7 +50,7 @@ Example:
35
50
  ``` ruby
36
51
  Sidekiq.configure_server do |config|
37
52
  config.server_middleware do |chain|
38
- chain.insert_after Sidekiq::Middleware::Server::RetryJobs, Sidekiq::Middleware::Server::Freekiqs, freekiqs: 3
53
+ chain.add Sidekiq::Middleware::Server::Freekiqs, freekiqs: 3
39
54
  end
40
55
  end
41
56
  ```
@@ -44,6 +59,17 @@ A callback can be fired when a freekiq happens.
44
59
  This can be useful for tracking or logging freekiqs separately from the sidekiq logs.
45
60
 
46
61
  Example:
62
+ ``` ruby
63
+ Sidekiq.configure_server do |config|
64
+ config.server_middleware do |chain|
65
+ chain.add Sidekiq::Middleware::Server::Freekiqs, callback: ->(worker, msg, queue) do
66
+ Librato::Metrics.submit freekiqs: { value: 1, source: worker.class.name }
67
+ end
68
+ end
69
+ end
70
+ ```
71
+
72
+ Or callback can be set outside middleware configuration:
47
73
  ``` ruby
48
74
  Sidekiq::Middleware::Server::Freekiqs::callback = ->(worker, msg, queue) do
49
75
  Librato::Metrics.submit freekiqs: { value: 1, source: worker.class.name }
@@ -57,7 +83,7 @@ Example:
57
83
  class MyWorker
58
84
  include Sidekiq::Worker
59
85
  sidekiq_options freekiqs: 1, freekiq_for: ['MyError']
60
- #
86
+
61
87
  def perform(param)
62
88
  ...
63
89
  end
@@ -67,8 +93,8 @@ In this case, if MyWorker fails with a MyError it will get 1 freekiq.
67
93
  All other errors thrown by this worker will get no freekiqs.
68
94
 
69
95
 
70
- If a `freekiq_for` contains a class name as a constant, any exception is that class
71
- *or a subclass* of that class will get freekiq'd.
96
+ If a `freekiq_for` contains a class name as a constant, any exception of that class
97
+ type *or a subclass* of that class will get freekiq'd.
72
98
 
73
99
  Example:
74
100
  ``` ruby
@@ -77,7 +103,7 @@ Example:
77
103
  class MyWorker
78
104
  include Sidekiq::Worker
79
105
  sidekiq_options freekiqs: 1, freekiq_for: [MyError]
80
- #
106
+
81
107
  def perform(param)
82
108
  ...
83
109
  end
@@ -85,26 +111,9 @@ Example:
85
111
  ```
86
112
  If MyWorker throws a SubMyError or MyError, it will get freekiq'd.
87
113
 
114
+ ## Sidekiq Versions
88
115
 
89
- ## Overview
116
+ Version 5 of this gem only works with Sidekiq 5 and higher. If you are using
117
+ an older version of Sidekiq, you'll need to use version [4.1.0](https://github.com/BookBub/freekiqs/tree/v4.1.0).
90
118
 
91
- Up to the configured number of "freekiqs", catch exceptions thrown
92
- from job and wrap them with the `FreekiqException` exception (which is a
93
- `RuntimeError`). That exception type can be ignored by monitoring
94
- tools. If job fails more times than configured "freekiqs", thrown
95
- exception will not be wrapped. That should result in normal operation
96
- and the exception showing up in monitoring tools.
97
-
98
- Implementation Details:
99
- This relies on Sidekiq's built-in retry handling. Specifically, its
100
- `retry_count` value. When a job first fails, its `retry_count` value
101
- is nil (because it hasn't actually been retried yet). That exception,
102
- along with subsequent exceptions from retries, are caught and wrapped
103
- with the `FreekiqException` exception.
104
- Cases where Freekiqs does NOT wrap the exception:
105
- - The `retry` option is false
106
- - The `freekiqs` option is not set on the worker nor globally
107
- - The `freekiqs` option is not set on worker and set to `false` globally
108
- - The `freekiqs` option is set to `false` on the worker
109
- - The job threw an exception that is not a StandardError (nor a subclass)
110
- - The number of thrown exceptions is more than specified freekiqs
119
+ Version 6.5.0 of this gem works with Sidekiq 6.5 and higher.
data/Vagrantfile ADDED
@@ -0,0 +1,31 @@
1
+ Vagrant.configure(2) do |config|
2
+ config.vm.box = 'ubuntu/xenial64'
3
+
4
+ config.vm.network :private_network, ip: '192.168.33.69'
5
+ config.vm.network :forwarded_port, host: 2202, guest: 22, id: 'ssh', auto_correct: true
6
+
7
+ config.vm.synced_folder '.', '/vagrant', type: 'nfs'
8
+
9
+ config.vm.provider :virtualbox do |vb|
10
+ vb.name = 'freekiqs'
11
+ vb.memory = 2048
12
+ vb.cpus = 2
13
+ end
14
+
15
+ config.vm.provision :shell, inline: <<-SCRIPT
16
+ apt-get update
17
+ apt-get upgrade
18
+ apt-get install -y git gnupg2 redis-server
19
+ SCRIPT
20
+
21
+ config.vm.provision :shell, privileged: false, inline: <<-SCRIPT
22
+ gpg --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
23
+ curl -sSL https://get.rvm.io | bash -s stable --quiet-curl --with-gems=bundler
24
+ source "$HOME/.rvm/scripts/rvm"
25
+ rvm --quiet-curl install 2.3.4
26
+ rvm use 2.3.4 --default
27
+ cd /vagrant
28
+ bundle install --full-index -j4
29
+ echo 'cd /vagrant' >> ~/.bash_profile
30
+ SCRIPT
31
+ end
data/circle.yml CHANGED
@@ -1,5 +1,5 @@
1
1
  machine:
2
2
  ruby:
3
- version: 2.3.0
3
+ version: 2.3.4
4
4
  services:
5
5
  - redis
@@ -0,0 +1,49 @@
1
+ protected-mode yes
2
+ port 6379
3
+ tcp-backlog 511
4
+ timeout 0
5
+ tcp-keepalive 300
6
+ daemonize no
7
+ supervised no
8
+ pidfile development/redis/redis_6379.pid
9
+ loglevel warning
10
+ logfile ""
11
+ databases 16
12
+ save 900 1
13
+ save 300 10
14
+ save 60 10000
15
+ stop-writes-on-bgsave-error yes
16
+ rdbcompression yes
17
+ rdbchecksum yes
18
+ dbfilename dump.rdb
19
+ dir development/redis
20
+ slave-serve-stale-data yes
21
+ slave-read-only yes
22
+ repl-diskless-sync no
23
+ repl-diskless-sync-delay 5
24
+ repl-disable-tcp-nodelay no
25
+ slave-priority 100
26
+ appendonly no
27
+ appendfilename "appendonly.aof"
28
+ appendfsync no
29
+ no-appendfsync-on-rewrite no
30
+ auto-aof-rewrite-percentage 100
31
+ auto-aof-rewrite-min-size 64mb
32
+ aof-load-truncated yes
33
+ lua-time-limit 5000
34
+ latency-monitor-threshold 0
35
+ notify-keyspace-events ""
36
+ hash-max-ziplist-entries 512
37
+ hash-max-ziplist-value 64
38
+ list-max-ziplist-size -2
39
+ list-compress-depth 0
40
+ set-max-intset-entries 512
41
+ zset-max-ziplist-entries 128
42
+ zset-max-ziplist-value 64
43
+ hll-sparse-max-bytes 3000
44
+ activerehashing yes
45
+ client-output-buffer-limit normal 0 0 0
46
+ client-output-buffer-limit slave 256mb 64mb 60
47
+ client-output-buffer-limit pubsub 32mb 8mb 60
48
+ hz 10
49
+ aof-rewrite-incremental-fsync yes
data/freekiqs.gemspec CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  Gem::Specification.new do |gem|
4
4
  gem.name = 'freekiqs'
5
- gem.version = '4.1.0'
5
+ gem.version = '6.5.0'
6
6
  gem.authors = ['Rob Lewis']
7
7
  gem.email = ['rob@bookbub.com']
8
8
  gem.summary = 'Sidekiq middleware extending RetryJobs to allow silient errors.'
@@ -14,8 +14,8 @@ Gem::Specification.new do |gem|
14
14
  gem.files = `git ls-files`.split("\n")
15
15
  gem.test_files = `git ls-files -- spec/*`.split("\n")
16
16
  gem.require_paths = ['lib']
17
- gem.required_ruby_version = '>= 1.9.3'
17
+ gem.required_ruby_version = '>= 2.2.2'
18
18
 
19
- gem.add_dependency 'sidekiq', '>= 4.0.0', '< 5.0.0'
20
- gem.add_development_dependency 'rspec', '>= 2.14.1'
19
+ gem.add_dependency 'sidekiq', '>= 6.5.0'
20
+ gem.add_development_dependency 'rspec', '>= 3.6.0'
21
21
  end
@@ -1,5 +1,5 @@
1
1
  require 'sidekiq'
2
- require 'sidekiq/util'
2
+ require 'sidekiq/middleware/modules'
3
3
 
4
4
  module Sidekiq
5
5
  class FreekiqException < RuntimeError; end
@@ -7,37 +7,38 @@ module Sidekiq
7
7
  module Middleware
8
8
  module Server
9
9
  class Freekiqs
10
- include Sidekiq::Util
10
+ include Sidekiq::ServerMiddleware
11
11
  @@callback = nil
12
12
 
13
13
  def initialize(opts={})
14
14
  @default_freekiqs = opts[:freekiqs]
15
15
  @default_freekiq_for = opts[:freekiq_for]
16
+ @@callback = opts[:callback]
16
17
  end
17
18
 
18
19
  def call(worker, msg, queue)
19
20
  yield
20
21
  rescue => ex
21
- freekiqs = get_freekiqs_if_enabled(worker, msg, ex)
22
+ freekiqs = get_freekiqs_if_enabled(worker, ex)
22
23
  if freekiqs
23
24
  if msg['retry_count'].nil? || msg['retry_count'] < freekiqs-1
24
25
  begin
25
26
  @@callback.call(worker, msg, queue) if @@callback
26
27
  rescue => callback_exception
27
- Sidekiq.logger.info { "Freekiq callback failed for #{msg['class']} job #{msg['jid']}" }
28
+ logger.info { "Freekiq callback failed for #{msg['class']} job #{msg['jid']}" }
28
29
  ensure
29
30
  raise FreekiqException, ex.message
30
31
  end
31
32
  else
32
- Sidekiq.logger.info { "Out of free kiqs for #{msg['class']} job #{msg['jid']}" }
33
+ logger.info { "Out of freekiqs for #{msg['class']} job #{msg['jid']}" }
33
34
  end
34
35
  end
35
36
  raise ex
36
37
  end
37
38
 
38
- def get_freekiqs_if_enabled(worker, msg, ex)
39
+ def get_freekiqs_if_enabled(worker, ex)
39
40
  freekiqs = nil
40
- if msg['retry']
41
+ if worker.class.get_sidekiq_options['retry']
41
42
  if worker.class.get_sidekiq_options['freekiqs'] != false
42
43
  errors = get_freekiq_errors(worker)
43
44
  if worker.class.get_sidekiq_options['freekiqs']
data/shell.nix ADDED
@@ -0,0 +1,42 @@
1
+ { system ? builtins.currentSystem }:
2
+
3
+ with (import (fetchTarball {
4
+ name = "nixpkgs-21.11";
5
+ url = "https://github.com/nixos/nixpkgs/archive/21.11.tar.gz";
6
+ sha256 = "162dywda2dvfj1248afxc45kcrg83appjd0nmdb541hl7rnncf02";
7
+ }) { inherit system; });
8
+
9
+
10
+ let
11
+ requiredNixVersion = "2.3";
12
+
13
+ pwd = builtins.getEnv "PWD";
14
+ in
15
+
16
+ if lib.versionOlder builtins.nixVersion requiredNixVersion == true then
17
+ abort "This project requires Nix >= ${requiredNixVersion}, please run 'nix-channel --update && nix-env -i nix'."
18
+ else
19
+
20
+ mkShell {
21
+ buildInputs = [
22
+ stdenv
23
+ git
24
+ cacert
25
+
26
+ # Ruby and Rails dependencies
27
+ ruby_2_7.devEnv
28
+ bundler
29
+
30
+ # Services
31
+ overmind
32
+ redis
33
+
34
+ ] ++ lib.optional (!stdenv.isDarwin) [
35
+ # linux-only packages
36
+ glibcLocales
37
+ ];
38
+
39
+ HOST = "127.0.0.24";
40
+ REDIS_URL="redis://127.0.0.24:6379/1";
41
+ BUNDLE_PATH = "vendor/bundle";
42
+ }
@@ -1,329 +1,249 @@
1
- require 'spec_helper'
2
- require 'sidekiq/cli'
3
- require 'sidekiq/middleware/server/retry_jobs'
1
+ require 'sidekiq'
2
+ require 'sidekiq/api'
3
+ require 'sidekiq/processor'
4
4
 
5
- shared_examples_for 'it should have 2 freekiqs for an ArgumentError' do
6
- it 'throws Freekiq exception for specified number of free kiqs' do
7
- expect {
8
- handler.invoke(worker, job, 'default') do
9
- raise ArgumentError, 'overlooked'
10
- end
11
- }.to raise_error(Sidekiq::FreekiqException, 'overlooked')
12
- expect(job['retry_count']).to eq(0)
13
- expect(job['error_class']).to eq('Sidekiq::FreekiqException')
14
- expect(job['error_message']).to eq('overlooked')
15
- expect {
16
- handler.invoke(worker, job, 'default') do
17
- raise ArgumentError, 'overlooked'
18
- end
19
- }.to raise_error(Sidekiq::FreekiqException, 'overlooked')
20
- expect(job['retry_count']).to eq(1)
21
- expect(job['error_class']).to eq('Sidekiq::FreekiqException')
22
- expect(job['error_message']).to eq('overlooked')
23
- expect {
24
- handler.invoke(worker, job, 'default') do
25
- raise ArgumentError, 'not overlooked'
26
- end
27
- }.to raise_error(ArgumentError, 'not overlooked')
28
- expect(job['retry_count']).to eq(2)
29
- expect(job['error_class']).to eq('ArgumentError')
30
- expect(job['error_message']).to eq('not overlooked')
31
- expect(Sidekiq::RetrySet.new.size).to eq(3)
32
- expect(Sidekiq::DeadSet.new.size).to eq(0)
33
- end
34
- end
5
+ RSpec.describe Sidekiq::Middleware::Server::Freekiqs do
6
+ class NonArgumentError < StandardError; end
35
7
 
36
- shared_examples_for 'it should have 0 freekiqs for an ArgumentError' do
37
- it 'raises the original error' do
38
- expect {
39
- handler.invoke(worker, job, 'default') do
40
- raise ArgumentError, 'not overlooked'
41
- end
42
- }.to raise_error(ArgumentError, 'not overlooked')
43
- expect(job['retry_count']).to eq(0)
44
- expect(job['error_class']).to eq('ArgumentError')
45
- expect(job['error_message']).to eq('not overlooked')
46
- expect(Sidekiq::RetrySet.new.size).to eq(1)
47
- expect(Sidekiq::DeadSet.new.size).to eq(0)
8
+ def build_job_hash(worker_class, args=[])
9
+ {'class' => worker_class, 'args' => args}
48
10
  end
49
- end
50
11
 
51
- describe Sidekiq::Middleware::Server::Freekiqs do
52
- class DummyWorkerPlain
53
- include Sidekiq::Worker
12
+ def fetch_retry_job
13
+ retry_set = Sidekiq::RetrySet.new
14
+ retry_job = retry_set.first
15
+ retry_set.clear
16
+ retry_job
54
17
  end
55
- class DummyWorkerWithFreekiqsEnabled
56
- include Sidekiq::Worker
57
- sidekiq_options freekiqs: 2
18
+
19
+ def process_job(job_hash)
20
+ processor = Sidekiq::Processor.new(Sidekiq)
21
+ job_msg = Sidekiq.dump_json(job_hash)
22
+ processor.process(Sidekiq::BasicFetch::UnitOfWork.new('queue:default', job_msg))
58
23
  end
59
24
 
60
- def build_handler_chain(freekiq_options={}, retry_options={})
61
- Sidekiq::Middleware::Chain.new do |chain|
62
- chain.add Sidekiq::Middleware::Server::RetryJobs, retry_options
63
- chain.add Sidekiq::Middleware::Server::Freekiqs, freekiq_options
25
+ def initialize_middleware(middleware_opts={})
26
+ Sidekiq.server_middleware do |chain|
27
+ chain.add Sidekiq::Middleware::Server::Freekiqs, middleware_opts
64
28
  end
65
29
  end
66
30
 
67
- def build_job(options={})
68
- {'class' => 'FreekiqDummyWorker', 'args' => [], 'retry' => true}.merge(options)
31
+ def initialize_worker_class(sidekiq_opts=nil)
32
+ worker_class_name = :TestDummyWorker
33
+ Object.send(:remove_const, worker_class_name) if Object.const_defined?(worker_class_name)
34
+ klass = Class.new do
35
+ include Sidekiq::Worker
36
+ sidekiq_options sidekiq_opts if sidekiq_opts
37
+ def perform
38
+ raise ArgumentError, 'Oops'
39
+ end
40
+ end
41
+ Object.const_set(worker_class_name, klass)
69
42
  end
70
43
 
71
44
  def cleanup_redis
72
- Sidekiq::RetrySet.new.select{|job| job.klass == 'FreekiqDummyWorker'}.each(&:delete)
73
- Sidekiq::DeadSet.new.clear
45
+ Sidekiq.redis {|c| c.flushdb }
46
+ end
47
+
48
+ shared_examples_for 'it should have 2 freekiqs for an ArgumentError' do
49
+ it 'throws Freekiq exception for specified number of freekiqs' do
50
+ args ||= []
51
+ expect {
52
+ process_job(build_job_hash(worker_class, args))
53
+ }.to raise_error(Sidekiq::FreekiqException, 'Oops')
54
+ expect(Sidekiq::RetrySet.new.size).to eq(1)
55
+ retry_job = fetch_retry_job
56
+ expect(retry_job['retry_count']).to eq(0)
57
+ expect(retry_job['error_class']).to eq('Sidekiq::FreekiqException')
58
+ expect(retry_job['error_message']).to eq('Oops')
59
+
60
+ expect {
61
+ process_job(retry_job.item)
62
+ }.to raise_error(Sidekiq::FreekiqException, 'Oops')
63
+ expect(Sidekiq::RetrySet.new.size).to eq(1)
64
+ retry_job = fetch_retry_job
65
+ expect(retry_job['retry_count']).to eq(1)
66
+ expect(retry_job['error_class']).to eq('Sidekiq::FreekiqException')
67
+ expect(retry_job['error_message']).to eq('Oops')
68
+
69
+ expect {
70
+ process_job(retry_job.item)
71
+ }.to raise_error(ArgumentError, 'Oops')
72
+ expect(Sidekiq::RetrySet.new.size).to eq(1)
73
+ retry_job = fetch_retry_job
74
+ expect(retry_job['retry_count']).to eq(2)
75
+ expect(retry_job['error_class']).to eq('ArgumentError')
76
+ expect(retry_job['error_message']).to eq('Oops')
77
+ end
78
+ end
79
+
80
+ shared_examples_for 'it should have 0 freekiqs for an ArgumentError' do
81
+ it 'raises the original error' do
82
+ args ||= []
83
+ expect {
84
+ process_job(build_job_hash(worker_class, args))
85
+ }.to raise_error(ArgumentError, 'Oops')
86
+ expect(Sidekiq::RetrySet.new.size).to eq(1)
87
+ retry_job = fetch_retry_job
88
+ expect(retry_job['retry_count']).to eq(0)
89
+ expect(retry_job['error_class']).to eq('ArgumentError')
90
+ expect(retry_job['error_message']).to eq('Oops')
91
+ end
92
+ end
93
+
94
+ shared_examples_for 'it should only raise exception for an ArgumentError' do
95
+ it 'raises the original error' do
96
+ args ||= []
97
+ expect {
98
+ process_job(build_job_hash(worker_class, args))
99
+ }.to raise_error(ArgumentError, 'Oops')
100
+ expect(Sidekiq::RetrySet.new.size).to eq(0)
101
+ # Note: Sidekiq doesn't send job to morgue when retries are diabled
102
+ expect(Sidekiq::DeadSet.new.size).to eq(0)
103
+ end
74
104
  end
75
105
 
76
106
  before(:each) do
77
107
  cleanup_redis
78
108
  end
79
109
 
80
- let(:worker_plain) { DummyWorkerPlain.new }
81
- let(:worker_with_freekiqs_enabled) { DummyWorkerWithFreekiqsEnabled.new }
110
+ context 'with default middleware config' do
111
+ before(:each) do
112
+ initialize_middleware
113
+ end
82
114
 
83
- it 'requires RetryJobs to update retry_count' do
84
- handler = Sidekiq::Middleware::Server::RetryJobs.new
85
- worker = worker_plain
86
- job = build_job
87
- expect {
88
- handler.call(worker, job, 'default') do
89
- raise 'Oops'
115
+ describe 'with nothing explicitly enabled' do
116
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
117
+ let!(:worker_class) { initialize_worker_class }
90
118
  end
91
- }.to raise_error(RuntimeError)
92
- expect(job['retry_count']).to eq(0)
93
- expect {
94
- handler.call(worker, job, 'default') do
95
- raise 'Oops'
96
- end
97
- }.to raise_error(RuntimeError)
98
- expect(job['retry_count']).to eq(1)
99
- end
100
-
101
- it 'should execute a defined callback' do
102
- Sidekiq::Middleware::Server::Freekiqs::callback = ->(worker, msg, queue) do
103
- return true
104
119
  end
105
120
 
106
- handler = build_handler_chain
107
- worker = worker_with_freekiqs_enabled
108
- job = build_job
109
-
110
- expect(Sidekiq::Middleware::Server::Freekiqs::callback).to receive(:call)
111
- expect {
112
- handler.invoke(worker, job, 'default') do
113
- raise ArgumentError, 'overlooked'
121
+ describe 'with freekiqs explicitly disabled' do
122
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
123
+ let!(:worker_class) { initialize_worker_class(freekiqs: false) }
114
124
  end
115
- }.to raise_error(Sidekiq::FreekiqException, 'overlooked')
116
- end
117
-
118
- it 'should still raise FreekiqException if the callback fails' do
119
- Sidekiq::Middleware::Server::Freekiqs::callback = ->(worker, msg, queue) do
120
- raise 'callback error'
121
125
  end
122
126
 
123
- handler = build_handler_chain
124
- worker = worker_with_freekiqs_enabled
125
- job = build_job
126
-
127
- expect {
128
- handler.invoke(worker, job, 'default') do
129
- raise ArgumentError, 'overlooked'
127
+ describe 'with 2 freekiqs in the worker' do
128
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
129
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2) }
130
130
  end
131
- }.to raise_error(Sidekiq::FreekiqException, 'overlooked')
132
- end
131
+ end
133
132
 
134
- describe 'with nothing explicitly enabled' do
135
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
136
- let (:handler) { build_handler_chain }
137
- let (:job) { build_job }
138
- let (:worker) do
139
- Class.new do
140
- include Sidekiq::Worker
141
- end.new
133
+ describe 'with freekiq_for ArgumentError and 2 freekiqs in worker' do
134
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
135
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [ArgumentError]) }
142
136
  end
143
137
  end
144
- end
145
138
 
146
- describe 'with freekiqs explicitly disabled' do
147
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
148
- let (:handler) { build_handler_chain(freekiqs: 3) }
149
- let (:job) { build_job }
150
- let (:worker) do
151
- Class.new do
152
- include Sidekiq::Worker
153
- sidekiq_options freekiqs: false
154
- end.new
139
+ describe 'with freekiq_for ArgumentError in worker and no freekiqs' do
140
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
141
+ let!(:worker_class) { initialize_worker_class(freekiqs: false, freekiq_for: [ArgumentError]) }
155
142
  end
156
143
  end
157
- end
158
144
 
159
- describe 'with 2 freekiqs in the initializer' do
160
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
161
- let (:handler) { build_handler_chain(freekiqs: 2) }
162
- let (:job) { build_job }
163
- let (:worker) do
164
- Class.new do
165
- include Sidekiq::Worker
166
- end.new
145
+ describe 'with freekiq_for NonArgumentError in worker and 2 freekiqs in worker' do
146
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
147
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [NonArgumentError]) }
167
148
  end
168
149
  end
169
- end
170
150
 
171
- describe 'with 2 freekiqs in the worker' do
172
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
173
- let (:handler) { build_handler_chain }
174
- let (:job) { build_job }
175
- let (:worker) do
176
- Class.new do
177
- include Sidekiq::Worker
178
- sidekiq_options freekiqs: 2
179
- end.new
151
+ describe 'with freekiq_for ArgumentError as a string' do
152
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
153
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: ['ArgumentError']) }
180
154
  end
181
155
  end
182
- end
183
156
 
184
- describe 'with freekiq_for ArgumentError and 2 freekiqs in worker' do
185
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
186
- let (:handler) { build_handler_chain }
187
- let (:job) { build_job }
188
- let (:worker) do
189
- Class.new do
190
- include Sidekiq::Worker
191
- sidekiq_options freekiqs: 2, freekiq_for: [ArgumentError]
192
- end.new
157
+ describe 'with freekiq_for the super class of ArgumentError' do
158
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
159
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [ArgumentError.superclass]) }
193
160
  end
194
161
  end
195
- end
196
162
 
197
- describe 'with freekiq_for ArgumentError in worker and no freekiqs' do
198
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
199
- let (:handler) { build_handler_chain }
200
- let (:job) { build_job }
201
- let (:worker) do
202
- Class.new do
203
- include Sidekiq::Worker
204
- class NonArgumentError < StandardError; end
205
- sidekiq_options freekiqs: 2, freekiq_for: [NonArgumentError]
206
- end.new
163
+ describe 'with 2 freekiqs in the worker and retries disabled' do
164
+ it_behaves_like 'it should only raise exception for an ArgumentError' do
165
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, retry: false) }
207
166
  end
208
167
  end
209
168
  end
210
169
 
211
- describe 'with freekiq_for ArgumentError in worker and 2 freekiqs in initializer' do
212
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
213
- let (:handler) { build_handler_chain(freekiqs: 2) }
214
- let (:job) { build_job }
215
- let (:worker) do
216
- Class.new do
217
- include Sidekiq::Worker
218
- sidekiq_options freekiq_for: [ArgumentError]
219
- end.new
220
- end
170
+ context 'with middleware configured with 2 freekiqs' do
171
+ before(:each) do
172
+ initialize_middleware(freekiqs: 2)
221
173
  end
222
- end
223
174
 
224
- describe 'with freekiq_for ArgumentError in initializer and 2 freekiqs in initializer' do
225
175
  it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
226
- let (:handler) { build_handler_chain(freekiqs: 2, freekiq_for: [ArgumentError]) }
227
- let (:job) { build_job }
228
- let (:worker) do
229
- Class.new do
230
- include Sidekiq::Worker
231
- end.new
232
- end
176
+ let!(:worker_class) { initialize_worker_class }
233
177
  end
234
- end
235
178
 
236
- describe 'with freekiq_for ArgumentError in initializer and 2 freekiqs in worker' do
237
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
238
- let (:handler) { build_handler_chain(freekiq_for: [ArgumentError]) }
239
- let (:job) { build_job }
240
- let (:worker) do
241
- Class.new do
242
- include Sidekiq::Worker
243
- sidekiq_options freekiqs: 2
244
- end.new
179
+ describe 'with freekiq_for ArgumentError in worker' do
180
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
181
+ let!(:worker_class) { initialize_worker_class(freekiq_for: [ArgumentError]) }
245
182
  end
246
183
  end
247
184
  end
248
185
 
249
- describe 'with freekiq_for NonArgumentError in worker and 2 freekiqs in worker' do
250
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
251
- let (:handler) { build_handler_chain }
252
- let (:job) { build_job }
253
- let (:worker) do
254
- Class.new do
255
- include Sidekiq::Worker
256
- class NonArgumentError < StandardError; end
257
- sidekiq_options freekiqs: 2, freekiq_for: [NonArgumentError]
258
- end.new
259
- end
186
+ context 'with middleware configured with freekiq_for ArgumentError' do
187
+ before(:each) do
188
+ initialize_middleware(freekiq_for: [ArgumentError])
260
189
  end
261
- end
262
190
 
263
- describe 'with freekiq_for NonArgumentError in initializer and 2 freekiqs in worker' do
264
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
265
- let (:handler) { build_handler_chain(freekiq_for: [NonArgumentError]) }
266
- let (:job) { build_job }
267
- let (:worker) do
268
- Class.new do
269
- include Sidekiq::Worker
270
- class NonArgumentError < StandardError; end
271
- sidekiq_options freekiqs: 2
272
- end.new
191
+ describe 'with 2 freekiqs in worker' do
192
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
193
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2) }
273
194
  end
274
195
  end
275
- end
276
196
 
277
- describe 'with freekiq_for NonArgumentError in initializer and freekiq for ArgumentError in worker' do
278
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
279
- let (:handler) { build_handler_chain(freekiq_for: [NonArgumentError]) }
280
- let (:job) { build_job }
281
- let (:worker) do
282
- Class.new do
283
- include Sidekiq::Worker
284
- sidekiq_options freekiqs: 2, freekiq_for: [ArgumentError]
285
- end.new
197
+ describe 'with freekiq_for NonArgumentError in worker' do
198
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
199
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [NonArgumentError]) }
286
200
  end
287
201
  end
288
202
  end
289
203
 
290
- describe 'with freekiq_for ArgumentError in initializer and freekiq for NonArgumentError in worker' do
291
- it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
292
- let (:handler) { build_handler_chain(freekiq_for: [ArgumentError]) }
293
- let (:job) { build_job }
294
- let (:worker) do
295
- Class.new do
296
- include Sidekiq::Worker
297
- class NonArgumentError < StandardError; end
298
- sidekiq_options freekiqs: 2, freekiq_for: [NonArgumentError]
299
- end.new
204
+ context 'with middleware configured with freekiq_for NonArgumentError' do
205
+ before(:each) do
206
+ initialize_middleware(freekiq_for: [NonArgumentError])
207
+ end
208
+
209
+ describe 'with 2 freekiqs in worker' do
210
+ it_behaves_like 'it should have 0 freekiqs for an ArgumentError' do
211
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2) }
300
212
  end
301
213
  end
302
- end
303
214
 
304
- describe 'with freekiq_for ArgumentError as a string' do
305
- it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
306
- let (:handler) { build_handler_chain }
307
- let (:job) { build_job }
308
- let (:worker) do
309
- Class.new do
310
- include Sidekiq::Worker
311
- sidekiq_options freekiqs: 2, freekiq_for: ['ArgumentError']
312
- end.new
215
+ describe 'with freekiq_for ArgumentError in worker' do
216
+ it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
217
+ let!(:worker_class) { initialize_worker_class(freekiqs: 2, freekiq_for: [ArgumentError]) }
313
218
  end
314
219
  end
315
220
  end
316
221
 
317
- describe 'with freekiq_for the super class of ArgumentError' do
222
+ context 'with middleware configured with 2 freekiqs and freekiq_for ArgumentError' do
223
+ before(:each) do
224
+ initialize_middleware(freekiqs: 2, freekiq_for: [ArgumentError])
225
+ end
226
+
318
227
  it_behaves_like 'it should have 2 freekiqs for an ArgumentError' do
319
- let (:handler) { build_handler_chain }
320
- let (:job) { build_job }
321
- let (:worker) do
322
- Class.new do
323
- include Sidekiq::Worker
324
- sidekiq_options freekiqs: 2, freekiq_for: [ArgumentError.superclass]
325
- end.new
326
- end
228
+ let!(:worker_class) { initialize_worker_class }
327
229
  end
328
230
  end
231
+
232
+ it 'should execute a defined callback' do
233
+ called = false
234
+ initialize_middleware(callback: ->(worker, msg, queue){called = true})
235
+
236
+ expect {
237
+ process_job(build_job_hash(initialize_worker_class(freekiqs: 2)))
238
+ }.to raise_error(Sidekiq::FreekiqException, 'Oops')
239
+ expect(called).to eq(true)
240
+ end
241
+
242
+ it 'should still raise FreekiqException if the callback fails' do
243
+ initialize_middleware(callback: ->(worker, msg, queue){raise 'callback error'})
244
+
245
+ expect {
246
+ process_job(build_job_hash(initialize_worker_class(freekiqs: 2)))
247
+ }.to raise_error(Sidekiq::FreekiqException, 'Oops')
248
+ end
329
249
  end
data/spec/spec_helper.rb CHANGED
@@ -1,11 +1,102 @@
1
1
  require 'freekiqs'
2
2
 
3
+ $TESTING = true
4
+
5
+ Sidekiq.logger = nil
6
+
7
+ # This file was generated by the `rspec --init` command. Conventionally, all
8
+ # specs live under a `spec` directory, which RSpec adds to the `$LOAD_PATH`.
9
+ # The generated `.rspec` file contains `--require spec_helper` which will cause
10
+ # this file to always be loaded, without a need to explicitly require it in any
11
+ # files.
12
+ #
13
+ # Given that it is always loaded, you are encouraged to keep this file as
14
+ # light-weight as possible. Requiring heavyweight dependencies from this file
15
+ # will add to the boot time of your test suite on EVERY test run, even for an
16
+ # individual file that may not need all of that loaded. Instead, consider making
17
+ # a separate helper file that requires the additional dependencies and performs
18
+ # the additional setup, and require it from the spec files that actually need
19
+ # it.
20
+ #
21
+ # See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
3
22
  RSpec.configure do |config|
23
+ # rspec-expectations config goes here. You can use an alternate
24
+ # assertion/expectation library such as wrong or the stdlib/minitest
25
+ # assertions if you prefer.
4
26
  config.expect_with :rspec do |expectations|
27
+ # This option will default to `true` in RSpec 4. It makes the `description`
28
+ # and `failure_message` of custom matchers include text for helper methods
29
+ # defined using `chain`, e.g.:
30
+ # be_bigger_than(2).and_smaller_than(4).description
31
+ # # => "be bigger than 2 and smaller than 4"
32
+ # ...rather than:
33
+ # # => "be bigger than 2"
5
34
  expectations.include_chain_clauses_in_custom_matcher_descriptions = true
6
35
  end
7
36
 
37
+ # rspec-mocks config goes here. You can use an alternate test double
38
+ # library (such as bogus or mocha) by changing the `mock_with` option here.
8
39
  config.mock_with :rspec do |mocks|
40
+ # Prevents you from mocking or stubbing a method that does not exist on
41
+ # a real object. This is generally recommended, and will default to
42
+ # `true` in RSpec 4.
9
43
  mocks.verify_partial_doubles = true
10
44
  end
45
+
46
+ # This option will default to `:apply_to_host_groups` in RSpec 4 (and will
47
+ # have no way to turn it off -- the option exists only for backwards
48
+ # compatibility in RSpec 3). It causes shared context metadata to be
49
+ # inherited by the metadata hash of host groups and examples, rather than
50
+ # triggering implicit auto-inclusion in groups with matching metadata.
51
+ config.shared_context_metadata_behavior = :apply_to_host_groups
52
+
53
+ # This allows you to limit a spec run to individual examples or groups
54
+ # you care about by tagging them with `:focus` metadata. When nothing
55
+ # is tagged with `:focus`, all examples get run. RSpec also provides
56
+ # aliases for `it`, `describe`, and `context` that include `:focus`
57
+ # metadata: `fit`, `fdescribe` and `fcontext`, respectively.
58
+ config.filter_run_when_matching :focus
59
+
60
+ # Allows RSpec to persist some state between runs in order to support
61
+ # the `--only-failures` and `--next-failure` CLI options. We recommend
62
+ # you configure your source control system to ignore this file.
63
+ # config.example_status_persistence_file_path = "spec/examples.txt"
64
+
65
+ # Limits the available syntax to the non-monkey patched syntax that is
66
+ # recommended. For more details, see:
67
+ # - http://rspec.info/blog/2012/06/rspecs-new-expectation-syntax/
68
+ # - http://www.teaisaweso.me/blog/2013/05/27/rspecs-new-message-expectation-syntax/
69
+ # - http://rspec.info/blog/2014/05/notable-changes-in-rspec-3/#zero-monkey-patching-mode
70
+ config.disable_monkey_patching!
71
+
72
+ # This setting enables warnings. It's recommended, but in some cases may
73
+ # be too noisy due to issues in dependencies.
74
+ config.warnings = true
75
+
76
+ # Many RSpec users commonly either run the entire suite or an individual
77
+ # file, and it's useful to allow more verbose output when running an
78
+ # individual spec file.
79
+ if config.files_to_run.one?
80
+ # Use the documentation formatter for detailed output,
81
+ # unless a formatter has already been configured
82
+ # (e.g. via a command-line flag).
83
+ config.default_formatter = "doc"
84
+ end
85
+
86
+ # Print the 10 slowest examples and example groups at the
87
+ # end of the spec run, to help surface which specs are running
88
+ # particularly slow.
89
+ config.profile_examples = 10
90
+
91
+ # Run specs in random order to surface order dependencies. If you find an
92
+ # order dependency and want to debug it, you can fix the order by providing
93
+ # the seed, which is printed after each run.
94
+ # --seed 1234
95
+ config.order = :random
96
+
97
+ # Seed global randomization in this process using the `--seed` CLI option.
98
+ # Setting this allows you to use `--seed` to deterministically reproduce
99
+ # test failures related to randomization by passing the same `--seed` value
100
+ # as the one that triggered the failure.
101
+ Kernel.srand config.seed
11
102
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: freekiqs
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.1.0
4
+ version: 6.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Rob Lewis
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-06-24 00:00:00.000000000 Z
11
+ date: 2022-10-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: sidekiq
@@ -16,34 +16,28 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 4.0.0
20
- - - "<"
21
- - !ruby/object:Gem::Version
22
- version: 5.0.0
19
+ version: 6.5.0
23
20
  type: :runtime
24
21
  prerelease: false
25
22
  version_requirements: !ruby/object:Gem::Requirement
26
23
  requirements:
27
24
  - - ">="
28
25
  - !ruby/object:Gem::Version
29
- version: 4.0.0
30
- - - "<"
31
- - !ruby/object:Gem::Version
32
- version: 5.0.0
26
+ version: 6.5.0
33
27
  - !ruby/object:Gem::Dependency
34
28
  name: rspec
35
29
  requirement: !ruby/object:Gem::Requirement
36
30
  requirements:
37
31
  - - ">="
38
32
  - !ruby/object:Gem::Version
39
- version: 2.14.1
33
+ version: 3.6.0
40
34
  type: :development
41
35
  prerelease: false
42
36
  version_requirements: !ruby/object:Gem::Requirement
43
37
  requirements:
44
38
  - - ">="
45
39
  - !ruby/object:Gem::Version
46
- version: 2.14.1
40
+ version: 3.6.0
47
41
  description: Sidekiq middleware extending RetryJobs to allow configuring how many
48
42
  exceptions a job can throw and be wrapped by a silenceable exception.
49
43
  email:
@@ -53,14 +47,19 @@ extensions: []
53
47
  extra_rdoc_files: []
54
48
  files:
55
49
  - ".gitignore"
50
+ - ".rspec"
56
51
  - Gemfile
57
52
  - MIT-LICENSE
53
+ - Procfile
58
54
  - README.md
59
55
  - Rakefile
56
+ - Vagrantfile
60
57
  - circle.yml
58
+ - development/redis/redis.conf
61
59
  - freekiqs.gemspec
62
60
  - lib/freekiqs.rb
63
61
  - lib/sidekiq/middleware/server/freekiqs.rb
62
+ - shell.nix
64
63
  - spec/freekiqs_spec.rb
65
64
  - spec/spec_helper.rb
66
65
  homepage: https://github.com/BookBub/freekiqs
@@ -75,15 +74,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
75
74
  requirements:
76
75
  - - ">="
77
76
  - !ruby/object:Gem::Version
78
- version: 1.9.3
77
+ version: 2.2.2
79
78
  required_rubygems_version: !ruby/object:Gem::Requirement
80
79
  requirements:
81
80
  - - ">="
82
81
  - !ruby/object:Gem::Version
83
82
  version: '0'
84
83
  requirements: []
85
- rubyforge_project:
86
- rubygems_version: 2.5.1
84
+ rubygems_version: 3.0.3.1
87
85
  signing_key:
88
86
  specification_version: 4
89
87
  summary: Sidekiq middleware extending RetryJobs to allow silient errors.