sidekiq-job-manager 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 33349e621f83f560be352fce901cadcab2899217
4
+ data.tar.gz: 37a8d1a936172860dcf4ceef293d6885d37857f6
5
+ SHA512:
6
+ metadata.gz: d95b1cadc97b81b499f0d033da06e3f128277ebee7e639d438f477e36536507431646e38484c8c84c57ff3b97ff9e018ac48a7a1b6eb04f74d04fd77f05ac2fd
7
+ data.tar.gz: 74d819970b40058087d54ce05a73c6d6e56b27ee154513867a5f95600a438946d77db4561744b1d9cf14f53d54682bb2910797f9069751a528a46efbe133f036
data/.gitignore ADDED
@@ -0,0 +1,16 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ InstalledFiles
7
+ _yardoc
8
+ coverage
9
+ doc/
10
+ lib/bundler/man
11
+ pkg
12
+ rdoc
13
+ spec/reports
14
+ test/tmp
15
+ test/version_tmp
16
+ tmp
data/.ruby-version ADDED
@@ -0,0 +1 @@
1
+ ruby-2.0.0-p195
data/CHANGELOG.md ADDED
File without changes
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in sidekiq-failures.gemspec
4
+ gemspec
data/Gemfile.lock ADDED
@@ -0,0 +1,56 @@
1
+ PATH
2
+ remote: .
3
+ specs:
4
+ sidekiq-job-manager (0.1.0)
5
+ sidekiq (>= 2.9.0)
6
+
7
+ GEM
8
+ remote: https://rubygems.org/
9
+ specs:
10
+ celluloid (0.14.1)
11
+ timers (>= 1.0.0)
12
+ connection_pool (1.1.0)
13
+ hike (1.2.1)
14
+ json (1.8.0)
15
+ multi_json (1.7.3)
16
+ rack (1.4.1)
17
+ rack-protection (1.2.0)
18
+ rack
19
+ rack-test (0.6.2)
20
+ rack (>= 1.0)
21
+ rake (0.9.2.2)
22
+ redis (3.0.4)
23
+ redis-namespace (1.3.0)
24
+ redis (~> 3.0.0)
25
+ sidekiq (2.13.0)
26
+ celluloid (>= 0.14.1)
27
+ connection_pool (>= 1.0.0)
28
+ json
29
+ redis (>= 3.0)
30
+ redis-namespace
31
+ sinatra (1.3.3)
32
+ rack (~> 1.3, >= 1.3.6)
33
+ rack-protection (~> 1.2)
34
+ tilt (~> 1.3, >= 1.3.3)
35
+ slim (1.3.4)
36
+ temple (~> 0.5.5)
37
+ tilt (~> 1.3.3)
38
+ sprockets (2.8.1)
39
+ hike (~> 1.2)
40
+ multi_json (~> 1.0)
41
+ rack (~> 1.0)
42
+ tilt (~> 1.1, != 1.3.0)
43
+ temple (0.5.5)
44
+ tilt (1.3.3)
45
+ timers (1.1.0)
46
+
47
+ PLATFORMS
48
+ ruby
49
+
50
+ DEPENDENCIES
51
+ rack-test
52
+ rake
53
+ sidekiq-job-manager!
54
+ sinatra
55
+ slim
56
+ sprockets
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2012 Marcelo Silveira
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,132 @@
1
+ # Sidekiq::Failures [![Build Status](https://secure.travis-ci.org/mhfs/sidekiq-failures.png)](http://travis-ci.org/mhfs/sidekiq-failures)
2
+
3
+ Keeps track of Sidekiq failed jobs and adds a tab to the Web UI to let you browse
4
+ them. Makes use of Sidekiq's custom tabs and middleware chain.
5
+
6
+ It mimics the way Resque keeps track of failures.
7
+
8
+ WARNING: by default sidekiq-failures will keep up to 1000 failures. See [Maximum Tracked Failures](https://github.com/mhfs/sidekiq-failures#maximum-tracked-failures) below.
9
+
10
+ ## Installation
11
+
12
+ Add this line to your application's Gemfile:
13
+
14
+ ```ruby
15
+ gem 'sidekiq-failures'
16
+ ```
17
+
18
+ ## Usage
19
+
20
+ Simply having the gem in your Gemfile is enough to get you started. Your failed
21
+ jobs will be visible via a Failures tab in the Web UI.
22
+
23
+ ## Configuring
24
+
25
+ ### Maximum Tracked Failures
26
+
27
+ Since each failed job/retry creates a new failure entry that will only be removed
28
+ by you manually, your failures list might consume more resources than you have
29
+ available.
30
+
31
+ To avoid this sidekiq-failures adopts a default of 1000 maximum tracked failures.
32
+
33
+ To change the maximum amount:
34
+
35
+ ```ruby
36
+ Sidekiq.configure_server do |config|
37
+ config.failures_max_count = 5000
38
+ end
39
+ ```
40
+
41
+ To disable the limit entirely:
42
+
43
+ ```ruby
44
+ Sidekiq.configure_server do |config|
45
+ config.failures_max_count = false
46
+ end
47
+ ```
48
+
49
+ ### Failures Tracking Mode
50
+
51
+ Sidekiq-failures offers three failures tracking options (per worker):
52
+
53
+
54
+ #### :all (default)
55
+
56
+ Tracks failures every time a background job fails. This mean a job with 25 retries
57
+ enabled might generate up to 25 failure entries. If the worker has retry disabled
58
+ only one failure will be tracked.
59
+
60
+ This is the default behavior but can be made explicit with:
61
+
62
+ ```ruby
63
+ class MyWorker
64
+ include Sidekiq::Worker
65
+
66
+ sidekiq_options :failures => true # or :all
67
+
68
+ def perform; end
69
+ end
70
+ ```
71
+
72
+ #### :exhausted
73
+
74
+ Only track failures if the job exhausts all its retries (or doesn't have retries
75
+ enabled).
76
+
77
+ You can set this mode as follows:
78
+
79
+ ```ruby
80
+ class MyWorker
81
+ include Sidekiq::Worker
82
+
83
+ sidekiq_options :failures => :exhausted
84
+
85
+ def perform; end
86
+ end
87
+ ```
88
+
89
+ #### :off
90
+
91
+ You can also completely turn off failures tracking for a given worker as follows:
92
+
93
+ ```ruby
94
+ class MyWorker
95
+ include Sidekiq::Worker
96
+
97
+ sidekiq_options :failures => false # or :off
98
+
99
+ def perform; end
100
+ end
101
+ ```
102
+
103
+ #### Change the default mode
104
+
105
+ You can also change the default of all your workers at once by setting the following
106
+ server config:
107
+
108
+ ```ruby
109
+ Sidekiq.configure_server do |config|
110
+ config.failures_default_mode = :off
111
+ end
112
+ ```
113
+
114
+ The valid modes are `:all`, `:exhausted` or `:off`.
115
+
116
+ ## Dependencies
117
+
118
+ Depends on Sidekiq >= 2.9.0
119
+
120
+ ## Contributing
121
+
122
+ 1. Fork it
123
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
124
+ 3. Commit your changes (`git commit -am 'Added some feature'`)
125
+ 4. Push to the branch (`git push origin my-new-feature`)
126
+ 5. Create new Pull Request
127
+
128
+ ## License
129
+
130
+ Released under the MIT License. See the [LICENSE][license] file for further details.
131
+
132
+ [license]: https://github.com/mhfs/sidekiq-failures/blob/master/LICENSE
data/Rakefile ADDED
@@ -0,0 +1,12 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
3
+ require "rake/testtask"
4
+
5
+ task :default => :test
6
+
7
+ Rake::TestTask.new do |t|
8
+ t.libs << "lib"
9
+ t.libs << "test"
10
+ t.test_files = FileList["test/**/*_test.rb"]
11
+ t.verbose = true
12
+ end
data/config.ru ADDED
@@ -0,0 +1,37 @@
1
+ # Used for debugging Web UI
2
+ #
3
+ # bundle exec rackup config.ru
4
+ #
5
+ require 'sidekiq'
6
+ require 'sidekiq/web'
7
+ require './lib/sidekiq-manager'
8
+
9
+ REDIS = Sidekiq::RedisConnection.create(:url => "redis://localhost/15", :namespace => 'sidekiq_manager_test')
10
+ Sidekiq.redis = REDIS
11
+ Sidekiq.redis {|c| c.flushdb }
12
+
13
+
14
+ 100.times do |t|
15
+ data = {
16
+ :finished_at => Time.at(Time.now.to_i-rand(60*t)).strftime("%Y/%m/%d %H:%M:%S %Z"),
17
+ :payload => { :args => ["test", rand(10)], :class => "Worker#{rand(5)}" },
18
+ :queue => 'default'
19
+ }
20
+
21
+ if rand(3) == 0
22
+ data[:error] = {
23
+ :exception => "ArgumentError",
24
+ :error => "Some new message",
25
+ :backtrace => ["path/file1.rb", "path/file2.rb"]
26
+ }
27
+ end
28
+
29
+ Sidekiq.redis do |c|
30
+ c.multi do
31
+ c.zadd(:unique_jobs, 0, data[:payload][:class])
32
+ c.lpush("#{data[:payload][:class]}:details", Sidekiq.dump_json(data))
33
+ end
34
+ end
35
+ end
36
+
37
+ run Sidekiq::Web
@@ -0,0 +1,62 @@
1
+ module Sidekiq
2
+ module JobManager
3
+
4
+ class Middleware
5
+ include Sidekiq::Util
6
+ attr_accessor :msg
7
+
8
+ def call(worker, msg, queue)
9
+ error = nil
10
+
11
+ self.msg = msg
12
+ yield
13
+ rescue => e
14
+ error = {
15
+ :exception => e.class.to_s,
16
+ :error => e.message,
17
+ :backtrace => e.backtrace
18
+ }
19
+
20
+ raise e
21
+ ensure
22
+ data = {
23
+ :finished_at => Time.now.strftime("%Y/%m/%d %H:%M:%S %Z"),
24
+ :payload => msg,
25
+ :queue => queue,
26
+ :error => error
27
+ }
28
+
29
+ Sidekiq.redis do |conn|
30
+ conn.zadd(:unique_jobs, 0, msg['class'])
31
+ conn.lpush("#{msg['class']}:details", Sidekiq.dump_json(data))
32
+
33
+ unless Sidekiq.job_details_max_count == false
34
+ conn.ltrim("#{msg['class']}:details", 0, Sidekiq.job_details_max_count - 1)
35
+ end
36
+ end
37
+ end
38
+
39
+ private
40
+
41
+ def last_try?
42
+ !msg['retry'] || retry_count == max_retries - 1
43
+ end
44
+
45
+ def retry_count
46
+ msg['retry_count'] || 0
47
+ end
48
+
49
+ def max_retries
50
+ retry_middleware.retry_attempts_from(msg['retry'], default_max_retries)
51
+ end
52
+
53
+ def retry_middleware
54
+ @retry_middleware ||= Sidekiq::Middleware::Server::RetryJobs.new
55
+ end
56
+
57
+ def default_max_retries
58
+ Sidekiq::Middleware::Server::RetryJobs::DEFAULT_MAX_RETRY_ATTEMPTS
59
+ end
60
+ end
61
+ end
62
+ end
@@ -0,0 +1,5 @@
1
+ module Sidekiq
2
+ module JobManager
3
+ VERSION = "0.1.0"
4
+ end
5
+ end
@@ -0,0 +1,61 @@
1
+ header.row
2
+ .span5
3
+ h3 Last calls of #{@worker} worker
4
+ .span3
5
+ input.filter type="search" style="margin: 12px 0;margin-top: 18px;" placeholder="Filter"
6
+ .span2
7
+ - if @messages.size > 0
8
+ == slim :_paging, :locals => { :url => "#{root_path}manager/worker/#{@worker}#@name" }
9
+
10
+ - if @messages.size > 0
11
+ table class="table table-striped table-bordered table-white" style="width: 100%; margin: 0; table-layout:fixed;"
12
+ thead
13
+ th style="width: 25%" Payload
14
+ th style="width: 10%" Queue
15
+ th style="width: 15%" Failed At
16
+ th style="width: 38%" Exception
17
+ th style="width: 12%" Action
18
+ - @messages.each do |msg|
19
+ tr class="#{msg['error'].nil? ? '' : 'error'}"
20
+ td style="overflow: hidden; text-overflow: ellipsis;"
21
+ = msg['payload']
22
+ td= msg['queue']
23
+ td
24
+ time datetime="#{Time.parse(msg['finished_at']).getutc.iso8601}"
25
+ = msg['finished_at']
26
+ td style="overflow: auto; padding: 10px;"
27
+ - if msg['error']
28
+ a.backtrace href="#" onclick="$(this).next().toggle(); return false" = "#{msg['error']['exception']}: #{msg['error']['error']}"
29
+ pre style="display: none; background: none; border: 0; width: 100%; max-height: 30em; font-size: 0.8em; white-space: nowrap;" == msg['error']['backtrace'].join("<br />")
30
+ td
31
+ input.btn.btn-small.add-to-queue type="button" name="run" value="Add to queue" data-args="#{msg['payload']['args'].join(',')}" data-worker="#{@worker}" data-queue="#{msg['queue']}"
32
+
33
+ div.row
34
+ .span5
35
+ form.form-inline action="#{root_path}manager/worker/#{@worker}/remove" method="post" style="margin: 20px 0"
36
+ input.btn.btn-danger.btn-small type="submit" name="delete" value="Clear All"
37
+ .span4
38
+ == slim :_paging, :locals => { :url => "#{root_path}manager/worker/#{@worker}#@name" }
39
+ - else
40
+ .alert.alert-success No failed jobs found.
41
+
42
+ javascript:
43
+ $('input.filter').live('keyup', function() {
44
+ var rex = new RegExp($(this).val(), 'i');
45
+ $('.table-striped tbody tr').hide();
46
+ $('.table-striped tbody tr').filter(function() {
47
+ return rex.test($(this).text());
48
+ }).show();
49
+ });
50
+
51
+ $('input.add-to-queue').live('click', function(){
52
+ var params = {
53
+ args: prompt('Enter arguments, comma separated', $(this).data('args')),
54
+ worker: $(this).data('worker'),
55
+ queue: $(this).data('queue')
56
+ }
57
+
58
+ $.post('#{root_path}manager/add_to_queue', params, function(data) {
59
+ window.location = "#{root_path}queues/"+params['queue']
60
+ });
61
+ })
@@ -0,0 +1,36 @@
1
+ header.row
2
+ .span5
3
+ h3 Recent Jobs
4
+
5
+ - if @jobs.length > 0
6
+ table class="table table-striped table-bordered table-white" style="width: 100%; margin: 0; table-layout:fixed;"
7
+ thead
8
+ th style="width: 44%" Worker
9
+ th style="width: 44%" Last run
10
+ th style="widht: 12%" Action
11
+ - @jobs.each do |job|
12
+ tr class="#{job[:last_call]['error'].nil? ? '' : 'error'}"
13
+ td style="overflow: hidden; text-overflow: ellipsis;"
14
+ a href="#{root_path}manager/worker/#{job[:name]}"
15
+ = job[:name]
16
+ td
17
+ time datetime="#{Time.parse(job[:last_call]['finished_at']).getutc.iso8601}"
18
+ = job[:last_call]['finished_at']
19
+
20
+ td
21
+ input.btn.btn-small.add-to-queue type="button" name="run" value="Add to queue" data-worker="#{job[:name]}" data-queue="#{job[:last_call]['queue']}"
22
+ - else
23
+ .alert.alert-success No failed jobs found.
24
+
25
+ javascript:
26
+ $('input.add-to-queue').live('click', function(){
27
+ var params = {
28
+ args: prompt('Enter arguments, comma separated', $(this).data('args')),
29
+ worker: $(this).data('worker'),
30
+ queue: $(this).data('queue')
31
+ }
32
+
33
+ $.post('#{root_path}manager/add_to_queue', params, function(data) {
34
+ window.location = "#{root_path}queues/"+params['queue']
35
+ });
36
+ })
@@ -0,0 +1,58 @@
1
+ module Sidekiq
2
+ module JobManager
3
+ module WebExtension
4
+
5
+ def self.registered(app)
6
+ app.get "/manager" do
7
+ view_path = File.join(File.expand_path("..", __FILE__), "views")
8
+
9
+ @jobs = Sidekiq.redis { |conn|conn.zrange('unique_jobs',0,-1) } || []
10
+
11
+ @jobs.map! do |job|
12
+ last_call = Sidekiq.redis { |conn|conn.lindex("#{job}:details",0) }
13
+
14
+ {
15
+ name: job,
16
+ last_call: Sidekiq.load_json(last_call)
17
+ }
18
+ end
19
+
20
+ render(:slim, File.read(File.join(view_path, "manager.slim")))
21
+ end
22
+
23
+ app.get "/manager/worker/:name" do |name|
24
+ @worker = name
25
+ view_path = File.join(File.expand_path("..", __FILE__), "views")
26
+
27
+ @count = (params[:count] || 50).to_i
28
+ (@current_page, @total_size, @messages) = page("#{name}:details", params[:page], @count)
29
+ @messages = @messages.map { |msg| Sidekiq.load_json(msg) }
30
+
31
+ render(:slim, File.read(File.join(view_path, "job_details.slim")))
32
+ end
33
+
34
+ app.post "/manager/worker/:name/remove" do |name|
35
+ Sidekiq.redis {|c|
36
+ c.multi do
37
+ c.del("#{name}:details")
38
+ c.zrem("unique_jobs", name)
39
+ end
40
+ }
41
+
42
+ redirect "#{root_path}manager"
43
+ end
44
+
45
+ app.post "/manager/add_to_queue" do
46
+ msg = {
47
+ 'class' => params[:worker],
48
+ 'args' => params[:args].split(','),
49
+ 'queue' => params[:queue] || "default",
50
+ 'retry' => false
51
+ }
52
+
53
+ Sidekiq::Client.push(msg)
54
+ end
55
+ end
56
+ end
57
+ end
58
+ end
@@ -0,0 +1,71 @@
1
+ begin
2
+ require "sidekiq/web"
3
+ rescue LoadError
4
+ # client-only usage
5
+ end
6
+
7
+ require "sidekiq/job-manager/version"
8
+ require "sidekiq/job-manager/middleware"
9
+ require "sidekiq/job-manager/web_extension"
10
+
11
+ module Sidekiq
12
+
13
+ SIDEKIQ_FAILURES_MODES = [:all, :exhausted, :off].freeze
14
+
15
+ # Sets the default failure tracking mode.
16
+ #
17
+ # The value provided here will be the default behavior but can be overwritten
18
+ # per worker by using `sidekiq_options :failures => :mode`
19
+ #
20
+ # Defaults to :all
21
+ def self.failures_default_mode=(mode)
22
+ unless SIDEKIQ_FAILURES_MODES.include?(mode.to_sym)
23
+ raise ArgumentError, "Sidekiq#failures_default_mode valid options: #{SIDEKIQ_FAILURES_MODES}"
24
+ end
25
+
26
+ @failures_default_mode = mode.to_sym
27
+ end
28
+
29
+ # Fetches the default failure tracking mode.
30
+ def self.failures_default_mode
31
+ @failures_default_mode || :all
32
+ end
33
+
34
+ # Sets the maximum number of failures to track
35
+ #
36
+ # If the number of failures exceeds this number the list will be trimmed (oldest
37
+ # failures will be purged).
38
+ #
39
+ # Defaults to 1000
40
+ # Set to false to disable rotation
41
+ def self.job_details_max_count=(value)
42
+ @job_details_max_count = value
43
+ end
44
+
45
+ # Fetches the failures max count value
46
+ def self.job_details_max_count
47
+ return 1000 if @job_details_max_count.nil?
48
+
49
+ @job_details_max_count
50
+ end
51
+
52
+ module JobManager
53
+ end
54
+ end
55
+
56
+ Sidekiq.configure_server do |config|
57
+ config.server_middleware do |chain|
58
+ chain.add Sidekiq::JobManager::Middleware
59
+ end
60
+ end
61
+
62
+ if defined?(Sidekiq::Web)
63
+ Sidekiq::Web.register Sidekiq::JobManager::WebExtension
64
+
65
+ if Sidekiq::Web.tabs.is_a?(Array)
66
+ # For sidekiq < 2.5
67
+ Sidekiq::Web.tabs << "manager"
68
+ else
69
+ Sidekiq::Web.tabs["Job Manager"] = "manager"
70
+ end
71
+ end
@@ -0,0 +1 @@
1
+ require "sidekiq/job-manager"
@@ -0,0 +1,25 @@
1
+ # -*- encoding: utf-8 -*-
2
+ require File.expand_path('../lib/sidekiq/job-manager/version', __FILE__)
3
+
4
+ Gem::Specification.new do |gem|
5
+ gem.authors = ["Leonid Bugaev"]
6
+ gem.email = ["leonsbox@gmail.com"]
7
+ gem.description = %q{Manage your sidekiq jobs}
8
+ gem.summary = %q{Keeps track of Sidekiq jobs and adds a tab to the Web UI to let you browse them.}
9
+ gem.homepage = "https://github.com/buger/sidekiq-job-manager/"
10
+
11
+ gem.files = `git ls-files`.split($\)
12
+ gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
13
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
14
+ gem.name = "sidekiq-job-manager"
15
+ gem.require_paths = ["lib"]
16
+ gem.version = Sidekiq::JobManager::VERSION
17
+
18
+ gem.add_dependency "sidekiq", ">= 2.9.0"
19
+
20
+ gem.add_development_dependency "rake"
21
+ gem.add_development_dependency "rack-test"
22
+ gem.add_development_dependency "sprockets"
23
+ gem.add_development_dependency "sinatra"
24
+ gem.add_development_dependency "slim"
25
+ end
@@ -0,0 +1,127 @@
1
+ require "test_helper"
2
+ require "sidekiq"
3
+
4
+ module Sidekiq
5
+ module JobManager
6
+ describe "Middleware" do
7
+ before do
8
+ $invokes = 0
9
+ Sidekiq.job_details_max_count = nil
10
+ @boss = MiniTest::Mock.new
11
+ @processor = ::Sidekiq::Processor.new(@boss)
12
+ Sidekiq.server_middleware {|chain| chain.add Sidekiq::JobManager::Middleware }
13
+ Sidekiq.redis = REDIS
14
+ Sidekiq.redis { |c| c.flushdb }
15
+ Sidekiq.instance_eval { @failures_default_mode = nil }
16
+ end
17
+
18
+ TestException = Class.new(StandardError)
19
+ ShutdownException = Class.new(Sidekiq::Shutdown)
20
+
21
+ class MockWorker
22
+ include Sidekiq::Worker
23
+
24
+ def perform(*args)
25
+ $invokes += 1
26
+
27
+ if args[1] != "success"
28
+ raise ShutdownException.new if args[0] == "shutdown"
29
+ raise TestException.new("failed!")
30
+ end
31
+ end
32
+ end
33
+
34
+ class MockWorker1 < MockWorker
35
+ end
36
+
37
+ class MockWorker2 < MockWorker
38
+ end
39
+
40
+ # TESTS FOR MANAGER
41
+ it 'should update active jobs' do
42
+ assert_equal 0, unique_jobs_count
43
+
44
+ [MockWorker, MockWorker, MockWorker1, MockWorker2].each do |worker|
45
+ processor = mock_actor
46
+ msg = create_work('class' => worker.to_s, 'args' => ['myarg', 'success'])
47
+ processor.process(msg)
48
+ end
49
+
50
+ assert_equal 3, unique_jobs_count
51
+ end
52
+
53
+ it 'should update job details' do
54
+ assert_equal 0, job_details(MockWorker.to_s).length
55
+
56
+ 2.times do
57
+ processor = mock_actor
58
+ msg = create_work('class' => MockWorker.to_s, 'args' => ['myarg', 'success'])
59
+ processor.process(msg)
60
+ end
61
+
62
+ assert_equal 2, job_details(MockWorker.to_s).length
63
+ end
64
+
65
+ it 'should update both success and failed runs' do
66
+ assert_equal 0, job_details(MockWorker.to_s).length
67
+
68
+ 2.times do
69
+ processor = mock_actor
70
+ msg = create_work('class' => MockWorker.to_s, 'args' => ['myarg', 'success'])
71
+ processor.process(msg)
72
+ end
73
+
74
+ 3.times do
75
+ processor = mock_actor
76
+ msg = create_work('class' => MockWorker.to_s, 'args' => ['myarg'])
77
+
78
+ assert_raises TestException do
79
+ processor.process(msg)
80
+ end
81
+ end
82
+
83
+ details = job_details(MockWorker.to_s)
84
+
85
+ assert_equal 3, details.select{|j| j['error']}.length
86
+ assert_equal 2, details.select{|j| !j['error']}.length
87
+ end
88
+
89
+ it "removes old details when job_details_max_count has been reached" do
90
+ assert_equal 1000, Sidekiq.job_details_max_count
91
+ Sidekiq.job_details_max_count = 2
92
+
93
+ 5.times do
94
+ processor = mock_actor
95
+ msg = create_work('class' => MockWorker.to_s, 'args' => ['myarg', 'success'])
96
+ processor.process(msg)
97
+ end
98
+
99
+ assert_equal 2, job_details(MockWorker.to_s).length
100
+ end
101
+
102
+ def create_work(msg)
103
+ Sidekiq::BasicFetch::UnitOfWork.new('default', Sidekiq.dump_json(msg))
104
+ end
105
+
106
+ def unique_jobs_count
107
+ Sidekiq.redis { |conn|conn.zcard('unique_jobs') } || 0
108
+ end
109
+
110
+ def job_details job_name
111
+ Sidekiq.redis { |conn|conn.lrange("#{job_name}:details", 0, 100).map{|j| Sidekiq.load_json(j)} }
112
+ end
113
+
114
+ def mock_actor
115
+ boss = MiniTest::Mock.new
116
+ processor = ::Sidekiq::Processor.new(boss)
117
+
118
+ actor = MiniTest::Mock.new
119
+ actor.expect(:processor_done, nil, [processor])
120
+ actor.expect(:real_thread, nil, [nil, Celluloid::Thread])
121
+ 2.times { boss.expect(:async, actor, []) }
122
+
123
+ return processor
124
+ end
125
+ end
126
+ end
127
+ end
@@ -0,0 +1,27 @@
1
+ Encoding.default_external = Encoding::UTF_8
2
+ Encoding.default_internal = Encoding::UTF_8
3
+
4
+ require "minitest/autorun"
5
+ require "minitest/spec"
6
+ require "minitest/mock"
7
+
8
+ # FIXME Remove once https://github.com/mperham/sidekiq/pull/548 is released.
9
+ class String
10
+ def blank?
11
+ self !~ /[^[:space:]]/
12
+ end
13
+ end
14
+
15
+ require "rack/test"
16
+
17
+ require "celluloid"
18
+ require "sidekiq"
19
+ require "sidekiq-manager"
20
+ require "sidekiq/processor"
21
+ require "sidekiq/fetch"
22
+ require "sidekiq/cli"
23
+
24
+ Celluloid.logger = nil
25
+ Sidekiq.logger.level = Logger::ERROR
26
+
27
+ REDIS = Sidekiq::RedisConnection.create(:url => "redis://localhost/15", :namespace => 'sidekiq_manager_test')
@@ -0,0 +1,72 @@
1
+ require "test_helper"
2
+ require "sidekiq/web"
3
+
4
+ module Sidekiq
5
+ describe "WebExtension" do
6
+ include Rack::Test::Methods
7
+
8
+ def app
9
+ Sidekiq::Web
10
+ end
11
+
12
+ before do
13
+ Sidekiq.redis = REDIS
14
+ Sidekiq.redis {|c| c.flushdb }
15
+ end
16
+
17
+ it 'can display home with manager tab' do
18
+ get '/'
19
+
20
+ last_response.status.must_equal 200
21
+ last_response.body.must_match /Sidekiq/
22
+ last_response.body.must_match /Job Manager/
23
+ end
24
+
25
+ it 'can display jobs page without any failures' do
26
+ get '/manager'
27
+ last_response.status.must_equal 200
28
+ last_response.body.must_match /Recent jobs/
29
+ last_response.body.must_match /No jobs found/
30
+ end
31
+
32
+ describe 'when there are jobs' do
33
+ before do
34
+ create_sample_failure
35
+ get '/manager'
36
+ end
37
+
38
+ it 'should be successful' do
39
+ last_response.status.must_equal 200
40
+ end
41
+
42
+ it 'can display failures page with failures listed' do
43
+
44
+ end
45
+ end
46
+
47
+ def create_sample_failure
48
+ data = {
49
+ :finished_at => Time.now.strftime("%Y/%m/%d %H:%M:%S %Z"),
50
+ :payload => { :args => ["test", 5], :class => 'Worker' },
51
+ :error => {
52
+ :exception => "ArgumentError",
53
+ :error => "Some new message",
54
+ :backtrace => ["path/file1.rb", "path/file2.rb"]
55
+ },
56
+ :queue => 'default'
57
+ }
58
+
59
+ Sidekiq.redis do |c|
60
+ c.multi do
61
+ c.zadd(:unique_jobs, 0, data[:payload][:class])
62
+ c.lpush("#{data[:payload][:class]}:details", Sidekiq.dump_json(data))
63
+ end
64
+ end
65
+ end
66
+
67
+ def unique_jobs_count
68
+ Sidekiq.redis { |conn|conn.zcard('unique_jobs') } || 0
69
+ end
70
+
71
+ end
72
+ end
metadata ADDED
@@ -0,0 +1,151 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: sidekiq-job-manager
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Leonid Bugaev
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2013-08-16 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: sidekiq
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - '>='
18
+ - !ruby/object:Gem::Version
19
+ version: 2.9.0
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - '>='
25
+ - !ruby/object:Gem::Version
26
+ version: 2.9.0
27
+ - !ruby/object:Gem::Dependency
28
+ name: rake
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - '>='
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - '>='
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: rack-test
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - '>='
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - '>='
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: sprockets
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - '>='
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - '>='
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: sinatra
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - '>='
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - '>='
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: slim
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - '>='
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - '>='
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ description: Manage your sidekiq jobs
98
+ email:
99
+ - leonsbox@gmail.com
100
+ executables: []
101
+ extensions: []
102
+ extra_rdoc_files: []
103
+ files:
104
+ - .gitignore
105
+ - .ruby-version
106
+ - CHANGELOG.md
107
+ - Gemfile
108
+ - Gemfile.lock
109
+ - LICENSE
110
+ - README.md
111
+ - Rakefile
112
+ - config.ru
113
+ - lib/sidekiq-manager.rb
114
+ - lib/sidekiq/job-manager.rb
115
+ - lib/sidekiq/job-manager/middleware.rb
116
+ - lib/sidekiq/job-manager/version.rb
117
+ - lib/sidekiq/job-manager/views/job_details.slim
118
+ - lib/sidekiq/job-manager/views/manager.slim
119
+ - lib/sidekiq/job-manager/web_extension.rb
120
+ - sidekiq-job-manager.gemspec
121
+ - test/middleware_test.rb
122
+ - test/test_helper.rb
123
+ - test/web_extension_test.rb
124
+ homepage: https://github.com/buger/sidekiq-job-manager/
125
+ licenses: []
126
+ metadata: {}
127
+ post_install_message:
128
+ rdoc_options: []
129
+ require_paths:
130
+ - lib
131
+ required_ruby_version: !ruby/object:Gem::Requirement
132
+ requirements:
133
+ - - '>='
134
+ - !ruby/object:Gem::Version
135
+ version: '0'
136
+ required_rubygems_version: !ruby/object:Gem::Requirement
137
+ requirements:
138
+ - - '>='
139
+ - !ruby/object:Gem::Version
140
+ version: '0'
141
+ requirements: []
142
+ rubyforge_project:
143
+ rubygems_version: 2.0.3
144
+ signing_key:
145
+ specification_version: 4
146
+ summary: Keeps track of Sidekiq jobs and adds a tab to the Web UI to let you browse
147
+ them.
148
+ test_files:
149
+ - test/middleware_test.rb
150
+ - test/test_helper.rb
151
+ - test/web_extension_test.rb