delayed_job 4.0.6 → 4.1.0

This diff has not been reviewed by any users.
Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 437dd17835ae0e764f1fbd9ce13c55be2b7ddead
4
- data.tar.gz: abb136834dd2e39edb5a6cf5c846613734c3b518
3
+ metadata.gz: 088a5a5e5912c5a02be5f22d9aa48cc28813396d
4
+ data.tar.gz: 5c50a72be0ce0ad140ced22858f9f4ccde83d5c7
5
5
  SHA512:
6
- metadata.gz: b89e00ad9d939277b9f2da9e94a1b45310c9b7ea3b7e3f7535f434eb8b9ad46a6589ef541e76501e5c2da07cb982fb337453d6c5b642b2e20274daa1c0321452
7
- data.tar.gz: 3cb7e586ac2a1a7ea5967d5e35bf3cd325943bf37e28e913e77a787363c0efa46ab891c07c5f157b88a19f8bfda5444839b9b8071e9bc1feb8bcb715773089f9
6
+ metadata.gz: bc8b0b28d3f6fb68c32549cbace9f0e222308d5537c9f3efc7db3a06b568569c460909ca7225d2fe34c3df14bd6d9cb104fec23e60f8885661bbfba1d3308384
7
+ data.tar.gz: 28050ab20af638f0cb4facc41d410f08d6e8d2becbde4d157c634371cb2b99588b41bdc29eba156499430de0fcc6045f906a9fd8c883bdfb1bf55e10cc55805e
@@ -1,3 +1,13 @@
1
+ 4.1.0 - 2015-09-22
2
+ ==================
3
+ * Alter `Delayed::Command` to work with or without Rails
4
+ * Allow `Delayed::Worker.delay_jobs` configuration to be a proc
5
+ * Add ability to set destroy failed jobs on a per job basis
6
+ * Make `Delayed::Worker.new` idempotent
7
+ * Set quiet from the environment
8
+ * Rescue `Exception` instead of `StandardError` in worker
9
+ * Fix worker crash on serialization error
10
+
1
11
  4.0.6 - 2014-12-22
2
12
  ==================
3
13
  * Revert removing test files from the gem
data/README.md CHANGED
@@ -1,3 +1,8 @@
1
+ **If you're viewing this at https://github.com/collectiveidea/delayed_job,
2
+ you're reading the documentation for the master branch.
3
+ [View documentation for the latest release
4
+ (4.0.6).](https://github.com/collectiveidea/delayed_job/tree/v4.0.6)**
5
+
1
6
  Delayed::Job
2
7
  ============
3
8
  [![Gem Version](https://badge.fury.io/rb/delayed_job.png)][gem]
@@ -58,14 +63,30 @@ running the following command:
58
63
  rails generate delayed_job:active_record
59
64
  rake db:migrate
60
65
 
66
+ For Rails 4.2, see [below](#rails-42)
67
+
61
68
  Development
62
69
  ===========
63
70
  In development mode, if you are using Rails 3.1+, your application code will automatically reload every 100 jobs or when the queue finishes.
64
71
  You no longer need to restart Delayed Job every time you update your code in development.
65
72
 
66
- Rails 4
67
- =======
68
- If you are using the protected_attributes gem, it must appear before delayed_job in your gemfile.
73
+ Rails 4.2
74
+ =========
75
+ Set the queue_adapter in config/application.rb
76
+
77
+ ```ruby
78
+ config.active_job.queue_adapter = :delayed_job
79
+ ```
80
+
81
+ See the [rails guide](http://guides.rubyonrails.org/active_job_basics.html#setting-the-backend) for more details.
82
+
83
+ Rails 4.x
84
+ =========
85
+ If you are using the protected_attributes gem, it must appear before delayed_job in your gemfile. If your jobs are failing with:
86
+
87
+ ActiveRecord::StatementInvalid: PG::NotNullViolation: ERROR: null value in column "handler" violates not-null constraint
88
+
89
+ then this is the fix you're looking for.
69
90
 
70
91
  Upgrading from 2.x to 3.0.0 on Active Record
71
92
  ============================================
@@ -103,9 +124,17 @@ device = Device.new
103
124
  device.deliver
104
125
  ```
105
126
 
106
- handle_asynchronously can take as options anything you can pass to delay. In
107
- addition, the values can be Proc objects allowing call time evaluation of the
108
- value. For some examples:
127
+ ## Parameters
128
+
129
+ `#handle_asynchronously` and `#delay` take these parameters:
130
+
131
+ - `:priority` (number): lower numbers run first; default is 0 but can be reconfigured (see below)
132
+ - `:run_at` (Time): run the job after this time (probably in the future)
133
+ - `:queue` (string): named queue to put this job in, an alternative to priorities (see below)
134
+
135
+ These params can be Proc objects, allowing call-time evaluation of the value.
136
+
137
+ For example:
109
138
 
110
139
  ```ruby
111
140
  class LongTasks
@@ -282,6 +311,20 @@ NewsletterJob = Struct.new(:text, :emails) do
282
311
  end
283
312
  ```
284
313
 
314
+ To set a per-job default for destroying failed jobs that overrides the Delayed::Worker.destroy_failed_jobs you can define a destroy_failed_jobs? method on the job
315
+
316
+ ```ruby
317
+ NewsletterJob = Struct.new(:text, :emails) do
318
+ def perform
319
+ emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
320
+ end
321
+
322
+ def destroy_failed_jobs?
323
+ false
324
+ end
325
+ end
326
+ ```
327
+
285
328
  To set a default queue name for a custom job that overrides Delayed::Worker.default_queue_name, you can define a queue_name method on the job
286
329
 
287
330
  ```ruby
@@ -384,6 +427,14 @@ By default all jobs will be queued without a named queue. A default named queue
384
427
 
385
428
  It is possible to disable delayed jobs for testing purposes. Set `Delayed::Worker.delay_jobs = false` to execute all jobs realtime.
386
429
 
430
+ Or `Delayed::Worker.delay_jobs` can be a Proc that decides whether to execute jobs inline on a per-job basis:
431
+
432
+ ```ruby
433
+ Delayed::Worker.delay_jobs = ->(job) {
434
+ job.queue != 'inline'
435
+ }
436
+ ```
437
+
387
438
  You may need to raise exceptions on SIGTERM signals, `Delayed::Worker.raise_signal_exceptions = :term` will cause the worker to raise a `SignalException` causing the running job to abort and be unlocked, which makes the job available to other workers. The default for this option is false.
388
439
 
389
440
  Here is an example of changing job parameters in Rails:
@@ -4,12 +4,12 @@ Gem::Specification.new do |spec|
4
4
  spec.description = 'Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks.'
5
5
  spec.email = ['brian@collectiveidea.com']
6
6
  spec.files = %w[CHANGELOG.md CONTRIBUTING.md LICENSE.md README.md Rakefile delayed_job.gemspec]
7
- spec.files += Dir.glob('{contrib,lib,recipes,spec}/**/*')
7
+ spec.files += Dir.glob('{contrib,lib,recipes,spec}/**/*')
8
8
  spec.homepage = 'http://github.com/collectiveidea/delayed_job'
9
9
  spec.licenses = ['MIT']
10
10
  spec.name = 'delayed_job'
11
11
  spec.require_paths = ['lib']
12
12
  spec.summary = 'Database-backed asynchronous priority queue system -- Extracted from Shopify'
13
13
  spec.test_files = Dir.glob('spec/**/*')
14
- spec.version = '4.0.6'
14
+ spec.version = '4.1.0'
15
15
  end
@@ -10,7 +10,7 @@ module Delayed
10
10
  def enqueue(*args) # rubocop:disable CyclomaticComplexity
11
11
  options = args.extract_options!
12
12
  options[:payload_object] ||= args.shift
13
- options[:priority] ||= Delayed::Worker.default_priority
13
+ options[:priority] ||= Delayed::Worker.default_priority
14
14
 
15
15
  if options[:queue].nil?
16
16
  if options[:payload_object].respond_to?(:queue_name)
@@ -32,7 +32,7 @@ module Delayed
32
32
  new(options).tap do |job|
33
33
  Delayed::Worker.lifecycle.run_callbacks(:enqueue, job) do
34
34
  job.hook(:enqueue)
35
- Delayed::Worker.delay_jobs ? job.save : job.invoke_job
35
+ Delayed::Worker.delay_job?(job) ? job.save : job.invoke_job
36
36
  end
37
37
  end
38
38
  end
@@ -63,6 +63,12 @@ module Delayed
63
63
  end
64
64
  end
65
65
 
66
+ attr_reader :error
67
+ def error=(error)
68
+ @error = error
69
+ self.last_error = "#{error.message}\n#{error.backtrace.join("\n")}" if self.respond_to?(:last_error=)
70
+ end
71
+
66
72
  def failed?
67
73
  !!failed_at
68
74
  end
@@ -93,7 +99,7 @@ module Delayed
93
99
  hook :before
94
100
  payload_object.perform
95
101
  hook :success
96
- rescue => e
102
+ rescue Exception => e # rubocop:disable RescueException
97
103
  hook :error, e
98
104
  raise e
99
105
  ensure
@@ -139,6 +145,12 @@ module Delayed
139
145
  end
140
146
  end
141
147
 
148
+ def destroy_failed_jobs?
149
+ payload_object.respond_to?(:destroy_failed_jobs?) ? payload_object.destroy_failed_jobs? : Delayed::Worker.destroy_failed_jobs
150
+ rescue DeserializationError
151
+ Delayed::Worker.destroy_failed_jobs
152
+ end
153
+
142
154
  def fail!
143
155
  update_attributes(:failed_at => self.class.db_time_now)
144
156
  end
@@ -161,7 +161,7 @@ shared_examples_for 'a delayed_job backend' do
161
161
  job = described_class.enqueue(CallbackJob.new)
162
162
  expect(job.payload_object).to receive(:perform).and_raise(RuntimeError.new('fail'))
163
163
 
164
- expect { job.invoke_job }.to raise_error
164
+ expect { job.invoke_job }.to raise_error(RuntimeError)
165
165
  expect(CallbackJob.messages).to eq(['enqueue', 'before', 'error: RuntimeError', 'after'])
166
166
  end
167
167
 
@@ -450,6 +450,34 @@ shared_examples_for 'a delayed_job backend' do
450
450
  end
451
451
  end
452
452
 
453
+ describe 'destroy_failed_jobs' do
454
+ context 'with a SimpleJob' do
455
+ before(:each) do
456
+ @job = described_class.enqueue SimpleJob.new
457
+ end
458
+
459
+ it 'is not defined' do
460
+ expect(@job.destroy_failed_jobs?).to be true
461
+ end
462
+
463
+ it 'uses the destroy failed jobs value on the payload when defined' do
464
+ expect(@job.payload_object).to receive(:destroy_failed_jobs?).and_return(false)
465
+ expect(@job.destroy_failed_jobs?).to be false
466
+ end
467
+ end
468
+
469
+ context 'with a job that raises DserializationError' do
470
+ before(:each) do
471
+ @job = described_class.new :handler => '--- !ruby/struct:GoingToRaiseArgError {}'
472
+ end
473
+
474
+ it 'falls back reasonably' do
475
+ expect(YAML).to receive(:load_dj).and_raise(ArgumentError)
476
+ expect(@job.destroy_failed_jobs?).to be true
477
+ end
478
+ end
479
+ end
480
+
453
481
  describe 'yaml serialization' do
454
482
  context 'when serializing jobs' do
455
483
  it 'raises error ArgumentError for new records' do
@@ -500,6 +528,7 @@ shared_examples_for 'a delayed_job backend' do
500
528
  Delayed::Worker.max_run_time = 1.second
501
529
  job = Delayed::Job.create :payload_object => LongRunningJob.new
502
530
  worker.run(job)
531
+ expect(job.error).to_not be_nil
503
532
  expect(job.reload.last_error).to match(/expired/)
504
533
  expect(job.reload.last_error).to match(/Delayed::Worker\.max_run_time is only 1 second/)
505
534
  expect(job.attempts).to eq(1)
@@ -513,6 +542,7 @@ shared_examples_for 'a delayed_job backend' do
513
542
  it 'marks the job as failed' do
514
543
  Delayed::Worker.destroy_failed_jobs = false
515
544
  job = described_class.create! :handler => '--- !ruby/object:JobThatDoesNotExist {}'
545
+ expect(job).to receive(:destroy_failed_jobs?).and_return(false)
516
546
  worker.work_off
517
547
  job.reload
518
548
  expect(job).to be_failed
@@ -535,6 +565,7 @@ shared_examples_for 'a delayed_job backend' do
535
565
  Delayed::Worker.max_attempts = 1
536
566
  worker.run(@job)
537
567
  @job.reload
568
+ expect(@job.error).to_not be_nil
538
569
  expect(@job.last_error).to match(/did not work/)
539
570
  expect(@job.attempts).to eq(1)
540
571
  expect(@job).to be_failed
@@ -543,6 +574,7 @@ shared_examples_for 'a delayed_job backend' do
543
574
  it 're-schedules jobs after failing' do
544
575
  worker.work_off
545
576
  @job.reload
577
+ expect(@job.error).to_not be_nil
546
578
  expect(@job.last_error).to match(/did not work/)
547
579
  expect(@job.last_error).to match(/sample_jobs.rb:\d+:in `perform'/)
548
580
  expect(@job.attempts).to eq(1)
@@ -617,6 +649,10 @@ shared_examples_for 'a delayed_job backend' do
617
649
  end
618
650
 
619
651
  context 'and we want to destroy jobs' do
652
+ after do
653
+ Delayed::Worker.destroy_failed_jobs = true
654
+ end
655
+
620
656
  it_behaves_like 'any failure more than Worker.max_attempts times'
621
657
 
622
658
  it 'is destroyed if it failed more than Worker.max_attempts times' do
@@ -624,6 +660,13 @@ shared_examples_for 'a delayed_job backend' do
624
660
  Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
625
661
  end
626
662
 
663
+ it 'is destroyed if the job has destroy failed jobs set' do
664
+ Delayed::Worker.destroy_failed_jobs = false
665
+ expect(@job).to receive(:destroy_failed_jobs?).and_return(true)
666
+ expect(@job).to receive(:destroy)
667
+ Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
668
+ end
669
+
627
670
  it 'is not destroyed if failed fewer than Worker.max_attempts times' do
628
671
  expect(@job).not_to receive(:destroy)
629
672
  (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) }
@@ -641,15 +684,35 @@ shared_examples_for 'a delayed_job backend' do
641
684
 
642
685
  it_behaves_like 'any failure more than Worker.max_attempts times'
643
686
 
644
- it 'is failed if it failed more than Worker.max_attempts times' do
645
- expect(@job.reload).not_to be_failed
646
- Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
647
- expect(@job.reload).to be_failed
687
+ context 'and destroy failed jobs is false' do
688
+ it 'is failed if it failed more than Worker.max_attempts times' do
689
+ expect(@job.reload).not_to be_failed
690
+ Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
691
+ expect(@job.reload).to be_failed
692
+ end
693
+
694
+ it 'is not failed if it failed fewer than Worker.max_attempts times' do
695
+ (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) }
696
+ expect(@job.reload).not_to be_failed
697
+ end
648
698
  end
649
699
 
650
- it 'is not failed if it failed fewer than Worker.max_attempts times' do
651
- (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) }
652
- expect(@job.reload).not_to be_failed
700
+ context 'and destroy failed jobs for job is false' do
701
+ before do
702
+ Delayed::Worker.destroy_failed_jobs = true
703
+ end
704
+
705
+ it 'is failed if it failed more than Worker.max_attempts times' do
706
+ expect(@job).to receive(:destroy_failed_jobs?).and_return(false)
707
+ expect(@job.reload).not_to be_failed
708
+ Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
709
+ expect(@job.reload).to be_failed
710
+ end
711
+
712
+ it 'is not failed if it failed fewer than Worker.max_attempts times' do
713
+ (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) }
714
+ expect(@job.reload).not_to be_failed
715
+ end
653
716
  end
654
717
  end
655
718
  end
@@ -6,15 +6,19 @@ unless ENV['RAILS_ENV'] == 'test'
6
6
  end
7
7
  end
8
8
  require 'optparse'
9
+ require 'pathname'
9
10
 
10
11
  module Delayed
11
12
  class Command # rubocop:disable ClassLength
12
13
  attr_accessor :worker_count, :worker_pools
13
14
 
15
+ DIR_PWD = Pathname.new Dir.pwd
16
+
14
17
  def initialize(args) # rubocop:disable MethodLength
15
18
  @options = {
16
19
  :quiet => true,
17
- :pid_dir => "#{Rails.root}/tmp/pids"
20
+ :pid_dir => "#{root}/tmp/pids",
21
+ :log_dir => "#{root}/log"
18
22
  }
19
23
 
20
24
  @worker_count = 1
@@ -37,11 +41,14 @@ module Delayed
37
41
  @options[:max_priority] = n
38
42
  end
39
43
  opt.on('-n', '--number_of_workers=workers', 'Number of unique workers to spawn') do |worker_count|
40
- @worker_count = worker_count.to_i rescue 1 # rubocop:disable RescueModifier
44
+ @worker_count = worker_count.to_i rescue 1
41
45
  end
42
46
  opt.on('--pid-dir=DIR', 'Specifies an alternate directory in which to store the process ids.') do |dir|
43
47
  @options[:pid_dir] = dir
44
48
  end
49
+ opt.on('--log-dir=DIR', 'Specifies an alternate directory in which to store the delayed_job log.') do |dir|
50
+ @options[:log_dir] = dir
51
+ end
45
52
  opt.on('-i', '--identifier=n', 'A numeric identifier for the worker.') do |n|
46
53
  @options[:identifier] = n
47
54
  end
@@ -114,18 +121,19 @@ module Delayed
114
121
  end
115
122
 
116
123
  def run(worker_name = nil, options = {})
117
- Dir.chdir(Rails.root)
124
+ Dir.chdir(root)
118
125
 
119
126
  Delayed::Worker.after_fork
120
- Delayed::Worker.logger ||= Logger.new(File.join(Rails.root, 'log', 'delayed_job.log'))
127
+ Delayed::Worker.logger ||= Logger.new(File.join(@options[:log_dir], 'delayed_job.log'))
121
128
 
122
129
  worker = Delayed::Worker.new(options)
123
130
  worker.name_prefix = "#{worker_name} "
124
131
  worker.start
125
132
  rescue => e
126
- Rails.logger.fatal e
127
133
  STDERR.puts e.message
128
- exit 1
134
+ STDERR.puts e.backtrace
135
+ ::Rails.logger.fatal(e) if rails_logger_defined?
136
+ exit_with_error_status
129
137
  end
130
138
 
131
139
  private
@@ -142,5 +150,21 @@ module Delayed
142
150
  worker_count = (worker_count || 1).to_i rescue 1
143
151
  @worker_pools << [queues, worker_count]
144
152
  end
153
+
154
+ def root
155
+ @root ||= rails_root_defined? ? ::Rails.root : DIR_PWD
156
+ end
157
+
158
+ def rails_root_defined?
159
+ defined?(::Rails.root)
160
+ end
161
+
162
+ def rails_logger_defined?
163
+ defined?(::Rails.logger)
164
+ end
165
+
166
+ def exit_with_error_status
167
+ exit 1
168
+ end
145
169
  end
146
170
  end
@@ -31,8 +31,10 @@ module Delayed
31
31
 
32
32
  module ClassMethods
33
33
  def handle_asynchronously(method, opts = {})
34
- aliased_method, punctuation = method.to_s.sub(/([?!=])$/, ''), $1 # rubocop:disable PerlBackrefs
35
- with_method, without_method = "#{aliased_method}_with_delay#{punctuation}", "#{aliased_method}_without_delay#{punctuation}"
34
+ aliased_method = method.to_s.sub(/([?!=])$/, '')
35
+ punctuation = $1 # rubocop:disable PerlBackrefs
36
+ with_method = "#{aliased_method}_with_delay#{punctuation}"
37
+ without_method = "#{aliased_method}_without_delay#{punctuation}"
36
38
  define_method(with_method) do |*args|
37
39
  curr_opts = opts.clone
38
40
  curr_opts.each_key do |key|
@@ -38,17 +38,17 @@ Capistrano::Configuration.instance.load do
38
38
 
39
39
  desc 'Stop the delayed_job process'
40
40
  task :stop, :roles => lambda { roles } do
41
- run "cd #{current_path};#{rails_env} #{delayed_job_command} stop #{args}"
41
+ run "cd #{current_path} && #{rails_env} #{delayed_job_command} stop #{args}"
42
42
  end
43
43
 
44
44
  desc 'Start the delayed_job process'
45
45
  task :start, :roles => lambda { roles } do
46
- run "cd #{current_path};#{rails_env} #{delayed_job_command} start #{args}"
46
+ run "cd #{current_path} && #{rails_env} #{delayed_job_command} start #{args}"
47
47
  end
48
48
 
49
49
  desc 'Restart the delayed_job process'
50
50
  task :restart, :roles => lambda { roles } do
51
- run "cd #{current_path};#{rails_env} #{delayed_job_command} restart #{args}"
51
+ run "cd #{current_path} && #{rails_env} #{delayed_job_command} restart #{args}"
52
52
  end
53
53
  end
54
54
  end
@@ -29,7 +29,7 @@ class Struct
29
29
  # Constantize the object so that ActiveSupport can attempt
30
30
  # its auto loading magic. Will raise LoadError if not successful.
31
31
  name.constantize
32
- "Struct::#{ name }"
32
+ "Struct::#{name}"
33
33
  end
34
34
  end
35
35
 
@@ -19,7 +19,7 @@ namespace :jobs do
19
19
  :min_priority => ENV['MIN_PRIORITY'],
20
20
  :max_priority => ENV['MAX_PRIORITY'],
21
21
  :queues => (ENV['QUEUES'] || ENV['QUEUE'] || '').split(','),
22
- :quiet => false
22
+ :quiet => ENV['QUIET']
23
23
  }
24
24
 
25
25
  @worker_options[:sleep_delay] = ENV['SLEEP_DELAY'].to_i if ENV['SLEEP_DELAY']
@@ -39,6 +39,7 @@ module Delayed
39
39
  self.delay_jobs = DEFAULT_DELAY_JOBS
40
40
  self.queues = DEFAULT_QUEUES
41
41
  self.read_ahead = DEFAULT_READ_AHEAD
42
+ @lifecycle = nil
42
43
  end
43
44
 
44
45
  reset
@@ -97,13 +98,29 @@ module Delayed
97
98
  end
98
99
 
99
100
  def self.lifecycle
100
- @lifecycle ||= Delayed::Lifecycle.new
101
+ # In case a worker has not been set up, job enqueueing needs a lifecycle.
102
+ setup_lifecycle unless @lifecycle
103
+
104
+ @lifecycle
105
+ end
106
+
107
+ def self.setup_lifecycle
108
+ @lifecycle = Delayed::Lifecycle.new
109
+ plugins.each { |klass| klass.new }
101
110
  end
102
111
 
103
112
  def self.reload_app?
104
113
  defined?(ActionDispatch::Reloader) && Rails.application.config.cache_classes == false
105
114
  end
106
115
 
116
+ def self.delay_job?(job)
117
+ if delay_jobs.is_a?(Proc)
118
+ delay_jobs.arity == 1 ? delay_jobs.call(job) : delay_jobs.call
119
+ else
120
+ delay_jobs
121
+ end
122
+ end
123
+
107
124
  def initialize(options = {})
108
125
  @quiet = options.key?(:quiet) ? options[:quiet] : true
109
126
  @failed_reserve_count = 0
@@ -112,7 +129,9 @@ module Delayed
112
129
  self.class.send("#{option}=", options[option]) if options.key?(option)
113
130
  end
114
131
 
115
- plugins.each { |klass| klass.new }
132
+ # Reset lifecycle on the offhand chance that something lazily
133
+ # triggered its creation before all plugins had been registered.
134
+ self.class.setup_lifecycle
116
135
  end
117
136
 
118
137
  # Every worker has a unique name which by default is the pid of the process. There are some
@@ -121,7 +140,7 @@ module Delayed
121
140
  # it crashed before.
122
141
  def name
123
142
  return @name unless @name.nil?
124
- "#{@name_prefix}host:#{Socket.gethostname} pid:#{Process.pid}" rescue "#{@name_prefix}pid:#{Process.pid}" # rubocop:disable RescueModifier
143
+ "#{@name_prefix}host:#{Socket.gethostname} pid:#{Process.pid}" rescue "#{@name_prefix}pid:#{Process.pid}"
125
144
  end
126
145
 
127
146
  # Sets the name of the worker.
@@ -181,7 +200,8 @@ module Delayed
181
200
  # Do num jobs and return stats on success/failure.
182
201
  # Exit early if interrupted.
183
202
  def work_off(num = 100)
184
- success, failure = 0, 0
203
+ success = 0
204
+ failure = 0
185
205
 
186
206
  num.times do
187
207
  case reserve_and_run_one_job
@@ -190,7 +210,7 @@ module Delayed
190
210
  when false
191
211
  failure += 1
192
212
  else
193
- break # leave if no work could be done
213
+ break # leave if no work could be done
194
214
  end
195
215
  break if stop? # leave if we're exiting
196
216
  end
@@ -200,18 +220,18 @@ module Delayed
200
220
 
201
221
  def run(job)
202
222
  job_say job, 'RUNNING'
203
- runtime = Benchmark.realtime do
223
+ runtime = Benchmark.realtime do
204
224
  Timeout.timeout(max_run_time(job).to_i, WorkerTimeout) { job.invoke_job }
205
225
  job.destroy
206
226
  end
207
227
  job_say job, format('COMPLETED after %.4f', runtime)
208
- return true # did work
228
+ return true # did work
209
229
  rescue DeserializationError => error
210
- job.last_error = "#{error.message}\n#{error.backtrace.join("\n")}"
230
+ job.error = error
211
231
  failed(job)
212
- rescue => error
232
+ rescue Exception => error # rubocop:disable RescueException
213
233
  self.class.lifecycle.run_callbacks(:error, self, job) { handle_failed_job(job, error) }
214
- return false # work failed
234
+ return false # work failed
215
235
  end
216
236
 
217
237
  # Reschedule the job in the future (when a job fails).
@@ -236,7 +256,7 @@ module Delayed
236
256
  say "Error when running failure callback: #{error}", 'error'
237
257
  say error.backtrace.join("\n"), 'error'
238
258
  ensure
239
- self.class.destroy_failed_jobs ? job.destroy : job.fail!
259
+ job.destroy_failed_jobs? ? job.destroy : job.fail!
240
260
  end
241
261
  end
242
262
  end
@@ -268,7 +288,7 @@ module Delayed
268
288
  protected
269
289
 
270
290
  def handle_failed_job(job, error)
271
- job.last_error = "#{error.message}\n#{error.backtrace.join("\n")}"
291
+ job.error = error
272
292
  job_say job, "FAILED (#{job.attempts} prior attempts) with #{error.class.name}: #{error.message}", 'error'
273
293
  reschedule(job)
274
294
  end
@@ -1,5 +1,6 @@
1
1
  module Autoloaded
2
- class InstanceStruct < ::Struct.new(nil)
2
+ InstanceStruct = ::Struct.new(nil)
3
+ class InstanceStruct
3
4
  def perform
4
5
  end
5
6
  end
@@ -1,6 +1,7 @@
1
1
  # Make sure this file does not get required manually
2
2
  module Autoloaded
3
- class Struct < ::Struct.new(nil)
3
+ Struct = ::Struct.new(nil)
4
+ class Struct
4
5
  def perform
5
6
  end
6
7
  end
@@ -0,0 +1,2 @@
1
+ # Fake "daemons" file on the spec load path to allow spec/delayed/command_spec.rb
2
+ # to test the Delayed::Command class without actually adding daemons as a dependency.
@@ -2,6 +2,129 @@ require 'helper'
2
2
  require 'delayed/command'
3
3
 
4
4
  describe Delayed::Command do
5
+ let(:options) { [] }
6
+ let(:logger) { double('Logger') }
7
+
8
+ subject { Delayed::Command.new options }
9
+
10
+ before do
11
+ allow(Delayed::Worker).to receive(:after_fork)
12
+ allow(Dir).to receive(:chdir)
13
+ allow(Logger).to receive(:new).and_return(logger)
14
+ allow_any_instance_of(Delayed::Worker).to receive(:start)
15
+ allow(Delayed::Worker).to receive(:logger=)
16
+ allow(Delayed::Worker).to receive(:logger).and_return(nil, logger)
17
+ end
18
+
19
+ shared_examples_for 'uses --log-dir option' do
20
+ context 'when --log-dir is specified' do
21
+ let(:options) { ['--log-dir=/custom/log/dir'] }
22
+
23
+ it 'creates the delayed_job.log in the specified directory' do
24
+ expect(Logger).to receive(:new).with('/custom/log/dir/delayed_job.log')
25
+ subject.run
26
+ end
27
+ end
28
+ end
29
+
30
+ describe 'run' do
31
+ it 'sets the Delayed::Worker logger' do
32
+ expect(Delayed::Worker).to receive(:logger=).with(logger)
33
+ subject.run
34
+ end
35
+
36
+ context 'when Rails root is defined' do
37
+ let(:rails_root) { Pathname.new '/rails/root' }
38
+ let(:rails) { double('Rails', :root => rails_root) }
39
+
40
+ before do
41
+ stub_const('Rails', rails)
42
+ end
43
+
44
+ it 'runs the Delayed::Worker process in Rails.root' do
45
+ expect(Dir).to receive(:chdir).with(rails_root)
46
+ subject.run
47
+ end
48
+
49
+ context 'when --log-dir is not specified' do
50
+ it 'creates the delayed_job.log in Rails.root/log' do
51
+ expect(Logger).to receive(:new).with('/rails/root/log/delayed_job.log')
52
+ subject.run
53
+ end
54
+ end
55
+
56
+ include_examples 'uses --log-dir option'
57
+ end
58
+
59
+ context 'when Rails root is not defined' do
60
+ let(:rails_without_root) { double('Rails') }
61
+
62
+ before do
63
+ stub_const('Rails', rails_without_root)
64
+ end
65
+
66
+ it 'runs the Delayed::Worker process in $PWD' do
67
+ expect(Dir).to receive(:chdir).with(Delayed::Command::DIR_PWD)
68
+ subject.run
69
+ end
70
+
71
+ context 'when --log-dir is not specified' do
72
+ it 'creates the delayed_job.log in $PWD/log' do
73
+ expect(Logger).to receive(:new).with("#{Delayed::Command::DIR_PWD}/log/delayed_job.log")
74
+ subject.run
75
+ end
76
+ end
77
+
78
+ include_examples 'uses --log-dir option'
79
+ end
80
+
81
+ context 'when an error is raised' do
82
+ let(:test_error) { Class.new(StandardError) }
83
+
84
+ before do
85
+ allow(Delayed::Worker).to receive(:new).and_raise(test_error.new('An error'))
86
+ allow(subject).to receive(:exit_with_error_status)
87
+ allow(STDERR).to receive(:puts)
88
+ end
89
+
90
+ it 'prints the error message to STDERR' do
91
+ expect(STDERR).to receive(:puts).with('An error')
92
+ subject.run
93
+ end
94
+
95
+ it 'exits with an error status' do
96
+ expect(subject).to receive(:exit_with_error_status)
97
+ subject.run
98
+ end
99
+
100
+ context 'when Rails logger is not defined' do
101
+ let(:rails) { double('Rails') }
102
+
103
+ before do
104
+ stub_const('Rails', rails)
105
+ end
106
+
107
+ it 'does not attempt to use the Rails logger' do
108
+ subject.run
109
+ end
110
+ end
111
+
112
+ context 'when Rails logger is defined' do
113
+ let(:rails_logger) { double('Rails logger') }
114
+ let(:rails) { double('Rails', :logger => rails_logger) }
115
+
116
+ before do
117
+ stub_const('Rails', rails)
118
+ end
119
+
120
+ it 'logs the error to the Rails logger' do
121
+ expect(rails_logger).to receive(:fatal).with(test_error)
122
+ subject.run
123
+ end
124
+ end
125
+ end
126
+ end
127
+
5
128
  describe 'parsing --pool argument' do
6
129
  it 'should parse --pool correctly' do
7
130
  command = Delayed::Command.new(['--pool=*:1', '--pool=test_queue:4', '--pool=mailers,misc:2'])
@@ -40,13 +163,13 @@ describe Delayed::Command do
40
163
  expect(Dir).to receive(:mkdir).with('./tmp/pids').once
41
164
 
42
165
  [
43
- ['delayed_job.0', {:quiet => true, :pid_dir => './tmp/pids', :queues => []}],
44
- ['delayed_job.1', {:quiet => true, :pid_dir => './tmp/pids', :queues => ['test_queue']}],
45
- ['delayed_job.2', {:quiet => true, :pid_dir => './tmp/pids', :queues => ['test_queue']}],
46
- ['delayed_job.3', {:quiet => true, :pid_dir => './tmp/pids', :queues => ['test_queue']}],
47
- ['delayed_job.4', {:quiet => true, :pid_dir => './tmp/pids', :queues => ['test_queue']}],
48
- ['delayed_job.5', {:quiet => true, :pid_dir => './tmp/pids', :queues => %w[mailers misc]}],
49
- ['delayed_job.6', {:quiet => true, :pid_dir => './tmp/pids', :queues => %w[mailers misc]}]
166
+ ['delayed_job.0', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => []}],
167
+ ['delayed_job.1', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => ['test_queue']}],
168
+ ['delayed_job.2', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => ['test_queue']}],
169
+ ['delayed_job.3', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => ['test_queue']}],
170
+ ['delayed_job.4', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => ['test_queue']}],
171
+ ['delayed_job.5', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => %w[mailers misc]}],
172
+ ['delayed_job.6', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => %w[mailers misc]}]
50
173
  ].each do |args|
51
174
  expect(command).to receive(:run_process).with(*args).once
52
175
  end
@@ -62,6 +62,7 @@ describe Delayed::MessageSending do
62
62
  class FairyTail
63
63
  attr_accessor :happy_ending
64
64
  def self.princesses; end
65
+
65
66
  def tell
66
67
  @happy_ending = true
67
68
  end
@@ -118,5 +119,25 @@ describe Delayed::MessageSending do
118
119
  end.not_to change(fairy_tail, :happy_ending)
119
120
  end.to change { Delayed::Job.count }.by(1)
120
121
  end
122
+
123
+ it 'does delay when delay_jobs is a proc returning true' do
124
+ Delayed::Worker.delay_jobs = ->(_job) { true }
125
+ fairy_tail = FairyTail.new
126
+ expect do
127
+ expect do
128
+ fairy_tail.delay.tell
129
+ end.not_to change(fairy_tail, :happy_ending)
130
+ end.to change { Delayed::Job.count }.by(1)
131
+ end
132
+
133
+ it 'does not delay the job when delay_jobs is a proc returning false' do
134
+ Delayed::Worker.delay_jobs = ->(_job) { false }
135
+ fairy_tail = FairyTail.new
136
+ expect do
137
+ expect do
138
+ fairy_tail.delay.tell
139
+ end.to change(fairy_tail, :happy_ending).from(nil).to(true)
140
+ end.not_to change { Delayed::Job.count }
141
+ end
121
142
  end
122
143
  end
@@ -68,7 +68,7 @@ describe Delayed::PerformableMethod do
68
68
  story = Story.create
69
69
  expect(story).to receive(:error).with(an_instance_of(Delayed::Job), an_instance_of(RuntimeError))
70
70
  expect(story).to receive(:tell).and_raise(RuntimeError)
71
- expect { story.delay.tell.invoke_job }.to raise_error
71
+ expect { story.delay.tell.invoke_job }.to raise_error(RuntimeError)
72
72
  end
73
73
 
74
74
  it 'delegates failure hook to object' do
@@ -98,7 +98,7 @@ describe Delayed::PerformableMethod do
98
98
  story = Story.create
99
99
  expect(story).to receive(:error).with(an_instance_of(Delayed::Job), an_instance_of(RuntimeError))
100
100
  expect(story).to receive(:tell).and_raise(RuntimeError)
101
- expect { story.delay.tell }.to raise_error
101
+ expect { story.delay.tell }.to raise_error(RuntimeError)
102
102
  end
103
103
 
104
104
  it 'delegates failure hook to object' do
@@ -1,4 +1,5 @@
1
- class NamedJob < Struct.new(:perform)
1
+ NamedJob = Struct.new(:perform)
2
+ class NamedJob
2
3
  def display_name
3
4
  'named_job'
4
5
  end
@@ -22,11 +23,12 @@ class ErrorJob
22
23
  cattr_accessor :runs
23
24
  @runs = 0
24
25
  def perform
25
- raise 'did not work'
26
+ raise Exception, 'did not work'
26
27
  end
27
28
  end
28
29
 
29
- class CustomRescheduleJob < Struct.new(:offset)
30
+ CustomRescheduleJob = Struct.new(:offset)
31
+ class CustomRescheduleJob
30
32
  cattr_accessor :runs
31
33
  @runs = 0
32
34
  def perform
@@ -154,4 +154,22 @@ describe Delayed::Worker do
154
154
  @worker.say(@text, Delayed::Worker.default_log_level)
155
155
  end
156
156
  end
157
+
158
+ describe 'plugin registration' do
159
+ it 'does not double-register plugins on worker instantiation' do
160
+ performances = 0
161
+ plugin = Class.new(Delayed::Plugin) do
162
+ callbacks do |lifecycle|
163
+ lifecycle.before(:enqueue) { performances += 1 }
164
+ end
165
+ end
166
+ Delayed::Worker.plugins << plugin
167
+
168
+ Delayed::Worker.new
169
+ Delayed::Worker.new
170
+ Delayed::Worker.lifecycle.run_callbacks(:enqueue, nil) {}
171
+
172
+ expect(performances).to eq(1)
173
+ end
174
+ end
157
175
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: delayed_job
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.0.6
4
+ version: 4.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Brandon Keepers
@@ -15,7 +15,7 @@ authors:
15
15
  autorequire:
16
16
  bindir: bin
17
17
  cert_chain: []
18
- date: 2014-12-22 00:00:00.000000000 Z
18
+ date: 2015-09-22 00:00:00.000000000 Z
19
19
  dependencies:
20
20
  - !ruby/object:Gem::Dependency
21
21
  name: activesupport
@@ -84,6 +84,7 @@ files:
84
84
  - spec/autoloaded/instance_clazz.rb
85
85
  - spec/autoloaded/instance_struct.rb
86
86
  - spec/autoloaded/struct.rb
87
+ - spec/daemons.rb
87
88
  - spec/delayed/backend/test.rb
88
89
  - spec/delayed/command_spec.rb
89
90
  - spec/delayed/serialization/test.rb
@@ -117,7 +118,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
117
118
  version: '0'
118
119
  requirements: []
119
120
  rubyforge_project:
120
- rubygems_version: 2.4.4
121
+ rubygems_version: 2.4.8
121
122
  signing_key:
122
123
  specification_version: 4
123
124
  summary: Database-backed asynchronous priority queue system -- Extracted from Shopify
@@ -126,6 +127,7 @@ test_files:
126
127
  - spec/autoloaded/instance_clazz.rb
127
128
  - spec/autoloaded/instance_struct.rb
128
129
  - spec/autoloaded/struct.rb
130
+ - spec/daemons.rb
129
131
  - spec/delayed/backend/test.rb
130
132
  - spec/delayed/command_spec.rb
131
133
  - spec/delayed/serialization/test.rb