good_job 3.9.0 → 3.10.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 69560b4af2d1cab2a3783fb80573794066c3704aacafa7bd315a9dc6e1b75441
4
- data.tar.gz: 55dff98e3c8ebbcebd5f09664d02f4ddd23b298c0ee2bc7cb163e43f6963de96
3
+ metadata.gz: 3e632ae76e69e8601479181fb0093e9c6777510ac0743184a8f578b3c4dacee8
4
+ data.tar.gz: 577f36ad182e6f4ff841abcd073925f4477f1086b1a0c7a454c12037f781ace7
5
5
  SHA512:
6
- metadata.gz: dd4763e6d473a22ea7b0f6f7b501249c7e5e29d06c1fdd9b3d028c923ba407fb485eea748fd37857a5eb09f5b26659d67f4792f4a66848542bfc1f47f7384577
7
- data.tar.gz: 89e165d003d156fa16450762b8a8a269172509557ab59be60e7d54794d079c9963f8c11f45047ecfdea8e0963d901c8d9e1c297743392a05817d21d417c78dab
6
+ metadata.gz: f9211423d43cee468602aa1476d0d4b71a3831448157dc7dc540f98e0d1cb476e60bed884679ef10733af21c21e1c595dac14dc2d1c0b25d4b49b8d246f7be98
7
+ data.tar.gz: 25a51a4196ea6618ca44ac73ed6cb9f97b63d99596bd144a9d51f439033a22d15ab16a73cd9d48954dc9b447b19cc9f03a21a9e5667d5021c85b5510ad7ca577
data/CHANGELOG.md CHANGED
@@ -1,5 +1,36 @@
1
1
  # Changelog
2
2
 
3
+ ## [v3.10.1](https://github.com/bensheldon/good_job/tree/v3.10.1) (2023-02-06)
4
+
5
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.10.0...v3.10.1)
6
+
7
+ **Fixed bugs:**
8
+
9
+ - Ensure batch is reloaded before updating on multiple enqueues [\#824](https://github.com/bensheldon/good_job/pull/824) ([bensheldon](https://github.com/bensheldon))
10
+
11
+ **Closed issues:**
12
+
13
+ - Can't batch.enqueue the callback after retrying a job within the batch [\#822](https://github.com/bensheldon/good_job/issues/822)
14
+
15
+ **Merged pull requests:**
16
+
17
+ - In tests, retry when connecting to Puma returns Net::ReadTimeout [\#825](https://github.com/bensheldon/good_job/pull/825) ([bensheldon](https://github.com/bensheldon))
18
+ - Add Batch enqueue example to Demo's cron configuration [\#823](https://github.com/bensheldon/good_job/pull/823) ([bensheldon](https://github.com/bensheldon))
19
+
20
+ ## [v3.10.0](https://github.com/bensheldon/good_job/tree/v3.10.0) (2023-02-04)
21
+
22
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.9.0...v3.10.0)
23
+
24
+ **Implemented enhancements:**
25
+
26
+ - Implement `GoodJob::Batch` [\#712](https://github.com/bensheldon/good_job/pull/712) ([bensheldon](https://github.com/bensheldon))
27
+
28
+ **Closed issues:**
29
+
30
+ - Support for Ruby 3.2 [\#785](https://github.com/bensheldon/good_job/issues/785)
31
+ - Custom table names [\#748](https://github.com/bensheldon/good_job/issues/748)
32
+ - Health check issue with cron scheduler job [\#741](https://github.com/bensheldon/good_job/issues/741)
33
+
3
34
  ## [v3.9.0](https://github.com/bensheldon/good_job/tree/v3.9.0) (2023-01-31)
4
35
 
5
36
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.8.0...v3.9.0)
data/README.md CHANGED
@@ -41,9 +41,11 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
41
41
  - [Dashboard](#dashboard)
42
42
  - [API-only Rails applications](#api-only-rails-applications)
43
43
  - [Live Polling](#live-polling)
44
- - [ActiveJob concurrency](#activejob-concurrency)
44
+ - [Concurrency controls](#concurrency-controls)
45
45
  - [How concurrency controls work](#how-concurrency-controls-work)
46
46
  - [Cron-style repeating/recurring jobs](#cron-style-repeatingrecurring-jobs)
47
+ - [Bulk enqueue](#bulk-enqueue)
48
+ - [Batches](#batches)
47
49
  - [Updating](#updating)
48
50
  - [Upgrading minor versions](#upgrading-minor-versions)
49
51
  - [Upgrading v2 to v3](#upgrading-v2-to-v3)
@@ -58,7 +60,6 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
58
60
  - [Database connections](#database-connections)
59
61
  - [Production setup](#production-setup)
60
62
  - [Queue performance with Queue Select Limit](#queue-performance-with-queue-select-limit)
61
- - [Bulk enqueue](#bulk-enqueue)
62
63
  - [Execute jobs async / in-process](#execute-jobs-async--in-process)
63
64
  - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
64
65
  - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
@@ -392,7 +393,7 @@ end
392
393
 
393
394
  The Dashboard can be set to automatically refresh by checking "Live Poll" in the Dashboard header, or by setting `?poll=10` with the interval in seconds (default 30 seconds).
394
395
 
395
- ### ActiveJob concurrency
396
+ ### Concurrency controls
396
397
 
397
398
  GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_. Limiting concurrency can help prevent duplicate, double or unnecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
398
399
 
@@ -485,6 +486,196 @@ config.good_job.cron = {
485
486
  }
486
487
  ```
487
488
 
489
+ ### Bulk enqueue
490
+
491
+ GoodJob's Bulk-enqueue functionality can buffer and enqueue multiple jobs at once, using a single INSERT statement. This can more performant when enqueuing a large number of jobs.
492
+
493
+ ```ruby
494
+ # Capture jobs using `.perform_later`:
495
+ active_jobs = GoodJob::Bulk.enqueue do
496
+ MyJob.perform_later
497
+ AnotherJob.perform_later
498
+ # If an exception is raised within this block, no jobs will be inserted.
499
+ end
500
+
501
+ # All ActiveJob instances are returned from GoodJob::Bulk.enqueue.
502
+ # Jobs that have been successfully enqueued have a `provider_job_id` set.
503
+ active_jobs.all?(&:provider_job_id)
504
+
505
+ # Bulk enqueue ActiveJob instances directly without using `.perform_later`:
506
+ GoodJob::Bulk.enqueue(MyJob.new, AnotherJob.new)
507
+ ```
508
+
509
+ ### Batches
510
+
511
+ Batches track a set of jobs, and enqueue an optional callback job when all of the jobs have finished (succeeded or discarded).
512
+
513
+ - A simple example that enqueues your `MyBatchCallbackJob` after the two jobs have finished, and passes along the current user as a batch property:
514
+
515
+ ```ruby
516
+ GoodJob::Batch.enqueue(on_finish: MyBatchCallbackJob, user: current_user) do
517
+ MyJob.perform_later
518
+ OtherJob.perform_later
519
+ end
520
+
521
+ # When these jobs have finished, it will enqueue your `MyBatchCallbackJob.perform_later(batch, options)`
522
+ class MyBatchCallbackJob < ApplicationJob
523
+ # Callback jobs must accept a `batch` and `options` argument
524
+ def perform(batch, params)
525
+ # The batch object will contain the Batch's properties, which are mutable
526
+ batch.properties[:user] # => <User id: 1, ...>
527
+
528
+ # Params is a hash containing additional context (more may be added in the future)
529
+ params[:event] # => :finish, :success, :discard
530
+ end
531
+ end
532
+ ```
533
+
534
+ - Jobs can be added to an existing batch. Jobs in a batch are enqueued and performed immediately/asynchronously. The final callback job will not be enqueued until `GoodJob::Batch#enqueue` is called.
535
+
536
+ ```ruby
537
+ batch = GoodJob::Batch.add do
538
+ 10.times { MyJob.perform_later }
539
+ end
540
+ batch.add do
541
+ 10.times { OtherJob.perform_later }
542
+ end
543
+ batch.enqueue(on_finish: MyBatchCallbackJob, age: 42)
544
+ ```
545
+
546
+ - If you need to access the batch within a job that is part of the batch, include [`GoodJob::ActiveJobExtensions::Batches`](lib/good_job/active_job_extensions/batches.rb) in your job class:
547
+
548
+ ```ruby
549
+ class MyJob < ApplicationJob
550
+ include GoodJob::ActiveJobExtensions::Batches
551
+
552
+ def perform
553
+ self.batch # => <GoodJob::Batch id: 1, ...>
554
+ end
555
+ end
556
+ ```
557
+
558
+ - [`GoodJob::Batch`](app/models/good_job/batch.rb) has a number of assignable attributes and methods:
559
+
560
+ ```ruby
561
+ batch = GoodJob::Batch.new
562
+ batch.description = "My batch"
563
+ batch.on_finish = "MyBatchCallbackJob" # Callback job when all jobs have finished
564
+ batch.on_success = "MyBatchCallbackJob" # Callback job when/if all jobs have succeeded
565
+ batch.on_discard = "MyBatchCallbackJob" # Callback job when the first job in the batch is discarded
566
+ batch.callback_queue_name = "special_queue" # Optional queue for callback jobs, otherwise will defer to job class
567
+ batch.callback_priority = 10 # Optional priority name for callback jobs, otherwise will defer to job class
568
+ batch.properties = { age: 42 } # Custom data and state to attach to the batch
569
+ batch.add do
570
+ MyJob.perform_later
571
+ end
572
+ batch.enqueue
573
+
574
+ batch.discarded? # => Boolean
575
+ batch.discarded_at # => <DateTime>
576
+ batch.finished? # => Boolean
577
+ batch.finished_at # => <DateTime>
578
+ batch.succeeded? # => Boolean
579
+ batch.active_jobs # => Array of ActiveJob::Base-inherited jobs that are part of the batch
580
+
581
+ batch = GoodJob::Batch.find(batch.id)
582
+ batch.description = "Updated batch description"
583
+ batch.save
584
+ batch.reload
585
+ ```
586
+
587
+ ### Batch callback jobs
588
+
589
+ Batch callbacks are Active Job jobs that are enqueued at certain events during the execution of jobs within the batch:
590
+
591
+ - `:finish` - Enqueued when all jobs in the batch have finished, after all retries. Jobs will either be discarded or succeeded.
592
+ - `:success` - Enqueued only when all jobs in the batch have finished and succeeded.
593
+ - `:discard` - Enqueued immediately the first time a job in the batch is discarded.
594
+
595
+ Callback jobs must accept a `batch` and `params` argument in their `perform` method:
596
+
597
+ ```ruby
598
+ class MyBatchCallbackJob < ApplicationJob
599
+ def perform(batch, params)
600
+ # The batch object will contain the Batch's properties
601
+ batch.properties[:user] # => <User id: 1, ...>
602
+ # Batches are mutable
603
+ batch.properties[:user] = User.find(2)
604
+ batch.save
605
+
606
+ # Params is a hash containing additional context (more may be added in the future)
607
+ params[:event] # => :finish, :success, :discard
608
+ end
609
+ end
610
+ ```
611
+
612
+ #### Complex batches
613
+
614
+ Consider a multi-stage batch with both parallel and serial job steps:
615
+
616
+ ```mermaid
617
+ graph TD
618
+ 0{"BatchJob\n{ stage: nil }"}
619
+ 0 --> a["WorkJob]\n{ step: a }"]
620
+ 0 --> b["WorkJob]\n{ step: b }"]
621
+ 0 --> c["WorkJob]\n{ step: c }"]
622
+ a --> 1
623
+ b --> 1
624
+ c --> 1
625
+ 1{"BatchJob\n{ stage: 1 }"}
626
+ 1 --> d["WorkJob]\n{ step: d }"]
627
+ 1 --> e["WorkJob]\n{ step: e }"]
628
+ e --> f["WorkJob]\n{ step: f }"]
629
+ d --> 2
630
+ f --> 2
631
+ 2{"BatchJob\n{ stage: 2 }"}
632
+ ```
633
+
634
+ This can be implemented with a single, mutable batch job:
635
+
636
+ ```ruby
637
+ class WorkJob < ApplicationJob
638
+ include GoodJob::ActiveJobExtensions::Batches
639
+
640
+ def perform(step)
641
+ # ...
642
+ if step == 'e'
643
+ batch.add { WorkJob.perform_later('f') }
644
+ end
645
+ end
646
+ end
647
+
648
+ class BatchJob < ApplicationJob
649
+ def perform(batch, options)
650
+ if batch.properties[:stage].nil?
651
+ batch.enqueue(stage: 1) do
652
+ WorkJob.perform_later('a')
653
+ WorkJob.perform_later('b')
654
+ WorkJob.perform_later('c')
655
+ end
656
+ elsif batch.properties[:stage] == 1
657
+ batch.enqueue(stage: 2) do
658
+ WorkJob.perform_later('d')
659
+ WorkJob.perform_later('e')
660
+ end
661
+ elsif batch.properties[:stage] == 2
662
+ # ...
663
+ end
664
+ end
665
+ end
666
+
667
+ GoodJob::Batch.enqueue(on_finish: BatchJob)
668
+ ```
669
+
670
+ #### Other batch details
671
+
672
+ - Whether to enqueue a callback job is evaluated once the batch is in an `enqueued?`-state by using `GoodJob::Batch.enqueue` or `batch.enqueue`.
673
+ - Callback job enqueueing will be re-triggered if additional jobs are `enqueue`'d to the batch; use `add` to add jobs to the batch without retriggering callback jobs.
674
+ - Callback jobs will be enqueued even if the batch contains no jobs.
675
+ - Callback jobs perform asynchronously. It's possible that `:finish` and `:success` or `:discard` callback jobs perform at the same time. Keep this in mind when updating batch properties.
676
+ - Batch properties are serialized using Active Job serialization. This is flexible, but can lead to deserialization errors if a GlobalID record is directly referenced but is subsequently deleted and thus unloadable.
677
+ - 🚧Batches are a work in progress. Please let us know what would be helpful to improve their functionality and usefulness.
678
+
488
679
  ### Updating
489
680
 
490
681
  GoodJob follows semantic versioning, though updates may be encouraged through deprecation warnings in minor versions.
@@ -794,26 +985,6 @@ To explain where this value is used, here is the pseudo-query that GoodJob uses
794
985
  )
795
986
  ```
796
987
 
797
- ### Bulk enqueue
798
-
799
- GoodJob's Bulk-enqueue functionality can buffer and enqueue multiple jobs at once, using a single INSERT statement. This can more performant when enqueuing a large number of jobs.
800
-
801
- ```ruby
802
- # Capture jobs using `.perform_later`:
803
- active_jobs = GoodJob::Bulk.enqueue do
804
- MyJob.perform_later
805
- AnotherJob.perform_later
806
- # If an exception is raised within this block, no jobs will be inserted.
807
- end
808
-
809
- # All ActiveJob instances are returned from GoodJob::Bulk.enqueue.
810
- # Jobs that have been successfully enqueued have a `provider_job_id` set.
811
- active_jobs.all?(&:provider_job_id)
812
-
813
- # Bulk enqueue ActiveJob instances directly without using `.perform_later`:
814
- GoodJob::Bulk.enqueue(MyJob.new, AnotherJob.new)
815
- ```
816
-
817
988
  ### Execute jobs async / in-process
818
989
 
819
990
  GoodJob can execute jobs "async" in the same process as the web server (e.g. `bin/rails s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
@@ -0,0 +1,12 @@
1
+ # frozen_string_literal: true
2
+ module GoodJob
3
+ class BatchesController < GoodJob::ApplicationController
4
+ def index
5
+ @filter = BatchesFilter.new(params)
6
+ end
7
+
8
+ def show
9
+ @batch = GoodJob::BatchRecord.find(params[:id])
10
+ end
11
+ end
12
+ end
@@ -0,0 +1,21 @@
1
+ # frozen_string_literal: true
2
+ module GoodJob
3
+ class BatchesFilter < BaseFilter
4
+ def records
5
+ after_created_at = params[:after_created_at].present? ? Time.zone.parse(params[:after_created_at]) : nil
6
+
7
+ filtered_query.display_all(
8
+ after_created_at: after_created_at,
9
+ after_id: params[:after_id]
10
+ ).limit(params.fetch(:limit, DEFAULT_LIMIT))
11
+ end
12
+
13
+ def filtered_query(_filtered_params = params)
14
+ base_query
15
+ end
16
+
17
+ def default_base_query
18
+ GoodJob::BatchRecord.all.includes(:jobs)
19
+ end
20
+ end
21
+ end
@@ -0,0 +1,149 @@
1
+ # frozen_string_literal: true
2
+
3
+ module GoodJob
4
+ # NOTE: This class delegates to {GoodJob::BatchRecord} and is intended to be the public interface for Batches.
5
+ class Batch
6
+ include GlobalID::Identification
7
+
8
+ thread_cattr_accessor :current_batch_id
9
+ thread_cattr_accessor :current_batch_callback_id
10
+
11
+ PROTECTED_PROPERTIES = %i[
12
+ on_finish
13
+ on_success
14
+ on_discard
15
+ callback_queue_name
16
+ callback_priority
17
+ description
18
+ properties
19
+ ].freeze
20
+
21
+ delegate(
22
+ :id,
23
+ :created_at,
24
+ :updated_at,
25
+ :persisted?,
26
+ :enqueued_at,
27
+ :finished_at,
28
+ :discarded_at,
29
+ :enqueued?,
30
+ :finished?,
31
+ :succeeded?,
32
+ :discarded?,
33
+ :description,
34
+ :description=,
35
+ :on_finish,
36
+ :on_finish=,
37
+ :on_success,
38
+ :on_success=,
39
+ :on_discard,
40
+ :on_discard=,
41
+ :callback_queue_name,
42
+ :callback_queue_name=,
43
+ :callback_priority,
44
+ :callback_priority=,
45
+ :properties,
46
+ :properties=,
47
+ :save,
48
+ :reload,
49
+ to: :record
50
+ )
51
+
52
+ # Create a new batch and enqueue it
53
+ # @param on_finish [String, Object] The class name of the callback job to be enqueued after the batch is finished
54
+ # @param properties [Hash] Additional properties to be stored on the batch
55
+ # @param block [Proc] Enqueue jobs within the block to add them to the batch
56
+ # @return [GoodJob::BatchRecord]
57
+ def self.enqueue(active_jobs = [], **properties, &block)
58
+ new.tap do |batch|
59
+ batch.enqueue(active_jobs, **properties, &block)
60
+ end
61
+ end
62
+
63
+ def self.find(id)
64
+ new _record: BatchRecord.find(id)
65
+ end
66
+
67
+ # Helper method to enqueue jobs and assign them to a batch
68
+ def self.within_thread(batch_id: nil, batch_callback_id: nil)
69
+ original_batch_id = current_batch_id
70
+ original_batch_callback_id = current_batch_callback_id
71
+
72
+ self.current_batch_id = batch_id
73
+ self.current_batch_callback_id = batch_callback_id
74
+
75
+ yield
76
+ ensure
77
+ self.current_batch_id = original_batch_id
78
+ self.current_batch_callback_id = original_batch_callback_id
79
+ end
80
+
81
+ def initialize(_record: nil, **properties) # rubocop:disable Lint/UnderscorePrefixedVariableName
82
+ self.record = _record || BatchRecord.new
83
+ assign_properties(properties)
84
+ end
85
+
86
+ # @return [Array<ActiveJob::Base>] Active jobs added to the batch
87
+ def enqueue(active_jobs = [], **properties, &block)
88
+ assign_properties(properties)
89
+ if record.new_record?
90
+ record.save!
91
+ else
92
+ record.with_advisory_lock(function: "pg_advisory_lock") do
93
+ record.enqueued_at_will_change!
94
+ record.finished_at_will_change!
95
+ record.update!(enqueued_at: nil, finished_at: nil)
96
+ end
97
+ end
98
+
99
+ active_jobs = add(active_jobs, &block)
100
+
101
+ record.with_advisory_lock(function: "pg_advisory_lock") do
102
+ record.update!(enqueued_at: Time.current)
103
+ record._continue_discard_or_finish(lock: false)
104
+ end
105
+
106
+ active_jobs
107
+ end
108
+
109
+ # Enqueue jobs and add them to the batch
110
+ # @param block [Proc] Enqueue jobs within the block to add them to the batch
111
+ # @return [Array<ActiveJob::Base>] Active jobs added to the batch
112
+ def add(active_jobs = nil, &block)
113
+ record.save if record.new_record?
114
+
115
+ buffer = Bulk::Buffer.new
116
+ buffer.add(active_jobs)
117
+ buffer.capture(&block) if block
118
+
119
+ self.class.within_thread(batch_id: id) do
120
+ buffer.enqueue
121
+ end
122
+
123
+ buffer.active_jobs
124
+ end
125
+
126
+ def active_jobs
127
+ record.jobs.map(&:head_execution).map(&:active_job)
128
+ end
129
+
130
+ def callback_active_jobs
131
+ record.callback_jobs.map(&:head_execution).map(&:active_job)
132
+ end
133
+
134
+ def assign_properties(properties)
135
+ properties = properties.dup
136
+ batch_attrs = PROTECTED_PROPERTIES.index_with { |key| properties.delete(key) }.compact
137
+ record.assign_attributes(batch_attrs)
138
+ self.properties.merge!(properties)
139
+ end
140
+
141
+ def _record
142
+ record
143
+ end
144
+
145
+ private
146
+
147
+ attr_accessor :record
148
+ end
149
+ end
@@ -0,0 +1,95 @@
1
+ # frozen_string_literal: true
2
+
3
+ module GoodJob
4
+ class BatchRecord < BaseRecord
5
+ include Lockable
6
+
7
+ self.table_name = 'good_job_batches'
8
+
9
+ has_many :jobs, class_name: 'GoodJob::Job', inverse_of: :batch, foreign_key: :batch_id, dependent: nil
10
+ has_many :executions, class_name: 'GoodJob::Execution', foreign_key: :batch_id, inverse_of: :batch, dependent: nil
11
+ has_many :callback_jobs, class_name: 'GoodJob::Job', foreign_key: :batch_callback_id, dependent: nil # rubocop:disable Rails/InverseOf
12
+
13
+ scope :finished, -> { where.not(finished_at: nil) }
14
+ scope :discarded, -> { where.not(discarded_at: nil) }
15
+ scope :not_discarded, -> { where(discarded_at: nil) }
16
+ scope :succeeded, -> { finished.not_discarded }
17
+
18
+ alias_attribute :enqueued?, :enqueued_at
19
+ alias_attribute :discarded?, :discarded_at
20
+ alias_attribute :finished?, :finished_at
21
+
22
+ scope :display_all, (lambda do |after_created_at: nil, after_id: nil|
23
+ query = order(created_at: :desc, id: :desc)
24
+ if after_created_at.present? && after_id.present?
25
+ query = query.where(Arel.sql('(created_at, id) < (:after_created_at, :after_id)'), after_created_at: after_created_at, after_id: after_id)
26
+ elsif after_created_at.present?
27
+ query = query.where(Arel.sql('(after_created_at) < (:after_created_at)'), after_created_at: after_created_at)
28
+ end
29
+ query
30
+ end)
31
+
32
+ # Whether the batch has finished and no jobs were discarded
33
+ # @return [Boolean]
34
+ def succeeded?
35
+ !discarded? && finished?
36
+ end
37
+
38
+ def to_batch
39
+ Batch.new(_record: self)
40
+ end
41
+
42
+ def display_attributes
43
+ attributes.except('serialized_properties').merge(properties: properties)
44
+ end
45
+
46
+ def _continue_discard_or_finish(execution = nil, lock: true)
47
+ execution_discarded = execution && execution.error.present? && execution.retried_good_job_id.nil?
48
+ take_advisory_lock(lock) do
49
+ Batch.within_thread(batch_id: nil, batch_callback_id: id) do
50
+ reload
51
+ if execution_discarded && !discarded_at
52
+ update(discarded_at: Time.current)
53
+ on_discard.constantize.set(priority: callback_priority, queue: callback_queue_name).perform_later(to_batch, { event: :discard }) if on_discard.present?
54
+ end
55
+
56
+ if enqueued_at && !finished_at && jobs.where(finished_at: nil).count.zero?
57
+ update(finished_at: Time.current)
58
+ on_success.constantize.set(priority: callback_priority, queue: callback_queue_name).perform_later(to_batch, { event: :success }) if !discarded_at && on_success.present?
59
+ on_finish.constantize.set(priority: callback_priority, queue: callback_queue_name).perform_later(to_batch, { event: :finish }) if on_finish.present?
60
+ end
61
+ end
62
+ end
63
+ end
64
+
65
+ class PropertySerializer
66
+ def self.dump(value)
67
+ ActiveJob::Arguments.serialize([value]).first
68
+ end
69
+
70
+ def self.load(value)
71
+ ActiveJob::Arguments.deserialize([value]).first
72
+ end
73
+ end
74
+
75
+ attribute :serialized_properties, :json, default: -> { {} } # Can be set as default value in `serialize` as of Rails v6.1
76
+ serialize :serialized_properties, PropertySerializer
77
+ alias_attribute :properties, :serialized_properties
78
+
79
+ def properties=(value)
80
+ raise ArgumentError, "Properties must be a Hash" unless value.is_a?(Hash)
81
+
82
+ self.serialized_properties = value
83
+ end
84
+
85
+ private
86
+
87
+ def take_advisory_lock(value, &block)
88
+ if value
89
+ with_advisory_lock(function: "pg_advisory_lock", &block)
90
+ else
91
+ yield
92
+ end
93
+ end
94
+ end
95
+ end
@@ -20,8 +20,12 @@ module GoodJob
20
20
  self.table_name = 'good_jobs'
21
21
  self.advisory_lockable_column = 'active_job_id'
22
22
 
23
+ define_model_callbacks :perform
23
24
  define_model_callbacks :perform_unlocked, only: :after
24
25
 
26
+ set_callback :perform, :around, :reset_batch_values
27
+ set_callback :perform_unlocked, :after, :continue_discard_or_finish_batch
28
+
25
29
  # Parse a string representing a group of queues into a more readable data
26
30
  # structure.
27
31
  # @param string [String] Queue string
@@ -65,6 +69,9 @@ module GoodJob
65
69
  end
66
70
  end
67
71
 
72
+ belongs_to :batch, class_name: 'GoodJob::BatchRecord', optional: true, inverse_of: :executions
73
+ belongs_to :batch_callback, class_name: 'GoodJob::Batch', optional: true
74
+
68
75
  belongs_to :job, class_name: 'GoodJob::Job', foreign_key: 'active_job_id', primary_key: 'active_job_id', optional: true, inverse_of: :executions
69
76
  after_destroy -> { self.class.active_job_id(active_job_id).delete_all }, if: -> { @_destroy_job }
70
77
 
@@ -206,14 +213,24 @@ module GoodJob
206
213
  serialized_params: active_job.serialize,
207
214
  scheduled_at: active_job.scheduled_at,
208
215
  }
209
-
210
216
  execution_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
211
217
 
212
- if CurrentThread.cron_key
218
+ reenqueued_current_execution = CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
219
+ current_execution = CurrentThread.execution
220
+
221
+ if reenqueued_current_execution
222
+ if GoodJob::BatchRecord.migrated?
223
+ execution_args[:batch_id] = current_execution.batch_id
224
+ execution_args[:batch_callback_id] = current_execution.batch_callback_id
225
+ end
226
+ execution_args[:cron_key] = current_execution.cron_key
227
+ else
228
+ if GoodJob::BatchRecord.migrated?
229
+ execution_args[:batch_id] = GoodJob::Batch.current_batch_id
230
+ execution_args[:batch_callback_id] = GoodJob::Batch.current_batch_callback_id
231
+ end
213
232
  execution_args[:cron_key] = CurrentThread.cron_key
214
233
  execution_args[:cron_at] = CurrentThread.cron_at
215
- elsif CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
216
- execution_args[:cron_key] = CurrentThread.execution.cron_key
217
234
  end
218
235
 
219
236
  new(**execution_args.merge(overrides))
@@ -283,7 +300,6 @@ module GoodJob
283
300
 
284
301
  execution.save!
285
302
  active_job.provider_job_id = execution.id
286
-
287
303
  CurrentThread.execution.retried_good_job_id = execution.id if CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
288
304
 
289
305
  execution
@@ -296,27 +312,29 @@ module GoodJob
296
312
  # exception raised by the job, if any. If the job completed successfully,
297
313
  # the second array entry (the exception) will be +nil+ and vice versa.
298
314
  def perform
299
- raise PreviouslyPerformedError, 'Cannot perform a job that has already been performed' if finished_at
315
+ run_callbacks(:perform) do
316
+ raise PreviouslyPerformedError, 'Cannot perform a job that has already been performed' if finished_at
300
317
 
301
- self.performed_at = Time.current
302
- save! if GoodJob.preserve_job_records
318
+ self.performed_at = Time.current
319
+ save! if GoodJob.preserve_job_records
303
320
 
304
- result = execute
321
+ result = execute
305
322
 
306
- job_error = result.handled_error || result.unhandled_error
307
- self.error = [job_error.class, ERROR_MESSAGE_SEPARATOR, job_error.message].join if job_error
323
+ job_error = result.handled_error || result.unhandled_error
324
+ self.error = [job_error.class, ERROR_MESSAGE_SEPARATOR, job_error.message].join if job_error
308
325
 
309
- reenqueued = result.retried? || retried_good_job_id.present?
310
- if result.unhandled_error && GoodJob.retry_on_unhandled_error
311
- save!
312
- elsif GoodJob.preserve_job_records == true || reenqueued || (result.unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error) || cron_key.present?
313
- self.finished_at = Time.current
314
- save!
315
- else
316
- destroy_job
317
- end
326
+ reenqueued = result.retried? || retried_good_job_id.present?
327
+ if result.unhandled_error && GoodJob.retry_on_unhandled_error
328
+ save!
329
+ elsif GoodJob.preserve_job_records == true || reenqueued || (result.unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error) || cron_key.present?
330
+ self.finished_at = Time.current
331
+ save!
332
+ else
333
+ destroy_job
334
+ end
318
335
 
319
- result
336
+ result
337
+ end
320
338
  end
321
339
 
322
340
  # Tests whether this job is safe to be executed by this thread.
@@ -411,5 +429,13 @@ module GoodJob
411
429
  end
412
430
  end
413
431
  end
432
+
433
+ def reset_batch_values(&block)
434
+ GoodJob::Batch.within_thread(batch_id: nil, batch_callback_id: nil, &block)
435
+ end
436
+
437
+ def continue_discard_or_finish_batch
438
+ batch._continue_discard_or_finish(self) if GoodJob::BatchRecord.migrated? && batch.present?
439
+ end
414
440
  end
415
441
  end
@@ -28,6 +28,7 @@ module GoodJob
28
28
  self.primary_key = 'active_job_id'
29
29
  self.advisory_lockable_column = 'active_job_id'
30
30
 
31
+ belongs_to :batch, class_name: 'GoodJob::BatchRecord', inverse_of: :jobs, optional: true
31
32
  has_many :executions, -> { order(created_at: :asc) }, class_name: 'GoodJob::Execution', foreign_key: 'active_job_id', inverse_of: :job # rubocop:disable Rails/HasManyOrHasOneDependent
32
33
 
33
34
  # Only the most-recent unretried execution represents a "Job"
@@ -0,0 +1,108 @@
1
+ <div class="my-3 card" data-gj-poll-replace id="jobs-table">
2
+ <div class="list-group list-group-flush text-nowrap table-jobs" role="table">
3
+ <header class="list-group-item bg-light">
4
+ <div class="row small text-muted text-uppercase align-items-center">
5
+ <div class="col-4">Jobs</div>
6
+ <div class="col-1">Queue</div>
7
+ <div class="col-1">Priority</div>
8
+ <div class="col-1 text-end">Attempts</div>
9
+ <div class="col text-end">
10
+ <%= tag.button type: "button", class: "btn btn-sm text-muted", role: "button",
11
+ data: { bs_toggle: "collapse", bs_target: ".job-params" },
12
+ aria: { expanded: false, controls: jobs.map { |job| "##{dom_id(job, "params")}" }.join(" ") } do %>
13
+ <%= render_icon "info" %>
14
+ <span class="visually-hidden">Inspect</span>
15
+ <% end %>
16
+ </div>
17
+ </div>
18
+ </header>
19
+
20
+ <% if jobs.present? %>
21
+ <% jobs.each do |job| %>
22
+ <div role="row" class="list-group-item list-group-item-action py-3">
23
+ <div class="row align-items-center">
24
+ <div class="col-4">
25
+ <%= tag.code link_to(job.id, job_path(job), class: "small text-muted text-decoration-none") %>
26
+ <%= tag.h5 tag.code(link_to(job.job_class, job_path(job), class: "text-reset text-decoration-none")), class: "text-reset mb-0" %>
27
+ </div>
28
+ <div class="col-1">
29
+ <span class="badge bg-primary bg-opacity-25 text-dark font-monospace"><%= job.queue_name %></span>
30
+ </div>
31
+ <div class="col-1 small text-center">
32
+ <span class="font-monospace fw-bold"><%= job.priority %></span>
33
+ </div>
34
+ <div class="col-1 text-center">
35
+ <% if job.executions_count > 0 && job.status != :finished %>
36
+ <%= tag.span job.executions_count, class: "badge rounded-pill bg-danger", data: {
37
+ bs_toggle: "popover",
38
+ bs_trigger: "hover focus click",
39
+ bs_placement: "bottom",
40
+ bs_content: job.recent_error
41
+ } %>
42
+ <% else %>
43
+ <span class="badge bg-secondary bg-opacity-50 rounded-pill"><%= job.executions_count %></span>
44
+ <% end %>
45
+ </div>
46
+ <div class="col d-flex gap-3 align-items-center justify-content-end">
47
+ <%= tag.span relative_time(job.last_status_at), class: "small" %>
48
+ <%= status_badge job.status %>
49
+ </div>
50
+ <div class="col-auto">
51
+ <div class="dropdown float-end">
52
+ <button class="d-flex align-items-center btn btn-sm" type="button" id="<%= dom_id(job, :actions) %>" data-bs-toggle="dropdown" aria-expanded="false">
53
+ <%= render "good_job/shared/icons/dots" %>
54
+ <span class="visually-hidden">Actions</span>
55
+ </button>
56
+ <ul class="dropdown-menu shadow" aria-labelledby="<%= dom_id(job, :actions) %>">
57
+ <li>
58
+ <% job_reschedulable = job.status.in? [:scheduled, :retried, :queued] %>
59
+ <%= link_to reschedule_job_path(job.id), method: :put, class: "dropdown-item #{'disabled' unless job_reschedulable}", title: "Reschedule job", data: { confirm: "Confirm reschedule", disable: true } do %>
60
+ <%= render "good_job/shared/icons/skip_forward" %>
61
+ Reschedule
62
+ <% end %>
63
+ </li>
64
+ <li>
65
+ <% job_discardable = job.status.in? [:scheduled, :retried, :queued] %>
66
+ <%= link_to discard_job_path(job.id), method: :put, class: "dropdown-item #{'disabled' unless job_discardable}", title: "Discard job", data: { confirm: "Confirm discard", disable: true } do %>
67
+ <%= render "good_job/shared/icons/stop" %>
68
+ Discard
69
+ <% end %>
70
+ </li>
71
+ <li>
72
+ <%= link_to retry_job_path(job.id), method: :put, class: "dropdown-item #{'disabled' unless job.status == :discarded}", title: "Retry job", data: { confirm: "Confirm retry", disable: true } do %>
73
+ <%= render "good_job/shared/icons/arrow_clockwise" %>
74
+ Retry
75
+ <% end %>
76
+ </li>
77
+ <li>
78
+ <%= link_to job_path(job.id), method: :delete, class: "dropdown-item #{'disabled' unless job.status.in? [:discarded, :finished]}", title: "Destroy job", data: { confirm: "Confirm destroy", disable: true } do %>
79
+ <%= render_icon "trash" %>
80
+ Destroy
81
+ <% end %>
82
+ </li>
83
+
84
+ <li>
85
+ <%= link_to "##{dom_id(job, 'params')}",
86
+ class: "dropdown-item",
87
+ data: { bs_toggle: "collapse" },
88
+ aria: { expanded: false, controls: dom_id(job, "params") } do %>
89
+ <%= render_icon "info" %>
90
+ Inspect
91
+ <% end %>
92
+ </li>
93
+ </ul>
94
+ </div>
95
+ </div>
96
+ </div>
97
+ </div>
98
+ <%= tag.div id: dom_id(job, "params"), class: "job-params list-group-item collapse small bg-dark text-light" do %>
99
+ <%= tag.pre JSON.pretty_generate(job.display_serialized_params) %>
100
+ <% end %>
101
+ <% end %>
102
+ <% else %>
103
+ <div class="list-group-item py-4 text-center text-muted">
104
+ No jobs found.
105
+ </div>
106
+ <% end %>
107
+ </>
108
+ </div>
@@ -0,0 +1,61 @@
1
+ <div class="my-3 card" data-gj-poll-replace id="batches-table">
2
+ <div class="list-group list-group-flush text-nowrap table-batches" role="table">
3
+ <header class="list-group-item bg-light">
4
+ <div class="row small text-muted text-uppercase align-items-center">
5
+ <div class="col-4">Name</div>
6
+ <div class="col-1">Created</div>
7
+ <div class="col-1">Enqueued</div>
8
+ <div class="col-1">Discarded</div>
9
+ <div class="col-1">Finished</div>
10
+ <div class="col">Jobs</div>
11
+ <div class="col text-end">
12
+ <%= tag.button type: "button", class: "btn btn-sm text-muted", role: "button",
13
+ data: { bs_toggle: "collapse", bs_target: ".batch-properties" },
14
+ aria: { expanded: false, controls: batches.map { |batch| "##{dom_id(batch, "params")}" }.join(" ") } do %>
15
+ <%= render_icon "info" %>
16
+ <span class="visually-hidden">Inspect</span>
17
+ <% end %>
18
+ </div>
19
+ </div>
20
+ </header>
21
+
22
+ <% if batches.present? %>
23
+ <% batches.each do |batch| %>
24
+ <div id="<%= dom_id(batch) %>" class="list-group-item py-3" role="row">
25
+ <div class="row align-items-center">
26
+ <div class="col-4">
27
+ <%= link_to batch_path(batch), class: "text-decoration-none" do %>
28
+ <code class="small text-muted">
29
+ <%= batch.id %>
30
+ </code>
31
+ <h5 class=""><code><%= batch.on_finish %></code></h5>
32
+ <div class="text-muted"><%= batch.description %></div>
33
+ <% end %>
34
+ </div>
35
+ <div class="col-1 text-wrap"><%= relative_time(batch.created_at) %></div>
36
+ <div class="col-1 text-wrap"><%= relative_time(batch.enqueued_at) if batch.enqueued_at %></div>
37
+ <div class="col-1 text-wrap"><%= relative_time(batch.discarded_at) if batch.discarded_at %></div>
38
+ <div class="col-1 text-wrap"><%= relative_time(batch.finished_at) if batch.finished_at %></div>
39
+ <div class="col"><%= batch.jobs.count %></div>
40
+ <div class="col text-end">
41
+ <%= tag.button type: "button", class: "btn btn-sm text-muted ms-auto", role: "button",
42
+ title: "Inspect",
43
+ data: { bs_toggle: "collapse", bs_target: "##{dom_id(batch, 'properties')}" },
44
+ aria: { expanded: false, controls: dom_id(batch, "state") } do %>
45
+ <%= render_icon "info" %>
46
+ <span class="visually-hidden">Inspect</span>
47
+ <% end %>
48
+ </div>
49
+ </div>
50
+ </div>
51
+ <%= tag.div id: dom_id(batch, "properties"), class: "batch-properties list-group-item collapse small bg-dark text-light" do %>
52
+ <%= tag.pre JSON.pretty_generate(batch.properties) %>
53
+ <% end %>
54
+ <% end %>
55
+ <% else %>
56
+ <div class="list-group-item py-4 text-center text-muted">
57
+ No batches found.
58
+ </div>
59
+ <% end %>
60
+ </div>
61
+ </div>
@@ -0,0 +1,16 @@
1
+ <% if GoodJob::BatchRecord.migrated? %>
2
+ <%= render 'good_job/batches/table', batches: @filter.records, filter: @filter %>
3
+ <% if @filter.records.present? %>
4
+ <nav aria-label="Batch pagination" class="mt-3">
5
+ <ul class="pagination">
6
+ <li class="page-item">
7
+ <%= link_to(@filter.to_params(after_created_at: @filter.last.created_at, after_id: @filter.last.id), class: "page-link") do %>
8
+ Older batches <span aria-hidden="true">&raquo;</span>
9
+ <% end %>
10
+ </li>
11
+ </ul>
12
+ </nav>
13
+ <% end %>
14
+ <% else %>
15
+ <h3 class="text-center my-5">GoodJob has pending database migrations.</h3>
16
+ <% end %>
@@ -0,0 +1,30 @@
1
+ <div class="break-out bg-light border-bottom py-2 mb-3">
2
+ <div class="container-fluid pt-2">
3
+ <div class="row align-items-center">
4
+ <div class="col-5">
5
+ <nav aria-label="breadcrumb">
6
+ <ol class="breadcrumb small mb-0">
7
+ <li class="breadcrumb-item"><%= link_to "Batches", batches_path %></li>
8
+ <li class="breadcrumb-item active" aria-current="page"><%= tag.code @batch.id, class: "text-muted" %></li>
9
+ </ol>
10
+ <h2 class="h5 mt-2"><%= @batch.description %></h2>
11
+ </nav>
12
+ </div>
13
+ </div>
14
+ </div>
15
+ </div>
16
+
17
+ <div class="my-4">
18
+ <h5>Attributes</h5>
19
+ <%= tag.pre JSON.pretty_generate @batch.display_attributes, class: 'text-wrap text-break' %>
20
+ </div>
21
+
22
+ <div class="my-4">
23
+ <h5>Callback Jobs</h5>
24
+ <%= render 'jobs', jobs: @batch.callback_jobs.reverse %>
25
+ </div>
26
+
27
+ <div class="my-4">
28
+ <h5>Batched Jobs</h5>
29
+ <%= render 'jobs', jobs: @batch.jobs.reverse %>
30
+ </div>
@@ -14,6 +14,13 @@
14
14
  <span class="badge bg-secondary rounded-pill"><%= number_to_human(jobs_count) %></span>
15
15
  <% end %>
16
16
  </li>
17
+ <li class="nav-item">
18
+ <%= link_to batches_path, class: ["nav-link", ("active" if controller_name == 'batches')] do %>
19
+ <%= "Batches" %>
20
+ <% batches_count = GoodJob::BatchRecord.migrated? ? GoodJob::BatchRecord.all.size : 0 %>
21
+ <span class="badge bg-secondary rounded-pill"><%= batches_count %></span>
22
+ <% end %>
23
+ </li>
17
24
  <li class="nav-item">
18
25
  <%= link_to cron_entries_path, class: ["nav-link", ("active" if controller_name == 'cron_entries')] do %>
19
26
  <%= t(".cron_schedules") %>
data/config/routes.rb CHANGED
@@ -15,6 +15,8 @@ GoodJob::Engine.routes.draw do
15
15
  end
16
16
  end
17
17
 
18
+ resources :batches, only: %i[index show]
19
+
18
20
  resources :cron_entries, only: %i[index show], param: :cron_key do
19
21
  member do
20
22
  post :enqueue
@@ -19,6 +19,23 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
19
19
  t.text :cron_key
20
20
  t.uuid :retried_good_job_id
21
21
  t.datetime :cron_at
22
+
23
+ t.uuid :batch_id
24
+ t.uuid :batch_callback_id
25
+ end
26
+
27
+ create_table :good_job_batches, id: :uuid do |t|
28
+ t.timestamps
29
+ t.text :description
30
+ t.jsonb :serialized_properties
31
+ t.text :on_finish
32
+ t.text :on_success
33
+ t.text :on_discard
34
+ t.text :callback_queue_name
35
+ t.integer :callback_priority
36
+ t.datetime :enqueued_at
37
+ t.datetime :discarded_at
38
+ t.datetime :finished_at
22
39
  end
23
40
 
24
41
  create_table :good_job_processes, id: :uuid do |t|
@@ -43,5 +60,7 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
43
60
  add_index :good_jobs, [:finished_at], where: "retried_good_job_id IS NULL AND finished_at IS NOT NULL", name: :index_good_jobs_jobs_on_finished_at
44
61
  add_index :good_jobs, [:priority, :created_at], order: { priority: "DESC NULLS LAST", created_at: :asc },
45
62
  where: "finished_at IS NULL", name: :index_good_jobs_jobs_on_priority_created_at_when_unfinished
63
+ add_index :good_jobs, [:batch_id], where: "batch_id IS NOT NULL"
64
+ add_index :good_jobs, [:batch_callback_id], where: "batch_callback_id IS NOT NULL"
46
65
  end
47
66
  end
@@ -0,0 +1,34 @@
1
+ # frozen_string_literal: true
2
+ class CreateGoodJobBatches < ActiveRecord::Migration<%= migration_version %>
3
+ def change
4
+ reversible do |dir|
5
+ dir.up do
6
+ # Ensure this incremental update migration is idempotent
7
+ # with monolithic install migration.
8
+ return if connection.table_exists?(:good_job_batches)
9
+ end
10
+ end
11
+
12
+ create_table :good_job_batches, id: :uuid do |t|
13
+ t.timestamps
14
+ t.text :description
15
+ t.jsonb :serialized_properties
16
+ t.text :on_finish
17
+ t.text :on_success
18
+ t.text :on_discard
19
+ t.text :callback_queue_name
20
+ t.integer :callback_priority
21
+ t.datetime :enqueued_at
22
+ t.datetime :discarded_at
23
+ t.datetime :finished_at
24
+ end
25
+
26
+ change_table :good_jobs do |t|
27
+ t.uuid :batch_id
28
+ t.uuid :batch_callback_id
29
+
30
+ t.index :batch_id, where: "batch_id IS NOT NULL"
31
+ t.index :batch_callback_id, where: "batch_callback_id IS NOT NULL"
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+ module GoodJob
3
+ module ActiveJobExtensions
4
+ module Batches
5
+ extend ActiveSupport::Concern
6
+
7
+ def batch
8
+ @_batch ||= CurrentThread.execution&.batch&.to_batch
9
+ end
10
+ alias batch? batch
11
+ end
12
+ end
13
+ end
data/lib/good_job/bulk.rb CHANGED
@@ -15,19 +15,14 @@ module GoodJob
15
15
  # @param active_jobs [Array<ActiveJob::Base>] Active Jobs to be buffered.
16
16
  # @param queue_adapter Override the jobs implict queue adapter with an explicit one.
17
17
  # @return [nil, Array<ActiveJob::Base>] The ActiveJob instances that have been buffered; nil if no active buffer
18
- def self.capture(active_jobs = nil, queue_adapter: nil)
19
- raise(ArgumentError, "Use either the block form or the argument form, not both") if block_given? && active_jobs
20
-
21
- if block_given?
22
- begin
23
- original_buffer = current_buffer
24
- self.current_buffer = Buffer.new
25
- yield
26
- current_buffer.active_jobs
27
- ensure
28
- self.current_buffer = original_buffer
29
- end
30
- else
18
+ def self.capture(active_jobs = nil, queue_adapter: nil, &block)
19
+ raise(ArgumentError, "Use either the block form or the argument form, not both") if block && active_jobs
20
+
21
+ if block
22
+ buffer = Buffer.new
23
+ buffer.capture(&block)
24
+ buffer.active_jobs
25
+ elsif current_buffer
31
26
  current_buffer&.add(active_jobs, queue_adapter: queue_adapter)
32
27
  end
33
28
  end
@@ -35,16 +30,15 @@ module GoodJob
35
30
  # Capture jobs to a buffer and enqueue them all at once; or enqueue the current buffer.
36
31
  # @param active_jobs [Array<ActiveJob::Base>] Active Jobs to be enqueued.
37
32
  # @return [Array<ActiveJob::Base>] The ActiveJob instances that have been captured; check provider_job_id to confirm enqueued.
38
- def self.enqueue(active_jobs = nil)
39
- raise(ArgumentError, "Use either the block form or the argument form, not both") if block_given? && active_jobs
33
+ def self.enqueue(active_jobs = nil, &block)
34
+ raise(ArgumentError, "Use either the block form or the argument form, not both") if block && active_jobs
40
35
 
41
- if block_given?
42
- capture do
43
- yield
44
- current_buffer&.enqueue
45
- end
36
+ buffer = Buffer.new
37
+ if block
38
+ buffer.capture(&block)
39
+ buffer.enqueue
40
+ buffer.active_jobs
46
41
  elsif active_jobs.present?
47
- buffer = Buffer.new
48
42
  buffer.add(active_jobs)
49
43
  buffer.enqueue
50
44
  buffer.active_jobs
@@ -69,8 +63,18 @@ module GoodJob
69
63
  @values = []
70
64
  end
71
65
 
66
+ def capture
67
+ original_buffer = Bulk.current_buffer
68
+ Bulk.current_buffer = self
69
+ yield
70
+ ensure
71
+ Bulk.current_buffer = original_buffer
72
+ end
73
+
72
74
  def add(active_jobs, queue_adapter: nil)
73
75
  new_pairs = Array(active_jobs).map do |active_job|
76
+ raise ArgumentError, "Expected an ActiveJob::Base instance, got #{active_job.class}" unless active_job.is_a?(ActiveJob::Base)
77
+
74
78
  adapter = queue_adapter || active_job.class.queue_adapter
75
79
  raise Error, "Jobs must have a Queue Adapter" unless adapter
76
80
 
@@ -156,7 +156,7 @@ module GoodJob
156
156
  destroyed_records_count = event.payload[:destroyed_records_count]
157
157
 
158
158
  info do
159
- "GoodJob destroyed #{destroyed_records_count} preserved #{'job'.pluralize(destroyed_records_count)} finished before #{timestamp}."
159
+ "GoodJob destroyed #{destroyed_records_count} preserved job execution #{'records'.pluralize(destroyed_records_count)} finished before #{timestamp}."
160
160
  end
161
161
  end
162
162
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
  module GoodJob
3
3
  # GoodJob gem version.
4
- VERSION = '3.9.0'
4
+ VERSION = '3.10.1'
5
5
  end
data/lib/good_job.rb CHANGED
@@ -8,6 +8,7 @@ require "good_job/engine"
8
8
  require "good_job/adapter"
9
9
  require "active_job/queue_adapters/good_job_adapter"
10
10
  require "good_job/active_job_extensions/concurrency"
11
+ require "good_job/active_job_extensions/batches"
11
12
 
12
13
  require "good_job/assignable_connection"
13
14
  require "good_job/bulk"
@@ -143,7 +144,7 @@ module GoodJob
143
144
  end
144
145
  end
145
146
 
146
- # Destroys preserved job records.
147
+ # Destroys preserved job and batch records.
147
148
  # By default, GoodJob destroys job records when the job is performed and this
148
149
  # method is not necessary. However, when `GoodJob.preserve_job_records = true`,
149
150
  # the jobs will be preserved in the database. This is useful when wanting to
@@ -151,7 +152,7 @@ module GoodJob
151
152
  # If you are preserving job records this way, use this method regularly to
152
153
  # destroy old records and preserve space in your database.
153
154
  # @params older_than [nil,Numeric,ActiveSupport::Duration] Jobs older than this will be destroyed (default: +86400+).
154
- # @return [Integer] Number of jobs that were destroyed.
155
+ # @return [Integer] Number of job execution records and batches that were destroyed.
155
156
  def self.cleanup_preserved_jobs(older_than: nil)
156
157
  older_than ||= GoodJob.configuration.cleanup_preserved_jobs_before_seconds_ago
157
158
  timestamp = Time.current - older_than
@@ -160,10 +161,19 @@ module GoodJob
160
161
  ActiveSupport::Notifications.instrument("cleanup_preserved_jobs.good_job", { older_than: older_than, timestamp: timestamp }) do |payload|
161
162
  old_jobs = GoodJob::Job.where('finished_at <= ?', timestamp)
162
163
  old_jobs = old_jobs.succeeded unless include_discarded
163
- old_jobs_count = old_jobs.count
164
-
165
- GoodJob::Execution.where(job: old_jobs).delete_all
166
- payload[:destroyed_records_count] = old_jobs_count
164
+ deleted_executions_count = GoodJob::Execution.where(job: old_jobs).delete_all
165
+
166
+ if GoodJob::BatchRecord.migrated?
167
+ old_batches = GoodJob::BatchRecord.where('finished_at <= ?', timestamp)
168
+ old_batches = old_batches.succeeded unless include_discarded
169
+ deleted_batches_count = old_batches.delete_all
170
+ else
171
+ deleted_batches_count = 0
172
+ end
173
+
174
+ payload[:destroyed_executions_count] = deleted_executions_count
175
+ payload[:destroyed_batches_count] = deleted_batches_count
176
+ payload[:destroyed_records_count] = deleted_executions_count + deleted_batches_count
167
177
  end
168
178
  end
169
179
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: good_job
3
3
  version: !ruby/object:Gem::Version
4
- version: 3.9.0
4
+ version: 3.10.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ben Sheldon
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2023-01-31 00:00:00.000000000 Z
11
+ date: 2023-02-06 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activejob
@@ -334,16 +334,20 @@ files:
334
334
  - app/charts/good_job/scheduled_by_queue_chart.rb
335
335
  - app/controllers/good_job/application_controller.rb
336
336
  - app/controllers/good_job/assets_controller.rb
337
+ - app/controllers/good_job/batches_controller.rb
337
338
  - app/controllers/good_job/cron_entries_controller.rb
338
339
  - app/controllers/good_job/jobs_controller.rb
339
340
  - app/controllers/good_job/processes_controller.rb
340
341
  - app/filters/good_job/base_filter.rb
342
+ - app/filters/good_job/batches_filter.rb
341
343
  - app/filters/good_job/jobs_filter.rb
342
344
  - app/helpers/good_job/application_helper.rb
343
345
  - app/models/concerns/good_job/filterable.rb
344
346
  - app/models/concerns/good_job/reportable.rb
345
347
  - app/models/good_job/active_job_job.rb
346
348
  - app/models/good_job/base_record.rb
349
+ - app/models/good_job/batch.rb
350
+ - app/models/good_job/batch_record.rb
347
351
  - app/models/good_job/cron_entry.rb
348
352
  - app/models/good_job/execution.rb
349
353
  - app/models/good_job/execution_result.rb
@@ -351,6 +355,10 @@ files:
351
355
  - app/models/good_job/lockable.rb
352
356
  - app/models/good_job/process.rb
353
357
  - app/models/good_job/setting.rb
358
+ - app/views/good_job/batches/_jobs.erb
359
+ - app/views/good_job/batches/_table.erb
360
+ - app/views/good_job/batches/index.html.erb
361
+ - app/views/good_job/batches/show.html.erb
354
362
  - app/views/good_job/cron_entries/index.html.erb
355
363
  - app/views/good_job/cron_entries/show.html.erb
356
364
  - app/views/good_job/jobs/_executions.erb
@@ -390,8 +398,10 @@ files:
390
398
  - lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb.erb
391
399
  - lib/generators/good_job/templates/update/migrations/02_create_good_job_settings.rb.erb
392
400
  - lib/generators/good_job/templates/update/migrations/03_create_index_good_jobs_jobs_on_priority_created_at_when_unfinished.rb.erb
401
+ - lib/generators/good_job/templates/update/migrations/04_create_good_job_batches.rb.erb
393
402
  - lib/generators/good_job/update_generator.rb
394
403
  - lib/good_job.rb
404
+ - lib/good_job/active_job_extensions/batches.rb
395
405
  - lib/good_job/active_job_extensions/concurrency.rb
396
406
  - lib/good_job/adapter.rb
397
407
  - lib/good_job/assignable_connection.rb