good_job 3.9.0 → 3.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 69560b4af2d1cab2a3783fb80573794066c3704aacafa7bd315a9dc6e1b75441
4
- data.tar.gz: 55dff98e3c8ebbcebd5f09664d02f4ddd23b298c0ee2bc7cb163e43f6963de96
3
+ metadata.gz: 22496fe725c6ad616c39f243cf8f53381dee64a5402d3a199c8c604bff2c4eb3
4
+ data.tar.gz: 6d9791dd28f28ddf9317f2770029323754fa32c60403f336f23db8020cdecc51
5
5
  SHA512:
6
- metadata.gz: dd4763e6d473a22ea7b0f6f7b501249c7e5e29d06c1fdd9b3d028c923ba407fb485eea748fd37857a5eb09f5b26659d67f4792f4a66848542bfc1f47f7384577
7
- data.tar.gz: 89e165d003d156fa16450762b8a8a269172509557ab59be60e7d54794d079c9963f8c11f45047ecfdea8e0963d901c8d9e1c297743392a05817d21d417c78dab
6
+ metadata.gz: e182b3aa78b38fd75baa5e0c14704baa99b7aa98f34ff011006543ce7a356e6eae8ac45e605d65d985b8631f6f185666f78e5bb1be1f39feebd62c159535a7e5
7
+ data.tar.gz: acf8a1f601e07142b0dbaea2f2e229fa3d9b7f8c50dd6232531f8ff50cf888d61f88f2a02ae423ffaa8a9a6533bd1cd0522e7cfa849abb20db8dc538adf75ae3
data/CHANGELOG.md CHANGED
@@ -1,5 +1,19 @@
1
1
  # Changelog
2
2
 
3
+ ## [v3.10.0](https://github.com/bensheldon/good_job/tree/v3.10.0) (2023-02-04)
4
+
5
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.9.0...v3.10.0)
6
+
7
+ **Implemented enhancements:**
8
+
9
+ - Implement `GoodJob::Batch` [\#712](https://github.com/bensheldon/good_job/pull/712) ([bensheldon](https://github.com/bensheldon))
10
+
11
+ **Closed issues:**
12
+
13
+ - Support for Ruby 3.2 [\#785](https://github.com/bensheldon/good_job/issues/785)
14
+ - Custom table names [\#748](https://github.com/bensheldon/good_job/issues/748)
15
+ - Health check issue with cron scheduler job [\#741](https://github.com/bensheldon/good_job/issues/741)
16
+
3
17
  ## [v3.9.0](https://github.com/bensheldon/good_job/tree/v3.9.0) (2023-01-31)
4
18
 
5
19
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v3.8.0...v3.9.0)
data/README.md CHANGED
@@ -41,9 +41,11 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
41
41
  - [Dashboard](#dashboard)
42
42
  - [API-only Rails applications](#api-only-rails-applications)
43
43
  - [Live Polling](#live-polling)
44
- - [ActiveJob concurrency](#activejob-concurrency)
44
+ - [Concurrency controls](#concurrency-controls)
45
45
  - [How concurrency controls work](#how-concurrency-controls-work)
46
46
  - [Cron-style repeating/recurring jobs](#cron-style-repeatingrecurring-jobs)
47
+ - [Bulk enqueue](#bulk-enqueue)
48
+ - [Batches](#batches)
47
49
  - [Updating](#updating)
48
50
  - [Upgrading minor versions](#upgrading-minor-versions)
49
51
  - [Upgrading v2 to v3](#upgrading-v2-to-v3)
@@ -58,7 +60,6 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
58
60
  - [Database connections](#database-connections)
59
61
  - [Production setup](#production-setup)
60
62
  - [Queue performance with Queue Select Limit](#queue-performance-with-queue-select-limit)
61
- - [Bulk enqueue](#bulk-enqueue)
62
63
  - [Execute jobs async / in-process](#execute-jobs-async--in-process)
63
64
  - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
64
65
  - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
@@ -392,7 +393,7 @@ end
392
393
 
393
394
  The Dashboard can be set to automatically refresh by checking "Live Poll" in the Dashboard header, or by setting `?poll=10` with the interval in seconds (default 30 seconds).
394
395
 
395
- ### ActiveJob concurrency
396
+ ### Concurrency controls
396
397
 
397
398
  GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_. Limiting concurrency can help prevent duplicate, double or unnecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
398
399
 
@@ -485,6 +486,196 @@ config.good_job.cron = {
485
486
  }
486
487
  ```
487
488
 
489
+ ### Bulk enqueue
490
+
491
+ GoodJob's Bulk-enqueue functionality can buffer and enqueue multiple jobs at once, using a single INSERT statement. This can more performant when enqueuing a large number of jobs.
492
+
493
+ ```ruby
494
+ # Capture jobs using `.perform_later`:
495
+ active_jobs = GoodJob::Bulk.enqueue do
496
+ MyJob.perform_later
497
+ AnotherJob.perform_later
498
+ # If an exception is raised within this block, no jobs will be inserted.
499
+ end
500
+
501
+ # All ActiveJob instances are returned from GoodJob::Bulk.enqueue.
502
+ # Jobs that have been successfully enqueued have a `provider_job_id` set.
503
+ active_jobs.all?(&:provider_job_id)
504
+
505
+ # Bulk enqueue ActiveJob instances directly without using `.perform_later`:
506
+ GoodJob::Bulk.enqueue(MyJob.new, AnotherJob.new)
507
+ ```
508
+
509
+ ### Batches
510
+
511
+ Batches track a set of jobs, and enqueue an optional callback job when all of the jobs have finished (succeeded or discarded).
512
+
513
+ - A simple example that enqueues your `MyBatchCallbackJob` after the two jobs have finished, and passes along the current user as a batch property:
514
+
515
+ ```ruby
516
+ GoodJob::Batch.enqueue(on_finish: MyBatchCallbackJob, user: current_user) do
517
+ MyJob.perform_later
518
+ OtherJob.perform_later
519
+ end
520
+
521
+ # When these jobs have finished, it will enqueue your `MyBatchCallbackJob.perform_later(batch, options)`
522
+ class MyBatchCallbackJob < ApplicationJob
523
+ # Callback jobs must accept a `batch` and `options` argument
524
+ def perform(batch, params)
525
+ # The batch object will contain the Batch's properties, which are mutable
526
+ batch.properties[:user] # => <User id: 1, ...>
527
+
528
+ # Params is a hash containing additional context (more may be added in the future)
529
+ params[:event] # => :finish, :success, :discard
530
+ end
531
+ end
532
+ ```
533
+
534
+ - Jobs can be added to an existing batch. Jobs in a batch are enqueued and performed immediately/asynchronously. The final callback job will not be enqueued until `GoodJob::Batch#enqueue` is called.
535
+
536
+ ```ruby
537
+ batch = GoodJob::Batch.add do
538
+ 10.times { MyJob.perform_later }
539
+ end
540
+ batch.add do
541
+ 10.times { OtherJob.perform_later }
542
+ end
543
+ batch.enqueue(on_finish: MyBatchCallbackJob, age: 42)
544
+ ```
545
+
546
+ - If you need to access the batch within a job that is part of the batch, include [`GoodJob::ActiveJobExtensions::Batches`](lib/good_job/active_job_extensions/batches.rb) in your job class:
547
+
548
+ ```ruby
549
+ class MyJob < ApplicationJob
550
+ include GoodJob::ActiveJobExtensions::Batches
551
+
552
+ def perform
553
+ self.batch # => <GoodJob::Batch id: 1, ...>
554
+ end
555
+ end
556
+ ```
557
+
558
+ - [`GoodJob::Batch`](app/models/good_job/batch.rb) has a number of assignable attributes and methods:
559
+
560
+ ```ruby
561
+ batch = GoodJob::Batch.new
562
+ batch.description = "My batch"
563
+ batch.on_finish = "MyBatchCallbackJob" # Callback job when all jobs have finished
564
+ batch.on_success = "MyBatchCallbackJob" # Callback job when/if all jobs have succeeded
565
+ batch.on_discard = "MyBatchCallbackJob" # Callback job when the first job in the batch is discarded
566
+ batch.callback_queue_name = "special_queue" # Optional queue for callback jobs, otherwise will defer to job class
567
+ batch.callback_priority = 10 # Optional priority name for callback jobs, otherwise will defer to job class
568
+ batch.properties = { age: 42 } # Custom data and state to attach to the batch
569
+ batch.add do
570
+ MyJob.perform_later
571
+ end
572
+ batch.enqueue
573
+
574
+ batch.discarded? # => Boolean
575
+ batch.discarded_at # => <DateTime>
576
+ batch.finished? # => Boolean
577
+ batch.finished_at # => <DateTime>
578
+ batch.succeeded? # => Boolean
579
+ batch.active_jobs # => Array of ActiveJob::Base-inherited jobs that are part of the batch
580
+
581
+ batch = GoodJob::Batch.find(batch.id)
582
+ batch.description = "Updated batch description"
583
+ batch.save
584
+ batch.reload
585
+ ```
586
+
587
+ ### Batch callback jobs
588
+
589
+ Batch callbacks are Active Job jobs that are enqueued at certain events during the execution of jobs within the batch:
590
+
591
+ - `:finish` - Enqueued when all jobs in the batch have finished, after all retries. Jobs will either be discarded or succeeded.
592
+ - `:success` - Enqueued only when all jobs in the batch have finished and succeeded.
593
+ - `:discard` - Enqueued immediately the first time a job in the batch is discarded.
594
+
595
+ Callback jobs must accept a `batch` and `params` argument in their `perform` method:
596
+
597
+ ```ruby
598
+ class MyBatchCallbackJob < ApplicationJob
599
+ def perform(batch, params)
600
+ # The batch object will contain the Batch's properties
601
+ batch.properties[:user] # => <User id: 1, ...>
602
+ # Batches are mutable
603
+ batch.properties[:user] = User.find(2)
604
+ batch.save
605
+
606
+ # Params is a hash containing additional context (more may be added in the future)
607
+ params[:event] # => :finish, :success, :discard
608
+ end
609
+ end
610
+ ```
611
+
612
+ #### Complex batches
613
+
614
+ Consider a multi-stage batch with both parallel and serial job steps:
615
+
616
+ ```mermaid
617
+ graph TD
618
+ 0{"BatchJob\n{ stage: nil }"}
619
+ 0 --> a["WorkJob]\n{ step: a }"]
620
+ 0 --> b["WorkJob]\n{ step: b }"]
621
+ 0 --> c["WorkJob]\n{ step: c }"]
622
+ a --> 1
623
+ b --> 1
624
+ c --> 1
625
+ 1{"BatchJob\n{ stage: 1 }"}
626
+ 1 --> d["WorkJob]\n{ step: d }"]
627
+ 1 --> e["WorkJob]\n{ step: e }"]
628
+ e --> f["WorkJob]\n{ step: f }"]
629
+ d --> 2
630
+ f --> 2
631
+ 2{"BatchJob\n{ stage: 2 }"}
632
+ ```
633
+
634
+ This can be implemented with a single, mutable batch job:
635
+
636
+ ```ruby
637
+ class WorkJob < ApplicationJob
638
+ include GoodJob::ActiveJobExtensions::Batches
639
+
640
+ def perform(step)
641
+ # ...
642
+ if step == 'e'
643
+ batch.add { WorkJob.perform_later('f') }
644
+ end
645
+ end
646
+ end
647
+
648
+ class BatchJob < ApplicationJob
649
+ def perform(batch, options)
650
+ if batch.properties[:stage].nil?
651
+ batch.enqueue(stage: 1) do
652
+ WorkJob.perform_later('a')
653
+ WorkJob.perform_later('b')
654
+ WorkJob.perform_later('c')
655
+ end
656
+ elsif batch.properties[:stage] == 1
657
+ batch.enqueue(stage: 2) do
658
+ WorkJob.perform_later('d')
659
+ WorkJob.perform_later('e')
660
+ end
661
+ elsif batch.properties[:stage] == 2
662
+ # ...
663
+ end
664
+ end
665
+ end
666
+
667
+ GoodJob::Batch.enqueue(on_finish: BatchJob)
668
+ ```
669
+
670
+ #### Other batch details
671
+
672
+ - Whether to enqueue a callback job is evaluated once the batch is in an `enqueued?`-state by using `GoodJob::Batch.enqueue` or `batch.enqueue`.
673
+ - Callback job enqueueing will be re-triggered if additional jobs are `enqueue`'d to the batch; use `add` to add jobs to the batch without retriggering callback jobs.
674
+ - Callback jobs will be enqueued even if the batch contains no jobs.
675
+ - Callback jobs perform asynchronously. It's possible that `:finish` and `:success` or `:discard` callback jobs perform at the same time. Keep this in mind when updating batch properties.
676
+ - Batch properties are serialized using Active Job serialization. This is flexible, but can lead to deserialization errors if a GlobalID record is directly referenced but is subsequently deleted and thus unloadable.
677
+ - 🚧Batches are a work in progress. Please let us know what would be helpful to improve their functionality and usefulness.
678
+
488
679
  ### Updating
489
680
 
490
681
  GoodJob follows semantic versioning, though updates may be encouraged through deprecation warnings in minor versions.
@@ -794,26 +985,6 @@ To explain where this value is used, here is the pseudo-query that GoodJob uses
794
985
  )
795
986
  ```
796
987
 
797
- ### Bulk enqueue
798
-
799
- GoodJob's Bulk-enqueue functionality can buffer and enqueue multiple jobs at once, using a single INSERT statement. This can more performant when enqueuing a large number of jobs.
800
-
801
- ```ruby
802
- # Capture jobs using `.perform_later`:
803
- active_jobs = GoodJob::Bulk.enqueue do
804
- MyJob.perform_later
805
- AnotherJob.perform_later
806
- # If an exception is raised within this block, no jobs will be inserted.
807
- end
808
-
809
- # All ActiveJob instances are returned from GoodJob::Bulk.enqueue.
810
- # Jobs that have been successfully enqueued have a `provider_job_id` set.
811
- active_jobs.all?(&:provider_job_id)
812
-
813
- # Bulk enqueue ActiveJob instances directly without using `.perform_later`:
814
- GoodJob::Bulk.enqueue(MyJob.new, AnotherJob.new)
815
- ```
816
-
817
988
  ### Execute jobs async / in-process
818
989
 
819
990
  GoodJob can execute jobs "async" in the same process as the web server (e.g. `bin/rails s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
@@ -0,0 +1,12 @@
1
+ # frozen_string_literal: true
2
+ module GoodJob
3
+ class BatchesController < GoodJob::ApplicationController
4
+ def index
5
+ @filter = BatchesFilter.new(params)
6
+ end
7
+
8
+ def show
9
+ @batch = GoodJob::BatchRecord.find(params[:id])
10
+ end
11
+ end
12
+ end
@@ -0,0 +1,21 @@
1
+ # frozen_string_literal: true
2
+ module GoodJob
3
+ class BatchesFilter < BaseFilter
4
+ def records
5
+ after_created_at = params[:after_created_at].present? ? Time.zone.parse(params[:after_created_at]) : nil
6
+
7
+ filtered_query.display_all(
8
+ after_created_at: after_created_at,
9
+ after_id: params[:after_id]
10
+ ).limit(params.fetch(:limit, DEFAULT_LIMIT))
11
+ end
12
+
13
+ def filtered_query(_filtered_params = params)
14
+ base_query
15
+ end
16
+
17
+ def default_base_query
18
+ GoodJob::BatchRecord.all.includes(:jobs)
19
+ end
20
+ end
21
+ end
@@ -0,0 +1,141 @@
1
+ # frozen_string_literal: true
2
+
3
+ module GoodJob
4
+ # NOTE: This class delegates to {GoodJob::BatchRecord} and is intended to be the public interface for Batches.
5
+ class Batch
6
+ include GlobalID::Identification
7
+
8
+ thread_cattr_accessor :current_batch_id
9
+ thread_cattr_accessor :current_batch_callback_id
10
+
11
+ PROTECTED_PROPERTIES = %i[
12
+ on_finish
13
+ on_success
14
+ on_discard
15
+ callback_queue_name
16
+ callback_priority
17
+ description
18
+ properties
19
+ ].freeze
20
+
21
+ delegate(
22
+ :id,
23
+ :created_at,
24
+ :updated_at,
25
+ :persisted?,
26
+ :enqueued_at,
27
+ :finished_at,
28
+ :discarded_at,
29
+ :enqueued?,
30
+ :finished?,
31
+ :succeeded?,
32
+ :discarded?,
33
+ :description,
34
+ :description=,
35
+ :on_finish,
36
+ :on_finish=,
37
+ :on_success,
38
+ :on_success=,
39
+ :on_discard,
40
+ :on_discard=,
41
+ :callback_queue_name,
42
+ :callback_queue_name=,
43
+ :callback_priority,
44
+ :callback_priority=,
45
+ :properties,
46
+ :properties=,
47
+ :save,
48
+ :reload,
49
+ to: :record
50
+ )
51
+
52
+ # Create a new batch and enqueue it
53
+ # @param on_finish [String, Object] The class name of the callback job to be enqueued after the batch is finished
54
+ # @param properties [Hash] Additional properties to be stored on the batch
55
+ # @param block [Proc] Enqueue jobs within the block to add them to the batch
56
+ # @return [GoodJob::BatchRecord]
57
+ def self.enqueue(active_jobs = [], **properties, &block)
58
+ new.tap do |batch|
59
+ batch.enqueue(active_jobs, **properties, &block)
60
+ end
61
+ end
62
+
63
+ def self.find(id)
64
+ new _record: BatchRecord.find(id)
65
+ end
66
+
67
+ # Helper method to enqueue jobs and assign them to a batch
68
+ def self.within_thread(batch_id: nil, batch_callback_id: nil)
69
+ original_batch_id = current_batch_id
70
+ original_batch_callback_id = current_batch_callback_id
71
+
72
+ self.current_batch_id = batch_id
73
+ self.current_batch_callback_id = batch_callback_id
74
+
75
+ yield
76
+ ensure
77
+ self.current_batch_id = original_batch_id
78
+ self.current_batch_callback_id = original_batch_callback_id
79
+ end
80
+
81
+ def initialize(_record: nil, **properties) # rubocop:disable Lint/UnderscorePrefixedVariableName
82
+ self.record = _record || BatchRecord.new
83
+ assign_properties(properties)
84
+ end
85
+
86
+ # @return [Array<ActiveJob::Base>] Active jobs added to the batch
87
+ def enqueue(active_jobs = [], **properties, &block)
88
+ assign_properties(properties)
89
+ record.save!
90
+
91
+ active_jobs = add(active_jobs, &block)
92
+
93
+ record.finished_at = nil
94
+ record.enqueued_at = Time.current if enqueued_at.nil?
95
+ record.save!
96
+
97
+ record._continue_discard_or_finish
98
+ active_jobs
99
+ end
100
+
101
+ # Enqueue jobs and add them to the batch
102
+ # @param block [Proc] Enqueue jobs within the block to add them to the batch
103
+ # @return [Array<ActiveJob::Base>] Active jobs added to the batch
104
+ def add(active_jobs = nil, &block)
105
+ record.save if record.new_record?
106
+
107
+ buffer = Bulk::Buffer.new
108
+ buffer.add(active_jobs)
109
+ buffer.capture(&block) if block
110
+
111
+ self.class.within_thread(batch_id: id) do
112
+ buffer.enqueue
113
+ end
114
+
115
+ buffer.active_jobs
116
+ end
117
+
118
+ def active_jobs
119
+ record.jobs.map(&:head_execution).map(&:active_job)
120
+ end
121
+
122
+ def callback_active_jobs
123
+ record.callback_jobs.map(&:head_execution).map(&:active_job)
124
+ end
125
+
126
+ def assign_properties(properties)
127
+ properties = properties.dup
128
+ batch_attrs = PROTECTED_PROPERTIES.index_with { |key| properties.delete(key) }.compact
129
+ record.assign_attributes(batch_attrs)
130
+ self.properties.merge!(properties)
131
+ end
132
+
133
+ def _record
134
+ record
135
+ end
136
+
137
+ private
138
+
139
+ attr_accessor :record
140
+ end
141
+ end
@@ -0,0 +1,84 @@
1
+ # frozen_string_literal: true
2
+
3
+ module GoodJob
4
+ class BatchRecord < BaseRecord
5
+ include Lockable
6
+
7
+ self.table_name = 'good_job_batches'
8
+
9
+ has_many :jobs, class_name: 'GoodJob::Job', inverse_of: :batch, foreign_key: :batch_id, dependent: nil
10
+ has_many :executions, class_name: 'GoodJob::Execution', foreign_key: :batch_id, inverse_of: :batch, dependent: nil
11
+ has_many :callback_jobs, class_name: 'GoodJob::Job', foreign_key: :batch_callback_id, dependent: nil # rubocop:disable Rails/InverseOf
12
+
13
+ scope :finished, -> { where.not(finished_at: nil) }
14
+ scope :discarded, -> { where.not(discarded_at: nil) }
15
+ scope :not_discarded, -> { where(discarded_at: nil) }
16
+ scope :succeeded, -> { finished.not_discarded }
17
+
18
+ alias_attribute :enqueued?, :enqueued_at
19
+ alias_attribute :discarded?, :discarded_at
20
+ alias_attribute :finished?, :finished_at
21
+
22
+ scope :display_all, (lambda do |after_created_at: nil, after_id: nil|
23
+ query = order(created_at: :desc, id: :desc)
24
+ if after_created_at.present? && after_id.present?
25
+ query = query.where(Arel.sql('(created_at, id) < (:after_created_at, :after_id)'), after_created_at: after_created_at, after_id: after_id)
26
+ elsif after_created_at.present?
27
+ query = query.where(Arel.sql('(after_created_at) < (:after_created_at)'), after_created_at: after_created_at)
28
+ end
29
+ query
30
+ end)
31
+
32
+ # Whether the batch has finished and no jobs were discarded
33
+ # @return [Boolean]
34
+ def succeeded?
35
+ !discarded? && finished?
36
+ end
37
+
38
+ def to_batch
39
+ Batch.new(_record: self)
40
+ end
41
+
42
+ def display_attributes
43
+ attributes.except('serialized_properties').merge(properties: properties)
44
+ end
45
+
46
+ def _continue_discard_or_finish(execution = nil)
47
+ execution_discarded = execution && execution.error.present? && execution.retried_good_job_id.nil?
48
+ with_advisory_lock(function: "pg_advisory_lock") do
49
+ Batch.within_thread(batch_id: nil, batch_callback_id: id) do
50
+ if execution_discarded && discarded_at.blank?
51
+ update(discarded_at: Time.current)
52
+ on_discard.constantize.set(priority: callback_priority, queue: callback_queue_name).perform_later(to_batch, { event: :discard }) if on_discard.present?
53
+ end
54
+
55
+ if !finished_at && enqueued_at && jobs.where(finished_at: nil).count.zero?
56
+ update(finished_at: Time.current)
57
+ on_success.constantize.set(priority: callback_priority, queue: callback_queue_name).perform_later(to_batch, { event: :success }) if !discarded_at && on_success.present?
58
+ on_finish.constantize.set(priority: callback_priority, queue: callback_queue_name).perform_later(to_batch, { event: :finish }) if on_finish.present?
59
+ end
60
+ end
61
+ end
62
+ end
63
+
64
+ class PropertySerializer
65
+ def self.dump(value)
66
+ ActiveJob::Arguments.serialize([value]).first
67
+ end
68
+
69
+ def self.load(value)
70
+ ActiveJob::Arguments.deserialize([value]).first
71
+ end
72
+ end
73
+
74
+ attribute :serialized_properties, :json, default: -> { {} } # Can be set as default value in `serialize` as of Rails v6.1
75
+ serialize :serialized_properties, PropertySerializer
76
+ alias_attribute :properties, :serialized_properties
77
+
78
+ def properties=(value)
79
+ raise ArgumentError, "Properties must be a Hash" unless value.is_a?(Hash)
80
+
81
+ self.serialized_properties = value
82
+ end
83
+ end
84
+ end
@@ -20,8 +20,12 @@ module GoodJob
20
20
  self.table_name = 'good_jobs'
21
21
  self.advisory_lockable_column = 'active_job_id'
22
22
 
23
+ define_model_callbacks :perform
23
24
  define_model_callbacks :perform_unlocked, only: :after
24
25
 
26
+ set_callback :perform, :around, :reset_batch_values
27
+ set_callback :perform_unlocked, :after, :continue_discard_or_finish_batch
28
+
25
29
  # Parse a string representing a group of queues into a more readable data
26
30
  # structure.
27
31
  # @param string [String] Queue string
@@ -65,6 +69,9 @@ module GoodJob
65
69
  end
66
70
  end
67
71
 
72
+ belongs_to :batch, class_name: 'GoodJob::BatchRecord', optional: true, inverse_of: :executions
73
+ belongs_to :batch_callback, class_name: 'GoodJob::Batch', optional: true
74
+
68
75
  belongs_to :job, class_name: 'GoodJob::Job', foreign_key: 'active_job_id', primary_key: 'active_job_id', optional: true, inverse_of: :executions
69
76
  after_destroy -> { self.class.active_job_id(active_job_id).delete_all }, if: -> { @_destroy_job }
70
77
 
@@ -206,14 +213,24 @@ module GoodJob
206
213
  serialized_params: active_job.serialize,
207
214
  scheduled_at: active_job.scheduled_at,
208
215
  }
209
-
210
216
  execution_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
211
217
 
212
- if CurrentThread.cron_key
218
+ reenqueued_current_execution = CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
219
+ current_execution = CurrentThread.execution
220
+
221
+ if reenqueued_current_execution
222
+ if GoodJob::BatchRecord.migrated?
223
+ execution_args[:batch_id] = current_execution.batch_id
224
+ execution_args[:batch_callback_id] = current_execution.batch_callback_id
225
+ end
226
+ execution_args[:cron_key] = current_execution.cron_key
227
+ else
228
+ if GoodJob::BatchRecord.migrated?
229
+ execution_args[:batch_id] = GoodJob::Batch.current_batch_id
230
+ execution_args[:batch_callback_id] = GoodJob::Batch.current_batch_callback_id
231
+ end
213
232
  execution_args[:cron_key] = CurrentThread.cron_key
214
233
  execution_args[:cron_at] = CurrentThread.cron_at
215
- elsif CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
216
- execution_args[:cron_key] = CurrentThread.execution.cron_key
217
234
  end
218
235
 
219
236
  new(**execution_args.merge(overrides))
@@ -283,7 +300,6 @@ module GoodJob
283
300
 
284
301
  execution.save!
285
302
  active_job.provider_job_id = execution.id
286
-
287
303
  CurrentThread.execution.retried_good_job_id = execution.id if CurrentThread.active_job_id && CurrentThread.active_job_id == active_job.job_id
288
304
 
289
305
  execution
@@ -296,27 +312,29 @@ module GoodJob
296
312
  # exception raised by the job, if any. If the job completed successfully,
297
313
  # the second array entry (the exception) will be +nil+ and vice versa.
298
314
  def perform
299
- raise PreviouslyPerformedError, 'Cannot perform a job that has already been performed' if finished_at
315
+ run_callbacks(:perform) do
316
+ raise PreviouslyPerformedError, 'Cannot perform a job that has already been performed' if finished_at
300
317
 
301
- self.performed_at = Time.current
302
- save! if GoodJob.preserve_job_records
318
+ self.performed_at = Time.current
319
+ save! if GoodJob.preserve_job_records
303
320
 
304
- result = execute
321
+ result = execute
305
322
 
306
- job_error = result.handled_error || result.unhandled_error
307
- self.error = [job_error.class, ERROR_MESSAGE_SEPARATOR, job_error.message].join if job_error
323
+ job_error = result.handled_error || result.unhandled_error
324
+ self.error = [job_error.class, ERROR_MESSAGE_SEPARATOR, job_error.message].join if job_error
308
325
 
309
- reenqueued = result.retried? || retried_good_job_id.present?
310
- if result.unhandled_error && GoodJob.retry_on_unhandled_error
311
- save!
312
- elsif GoodJob.preserve_job_records == true || reenqueued || (result.unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error) || cron_key.present?
313
- self.finished_at = Time.current
314
- save!
315
- else
316
- destroy_job
317
- end
326
+ reenqueued = result.retried? || retried_good_job_id.present?
327
+ if result.unhandled_error && GoodJob.retry_on_unhandled_error
328
+ save!
329
+ elsif GoodJob.preserve_job_records == true || reenqueued || (result.unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error) || cron_key.present?
330
+ self.finished_at = Time.current
331
+ save!
332
+ else
333
+ destroy_job
334
+ end
318
335
 
319
- result
336
+ result
337
+ end
320
338
  end
321
339
 
322
340
  # Tests whether this job is safe to be executed by this thread.
@@ -411,5 +429,13 @@ module GoodJob
411
429
  end
412
430
  end
413
431
  end
432
+
433
+ def reset_batch_values(&block)
434
+ GoodJob::Batch.within_thread(batch_id: nil, batch_callback_id: nil, &block)
435
+ end
436
+
437
+ def continue_discard_or_finish_batch
438
+ batch._continue_discard_or_finish(self) if GoodJob::BatchRecord.migrated? && batch.present?
439
+ end
414
440
  end
415
441
  end
@@ -28,6 +28,7 @@ module GoodJob
28
28
  self.primary_key = 'active_job_id'
29
29
  self.advisory_lockable_column = 'active_job_id'
30
30
 
31
+ belongs_to :batch, class_name: 'GoodJob::BatchRecord', inverse_of: :jobs, optional: true
31
32
  has_many :executions, -> { order(created_at: :asc) }, class_name: 'GoodJob::Execution', foreign_key: 'active_job_id', inverse_of: :job # rubocop:disable Rails/HasManyOrHasOneDependent
32
33
 
33
34
  # Only the most-recent unretried execution represents a "Job"
@@ -0,0 +1,108 @@
1
+ <div class="my-3 card" data-gj-poll-replace id="jobs-table">
2
+ <div class="list-group list-group-flush text-nowrap table-jobs" role="table">
3
+ <header class="list-group-item bg-light">
4
+ <div class="row small text-muted text-uppercase align-items-center">
5
+ <div class="col-4">Jobs</div>
6
+ <div class="col-1">Queue</div>
7
+ <div class="col-1">Priority</div>
8
+ <div class="col-1 text-end">Attempts</div>
9
+ <div class="col text-end">
10
+ <%= tag.button type: "button", class: "btn btn-sm text-muted", role: "button",
11
+ data: { bs_toggle: "collapse", bs_target: ".job-params" },
12
+ aria: { expanded: false, controls: jobs.map { |job| "##{dom_id(job, "params")}" }.join(" ") } do %>
13
+ <%= render_icon "info" %>
14
+ <span class="visually-hidden">Inspect</span>
15
+ <% end %>
16
+ </div>
17
+ </div>
18
+ </header>
19
+
20
+ <% if jobs.present? %>
21
+ <% jobs.each do |job| %>
22
+ <div role="row" class="list-group-item list-group-item-action py-3">
23
+ <div class="row align-items-center">
24
+ <div class="col-4">
25
+ <%= tag.code link_to(job.id, job_path(job), class: "small text-muted text-decoration-none") %>
26
+ <%= tag.h5 tag.code(link_to(job.job_class, job_path(job), class: "text-reset text-decoration-none")), class: "text-reset mb-0" %>
27
+ </div>
28
+ <div class="col-1">
29
+ <span class="badge bg-primary bg-opacity-25 text-dark font-monospace"><%= job.queue_name %></span>
30
+ </div>
31
+ <div class="col-1 small text-center">
32
+ <span class="font-monospace fw-bold"><%= job.priority %></span>
33
+ </div>
34
+ <div class="col-1 text-center">
35
+ <% if job.executions_count > 0 && job.status != :finished %>
36
+ <%= tag.span job.executions_count, class: "badge rounded-pill bg-danger", data: {
37
+ bs_toggle: "popover",
38
+ bs_trigger: "hover focus click",
39
+ bs_placement: "bottom",
40
+ bs_content: job.recent_error
41
+ } %>
42
+ <% else %>
43
+ <span class="badge bg-secondary bg-opacity-50 rounded-pill"><%= job.executions_count %></span>
44
+ <% end %>
45
+ </div>
46
+ <div class="col d-flex gap-3 align-items-center justify-content-end">
47
+ <%= tag.span relative_time(job.last_status_at), class: "small" %>
48
+ <%= status_badge job.status %>
49
+ </div>
50
+ <div class="col-auto">
51
+ <div class="dropdown float-end">
52
+ <button class="d-flex align-items-center btn btn-sm" type="button" id="<%= dom_id(job, :actions) %>" data-bs-toggle="dropdown" aria-expanded="false">
53
+ <%= render "good_job/shared/icons/dots" %>
54
+ <span class="visually-hidden">Actions</span>
55
+ </button>
56
+ <ul class="dropdown-menu shadow" aria-labelledby="<%= dom_id(job, :actions) %>">
57
+ <li>
58
+ <% job_reschedulable = job.status.in? [:scheduled, :retried, :queued] %>
59
+ <%= link_to reschedule_job_path(job.id), method: :put, class: "dropdown-item #{'disabled' unless job_reschedulable}", title: "Reschedule job", data: { confirm: "Confirm reschedule", disable: true } do %>
60
+ <%= render "good_job/shared/icons/skip_forward" %>
61
+ Reschedule
62
+ <% end %>
63
+ </li>
64
+ <li>
65
+ <% job_discardable = job.status.in? [:scheduled, :retried, :queued] %>
66
+ <%= link_to discard_job_path(job.id), method: :put, class: "dropdown-item #{'disabled' unless job_discardable}", title: "Discard job", data: { confirm: "Confirm discard", disable: true } do %>
67
+ <%= render "good_job/shared/icons/stop" %>
68
+ Discard
69
+ <% end %>
70
+ </li>
71
+ <li>
72
+ <%= link_to retry_job_path(job.id), method: :put, class: "dropdown-item #{'disabled' unless job.status == :discarded}", title: "Retry job", data: { confirm: "Confirm retry", disable: true } do %>
73
+ <%= render "good_job/shared/icons/arrow_clockwise" %>
74
+ Retry
75
+ <% end %>
76
+ </li>
77
+ <li>
78
+ <%= link_to job_path(job.id), method: :delete, class: "dropdown-item #{'disabled' unless job.status.in? [:discarded, :finished]}", title: "Destroy job", data: { confirm: "Confirm destroy", disable: true } do %>
79
+ <%= render_icon "trash" %>
80
+ Destroy
81
+ <% end %>
82
+ </li>
83
+
84
+ <li>
85
+ <%= link_to "##{dom_id(job, 'params')}",
86
+ class: "dropdown-item",
87
+ data: { bs_toggle: "collapse" },
88
+ aria: { expanded: false, controls: dom_id(job, "params") } do %>
89
+ <%= render_icon "info" %>
90
+ Inspect
91
+ <% end %>
92
+ </li>
93
+ </ul>
94
+ </div>
95
+ </div>
96
+ </div>
97
+ </div>
98
+ <%= tag.div id: dom_id(job, "params"), class: "job-params list-group-item collapse small bg-dark text-light" do %>
99
+ <%= tag.pre JSON.pretty_generate(job.display_serialized_params) %>
100
+ <% end %>
101
+ <% end %>
102
+ <% else %>
103
+ <div class="list-group-item py-4 text-center text-muted">
104
+ No jobs found.
105
+ </div>
106
+ <% end %>
107
+ </>
108
+ </div>
@@ -0,0 +1,61 @@
1
+ <div class="my-3 card" data-gj-poll-replace id="batches-table">
2
+ <div class="list-group list-group-flush text-nowrap table-batches" role="table">
3
+ <header class="list-group-item bg-light">
4
+ <div class="row small text-muted text-uppercase align-items-center">
5
+ <div class="col-4">Name</div>
6
+ <div class="col-1">Created</div>
7
+ <div class="col-1">Enqueued</div>
8
+ <div class="col-1">Discarded</div>
9
+ <div class="col-1">Finished</div>
10
+ <div class="col">Jobs</div>
11
+ <div class="col text-end">
12
+ <%= tag.button type: "button", class: "btn btn-sm text-muted", role: "button",
13
+ data: { bs_toggle: "collapse", bs_target: ".batch-properties" },
14
+ aria: { expanded: false, controls: batches.map { |batch| "##{dom_id(batch, "params")}" }.join(" ") } do %>
15
+ <%= render_icon "info" %>
16
+ <span class="visually-hidden">Inspect</span>
17
+ <% end %>
18
+ </div>
19
+ </div>
20
+ </header>
21
+
22
+ <% if batches.present? %>
23
+ <% batches.each do |batch| %>
24
+ <div id="<%= dom_id(batch) %>" class="list-group-item py-3" role="row">
25
+ <div class="row align-items-center">
26
+ <div class="col-4">
27
+ <%= link_to batch_path(batch), class: "text-decoration-none" do %>
28
+ <code class="small text-muted">
29
+ <%= batch.id %>
30
+ </code>
31
+ <h5 class=""><code><%= batch.on_finish %></code></h5>
32
+ <div class="text-muted"><%= batch.description %></div>
33
+ <% end %>
34
+ </div>
35
+ <div class="col-1 text-wrap"><%= relative_time(batch.created_at) %></div>
36
+ <div class="col-1 text-wrap"><%= relative_time(batch.enqueued_at) if batch.enqueued_at %></div>
37
+ <div class="col-1 text-wrap"><%= relative_time(batch.discarded_at) if batch.discarded_at %></div>
38
+ <div class="col-1 text-wrap"><%= relative_time(batch.finished_at) if batch.finished_at %></div>
39
+ <div class="col"><%= batch.jobs.count %></div>
40
+ <div class="col text-end">
41
+ <%= tag.button type: "button", class: "btn btn-sm text-muted ms-auto", role: "button",
42
+ title: "Inspect",
43
+ data: { bs_toggle: "collapse", bs_target: "##{dom_id(batch, 'properties')}" },
44
+ aria: { expanded: false, controls: dom_id(batch, "state") } do %>
45
+ <%= render_icon "info" %>
46
+ <span class="visually-hidden">Inspect</span>
47
+ <% end %>
48
+ </div>
49
+ </div>
50
+ </div>
51
+ <%= tag.div id: dom_id(batch, "properties"), class: "batch-properties list-group-item collapse small bg-dark text-light" do %>
52
+ <%= tag.pre JSON.pretty_generate(batch.properties) %>
53
+ <% end %>
54
+ <% end %>
55
+ <% else %>
56
+ <div class="list-group-item py-4 text-center text-muted">
57
+ No batches found.
58
+ </div>
59
+ <% end %>
60
+ </div>
61
+ </div>
@@ -0,0 +1,16 @@
1
+ <% if GoodJob::BatchRecord.migrated? %>
2
+ <%= render 'good_job/batches/table', batches: @filter.records, filter: @filter %>
3
+ <% if @filter.records.present? %>
4
+ <nav aria-label="Batch pagination" class="mt-3">
5
+ <ul class="pagination">
6
+ <li class="page-item">
7
+ <%= link_to(@filter.to_params(after_created_at: @filter.last.created_at, after_id: @filter.last.id), class: "page-link") do %>
8
+ Older batches <span aria-hidden="true">&raquo;</span>
9
+ <% end %>
10
+ </li>
11
+ </ul>
12
+ </nav>
13
+ <% end %>
14
+ <% else %>
15
+ <h3 class="text-center my-5">GoodJob has pending database migrations.</h3>
16
+ <% end %>
@@ -0,0 +1,30 @@
1
+ <div class="break-out bg-light border-bottom py-2 mb-3">
2
+ <div class="container-fluid pt-2">
3
+ <div class="row align-items-center">
4
+ <div class="col-5">
5
+ <nav aria-label="breadcrumb">
6
+ <ol class="breadcrumb small mb-0">
7
+ <li class="breadcrumb-item"><%= link_to "Batches", batches_path %></li>
8
+ <li class="breadcrumb-item active" aria-current="page"><%= tag.code @batch.id, class: "text-muted" %></li>
9
+ </ol>
10
+ <h2 class="h5 mt-2"><%= @batch.description %></h2>
11
+ </nav>
12
+ </div>
13
+ </div>
14
+ </div>
15
+ </div>
16
+
17
+ <div class="my-4">
18
+ <h5>Attributes</h5>
19
+ <%= tag.pre JSON.pretty_generate @batch.display_attributes, class: 'text-wrap text-break' %>
20
+ </div>
21
+
22
+ <div class="my-4">
23
+ <h5>Callback Jobs</h5>
24
+ <%= render 'jobs', jobs: @batch.callback_jobs.reverse %>
25
+ </div>
26
+
27
+ <div class="my-4">
28
+ <h5>Batched Jobs</h5>
29
+ <%= render 'jobs', jobs: @batch.jobs.reverse %>
30
+ </div>
@@ -14,6 +14,13 @@
14
14
  <span class="badge bg-secondary rounded-pill"><%= number_to_human(jobs_count) %></span>
15
15
  <% end %>
16
16
  </li>
17
+ <li class="nav-item">
18
+ <%= link_to batches_path, class: ["nav-link", ("active" if controller_name == 'batches')] do %>
19
+ <%= "Batches" %>
20
+ <% batches_count = GoodJob::BatchRecord.migrated? ? GoodJob::BatchRecord.all.size : 0 %>
21
+ <span class="badge bg-secondary rounded-pill"><%= batches_count %></span>
22
+ <% end %>
23
+ </li>
17
24
  <li class="nav-item">
18
25
  <%= link_to cron_entries_path, class: ["nav-link", ("active" if controller_name == 'cron_entries')] do %>
19
26
  <%= t(".cron_schedules") %>
data/config/routes.rb CHANGED
@@ -15,6 +15,8 @@ GoodJob::Engine.routes.draw do
15
15
  end
16
16
  end
17
17
 
18
+ resources :batches, only: %i[index show]
19
+
18
20
  resources :cron_entries, only: %i[index show], param: :cron_key do
19
21
  member do
20
22
  post :enqueue
@@ -19,6 +19,23 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
19
19
  t.text :cron_key
20
20
  t.uuid :retried_good_job_id
21
21
  t.datetime :cron_at
22
+
23
+ t.uuid :batch_id
24
+ t.uuid :batch_callback_id
25
+ end
26
+
27
+ create_table :good_job_batches, id: :uuid do |t|
28
+ t.timestamps
29
+ t.text :description
30
+ t.jsonb :serialized_properties
31
+ t.text :on_finish
32
+ t.text :on_success
33
+ t.text :on_discard
34
+ t.text :callback_queue_name
35
+ t.integer :callback_priority
36
+ t.datetime :enqueued_at
37
+ t.datetime :discarded_at
38
+ t.datetime :finished_at
22
39
  end
23
40
 
24
41
  create_table :good_job_processes, id: :uuid do |t|
@@ -43,5 +60,7 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
43
60
  add_index :good_jobs, [:finished_at], where: "retried_good_job_id IS NULL AND finished_at IS NOT NULL", name: :index_good_jobs_jobs_on_finished_at
44
61
  add_index :good_jobs, [:priority, :created_at], order: { priority: "DESC NULLS LAST", created_at: :asc },
45
62
  where: "finished_at IS NULL", name: :index_good_jobs_jobs_on_priority_created_at_when_unfinished
63
+ add_index :good_jobs, [:batch_id], where: "batch_id IS NOT NULL"
64
+ add_index :good_jobs, [:batch_callback_id], where: "batch_callback_id IS NOT NULL"
46
65
  end
47
66
  end
@@ -0,0 +1,34 @@
1
+ # frozen_string_literal: true
2
+ class CreateGoodJobBatches < ActiveRecord::Migration<%= migration_version %>
3
+ def change
4
+ reversible do |dir|
5
+ dir.up do
6
+ # Ensure this incremental update migration is idempotent
7
+ # with monolithic install migration.
8
+ return if connection.table_exists?(:good_job_batches)
9
+ end
10
+ end
11
+
12
+ create_table :good_job_batches, id: :uuid do |t|
13
+ t.timestamps
14
+ t.text :description
15
+ t.jsonb :serialized_properties
16
+ t.text :on_finish
17
+ t.text :on_success
18
+ t.text :on_discard
19
+ t.text :callback_queue_name
20
+ t.integer :callback_priority
21
+ t.datetime :enqueued_at
22
+ t.datetime :discarded_at
23
+ t.datetime :finished_at
24
+ end
25
+
26
+ change_table :good_jobs do |t|
27
+ t.uuid :batch_id
28
+ t.uuid :batch_callback_id
29
+
30
+ t.index :batch_id, where: "batch_id IS NOT NULL"
31
+ t.index :batch_callback_id, where: "batch_callback_id IS NOT NULL"
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+ module GoodJob
3
+ module ActiveJobExtensions
4
+ module Batches
5
+ extend ActiveSupport::Concern
6
+
7
+ def batch
8
+ @_batch ||= CurrentThread.execution&.batch&.to_batch
9
+ end
10
+ alias batch? batch
11
+ end
12
+ end
13
+ end
data/lib/good_job/bulk.rb CHANGED
@@ -15,19 +15,14 @@ module GoodJob
15
15
  # @param active_jobs [Array<ActiveJob::Base>] Active Jobs to be buffered.
16
16
  # @param queue_adapter Override the jobs implict queue adapter with an explicit one.
17
17
  # @return [nil, Array<ActiveJob::Base>] The ActiveJob instances that have been buffered; nil if no active buffer
18
- def self.capture(active_jobs = nil, queue_adapter: nil)
19
- raise(ArgumentError, "Use either the block form or the argument form, not both") if block_given? && active_jobs
20
-
21
- if block_given?
22
- begin
23
- original_buffer = current_buffer
24
- self.current_buffer = Buffer.new
25
- yield
26
- current_buffer.active_jobs
27
- ensure
28
- self.current_buffer = original_buffer
29
- end
30
- else
18
+ def self.capture(active_jobs = nil, queue_adapter: nil, &block)
19
+ raise(ArgumentError, "Use either the block form or the argument form, not both") if block && active_jobs
20
+
21
+ if block
22
+ buffer = Buffer.new
23
+ buffer.capture(&block)
24
+ buffer.active_jobs
25
+ elsif current_buffer
31
26
  current_buffer&.add(active_jobs, queue_adapter: queue_adapter)
32
27
  end
33
28
  end
@@ -35,16 +30,15 @@ module GoodJob
35
30
  # Capture jobs to a buffer and enqueue them all at once; or enqueue the current buffer.
36
31
  # @param active_jobs [Array<ActiveJob::Base>] Active Jobs to be enqueued.
37
32
  # @return [Array<ActiveJob::Base>] The ActiveJob instances that have been captured; check provider_job_id to confirm enqueued.
38
- def self.enqueue(active_jobs = nil)
39
- raise(ArgumentError, "Use either the block form or the argument form, not both") if block_given? && active_jobs
33
+ def self.enqueue(active_jobs = nil, &block)
34
+ raise(ArgumentError, "Use either the block form or the argument form, not both") if block && active_jobs
40
35
 
41
- if block_given?
42
- capture do
43
- yield
44
- current_buffer&.enqueue
45
- end
36
+ buffer = Buffer.new
37
+ if block
38
+ buffer.capture(&block)
39
+ buffer.enqueue
40
+ buffer.active_jobs
46
41
  elsif active_jobs.present?
47
- buffer = Buffer.new
48
42
  buffer.add(active_jobs)
49
43
  buffer.enqueue
50
44
  buffer.active_jobs
@@ -69,8 +63,18 @@ module GoodJob
69
63
  @values = []
70
64
  end
71
65
 
66
+ def capture
67
+ original_buffer = Bulk.current_buffer
68
+ Bulk.current_buffer = self
69
+ yield
70
+ ensure
71
+ Bulk.current_buffer = original_buffer
72
+ end
73
+
72
74
  def add(active_jobs, queue_adapter: nil)
73
75
  new_pairs = Array(active_jobs).map do |active_job|
76
+ raise ArgumentError, "Expected an ActiveJob::Base instance, got #{active_job.class}" unless active_job.is_a?(ActiveJob::Base)
77
+
74
78
  adapter = queue_adapter || active_job.class.queue_adapter
75
79
  raise Error, "Jobs must have a Queue Adapter" unless adapter
76
80
 
@@ -156,7 +156,7 @@ module GoodJob
156
156
  destroyed_records_count = event.payload[:destroyed_records_count]
157
157
 
158
158
  info do
159
- "GoodJob destroyed #{destroyed_records_count} preserved #{'job'.pluralize(destroyed_records_count)} finished before #{timestamp}."
159
+ "GoodJob destroyed #{destroyed_records_count} preserved job execution #{'records'.pluralize(destroyed_records_count)} finished before #{timestamp}."
160
160
  end
161
161
  end
162
162
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
  module GoodJob
3
3
  # GoodJob gem version.
4
- VERSION = '3.9.0'
4
+ VERSION = '3.10.0'
5
5
  end
data/lib/good_job.rb CHANGED
@@ -8,6 +8,7 @@ require "good_job/engine"
8
8
  require "good_job/adapter"
9
9
  require "active_job/queue_adapters/good_job_adapter"
10
10
  require "good_job/active_job_extensions/concurrency"
11
+ require "good_job/active_job_extensions/batches"
11
12
 
12
13
  require "good_job/assignable_connection"
13
14
  require "good_job/bulk"
@@ -143,7 +144,7 @@ module GoodJob
143
144
  end
144
145
  end
145
146
 
146
- # Destroys preserved job records.
147
+ # Destroys preserved job and batch records.
147
148
  # By default, GoodJob destroys job records when the job is performed and this
148
149
  # method is not necessary. However, when `GoodJob.preserve_job_records = true`,
149
150
  # the jobs will be preserved in the database. This is useful when wanting to
@@ -151,7 +152,7 @@ module GoodJob
151
152
  # If you are preserving job records this way, use this method regularly to
152
153
  # destroy old records and preserve space in your database.
153
154
  # @params older_than [nil,Numeric,ActiveSupport::Duration] Jobs older than this will be destroyed (default: +86400+).
154
- # @return [Integer] Number of jobs that were destroyed.
155
+ # @return [Integer] Number of job execution records and batches that were destroyed.
155
156
  def self.cleanup_preserved_jobs(older_than: nil)
156
157
  older_than ||= GoodJob.configuration.cleanup_preserved_jobs_before_seconds_ago
157
158
  timestamp = Time.current - older_than
@@ -160,10 +161,19 @@ module GoodJob
160
161
  ActiveSupport::Notifications.instrument("cleanup_preserved_jobs.good_job", { older_than: older_than, timestamp: timestamp }) do |payload|
161
162
  old_jobs = GoodJob::Job.where('finished_at <= ?', timestamp)
162
163
  old_jobs = old_jobs.succeeded unless include_discarded
163
- old_jobs_count = old_jobs.count
164
-
165
- GoodJob::Execution.where(job: old_jobs).delete_all
166
- payload[:destroyed_records_count] = old_jobs_count
164
+ deleted_executions_count = GoodJob::Execution.where(job: old_jobs).delete_all
165
+
166
+ if GoodJob::BatchRecord.migrated?
167
+ old_batches = GoodJob::BatchRecord.where('finished_at <= ?', timestamp)
168
+ old_batches = old_batches.succeeded unless include_discarded
169
+ deleted_batches_count = old_batches.delete_all
170
+ else
171
+ deleted_batches_count = 0
172
+ end
173
+
174
+ payload[:destroyed_executions_count] = deleted_executions_count
175
+ payload[:destroyed_batches_count] = deleted_batches_count
176
+ payload[:destroyed_records_count] = deleted_executions_count + deleted_batches_count
167
177
  end
168
178
  end
169
179
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: good_job
3
3
  version: !ruby/object:Gem::Version
4
- version: 3.9.0
4
+ version: 3.10.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ben Sheldon
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2023-01-31 00:00:00.000000000 Z
11
+ date: 2023-02-04 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activejob
@@ -334,16 +334,20 @@ files:
334
334
  - app/charts/good_job/scheduled_by_queue_chart.rb
335
335
  - app/controllers/good_job/application_controller.rb
336
336
  - app/controllers/good_job/assets_controller.rb
337
+ - app/controllers/good_job/batches_controller.rb
337
338
  - app/controllers/good_job/cron_entries_controller.rb
338
339
  - app/controllers/good_job/jobs_controller.rb
339
340
  - app/controllers/good_job/processes_controller.rb
340
341
  - app/filters/good_job/base_filter.rb
342
+ - app/filters/good_job/batches_filter.rb
341
343
  - app/filters/good_job/jobs_filter.rb
342
344
  - app/helpers/good_job/application_helper.rb
343
345
  - app/models/concerns/good_job/filterable.rb
344
346
  - app/models/concerns/good_job/reportable.rb
345
347
  - app/models/good_job/active_job_job.rb
346
348
  - app/models/good_job/base_record.rb
349
+ - app/models/good_job/batch.rb
350
+ - app/models/good_job/batch_record.rb
347
351
  - app/models/good_job/cron_entry.rb
348
352
  - app/models/good_job/execution.rb
349
353
  - app/models/good_job/execution_result.rb
@@ -351,6 +355,10 @@ files:
351
355
  - app/models/good_job/lockable.rb
352
356
  - app/models/good_job/process.rb
353
357
  - app/models/good_job/setting.rb
358
+ - app/views/good_job/batches/_jobs.erb
359
+ - app/views/good_job/batches/_table.erb
360
+ - app/views/good_job/batches/index.html.erb
361
+ - app/views/good_job/batches/show.html.erb
354
362
  - app/views/good_job/cron_entries/index.html.erb
355
363
  - app/views/good_job/cron_entries/show.html.erb
356
364
  - app/views/good_job/jobs/_executions.erb
@@ -390,8 +398,10 @@ files:
390
398
  - lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb.erb
391
399
  - lib/generators/good_job/templates/update/migrations/02_create_good_job_settings.rb.erb
392
400
  - lib/generators/good_job/templates/update/migrations/03_create_index_good_jobs_jobs_on_priority_created_at_when_unfinished.rb.erb
401
+ - lib/generators/good_job/templates/update/migrations/04_create_good_job_batches.rb.erb
393
402
  - lib/generators/good_job/update_generator.rb
394
403
  - lib/good_job.rb
404
+ - lib/good_job/active_job_extensions/batches.rb
395
405
  - lib/good_job/active_job_extensions/concurrency.rb
396
406
  - lib/good_job/adapter.rb
397
407
  - lib/good_job/assignable_connection.rb