good_job 2.4.0 → 2.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9a8150ef0c4bf16e9091908964e629175bf1b3543c0955e796be67c46a842a55
-  data.tar.gz: 707260ea8989fa5d108b533c0b61a39741be036de397980d37a725e1e188d6f2
+  metadata.gz: a0546a16ecd5760e3ca0dd5097ec950767b5993ac613489d5591d5d44eadc021
+  data.tar.gz: 9e21d8930806b1843d4c32abef233f89f48292229386da593ba82b3a94617eae
 SHA512:
-  metadata.gz: 9b6b1292908f04a1e1bc72a541c1e8751856b8deb836046660290d766494f94d54f37b6070b614690c59ee34849c1051a88aa5f0c66f1c5931b765d2d9e0c685
-  data.tar.gz: a3b6ed6bb8032e0ee772186b6846af655fee07e097bc2dc0be82185d8369141f6f14c611223c2e5c5239a2287dea4b2969d90ededb1e817b64f253a0b2b48699
+  metadata.gz: 6e3aa1deaa9973f0d435bc6eb77e4557f509a5e9dbd38e267a4268987561576323eda9bfb8bfa3af5ad7aec72c51b17f88121f7d202b1455c1ea17c8272cb6de
+  data.tar.gz: b812a3371f589db1464f3dabe35badee94bf2a8f132e68e64c438e8ddde4415598d133f2e956d335e021a0ea82b59fbc61c3eb77cab230be760516dbca270402

data/CHANGELOG.md

@@ -1,5 +1,38 @@
 # Changelog
 
+## [v2.4.1](https://github.com/bensheldon/good_job/tree/v2.4.1) (2021-10-11)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.4.0...v2.4.1)
+
+**Implemented enhancements:**
+
+- Support Datadog APM / `dd-trace-rb` [\#323](https://github.com/bensheldon/good_job/issues/323)
+- Display info about used timezone. [\#398](https://github.com/bensheldon/good_job/pull/398) ([morgoth](https://github.com/morgoth))
+- Display cron schedules args in dashboard [\#396](https://github.com/bensheldon/good_job/pull/396) ([aried3r](https://github.com/aried3r))
+
+**Fixed bugs:**
+
+- Inline adapter should raise unhandled exceptions during execution [\#416](https://github.com/bensheldon/good_job/pull/416) ([bensheldon](https://github.com/bensheldon))
+- Enforce english locale in UI [\#407](https://github.com/bensheldon/good_job/pull/407) ([morgoth](https://github.com/morgoth))
+
+**Closed issues:**
+
+- Finished jobs don't show up as finished [\#415](https://github.com/bensheldon/good_job/issues/415)
+- Inline adapter should raise unhandled exceptions during execution [\#410](https://github.com/bensheldon/good_job/issues/410)
+- Rewrite Scheduler "worker" thread name to be `thread` [\#406](https://github.com/bensheldon/good_job/issues/406)
+- "WARNING: you don't own a lock of type ExclusiveLock" in Development [\#388](https://github.com/bensheldon/good_job/issues/388)
+- Improve Readme's "Optimize queues, threads, processes" section [\#132](https://github.com/bensheldon/good_job/issues/132)
+
+**Merged pull requests:**
+
+- Ignore Rails HEAD Appraisal until `rails new` fixed [\#419](https://github.com/bensheldon/good_job/pull/419) ([bensheldon](https://github.com/bensheldon))
+- Warn in Readme that configuration should not go into `config/initializers/*.rb` [\#418](https://github.com/bensheldon/good_job/pull/418) ([bensheldon](https://github.com/bensheldon))
+- Replace worker wording [\#409](https://github.com/bensheldon/good_job/pull/409) ([Hugo-Hache](https://github.com/Hugo-Hache))
+- Improve Readme's "Optimize queues, threads, processes" section [\#405](https://github.com/bensheldon/good_job/pull/405) ([Hugo-Hache](https://github.com/Hugo-Hache))
+- Update GH Test Matrix with more PG versions [\#401](https://github.com/bensheldon/good_job/pull/401) ([tedhexaflow](https://github.com/tedhexaflow))
+- Extract cron configuration hash into CronEntry ActiveModel objects [\#400](https://github.com/bensheldon/good_job/pull/400) ([bensheldon](https://github.com/bensheldon))
+- Remove errant copy-paste from app.json [\#397](https://github.com/bensheldon/good_job/pull/397) ([morgoth](https://github.com/morgoth))
+
 ## [v2.4.0](https://github.com/bensheldon/good_job/tree/v2.4.0) (2021-10-02)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v2.3.1...v2.4.0)

data/README.md

@@ -212,7 +212,11 @@ to delete old records and preserve space in your database.
 
 To use GoodJob, you can set `config.active_job.queue_adapter` to a `:good_job`.
 
-Additional configuration can be provided via `config.good_job.OPTION = ...` for example:
+Additional configuration can be provided via `config.good_job.OPTION = ...`.
+
+_Configuration **must** be placed into `config/application.rb` or `config/environments/{RAILS_ENV}.rb`; configuration may not work correctly if placed into `config/initializers/*.rb` because application initializers run _after_ gem initialization (see [Rails#36650](https://github.com/rails/rails/issues/36650) and [GoodJob#380](https://github.com/bensheldon/good_job/issues/380))._
+
+Configuration examples:
 
 ```ruby
 # config/application.rb
@@ -576,43 +580,62 @@ end
 
 By default, GoodJob creates a single thread execution pool that will execute jobs from any queue. Depending on your application's workload, job types, and service level objectives, you may wish to optimize execution resources. For example, providing dedicated execution resources for transactional emails so they are not delayed by long-running batch jobs. Some options:
 
-- Multiple execution pools within a single process:
+- Multiple isolated execution pools within a single process:
+
+    For moderate workloads, multiple isolated thread execution pools offers a good balance between congestion management and economy.
+
+    A pool is configured with the following syntax `<participating_queues>:<thread_count>`:
+
+    - `<participating_queues>`: either `queue1,queue2` (only those queues), `*` (all) or `-queue1,queue2` (all except those queues).
+    - `<thread_count>`: a count overriding for this specific pool the global `max-threads`.
+
+    Pool configurations are separated with a semicolon (;) in the `queues` configuration
 
     ```bash
-    $ bundle exec good_job --queues="transactional_messages:2;batch_processing:1;-transactional_messages,batch_processing:2;*" --max-threads=5
+    $ bundle exec good_job \
+        --queues="transactional_messages:2;batch_processing:1;-transactional_messages,batch_processing:2;*" \
+        --max-threads=5
     ```
 
-    This configuration will result in a single process with 4 isolated thread execution pools. Isolated execution pools are separated with a semicolon (`;`) and queue names and thread counts with a colon (`:`)
+    This configuration will result in a single process with 4 isolated thread execution pools.
 
-    - `transactional_messages:2`: execute jobs enqueued on `transactional_messages` with up to 2 threads.
-    - `batch_processing:1` execute jobs enqueued on `batch_processing` with a single thread.
-    - `-transactional_messages,batch_processing`: execute jobs enqueued on _any_ queue _excluding_ `transactional_messages` or `batch_processing` with up to 2 threads.
-    - `*`: execute jobs on any queue on up to 5 threads, as configured by `--max-threads=5`
-
-    For moderate workloads, multiple isolated thread execution pools offers a good balance between congestion management and economy.
+    - `transactional_messages:2`: execute jobs enqueued on `transactional_messages`, with up to 2 threads.
+    - `batch_processing:1` execute jobs enqueued on `batch_processing`, with a single thread.
+    - `-transactional_messages,batch_processing`: execute jobs enqueued on _any_ queue _excluding_ `transactional_messages` or `batch_processing`, with up to 2 threads.
+    - `*`: execute jobs on any queue, with up to 5 threads (as configured by `--max-threads=5`).
 
     Configuration can be injected by environment variables too:
 
     ```bash
-    $ GOOD_JOB_QUEUES="transactional_messages:2;batch_processing:1;-transactional_messages,batch_processing:2;*" GOOD_JOB_MAX_THREADS=5 bundle exec good_job
+    $ GOOD_JOB_QUEUES="transactional_messages:2;batch_processing:1;-transactional_messages,batch_processing:2;*" \
+      GOOD_JOB_MAX_THREADS=5 \
+      bundle exec good_job
     ```
 
-- Multiple processes; for example, on Heroku:
+- Multiple processes:
+
+    While multiple isolated thread execution pools offer a way to provide dedicated execution resources, those resources are bound to a single machine. To scale them independently, define several processes.
+
+    For example, this configuration on Heroku allows to customize the dyno count (instances), or type (CPU/RAM), per process type:
 
     ```procfile
     # Procfile
 
-    # Separate dyno types
+    # Separate process types
     worker: bundle exec good_job --max-threads=5
     transactional_worker: bundle exec good_job --queues="transactional_messages" --max-threads=2
     batch_worker: bundle exec good_job --queues="batch_processing" --max-threads=1
+    ```
+
+    To optimize for CPU performance at the expense of greater memory and system resource usage, while keeping a single process type (and thus a single dyno), combine several processes and wait for them:
 
-    # Combined multi-process dyno
+    ```procfile
+    # Procfile
+
+    # Combined multi-process
     combined_worker: bundle exec good_job --max-threads=5 & bundle exec good_job --queues="transactional_messages" --max-threads=2 & bundle exec good_job --queues="batch_processing" --max-threads=1 & wait -n
     ```
 
-    Running multiple processes can optimize for CPU performance at the expense of greater memory and system resource usage.
-
 Keep in mind, queue operations and management is an advanced discipline. This stuff is complex, especially for heavy workloads and unique processing requirements. Good job 👍
 
 ### Database connections

data/engine/app/controllers/good_job/base_controller.rb

@@ -2,5 +2,13 @@
 module GoodJob
   class BaseController < ActionController::Base # rubocop:disable Rails/ApplicationController
     protect_from_forgery with: :exception
+
+    around_action :switch_locale
+
+    private
+
+    def switch_locale(&action)
+      I18n.with_locale(:en, &action)
+    end
   end
 end

data/engine/app/controllers/good_job/cron_schedules_controller.rb

@@ -3,7 +3,7 @@ module GoodJob
   class CronSchedulesController < GoodJob::BaseController
     def index
       configuration = GoodJob::Configuration.new({})
-      @cron_schedules = configuration.cron
+      @cron_entries = configuration.cron_entries
     end
   end
 end

data/engine/app/views/good_job/cron_schedules/index.html.erb

@@ -1,4 +1,4 @@
-<% if @cron_schedules.present? %>
+<% if @cron_entries.present? %>
   <div class="card my-3">
     <div class="table-responsive">
       <table class="table card-table table-bordered table-hover table-sm mb-0">
@@ -9,7 +9,14 @@
             Set&nbsp;
             <%= tag.button "Toggle", type: "button", class: "btn btn-sm btn-outline-primary", role: "button",
                            data: { bs_toggle: "collapse", bs_target: ".job-properties" },
-                           aria: { expanded: false, controls: @cron_schedules.map { |job_key, _| "##{job_key.to_param}" }.join(" ") }
+                           aria: { expanded: false, controls: @cron_entries.map { |cron_entry| dom_id(cron_entry, 'properties') }.join(" ") }
+            %>
+          </th>
+          <th>
+            Args&nbsp;
+            <%= tag.button "Toggle", type: "button", class: "btn btn-sm btn-outline-primary", role: "button",
+                           data: { bs_toggle: "collapse", bs_target: ".job-args" },
+                           aria: { expanded: false, controls: @cron_entries.map { |cron_entry| dom_id(cron_entry, 'args') }.join(" ") }
             %>
           </th>
           <th>Class</th>
@@ -17,28 +24,43 @@
           <th>Next scheduled</th>
         </thead>
         <tbody>
-          <% @cron_schedules.each do |job_key, job| %>
+          <% @cron_entries.each do |cron_entry| %>
             <tr>
-              <td class="font-monospace"><%= job_key %></td>
-              <td class="font-monospace"><%= job[:cron] %></td>
+              <td class="font-monospace"><%= cron_entry.key %></td>
+              <td class="font-monospace"><%= cron_entry.cron %></td>
+              <td>
+                <%=
+                  case cron_entry.set
+                  when NilClass
+                    "None"
+                  when Proc
+                    "Lambda/Callable"
+                  when Hash
+                    tag.button("Preview", type: "button", class: "btn btn-sm btn-outline-primary", role: "button",
+                        data: { bs_toggle: "collapse", bs_target: "##{dom_id(cron_entry, 'properties')}" },
+                        aria: { expanded: false, controls: dom_id(cron_entry, 'properties') }) +
+                      tag.pre(JSON.pretty_generate(cron_entry.set), id: dom_id(cron_entry, 'properties'), class: "collapse job-properties")
+                  end
+                %>
+              </td>
               <td>
                 <%=
-                  case job[:set]
+                  case cron_entry.args
                   when NilClass
                     "None"
                   when Proc
                     "Lambda/Callable"
                   when Hash
                     tag.button("Preview", type: "button", class: "btn btn-sm btn-outline-primary", role: "button",
-                        data: { bs_toggle: "collapse", bs_target: "##{job_key.to_param}" },
-                        aria: { expanded: false, controls: job_key.to_param }) +
-                      tag.pre(JSON.pretty_generate(job[:set]), id: job_key.to_param, class: "collapse job-properties")
+                        data: { bs_toggle: "collapse", bs_target: "##{dom_id(cron_entry, 'args')}" },
+                        aria: { expanded: false, controls: dom_id(cron_entry, 'args') }) +
+                      tag.pre(JSON.pretty_generate(cron_entry.args), id: dom_id(cron_entry, 'args'), class: "collapse job-args")
                   end
                 %>
               </td>
-              <td class="font-monospace"><%= job[:class] %></td>
-              <td><%= job[:description] %></td>
-              <td><%= Fugit.parse_cron(job[:cron]).next_time.to_local_time %></td>
+              <td class="font-monospace"><%= cron_entry.job_class %></td>
+              <td><%= cron_entry.description %></td>
+              <td><%= cron_entry.next_at.to_local_time %></td>
             </tr>
           <% end %>
         </tbody>

data/engine/app/views/layouts/good_job/base.html.erb

@@ -49,6 +49,7 @@
             </li>
           -->
         </ul>
+        <div class="text-muted" title="Now is <%= Time.current %>">Times are displayed in <%= Time.current.zone %> timezone</div>
       </div>
     </div>
   </nav>

data/lib/good_job/adapter.rb

@@ -38,7 +38,7 @@ module GoodJob
         @notifier.recipients << [@scheduler, :create_thread]
         @poller.recipients << [@scheduler, :create_thread]
 
-        @cron_manager = GoodJob::CronManager.new(@configuration.cron, start_on_initialize: Rails.application.initialized?) if @configuration.enable_cron?
+        @cron_manager = GoodJob::CronManager.new(@configuration.cron_entries, start_on_initialize: Rails.application.initialized?) if @configuration.enable_cron?
       end
     end
 
@@ -64,10 +64,12 @@ module GoodJob
 
       if execute_inline?
         begin
-          execution.perform
+          result = execution.perform
         ensure
           execution.advisory_unlock
         end
+
+        raise result.unhandled_error if result.unhandled_error
       else
         job_state = { queue_name: execution.queue_name }
         job_state[:scheduled_at] = execution.scheduled_at if execution.scheduled_at

data/lib/good_job/cli.rb

@@ -91,7 +91,9 @@ module GoodJob
       scheduler = GoodJob::Scheduler.from_configuration(configuration, warm_cache_on_initialize: true)
       notifier.recipients << [scheduler, :create_thread]
       poller.recipients << [scheduler, :create_thread]
-      cron_manager = GoodJob::CronManager.new(configuration.cron, start_on_initialize: true) if configuration.enable_cron?
+
+      cron_manager = GoodJob::CronManager.new(configuration.cron_entries, start_on_initialize: true) if configuration.enable_cron?
+
       @stop_good_job_executable = false
       %w[INT TERM].each do |signal|
         trap(signal) { @stop_good_job_executable = true }

data/lib/good_job/configuration.rb

@@ -165,6 +165,10 @@ module GoodJob
         {}
     end
 
+    def cron_entries
+      cron.map { |cron_key, params| GoodJob::CronEntry.new(params.merge(key: cron_key)) }
+    end
+
     # Number of seconds to preserve jobs when using the +good_job cleanup_preserved_jobs+ CLI command.
     # This configuration is only used when {GoodJob.preserve_job_records} is +true+.
     # @return [Integer]

data/lib/good_job/cron_entry.rb

@@ -0,0 +1,65 @@
+# frozen_string_literal: true
+require "concurrent/hash"
+require "concurrent/scheduled_task"
+require "fugit"
+
+module GoodJob # :nodoc:
+  #
+  # A CronEntry represents a single scheduled item's properties.
+  #
+  class CronEntry
+    include ActiveModel::Model
+
+    attr_reader :params
+
+    def initialize(params = {})
+      @params = params.with_indifferent_access
+    end
+
+    def key
+      params.fetch(:key)
+    end
+    alias id key
+
+    def job_class
+      params.fetch(:class)
+    end
+
+    def cron
+      params.fetch(:cron)
+    end
+
+    def set
+      params[:set]
+    end
+
+    def args
+      params[:args]
+    end
+
+    def description
+      params[:description]
+    end
+
+    def next_at
+      fugit = Fugit::Cron.parse(cron)
+      fugit.next_time
+    end
+
+    def enqueue
+      job_class.constantize.set(set_value).perform_later(*args_value)
+    end
+
+    private
+
+    def set_value
+      value = set || {}
+      value.respond_to?(:call) ? value.call : value
+    end
+
+    def args_value
+      value = args || []
+      value.respond_to?(:call) ? value.call : value
+    end
+  end
+end

data/lib/good_job/cron_manager.rb

@@ -11,7 +11,7 @@ module GoodJob # :nodoc:
     # @!attribute [r] instances
     #   @!scope class
     #   List of all instantiated CronManagers in the current process.
-    #   @return [Array<GoodJob::CronManagers>, nil]
+    #   @return [Array<GoodJob::CronManager>, nil]
     cattr_reader :instances, default: [], instance_reader: false
 
     # Task observer for cron task
@@ -26,13 +26,13 @@ module GoodJob # :nodoc:
 
     # Execution configuration to be scheduled
     # @return [Hash]
-    attr_reader :schedules
+    attr_reader :cron_entries
 
-    # @param schedules [Hash]
+    # @param cron_entries [Array<CronEntry>]
     # @param start_on_initialize [Boolean]
-    def initialize(schedules = {}, start_on_initialize: false)
+    def initialize(cron_entries = [], start_on_initialize: false)
       @running = false
-      @schedules = schedules
+      @cron_entries = cron_entries
       @tasks = Concurrent::Hash.new
 
       self.class.instances << self
@@ -42,9 +42,11 @@ module GoodJob # :nodoc:
 
     # Schedule tasks that will enqueue jobs based on their schedule
     def start
-      ActiveSupport::Notifications.instrument("cron_manager_start.good_job", cron_jobs: @schedules) do
+      ActiveSupport::Notifications.instrument("cron_manager_start.good_job", cron_entries: cron_entries) do
         @running = true
-        schedules.each_key { |cron_key| create_task(cron_key) }
+        cron_entries.each do |cron_entry|
+          create_task(cron_entry)
+        end
       end
     end
 
@@ -78,36 +80,22 @@ module GoodJob # :nodoc:
     end
 
     # Enqueues a scheduled task
-    # @param cron_key [Symbol, String] the key within the schedule to use
-    def create_task(cron_key)
-      schedule = @schedules[cron_key]
-      return false if schedule.blank?
-
-      fugit = Fugit::Cron.parse(schedule.fetch(:cron))
-      delay = [(fugit.next_time - Time.current).to_f, 0].max
-
-      future = Concurrent::ScheduledTask.new(delay, args: [self, cron_key]) do |thr_scheduler, thr_cron_key|
+    # @param cron_entry [CronEntry] the CronEntry object to schedule
+    def create_task(cron_entry)
+      delay = [(cron_entry.next_at - Time.current).to_f, 0].max
+      future = Concurrent::ScheduledTask.new(delay, args: [self, cron_entry]) do |thr_scheduler, thr_cron_entry|
         # Re-schedule the next cron task before executing the current task
-        thr_scheduler.create_task(thr_cron_key)
-
-        CurrentThread.reset
-        CurrentThread.cron_key = thr_cron_key
+        thr_scheduler.create_task(thr_cron_entry)
 
         Rails.application.executor.wrap do
-          schedule = thr_scheduler.schedules.fetch(thr_cron_key).with_indifferent_access
-          job_class = schedule.fetch(:class).constantize
-
-          job_set_value = schedule.fetch(:set, {})
-          job_set = job_set_value.respond_to?(:call) ? job_set_value.call : job_set_value
-
-          job_args_value = schedule.fetch(:args, [])
-          job_args = job_args_value.respond_to?(:call) ? job_args_value.call : job_args_value
+          CurrentThread.reset
+          CurrentThread.cron_key = thr_cron_entry.key
 
-          job_class.set(job_set).perform_later(*job_args)
+          cron_entry.enqueue
         end
       end
 
-      @tasks[cron_key] = future
+      @tasks[cron_entry.key] = future
       future.add_observer(self.class, :task_observer)
       future.execute
     end

data/lib/good_job/log_subscriber.rb

@@ -59,11 +59,11 @@ module GoodJob
 
     # @!macro notification_responder
     def cron_manager_start(event)
-      cron_jobs = event.payload[:cron_jobs]
-      cron_jobs_count = cron_jobs.size
+      cron_entries = event.payload[:cron_entries]
+      cron_jobs_count = cron_entries.size
 
       info do
-        "GoodJob started cron with #{cron_jobs_count} #{'jobs'.pluralize(cron_jobs_count)}."
+        "GoodJob started cron with #{cron_jobs_count} #{'job'.pluralize(cron_jobs_count)}."
       end
     end
 

data/lib/good_job/scheduler.rb

@@ -230,6 +230,7 @@ module GoodJob # :nodoc:
     # @return [void]
     def create_task(delay = 0)
       future = Concurrent::ScheduledTask.new(delay, args: [performer], executor: executor, timer_set: timer_set) do |thr_performer|
+        Thread.current.name = Thread.current.name.sub("-worker-", "-thread-") if Thread.current.name
         Rails.application.reloader.wrap do
           thr_performer.next
         end

data/lib/good_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 module GoodJob
   # GoodJob gem version.
-  VERSION = '2.4.0'
+  VERSION = '2.4.1'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 2.4.0
+  version: 2.4.1
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-10-02 00:00:00.000000000 Z
+date: 2021-10-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -387,6 +387,7 @@ files:
 - lib/good_job/adapter.rb
 - lib/good_job/cli.rb
 - lib/good_job/configuration.rb
+- lib/good_job/cron_entry.rb
 - lib/good_job/cron_manager.rb
 - lib/good_job/current_thread.rb
 - lib/good_job/daemon.rb