good_job 1.0.2 → 1.0.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f7c64e8766a52e6bbbb5e55b409bb0100e54790d6d88340893e61f930cbf8b9e
-  data.tar.gz: 7e89b7bf2d3aaa45cc5b07deab36f5334a61cac44c0aec06206ddbd6eb0ebc22
+  metadata.gz: fc7b3997aaa38583efb4c91e66a2a5f9b8bfd75d4c14c9baaf8f1af7c236a42e
+  data.tar.gz: 285b441a5e5add440142960884370a1b74168266252fa48ce569cbb56e80f240
 SHA512:
-  metadata.gz: 944b64f713ad584b56fe386dc63aa6afc0538e498cff4fb46aec9098306b7e436a3d29cebd968157f34e62c9b1b0ce932e15967b4a631f9f04c60ca43d77f4c8
-  data.tar.gz: 5000ffc97bc381cb94999478c1fe90202b681d5cf87d697f4700cc60b94a7417fd67ca3f5428c4b50acb195bde7850a3a5d809e32c5baaf08ae3a58a2101624a
+  metadata.gz: f54e83d6ac19d80da2123a395ccf6ffd7eb2faab8e68bd5d50a572c8f0bf64fbc907125a47b7ff91dccac5497af113d043bba4f62ee6ae10fecaf3c004a1114d
+  data.tar.gz: 0b65a60bde26db0cfcfa55f166e80043bcbccb6e2c80f5de1840d2deae3ee28880c904bb53ffc2478ad834ffef8cfe567cee9a179932e20e4f92fdf8504838e4

data/CHANGELOG.md

@@ -1,6 +1,23 @@
 # Changelog
 
-## [v1.0.2](https://github.com/bensheldon/good_job/tree/v1.0.2) (2020-07-24)
+## [v1.0.3](https://github.com/bensheldon/good_job/tree/v1.0.3) (2020-07-25)
+
+[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.2...v1.0.3)
+
+**Fixed bugs:**
+
+- Preserve GoodJob::Jobs when a StandardError is raised [\#60](https://github.com/bensheldon/good_job/issues/60)
+
+**Closed issues:**
+
+- Have an initial setup generator [\#6](https://github.com/bensheldon/good_job/issues/6)
+
+**Merged pull requests:**
+
+- Re-perform a job if a StandardError bubbles up; better document job reliability [\#62](https://github.com/bensheldon/good_job/pull/62) ([bensheldon](https://github.com/bensheldon))
+- Update the setup documentation to use correct bin setup command [\#61](https://github.com/bensheldon/good_job/pull/61) ([jm96441n](https://github.com/jm96441n))
+
+## [v1.0.2](https://github.com/bensheldon/good_job/tree/v1.0.2) (2020-07-25)
 
 [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.1...v1.0.2)
 
@@ -96,6 +113,7 @@
 - Add configuration options to good\_job executable [\#33](https://github.com/bensheldon/good_job/pull/33) ([bensheldon](https://github.com/bensheldon))
 - Extract Job querying behavior out of Scheduler [\#31](https://github.com/bensheldon/good_job/pull/31) ([bensheldon](https://github.com/bensheldon))
 - Allow configuration of Rails queue adapter with `:good\_job` [\#28](https://github.com/bensheldon/good_job/pull/28) ([bensheldon](https://github.com/bensheldon))
+- Update development Ruby to 2.6.5 [\#22](https://github.com/bensheldon/good_job/pull/22) ([bensheldon](https://github.com/bensheldon))
 
 ## [v0.5.0](https://github.com/bensheldon/good_job/tree/v0.5.0) (2020-07-13)
 
@@ -120,7 +138,6 @@
 
 **Merged pull requests:**
 
-- Update development Ruby to 2.6.5 [\#22](https://github.com/bensheldon/good_job/pull/22) ([bensheldon](https://github.com/bensheldon))
 - Simplify the internal API, removing JobWrapper and InlineScheduler [\#21](https://github.com/bensheldon/good_job/pull/21) ([bensheldon](https://github.com/bensheldon))
 - Generate a new future for every executed job [\#20](https://github.com/bensheldon/good_job/pull/20) ([bensheldon](https://github.com/bensheldon))
 - Configuration for maximum number of job execution threads [\#18](https://github.com/bensheldon/good_job/pull/18) ([bensheldon](https://github.com/bensheldon))

data/README.md

@@ -79,25 +79,85 @@ $ bundle install
    #   [--poll-interval=N]       # Interval between polls for available jobs in seconds (default: 1)
    ```
 
-### Taking advantage of ActiveJob
+### Error handling, retries, and reliability
 
-ActiveJob has a rich set of built-in functionality for timeouts, error handling, and retrying. For example:
+GoodJob guarantees _at-least-once_ performance of jobs. GoodJob fully supports ActiveJob's built-in functionality for error handling, retries and timeouts. 
+
+#### Error handling
+
+By default, if a job raises an error while it is being performed, _and it bubbles up to the GoodJob backend_, GoodJob will be immediately re-perform the job until it finishes successfully.
+
+- `Exception`-type errors, such as a SIGINT, will always cause a job to be re-performed.
+- `StandardError`-type errors, by default, will cause a job to be re-performed, though this is configurable:
+   
+    ```ruby
+    # config/initializers/good_job.rb
+    GoodJob.reperform_jobs_on_standard_error = true # => default
+    ```
+
+### Retrying jobs
+
+ActiveJob can be configured to retry an infinite number of times, with an exponential backoff. Using ActiveJob's `retry_on` will ensure that errors do not bubble up to the GoodJob backend:
 
 ```ruby
 class ApplicationJob < ActiveJob::Base  
-  # Retry errors an infinite number of times with exponential back-off
   retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
+  # ...
+end
+```
+
+When specifying a limited number of retries, care must be taken to ensure that an error does not bubble up to the GoodJob backend because that will result in the job being re-performed:
+
+```ruby
+class ApplicationJob < ActiveJob::Base  
+  retry_on StandardError, attempts: 5 do |_job, _exception|
+    # Log error, etc.
+    # You must implement this block, otherwise, 
+    #   Active Job will re-raise the error.
+    # Do not re-raise the error, otherwise 
+    #   GoodJob will immediately re-perform the job. 
+  end
+  # ...
+end
+```
+
+GoodJob can be configured to allow omitting `retry_on`'s block argument and implicitly discard un-handled errors:
 
-  # Timeout jobs after 10 minutes
+    ```ruby
+    # config/initializers/good_job.rb
+    
+    # Do NOT re-perform a job if a StandardError bubbles up to the GoodJob backend
+    GoodJob.reperform_jobs_on_standard_error = false 
+    ```
+
+ActiveJob's `discard_on` functionality is supported too.
+
+#### ActionMailer retries
+
+Using a Mailer's `#deliver_later` will enqueue an instance of `ActionMailer::DeliveryJob` which inherits from `ActiveJob::Base` rather than your applications `ApplicationJob`. You can use an initializer to configure retries on `ActionMailer::DeliveryJob`:
+
+```ruby
+# config/initializers/good_job.rb
+ActionMailer::DeliveryJob.retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
+```
+
+#### Timeouts
+
+Job timeouts can be configured with an `around_perform`:
+
+```ruby
+class ApplicationJob < ActiveJob::Base  
   JobTimeoutError = Class.new(StandardError)
+  
   around_perform do |_job, block|
+    # Timeout jobs after 10 minutes
     Timeout.timeout(10.minutes, JobTimeoutError) do
       block.call
     end
   end
 end
 ```
-   
+
 ### Configuring Job Execution Threads
     
 GoodJob executes enqueued jobs using threads. There is a lot than can be said about [multithreaded behavior in Ruby on Rails](https://guides.rubyonrails.org/threading_and_code_execution.html), but briefly:
@@ -167,7 +227,7 @@ To run tests:
 $ git clone git@github.com:bensheldon/good_job.git
 
 # Set up the local environment
-$ bin/setup_test
+$ bin/setup
 
 # Run the tests
 $ bin/rspec

data/lib/good_job.rb

@@ -13,6 +13,7 @@ require 'active_job/queue_adapters/good_job_adapter'
 
 module GoodJob
   mattr_accessor :preserve_job_records, default: false
+  mattr_accessor :reperform_jobs_on_standard_error, default: true
   include Logging
 
   ActiveSupport.run_load_hooks(:good_job, self)

data/lib/good_job/cli.rb

@@ -39,13 +39,7 @@ module GoodJob
       job_query = GoodJob::Job.all.priority_ordered
       queue_names_without_all = queue_names.reject { |q| q == '*' }
       job_query = job_query.where(queue_name: queue_names_without_all) unless queue_names_without_all.size.zero?
-
-      performer_method = if GoodJob.preserve_job_records
-                           :perform_with_advisory_lock_and_preserve_job_records
-                         else
-                           :perform_with_advisory_lock_and_destroy_job_records
-                         end
-      job_performer = GoodJob::Performer.new(job_query, performer_method)
+      job_performer = GoodJob::Performer.new(job_query, :perform_with_advisory_lock)
 
       $stdout.puts "GoodJob worker starting with max_threads=#{max_threads} on queues=#{queue_names.join(',')}"
 

data/lib/good_job/job.rb

@@ -21,7 +21,7 @@ module GoodJob
     scope :priority_ordered, -> { order(priority: :desc) }
     scope :finished, ->(timestamp = nil) { timestamp ? where(arel_table['finished_at'].lteq(timestamp)) : where.not(finished_at: nil) }
 
-    def self.perform_with_advisory_lock(destroy_after: !GoodJob.preserve_job_records)
+    def self.perform_with_advisory_lock
       good_job = nil
       result = nil
       error = nil
@@ -30,20 +30,12 @@ module GoodJob
         good_job = good_jobs.first
         break unless good_job
 
-        result, error = good_job.perform(destroy_after: destroy_after)
+        result, error = good_job.perform
       end
 
       [good_job, result, error] if good_job
     end
 
-    def self.perform_with_advisory_lock_and_preserve_job_records
-      perform_with_advisory_lock(destroy_after: false)
-    end
-
-    def self.perform_with_advisory_lock_and_destroy_job_records
-      perform_with_advisory_lock(destroy_after: true)
-    end
-
     def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
       good_job = nil
       ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|
@@ -64,40 +56,49 @@ module GoodJob
       good_job
     end
 
-    def perform(destroy_after: true)
+    def perform(destroy_after: !GoodJob.preserve_job_records, reperform_on_standard_error: GoodJob.reperform_jobs_on_standard_error)
       raise PreviouslyPerformedError, 'Cannot perform a job that has already been performed' if finished_at
 
       result = nil
+      rescued_error = nil
       error = nil
 
       ActiveSupport::Notifications.instrument("before_perform_job.good_job", { good_job: self })
       self.performed_at = Time.current
       save! unless destroy_after
 
-      ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self }) do
-        params = serialized_params.merge(
-          "provider_job_id" => id
-        )
-        begin
+      params = serialized_params.merge(
+        "provider_job_id" => id
+      )
+
+      begin
+        ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self }) do
           result = ActiveJob::Base.execute(params)
-        rescue StandardError => e
-          error = e
         end
+      rescue StandardError => e
+        rescued_error = e
       end
 
-      if error.nil? && result.is_a?(Exception)
+      if rescued_error
+        error = rescued_error
+      elsif result.is_a?(Exception)
         error = result
         result = nil
       end
 
       error_message = "#{error.class}: #{error.message}" if error
       self.error = error_message
-      self.finished_at = Time.current
 
-      if destroy_after
-        destroy!
-      else
+      if rescued_error && reperform_on_standard_error
         save!
+      else
+        self.finished_at = Time.current
+
+        if destroy_after
+          destroy!
+        else
+          save!
+        end
       end
 
       [result, error]

data/lib/good_job/version.rb

@@ -1,3 +1,3 @@
 module GoodJob
-  VERSION = '1.0.2'.freeze
+  VERSION = '1.0.3'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: good_job
 version: !ruby/object:Gem::Version
-  version: 1.0.2
+  version: 1.0.3
 platform: ruby
 authors:
 - Ben Sheldon
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-07-25 00:00:00.000000000 Z
+date: 2020-07-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: concurrent-ruby