maintenance_tasks 2.12.0 → 2.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +68 -1
- data/app/controllers/maintenance_tasks/application_controller.rb +6 -0
- data/app/jobs/concerns/maintenance_tasks/task_job_concern.rb +24 -4
- data/app/models/maintenance_tasks/run.rb +85 -66
- data/app/models/maintenance_tasks/task.rb +17 -6
- data/app/views/layouts/maintenance_tasks/application.html.erb +2 -1
- data/db/migrate/20201211151756_create_maintenance_tasks_runs.rb +1 -1
- data/db/migrate/20210219212931_change_cursor_to_string.rb +1 -1
- data/db/migrate/20210225152418_remove_index_on_task_name.rb +1 -1
- data/db/migrate/20210517131953_add_arguments_to_maintenance_tasks_runs.rb +1 -1
- data/db/migrate/20211210152329_add_lock_version_to_maintenance_tasks_runs.rb +1 -1
- data/db/migrate/20220706101937_change_runs_tick_columns_to_bigints.rb +1 -1
- data/db/migrate/20220713131925_add_index_on_task_name_and_status_to_runs.rb +1 -1
- data/db/migrate/20230622035229_add_metadata_to_runs.rb +1 -1
- data/lib/maintenance_tasks/engine.rb +2 -4
- data/lib/maintenance_tasks.rb +24 -0
- metadata +12 -12
    
        checksums.yaml
    CHANGED
    
    | @@ -1,7 +1,7 @@ | |
| 1 1 | 
             
            ---
         | 
| 2 2 | 
             
            SHA256:
         | 
| 3 | 
            -
              metadata.gz:  | 
| 4 | 
            -
              data.tar.gz:  | 
| 3 | 
            +
              metadata.gz: 8402f1f4b8919e892a6334373ad397d49b5c6c78893f2f107b990b90653435d1
         | 
| 4 | 
            +
              data.tar.gz: 1459d5b92233a1196aeb83c80bec9da525f04e975a50f6a415cdcd8dc647bd59
         | 
| 5 5 | 
             
            SHA512:
         | 
| 6 | 
            -
              metadata.gz:  | 
| 7 | 
            -
              data.tar.gz:  | 
| 6 | 
            +
              metadata.gz: 3dff16b85a650f4d73cf79df20267d1278b1181f2ba6717742581909ebfd39145dfdd9b9cf1453edaa785474fb6f7dda87af2bad534e6398bf010945b02f2da3
         | 
| 7 | 
            +
              data.tar.gz: c7b75c1edc684fad66c5080ee3642fe469038e1f4ab2329ead0fd3c65e5f0ca2a940250dc988658dce672e2de0d056741b9a317310afa23165ba27794722ce1d
         | 
    
        data/README.md
    CHANGED
    
    | @@ -1027,7 +1027,7 @@ be placed in a `maintenance_tasks.rb` initializer. | |
| 1027 1027 | 
             
            Exceptions raised while a Task is performing are rescued and information about
         | 
| 1028 1028 | 
             
            the error is persisted and visible in the UI.
         | 
| 1029 1029 |  | 
| 1030 | 
            -
            Errors are also sent to  | 
| 1030 | 
            +
            Errors are also sent to `Rails.error.report`, which can be configured by
         | 
| 1031 1031 | 
             
            your application. See the [Error Reporting in Rails
         | 
| 1032 1032 | 
             
            Applications][rails-error-reporting] guide for more details.
         | 
| 1033 1033 |  | 
| @@ -1042,11 +1042,42 @@ Reports to the error reporter will contain the following data: | |
| 1042 1042 | 
             
               * `tick_count`: The tick count at the time of the error
         | 
| 1043 1043 | 
             
               * `errored_element`: The element, if any, that was being processed when the
         | 
| 1044 1044 | 
             
            * `source`: This will be `maintenance-tasks`
         | 
| 1045 | 
            +
            * `handled`: the value of `MaintenanceTasks.report_errors_as_handled` (default `true`, see below)
         | 
| 1045 1046 |  | 
| 1046 1047 | 
             
            Note that `context` may be empty if the Task produced an error before any
         | 
| 1047 1048 | 
             
            context could be gathered (for example, if deserializing the job to process your
         | 
| 1048 1049 | 
             
            Task failed).
         | 
| 1049 1050 |  | 
| 1051 | 
            +
            Here's an example custom subscriber to the Rails error reporter for integrating
         | 
| 1052 | 
            +
            with an exception monitoring service (Bugsnag):
         | 
| 1053 | 
            +
             | 
| 1054 | 
            +
            ```ruby
         | 
| 1055 | 
            +
            # config/initializers/maintenance_tasks.rb
         | 
| 1056 | 
            +
            MaintenanceTasks.report_errors_as_handled = false
         | 
| 1057 | 
            +
             | 
| 1058 | 
            +
            class MaintenanceTasksErrorSubscriber
         | 
| 1059 | 
            +
              def report(error, handled:, severity:, context:, source: nil)
         | 
| 1060 | 
            +
                return unless source == "maintenance-tasks"
         | 
| 1061 | 
            +
             | 
| 1062 | 
            +
                unless handled
         | 
| 1063 | 
            +
                  Bugsnag.notify(error) do |notification|
         | 
| 1064 | 
            +
                    notification.add_metadata(:task, context)
         | 
| 1065 | 
            +
                  end
         | 
| 1066 | 
            +
                else
         | 
| 1067 | 
            +
                  Rails.logger.info(error)
         | 
| 1068 | 
            +
                end
         | 
| 1069 | 
            +
              end
         | 
| 1070 | 
            +
            end
         | 
| 1071 | 
            +
             | 
| 1072 | 
            +
            Rails.error.subscribe(MaintenanceTasksErrorSubscriber.new)
         | 
| 1073 | 
            +
            ```
         | 
| 1074 | 
            +
             | 
| 1075 | 
            +
            `MaintenanceTasks.report_errors_as_handled` determines the value for `handled` in this example.
         | 
| 1076 | 
            +
            By default (for backwards compatibility) this is `true`.
         | 
| 1077 | 
            +
            Setting this to `false` provides more accurate error reporting as it allows to distinguish between
         | 
| 1078 | 
            +
            expected (e.g., via `report_on`) and unexpected errors in error subscribers.
         | 
| 1079 | 
            +
            `false` will be the default in v3.0.
         | 
| 1080 | 
            +
             | 
| 1050 1081 | 
             
            #### Reporting errors during iteration
         | 
| 1051 1082 |  | 
| 1052 1083 | 
             
            By default, errors raised during task iteration will be raised to the
         | 
| @@ -1236,6 +1267,42 @@ The value for `MaintenanceTasks.stuck_task_duration` must be an | |
| 1236 1267 | 
             
            `ActiveSupport::Duration`. If no value is specified, it will default to 5
         | 
| 1237 1268 | 
             
            minutes.
         | 
| 1238 1269 |  | 
| 1270 | 
            +
            #### Configure status reload frequency
         | 
| 1271 | 
            +
             | 
| 1272 | 
            +
            `MaintenanceTasks.status_reload_frequency` can be configured to specify how often
         | 
| 1273 | 
            +
            the run status should be reloaded during iteration. By default, the status is
         | 
| 1274 | 
            +
            reloaded every second, but this can be increased to improve performance. Note that increasing the reload interval impacts how quickly
         | 
| 1275 | 
            +
            your task will stop if it is paused or interrupted.
         | 
| 1276 | 
            +
             | 
| 1277 | 
            +
            ```ruby
         | 
| 1278 | 
            +
            # config/initializers/maintenance_tasks.rb
         | 
| 1279 | 
            +
            MaintenanceTasks.status_reload_frequency = 10.seconds  # Reload status every 10 seconds
         | 
| 1280 | 
            +
            ```
         | 
| 1281 | 
            +
             | 
| 1282 | 
            +
            Individual tasks can also override this setting using the `reload_status_every` method:
         | 
| 1283 | 
            +
             | 
| 1284 | 
            +
            ```ruby
         | 
| 1285 | 
            +
            # app/tasks/maintenance/update_posts_task.rb
         | 
| 1286 | 
            +
             | 
| 1287 | 
            +
            module Maintenance
         | 
| 1288 | 
            +
              class UpdatePostsTask < MaintenanceTasks::Task
         | 
| 1289 | 
            +
                # Reload status every 5 seconds instead of the global default
         | 
| 1290 | 
            +
                reload_status_every(5.seconds)
         | 
| 1291 | 
            +
             | 
| 1292 | 
            +
                def collection
         | 
| 1293 | 
            +
                  Post.all
         | 
| 1294 | 
            +
                end
         | 
| 1295 | 
            +
             | 
| 1296 | 
            +
                def process(post)
         | 
| 1297 | 
            +
                  post.update!(content: "New content!")
         | 
| 1298 | 
            +
                end
         | 
| 1299 | 
            +
              end
         | 
| 1300 | 
            +
            end
         | 
| 1301 | 
            +
            ```
         | 
| 1302 | 
            +
             | 
| 1303 | 
            +
            This optimization can significantly reduce database queries, especially for short iterations.
         | 
| 1304 | 
            +
            This is especially useful if the task doesn't need to check for cancellation/pausing very often.
         | 
| 1305 | 
            +
             | 
| 1239 1306 | 
             
            #### Metadata
         | 
| 1240 1307 |  | 
| 1241 1308 | 
             
            `MaintenanceTasks.metadata` can be configured to specify a proc from which to
         | 
| @@ -13,9 +13,15 @@ module MaintenanceTasks | |
| 13 13 | 
             
                    # <style> tag in app/views/layouts/maintenance_tasks/application.html.erb
         | 
| 14 14 | 
             
                    "'sha256-WHHDQLdkleXnAN5zs0GDXC5ls41CHUaVsJtVpaNx+EM='",
         | 
| 15 15 | 
             
                  )
         | 
| 16 | 
            +
                  capybara_lockstep_scripts = [
         | 
| 17 | 
            +
                    "'sha256-1AoN3ZtJC5OvqkMgrYvhZjp4kI8QjJjO7TAyKYiDw+U='",
         | 
| 18 | 
            +
                    "'sha256-QVSzZi6ZsX/cu4h+hIs1iVivG1BxUmJggiEsGDIXBG0='", # with debug on
         | 
| 19 | 
            +
                  ] if defined?(Capybara::Lockstep)
         | 
| 16 20 | 
             
                  policy.script_src_elem(
         | 
| 17 21 | 
             
                    # <script> tag in app/views/layouts/maintenance_tasks/application.html.erb
         | 
| 18 22 | 
             
                    "'sha256-NiHKryHWudRC2IteTqmY9v1VkaDUA/5jhgXkMTkgo2w='",
         | 
| 23 | 
            +
                    # <script> tag for capybara-lockstep
         | 
| 24 | 
            +
                    *capybara_lockstep_scripts,
         | 
| 19 25 | 
             
                  )
         | 
| 20 26 |  | 
| 21 27 | 
             
                  policy.require_trusted_types_for # disable because we use new DOMParser().parseFromString
         | 
| @@ -101,7 +101,8 @@ module MaintenanceTasks | |
| 101 101 | 
             
                  throw(:abort, :skip_complete_callbacks) if @run.stopping?
         | 
| 102 102 | 
             
                  task_iteration(input)
         | 
| 103 103 | 
             
                  @ticker.tick
         | 
| 104 | 
            -
             | 
| 104 | 
            +
             | 
| 105 | 
            +
                  reload_run_status
         | 
| 105 106 | 
             
                end
         | 
| 106 107 |  | 
| 107 108 | 
             
                def task_iteration(input)
         | 
| @@ -127,6 +128,8 @@ module MaintenanceTasks | |
| 127 128 | 
             
                  @ticker = Ticker.new(MaintenanceTasks.ticker_delay) do |ticks, duration|
         | 
| 128 129 | 
             
                    @run.persist_progress(ticks, duration)
         | 
| 129 130 | 
             
                  end
         | 
| 131 | 
            +
             | 
| 132 | 
            +
                  @last_status_reload = nil
         | 
| 130 133 | 
             
                end
         | 
| 131 134 |  | 
| 132 135 | 
             
                def on_start
         | 
| @@ -190,11 +193,28 @@ module MaintenanceTasks | |
| 190 193 | 
             
                  if MaintenanceTasks.instance_variable_get(:@error_handler)
         | 
| 191 194 | 
             
                    errored_element = task_context.delete(:errored_element)
         | 
| 192 195 | 
             
                    MaintenanceTasks.error_handler.call(error, task_context.except(:run_id, :tick_count), errored_element)
         | 
| 193 | 
            -
                  elsif Rails.gem_version >= Gem::Version.new("7.1")
         | 
| 194 | 
            -
                    Rails.error.report(error, context: task_context, source: "maintenance-tasks")
         | 
| 195 196 | 
             
                  else
         | 
| 196 | 
            -
                    Rails.error.report( | 
| 197 | 
            +
                    Rails.error.report(
         | 
| 198 | 
            +
                      error,
         | 
| 199 | 
            +
                      handled: MaintenanceTasks.report_errors_as_handled,
         | 
| 200 | 
            +
                      context: task_context,
         | 
| 201 | 
            +
                      source: "maintenance-tasks",
         | 
| 202 | 
            +
                    )
         | 
| 197 203 | 
             
                  end
         | 
| 198 204 | 
             
                end
         | 
| 205 | 
            +
             | 
| 206 | 
            +
                def reload_run_status
         | 
| 207 | 
            +
                  return unless should_reload_status?
         | 
| 208 | 
            +
             | 
| 209 | 
            +
                  @run.reload_status
         | 
| 210 | 
            +
                  @last_status_reload = Time.now
         | 
| 211 | 
            +
                end
         | 
| 212 | 
            +
             | 
| 213 | 
            +
                def should_reload_status?
         | 
| 214 | 
            +
                  return true if @last_status_reload.nil?
         | 
| 215 | 
            +
             | 
| 216 | 
            +
                  time_since_last_reload = Time.now - @last_status_reload
         | 
| 217 | 
            +
                  time_since_last_reload >= @task.status_reload_frequency
         | 
| 218 | 
            +
                end
         | 
| 199 219 | 
             
              end
         | 
| 200 220 | 
             
            end
         | 
| @@ -44,15 +44,9 @@ module MaintenanceTasks | |
| 44 44 |  | 
| 45 45 | 
             
                attr_readonly :task_name
         | 
| 46 46 |  | 
| 47 | 
            -
                 | 
| 48 | 
            -
             | 
| 49 | 
            -
             | 
| 50 | 
            -
                  serialize :metadata, coder: JSON
         | 
| 51 | 
            -
                else
         | 
| 52 | 
            -
                  serialize :backtrace
         | 
| 53 | 
            -
                  serialize :arguments, JSON
         | 
| 54 | 
            -
                  serialize :metadata, JSON
         | 
| 55 | 
            -
                end
         | 
| 47 | 
            +
                serialize :backtrace, coder: YAML
         | 
| 48 | 
            +
                serialize :arguments, coder: JSON
         | 
| 49 | 
            +
                serialize :metadata, coder: JSON
         | 
| 56 50 |  | 
| 57 51 | 
             
                scope :active, -> { where(status: ACTIVE_STATUSES) }
         | 
| 58 52 | 
             
                scope :completed, -> { where(status: COMPLETED_STATUSES) }
         | 
| @@ -79,11 +73,10 @@ module MaintenanceTasks | |
| 79 73 | 
             
                # Rescues and retries status transition if an ActiveRecord::StaleObjectError
         | 
| 80 74 | 
             
                # is encountered.
         | 
| 81 75 | 
             
                def enqueued!
         | 
| 82 | 
            -
                   | 
| 83 | 
            -
             | 
| 84 | 
            -
             | 
| 85 | 
            -
                   | 
| 86 | 
            -
                  retry
         | 
| 76 | 
            +
                  with_stale_object_retry do
         | 
| 77 | 
            +
                    status_will_change!
         | 
| 78 | 
            +
                    super
         | 
| 79 | 
            +
                  end
         | 
| 87 80 | 
             
                end
         | 
| 88 81 |  | 
| 89 82 | 
             
                CALLBACKS_TRANSITION = {
         | 
| @@ -92,23 +85,39 @@ module MaintenanceTasks | |
| 92 85 | 
             
                  paused: :pause,
         | 
| 93 86 | 
             
                  succeeded: :complete,
         | 
| 94 87 | 
             
                }.transform_keys(&:to_s)
         | 
| 95 | 
            -
             | 
| 88 | 
            +
             | 
| 89 | 
            +
                DELAYS_PER_ATTEMPT = [0.1, 0.2, 0.4, 0.8, 1.6]
         | 
| 90 | 
            +
                MAX_RETRIES = DELAYS_PER_ATTEMPT.size
         | 
| 91 | 
            +
             | 
| 92 | 
            +
                private_constant :CALLBACKS_TRANSITION, :DELAYS_PER_ATTEMPT, :MAX_RETRIES
         | 
| 96 93 |  | 
| 97 94 | 
             
                # Saves the run, persisting the transition of its status, and all other
         | 
| 98 95 | 
             
                # changes to the object.
         | 
| 99 96 | 
             
                def persist_transition
         | 
| 100 | 
            -
                   | 
| 97 | 
            +
                  retry_count = 0
         | 
| 98 | 
            +
                  begin
         | 
| 99 | 
            +
                    save!
         | 
| 100 | 
            +
                  rescue ActiveRecord::StaleObjectError
         | 
| 101 | 
            +
                    if retry_count < MAX_RETRIES
         | 
| 102 | 
            +
                      sleep(DELAYS_PER_ATTEMPT[retry_count])
         | 
| 103 | 
            +
                      retry_count += 1
         | 
| 104 | 
            +
             | 
| 105 | 
            +
                      success = succeeded?
         | 
| 106 | 
            +
                      reload_status
         | 
| 107 | 
            +
                      if success
         | 
| 108 | 
            +
                        self.status = :succeeded
         | 
| 109 | 
            +
                      else
         | 
| 110 | 
            +
                        job_shutdown
         | 
| 111 | 
            +
                      end
         | 
| 112 | 
            +
             | 
| 113 | 
            +
                      retry
         | 
| 114 | 
            +
                    else
         | 
| 115 | 
            +
                      raise
         | 
| 116 | 
            +
                    end
         | 
| 117 | 
            +
                  end
         | 
| 118 | 
            +
             | 
| 101 119 | 
             
                  callback = CALLBACKS_TRANSITION[status]
         | 
| 102 120 | 
             
                  run_task_callbacks(callback) if callback
         | 
| 103 | 
            -
                rescue ActiveRecord::StaleObjectError
         | 
| 104 | 
            -
                  success = succeeded?
         | 
| 105 | 
            -
                  reload_status
         | 
| 106 | 
            -
                  if success
         | 
| 107 | 
            -
                    self.status = :succeeded
         | 
| 108 | 
            -
                  else
         | 
| 109 | 
            -
                    job_shutdown
         | 
| 110 | 
            -
                  end
         | 
| 111 | 
            -
                  retry
         | 
| 112 121 | 
             
                end
         | 
| 113 122 |  | 
| 114 123 | 
             
                # Increments +tick_count+ by +number_of_ticks+ and +time_running+ by
         | 
| @@ -126,29 +135,23 @@ module MaintenanceTasks | |
| 126 135 | 
             
                    time_running: duration,
         | 
| 127 136 | 
             
                    touch: true,
         | 
| 128 137 | 
             
                  )
         | 
| 129 | 
            -
                  if locking_enabled?
         | 
| 130 | 
            -
                    locking_column = self.class.locking_column
         | 
| 131 | 
            -
                    self[locking_column] += 1
         | 
| 132 | 
            -
                    clear_attribute_change(locking_column)
         | 
| 133 | 
            -
                  end
         | 
| 134 138 | 
             
                end
         | 
| 135 139 |  | 
| 136 140 | 
             
                # Marks the run as errored and persists the error data.
         | 
| 137 141 | 
             
                #
         | 
| 138 142 | 
             
                # @param error [StandardError] the Error being persisted.
         | 
| 139 143 | 
             
                def persist_error(error)
         | 
| 140 | 
            -
                   | 
| 141 | 
            -
             | 
| 142 | 
            -
                     | 
| 143 | 
            -
             | 
| 144 | 
            -
             | 
| 145 | 
            -
             | 
| 146 | 
            -
             | 
| 147 | 
            -
             | 
| 144 | 
            +
                  with_stale_object_retry do
         | 
| 145 | 
            +
                    self.started_at ||= Time.now
         | 
| 146 | 
            +
                    update!(
         | 
| 147 | 
            +
                      status: :errored,
         | 
| 148 | 
            +
                      error_class: truncate(:error_class, error.class.name),
         | 
| 149 | 
            +
                      error_message: truncate(:error_message, error.message),
         | 
| 150 | 
            +
                      backtrace: MaintenanceTasks.backtrace_cleaner.clean(error.backtrace),
         | 
| 151 | 
            +
                      ended_at: Time.now,
         | 
| 152 | 
            +
                    )
         | 
| 153 | 
            +
                  end
         | 
| 148 154 | 
             
                  run_error_callback
         | 
| 149 | 
            -
                rescue ActiveRecord::StaleObjectError
         | 
| 150 | 
            -
                  reload_status
         | 
| 151 | 
            -
                  retry
         | 
| 152 155 | 
             
                end
         | 
| 153 156 |  | 
| 154 157 | 
             
                # Refreshes the status and lock version attributes on the Active Record
         | 
| @@ -241,11 +244,8 @@ module MaintenanceTasks | |
| 241 244 | 
             
                # is encountered.
         | 
| 242 245 | 
             
                def running
         | 
| 243 246 | 
             
                  if locking_enabled?
         | 
| 244 | 
            -
                     | 
| 247 | 
            +
                    with_stale_object_retry do
         | 
| 245 248 | 
             
                      running! unless stopping?
         | 
| 246 | 
            -
                    rescue ActiveRecord::StaleObjectError
         | 
| 247 | 
            -
                      reload_status
         | 
| 248 | 
            -
                      retry
         | 
| 249 249 | 
             
                    end
         | 
| 250 250 | 
             
                  else
         | 
| 251 251 | 
             
                    # Preserve swap-and-replace solution for data races until users
         | 
| @@ -268,11 +268,11 @@ module MaintenanceTasks | |
| 268 268 | 
             
                # @param count [Integer] the total iterations to be performed, as
         | 
| 269 269 | 
             
                #   specified by the Task.
         | 
| 270 270 | 
             
                def start(count)
         | 
| 271 | 
            -
                   | 
| 271 | 
            +
                  with_stale_object_retry do
         | 
| 272 | 
            +
                    update!(started_at: Time.now, tick_total: count)
         | 
| 273 | 
            +
                  end
         | 
| 274 | 
            +
             | 
| 272 275 | 
             
                  task.run_callbacks(:start)
         | 
| 273 | 
            -
                rescue ActiveRecord::StaleObjectError
         | 
| 274 | 
            -
                  reload_status
         | 
| 275 | 
            -
                  retry
         | 
| 276 276 | 
             
                end
         | 
| 277 277 |  | 
| 278 278 | 
             
                # Handles transitioning the status on a Run when the job shuts down.
         | 
| @@ -307,16 +307,15 @@ module MaintenanceTasks | |
| 307 307 | 
             
                # minutes ago, it will transition to cancelled, and the ended_at timestamp
         | 
| 308 308 | 
             
                # will be updated.
         | 
| 309 309 | 
             
                def cancel
         | 
| 310 | 
            -
                   | 
| 311 | 
            -
                     | 
| 312 | 
            -
             | 
| 313 | 
            -
             | 
| 314 | 
            -
             | 
| 315 | 
            -
                     | 
| 310 | 
            +
                  with_stale_object_retry do
         | 
| 311 | 
            +
                    if paused? || stuck?
         | 
| 312 | 
            +
                      self.status = :cancelled
         | 
| 313 | 
            +
                      self.ended_at = Time.now
         | 
| 314 | 
            +
                      persist_transition
         | 
| 315 | 
            +
                    else
         | 
| 316 | 
            +
                      cancelling!
         | 
| 317 | 
            +
                    end
         | 
| 316 318 | 
             
                  end
         | 
| 317 | 
            -
                rescue ActiveRecord::StaleObjectError
         | 
| 318 | 
            -
                  reload_status
         | 
| 319 | 
            -
                  retry
         | 
| 320 319 | 
             
                end
         | 
| 321 320 |  | 
| 322 321 | 
             
                # Marks a Run as pausing.
         | 
| @@ -327,15 +326,14 @@ module MaintenanceTasks | |
| 327 326 | 
             
                # Rescues and retries status transition if an ActiveRecord::StaleObjectError
         | 
| 328 327 | 
             
                # is encountered.
         | 
| 329 328 | 
             
                def pause
         | 
| 330 | 
            -
                   | 
| 331 | 
            -
                     | 
| 332 | 
            -
             | 
| 333 | 
            -
             | 
| 334 | 
            -
                     | 
| 329 | 
            +
                  with_stale_object_retry do
         | 
| 330 | 
            +
                    if stuck?
         | 
| 331 | 
            +
                      self.status = :paused
         | 
| 332 | 
            +
                      persist_transition
         | 
| 333 | 
            +
                    else
         | 
| 334 | 
            +
                      pausing!
         | 
| 335 | 
            +
                    end
         | 
| 335 336 | 
             
                  end
         | 
| 336 | 
            -
                rescue ActiveRecord::StaleObjectError
         | 
| 337 | 
            -
                  reload_status
         | 
| 338 | 
            -
                  retry
         | 
| 339 337 | 
             
                end
         | 
| 340 338 |  | 
| 341 339 | 
             
                # Returns whether a Run is stuck, which is defined as having a status of
         | 
| @@ -499,5 +497,26 @@ module MaintenanceTasks | |
| 499 497 | 
             
                    Rails.application.config.filter_parameters + task.masked_arguments,
         | 
| 500 498 | 
             
                  )
         | 
| 501 499 | 
             
                end
         | 
| 500 | 
            +
             | 
| 501 | 
            +
                def with_stale_object_retry(retry_count = 0)
         | 
| 502 | 
            +
                  yield
         | 
| 503 | 
            +
                rescue ActiveRecord::StaleObjectError
         | 
| 504 | 
            +
                  if retry_count < MAX_RETRIES
         | 
| 505 | 
            +
                    sleep(stale_object_retry_delay(retry_count))
         | 
| 506 | 
            +
                    retry_count += 1
         | 
| 507 | 
            +
                    reload_status
         | 
| 508 | 
            +
             | 
| 509 | 
            +
                    retry
         | 
| 510 | 
            +
                  else
         | 
| 511 | 
            +
                    raise
         | 
| 512 | 
            +
                  end
         | 
| 513 | 
            +
                end
         | 
| 514 | 
            +
             | 
| 515 | 
            +
                def stale_object_retry_delay(retry_count)
         | 
| 516 | 
            +
                  delay = DELAYS_PER_ATTEMPT[retry_count]
         | 
| 517 | 
            +
                  # Add jitter (±25% randomization) to prevent thundering herd
         | 
| 518 | 
            +
                  jitter = delay * 0.25
         | 
| 519 | 
            +
                  delay + (rand * 2 - 1) * jitter
         | 
| 520 | 
            +
                end
         | 
| 502 521 | 
             
              end
         | 
| 503 522 | 
             
            end
         | 
| @@ -33,6 +33,12 @@ module MaintenanceTasks | |
| 33 33 | 
             
                # @api private
         | 
| 34 34 | 
             
                class_attribute :masked_arguments, default: []
         | 
| 35 35 |  | 
| 36 | 
            +
                # The frequency at which to reload the run status during iteration.
         | 
| 37 | 
            +
                # Defaults to the global MaintenanceTasks.status_reload_frequency setting.
         | 
| 38 | 
            +
                #
         | 
| 39 | 
            +
                # @api private
         | 
| 40 | 
            +
                class_attribute :status_reload_frequency, default: MaintenanceTasks.status_reload_frequency
         | 
| 41 | 
            +
             | 
| 36 42 | 
             
                define_callbacks :start, :complete, :error, :cancel, :pause, :interrupt
         | 
| 37 43 |  | 
| 38 44 | 
             
                attr_accessor :metadata
         | 
| @@ -167,6 +173,15 @@ module MaintenanceTasks | |
| 167 173 | 
             
                    self.masked_arguments += attributes
         | 
| 168 174 | 
             
                  end
         | 
| 169 175 |  | 
| 176 | 
            +
                  # Configure how frequently the run status should be reloaded during iteration.
         | 
| 177 | 
            +
                  # Use this to reduce database queries when processing large collections.
         | 
| 178 | 
            +
                  #
         | 
| 179 | 
            +
                  # @param frequency [ActiveSupport::Duration, Numeric] reload status every N seconds (default: 1 second).
         | 
| 180 | 
            +
                  #   Setting this to 10.seconds means status will be reloaded every 10 seconds.
         | 
| 181 | 
            +
                  def reload_status_every(frequency)
         | 
| 182 | 
            +
                    self.status_reload_frequency = frequency
         | 
| 183 | 
            +
                  end
         | 
| 184 | 
            +
             | 
| 170 185 | 
             
                  # Initialize a callback to run after the task starts.
         | 
| 171 186 | 
             
                  #
         | 
| 172 187 | 
             
                  # @param filter_list apply filters to the callback
         | 
| @@ -220,14 +235,10 @@ module MaintenanceTasks | |
| 220 235 | 
             
                  #
         | 
| 221 236 | 
             
                  # @param exceptions list of exceptions to rescue and report
         | 
| 222 237 | 
             
                  # @param report_options [Hash] optionally, supply additional options for `Rails.error.report`.
         | 
| 223 | 
            -
                  #   By default: <code>{ source: " | 
| 238 | 
            +
                  #   By default: <code>{ handled: true, source: "maintenance-tasks" }</code>.
         | 
| 224 239 | 
             
                  def report_on(*exceptions, **report_options)
         | 
| 225 240 | 
             
                    rescue_from(*exceptions) do |exception|
         | 
| 226 | 
            -
                       | 
| 227 | 
            -
                        Rails.error.report(exception, source: "maintenance_tasks", **report_options)
         | 
| 228 | 
            -
                      else
         | 
| 229 | 
            -
                        Rails.error.report(exception, handled: true, **report_options)
         | 
| 230 | 
            -
                      end
         | 
| 241 | 
            +
                      Rails.error.report(exception, source: "maintenance-tasks", **report_options)
         | 
| 231 242 | 
             
                    end
         | 
| 232 243 | 
             
                  end
         | 
| 233 244 |  | 
| @@ -3,6 +3,7 @@ | |
| 3 3 | 
             
              <head>
         | 
| 4 4 | 
             
                <meta charset="utf-8">
         | 
| 5 5 | 
             
                <meta name="viewport" content="width=device-width, initial-scale=1">
         | 
| 6 | 
            +
                <%= capybara_lockstep if defined?(Capybara::Lockstep) %>
         | 
| 6 7 |  | 
| 7 8 | 
             
                <title>
         | 
| 8 9 | 
             
                  <% if content_for?(:page_title) %>
         | 
| @@ -99,6 +100,6 @@ | |
| 99 100 |  | 
| 100 101 | 
             
                    <%= yield %>
         | 
| 101 102 | 
             
                  </div>
         | 
| 102 | 
            -
                </ | 
| 103 | 
            +
                </section>
         | 
| 103 104 | 
             
              </body>
         | 
| 104 105 | 
             
            </html>
         | 
| @@ -1,6 +1,6 @@ | |
| 1 1 | 
             
            # frozen_string_literal: true
         | 
| 2 2 |  | 
| 3 | 
            -
            class CreateMaintenanceTasksRuns < ActiveRecord::Migration[ | 
| 3 | 
            +
            class CreateMaintenanceTasksRuns < ActiveRecord::Migration[7.0]
         | 
| 4 4 | 
             
              def change
         | 
| 5 5 | 
             
                create_table(:maintenance_tasks_runs, id: primary_key_type) do |t|
         | 
| 6 6 | 
             
                  t.string(:task_name, null: false)
         | 
| @@ -1,6 +1,6 @@ | |
| 1 1 | 
             
            # frozen_string_literal: true
         | 
| 2 2 |  | 
| 3 | 
            -
            class ChangeCursorToString < ActiveRecord::Migration[ | 
| 3 | 
            +
            class ChangeCursorToString < ActiveRecord::Migration[7.0]
         | 
| 4 4 | 
             
              # This migration will clear all existing data in the cursor column with MySQL.
         | 
| 5 5 | 
             
              # Ensure no Tasks are paused when this migration is deployed, or they will be resumed from the start.
         | 
| 6 6 | 
             
              # Running tasks are able to gracefully handle this change, even if interrupted.
         | 
| @@ -1,6 +1,6 @@ | |
| 1 1 | 
             
            # frozen_string_literal: true
         | 
| 2 2 |  | 
| 3 | 
            -
            class ChangeRunsTickColumnsToBigints < ActiveRecord::Migration[ | 
| 3 | 
            +
            class ChangeRunsTickColumnsToBigints < ActiveRecord::Migration[7.0]
         | 
| 4 4 | 
             
              def up
         | 
| 5 5 | 
             
                change_table(:maintenance_tasks_runs, bulk: true) do |t|
         | 
| 6 6 | 
             
                  t.change(:tick_count, :bigint)
         | 
| @@ -21,10 +21,8 @@ module MaintenanceTasks | |
| 21 21 | 
             
                  MaintenanceTasks.backtrace_cleaner = Rails.backtrace_cleaner
         | 
| 22 22 | 
             
                end
         | 
| 23 23 |  | 
| 24 | 
            -
                 | 
| 25 | 
            -
                   | 
| 26 | 
            -
                    Rails.application.deprecators[:maintenance_tasks] = MaintenanceTasks.deprecator
         | 
| 27 | 
            -
                  end
         | 
| 24 | 
            +
                initializer "maintenance_tasks.deprecator" do
         | 
| 25 | 
            +
                  Rails.application.deprecators[:maintenance_tasks] = MaintenanceTasks.deprecator
         | 
| 28 26 | 
             
                end
         | 
| 29 27 |  | 
| 30 28 | 
             
                config.to_prepare do
         | 
    
        data/lib/maintenance_tasks.rb
    CHANGED
    
    | @@ -85,6 +85,30 @@ module MaintenanceTasks | |
| 85 85 | 
             
              #  @return [ActiveSupport::Duration] the threshold in seconds after which a task is considered stuck.
         | 
| 86 86 | 
             
              mattr_accessor :stuck_task_duration, default: 5.minutes
         | 
| 87 87 |  | 
| 88 | 
            +
              # @!attribute status_reload_frequency
         | 
| 89 | 
            +
              #  @scope class
         | 
| 90 | 
            +
              #  The frequency at which to reload the run status during iteration.
         | 
| 91 | 
            +
              #  Defaults to 1 second, meaning reload status every second.
         | 
| 92 | 
            +
              #
         | 
| 93 | 
            +
              #  @return [ActiveSupport::Duration, Numeric] the time interval between status reloads.
         | 
| 94 | 
            +
              mattr_accessor :status_reload_frequency, default: 1.second
         | 
| 95 | 
            +
             | 
| 96 | 
            +
              # @!attribute report_errors_as_handled
         | 
| 97 | 
            +
              #  @scope class
         | 
| 98 | 
            +
              #  How unexpected errors are reported to Rails.error.report.
         | 
| 99 | 
            +
              #
         | 
| 100 | 
            +
              #  When an error occurs that isn't explicitly handled (e.g., via `report_on`),
         | 
| 101 | 
            +
              #  it gets reported to Rails.error.report. This setting determines whether
         | 
| 102 | 
            +
              #  these errors are marked as "handled" or "unhandled".
         | 
| 103 | 
            +
              #
         | 
| 104 | 
            +
              #  The current default of `true` is for backwards compatibility, but it prevents
         | 
| 105 | 
            +
              #  error subscribers from distinguishing between expected and unexpected errors.
         | 
| 106 | 
            +
              #  Setting this to `false` provides more accurate error reporting and will become the default in v3.0.
         | 
| 107 | 
            +
              #
         | 
| 108 | 
            +
              #  @see https://api.rubyonrails.org/classes/ActiveSupport/ErrorReporter.html#method-i-report
         | 
| 109 | 
            +
              #  @return [Boolean] whether to report unexpected errors as handled (true) or unhandled (false).
         | 
| 110 | 
            +
              mattr_accessor :report_errors_as_handled, default: true
         | 
| 111 | 
            +
             | 
| 88 112 | 
             
              class << self
         | 
| 89 113 | 
             
                DEPRECATION_MESSAGE = "MaintenanceTasks.error_handler is deprecated and will be removed in the 3.0 release. " \
         | 
| 90 114 | 
             
                  "Instead, reports will be sent to the Rails error reporter. Do not set a handler and subscribe " \
         | 
    
        metadata
    CHANGED
    
    | @@ -1,7 +1,7 @@ | |
| 1 1 | 
             
            --- !ruby/object:Gem::Specification
         | 
| 2 2 | 
             
            name: maintenance_tasks
         | 
| 3 3 | 
             
            version: !ruby/object:Gem::Version
         | 
| 4 | 
            -
              version: 2. | 
| 4 | 
            +
              version: 2.13.0
         | 
| 5 5 | 
             
            platform: ruby
         | 
| 6 6 | 
             
            authors:
         | 
| 7 7 | 
             
            - Shopify Engineering
         | 
| @@ -15,42 +15,42 @@ dependencies: | |
| 15 15 | 
             
                requirements:
         | 
| 16 16 | 
             
                - - ">="
         | 
| 17 17 | 
             
                  - !ruby/object:Gem::Version
         | 
| 18 | 
            -
                    version: '7. | 
| 18 | 
            +
                    version: '7.1'
         | 
| 19 19 | 
             
              type: :runtime
         | 
| 20 20 | 
             
              prerelease: false
         | 
| 21 21 | 
             
              version_requirements: !ruby/object:Gem::Requirement
         | 
| 22 22 | 
             
                requirements:
         | 
| 23 23 | 
             
                - - ">="
         | 
| 24 24 | 
             
                  - !ruby/object:Gem::Version
         | 
| 25 | 
            -
                    version: '7. | 
| 25 | 
            +
                    version: '7.1'
         | 
| 26 26 | 
             
            - !ruby/object:Gem::Dependency
         | 
| 27 27 | 
             
              name: activejob
         | 
| 28 28 | 
             
              requirement: !ruby/object:Gem::Requirement
         | 
| 29 29 | 
             
                requirements:
         | 
| 30 30 | 
             
                - - ">="
         | 
| 31 31 | 
             
                  - !ruby/object:Gem::Version
         | 
| 32 | 
            -
                    version: '7. | 
| 32 | 
            +
                    version: '7.1'
         | 
| 33 33 | 
             
              type: :runtime
         | 
| 34 34 | 
             
              prerelease: false
         | 
| 35 35 | 
             
              version_requirements: !ruby/object:Gem::Requirement
         | 
| 36 36 | 
             
                requirements:
         | 
| 37 37 | 
             
                - - ">="
         | 
| 38 38 | 
             
                  - !ruby/object:Gem::Version
         | 
| 39 | 
            -
                    version: '7. | 
| 39 | 
            +
                    version: '7.1'
         | 
| 40 40 | 
             
            - !ruby/object:Gem::Dependency
         | 
| 41 41 | 
             
              name: activerecord
         | 
| 42 42 | 
             
              requirement: !ruby/object:Gem::Requirement
         | 
| 43 43 | 
             
                requirements:
         | 
| 44 44 | 
             
                - - ">="
         | 
| 45 45 | 
             
                  - !ruby/object:Gem::Version
         | 
| 46 | 
            -
                    version: '7. | 
| 46 | 
            +
                    version: '7.1'
         | 
| 47 47 | 
             
              type: :runtime
         | 
| 48 48 | 
             
              prerelease: false
         | 
| 49 49 | 
             
              version_requirements: !ruby/object:Gem::Requirement
         | 
| 50 50 | 
             
                requirements:
         | 
| 51 51 | 
             
                - - ">="
         | 
| 52 52 | 
             
                  - !ruby/object:Gem::Version
         | 
| 53 | 
            -
                    version: '7. | 
| 53 | 
            +
                    version: '7.1'
         | 
| 54 54 | 
             
            - !ruby/object:Gem::Dependency
         | 
| 55 55 | 
             
              name: csv
         | 
| 56 56 | 
             
              requirement: !ruby/object:Gem::Requirement
         | 
| @@ -85,14 +85,14 @@ dependencies: | |
| 85 85 | 
             
                requirements:
         | 
| 86 86 | 
             
                - - ">="
         | 
| 87 87 | 
             
                  - !ruby/object:Gem::Version
         | 
| 88 | 
            -
                    version: '7. | 
| 88 | 
            +
                    version: '7.1'
         | 
| 89 89 | 
             
              type: :runtime
         | 
| 90 90 | 
             
              prerelease: false
         | 
| 91 91 | 
             
              version_requirements: !ruby/object:Gem::Requirement
         | 
| 92 92 | 
             
                requirements:
         | 
| 93 93 | 
             
                - - ">="
         | 
| 94 94 | 
             
                  - !ruby/object:Gem::Version
         | 
| 95 | 
            -
                    version: '7. | 
| 95 | 
            +
                    version: '7.1'
         | 
| 96 96 | 
             
            - !ruby/object:Gem::Dependency
         | 
| 97 97 | 
             
              name: zeitwerk
         | 
| 98 98 | 
             
              requirement: !ruby/object:Gem::Requirement
         | 
| @@ -182,7 +182,7 @@ homepage: https://github.com/Shopify/maintenance_tasks | |
| 182 182 | 
             
            licenses:
         | 
| 183 183 | 
             
            - MIT
         | 
| 184 184 | 
             
            metadata:
         | 
| 185 | 
            -
              source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v2. | 
| 185 | 
            +
              source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v2.13.0
         | 
| 186 186 | 
             
              allowed_push_host: https://rubygems.org
         | 
| 187 187 | 
             
            rdoc_options: []
         | 
| 188 188 | 
             
            require_paths:
         | 
| @@ -191,14 +191,14 @@ required_ruby_version: !ruby/object:Gem::Requirement | |
| 191 191 | 
             
              requirements:
         | 
| 192 192 | 
             
              - - ">="
         | 
| 193 193 | 
             
                - !ruby/object:Gem::Version
         | 
| 194 | 
            -
                  version: '3. | 
| 194 | 
            +
                  version: '3.2'
         | 
| 195 195 | 
             
            required_rubygems_version: !ruby/object:Gem::Requirement
         | 
| 196 196 | 
             
              requirements:
         | 
| 197 197 | 
             
              - - ">="
         | 
| 198 198 | 
             
                - !ruby/object:Gem::Version
         | 
| 199 199 | 
             
                  version: '0'
         | 
| 200 200 | 
             
            requirements: []
         | 
| 201 | 
            -
            rubygems_version: 3. | 
| 201 | 
            +
            rubygems_version: 3.7.2
         | 
| 202 202 | 
             
            specification_version: 4
         | 
| 203 203 | 
             
            summary: A Rails engine for queuing and managing maintenance tasks
         | 
| 204 204 | 
             
            test_files: []
         |