maintenance_tasks 1.5.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8c13ec8d79682d304378d636955e9726cd1d67ebb167a549a2d6479f7eea6712
-  data.tar.gz: bf7026b08f0f442c3c074118dfca1cbae4babd7d234b97dc63cf66cdf94db67b
+  metadata.gz: 8affacb1fe5aa5298899bb27875132a4b299b1bd89e239b8fc234df1d00e171d
+  data.tar.gz: cd9351e3abed12d1c1bede76cea7ac1d664159983b3ece57fe2707a0de971db5
 SHA512:
-  metadata.gz: 3b41ee8a4ccaf28d590d7acd2a81fd5123dd3e81b552454082313d3a183b5840eb3ff130ab6c0a40c3e05112899680cee03bbec74c39e11bf0f91dff9955edb1
-  data.tar.gz: 42321e823247c83d58ead579070a5b007c7c0089601c7bcf587af311be715eb791102b94c2fdff513010816977c4c318297bb9f946726b3bd08066bc0ce295b2
+  metadata.gz: c0459cf005532667f4d5bbc8a1b05adc5e4aa57bccb4b4a3e9b9338589ff73043272201e5cd0a86ed0abfa38430b18ae5021ec1991a81f064444665be8de4e3f
+  data.tar.gz: d1f9b671f215a2d4a908a9a2353f02d6800511ea8d3337b4074712ec3aed7bdcd3a6b243960915a11e4ee31ca1b756d1992ce3af5a5cd27457a9016b42136842

data/README.md

@@ -57,6 +57,7 @@ Example:
 
 ```ruby
 # app/tasks/maintenance/update_posts_task.rb
+
 module Maintenance
   class UpdatePostsTask < MaintenanceTasks::Task
     def collection
@@ -68,7 +69,7 @@ module Maintenance
     end
 
     def process(post)
-      post.update!(content: 'New content!')
+      post.update!(content: "New content!")
     end
   end
 end
@@ -78,8 +79,8 @@ end
 
 You can also write a Task that iterates on a CSV file. Note that writing CSV
 Tasks **requires Active Storage to be configured**. Ensure that the dependency
-is specified in your application's Gemfile, and that you've followed the
-[setup instuctions][setup].
+is specified in your application's Gemfile, and that you've followed the [setup
+instuctions][setup].
 
 [setup]: https://edgeguides.rubyonrails.org/active_storage_overview.html#setup
 
@@ -95,6 +96,7 @@ The generated task is a subclass of `MaintenanceTasks::Task` that implements:
 
 ```ruby
 # app/tasks/maintenance/import_posts_task.rb
+
 module Maintenance
   class ImportPostsTask < MaintenanceTasks::Task
     csv_collection
@@ -112,16 +114,20 @@ title,content
 My Title,Hello World!
 ```
 
+The files uploaded to your Active Storage service provider will be renamed first
+to include an ISO8601 timestamp and the Task name in snake case format.
+
 ### Processing Batch Collections
 
 The Maintenance Tasks gem supports processing Active Records in batches. This
 can reduce the number of calls your Task makes to the database. Use
-`ActiveRecord::Batches#in_batches` on the relation returned by your collection to specify that your Task should process
-batches instead of records. Active Record defaults to 1000 records by batch, but a custom size can be
-specified.
+`ActiveRecord::Batches#in_batches` on the relation returned by your collection
+to specify that your Task should process batches instead of records. Active
+Record defaults to 1000 records by batch, but a custom size can be specified.
 
 ```ruby
 # app/tasks/maintenance/update_posts_in_batches_task.rb
+
 module Maintenance
   class UpdatePostsInBatchesTask < MaintenanceTasks::Task
     def collection
@@ -147,10 +153,11 @@ your collection, and your Task's progress will be displayed in terms of batches
 **Important!** Batches should only be used if `#process` is performing a batch
 operation such as `#update_all` or `#delete_all`. If you need to iterate over
 individual records, you should define a collection that [returns an
-`ActiveRecord::Relation`](#creating-a-task). This uses batching
-internally, but loads the records with one SQL query. Conversely, batch
-collections load the primary keys of the records of the batch first, and then perform an additional query to load the
-records when calling `each` (or any `Enumerable` method) inside `#process`.
+`ActiveRecord::Relation`](#creating-a-task). This uses batching internally, but
+loads the records with one SQL query. Conversely, batch collections load the
+primary keys of the records of the batch first, and then perform an additional
+query to load the records when calling `each` (or any `Enumerable` method)
+inside `#process`.
 
 ### Throttling
 
@@ -162,6 +169,7 @@ Specify the throttle condition as a block:
 
 ```ruby
 # app/tasks/maintenance/update_posts_throttled_task.rb
+
 module Maintenance
   class UpdatePostsThrottledTask < MaintenanceTasks::Task
     throttle_on(backoff: 1.minute) do
@@ -195,12 +203,12 @@ conditions.
 ### Custom Task Parameters
 
 Tasks may need additional information, supplied via parameters, to run.
-Parameters can be defined as Active Model Attributes in a Task, and then
-become accessible to any of Task's methods: `#collection`, `#count`, or
-`#process`.
+Parameters can be defined as Active Model Attributes in a Task, and then become
+accessible to any of Task's methods: `#collection`, `#count`, or `#process`.
 
 ```ruby
 # app/tasks/maintenance/update_posts_via_params_task.rb
+
 module Maintenance
   class UpdatePostsViaParamsTask < MaintenanceTasks::Task
     attribute :updated_content, :string
@@ -227,6 +235,73 @@ to run. Since arguments are specified in the user interface via text area
 inputs, it's important to check that they conform to the format your Task
 expects, and to sanitize any inputs if necessary.
 
+### Using Task Callbacks
+
+The Task provides callbacks that hook into its life cycle.
+
+Available callbacks are:
+
+`after_start`
+`after_pause`
+`after_interrupt`
+`after_cancel`
+`after_complete`
+`after_error`
+
+```ruby
+module Maintenance
+  class UpdatePostsTask < MaintenanceTasks::Task
+    after_start :notify
+
+    def notify
+      NotifyJob.perform_later(self.class.name)
+    end
+
+    # ...
+  end
+end
+```
+
+Note: The `after_error` callback is guaranteed to complete,
+so any exceptions raised in your callback code are ignored.
+If your `after_error` callback code can raise an exception,
+you'll need to rescue it and handle it appropriately
+within the callback.
+
+```ruby
+module Maintenance
+  class UpdatePostsTask < MaintenanceTasks::Task
+    after_error :dangerous_notify
+
+    def dangerous_notify
+      # This error is rescued in favour of the original error causing the error flow.
+      raise NotDeliveredError
+    end
+
+    # ...
+  end
+end
+```
+
+If any of the other callbacks cause an exception,
+it will be handled by the error handler,
+and will cause the task to stop running.
+
+Callback behaviour can be shared across all tasks using an initializer.
+
+```ruby
+# config/initializer/maintenance_tasks.rb
+Rails.autoloaders.main.on_load("MaintenanceTasks::Task") do
+  MaintenanceTasks::Task.class_eval do
+    after_start(:notify)
+
+    private
+
+    def notify; end
+  end
+end
+```
+
 ### Considerations when writing Tasks
 
 MaintenanceTasks relies on the queue adapter configured for your application to
@@ -259,7 +334,7 @@ Example:
 ```ruby
 # test/tasks/maintenance/update_posts_task_test.rb
 
-require 'test_helper'
+require "test_helper"
 
 module Maintenance
   class UpdatePostsTaskTest < ActiveSupport::TestCase
@@ -268,7 +343,7 @@ module Maintenance
 
       Maintenance::UpdatePostsTask.process(post)
 
-      assert_equal 'New content!', post.content
+      assert_equal "New content!", post.content
     end
   end
 end
@@ -281,20 +356,50 @@ takes a `CSV::Row` as an argument. You can pass a row, or a hash with string
 keys to `#process` from your test.
 
 ```ruby
-# app/tasks/maintenance/import_posts_task_test.rb
+# test/tasks/maintenance/import_posts_task_test.rb
+
+require "test_helper"
+
 module Maintenance
   class ImportPostsTaskTest < ActiveSupport::TestCase
     test "#process performs a task iteration" do
       assert_difference -> { Post.count } do
         Maintenance::UpdatePostsTask.process({
-          'title' => 'My Title',
-          'content' => 'Hello World!',
+          "title" => "My Title",
+          "content" => "Hello World!",
         })
       end
 
       post = Post.last
-      assert_equal 'My Title', post.title
-      assert_equal 'Hello World!', post.content
+      assert_equal "My Title", post.title
+      assert_equal "Hello World!", post.content
+    end
+  end
+end
+```
+
+### Writing tests for a Task with parameters
+
+Tests for tasks with parameters need to instatiate the task class in order to
+assign attributes. Once the task instance is setup, you may test `#process`
+normally.
+
+```ruby
+# test/tasks/maintenance/update_posts_via_params_task_test.rb
+
+require "test_helper"
+
+module Maintenance
+  class UpdatePostsViaParamsTaskTest < ActiveSupport::TestCase
+    setup do
+      @task = UpdatePostsViaParamsTask.new
+      @task.updated_content = "Testing"
+    end
+
+    test "#process performs a task iteration" do
+      assert_difference -> { Post.first.content } do
+        task.process(Post.first)
+      end
     end
   end
 end
@@ -313,20 +418,21 @@ $ bundle exec maintenance_tasks perform Maintenance::UpdatePostsTask
 To run a Task that processes CSVs from the command line, use the --csv option:
 
 ```bash
-$ bundle exec maintenance_tasks perform Maintenance::ImportPostsTask --csv 'path/to/my_csv.csv'
+$ bundle exec maintenance_tasks perform Maintenance::ImportPostsTask --csv "path/to/my_csv.csv"
 ```
 
 To run a Task that takes arguments from the command line, use the --arguments
-option, passing arguments as a set of <key>:<value> pairs:
+option, passing arguments as a set of \<key>:\<value> pairs:
 
 ```bash
-$ bundle exec maintenance_tasks perform Maintenance::ParamsTask --arguments post_ids:1,2,3 content:"Hello, World!"
+$ bundle exec maintenance_tasks perform Maintenance::ParamsTask \
+  --arguments post_ids:1,2,3 content:"Hello, World!"
 ```
 
 You can also run a Task in Ruby by sending `run` with a Task name to Runner:
 
 ```ruby
-MaintenanceTasks::Runner.run(name: 'Maintenance::UpdatePostsTask')
+MaintenanceTasks::Runner.run(name: "Maintenance::UpdatePostsTask")
 ```
 
 To run a Task that processes CSVs using the Runner, provide a Hash containing an
@@ -334,8 +440,8 @@ open IO object and a filename to `run`:
 
 ```ruby
 MaintenanceTasks::Runner.run(
-  name: 'Maintenance::ImportPostsTask'
-  csv_file: { io: File.open('path/to/my_csv.csv'), filename: 'my_csv.csv' }
+  name: "Maintenance::ImportPostsTask",
+  csv_file: { io: File.open("path/to/my_csv.csv"), filename: "my_csv.csv" }
 )
 ```
 
@@ -358,8 +464,8 @@ a Task can be in:
 * **enqueued**: A Task that is waiting to be performed after a user has
   instructed it to run.
 * **running**: A Task that is currently being performed by a job worker.
-* **pausing**: A Task that was paused by a user, but needs to finish work
-  before stopping.
+* **pausing**: A Task that was paused by a user, but needs to finish work before
+  stopping.
 * **paused**: A Task that was paused by a user and is not performing. It can be
   resumed.
 * **interrupted**: A Task that has been momentarily interrupted by the job
@@ -433,6 +539,7 @@ you can define an error handler:
 
 ```ruby
 # config/initializers/maintenance_tasks.rb
+
 MaintenanceTasks.error_handler = ->(error, task_context, _errored_element) do
   Bugsnag.notify(error) do |notification|
     notification.add_tab(:task, task_context)
@@ -448,18 +555,18 @@ The error handler should be a lambda that accepts three arguments:
   * `task_name`: The name of the Task that errored
   * `started_at`: The time the Task started
   * `ended_at`: The time the Task errored
+
   Note that `task_context` may be empty if the Task produced an error before any
   context could be gathered (for example, if deserializing the job to process
   your Task failed).
-* `errored_element`: The element, if any, that was being processed when the
-  Task raised an exception. If you would like to pass this object to your
-  exception monitoring service, make sure you **sanitize the object** to avoid
-  leaking sensitive data and **convert it to a format** that is compatible with
-  your bug tracker. For example, Bugsnag only sends the id and class name of
-  Active Record objects in order to protect sensitive data. CSV rows, on the
-  other hand, are converted to strings and passed raw to Bugsnag, so make sure
-  to filter any personal data from these objects before adding them to a
-  report.
+* `errored_element`: The element, if any, that was being processed when the Task
+  raised an exception. If you would like to pass this object to your exception
+  monitoring service, make sure you **sanitize the object** to avoid leaking
+  sensitive data and **convert it to a format** that is compatible with your bug
+  tracker. For example, Bugsnag only sends the id and class name of Active
+  Record objects in order to protect sensitive data. CSV rows, on the other
+  hand, are converted to strings and passed raw to Bugsnag, so make sure to
+  filter any personal data from these objects before adding them to a report.
 
 #### Customizing the maintenance tasks module
 
@@ -468,7 +575,8 @@ tasks will be placed.
 
 ```ruby
 # config/initializers/maintenance_tasks.rb
-MaintenanceTasks.tasks_module = 'TaskModule'
+
+MaintenanceTasks.tasks_module = "TaskModule"
 ```
 
 If no value is specified, it will default to `Maintenance`.
@@ -481,9 +589,11 @@ maintenance tasks in your application.
 
 ```ruby
 # config/initializers/maintenance_tasks.rb
+
 MaintenanceTasks.job = 'CustomTaskJob'
 
 # app/jobs/custom_task_job.rb
+
 class CustomTaskJob < MaintenanceTasks::TaskJob
   queue_as :low_priority
 end
@@ -491,8 +601,8 @@ end
 
 The Job class **must inherit** from `MaintenanceTasks::TaskJob`.
 
-Note that `retry_on` is not supported for custom Job
-classes, so failed jobs cannot be retried.
+Note that `retry_on` is not supported for custom Job classes, so failed jobs
+cannot be retried.
 
 #### Customizing the rate at which task progress gets updated
 
@@ -502,6 +612,7 @@ task progress gets persisted to the database. It can be a `Numeric` value or an
 
 ```ruby
 # config/initializers/maintenance_tasks.rb
+
 MaintenanceTasks.ticker_delay = 2.seconds
 ```
 
@@ -516,6 +627,7 @@ key, as specified in your application's `config/storage.yml`:
 
 ```yaml
 # config/storage.yml
+
 user_data:
   service: GCS
   credentials: <%= Rails.root.join("path/to/user/data/keyfile.json") %>
@@ -531,6 +643,7 @@ internal:
 
 ```ruby
 # config/initializers/maintenance_tasks.rb
+
 MaintenanceTasks.active_storage_service = :internal
 ```
 
@@ -545,6 +658,7 @@ An `ActiveSupport::BacktraceCleaner` should be used.
 
 ```ruby
 # config/initializers/maintenance_tasks.rb
+
 cleaner = ActiveSupport::BacktraceCleaner.new
 cleaner.add_silencer { |line| line =~ /ignore_this_dir/ }
 
@@ -568,10 +682,11 @@ This ensures that new migrations are installed and run as well.
 **What if I've deleted my previous Maintenance Task migrations?**
 
 The install command will attempt to reinstall these old migrations and migrating
-the database will cause problems. Use `bin/rails generate maintenance_tasks:install:migrations`
-to copy the gem's migrations to your `db/migrate` folder. Check the release
-notes to see if any new migrations were added since your last gem upgrade.
-Ensure that these are kept, but remove any migrations that already ran.
+the database will cause problems. Use `bin/rails
+maintenance_tasks:install:migrations` to copy the gem's migrations to your
+`db/migrate` folder. Check the release notes to see if any new migrations were
+added since your last gem upgrade.  Ensure that these are kept, but remove any
+migrations that already ran.
 
 Run the migrations using `bin/rails db:migrate`.
 

data/app/helpers/maintenance_tasks/application_helper.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # Module for common view helpers.
   #

data/app/jobs/concerns/maintenance_tasks/task_job_concern.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # Concern that holds the behaviour of the job that runs the tasks. It is
   # included in {TaskJob} and if MaintenanceTasks.job is overridden, it must be
@@ -90,11 +91,11 @@ module MaintenanceTasks
     def before_perform
       @run = arguments.first
       @task = @run.task
-      if @task.respond_to?(:csv_content=)
+      if @task.has_csv_content?
         @task.csv_content = @run.csv_file.download
       end
 
-      @run.running! unless @run.stopping?
+      @run.running
 
       @ticker = Ticker.new(MaintenanceTasks.ticker_delay) do |ticks, duration|
         @run.persist_progress(ticks, duration)
@@ -105,22 +106,29 @@ module MaintenanceTasks
       count = @task.count
       count = @enumerator&.size if count == :no_count
       @run.update!(started_at: Time.now, tick_total: count)
+      @task.run_callbacks(:start)
     end
 
     def on_complete
       @run.status = :succeeded
       @run.ended_at = Time.now
+      @task.run_callbacks(:complete)
     end
 
     def on_shutdown
       if @run.cancelling?
         @run.status = :cancelled
+        @task.run_callbacks(:cancel)
         @run.ended_at = Time.now
+      elsif @run.pausing?
+        @run.status = :paused
+        @task.run_callbacks(:pause)
       else
-        @run.status = @run.pausing? ? :paused : :interrupted
-        @run.cursor = cursor_position
+        @run.status = :interrupted
+        @task.run_callbacks(:interrupt)
       end
 
+      @run.cursor = cursor_position
       @ticker.persist
     end
 
@@ -163,7 +171,15 @@ module MaintenanceTasks
         task_context = {}
       end
       errored_element = @errored_element if defined?(@errored_element)
+      run_error_callback
+    ensure
       MaintenanceTasks.error_handler.call(error, task_context, errored_element)
     end
+
+    def run_error_callback
+      @task.run_callbacks(:error) if defined?(@task)
+    rescue
+      nil
+    end
   end
 end

data/app/models/maintenance_tasks/application_record.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # Base class for all records used by this engine.
   #

data/app/models/maintenance_tasks/{csv_collection.rb → csv_collection_builder.rb}

@@ -3,22 +3,16 @@
 require "csv"
 
 module MaintenanceTasks
-  # Module that is included into Task classes by Task.csv_collection for
-  # processing CSV files.
+  # Strategy for building a Task that processes CSV files.
   #
   # @api private
-  module CsvCollection
-    # The contents of a CSV file to be processed by a Task.
-    #
-    # @return [String] the content of the CSV file to process.
-    attr_accessor :csv_content
-
+  class CsvCollectionBuilder
     # Defines the collection to be iterated over, based on the provided CSV.
     #
     # @return [CSV] the CSV object constructed from the specified CSV content,
     #   with headers.
-    def collection
-      CSV.new(csv_content, headers: true)
+    def collection(task)
+      CSV.new(task.csv_content, headers: true)
     end
 
     # The number of rows to be processed. Excludes the header row from the count
@@ -26,8 +20,13 @@ module MaintenanceTasks
     # an approximation based on the number of new lines.
     #
     # @return [Integer] the approximate number of rows to process.
-    def count
-      csv_content.count("\n") - 1
+    def count(task)
+      task.csv_content.count("\n") - 1
+    end
+
+    # Return that the Task processes CSV content.
+    def has_csv_content?
+      true
     end
   end
 end

data/app/models/maintenance_tasks/null_collection_builder.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module MaintenanceTasks
+  # Base strategy for building a collection-based Task to be performed.
+  class NullCollectionBuilder
+    # Placeholder method to raise in case a subclass fails to implement the
+    # expected instance method.
+    #
+    # @raise [NotImplementedError] with a message advising subclasses to
+    #   implement an override for this method.
+    def collection(task)
+      raise NoMethodError, "#{task.class.name} must implement `collection`."
+    end
+
+    # Total count of iterations to be performed.
+    #
+    # Tasks override this method to define the total amount of iterations
+    # expected at the start of the run. Return +nil+ if the amount is
+    # undefined, or counting would be prohibitive for your database.
+    #
+    # @return [Integer, nil]
+    def count(task)
+      :no_count
+    end
+
+    # Return that the Task does not process CSV content.
+    def has_csv_content?
+      false
+    end
+  end
+end

data/app/models/maintenance_tasks/progress.rb

@@ -50,14 +50,19 @@ module MaintenanceTasks
     def text
       count = @run.tick_count
       total = @run.tick_total
+
       if !total?
-        "Processed #{pluralize(count, "item")}."
+        "Processed #{number_to_delimited(count)} "\
+          "#{"item".pluralize(count)}."
       elsif over_total?
-        "Processed #{pluralize(count, "item")} (expected #{total})."
+        "Processed #{number_to_delimited(count)} "\
+          "#{"item".pluralize(count)} "\
+          "(expected #{number_to_delimited(total)})."
       else
         percentage = 100.0 * count / total
 
-        "Processed #{count} out of #{pluralize(total, "item")} "\
+        "Processed #{number_to_delimited(count)} out of "\
+          "#{number_to_delimited(total)} #{"item".pluralize(total)} "\
           "(#{number_to_percentage(percentage, precision: 0)})."
       end
     end

data/app/models/maintenance_tasks/run.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # Model that persists information related to a task being run from the UI.
   #
@@ -25,6 +26,10 @@ module MaintenanceTasks
       :cancelling,
       :interrupted,
     ]
+    STOPPING_STATUSES = [
+      :pausing,
+      :cancelling,
+    ]
     COMPLETED_STATUSES = [:succeeded, :errored, :cancelled]
     COMPLETED_RUNS_LIMIT = 10
     STUCK_TASK_TIMEOUT = 5.minutes
@@ -87,6 +92,7 @@ module MaintenanceTasks
     #
     # @param error [StandardError] the Error being persisted.
     def persist_error(error)
+      self.started_at ||= Time.now
       update!(
         status: :errored,
         error_class: error.class.to_s,
@@ -103,8 +109,8 @@ module MaintenanceTasks
     #
     # @return [MaintenanceTasks::Run] the Run record with its updated status.
     def reload_status
-      updated_status = Run.uncached do
-        Run.where(id: id).pluck(:status).first
+      updated_status = self.class.uncached do
+        self.class.where(id: id).pluck(:status).first
       end
       self.status = updated_status
       clear_attribute_changes([:status])
@@ -116,7 +122,7 @@ module MaintenanceTasks
     #
     # @return [Boolean] whether the Run is stopping.
     def stopping?
-      pausing? || cancelling?
+      STOPPING_STATUSES.include?(status.to_sym)
     end
 
     # Returns whether the Run is stopped, which is defined as having a status of
@@ -167,6 +173,21 @@ module MaintenanceTasks
       seconds_to_finished.seconds
     end
 
+    # Mark a Run as running.
+    #
+    # If the run is stopping already, it will not transition to running.
+    def running
+      return if stopping?
+      updated = self.class.where(id: id).where.not(status: STOPPING_STATUSES)
+        .update_all(status: :running, updated_at: Time.now) > 0
+      if updated
+        self.status = :running
+        clear_attribute_changes([:status])
+      else
+        reload_status
+      end
+    end
+
     # Cancels a Run.
     #
     # If the Run is paused, it will transition directly to cancelled, since the
@@ -200,9 +221,9 @@ module MaintenanceTasks
     # should not have an attachment to be valid. The appropriate error is added
     # if the Run does not meet the above criteria.
     def csv_attachment_presence
-      if Task.named(task_name) < CsvCollection && !csv_file.attached?
+      if Task.named(task_name).has_csv_content? && !csv_file.attached?
         errors.add(:csv_file, "must be attached to CSV Task.")
-      elsif !(Task.named(task_name) < CsvCollection) && csv_file.present?
+      elsif !Task.named(task_name).has_csv_content? && csv_file.present?
         errors.add(:csv_file, "should not be attached to non-CSV Task.")
       end
     rescue Task::NotFoundError

data/app/models/maintenance_tasks/runner.rb

@@ -47,11 +47,14 @@ module MaintenanceTasks
     #   creating the Run.
     # @raise [ActiveRecord::ValueTooLong] if the creation of the Run fails due
     #   to a value being too long for the column type.
-    def run(name:, csv_file: nil, arguments: {})
-      run = Run.active.find_by(task_name: name) ||
-        Run.new(task_name: name, arguments: arguments)
-      run.csv_file.attach(csv_file) if csv_file
-      job = MaintenanceTasks.job.constantize.new(run)
+    def run(name:, csv_file: nil, arguments: {}, run_model: Run)
+      run = run_model.active.find_by(task_name: name) ||
+        run_model.new(task_name: name, arguments: arguments)
+      if csv_file
+        run.csv_file.attach(csv_file)
+        run.csv_file.filename = filename(name)
+      end
+      job = instantiate_job(run)
       run.job_id = job.job_id
       yield run if block_given?
       run.enqueued!
@@ -70,5 +73,14 @@ module MaintenanceTasks
       run.persist_error(error)
       raise EnqueuingError, run
     end
+
+    def filename(task_name)
+      formatted_task_name = task_name.underscore.gsub("/", "_")
+      "#{Time.now.utc.strftime("%Y%m%dT%H%M%SZ")}_#{formatted_task_name}.csv"
+    end
+
+    def instantiate_job(run)
+      MaintenanceTasks.job.constantize.new(run)
+    end
   end
 end

data/app/models/maintenance_tasks/runs_page.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # This class is responsible for handling cursor-based pagination for Run
   # records.

data/app/{tasks → models}/maintenance_tasks/task.rb

@@ -1,8 +1,10 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # Base class that is inherited by the host application's task classes.
   class Task
     extend ActiveSupport::DescendantsTracker
+    include ActiveSupport::Callbacks
     include ActiveModel::Attributes
     include ActiveModel::AttributeAssignment
     include ActiveModel::Validations
@@ -16,6 +18,11 @@ module MaintenanceTasks
     # @api private
     class_attribute :throttle_conditions, default: []
 
+    class_attribute :collection_builder_strategy,
+      default: NullCollectionBuilder.new
+
+    define_callbacks :start, :complete, :error, :cancel, :pause, :interrupt
+
     class << self
       # Finds a Task with the given name.
       #
@@ -47,11 +54,19 @@ module MaintenanceTasks
       # An input to upload a CSV will be added in the form to start a Run. The
       # collection and count method are implemented.
       def csv_collection
-        if !defined?(ActiveStorage) || !ActiveStorage::Attachment.table_exists?
+        unless defined?(ActiveStorage)
           raise NotImplementedError, "Active Storage needs to be installed\n"\
             "To resolve this issue run: bin/rails active_storage:install"
         end
-        include(CsvCollection)
+
+        self.collection_builder_strategy = CsvCollectionBuilder.new
+      end
+
+      # Returns whether the Task handles CSV.
+      #
+      # @return [Boolean] whether the Task handles CSV.
+      def has_csv_content?
+        collection_builder_strategy.has_csv_content?
       end
 
       # Processes one item.
@@ -94,6 +109,54 @@ module MaintenanceTasks
         ]
       end
 
+      # Initialize a callback to run after the task starts.
+      #
+      # @param filter_list apply filters to the callback
+      #   (see https://api.rubyonrails.org/classes/ActiveSupport/Callbacks/ClassMethods.html#method-i-set_callback)
+      def after_start(*filter_list, &block)
+        set_callback(:start, :after, *filter_list, &block)
+      end
+
+      # Initialize a callback to run after the task completes.
+      #
+      # @param filter_list apply filters to the callback
+      #   (see https://api.rubyonrails.org/classes/ActiveSupport/Callbacks/ClassMethods.html#method-i-set_callback)
+      def after_complete(*filter_list, &block)
+        set_callback(:complete, :after, *filter_list, &block)
+      end
+
+      # Initialize a callback to run after the task pauses.
+      #
+      # @param filter_list apply filters to the callback
+      #   (see https://api.rubyonrails.org/classes/ActiveSupport/Callbacks/ClassMethods.html#method-i-set_callback)
+      def after_pause(*filter_list, &block)
+        set_callback(:pause, :after, *filter_list, &block)
+      end
+
+      # Initialize a callback to run after the task is interrupted.
+      #
+      # @param filter_list apply filters to the callback
+      #   (see https://api.rubyonrails.org/classes/ActiveSupport/Callbacks/ClassMethods.html#method-i-set_callback)
+      def after_interrupt(*filter_list, &block)
+        set_callback(:interrupt, :after, *filter_list, &block)
+      end
+
+      # Initialize a callback to run after the task is cancelled.
+      #
+      # @param filter_list apply filters to the callback
+      #   (see https://api.rubyonrails.org/classes/ActiveSupport/Callbacks/ClassMethods.html#method-i-set_callback)
+      def after_cancel(*filter_list, &block)
+        set_callback(:cancel, :after, *filter_list, &block)
+      end
+
+      # Initialize a callback to run after the task produces an error.
+      #
+      # @param filter_list apply filters to the callback
+      #   (see https://api.rubyonrails.org/classes/ActiveSupport/Callbacks/ClassMethods.html#method-i-set_callback)
+      def after_error(*filter_list, &block)
+        set_callback(:error, :after, *filter_list, &block)
+      end
+
       private
 
       def load_constants
@@ -103,13 +166,36 @@ module MaintenanceTasks
       end
     end
 
-    # Placeholder method to raise in case a subclass fails to implement the
-    # expected instance method.
+    # The contents of a CSV file to be processed by a Task.
     #
-    # @raise [NotImplementedError] with a message advising subclasses to
-    #   implement an override for this method.
+    # @return [String] the content of the CSV file to process.
+    def csv_content
+      raise NoMethodError unless has_csv_content?
+
+      @csv_content
+    end
+
+    # Set the contents of a CSV file to be processed by a Task.
+    #
+    # @param csv_content [String] the content of the CSV file to process.
+    def csv_content=(csv_content)
+      raise NoMethodError unless has_csv_content?
+
+      @csv_content = csv_content
+    end
+
+    # Returns whether the Task handles CSV.
+    #
+    # @return [Boolean] whether the Task handles CSV.
+    def has_csv_content?
+      self.class.has_csv_content?
+    end
+
+    # The collection to be processed, delegated to the strategy.
+    #
+    # @return the collection.
     def collection
-      raise NoMethodError, "#{self.class.name} must implement `collection`."
+      self.class.collection_builder_strategy.collection(self)
     end
 
     # Placeholder method to raise in case a subclass fails to implement the
@@ -123,15 +209,11 @@ module MaintenanceTasks
       raise NoMethodError, "#{self.class.name} must implement `process`."
     end
 
-    # Total count of iterations to be performed.
-    #
-    # Tasks override this method to define the total amount of iterations
-    # expected at the start of the run. Return +nil+ if the amount is
-    # undefined, or counting would be prohibitive for your database.
+    # Total count of iterations to be performed, delegated to the strategy.
     #
     # @return [Integer, nil]
     def count
-      :no_count
+      self.class.collection_builder_strategy.count(self)
     end
   end
 end

data/app/models/maintenance_tasks/task_data.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # Class that represents the data related to a Task. Such information can be
   # sourced from a Task or from existing Run records for a Task that was since
@@ -129,7 +130,7 @@ module MaintenanceTasks
 
     # @return [Boolean] whether the Task inherits from CsvTask.
     def csv_task?
-      !deleted? && Task.named(name) < CsvCollection
+      !deleted? && Task.named(name).has_csv_content?
     end
 
     # @return [Array<String>] the names of parameters the Task accepts.

data/app/validators/maintenance_tasks/run_status_validator.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # Custom validator class responsible for ensuring that transitions between
   # Run statuses are valid.

data/config/routes.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 MaintenanceTasks::Engine.routes.draw do
   resources :tasks, only: [:index, :show], format: false do
     member do

data/db/migrate/20201211151756_create_maintenance_tasks_runs.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 class CreateMaintenanceTasksRuns < ActiveRecord::Migration[6.0]
   def change
     create_table(:maintenance_tasks_runs) do |t|

data/db/migrate/20210225152418_remove_index_on_task_name.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 class RemoveIndexOnTaskName < ActiveRecord::Migration[6.0]
   def up
     change_table(:maintenance_tasks_runs) do |t|

data/db/migrate/20210517131953_add_arguments_to_maintenance_tasks_runs.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 class AddArgumentsToMaintenanceTasksRuns < ActiveRecord::Migration[6.0]
   def change
     add_column(:maintenance_tasks_runs, :arguments, :text)

data/lib/generators/maintenance_tasks/install_generator.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module MaintenanceTasks
   # Generator used to set up the engine in the host application. It handles
   # mounting the engine and installing migrations.

data/lib/generators/maintenance_tasks/templates/task.rb.tt

@@ -9,7 +9,9 @@ module <%= tasks_module %>
     end
 
     def process(element)
-      # The work to be done in a single iteration of the task
+      # The work to be done in a single iteration of the task.
+      # This should be idempotent, as the same element may be processed more
+      # than once if the task is interrupted and resumed.
     end
 
     def count

data/lib/maintenance_tasks/cli.rb

@@ -25,12 +25,13 @@ module MaintenanceTasks
     LONGDESC
 
     # Specify the CSV file to process for CSV Tasks
-    option :csv, desc: "Supply a CSV file to be processed by a CSV Task, "\
-      '--csv "path/to/csv/file.csv"'
-
+    desc = "Supply a CSV file to be processed by a CSV Task, "\
+      "--csv path/to/csv/file.csv"
+    option :csv, desc: desc
     # Specify arguments to supply to a Task supporting parameters
-    option :arguments, type: :hash, desc: "Supply arguments for a Task that "\
-      "accepts parameters as a set of <key>:<value> pairs."
+    desc = "Supply arguments for a Task that accepts parameters as a set of "\
+      "<key>:<value> pairs."
+    option :arguments, type: :hash, desc: desc
 
     # Command to run a Task.
     #

data/lib/maintenance_tasks/engine.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 require "active_record/railtie"
 
 module MaintenanceTasks

data/lib/maintenance_tasks.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 require "action_controller"
 require "action_view"
 require "active_job"
@@ -73,7 +74,7 @@ module MaintenanceTasks
     unless error_handler.arity == 3
       ActiveSupport::Deprecation.warn(
         "MaintenanceTasks.error_handler should be a lambda that takes three "\
-         "arguments: error, task_context, and errored_element."
+          "arguments: error, task_context, and errored_element."
       )
       @error_handler = ->(error, _task_context, _errored_element) do
         error_handler.call(error)

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: maintenance_tasks
 version: !ruby/object:Gem::Version
-  version: 1.5.0
+  version: 1.6.0
 platform: ruby
 authors:
 - Shopify Engineering
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-07-28 00:00:00.000000000 Z
+date: 2021-11-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: actionpack
@@ -97,14 +97,15 @@ files:
 - app/jobs/concerns/maintenance_tasks/task_job_concern.rb
 - app/jobs/maintenance_tasks/task_job.rb
 - app/models/maintenance_tasks/application_record.rb
-- app/models/maintenance_tasks/csv_collection.rb
+- app/models/maintenance_tasks/csv_collection_builder.rb
+- app/models/maintenance_tasks/null_collection_builder.rb
 - app/models/maintenance_tasks/progress.rb
 - app/models/maintenance_tasks/run.rb
 - app/models/maintenance_tasks/runner.rb
 - app/models/maintenance_tasks/runs_page.rb
+- app/models/maintenance_tasks/task.rb
 - app/models/maintenance_tasks/task_data.rb
 - app/models/maintenance_tasks/ticker.rb
-- app/tasks/maintenance_tasks/task.rb
 - app/validators/maintenance_tasks/run_status_validator.rb
 - app/views/layouts/maintenance_tasks/_navbar.html.erb
 - app/views/layouts/maintenance_tasks/application.html.erb
@@ -143,7 +144,7 @@ homepage: https://github.com/Shopify/maintenance_tasks
 licenses:
 - MIT
 metadata:
-  source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v1.5.0
+  source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v1.6.0
   allowed_push_host: https://rubygems.org
 post_install_message: 
 rdoc_options: []