maintenance_tasks 2.3.3 → 2.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1f292dbc04fbbb71588c0be3af0bc95bfb5de5ca27f9aa7562b681cad39c6241
-  data.tar.gz: d631953126ffbd114721fea778fc8d43f5c86c20461c04943c3127a1b6f9e0d3
+  metadata.gz: 1bca60024506654676fc6c094e6c4a4a1af4c87fc3a80c81f5a7be58a73b6061
+  data.tar.gz: 351ba1281e24013a2cbdd43302498769d398e38f0a695850d988d1d4aa4987bb
 SHA512:
-  metadata.gz: 146510c2a63f74b6084b43f9fa939c017e242b79ae0c453008b17b8e5c5e2332bd0d972ab4d8dce4d9d365836bcada8df8a7f1a7d717dc1d733232edd0830a4e
-  data.tar.gz: d1df2c04b6274843b32e4feb44a4bfb1eec3e64717ae67d78bbb440c08c34f24f63882d880dfc602529b2971f9bbfc9ce74345b848ecd3f780d1ab9270a610ec
+  metadata.gz: c4cb3cfe2d3ddeb5f614976f6b2c04ce655e3b4acdf739ea133e5a303a224e170e3f6b9ea0aa58c32b71e72f30017524c172750dd86308a29bcd46d309cb45ed
+  data.tar.gz: 1c990f1658545d4bd5e622598c5ce7a08f0dbdda9e1d8c9e0bd56c46d491fe5a440ce53100c976d196b7cc2535a839e16253a278d6e9e1ae1dc6a1121e25848e

data/README.md

@@ -5,35 +5,35 @@ A Rails engine for queuing and managing maintenance tasks.
 By ”maintenance task”, this project means a data migration, i.e. code that
 changes data in the database, often to support schema migrations. For example,
 in order to introduce a new `NOT NULL` column, it has to be added as nullable
-first, backfilled with values, before finally being changed to `NOT NULL`.
-This engine helps with the second part of this process, backfilling.
+first, backfilled with values, before finally being changed to `NOT NULL`. This
+engine helps with the second part of this process, backfilling.
 
-Maintenance tasks are collection-based tasks, usually using Active Record,
-that update the data in your database. They can be paused or interrupted.
-Maintenance tasks can operate [in batches](#processing-batch-collections) and
-use [throttling](#throttling) to control the load on your database.
+Maintenance tasks are collection-based tasks, usually using Active Record, that
+update the data in your database. They can be paused or interrupted. Maintenance
+tasks can operate [in batches](#processing-batch-collections) and use
+[throttling](#throttling) to control the load on your database.
 
 Maintenance tasks aren't meant to happen on a regular basis. They're used as
 needed, or as one-offs. Normally maintenance tasks are ephemeral, so they are
 used briefly and then deleted.
 
-The Rails engine has a web-based UI for listing maintenance tasks, seeing
-their status, and starting, pausing and restarting them.
+The Rails engine has a web-based UI for listing maintenance tasks, seeing their
+status, and starting, pausing and restarting them.
 
 [![Link to demo video](static/demo.png)](https://www.youtube.com/watch?v=BTuvTQxlFzs)
 
 ## Should I Use Maintenance Tasks?
 
-Maintenance tasks have a limited, specific job UI. While the engine can be
-used to provide a user interface for other data changes, such as data changes
-for support requests, we recommend you use regular application code for those
-use cases instead. These inevitably require more flexibility than this engine
-will be able to provide.
+Maintenance tasks have a limited, specific job UI. While the engine can be used
+to provide a user interface for other data changes, such as data changes for
+support requests, we recommend you use regular application code for those use
+cases instead. These inevitably require more flexibility than this engine will
+be able to provide.
 
-If your task shouldn't run as an Active Job, it probably isn't a good match
-for this gem. If your task doesn't need to run in the background,
-consider a runner script instead. If your task doesn't need to be
-interruptible, consider a normal Active Job.
+If your task shouldn't run as an Active Job, it probably isn't a good match for
+this gem. If your task doesn't need to run in the background, consider a runner
+script instead. If your task doesn't need to be interruptible, consider a normal
+Active Job.
 
 Maintenance tasks can be interrupted between iterations. If your task [isn't
 collection-based](#tasks-that-dont-need-a-collection) (no CSV file or database
@@ -48,9 +48,10 @@ If your task happens regularly, consider Active Jobs with a scheduler or cron,
 [job-iteration jobs](https://github.com/shopify/job-iteration) and/or [custom
 rails_admin UIs][rails-admin-engines] instead of the Maintenance Tasks gem.
 Maintenance tasks should be ephemeral, to suit their intentionally limited UI.
+They should not repeat.
 
-To create seed data for a new application, use the provided Rails
-`db/seeds.rb` file instead.
+To create seed data for a new application, use the provided Rails `db/seeds.rb`
+file instead.
 
 If your application can't handle a half-completed migration, maintenance tasks
 are probably the wrong tool. Remember that maintenance tasks are intentionally
@@ -99,7 +100,8 @@ constants](https://guides.rubyonrails.org/autoloading_and_reloading_constants.ht
 
 The typical Maintenance Tasks workflow is as follows:
 
-1. [Generate a class describing the Task](#creating-a-task) and the work to be done.
+1. [Generate a class describing the Task](#creating-a-task) and the work to be
+   done.
 2. Run the Task
     - either by [using the included web UI](#running-a-task-from-the-web-ui),
     - or by [using the command line](#running-a-task-from-the-command-line),
@@ -191,9 +193,9 @@ title,content
 My Title,Hello World!
 ```
 
-The files uploaded to your Active Storage service provider will be renamed
-to include an ISO 8601 timestamp and the Task name in snake case format.
-The CSV is expected to have a trailing newline at the end of the file.
+The files uploaded to your Active Storage service provider will be renamed to
+include an ISO 8601 timestamp and the Task name in snake case format. The CSV is
+expected to have a trailing newline at the end of the file.
 
 #### Batch CSV Tasks
 
@@ -270,8 +272,8 @@ inside `#process`.
 ### Tasks that don’t need a Collection
 
 Sometimes, you might want to run a Task that performs a single operation, such
-as enqueuing another background job or querying an external API. The gem supports
-collection-less tasks.
+as enqueuing another background job or querying an external API. The gem
+supports collection-less tasks.
 
 Generate a collection-less Task by running:
 
@@ -409,10 +411,9 @@ module Maintenance
 end
 ```
 
-Note: The `after_error` callback is guaranteed to complete,
-so any exceptions raised in your callback code are ignored.
-If your `after_error` callback code can raise an exception,
-you’ll need to rescue it and handle it appropriately
+Note: The `after_error` callback is guaranteed to complete, so any exceptions
+raised in your callback code are ignored. If your `after_error` callback code
+can raise an exception, you’ll need to rescue it and handle it appropriately
 within the callback.
 
 ```ruby
@@ -430,9 +431,8 @@ module Maintenance
 end
 ```
 
-If any of the other callbacks cause an exception,
-it will be handled by the error handler,
-and will cause the task to stop running.
+If any of the other callbacks cause an exception, it will be handled by the
+error handler, and will cause the task to stop running.
 
 Callback behaviour can be shared across all tasks using an initializer.
 
@@ -474,14 +474,14 @@ depend on the queue adapter but in general, you should follow these rules:
 When the Task runs or resumes, the Runner enqueues a job, which processes the
 Task. That job will instantiate a Task object which will live for the duration
 of the job. The first time the job runs, it will call `count`. Every time a job
-runs, it will call `collection` on the Task object, and then `process`
-for each item in the collection, until the job stops. The job stops when either the
+runs, it will call `collection` on the Task object, and then `process` for each
+item in the collection, until the job stops. The job stops when either the
 collection is finished processing or after the maximum job runtime has expired.
 
 This means memoization can be misleading within `process`, since the memoized
 values will be available for subsequent calls to `process` within the same job.
-Still, memoization can be used for throttling or reporting, and you can use [Task
-callbacks](#using-task-callbacks) to persist or log a report for example.
+Still, memoization can be used for throttling or reporting, and you can use
+[Task callbacks](#using-task-callbacks) to persist or log a report for example.
 
 ### Writing tests for a Task
 
@@ -559,7 +559,7 @@ module Maintenance
 
     test "#process performs a task iteration" do
       assert_difference -> { Post.first.content } do
-        task.process(Post.first)
+        @task.process(Post.first)
       end
     end
   end
@@ -671,7 +671,7 @@ tweaked in an initializer if necessary.
 [max-job-runtime]: https://github.com/Shopify/job-iteration/blob/-/guides/best-practices.md#max-job-runtime
 
 Running tasks will also be interrupted and re-enqueued when needed. For example
-[when Sidekiq workers shuts down for a deploy][sidekiq-deploy]:
+[when Sidekiq workers shut down for a deploy][sidekiq-deploy]:
 
 [sidekiq-deploy]: https://github.com/mperham/sidekiq/wiki/Deployment
 
@@ -684,19 +684,24 @@ Running tasks will also be interrupted and re-enqueued when needed. For example
 When Sidekiq is stopping, it will give workers 25 seconds to finish before
 forcefully terminating them (this is the default but can be configured with the
 `--timeout` option). Before the worker threads are terminated, Sidekiq will try
-to re-enqueue the job so your Task will be resumed. However, the position in
-the collection won’t be persisted so at least one iteration may run again.
+to re-enqueue the job so your Task will be resumed. However, the position in the
+collection won’t be persisted so at least one iteration may run again.
+
+Job queues other than Sidekiq may handle this in different ways.
 
 #### Help! My Task is stuck
 
-Finally, if the queue adapter configured for your application doesn’t have this
-property, or if Sidekiq crashes, is forcefully terminated, or is unable to
-re-enqueue the jobs that were in progress, the Task may be in a seemingly stuck
-situation where it appears to be running but is not. In that situation, pausing
-or cancelling it will not result in the Task being paused or cancelled, as the
-Task will get stuck in a state of `pausing` or `cancelling`. As a work-around,
-if a Task is `cancelling` for more than 5 minutes, you can cancel it again.
-It will then be marked as fully cancelled, allowing you to run it again.
+If the queue adapter configured for your application doesn’t have this property,
+or if Sidekiq crashes, is forcefully terminated, or is unable to re-enqueue the
+jobs that were in progress, the Task may be in a seemingly stuck situation where
+it appears to be running but is not. In that situation, pausing or cancelling it
+will not result in the Task being paused or cancelled, as the Task will get
+stuck in a state of `pausing` or `cancelling`. As a work-around, if a Task is
+`cancelling` for more than 5 minutes, you can cancel it again. It will then be
+marked as fully cancelled, allowing you to run it again.
+
+If you are stuck in `pausing` and wish to preserve your tasks's position
+(instead of cancelling and rerunning), you may click "Force pause".
 
 ### Configuring the gem
 
@@ -757,9 +762,10 @@ If no value is specified, it will default to `Maintenance`.
 
 #### Organizing tasks using namespaces
 
-Tasks may be nested arbitrarily deeply under `app/tasks/maintenance`, for example given a
-task file `app/tasks/maintenance/team_name/service_name/update_posts_task.rb` we
-can define the task as:
+Tasks may be nested arbitrarily deeply under `app/tasks/maintenance`, for
+example given a task file
+`app/tasks/maintenance/team_name/service_name/update_posts_task.rb` we can
+define the task as:
 
 ```ruby
 module Maintenance
@@ -848,8 +854,8 @@ default.
 #### Customizing the backtrace cleaner
 
 `MaintenanceTasks.backtrace_cleaner` can be configured to specify a backtrace
-cleaner to use when a Task errors and the backtrace is cleaned and persisted.
-An `ActiveSupport::BacktraceCleaner` should be used.
+cleaner to use when a Task errors and the backtrace is cleaned and persisted. An
+`ActiveSupport::BacktraceCleaner` should be used.
 
 ```ruby
 # config/initializers/maintenance_tasks.rb
@@ -865,8 +871,8 @@ clean backtraces.
 
 #### Customizing the parent controller for the web UI
 
-`MaintenanceTasks.parent_controller` can be configured to specify a controller class for all of the web UI engine's
-controllers to inherit from.
+`MaintenanceTasks.parent_controller` can be configured to specify a controller
+class for all of the web UI engine's controllers to inherit from.
 
 This allows applications with common logic in their `ApplicationController` (or
 any other controller) to optionally configure the web UI to inherit that logic
@@ -893,9 +899,10 @@ If no value is specified, it will default to `"ActionController::Base"`.
 
 ### Metadata
 
-`MaintenanceTasks.metadata` can be configured to specify a proc from which to get extra information about the run.
-Since this proc will be ran in the context of the `MaintenanceTasks.parent_controller`, it can be used to keep the id
-or email of the user who performed the maintenance task.
+`MaintenanceTasks.metadata` can be configured to specify a proc from which to
+get extra information about the run. Since this proc will be ran in the context
+of the `MaintenanceTasks.parent_controller`, it can be used to keep the id or
+email of the user who performed the maintenance task.
 
 ```ruby
 # config/initializers/maintenance_tasks.rb
@@ -915,7 +922,7 @@ bin/rails generate maintenance_tasks:install
 
 This ensures that new migrations are installed and run as well.
 
-**What if I’ve deleted my previous Maintenance Task migrations?**
+### What if I’ve deleted my previous Maintenance Task migrations?
 
 The install command will attempt to reinstall these old migrations and migrating
 the database will cause problems. Use `bin/rails

data/app/controllers/maintenance_tasks/runs_controller.rb

@@ -29,7 +29,7 @@ module MaintenanceTasks
 
     # Updates a Run status to paused.
     def pause
-      @run.pausing!
+      @run.pause
       redirect_to(task_path(@run.task_name))
     rescue ActiveRecord::RecordInvalid => error
       redirect_to(task_path(@run.task_name), alert: error.message)

data/app/models/maintenance_tasks/run.rb

@@ -320,21 +320,29 @@ module MaintenanceTasks
 
     # Marks a Run as pausing.
     #
+    # If the Run has been stuck on pausing for more than 5 minutes, it forces
+    # the transition to paused. The ended_at timestamp will be updated.
+    #
     # Rescues and retries status transition if an ActiveRecord::StaleObjectError
     # is encountered.
-    def pausing!
-      super
+    def pause
+      if stuck?
+        self.status = :paused
+        persist_transition
+      else
+        pausing!
+      end
     rescue ActiveRecord::StaleObjectError
       reload_status
       retry
     end
 
     # Returns whether a Run is stuck, which is defined as having a status of
-    # cancelling, and not having been updated in the last 5 minutes.
+    # cancelling or pausing, and not having been updated in the last 5 minutes.
     #
     # @return [Boolean] whether the Run is stuck.
     def stuck?
-      cancelling? && updated_at <= STUCK_TASK_TIMEOUT.ago
+      (cancelling? || pausing?) && updated_at <= STUCK_TASK_TIMEOUT.ago
     end
 
     # Performs validation on the task_name attribute.

data/app/models/maintenance_tasks/task.rb

@@ -187,13 +187,7 @@ module MaintenanceTasks
         namespace = MaintenanceTasks.tasks_module.safe_constantize
         return unless namespace
 
-        load_const = lambda do |root|
-          root.constants.each do |name|
-            object = root.const_get(name)
-            load_const.call(object) if object.instance_of?(Module)
-          end
-        end
-        load_const.call(namespace)
+        Rails.autoloaders.main.eager_load_namespace(namespace)
       end
     end
 

data/app/views/maintenance_tasks/runs/_run.html.erb

@@ -29,6 +29,9 @@
     <% elsif run.pausing? %>
       <%= button_to 'Pausing', pause_task_run_path(@task, run), method: :put, class: 'button is-warning', disabled: true %>
       <%= button_to 'Cancel', cancel_task_run_path(@task, run), method: :put, class: 'button is-danger' %>
+      <% if run.stuck? %>
+        <%= button_to 'Force pause', pause_task_run_path(@task, run), method: :put, class: 'button is-danger', disabled: @task.deleted? %>
+      <% end %>
     <% elsif run.active? %>
       <%= button_to 'Pause', pause_task_run_path(@task, run), method: :put, class: 'button is-warning', disabled: @task.deleted? %>
       <%= button_to 'Cancel', cancel_task_run_path(@task, run), method: :put, class: 'button is-danger' %>

data/lib/generators/maintenance_tasks/templates/task_spec.rb.tt

@@ -6,10 +6,14 @@ module <%= tasks_module %>
 <% module_namespacing do -%>
   RSpec.describe <%= class_name %>Task do
     describe "#process" do
+      <%- if no_collection? -%>
+      subject(:process) { described_class.process }
+      <%- else -%>
       subject(:process) { described_class.process(element) }
       let(:element) {
         # Object to be processed in a single iteration of this task
       }
+      <%- end -%>
       pending "add some examples to (or delete) #{__FILE__}"
     end
   end

data/lib/maintenance_tasks/cli.rb

@@ -15,6 +15,12 @@ module MaintenanceTasks
     end
 
     desc "perform [TASK NAME]", "Runs the given Maintenance Task"
+    long_desc <<~DESC
+      `maintenance_tasks perform` will run the Maintenance Task specified by
+      the [TASK NAME] argument.
+
+      Use `maintenance_tasks list` to get a list of all available tasks.
+    DESC
 
     # Specify the CSV file to process for CSV Tasks
     desc = "Supply a CSV file to be processed by a CSV Task, "\
@@ -41,19 +47,14 @@ module MaintenanceTasks
       say_status(:error, error.message, :red)
     end
 
-    # `long_desc` only allows us to use a static string as "long description".
-    # By redefining the `#long_description` method on the "perform" Command
-    # object instead, we make it dynamic, thus delaying the task loading
-    # process until it's actually required.
-    commands["perform"].define_singleton_method(:long_description) do
-      <<~LONGDESC
-        `maintenance_tasks perform` will run the Maintenance Task specified by
-        the [TASK NAME] argument.
-
-        Available Tasks:
+    desc "list", "Load and list all available tasks."
 
-        #{Task.load_all.map(&:name).sort.join("\n\n")}
-      LONGDESC
+    # Command to list all available Tasks.
+    #
+    # It needs to use `Task.load_all` in order to load all the tasks available
+    # in the project before displaying them.
+    def list
+      say(Task.load_all.map(&:name).sort.join("\n"))
     end
 
     private

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: maintenance_tasks
 version: !ruby/object:Gem::Version
-  version: 2.3.3
+  version: 2.4.0
 platform: ruby
 authors:
 - Shopify Engineering
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-10-11 00:00:00.000000000 Z
+date: 2023-12-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: actionpack
@@ -80,6 +80,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: zeitwerk
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 2.6.2
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 2.6.2
 description: 
 email: gems@shopify.com
 executables:
@@ -143,7 +157,6 @@ files:
 - lib/generators/maintenance_tasks/task_generator.rb
 - lib/generators/maintenance_tasks/templates/csv_task.rb.tt
 - lib/generators/maintenance_tasks/templates/no_collection_task.rb.tt
-- lib/generators/maintenance_tasks/templates/no_collection_task_test.rb.tt
 - lib/generators/maintenance_tasks/templates/task.rb.tt
 - lib/generators/maintenance_tasks/templates/task_spec.rb.tt
 - lib/generators/maintenance_tasks/templates/task_test.rb.tt
@@ -156,7 +169,7 @@ homepage: https://github.com/Shopify/maintenance_tasks
 licenses:
 - MIT
 metadata:
-  source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v2.3.3
+  source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v2.4.0
   allowed_push_host: https://rubygems.org
 post_install_message: 
 rdoc_options: []
@@ -173,7 +186,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.20
+rubygems_version: 3.4.22
 signing_key: 
 specification_version: 4
 summary: A Rails engine for queuing and managing maintenance tasks

data/lib/generators/maintenance_tasks/templates/no_collection_task_test.rb.tt

@@ -1,13 +0,0 @@
-# frozen_string_literal: true
-
-require "test_helper"
-
-module <%= tasks_module %>
-<% module_namespacing do -%>
-  class <%= class_name %>TaskTest < ActiveSupport::TestCase
-    # test "#process performs a task iteration" do
-    #   <%= tasks_module %>::<%= class_name %>Task.process
-    # end
-  end
-<% end -%>
-end