maintenance_tasks 2.3.2 → 2.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +121 -55
- data/app/controllers/maintenance_tasks/runs_controller.rb +2 -2
- data/app/models/maintenance_tasks/run.rb +12 -4
- data/app/models/maintenance_tasks/task.rb +1 -7
- data/app/views/maintenance_tasks/runs/_run.html.erb +3 -0
- data/app/views/maintenance_tasks/tasks/show.html.erb +1 -1
- data/lib/generators/maintenance_tasks/templates/task_spec.rb.tt +4 -0
- data/lib/maintenance_tasks/cli.rb +13 -12
- metadata +18 -5
- data/lib/generators/maintenance_tasks/templates/no_collection_task_test.rb.tt +0 -13
    
        checksums.yaml
    CHANGED
    
    | @@ -1,7 +1,7 @@ | |
| 1 1 | 
             
            ---
         | 
| 2 2 | 
             
            SHA256:
         | 
| 3 | 
            -
              metadata.gz:  | 
| 4 | 
            -
              data.tar.gz:  | 
| 3 | 
            +
              metadata.gz: 1bca60024506654676fc6c094e6c4a4a1af4c87fc3a80c81f5a7be58a73b6061
         | 
| 4 | 
            +
              data.tar.gz: 351ba1281e24013a2cbdd43302498769d398e38f0a695850d988d1d4aa4987bb
         | 
| 5 5 | 
             
            SHA512:
         | 
| 6 | 
            -
              metadata.gz:  | 
| 7 | 
            -
              data.tar.gz:  | 
| 6 | 
            +
              metadata.gz: c4cb3cfe2d3ddeb5f614976f6b2c04ce655e3b4acdf739ea133e5a303a224e170e3f6b9ea0aa58c32b71e72f30017524c172750dd86308a29bcd46d309cb45ed
         | 
| 7 | 
            +
              data.tar.gz: 1c990f1658545d4bd5e622598c5ce7a08f0dbdda9e1d8c9e0bd56c46d491fe5a440ce53100c976d196b7cc2535a839e16253a278d6e9e1ae1dc6a1121e25848e
         | 
    
        data/README.md
    CHANGED
    
    | @@ -1,9 +1,64 @@ | |
| 1 | 
            -
            #  | 
| 1 | 
            +
            # Maintenance Tasks
         | 
| 2 2 |  | 
| 3 3 | 
             
            A Rails engine for queuing and managing maintenance tasks.
         | 
| 4 4 |  | 
| 5 | 
            +
            By ”maintenance task”, this project means a data migration, i.e. code that
         | 
| 6 | 
            +
            changes data in the database, often to support schema migrations. For example,
         | 
| 7 | 
            +
            in order to introduce a new `NOT NULL` column, it has to be added as nullable
         | 
| 8 | 
            +
            first, backfilled with values, before finally being changed to `NOT NULL`. This
         | 
| 9 | 
            +
            engine helps with the second part of this process, backfilling.
         | 
| 10 | 
            +
             | 
| 11 | 
            +
            Maintenance tasks are collection-based tasks, usually using Active Record, that
         | 
| 12 | 
            +
            update the data in your database. They can be paused or interrupted. Maintenance
         | 
| 13 | 
            +
            tasks can operate [in batches](#processing-batch-collections) and use
         | 
| 14 | 
            +
            [throttling](#throttling) to control the load on your database.
         | 
| 15 | 
            +
             | 
| 16 | 
            +
            Maintenance tasks aren't meant to happen on a regular basis. They're used as
         | 
| 17 | 
            +
            needed, or as one-offs. Normally maintenance tasks are ephemeral, so they are
         | 
| 18 | 
            +
            used briefly and then deleted.
         | 
| 19 | 
            +
             | 
| 20 | 
            +
            The Rails engine has a web-based UI for listing maintenance tasks, seeing their
         | 
| 21 | 
            +
            status, and starting, pausing and restarting them.
         | 
| 22 | 
            +
             | 
| 5 23 | 
             
            [](https://www.youtube.com/watch?v=BTuvTQxlFzs)
         | 
| 6 24 |  | 
| 25 | 
            +
            ## Should I Use Maintenance Tasks?
         | 
| 26 | 
            +
             | 
| 27 | 
            +
            Maintenance tasks have a limited, specific job UI. While the engine can be used
         | 
| 28 | 
            +
            to provide a user interface for other data changes, such as data changes for
         | 
| 29 | 
            +
            support requests, we recommend you use regular application code for those use
         | 
| 30 | 
            +
            cases instead. These inevitably require more flexibility than this engine will
         | 
| 31 | 
            +
            be able to provide.
         | 
| 32 | 
            +
             | 
| 33 | 
            +
            If your task shouldn't run as an Active Job, it probably isn't a good match for
         | 
| 34 | 
            +
            this gem. If your task doesn't need to run in the background, consider a runner
         | 
| 35 | 
            +
            script instead. If your task doesn't need to be interruptible, consider a normal
         | 
| 36 | 
            +
            Active Job.
         | 
| 37 | 
            +
             | 
| 38 | 
            +
            Maintenance tasks can be interrupted between iterations. If your task [isn't
         | 
| 39 | 
            +
            collection-based](#tasks-that-dont-need-a-collection) (no CSV file or database
         | 
| 40 | 
            +
            table) or has very large batches, it will get limited benefit from throttling
         | 
| 41 | 
            +
            (pausing between iterations) or interrupting. This might be fine, or the added
         | 
| 42 | 
            +
            complexity of maintenance Tasks over normal Active Jobs may not be worthwhile.
         | 
| 43 | 
            +
             | 
| 44 | 
            +
            If your task updates your database schema instead of data, use a migration
         | 
| 45 | 
            +
            instead of a maintenance task.
         | 
| 46 | 
            +
             | 
| 47 | 
            +
            If your task happens regularly, consider Active Jobs with a scheduler or cron,
         | 
| 48 | 
            +
            [job-iteration jobs](https://github.com/shopify/job-iteration) and/or [custom
         | 
| 49 | 
            +
            rails_admin UIs][rails-admin-engines] instead of the Maintenance Tasks gem.
         | 
| 50 | 
            +
            Maintenance tasks should be ephemeral, to suit their intentionally limited UI.
         | 
| 51 | 
            +
            They should not repeat.
         | 
| 52 | 
            +
             | 
| 53 | 
            +
            To create seed data for a new application, use the provided Rails `db/seeds.rb`
         | 
| 54 | 
            +
            file instead.
         | 
| 55 | 
            +
             | 
| 56 | 
            +
            If your application can't handle a half-completed migration, maintenance tasks
         | 
| 57 | 
            +
            are probably the wrong tool. Remember that maintenance tasks are intentionally
         | 
| 58 | 
            +
            pausable and can be cancelled halfway.
         | 
| 59 | 
            +
             | 
| 60 | 
            +
            [rails-admin-engines]: https://www.ruby-toolbox.com/categories/rails_admin_interfaces
         | 
| 61 | 
            +
             | 
| 7 62 | 
             
            ## Installation
         | 
| 8 63 |  | 
| 9 64 | 
             
            To install the gem and run the install generator, execute:
         | 
| @@ -45,7 +100,8 @@ constants](https://guides.rubyonrails.org/autoloading_and_reloading_constants.ht | |
| 45 100 |  | 
| 46 101 | 
             
            The typical Maintenance Tasks workflow is as follows:
         | 
| 47 102 |  | 
| 48 | 
            -
            1. [Generate a class describing the Task](#creating-a-task) and the work to be | 
| 103 | 
            +
            1. [Generate a class describing the Task](#creating-a-task) and the work to be
         | 
| 104 | 
            +
               done.
         | 
| 49 105 | 
             
            2. Run the Task
         | 
| 50 106 | 
             
                - either by [using the included web UI](#running-a-task-from-the-web-ui),
         | 
| 51 107 | 
             
                - or by [using the command line](#running-a-task-from-the-command-line),
         | 
| @@ -137,9 +193,9 @@ title,content | |
| 137 193 | 
             
            My Title,Hello World!
         | 
| 138 194 | 
             
            ```
         | 
| 139 195 |  | 
| 140 | 
            -
            The files uploaded to your Active Storage service provider will be renamed
         | 
| 141 | 
            -
             | 
| 142 | 
            -
             | 
| 196 | 
            +
            The files uploaded to your Active Storage service provider will be renamed to
         | 
| 197 | 
            +
            include an ISO 8601 timestamp and the Task name in snake case format. The CSV is
         | 
| 198 | 
            +
            expected to have a trailing newline at the end of the file.
         | 
| 143 199 |  | 
| 144 200 | 
             
            #### Batch CSV Tasks
         | 
| 145 201 |  | 
| @@ -168,6 +224,9 @@ Note that `#count` is calculated automatically based on the number of batches in | |
| 168 224 | 
             
            your collection, and your Task’s progress will be displayed in terms of batches
         | 
| 169 225 | 
             
            (not the total number of rows in your CSV).
         | 
| 170 226 |  | 
| 227 | 
            +
            Non-batched CSV tasks will have an effective batch size of 1, which can reduce
         | 
| 228 | 
            +
            the efficiency of your database operations.
         | 
| 229 | 
            +
             | 
| 171 230 | 
             
            ### Processing Batch Collections
         | 
| 172 231 |  | 
| 173 232 | 
             
            The Maintenance Tasks gem supports processing Active Records in batches. This
         | 
| @@ -213,8 +272,8 @@ inside `#process`. | |
| 213 272 | 
             
            ### Tasks that don’t need a Collection
         | 
| 214 273 |  | 
| 215 274 | 
             
            Sometimes, you might want to run a Task that performs a single operation, such
         | 
| 216 | 
            -
            as enqueuing another background job or  | 
| 217 | 
            -
            collection-less tasks.
         | 
| 275 | 
            +
            as enqueuing another background job or querying an external API. The gem
         | 
| 276 | 
            +
            supports collection-less tasks.
         | 
| 218 277 |  | 
| 219 278 | 
             
            Generate a collection-less Task by running:
         | 
| 220 279 |  | 
| @@ -242,10 +301,12 @@ end | |
| 242 301 |  | 
| 243 302 | 
             
            ### Throttling
         | 
| 244 303 |  | 
| 245 | 
            -
            Maintenance  | 
| 304 | 
            +
            Maintenance tasks often modify a lot of data and can be taxing on your database.
         | 
| 246 305 | 
             
            The gem provides a throttling mechanism that can be used to throttle a Task when
         | 
| 247 | 
            -
            a given condition is met. If a Task is throttled | 
| 248 | 
            -
            retried after a backoff period has passed. The | 
| 306 | 
            +
            a given condition is met. If a Task is throttled (the throttle block returns
         | 
| 307 | 
            +
            true), it will be interrupted and retried after a backoff period has passed. The
         | 
| 308 | 
            +
            default backoff is 30 seconds.
         | 
| 309 | 
            +
             | 
| 249 310 | 
             
            Specify the throttle condition as a block:
         | 
| 250 311 |  | 
| 251 312 | 
             
            ```ruby
         | 
| @@ -277,7 +338,7 @@ Tasks can define multiple throttle conditions. Throttle conditions are inherited | |
| 277 338 | 
             
            by descendants, and new conditions will be appended without impacting existing
         | 
| 278 339 | 
             
            conditions.
         | 
| 279 340 |  | 
| 280 | 
            -
            The backoff can also be specified as a Proc:
         | 
| 341 | 
            +
            The backoff can also be specified as a Proc that receives no arguments:
         | 
| 281 342 |  | 
| 282 343 | 
             
            ```ruby
         | 
| 283 344 | 
             
            # app/tasks/maintenance/update_posts_throttled_task.rb
         | 
| @@ -350,10 +411,9 @@ module Maintenance | |
| 350 411 | 
             
            end
         | 
| 351 412 | 
             
            ```
         | 
| 352 413 |  | 
| 353 | 
            -
            Note: The `after_error` callback is guaranteed to complete,
         | 
| 354 | 
            -
             | 
| 355 | 
            -
             | 
| 356 | 
            -
            you’ll need to rescue it and handle it appropriately
         | 
| 414 | 
            +
            Note: The `after_error` callback is guaranteed to complete, so any exceptions
         | 
| 415 | 
            +
            raised in your callback code are ignored. If your `after_error` callback code
         | 
| 416 | 
            +
            can raise an exception, you’ll need to rescue it and handle it appropriately
         | 
| 357 417 | 
             
            within the callback.
         | 
| 358 418 |  | 
| 359 419 | 
             
            ```ruby
         | 
| @@ -362,7 +422,7 @@ module Maintenance | |
| 362 422 | 
             
                after_error :dangerous_notify
         | 
| 363 423 |  | 
| 364 424 | 
             
                def dangerous_notify
         | 
| 365 | 
            -
                  # This error is rescued in favour of the original error causing the error flow.
         | 
| 425 | 
            +
                  # This error is rescued and ignored in favour of the original error causing the error flow.
         | 
| 366 426 | 
             
                  raise NotDeliveredError
         | 
| 367 427 | 
             
                end
         | 
| 368 428 |  | 
| @@ -371,9 +431,8 @@ module Maintenance | |
| 371 431 | 
             
            end
         | 
| 372 432 | 
             
            ```
         | 
| 373 433 |  | 
| 374 | 
            -
            If any of the other callbacks cause an exception,
         | 
| 375 | 
            -
             | 
| 376 | 
            -
            and will cause the task to stop running.
         | 
| 434 | 
            +
            If any of the other callbacks cause an exception, it will be handled by the
         | 
| 435 | 
            +
            error handler, and will cause the task to stop running.
         | 
| 377 436 |  | 
| 378 437 | 
             
            Callback behaviour can be shared across all tasks using an initializer.
         | 
| 379 438 |  | 
| @@ -392,21 +451,21 @@ end | |
| 392 451 |  | 
| 393 452 | 
             
            ### Considerations when writing Tasks
         | 
| 394 453 |  | 
| 395 | 
            -
             | 
| 454 | 
            +
            Maintenance Tasks relies on the queue adapter configured for your application to
         | 
| 396 455 | 
             
            run the job which is processing your Task. The guidelines for writing Task may
         | 
| 397 456 | 
             
            depend on the queue adapter but in general, you should follow these rules:
         | 
| 398 457 |  | 
| 399 458 | 
             
            * Duration of `Task#process`: processing a single element of the collection
         | 
| 400 459 | 
             
              should take less than 25 seconds, or the duration set as a timeout for Sidekiq
         | 
| 401 | 
            -
              or the queue adapter configured in your application.  | 
| 402 | 
            -
              safely interrupted and resumed.
         | 
| 460 | 
            +
              or the queue adapter configured in your application. Short batches allow the
         | 
| 461 | 
            +
              Task to be safely interrupted and resumed.
         | 
| 403 462 | 
             
            * Idempotency of `Task#process`: it should be safe to run `process` multiple
         | 
| 404 463 | 
             
              times for the same element of the collection. Read more in [this Sidekiq best
         | 
| 405 464 | 
             
              practice][sidekiq-idempotent]. It’s important if the Task errors and you run
         | 
| 406 | 
            -
              it again, because the same element that  | 
| 407 | 
            -
              again. It especially matters in the situation described | 
| 408 | 
            -
              iteration duration exceeds the timeout: if the job is | 
| 409 | 
            -
              elements may be processed again.
         | 
| 465 | 
            +
              it again, because the same element that caused the Task to give an error may
         | 
| 466 | 
            +
              well be processed again. It especially matters in the situation described
         | 
| 467 | 
            +
              above, when the iteration duration exceeds the timeout: if the job is
         | 
| 468 | 
            +
              re-enqueued, multiple elements may be processed again.
         | 
| 410 469 |  | 
| 411 470 | 
             
            [sidekiq-idempotent]: https://github.com/mperham/sidekiq/wiki/Best-Practices#2-make-your-job-idempotent-and-transactional
         | 
| 412 471 |  | 
| @@ -415,14 +474,14 @@ depend on the queue adapter but in general, you should follow these rules: | |
| 415 474 | 
             
            When the Task runs or resumes, the Runner enqueues a job, which processes the
         | 
| 416 475 | 
             
            Task. That job will instantiate a Task object which will live for the duration
         | 
| 417 476 | 
             
            of the job. The first time the job runs, it will call `count`. Every time a job
         | 
| 418 | 
            -
            runs, it will call `collection` on the Task object, and then `process`
         | 
| 419 | 
            -
             | 
| 477 | 
            +
            runs, it will call `collection` on the Task object, and then `process` for each
         | 
| 478 | 
            +
            item in the collection, until the job stops. The job stops when either the
         | 
| 420 479 | 
             
            collection is finished processing or after the maximum job runtime has expired.
         | 
| 421 480 |  | 
| 422 481 | 
             
            This means memoization can be misleading within `process`, since the memoized
         | 
| 423 482 | 
             
            values will be available for subsequent calls to `process` within the same job.
         | 
| 424 | 
            -
            Still, memoization can be used for throttling or reporting, and you can use | 
| 425 | 
            -
            callbacks](#using-task-callbacks) to persist or log a report for example.
         | 
| 483 | 
            +
            Still, memoization can be used for throttling or reporting, and you can use
         | 
| 484 | 
            +
            [Task callbacks](#using-task-callbacks) to persist or log a report for example.
         | 
| 426 485 |  | 
| 427 486 | 
             
            ### Writing tests for a Task
         | 
| 428 487 |  | 
| @@ -500,7 +559,7 @@ module Maintenance | |
| 500 559 |  | 
| 501 560 | 
             
                test "#process performs a task iteration" do
         | 
| 502 561 | 
             
                  assert_difference -> { Post.first.content } do
         | 
| 503 | 
            -
                    task.process(Post.first)
         | 
| 562 | 
            +
                    @task.process(Post.first)
         | 
| 504 563 | 
             
                  end
         | 
| 505 564 | 
             
                end
         | 
| 506 565 | 
             
              end
         | 
| @@ -612,7 +671,7 @@ tweaked in an initializer if necessary. | |
| 612 671 | 
             
            [max-job-runtime]: https://github.com/Shopify/job-iteration/blob/-/guides/best-practices.md#max-job-runtime
         | 
| 613 672 |  | 
| 614 673 | 
             
            Running tasks will also be interrupted and re-enqueued when needed. For example
         | 
| 615 | 
            -
            [when Sidekiq workers  | 
| 674 | 
            +
            [when Sidekiq workers shut down for a deploy][sidekiq-deploy]:
         | 
| 616 675 |  | 
| 617 676 | 
             
            [sidekiq-deploy]: https://github.com/mperham/sidekiq/wiki/Deployment
         | 
| 618 677 |  | 
| @@ -625,19 +684,24 @@ Running tasks will also be interrupted and re-enqueued when needed. For example | |
| 625 684 | 
             
            When Sidekiq is stopping, it will give workers 25 seconds to finish before
         | 
| 626 685 | 
             
            forcefully terminating them (this is the default but can be configured with the
         | 
| 627 686 | 
             
            `--timeout` option). Before the worker threads are terminated, Sidekiq will try
         | 
| 628 | 
            -
            to re-enqueue the job so your Task will be resumed. However, the position in
         | 
| 629 | 
            -
             | 
| 687 | 
            +
            to re-enqueue the job so your Task will be resumed. However, the position in the
         | 
| 688 | 
            +
            collection won’t be persisted so at least one iteration may run again.
         | 
| 689 | 
            +
             | 
| 690 | 
            +
            Job queues other than Sidekiq may handle this in different ways.
         | 
| 630 691 |  | 
| 631 692 | 
             
            #### Help! My Task is stuck
         | 
| 632 693 |  | 
| 633 | 
            -
             | 
| 634 | 
            -
             | 
| 635 | 
            -
             | 
| 636 | 
            -
             | 
| 637 | 
            -
             | 
| 638 | 
            -
             | 
| 639 | 
            -
             | 
| 640 | 
            -
             | 
| 694 | 
            +
            If the queue adapter configured for your application doesn’t have this property,
         | 
| 695 | 
            +
            or if Sidekiq crashes, is forcefully terminated, or is unable to re-enqueue the
         | 
| 696 | 
            +
            jobs that were in progress, the Task may be in a seemingly stuck situation where
         | 
| 697 | 
            +
            it appears to be running but is not. In that situation, pausing or cancelling it
         | 
| 698 | 
            +
            will not result in the Task being paused or cancelled, as the Task will get
         | 
| 699 | 
            +
            stuck in a state of `pausing` or `cancelling`. As a work-around, if a Task is
         | 
| 700 | 
            +
            `cancelling` for more than 5 minutes, you can cancel it again. It will then be
         | 
| 701 | 
            +
            marked as fully cancelled, allowing you to run it again.
         | 
| 702 | 
            +
             | 
| 703 | 
            +
            If you are stuck in `pausing` and wish to preserve your tasks's position
         | 
| 704 | 
            +
            (instead of cancelling and rerunning), you may click "Force pause".
         | 
| 641 705 |  | 
| 642 706 | 
             
            ### Configuring the gem
         | 
| 643 707 |  | 
| @@ -698,9 +762,10 @@ If no value is specified, it will default to `Maintenance`. | |
| 698 762 |  | 
| 699 763 | 
             
            #### Organizing tasks using namespaces
         | 
| 700 764 |  | 
| 701 | 
            -
            Tasks may be nested arbitrarily deeply under `app/tasks/maintenance`, for | 
| 702 | 
            -
            task file | 
| 703 | 
            -
             | 
| 765 | 
            +
            Tasks may be nested arbitrarily deeply under `app/tasks/maintenance`, for
         | 
| 766 | 
            +
            example given a task file
         | 
| 767 | 
            +
            `app/tasks/maintenance/team_name/service_name/update_posts_task.rb` we can
         | 
| 768 | 
            +
            define the task as:
         | 
| 704 769 |  | 
| 705 770 | 
             
            ```ruby
         | 
| 706 771 | 
             
            module Maintenance
         | 
| @@ -789,8 +854,8 @@ default. | |
| 789 854 | 
             
            #### Customizing the backtrace cleaner
         | 
| 790 855 |  | 
| 791 856 | 
             
            `MaintenanceTasks.backtrace_cleaner` can be configured to specify a backtrace
         | 
| 792 | 
            -
            cleaner to use when a Task errors and the backtrace is cleaned and persisted.
         | 
| 793 | 
            -
             | 
| 857 | 
            +
            cleaner to use when a Task errors and the backtrace is cleaned and persisted. An
         | 
| 858 | 
            +
            `ActiveSupport::BacktraceCleaner` should be used.
         | 
| 794 859 |  | 
| 795 860 | 
             
            ```ruby
         | 
| 796 861 | 
             
            # config/initializers/maintenance_tasks.rb
         | 
| @@ -806,8 +871,8 @@ clean backtraces. | |
| 806 871 |  | 
| 807 872 | 
             
            #### Customizing the parent controller for the web UI
         | 
| 808 873 |  | 
| 809 | 
            -
            `MaintenanceTasks.parent_controller` can be configured to specify a controller | 
| 810 | 
            -
            controllers to inherit from.
         | 
| 874 | 
            +
            `MaintenanceTasks.parent_controller` can be configured to specify a controller
         | 
| 875 | 
            +
            class for all of the web UI engine's controllers to inherit from.
         | 
| 811 876 |  | 
| 812 877 | 
             
            This allows applications with common logic in their `ApplicationController` (or
         | 
| 813 878 | 
             
            any other controller) to optionally configure the web UI to inherit that logic
         | 
| @@ -834,14 +899,15 @@ If no value is specified, it will default to `"ActionController::Base"`. | |
| 834 899 |  | 
| 835 900 | 
             
            ### Metadata
         | 
| 836 901 |  | 
| 837 | 
            -
            `MaintenanceTasks.metadata` can be configured to specify a proc from which to | 
| 838 | 
            -
            Since this proc will be ran in the context | 
| 839 | 
            -
             | 
| 902 | 
            +
            `MaintenanceTasks.metadata` can be configured to specify a proc from which to
         | 
| 903 | 
            +
            get extra information about the run. Since this proc will be ran in the context
         | 
| 904 | 
            +
            of the `MaintenanceTasks.parent_controller`, it can be used to keep the id or
         | 
| 905 | 
            +
            email of the user who performed the maintenance task.
         | 
| 840 906 |  | 
| 841 907 | 
             
            ```ruby
         | 
| 842 908 | 
             
            # config/initializers/maintenance_tasks.rb
         | 
| 843 | 
            -
            MaintenanceTasks.metadata = ->( | 
| 844 | 
            -
             | 
| 909 | 
            +
            MaintenanceTasks.metadata = ->() do
         | 
| 910 | 
            +
              { user_email: current_user.email }
         | 
| 845 911 | 
             
            end
         | 
| 846 912 | 
             
            ```
         | 
| 847 913 |  | 
| @@ -856,7 +922,7 @@ bin/rails generate maintenance_tasks:install | |
| 856 922 |  | 
| 857 923 | 
             
            This ensures that new migrations are installed and run as well.
         | 
| 858 924 |  | 
| 859 | 
            -
             | 
| 925 | 
            +
            ### What if I’ve deleted my previous Maintenance Task migrations?
         | 
| 860 926 |  | 
| 861 927 | 
             
            The install command will attempt to reinstall these old migrations and migrating
         | 
| 862 928 | 
             
            the database will cause problems. Use `bin/rails
         | 
| @@ -13,7 +13,7 @@ module MaintenanceTasks | |
| 13 13 | 
             
                  task = Runner.run(
         | 
| 14 14 | 
             
                    name: params.fetch(:task_id),
         | 
| 15 15 | 
             
                    csv_file: params[:csv_file],
         | 
| 16 | 
            -
                    arguments: params.fetch(: | 
| 16 | 
            +
                    arguments: params.fetch(:task, {}).permit!.to_h,
         | 
| 17 17 | 
             
                    metadata: instance_exec(&MaintenanceTasks.metadata || -> {}),
         | 
| 18 18 | 
             
                    &block
         | 
| 19 19 | 
             
                  )
         | 
| @@ -29,7 +29,7 @@ module MaintenanceTasks | |
| 29 29 |  | 
| 30 30 | 
             
                # Updates a Run status to paused.
         | 
| 31 31 | 
             
                def pause
         | 
| 32 | 
            -
                  @run. | 
| 32 | 
            +
                  @run.pause
         | 
| 33 33 | 
             
                  redirect_to(task_path(@run.task_name))
         | 
| 34 34 | 
             
                rescue ActiveRecord::RecordInvalid => error
         | 
| 35 35 | 
             
                  redirect_to(task_path(@run.task_name), alert: error.message)
         | 
| @@ -320,21 +320,29 @@ module MaintenanceTasks | |
| 320 320 |  | 
| 321 321 | 
             
                # Marks a Run as pausing.
         | 
| 322 322 | 
             
                #
         | 
| 323 | 
            +
                # If the Run has been stuck on pausing for more than 5 minutes, it forces
         | 
| 324 | 
            +
                # the transition to paused. The ended_at timestamp will be updated.
         | 
| 325 | 
            +
                #
         | 
| 323 326 | 
             
                # Rescues and retries status transition if an ActiveRecord::StaleObjectError
         | 
| 324 327 | 
             
                # is encountered.
         | 
| 325 | 
            -
                def  | 
| 326 | 
            -
                   | 
| 328 | 
            +
                def pause
         | 
| 329 | 
            +
                  if stuck?
         | 
| 330 | 
            +
                    self.status = :paused
         | 
| 331 | 
            +
                    persist_transition
         | 
| 332 | 
            +
                  else
         | 
| 333 | 
            +
                    pausing!
         | 
| 334 | 
            +
                  end
         | 
| 327 335 | 
             
                rescue ActiveRecord::StaleObjectError
         | 
| 328 336 | 
             
                  reload_status
         | 
| 329 337 | 
             
                  retry
         | 
| 330 338 | 
             
                end
         | 
| 331 339 |  | 
| 332 340 | 
             
                # Returns whether a Run is stuck, which is defined as having a status of
         | 
| 333 | 
            -
                # cancelling, and not having been updated in the last 5 minutes.
         | 
| 341 | 
            +
                # cancelling or pausing, and not having been updated in the last 5 minutes.
         | 
| 334 342 | 
             
                #
         | 
| 335 343 | 
             
                # @return [Boolean] whether the Run is stuck.
         | 
| 336 344 | 
             
                def stuck?
         | 
| 337 | 
            -
                  cancelling? && updated_at <= STUCK_TASK_TIMEOUT.ago
         | 
| 345 | 
            +
                  (cancelling? || pausing?) && updated_at <= STUCK_TASK_TIMEOUT.ago
         | 
| 338 346 | 
             
                end
         | 
| 339 347 |  | 
| 340 348 | 
             
                # Performs validation on the task_name attribute.
         | 
| @@ -187,13 +187,7 @@ module MaintenanceTasks | |
| 187 187 | 
             
                    namespace = MaintenanceTasks.tasks_module.safe_constantize
         | 
| 188 188 | 
             
                    return unless namespace
         | 
| 189 189 |  | 
| 190 | 
            -
                     | 
| 191 | 
            -
                      root.constants.each do |name|
         | 
| 192 | 
            -
                        object = root.const_get(name)
         | 
| 193 | 
            -
                        load_const.call(object) if object.instance_of?(Module)
         | 
| 194 | 
            -
                      end
         | 
| 195 | 
            -
                    end
         | 
| 196 | 
            -
                    load_const.call(namespace)
         | 
| 190 | 
            +
                    Rails.autoloaders.main.eager_load_namespace(namespace)
         | 
| 197 191 | 
             
                  end
         | 
| 198 192 | 
             
                end
         | 
| 199 193 |  | 
| @@ -29,6 +29,9 @@ | |
| 29 29 | 
             
                <% elsif run.pausing? %>
         | 
| 30 30 | 
             
                  <%= button_to 'Pausing', pause_task_run_path(@task, run), method: :put, class: 'button is-warning', disabled: true %>
         | 
| 31 31 | 
             
                  <%= button_to 'Cancel', cancel_task_run_path(@task, run), method: :put, class: 'button is-danger' %>
         | 
| 32 | 
            +
                  <% if run.stuck? %>
         | 
| 33 | 
            +
                    <%= button_to 'Force pause', pause_task_run_path(@task, run), method: :put, class: 'button is-danger', disabled: @task.deleted? %>
         | 
| 34 | 
            +
                  <% end %>
         | 
| 32 35 | 
             
                <% elsif run.active? %>
         | 
| 33 36 | 
             
                  <%= button_to 'Pause', pause_task_run_path(@task, run), method: :put, class: 'button is-warning', disabled: @task.deleted? %>
         | 
| 34 37 | 
             
                  <%= button_to 'Cancel', cancel_task_run_path(@task, run), method: :put, class: 'button is-danger' %>
         | 
| @@ -15,7 +15,7 @@ | |
| 15 15 | 
             
                <% parameter_names = @task.parameter_names %>
         | 
| 16 16 | 
             
                <% if parameter_names.any? %>
         | 
| 17 17 | 
             
                  <div class="block">
         | 
| 18 | 
            -
                    <%=  | 
| 18 | 
            +
                    <%= fields_for :task, @task.new do |ff| %>
         | 
| 19 19 | 
             
                      <% parameter_names.each do |parameter_name| %>
         | 
| 20 20 | 
             
                        <div class="field">
         | 
| 21 21 | 
             
                          <%= ff.label parameter_name, parameter_name, class: "label is-family-monospace" %>
         | 
| @@ -6,10 +6,14 @@ module <%= tasks_module %> | |
| 6 6 | 
             
            <% module_namespacing do -%>
         | 
| 7 7 | 
             
              RSpec.describe <%= class_name %>Task do
         | 
| 8 8 | 
             
                describe "#process" do
         | 
| 9 | 
            +
                  <%- if no_collection? -%>
         | 
| 10 | 
            +
                  subject(:process) { described_class.process }
         | 
| 11 | 
            +
                  <%- else -%>
         | 
| 9 12 | 
             
                  subject(:process) { described_class.process(element) }
         | 
| 10 13 | 
             
                  let(:element) {
         | 
| 11 14 | 
             
                    # Object to be processed in a single iteration of this task
         | 
| 12 15 | 
             
                  }
         | 
| 16 | 
            +
                  <%- end -%>
         | 
| 13 17 | 
             
                  pending "add some examples to (or delete) #{__FILE__}"
         | 
| 14 18 | 
             
                end
         | 
| 15 19 | 
             
              end
         | 
| @@ -15,6 +15,12 @@ module MaintenanceTasks | |
| 15 15 | 
             
                end
         | 
| 16 16 |  | 
| 17 17 | 
             
                desc "perform [TASK NAME]", "Runs the given Maintenance Task"
         | 
| 18 | 
            +
                long_desc <<~DESC
         | 
| 19 | 
            +
                  `maintenance_tasks perform` will run the Maintenance Task specified by
         | 
| 20 | 
            +
                  the [TASK NAME] argument.
         | 
| 21 | 
            +
             | 
| 22 | 
            +
                  Use `maintenance_tasks list` to get a list of all available tasks.
         | 
| 23 | 
            +
                DESC
         | 
| 18 24 |  | 
| 19 25 | 
             
                # Specify the CSV file to process for CSV Tasks
         | 
| 20 26 | 
             
                desc = "Supply a CSV file to be processed by a CSV Task, "\
         | 
| @@ -41,19 +47,14 @@ module MaintenanceTasks | |
| 41 47 | 
             
                  say_status(:error, error.message, :red)
         | 
| 42 48 | 
             
                end
         | 
| 43 49 |  | 
| 44 | 
            -
                 | 
| 45 | 
            -
                # By redefining the `#long_description` method on the "perform" Command
         | 
| 46 | 
            -
                # object instead, we make it dynamic, thus delaying the task loading
         | 
| 47 | 
            -
                # process until it's actually required.
         | 
| 48 | 
            -
                commands["perform"].define_singleton_method(:long_description) do
         | 
| 49 | 
            -
                  <<~LONGDESC
         | 
| 50 | 
            -
                    `maintenance_tasks perform` will run the Maintenance Task specified by
         | 
| 51 | 
            -
                    the [TASK NAME] argument.
         | 
| 52 | 
            -
             | 
| 53 | 
            -
                    Available Tasks:
         | 
| 50 | 
            +
                desc "list", "Load and list all available tasks."
         | 
| 54 51 |  | 
| 55 | 
            -
             | 
| 56 | 
            -
             | 
| 52 | 
            +
                # Command to list all available Tasks.
         | 
| 53 | 
            +
                #
         | 
| 54 | 
            +
                # It needs to use `Task.load_all` in order to load all the tasks available
         | 
| 55 | 
            +
                # in the project before displaying them.
         | 
| 56 | 
            +
                def list
         | 
| 57 | 
            +
                  say(Task.load_all.map(&:name).sort.join("\n"))
         | 
| 57 58 | 
             
                end
         | 
| 58 59 |  | 
| 59 60 | 
             
                private
         | 
    
        metadata
    CHANGED
    
    | @@ -1,14 +1,14 @@ | |
| 1 1 | 
             
            --- !ruby/object:Gem::Specification
         | 
| 2 2 | 
             
            name: maintenance_tasks
         | 
| 3 3 | 
             
            version: !ruby/object:Gem::Version
         | 
| 4 | 
            -
              version: 2. | 
| 4 | 
            +
              version: 2.4.0
         | 
| 5 5 | 
             
            platform: ruby
         | 
| 6 6 | 
             
            authors:
         | 
| 7 7 | 
             
            - Shopify Engineering
         | 
| 8 8 | 
             
            autorequire: 
         | 
| 9 9 | 
             
            bindir: exe
         | 
| 10 10 | 
             
            cert_chain: []
         | 
| 11 | 
            -
            date: 2023- | 
| 11 | 
            +
            date: 2023-12-20 00:00:00.000000000 Z
         | 
| 12 12 | 
             
            dependencies:
         | 
| 13 13 | 
             
            - !ruby/object:Gem::Dependency
         | 
| 14 14 | 
             
              name: actionpack
         | 
| @@ -80,6 +80,20 @@ dependencies: | |
| 80 80 | 
             
                - - ">="
         | 
| 81 81 | 
             
                  - !ruby/object:Gem::Version
         | 
| 82 82 | 
             
                    version: '6.0'
         | 
| 83 | 
            +
            - !ruby/object:Gem::Dependency
         | 
| 84 | 
            +
              name: zeitwerk
         | 
| 85 | 
            +
              requirement: !ruby/object:Gem::Requirement
         | 
| 86 | 
            +
                requirements:
         | 
| 87 | 
            +
                - - ">="
         | 
| 88 | 
            +
                  - !ruby/object:Gem::Version
         | 
| 89 | 
            +
                    version: 2.6.2
         | 
| 90 | 
            +
              type: :runtime
         | 
| 91 | 
            +
              prerelease: false
         | 
| 92 | 
            +
              version_requirements: !ruby/object:Gem::Requirement
         | 
| 93 | 
            +
                requirements:
         | 
| 94 | 
            +
                - - ">="
         | 
| 95 | 
            +
                  - !ruby/object:Gem::Version
         | 
| 96 | 
            +
                    version: 2.6.2
         | 
| 83 97 | 
             
            description: 
         | 
| 84 98 | 
             
            email: gems@shopify.com
         | 
| 85 99 | 
             
            executables:
         | 
| @@ -143,7 +157,6 @@ files: | |
| 143 157 | 
             
            - lib/generators/maintenance_tasks/task_generator.rb
         | 
| 144 158 | 
             
            - lib/generators/maintenance_tasks/templates/csv_task.rb.tt
         | 
| 145 159 | 
             
            - lib/generators/maintenance_tasks/templates/no_collection_task.rb.tt
         | 
| 146 | 
            -
            - lib/generators/maintenance_tasks/templates/no_collection_task_test.rb.tt
         | 
| 147 160 | 
             
            - lib/generators/maintenance_tasks/templates/task.rb.tt
         | 
| 148 161 | 
             
            - lib/generators/maintenance_tasks/templates/task_spec.rb.tt
         | 
| 149 162 | 
             
            - lib/generators/maintenance_tasks/templates/task_test.rb.tt
         | 
| @@ -156,7 +169,7 @@ homepage: https://github.com/Shopify/maintenance_tasks | |
| 156 169 | 
             
            licenses:
         | 
| 157 170 | 
             
            - MIT
         | 
| 158 171 | 
             
            metadata:
         | 
| 159 | 
            -
              source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v2. | 
| 172 | 
            +
              source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v2.4.0
         | 
| 160 173 | 
             
              allowed_push_host: https://rubygems.org
         | 
| 161 174 | 
             
            post_install_message: 
         | 
| 162 175 | 
             
            rdoc_options: []
         | 
| @@ -173,7 +186,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement | |
| 173 186 | 
             
                - !ruby/object:Gem::Version
         | 
| 174 187 | 
             
                  version: '0'
         | 
| 175 188 | 
             
            requirements: []
         | 
| 176 | 
            -
            rubygems_version: 3.4. | 
| 189 | 
            +
            rubygems_version: 3.4.22
         | 
| 177 190 | 
             
            signing_key: 
         | 
| 178 191 | 
             
            specification_version: 4
         | 
| 179 192 | 
             
            summary: A Rails engine for queuing and managing maintenance tasks
         | 
| @@ -1,13 +0,0 @@ | |
| 1 | 
            -
            # frozen_string_literal: true
         | 
| 2 | 
            -
             | 
| 3 | 
            -
            require "test_helper"
         | 
| 4 | 
            -
             | 
| 5 | 
            -
            module <%= tasks_module %>
         | 
| 6 | 
            -
            <% module_namespacing do -%>
         | 
| 7 | 
            -
              class <%= class_name %>TaskTest < ActiveSupport::TestCase
         | 
| 8 | 
            -
                # test "#process performs a task iteration" do
         | 
| 9 | 
            -
                #   <%= tasks_module %>::<%= class_name %>Task.process
         | 
| 10 | 
            -
                # end
         | 
| 11 | 
            -
              end
         | 
| 12 | 
            -
            <% end -%>
         | 
| 13 | 
            -
            end
         |