maintenance_tasks 2.3.3 → 2.5.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1f292dbc04fbbb71588c0be3af0bc95bfb5de5ca27f9aa7562b681cad39c6241
4
- data.tar.gz: d631953126ffbd114721fea778fc8d43f5c86c20461c04943c3127a1b6f9e0d3
3
+ metadata.gz: f26c6afe0c4b45b6d191162ab2c37e3b058e8b6ae473b17cfc3724e29e638fce
4
+ data.tar.gz: 4308f3356279fa5879429077cb482d91033336bfa62c1c4afb51fc9d957b1785
5
5
  SHA512:
6
- metadata.gz: 146510c2a63f74b6084b43f9fa939c017e242b79ae0c453008b17b8e5c5e2332bd0d972ab4d8dce4d9d365836bcada8df8a7f1a7d717dc1d733232edd0830a4e
7
- data.tar.gz: d1df2c04b6274843b32e4feb44a4bfb1eec3e64717ae67d78bbb440c08c34f24f63882d880dfc602529b2971f9bbfc9ce74345b848ecd3f780d1ab9270a610ec
6
+ metadata.gz: 383bc985f1a465e00889b20997d5718ff2039d5cdc6ca6b14a427a183fef4df3ff445eda2893d10b41d0bf243a6150ad5699afdfcbe3beaf280e4d35f130c1f5
7
+ data.tar.gz: d2b54c567a3d89e78f3538ca22b35d83dcb150e7520fc3014aa2040836f2e2741eb2bda700211c6411b8355b54ee19814655cf79827358d2d2f922fa855934af
data/README.md CHANGED
@@ -5,35 +5,35 @@ A Rails engine for queuing and managing maintenance tasks.
5
5
  By ”maintenance task”, this project means a data migration, i.e. code that
6
6
  changes data in the database, often to support schema migrations. For example,
7
7
  in order to introduce a new `NOT NULL` column, it has to be added as nullable
8
- first, backfilled with values, before finally being changed to `NOT NULL`.
9
- This engine helps with the second part of this process, backfilling.
8
+ first, backfilled with values, before finally being changed to `NOT NULL`. This
9
+ engine helps with the second part of this process, backfilling.
10
10
 
11
- Maintenance tasks are collection-based tasks, usually using Active Record,
12
- that update the data in your database. They can be paused or interrupted.
13
- Maintenance tasks can operate [in batches](#processing-batch-collections) and
14
- use [throttling](#throttling) to control the load on your database.
11
+ Maintenance tasks are collection-based tasks, usually using Active Record, that
12
+ update the data in your database. They can be paused or interrupted. Maintenance
13
+ tasks can operate [in batches](#processing-batch-collections) and use
14
+ [throttling](#throttling) to control the load on your database.
15
15
 
16
16
  Maintenance tasks aren't meant to happen on a regular basis. They're used as
17
17
  needed, or as one-offs. Normally maintenance tasks are ephemeral, so they are
18
18
  used briefly and then deleted.
19
19
 
20
- The Rails engine has a web-based UI for listing maintenance tasks, seeing
21
- their status, and starting, pausing and restarting them.
20
+ The Rails engine has a web-based UI for listing maintenance tasks, seeing their
21
+ status, and starting, pausing and restarting them.
22
22
 
23
23
  [![Link to demo video](static/demo.png)](https://www.youtube.com/watch?v=BTuvTQxlFzs)
24
24
 
25
25
  ## Should I Use Maintenance Tasks?
26
26
 
27
- Maintenance tasks have a limited, specific job UI. While the engine can be
28
- used to provide a user interface for other data changes, such as data changes
29
- for support requests, we recommend you use regular application code for those
30
- use cases instead. These inevitably require more flexibility than this engine
31
- will be able to provide.
27
+ Maintenance tasks have a limited, specific job UI. While the engine can be used
28
+ to provide a user interface for other data changes, such as data changes for
29
+ support requests, we recommend you use regular application code for those use
30
+ cases instead. These inevitably require more flexibility than this engine will
31
+ be able to provide.
32
32
 
33
- If your task shouldn't run as an Active Job, it probably isn't a good match
34
- for this gem. If your task doesn't need to run in the background,
35
- consider a runner script instead. If your task doesn't need to be
36
- interruptible, consider a normal Active Job.
33
+ If your task shouldn't run as an Active Job, it probably isn't a good match for
34
+ this gem. If your task doesn't need to run in the background, consider a runner
35
+ script instead. If your task doesn't need to be interruptible, consider a normal
36
+ Active Job.
37
37
 
38
38
  Maintenance tasks can be interrupted between iterations. If your task [isn't
39
39
  collection-based](#tasks-that-dont-need-a-collection) (no CSV file or database
@@ -48,9 +48,10 @@ If your task happens regularly, consider Active Jobs with a scheduler or cron,
48
48
  [job-iteration jobs](https://github.com/shopify/job-iteration) and/or [custom
49
49
  rails_admin UIs][rails-admin-engines] instead of the Maintenance Tasks gem.
50
50
  Maintenance tasks should be ephemeral, to suit their intentionally limited UI.
51
+ They should not repeat.
51
52
 
52
- To create seed data for a new application, use the provided Rails
53
- `db/seeds.rb` file instead.
53
+ To create seed data for a new application, use the provided Rails `db/seeds.rb`
54
+ file instead.
54
55
 
55
56
  If your application can't handle a half-completed migration, maintenance tasks
56
57
  are probably the wrong tool. Remember that maintenance tasks are intentionally
@@ -99,7 +100,8 @@ constants](https://guides.rubyonrails.org/autoloading_and_reloading_constants.ht
99
100
 
100
101
  The typical Maintenance Tasks workflow is as follows:
101
102
 
102
- 1. [Generate a class describing the Task](#creating-a-task) and the work to be done.
103
+ 1. [Generate a class describing the Task](#creating-a-task) and the work to be
104
+ done.
103
105
  2. Run the Task
104
106
  - either by [using the included web UI](#running-a-task-from-the-web-ui),
105
107
  - or by [using the command line](#running-a-task-from-the-command-line),
@@ -191,9 +193,9 @@ title,content
191
193
  My Title,Hello World!
192
194
  ```
193
195
 
194
- The files uploaded to your Active Storage service provider will be renamed
195
- to include an ISO 8601 timestamp and the Task name in snake case format.
196
- The CSV is expected to have a trailing newline at the end of the file.
196
+ The files uploaded to your Active Storage service provider will be renamed to
197
+ include an ISO 8601 timestamp and the Task name in snake case format. The CSV is
198
+ expected to have a trailing newline at the end of the file.
197
199
 
198
200
  #### Batch CSV Tasks
199
201
 
@@ -270,8 +272,8 @@ inside `#process`.
270
272
  ### Tasks that don’t need a Collection
271
273
 
272
274
  Sometimes, you might want to run a Task that performs a single operation, such
273
- as enqueuing another background job or querying an external API. The gem supports
274
- collection-less tasks.
275
+ as enqueuing another background job or querying an external API. The gem
276
+ supports collection-less tasks.
275
277
 
276
278
  Generate a collection-less Task by running:
277
279
 
@@ -297,6 +299,35 @@ module Maintenance
297
299
  end
298
300
  ```
299
301
 
302
+ ### Tasks with Custom Enumerators
303
+
304
+ If you have a special use case requiring iteration over an unsupported
305
+ collection type, such as external resources fetched from some API, you can
306
+ implement the `enumerator_builder(cursor:)` method in your task.
307
+
308
+ This method should return an `Enumerator`, yielding pairs of
309
+ `[item, cursor]`. Maintenance Tasks takes care of persisting the current
310
+ cursor position and will provide it as the `cursor` argument if your task is
311
+ interrupted or resumed. The `cursor` is stored as a `String`, so your custom
312
+ enumerator should handle serializing/deserializing the value if required.
313
+
314
+ ```ruby
315
+ # app/tasks/maintenance/custom_enumerator_task.rb
316
+
317
+ module Maintenance
318
+ class CustomEnumeratorTask < MaintenanceTasks::Task
319
+ def enumerator_builder(cursor:)
320
+ after_id = cursor&.to_i
321
+ PostAPI.index(after_id: after_id).map { |post| [post, post.id] }.to_enum
322
+ end
323
+
324
+ def process(post)
325
+ Post.create!(post)
326
+ end
327
+ end
328
+ end
329
+ ```
330
+
300
331
  ### Throttling
301
332
 
302
333
  Maintenance tasks often modify a lot of data and can be taxing on your database.
@@ -409,10 +440,9 @@ module Maintenance
409
440
  end
410
441
  ```
411
442
 
412
- Note: The `after_error` callback is guaranteed to complete,
413
- so any exceptions raised in your callback code are ignored.
414
- If your `after_error` callback code can raise an exception,
415
- you’ll need to rescue it and handle it appropriately
443
+ Note: The `after_error` callback is guaranteed to complete, so any exceptions
444
+ raised in your callback code are ignored. If your `after_error` callback code
445
+ can raise an exception, you’ll need to rescue it and handle it appropriately
416
446
  within the callback.
417
447
 
418
448
  ```ruby
@@ -430,9 +460,8 @@ module Maintenance
430
460
  end
431
461
  ```
432
462
 
433
- If any of the other callbacks cause an exception,
434
- it will be handled by the error handler,
435
- and will cause the task to stop running.
463
+ If any of the other callbacks cause an exception, it will be handled by the
464
+ error handler, and will cause the task to stop running.
436
465
 
437
466
  Callback behaviour can be shared across all tasks using an initializer.
438
467
 
@@ -474,14 +503,14 @@ depend on the queue adapter but in general, you should follow these rules:
474
503
  When the Task runs or resumes, the Runner enqueues a job, which processes the
475
504
  Task. That job will instantiate a Task object which will live for the duration
476
505
  of the job. The first time the job runs, it will call `count`. Every time a job
477
- runs, it will call `collection` on the Task object, and then `process`
478
- for each item in the collection, until the job stops. The job stops when either the
506
+ runs, it will call `collection` on the Task object, and then `process` for each
507
+ item in the collection, until the job stops. The job stops when either the
479
508
  collection is finished processing or after the maximum job runtime has expired.
480
509
 
481
510
  This means memoization can be misleading within `process`, since the memoized
482
511
  values will be available for subsequent calls to `process` within the same job.
483
- Still, memoization can be used for throttling or reporting, and you can use [Task
484
- callbacks](#using-task-callbacks) to persist or log a report for example.
512
+ Still, memoization can be used for throttling or reporting, and you can use
513
+ [Task callbacks](#using-task-callbacks) to persist or log a report for example.
485
514
 
486
515
  ### Writing tests for a Task
487
516
 
@@ -559,7 +588,7 @@ module Maintenance
559
588
 
560
589
  test "#process performs a task iteration" do
561
590
  assert_difference -> { Post.first.content } do
562
- task.process(Post.first)
591
+ @task.process(Post.first)
563
592
  end
564
593
  end
565
594
  end
@@ -671,7 +700,7 @@ tweaked in an initializer if necessary.
671
700
  [max-job-runtime]: https://github.com/Shopify/job-iteration/blob/-/guides/best-practices.md#max-job-runtime
672
701
 
673
702
  Running tasks will also be interrupted and re-enqueued when needed. For example
674
- [when Sidekiq workers shuts down for a deploy][sidekiq-deploy]:
703
+ [when Sidekiq workers shut down for a deploy][sidekiq-deploy]:
675
704
 
676
705
  [sidekiq-deploy]: https://github.com/mperham/sidekiq/wiki/Deployment
677
706
 
@@ -684,19 +713,24 @@ Running tasks will also be interrupted and re-enqueued when needed. For example
684
713
  When Sidekiq is stopping, it will give workers 25 seconds to finish before
685
714
  forcefully terminating them (this is the default but can be configured with the
686
715
  `--timeout` option). Before the worker threads are terminated, Sidekiq will try
687
- to re-enqueue the job so your Task will be resumed. However, the position in
688
- the collection won’t be persisted so at least one iteration may run again.
716
+ to re-enqueue the job so your Task will be resumed. However, the position in the
717
+ collection won’t be persisted so at least one iteration may run again.
718
+
719
+ Job queues other than Sidekiq may handle this in different ways.
689
720
 
690
721
  #### Help! My Task is stuck
691
722
 
692
- Finally, if the queue adapter configured for your application doesn’t have this
693
- property, or if Sidekiq crashes, is forcefully terminated, or is unable to
694
- re-enqueue the jobs that were in progress, the Task may be in a seemingly stuck
695
- situation where it appears to be running but is not. In that situation, pausing
696
- or cancelling it will not result in the Task being paused or cancelled, as the
697
- Task will get stuck in a state of `pausing` or `cancelling`. As a work-around,
698
- if a Task is `cancelling` for more than 5 minutes, you can cancel it again.
699
- It will then be marked as fully cancelled, allowing you to run it again.
723
+ If the queue adapter configured for your application doesn’t have this property,
724
+ or if Sidekiq crashes, is forcefully terminated, or is unable to re-enqueue the
725
+ jobs that were in progress, the Task may be in a seemingly stuck situation where
726
+ it appears to be running but is not. In that situation, pausing or cancelling it
727
+ will not result in the Task being paused or cancelled, as the Task will get
728
+ stuck in a state of `pausing` or `cancelling`. As a work-around, if a Task is
729
+ `cancelling` for more than 5 minutes, you can cancel it again. It will then be
730
+ marked as fully cancelled, allowing you to run it again.
731
+
732
+ If you are stuck in `pausing` and wish to preserve your tasks's position
733
+ (instead of cancelling and rerunning), you may click "Force pause".
700
734
 
701
735
  ### Configuring the gem
702
736
 
@@ -757,9 +791,10 @@ If no value is specified, it will default to `Maintenance`.
757
791
 
758
792
  #### Organizing tasks using namespaces
759
793
 
760
- Tasks may be nested arbitrarily deeply under `app/tasks/maintenance`, for example given a
761
- task file `app/tasks/maintenance/team_name/service_name/update_posts_task.rb` we
762
- can define the task as:
794
+ Tasks may be nested arbitrarily deeply under `app/tasks/maintenance`, for
795
+ example given a task file
796
+ `app/tasks/maintenance/team_name/service_name/update_posts_task.rb` we can
797
+ define the task as:
763
798
 
764
799
  ```ruby
765
800
  module Maintenance
@@ -848,8 +883,8 @@ default.
848
883
  #### Customizing the backtrace cleaner
849
884
 
850
885
  `MaintenanceTasks.backtrace_cleaner` can be configured to specify a backtrace
851
- cleaner to use when a Task errors and the backtrace is cleaned and persisted.
852
- An `ActiveSupport::BacktraceCleaner` should be used.
886
+ cleaner to use when a Task errors and the backtrace is cleaned and persisted. An
887
+ `ActiveSupport::BacktraceCleaner` should be used.
853
888
 
854
889
  ```ruby
855
890
  # config/initializers/maintenance_tasks.rb
@@ -865,8 +900,8 @@ clean backtraces.
865
900
 
866
901
  #### Customizing the parent controller for the web UI
867
902
 
868
- `MaintenanceTasks.parent_controller` can be configured to specify a controller class for all of the web UI engine's
869
- controllers to inherit from.
903
+ `MaintenanceTasks.parent_controller` can be configured to specify a controller
904
+ class for all of the web UI engine's controllers to inherit from.
870
905
 
871
906
  This allows applications with common logic in their `ApplicationController` (or
872
907
  any other controller) to optionally configure the web UI to inherit that logic
@@ -891,11 +926,21 @@ controller class which **must inherit** from `ActionController::Base`.
891
926
 
892
927
  If no value is specified, it will default to `"ActionController::Base"`.
893
928
 
929
+ #### Configure time after which the task will be considered stuck
930
+
931
+ To specify a time duration after which a task is considered stuck if it has not been updated,
932
+ you can configure `MaintenanceTasks.stuck_task_duration`. This duration should account for
933
+ job infrastructure events that may prevent the maintenance tasks job from being executed and cancelling the task.
934
+
935
+ The value for `MaintenanceTasks.stuck_task_duration` must be an `ActiveSupport::Duration`.
936
+ If no value is specified, it will default to 5 minutes.
937
+
894
938
  ### Metadata
895
939
 
896
- `MaintenanceTasks.metadata` can be configured to specify a proc from which to get extra information about the run.
897
- Since this proc will be ran in the context of the `MaintenanceTasks.parent_controller`, it can be used to keep the id
898
- or email of the user who performed the maintenance task.
940
+ `MaintenanceTasks.metadata` can be configured to specify a proc from which to
941
+ get extra information about the run. Since this proc will be ran in the context
942
+ of the `MaintenanceTasks.parent_controller`, it can be used to keep the id or
943
+ email of the user who performed the maintenance task.
899
944
 
900
945
  ```ruby
901
946
  # config/initializers/maintenance_tasks.rb
@@ -915,7 +960,7 @@ bin/rails generate maintenance_tasks:install
915
960
 
916
961
  This ensures that new migrations are installed and run as well.
917
962
 
918
- **What if I’ve deleted my previous Maintenance Task migrations?**
963
+ ### What if I’ve deleted my previous Maintenance Task migrations?
919
964
 
920
965
  The install command will attempt to reinstall these old migrations and migrating
921
966
  the database will cause problems. Use `bin/rails
@@ -29,7 +29,7 @@ module MaintenanceTasks
29
29
 
30
30
  # Updates a Run status to paused.
31
31
  def pause
32
- @run.pausing!
32
+ @run.pause
33
33
  redirect_to(task_path(@run.task_name))
34
34
  rescue ActiveRecord::RecordInvalid => error
35
35
  redirect_to(task_path(@run.task_name), alert: error.message)
@@ -101,8 +101,17 @@ module MaintenanceTasks
101
101
  )
102
102
  end
103
103
 
104
- # Return the appropriate field tag for the parameter
104
+ # Return the appropriate field tag for the parameter, based on its type.
105
+ # If the parameter has a `validates_inclusion_of` validator, return a dropdown list of options instead.
105
106
  def parameter_field(form_builder, parameter_name)
107
+ inclusion_validator = form_builder.object.class.validators_on(parameter_name).find do |validator|
108
+ validator.kind == :inclusion
109
+ end
110
+
111
+ return form_builder.select(
112
+ parameter_name, inclusion_validator.options[:in], prompt: "Select a value"
113
+ ) if inclusion_validator
114
+
106
115
  case form_builder.object.class.attribute_types[parameter_name]
107
116
  when ActiveModel::Type::Integer
108
117
  form_builder.number_field(parameter_name)
@@ -32,45 +32,7 @@ module MaintenanceTasks
32
32
 
33
33
  def build_enumerator(_run, cursor:)
34
34
  cursor ||= @run.cursor
35
- collection = @task.collection
36
- @enumerator = nil
37
-
38
- @collection_enum = case collection
39
- when :no_collection
40
- enumerator_builder.build_once_enumerator(cursor: nil)
41
- when ActiveRecord::Relation
42
- enumerator_builder.active_record_on_records(collection, cursor: cursor)
43
- when ActiveRecord::Batches::BatchEnumerator
44
- if collection.start || collection.finish
45
- raise ArgumentError, <<~MSG.squish
46
- #{@task.class.name}#collection cannot support
47
- a batch enumerator with the "start" or "finish" options.
48
- MSG
49
- end
50
-
51
- # For now, only support automatic count based on the enumerator for
52
- # batches
53
- enumerator_builder.active_record_on_batch_relations(
54
- collection.relation,
55
- cursor: cursor,
56
- batch_size: collection.batch_size,
57
- )
58
- when Array
59
- enumerator_builder.build_array_enumerator(collection, cursor: cursor&.to_i)
60
- when BatchCsvCollectionBuilder::BatchCsv
61
- JobIteration::CsvEnumerator.new(collection.csv).batches(
62
- batch_size: collection.batch_size,
63
- cursor: cursor&.to_i,
64
- )
65
- when CSV
66
- JobIteration::CsvEnumerator.new(collection).rows(cursor: cursor&.to_i)
67
- else
68
- raise ArgumentError, <<~MSG.squish
69
- #{@task.class.name}#collection must be either an
70
- Active Record Relation, ActiveRecord::Batches::BatchEnumerator,
71
- Array, or CSV.
72
- MSG
73
- end
35
+ @collection_enum = @task.enumerator_builder(cursor: cursor)
74
36
  throttle_enumerator(@collection_enum)
75
37
  end
76
38
 
@@ -165,6 +127,7 @@ module MaintenanceTasks
165
127
  @ticker.persist if defined?(@ticker)
166
128
 
167
129
  if defined?(@run)
130
+ @run.cursor = cursor_position
168
131
  @run.persist_error(error)
169
132
 
170
133
  task_context = {
@@ -32,7 +32,6 @@ module MaintenanceTasks
32
32
  :cancelled,
33
33
  ]
34
34
  COMPLETED_STATUSES = [:succeeded, :errored, :cancelled]
35
- STUCK_TASK_TIMEOUT = 5.minutes
36
35
 
37
36
  enum status: STATUSES.to_h { |status| [status, status.to_s] }
38
37
 
@@ -320,21 +319,29 @@ module MaintenanceTasks
320
319
 
321
320
  # Marks a Run as pausing.
322
321
  #
322
+ # If the Run has been stuck on pausing for more than 5 minutes, it forces
323
+ # the transition to paused. The ended_at timestamp will be updated.
324
+ #
323
325
  # Rescues and retries status transition if an ActiveRecord::StaleObjectError
324
326
  # is encountered.
325
- def pausing!
326
- super
327
+ def pause
328
+ if stuck?
329
+ self.status = :paused
330
+ persist_transition
331
+ else
332
+ pausing!
333
+ end
327
334
  rescue ActiveRecord::StaleObjectError
328
335
  reload_status
329
336
  retry
330
337
  end
331
338
 
332
339
  # Returns whether a Run is stuck, which is defined as having a status of
333
- # cancelling, and not having been updated in the last 5 minutes.
340
+ # cancelling or pausing, and not having been updated in the last 5 minutes.
334
341
  #
335
342
  # @return [Boolean] whether the Run is stuck.
336
343
  def stuck?
337
- cancelling? && updated_at <= STUCK_TASK_TIMEOUT.ago
344
+ (cancelling? || pausing?) && updated_at <= MaintenanceTasks.stuck_task_duration.ago
338
345
  end
339
346
 
340
347
  # Performs validation on the task_name attribute.
@@ -187,13 +187,7 @@ module MaintenanceTasks
187
187
  namespace = MaintenanceTasks.tasks_module.safe_constantize
188
188
  return unless namespace
189
189
 
190
- load_const = lambda do |root|
191
- root.constants.each do |name|
192
- object = root.const_get(name)
193
- load_const.call(object) if object.instance_of?(Module)
194
- end
195
- end
196
- load_const.call(namespace)
190
+ Rails.autoloaders.main.eager_load_namespace(namespace)
197
191
  end
198
192
  end
199
193
 
@@ -253,5 +247,55 @@ module MaintenanceTasks
253
247
  def count
254
248
  self.class.collection_builder_strategy.count(self)
255
249
  end
250
+
251
+ # Default enumeration builder. You may override this method to return any
252
+ # Enumerator yielding pairs of `[item, item_cursor]`.
253
+ #
254
+ # @param cursor [String, nil] cursor position to resume from, or nil on
255
+ # initial call.
256
+ #
257
+ # @return [Enumerator]
258
+ def enumerator_builder(cursor:)
259
+ collection = self.collection
260
+
261
+ job_iteration_builder = JobIteration::EnumeratorBuilder.new(nil)
262
+
263
+ case collection
264
+ when :no_collection
265
+ job_iteration_builder.build_once_enumerator(cursor: nil)
266
+ when ActiveRecord::Relation
267
+ job_iteration_builder.active_record_on_records(collection, cursor: cursor)
268
+ when ActiveRecord::Batches::BatchEnumerator
269
+ if collection.start || collection.finish
270
+ raise ArgumentError, <<~MSG.squish
271
+ #{self.class.name}#collection cannot support
272
+ a batch enumerator with the "start" or "finish" options.
273
+ MSG
274
+ end
275
+
276
+ # For now, only support automatic count based on the enumerator for
277
+ # batches
278
+ job_iteration_builder.active_record_on_batch_relations(
279
+ collection.relation,
280
+ cursor: cursor,
281
+ batch_size: collection.batch_size,
282
+ )
283
+ when Array
284
+ job_iteration_builder.build_array_enumerator(collection, cursor: cursor&.to_i)
285
+ when BatchCsvCollectionBuilder::BatchCsv
286
+ JobIteration::CsvEnumerator.new(collection.csv).batches(
287
+ batch_size: collection.batch_size,
288
+ cursor: cursor&.to_i,
289
+ )
290
+ when CSV
291
+ JobIteration::CsvEnumerator.new(collection).rows(cursor: cursor&.to_i)
292
+ else
293
+ raise ArgumentError, <<~MSG.squish
294
+ #{self.class.name}#collection must be either an
295
+ Active Record Relation, ActiveRecord::Batches::BatchEnumerator,
296
+ Array, or CSV.
297
+ MSG
298
+ end
299
+ end
256
300
  end
257
301
  end
@@ -60,6 +60,9 @@ module MaintenanceTasks
60
60
  # interrupted -> errored occurs when the task is deleted while it is
61
61
  # interrupted.
62
62
  "interrupted" => ["running", "pausing", "cancelling", "errored"],
63
+ # errored -> enqueued occurs when the task is retried after encounting an
64
+ # error.
65
+ "errored" => ["enqueued"],
63
66
  }
64
67
 
65
68
  # Validate whether a transition from one Run status
@@ -1,22 +1,4 @@
1
- <% if run.arguments.present? %>
2
- <div class="table-container">
3
- <h6 class="title is-6">Arguments:</h6>
4
- <table class="table">
5
- <tbody>
6
- <% run.arguments.transform_values(&:to_s).each do |key, value| %>
7
- <tr>
8
- <td class="is-family-monospace"><%= key %></td>
9
- <td>
10
- <% next if value.empty? %>
11
- <% if value.include?("\n") %>
12
- <pre><%= value %></pre>
13
- <% else %>
14
- <code><%= value %></code>
15
- <% end %>
16
- </td>
17
- </tr>
18
- <% end %>
19
- </tbody>
20
- </table>
21
- </div>
1
+ <% if arguments.present? %>
2
+ <h6 class="title is-6">Arguments:</h6>
3
+ <%= render "maintenance_tasks/runs/serializable", serializable: arguments %>
22
4
  <% end %>
@@ -0,0 +1,4 @@
1
+ <% if metadata.present? %>
2
+ <h6 class="title is-6">Metadata:</h6>
3
+ <%= render "maintenance_tasks/runs/serializable", serializable: metadata %>
4
+ <% end %>
@@ -2,6 +2,7 @@
2
2
  <h5 class="title is-5">
3
3
  <%= time_tag run.created_at, title: run.created_at.utc.iso8601 %>
4
4
  <%= status_tag run.status %>
5
+ <span class="is-pulled-right" title="Run ID">#<%= run.id %></span>
5
6
  </h5>
6
7
 
7
8
  <%= progress run %>
@@ -16,12 +17,16 @@
16
17
 
17
18
  <%= render "maintenance_tasks/runs/csv", run: run %>
18
19
  <%= tag.hr if run.csv_file.present? && run.arguments.present? %>
19
- <%= render "maintenance_tasks/runs/arguments", run: run %>
20
+ <%= render "maintenance_tasks/runs/arguments", arguments: run.arguments %>
21
+ <%= tag.hr if run.csv_file.present? || run.arguments.present? && run.metadata.present? %>
22
+ <%= render "maintenance_tasks/runs/metadata", metadata: run.metadata %>
20
23
 
21
24
  <div class="buttons">
22
25
  <% if run.paused? %>
23
26
  <%= button_to 'Resume', resume_task_run_path(@task, run), method: :put, class: 'button is-primary', disabled: @task.deleted? %>
24
27
  <%= button_to 'Cancel', cancel_task_run_path(@task, run), method: :put, class: 'button is-danger' %>
28
+ <% elsif run.errored? %>
29
+ <%= button_to 'Resume', resume_task_run_path(@task, run), method: :put, class: 'button is-primary', disabled: @task.deleted? %>
25
30
  <% elsif run.cancelling? %>
26
31
  <% if run.stuck? %>
27
32
  <%= button_to 'Cancel', cancel_task_run_path(@task, run), method: :put, class: 'button is-danger', disabled: @task.deleted? %>
@@ -29,6 +34,9 @@
29
34
  <% elsif run.pausing? %>
30
35
  <%= button_to 'Pausing', pause_task_run_path(@task, run), method: :put, class: 'button is-warning', disabled: true %>
31
36
  <%= button_to 'Cancel', cancel_task_run_path(@task, run), method: :put, class: 'button is-danger' %>
37
+ <% if run.stuck? %>
38
+ <%= button_to 'Force pause', pause_task_run_path(@task, run), method: :put, class: 'button is-danger', disabled: @task.deleted? %>
39
+ <% end %>
32
40
  <% elsif run.active? %>
33
41
  <%= button_to 'Pause', pause_task_run_path(@task, run), method: :put, class: 'button is-warning', disabled: @task.deleted? %>
34
42
  <%= button_to 'Cancel', cancel_task_run_path(@task, run), method: :put, class: 'button is-danger' %>
@@ -0,0 +1,26 @@
1
+ <% if serializable.present? %>
2
+ <% case serializable %>
3
+ <% when Hash %>
4
+ <div class="table-container">
5
+ <table class="table">
6
+ <tbody>
7
+ <% serializable.transform_values(&:to_s).each do |key, value| %>
8
+ <tr>
9
+ <td class="is-family-monospace"><%= key %></td>
10
+ <td>
11
+ <% next if value.empty? %>
12
+ <% if value.include?("\n") %>
13
+ <pre><%= value %></pre>
14
+ <% else %>
15
+ <code><%= value %></code>
16
+ <% end %>
17
+ </td>
18
+ </tr>
19
+ <% end %>
20
+ </tbody>
21
+ </table>
22
+ </div>
23
+ <% else %>
24
+ <code><%= serializable.inspect %></code>
25
+ <% end %>
26
+ <% end %>
@@ -21,6 +21,8 @@
21
21
 
22
22
  <%= render "maintenance_tasks/runs/csv", run: run %>
23
23
  <%= tag.hr if run.csv_file.present? && run.arguments.present? %>
24
- <%= render "maintenance_tasks/runs/arguments", run: run %>
24
+ <%= render "maintenance_tasks/runs/arguments", arguments: run.arguments %>
25
+ <%= tag.hr if run.csv_file.present? || run.arguments.present? && run.metadata.present? %>
26
+ <%= render "maintenance_tasks/runs/metadata", metadata: run.metadata %>
25
27
  <% end %>
26
28
  </div>
@@ -6,10 +6,14 @@ module <%= tasks_module %>
6
6
  <% module_namespacing do -%>
7
7
  RSpec.describe <%= class_name %>Task do
8
8
  describe "#process" do
9
+ <%- if no_collection? -%>
10
+ subject(:process) { described_class.process }
11
+ <%- else -%>
9
12
  subject(:process) { described_class.process(element) }
10
13
  let(:element) {
11
14
  # Object to be processed in a single iteration of this task
12
15
  }
16
+ <%- end -%>
13
17
  pending "add some examples to (or delete) #{__FILE__}"
14
18
  end
15
19
  end
@@ -15,6 +15,12 @@ module MaintenanceTasks
15
15
  end
16
16
 
17
17
  desc "perform [TASK NAME]", "Runs the given Maintenance Task"
18
+ long_desc <<~DESC
19
+ `maintenance_tasks perform` will run the Maintenance Task specified by
20
+ the [TASK NAME] argument.
21
+
22
+ Use `maintenance_tasks list` to get a list of all available tasks.
23
+ DESC
18
24
 
19
25
  # Specify the CSV file to process for CSV Tasks
20
26
  desc = "Supply a CSV file to be processed by a CSV Task, "\
@@ -41,19 +47,14 @@ module MaintenanceTasks
41
47
  say_status(:error, error.message, :red)
42
48
  end
43
49
 
44
- # `long_desc` only allows us to use a static string as "long description".
45
- # By redefining the `#long_description` method on the "perform" Command
46
- # object instead, we make it dynamic, thus delaying the task loading
47
- # process until it's actually required.
48
- commands["perform"].define_singleton_method(:long_description) do
49
- <<~LONGDESC
50
- `maintenance_tasks perform` will run the Maintenance Task specified by
51
- the [TASK NAME] argument.
52
-
53
- Available Tasks:
50
+ desc "list", "Load and list all available tasks."
54
51
 
55
- #{Task.load_all.map(&:name).sort.join("\n\n")}
56
- LONGDESC
52
+ # Command to list all available Tasks.
53
+ #
54
+ # It needs to use `Task.load_all` in order to load all the tasks available
55
+ # in the project before displaying them.
56
+ def list
57
+ say(Task.load_all.map(&:name).sort.join("\n"))
57
58
  end
58
59
 
59
60
  private
@@ -89,4 +89,11 @@ module MaintenanceTasks
89
89
  #
90
90
  # @return [Proc] generates a hash containing the metadata to be stored on the Run
91
91
  mattr_accessor :metadata, default: nil
92
+
93
+ # @!attribute stuck_task_duration
94
+ # @scope class
95
+ # The duration after which a task is considered stuck and can be force cancelled.
96
+ #
97
+ # @return [ActiveSupport::Duration] the threshold in seconds after which a task is considered stuck.
98
+ mattr_accessor :stuck_task_duration, default: 5.minutes
92
99
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: maintenance_tasks
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.3.3
4
+ version: 2.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Shopify Engineering
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2023-10-11 00:00:00.000000000 Z
11
+ date: 2024-01-31 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: actionpack
@@ -80,6 +80,20 @@ dependencies:
80
80
  - - ">="
81
81
  - !ruby/object:Gem::Version
82
82
  version: '6.0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: zeitwerk
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: 2.6.2
90
+ type: :runtime
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: 2.6.2
83
97
  description:
84
98
  email: gems@shopify.com
85
99
  executables:
@@ -114,7 +128,9 @@ files:
114
128
  - app/views/layouts/maintenance_tasks/application.html.erb
115
129
  - app/views/maintenance_tasks/runs/_arguments.html.erb
116
130
  - app/views/maintenance_tasks/runs/_csv.html.erb
131
+ - app/views/maintenance_tasks/runs/_metadata.html.erb
117
132
  - app/views/maintenance_tasks/runs/_run.html.erb
133
+ - app/views/maintenance_tasks/runs/_serializable.html.erb
118
134
  - app/views/maintenance_tasks/runs/info/_cancelled.html.erb
119
135
  - app/views/maintenance_tasks/runs/info/_cancelling.html.erb
120
136
  - app/views/maintenance_tasks/runs/info/_custom.html.erb
@@ -143,7 +159,6 @@ files:
143
159
  - lib/generators/maintenance_tasks/task_generator.rb
144
160
  - lib/generators/maintenance_tasks/templates/csv_task.rb.tt
145
161
  - lib/generators/maintenance_tasks/templates/no_collection_task.rb.tt
146
- - lib/generators/maintenance_tasks/templates/no_collection_task_test.rb.tt
147
162
  - lib/generators/maintenance_tasks/templates/task.rb.tt
148
163
  - lib/generators/maintenance_tasks/templates/task_spec.rb.tt
149
164
  - lib/generators/maintenance_tasks/templates/task_test.rb.tt
@@ -156,7 +171,7 @@ homepage: https://github.com/Shopify/maintenance_tasks
156
171
  licenses:
157
172
  - MIT
158
173
  metadata:
159
- source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v2.3.3
174
+ source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v2.5.0
160
175
  allowed_push_host: https://rubygems.org
161
176
  post_install_message:
162
177
  rdoc_options: []
@@ -173,7 +188,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
173
188
  - !ruby/object:Gem::Version
174
189
  version: '0'
175
190
  requirements: []
176
- rubygems_version: 3.4.20
191
+ rubygems_version: 3.5.5
177
192
  signing_key:
178
193
  specification_version: 4
179
194
  summary: A Rails engine for queuing and managing maintenance tasks
@@ -1,13 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require "test_helper"
4
-
5
- module <%= tasks_module %>
6
- <% module_namespacing do -%>
7
- class <%= class_name %>TaskTest < ActiveSupport::TestCase
8
- # test "#process performs a task iteration" do
9
- # <%= tasks_module %>::<%= class_name %>Task.process
10
- # end
11
- end
12
- <% end -%>
13
- end