maintenance_tasks 1.10.3 → 2.1.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (31) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +94 -50
  3. data/app/controllers/maintenance_tasks/application_controller.rb +1 -2
  4. data/app/controllers/maintenance_tasks/runs_controller.rb +29 -1
  5. data/app/controllers/maintenance_tasks/tasks_controller.rb +8 -21
  6. data/app/helpers/maintenance_tasks/tasks_helper.rb +5 -4
  7. data/app/jobs/concerns/maintenance_tasks/task_job_concern.rb +5 -5
  8. data/app/models/maintenance_tasks/batch_csv_collection_builder.rb +1 -1
  9. data/app/models/maintenance_tasks/run.rb +27 -8
  10. data/app/models/maintenance_tasks/runner.rb +18 -10
  11. data/app/models/maintenance_tasks/runs_page.rb +10 -4
  12. data/app/models/maintenance_tasks/task.rb +7 -13
  13. data/app/models/maintenance_tasks/task_data_index.rb +87 -0
  14. data/app/models/maintenance_tasks/task_data_show.rb +96 -0
  15. data/app/validators/maintenance_tasks/run_status_validator.rb +1 -1
  16. data/app/views/layouts/maintenance_tasks/application.html.erb +2 -2
  17. data/app/views/maintenance_tasks/runs/_run.html.erb +17 -0
  18. data/app/views/maintenance_tasks/tasks/_task.html.erb +1 -1
  19. data/app/views/maintenance_tasks/tasks/show.html.erb +32 -62
  20. data/config/routes.rb +2 -5
  21. data/db/migrate/20211210152329_add_lock_version_to_maintenance_tasks_runs.rb +7 -2
  22. data/db/migrate/20220713131925_add_index_on_task_name_and_status_to_runs.rb +19 -0
  23. data/lib/generators/maintenance_tasks/task_generator.rb +9 -5
  24. data/lib/generators/maintenance_tasks/templates/no_collection_task_test.rb.tt +1 -0
  25. data/lib/generators/maintenance_tasks/templates/task_spec.rb.tt +1 -0
  26. data/lib/generators/maintenance_tasks/templates/task_test.rb.tt +1 -0
  27. data/lib/maintenance_tasks/cli.rb +19 -3
  28. data/lib/maintenance_tasks/engine.rb +3 -13
  29. data/lib/maintenance_tasks.rb +2 -21
  30. metadata +6 -4
  31. data/app/models/maintenance_tasks/task_data.rb +0 -166
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: de43aad0e8d7479e1f7a88d8f664e20213d42455ca85ef22e074e2321b9a183b
4
- data.tar.gz: 65bae8e5b9f793e5cd4dc08dfee03611378007d571d5fc027b349faa7c358b74
3
+ metadata.gz: 2dcb5b54d6f3fd47db28a3f57a487c201e2109b3eb8f78efbe3c972a374999e8
4
+ data.tar.gz: eeb95960a0cac8de08aa3af7d12d47f6cfbee4d875ab74f8989f4d17ef545f6c
5
5
  SHA512:
6
- metadata.gz: 5b088335a8ef4a0e1e92d376f4c5c2d2c05ca2dea2e07aa983f21783ce5a7bff53cf3b4f2a75607d67fea144119078f563882f6e67c6fd09adc3d20ebac285a1
7
- data.tar.gz: '0128f6891b9b0328b7ba80490b4843a70eb014ddb82295d50f7d22218164ac3e9ef1e7c08bb80e8aab0a4501e932a1d374e755e7f21bd709c996f33b3344575f'
6
+ metadata.gz: e99ebde772755c6f3d88d48e65df883805b2ceb4798334f00175bc8e4c83d093bef79931bf0c0c86b303c3d94d2e0b44b3b506b1813ea6e01597ccf47f2c8182
7
+ data.tar.gz: 58d11ad3fa36721e9f29a49cecf586d2f362aaf155667ce6b7702b7acd7bb43f1defa2a2a4ed1fa259d563876b9cec5d5f7b66f4d5dcf43747009e64552fa804
data/README.md CHANGED
@@ -33,8 +33,28 @@ take a look at the [Active Job documentation][active-job-docs].
33
33
  [async-adapter]: https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/AsyncAdapter.html
34
34
  [active-job-docs]: https://guides.rubyonrails.org/active_job_basics.html#setting-the-backend
35
35
 
36
+ ### Autoloading
37
+
38
+ The Maintenance Tasks framework does not support autoloading in `:classic` mode.
39
+ Please ensure your application is using
40
+ [Zeitwerk](https://github.com/fxn/zeitwerk) to load your code. For more
41
+ information, please consult the [Rails guides on autoloading and reloading
42
+ constants](https://guides.rubyonrails.org/autoloading_and_reloading_constants.html).
43
+
36
44
  ## Usage
37
45
 
46
+ The typical Maintenance Tasks workflow is as follows:
47
+
48
+ 1. [Generate a class describing the Task](#creating-a-task) and the work to be done.
49
+ 2. Run the Task
50
+ - either by [using the included web UI](#running-a-task-from-the-web-ui),
51
+ - or by [using the command line](#running-a-task-from-the-command-line),
52
+ - or by [using Ruby](#running-a-task-from-ruby).
53
+ 3. [Monitor the Task](#monitoring-your-tasks-status)
54
+ - either by using the included web UI,
55
+ - or by manually checking your task’s run’s status in your database.
56
+ 4. Optionally, delete the Task code if you no longer need it.
57
+
38
58
  ### Creating a Task
39
59
 
40
60
  A generator is provided to create tasks. Generate a new task by running:
@@ -50,8 +70,13 @@ The generated task is a subclass of `MaintenanceTasks::Task` that implements:
50
70
  * `collection`: return an Active Record Relation or an Array to be iterated
51
71
  over.
52
72
  * `process`: do the work of your maintenance task on a single record
53
- * `count`: return the number of rows that will be iterated over (optional, to be
54
- able to show progress)
73
+
74
+ Optionally, tasks can also implement a custom `#count` method, defining the
75
+ number of elements that will be iterated over. Your task’s `tick_total` will be
76
+ calculated automatically based on the collection size, but this value may be
77
+ overridden if desired using the `#count` method (this might be done, for
78
+ example, to avoid the query that would be produced to determine the size of your
79
+ collection).
55
80
 
56
81
  Example:
57
82
 
@@ -64,10 +89,6 @@ module Maintenance
64
89
  Post.all
65
90
  end
66
91
 
67
- def count
68
- collection.count
69
- end
70
-
71
92
  def process(post)
72
93
  post.update!(content: "New content!")
73
94
  end
@@ -79,10 +100,12 @@ end
79
100
 
80
101
  You can also write a Task that iterates on a CSV file. Note that writing CSV
81
102
  Tasks **requires Active Storage to be configured**. Ensure that the dependency
82
- is specified in your application's Gemfile, and that you've followed the [setup
83
- instuctions][setup].
103
+ is specified in your applications Gemfile, and that youve followed the [setup
104
+ instructions][storage-setup]. See also [Customizing which Active Storage service
105
+ to use][storage-customizing].
84
106
 
85
- [setup]: https://edgeguides.rubyonrails.org/active_storage_overview.html#setup
107
+ [storage-setup]: https://edgeguides.rubyonrails.org/active_storage_overview.html#setup
108
+ [storage-customizing]: #customizing-which-active-storage-service-to-use
86
109
 
87
110
  Generate a CSV Task by running:
88
111
 
@@ -115,12 +138,12 @@ My Title,Hello World!
115
138
  ```
116
139
 
117
140
  The files uploaded to your Active Storage service provider will be renamed
118
- to include an ISO8601 timestamp and the Task name in snake case format.
141
+ to include an ISO 8601 timestamp and the Task name in snake case format.
119
142
  The CSV is expected to have a trailing newline at the end of the file.
120
143
 
121
144
  #### Batch CSV Tasks
122
145
 
123
- Tasks can process CSVs in batches. Add the `in_batches` option to your task's
146
+ Tasks can process CSVs in batches. Add the `in_batches` option to your tasks
124
147
  `csv_collection` macro:
125
148
 
126
149
  ```ruby
@@ -137,12 +160,12 @@ module Maintenance
137
160
  end
138
161
  ```
139
162
 
140
- As with a regular CSV task, ensure you've implemented the following method:
163
+ As with a regular CSV task, ensure youve implemented the following method:
141
164
 
142
165
  * `process`: do the work of your Task on a batch (array of `CSV::Row` objects).
143
166
 
144
167
  Note that `#count` is calculated automatically based on the number of batches in
145
- your collection, and your Task's progress will be displayed in terms of batches
168
+ your collection, and your Tasks progress will be displayed in terms of batches
146
169
  (not the total number of rows in your CSV).
147
170
 
148
171
  ### Processing Batch Collections
@@ -169,13 +192,13 @@ module Maintenance
169
192
  end
170
193
  ```
171
194
 
172
- Ensure that you've implemented the following methods:
195
+ Ensure that youve implemented the following methods:
173
196
 
174
197
  * `collection`: return an `ActiveRecord::Batches::BatchEnumerator`.
175
198
  * `process`: do the work of your Task on a batch (`ActiveRecord::Relation`).
176
199
 
177
200
  Note that `#count` is calculated automatically based on the number of batches in
178
- your collection, and your Task's progress will be displayed in terms of batches
201
+ your collection, and your Tasks progress will be displayed in terms of batches
179
202
  (not the number of records in the relation).
180
203
 
181
204
  **Important!** Batches should only be used if `#process` is performing a batch
@@ -187,7 +210,7 @@ primary keys of the records of the batch first, and then perform an additional
187
210
  query to load the records when calling `each` (or any `Enumerable` method)
188
211
  inside `#process`.
189
212
 
190
- ### Tasks that don't need a Collection
213
+ ### Tasks that dont need a Collection
191
214
 
192
215
  Sometimes, you might want to run a Task that performs a single operation, such
193
216
  as enqueuing another background job or hitting an external API. The gem supports
@@ -238,10 +261,6 @@ module Maintenance
238
261
  Post.all
239
262
  end
240
263
 
241
- def count
242
- collection.count
243
- end
244
-
245
264
  def process(post)
246
265
  post.update!(content: "New content added on #{Time.now.utc}")
247
266
  end
@@ -249,7 +268,7 @@ module Maintenance
249
268
  end
250
269
  ```
251
270
 
252
- Note that it's up to you to define a throttling condition that makes sense for
271
+ Note that its up to you to define a throttling condition that makes sense for
253
272
  your app. Shopify implements `DatabaseStatus.healthy?` to check various MySQL
254
273
  metrics such as replication lag, DB threads, whether DB writes are available,
255
274
  etc.
@@ -258,7 +277,7 @@ Tasks can define multiple throttle conditions. Throttle conditions are inherited
258
277
  by descendants, and new conditions will be appended without impacting existing
259
278
  conditions.
260
279
 
261
- The backoff can also be specified as a proc:
280
+ The backoff can also be specified as a Proc:
262
281
 
263
282
  ```ruby
264
283
  # app/tasks/maintenance/update_posts_throttled_task.rb
@@ -272,11 +291,12 @@ module Maintenance
272
291
  end
273
292
  end
274
293
  ```
294
+
275
295
  ### Custom Task Parameters
276
296
 
277
297
  Tasks may need additional information, supplied via parameters, to run.
278
298
  Parameters can be defined as Active Model Attributes in a Task, and then become
279
- accessible to any of Task's methods: `#collection`, `#count`, or `#process`.
299
+ accessible to any of Tasks methods: `#collection`, `#count`, or `#process`.
280
300
 
281
301
  ```ruby
282
302
  # app/tasks/maintenance/update_posts_via_params_task.rb
@@ -290,10 +310,6 @@ module Maintenance
290
310
  Post.all
291
311
  end
292
312
 
293
- def count
294
- collection.count
295
- end
296
-
297
313
  def process(post)
298
314
  post.update!(content: updated_content)
299
315
  end
@@ -304,7 +320,7 @@ end
304
320
  Tasks can leverage Active Model Validations when defining parameters. Arguments
305
321
  supplied to a Task accepting parameters will be validated before the Task starts
306
322
  to run. Since arguments are specified in the user interface via text area
307
- inputs, it's important to check that they conform to the format your Task
323
+ inputs, its important to check that they conform to the format your Task
308
324
  expects, and to sanitize any inputs if necessary.
309
325
 
310
326
  ### Using Task Callbacks
@@ -337,7 +353,7 @@ end
337
353
  Note: The `after_error` callback is guaranteed to complete,
338
354
  so any exceptions raised in your callback code are ignored.
339
355
  If your `after_error` callback code can raise an exception,
340
- you'll need to rescue it and handle it appropriately
356
+ youll need to rescue it and handle it appropriately
341
357
  within the callback.
342
358
 
343
359
  ```ruby
@@ -386,7 +402,7 @@ depend on the queue adapter but in general, you should follow these rules:
386
402
  safely interrupted and resumed.
387
403
  * Idempotency of `Task#process`: it should be safe to run `process` multiple
388
404
  times for the same element of the collection. Read more in [this Sidekiq best
389
- practice][sidekiq-idempotent]. It's important if the Task errors and you run
405
+ practice][sidekiq-idempotent]. Its important if the Task errors and you run
390
406
  it again, because the same element that errored the Task may well be processed
391
407
  again. It especially matters in the situation described above, when the
392
408
  iteration duration exceeds the timeout: if the job is re-enqueued, multiple
@@ -394,10 +410,24 @@ depend on the queue adapter but in general, you should follow these rules:
394
410
 
395
411
  [sidekiq-idempotent]: https://github.com/mperham/sidekiq/wiki/Best-Practices#2-make-your-job-idempotent-and-transactional
396
412
 
413
+ #### Task object life cycle and memoization
414
+
415
+ When the Task runs or resumes, the Runner enqueues a job, which processes the
416
+ Task. That job will instantiate a Task object which will live for the duration
417
+ of the job. The first time the job runs, it will call `count`. Every time a job
418
+ runs, it will call `collection` on the Task object, and then `process`
419
+ for each item in the collection, until the job stops. The job stops when either the
420
+ collection is finished processing or after the maximum job runtime has expired.
421
+
422
+ This means memoization can be misleading within `process`, since the memoized
423
+ values will be available for subsequent calls to `process` within the same job.
424
+ Still, memoization can be used for throttling or reporting, and you can use [Task
425
+ callbacks](#using-task-callbacks) to persist or log a report for example.
426
+
397
427
  ### Writing tests for a Task
398
428
 
399
429
  The task generator will also create a test file for your task in the folder
400
- `test/tasks/maintenance/`. At a minimum, it's recommended that the `#process`
430
+ `test/tasks/maintenance/`. At a minimum, its recommended that the `#process`
401
431
  method in your task be tested. You may also want to test the `#collection` and
402
432
  `#count` methods for your task if they are sufficiently complex.
403
433
 
@@ -452,7 +482,7 @@ end
452
482
 
453
483
  ### Writing tests for a Task with parameters
454
484
 
455
- Tests for tasks with parameters need to instatiate the task class in order to
485
+ Tests for tasks with parameters need to instantiate the task class in order to
456
486
  assign attributes. Once the task instance is setup, you may test `#process`
457
487
  normally.
458
488
 
@@ -479,21 +509,32 @@ end
479
509
 
480
510
  ### Running a Task
481
511
 
512
+ #### Running a Task from the Web UI
513
+
482
514
  You can run your new Task by accessing the Web UI and clicking on "Run".
483
515
 
516
+ #### Running a Task from the command line
517
+
484
518
  Alternatively, you can run your Task in the command line:
485
519
 
486
520
  ```sh-session
487
521
  bundle exec maintenance_tasks perform Maintenance::UpdatePostsTask
488
522
  ```
489
523
 
490
- To run a Task that processes CSVs from the command line, use the --csv option:
524
+ To run a Task that processes CSVs from the command line, use the `--csv` option:
491
525
 
492
526
  ```sh-session
493
527
  bundle exec maintenance_tasks perform Maintenance::ImportPostsTask --csv "path/to/my_csv.csv"
494
528
  ```
495
529
 
496
- To run a Task that takes arguments from the command line, use the --arguments
530
+ The `--csv` option also works with CSV content coming from the standard input:
531
+
532
+ ```sh-session
533
+ curl "some/remote/csv" |
534
+ bundle exec maintenance_tasks perform Maintenance::ImportPostsTask --csv
535
+ ```
536
+
537
+ To run a Task that takes arguments from the command line, use the `--arguments`
497
538
  option, passing arguments as a set of \<key>:\<value> pairs:
498
539
 
499
540
  ```sh-session
@@ -501,6 +542,8 @@ bundle exec maintenance_tasks perform Maintenance::ParamsTask \
501
542
  --arguments post_ids:1,2,3 content:"Hello, World!"
502
543
  ```
503
544
 
545
+ #### Running a Task from Ruby
546
+
504
547
  You can also run a Task in Ruby by sending `run` with a Task name to Runner:
505
548
 
506
549
  ```ruby
@@ -527,7 +570,7 @@ MaintenanceTasks::Runner.run(
527
570
  )
528
571
  ```
529
572
 
530
- ### Monitoring your Task's status
573
+ ### Monitoring your Tasks status
531
574
 
532
575
  The web UI will provide updates on the status of your Task. Here are the states
533
576
  a Task can be in:
@@ -566,7 +609,7 @@ By default, a running Task will be interrupted after running for more 5 minutes.
566
609
  This is [configured in the `job-iteration` gem][max-job-runtime] and can be
567
610
  tweaked in an initializer if necessary.
568
611
 
569
- [max-job-runtime]: https://github.com/Shopify/job-iteration/blob/master/guides/best-practices.md#max-job-runtime
612
+ [max-job-runtime]: https://github.com/Shopify/job-iteration/blob/-/guides/best-practices.md#max-job-runtime
570
613
 
571
614
  Running tasks will also be interrupted and re-enqueued when needed. For example
572
615
  [when Sidekiq workers shuts down for a deploy][sidekiq-deploy]:
@@ -581,13 +624,13 @@ Running tasks will also be interrupted and re-enqueued when needed. For example
581
624
 
582
625
  When Sidekiq is stopping, it will give workers 25 seconds to finish before
583
626
  forcefully terminating them (this is the default but can be configured with the
584
- `--timeout` option). Before the worker threads are terminated, Sidekiq will try
585
- to re-enqueue the job so your Task will be resumed. However, the position in the
586
- collection won't be persisted so at least one iteration may run again.
627
+ `--timeout` option). Before the worker threads are terminated, Sidekiq will try
628
+ to re-enqueue the job so your Task will be resumed. However, the position in
629
+ the collection wont be persisted so at least one iteration may run again.
587
630
 
588
631
  #### Help! My Task is stuck
589
632
 
590
- Finally, if the queue adapter configured for your application doesn't have this
633
+ Finally, if the queue adapter configured for your application doesnt have this
591
634
  property, or if Sidekiq crashes, is forcefully terminated, or is unable to
592
635
  re-enqueue the jobs that were in progress, the Task may be in a seemingly stuck
593
636
  situation where it appears to be running but is not. In that situation, pausing
@@ -662,7 +705,7 @@ maintenance tasks in your application.
662
705
  ```ruby
663
706
  # config/initializers/maintenance_tasks.rb
664
707
 
665
- MaintenanceTasks.job = 'CustomTaskJob'
708
+ MaintenanceTasks.job = "CustomTaskJob"
666
709
 
667
710
  # app/jobs/custom_task_job.rb
668
711
 
@@ -693,9 +736,9 @@ If no value is specified, it will default to 1 second.
693
736
  #### Customizing which Active Storage service to use
694
737
 
695
738
  The Active Storage framework in Rails 6.1 and up supports multiple storage
696
- services per environment. To specify which service to use,
697
- `MaintenanceTasks.active_storage_service` can be configured with the service's
698
- key, as specified in your application's `config/storage.yml`:
739
+ services. To specify which service to use,
740
+ `MaintenanceTasks.active_storage_service` can be configured with the services
741
+ key, as specified in your applications `config/storage.yml`:
699
742
 
700
743
  ```yaml
701
744
  # config/storage.yml
@@ -720,7 +763,8 @@ MaintenanceTasks.active_storage_service = :internal
720
763
  ```
721
764
 
722
765
  There is no need to configure this option if your application uses only one
723
- storage service per environment.
766
+ storage service. `Rails.configuration.active_storage.service` is used by
767
+ default.
724
768
 
725
769
  #### Customizing the backtrace cleaner
726
770
 
@@ -751,13 +795,13 @@ bin/rails generate maintenance_tasks:install
751
795
 
752
796
  This ensures that new migrations are installed and run as well.
753
797
 
754
- **What if I've deleted my previous Maintenance Task migrations?**
798
+ **What if Ive deleted my previous Maintenance Task migrations?**
755
799
 
756
800
  The install command will attempt to reinstall these old migrations and migrating
757
801
  the database will cause problems. Use `bin/rails
758
- maintenance_tasks:install:migrations` to copy the gem's migrations to your
802
+ maintenance_tasks:install:migrations` to copy the gems migrations to your
759
803
  `db/migrate` folder. Check the release notes to see if any new migrations were
760
- added since your last gem upgrade. Ensure that these are kept, but remove any
804
+ added since your last gem upgrade. Ensure that these are kept, but remove any
761
805
  migrations that already ran.
762
806
 
763
807
  Run the migrations using `bin/rails db:migrate`.
@@ -783,8 +827,8 @@ Once a release is ready, follow these steps:
783
827
  * Deploy via [Shipit][shipit] and see the new version on
784
828
  <https://rubygems.org/gems/maintenance_tasks>.
785
829
  * Ensure the release has documented all changes and publish it.
786
- * Create a new [draft release on GitHub][release] with the title 'Upcoming
787
- Release'. The tag version can be left blank. This will be the starting point
830
+ * Create a new [draft release on GitHub][release] with the title Upcoming
831
+ Release”. The tag version can be left blank. This will be the starting point
788
832
  for documenting changes related to the next release.
789
833
 
790
834
  [release]: https://help.github.com/articles/creating-releases/
@@ -21,8 +21,7 @@ module MaintenanceTasks
21
21
  end
22
22
 
23
23
  before_action do
24
- request.content_security_policy_nonce_generator ||=
25
- ->(_request) { SecureRandom.base64(16) }
24
+ request.content_security_policy_nonce_generator ||= ->(_request) { SecureRandom.base64(16) }
26
25
  request.content_security_policy_nonce_directives = ["style-src"]
27
26
  end
28
27
 
@@ -6,7 +6,25 @@ module MaintenanceTasks
6
6
  #
7
7
  # @api private
8
8
  class RunsController < ApplicationController
9
- before_action :set_run
9
+ before_action :set_run, except: :create
10
+
11
+ # Creates a Run for a given Task and redirects to the Task page.
12
+ def create(&block)
13
+ task = Runner.run(
14
+ name: params.fetch(:task_id),
15
+ csv_file: params[:csv_file],
16
+ arguments: params.fetch(:task_arguments, {}).permit!.to_h,
17
+ &block
18
+ )
19
+ redirect_to(task_path(task))
20
+ rescue ActiveRecord::RecordInvalid => error
21
+ redirect_to(task_path(error.record.task_name), alert: error.message)
22
+ rescue ActiveRecord::ValueTooLong => error
23
+ task_name = params.fetch(:id)
24
+ redirect_to(task_path(task_name), alert: error.message)
25
+ rescue Runner::EnqueuingError => error
26
+ redirect_to(task_path(error.run.task_name), alert: error.message)
27
+ end
10
28
 
11
29
  # Updates a Run status to paused.
12
30
  def pause
@@ -24,6 +42,16 @@ module MaintenanceTasks
24
42
  redirect_to(task_path(@run.task_name), alert: error.message)
25
43
  end
26
44
 
45
+ # Resumes a previously paused Run.
46
+ def resume
47
+ Runner.resume(@run)
48
+ redirect_to(task_path(@run.task_name))
49
+ rescue ActiveRecord::RecordInvalid => error
50
+ redirect_to(task_path(@run.task_name), alert: error.message)
51
+ rescue Runner::EnqueuingError => error
52
+ redirect_to(task_path(@run.task_name), alert: error.message)
53
+ end
54
+
27
55
  private
28
56
 
29
57
  def set_run
@@ -12,33 +12,20 @@ module MaintenanceTasks
12
12
  # Renders the maintenance_tasks/tasks page, displaying
13
13
  # available tasks to users, grouped by category.
14
14
  def index
15
- @available_tasks = TaskData.available_tasks.group_by(&:category)
15
+ @available_tasks = TaskDataIndex.available_tasks.group_by(&:category)
16
16
  end
17
17
 
18
18
  # Renders the page responsible for providing Task actions to users.
19
19
  # Shows running and completed instances of the Task.
20
20
  def show
21
- @task = TaskData.find(params.fetch(:id))
22
- set_refresh if @task.last_run&.active?
23
- @runs_page = RunsPage.new(@task.previous_runs, params[:cursor])
24
- end
25
-
26
- # Runs a given Task and redirects to the Task page.
27
- def run(&block)
28
- task = Runner.run(
29
- name: params.fetch(:id),
30
- csv_file: params[:csv_file],
31
- arguments: params.fetch(:task_arguments, {}).permit!.to_h,
32
- &block
33
- )
34
- redirect_to(task_path(task))
35
- rescue ActiveRecord::RecordInvalid => error
36
- redirect_to(task_path(error.record.task_name), alert: error.message)
37
- rescue ActiveRecord::ValueTooLong => error
38
21
  task_name = params.fetch(:id)
39
- redirect_to(task_path(task_name), alert: error.message)
40
- rescue Runner::EnqueuingError => error
41
- redirect_to(task_path(error.run.task_name), alert: error.message)
22
+ @task = TaskDataShow.new(task_name)
23
+ @task.active_runs.load
24
+ set_refresh if @task.active_runs.any?
25
+ @runs_page = RunsPage.new(@task.completed_runs, params[:cursor])
26
+ if @task.active_runs.none? && @runs_page.records.none?
27
+ Task.named(task_name)
28
+ end
42
29
  end
43
30
 
44
31
  private
@@ -47,7 +47,7 @@ module MaintenanceTasks
47
47
  progress_bar = tag.progress(
48
48
  value: progress.value,
49
49
  max: progress.max,
50
- class: ["progress"] + STATUS_COLOURS.fetch(run.status)
50
+ class: ["progress"] + STATUS_COLOURS.fetch(run.status),
51
51
  )
52
52
  progress_text = tag.p(tag.i(progress.text))
53
53
  tag.div(progress_bar + progress_text, class: "block")
@@ -97,16 +97,17 @@ module MaintenanceTasks
97
97
  def csv_file_download_path(run)
98
98
  Rails.application.routes.url_helpers.rails_blob_path(
99
99
  run.csv_file,
100
- only_path: true
100
+ only_path: true,
101
101
  )
102
102
  end
103
103
 
104
104
  # Return the appropriate field tag for the parameter
105
105
  def parameter_field(form_builder, parameter_name)
106
106
  case form_builder.object.class.attribute_types[parameter_name]
107
- when ActiveModel::Type::Integer, ActiveModel::Type::Decimal,
108
- ActiveModel::Type::Float
107
+ when ActiveModel::Type::Integer
109
108
  form_builder.number_field(parameter_name)
109
+ when ActiveModel::Type::Decimal, ActiveModel::Type::Float
110
+ form_builder.number_field(parameter_name, { step: "any" })
110
111
  when ActiveModel::Type::DateTime
111
112
  form_builder.datetime_field(parameter_name)
112
113
  when ActiveModel::Type::Date
@@ -35,7 +35,7 @@ module MaintenanceTasks
35
35
  collection = @task.collection
36
36
  @enumerator = nil
37
37
 
38
- collection_enum = case collection
38
+ @collection_enum = case collection
39
39
  when :no_collection
40
40
  enumerator_builder.build_once_enumerator(cursor: nil)
41
41
  when ActiveRecord::Relation
@@ -50,7 +50,7 @@ module MaintenanceTasks
50
50
 
51
51
  # For now, only support automatic count based on the enumerator for
52
52
  # batches
53
- @enumerator = enumerator_builder.active_record_on_batch_relations(
53
+ enumerator_builder.active_record_on_batch_relations(
54
54
  collection.relation,
55
55
  cursor: cursor,
56
56
  batch_size: collection.batch_size,
@@ -71,7 +71,7 @@ module MaintenanceTasks
71
71
  Array, or CSV.
72
72
  MSG
73
73
  end
74
- throttle_enumerator(collection_enum)
74
+ throttle_enumerator(@collection_enum)
75
75
  end
76
76
 
77
77
  def throttle_enumerator(collection_enum)
@@ -79,7 +79,7 @@ module MaintenanceTasks
79
79
  enumerator_builder.build_throttle_enumerator(
80
80
  enum,
81
81
  throttle_on: condition[:throttle_on],
82
- backoff: condition[:backoff].call
82
+ backoff: condition[:backoff].call,
83
83
  )
84
84
  end
85
85
  end
@@ -123,7 +123,7 @@ module MaintenanceTasks
123
123
 
124
124
  def on_start
125
125
  count = @task.count
126
- count = @enumerator&.size if count == :no_count
126
+ count = @collection_enum.size if count == :no_count
127
127
  @run.start(count)
128
128
  end
129
129
 
@@ -22,7 +22,7 @@ module MaintenanceTasks
22
22
  def collection(task)
23
23
  BatchCsv.new(
24
24
  csv: CSV.new(task.csv_content, headers: true),
25
- batch_size: @batch_size
25
+ batch_size: @batch_size,
26
26
  )
27
27
  end
28
28
 
@@ -37,18 +37,25 @@ module MaintenanceTasks
37
37
 
38
38
  enum status: STATUSES.to_h { |status| [status, status.to_s] }
39
39
 
40
- validates :task_name, on: :create, inclusion: { in: ->(_) {
41
- Task.available_tasks.map(&:to_s)
42
- } }
40
+ validates :task_name, on: :create, inclusion: {
41
+ in: ->(_) { Task.available_tasks.map(&:to_s) },
42
+ }
43
43
  validate :csv_attachment_presence, on: :create
44
+ validate :csv_content_type, on: :create
44
45
  validate :validate_task_arguments, on: :create
45
46
 
46
47
  attr_readonly :task_name
47
48
 
48
- serialize :backtrace
49
- serialize :arguments, JSON
49
+ if Rails.gem_version >= Gem::Version.new("7.1.alpha")
50
+ serialize :backtrace, coder: YAML
51
+ serialize :arguments, coder: JSON
52
+ else
53
+ serialize :backtrace
54
+ serialize :arguments, JSON
55
+ end
50
56
 
51
57
  scope :active, -> { where(status: ACTIVE_STATUSES) }
58
+ scope :completed, -> { where(status: COMPLETED_STATUSES) }
52
59
 
53
60
  # Ensure ActiveStorage is in use before preloading the attachments
54
61
  scope :with_attached_csv, -> do
@@ -117,7 +124,7 @@ module MaintenanceTasks
117
124
  id,
118
125
  tick_count: number_of_ticks,
119
126
  time_running: duration,
120
- touch: true
127
+ touch: true,
121
128
  )
122
129
  if locking_enabled?
123
130
  locking_column = self.class.locking_column
@@ -346,6 +353,18 @@ module MaintenanceTasks
346
353
  nil
347
354
  end
348
355
 
356
+ # Performs validation on the content type of the :csv_file attachment.
357
+ # A Run for a Task that uses CsvCollection must have a present :csv_file
358
+ # and a content type of "text/csv" to be valid. The appropriate error is
359
+ # added if the Run does not meet the above criteria.
360
+ def csv_content_type
361
+ if csv_file.present? && csv_file.content_type != "text/csv"
362
+ errors.add(:csv_file, "must be a CSV")
363
+ end
364
+ rescue Task::NotFoundError
365
+ nil
366
+ end
367
+
349
368
  # Support iterating over ActiveModel::Errors in Rails 6.0 and Rails 6.1+.
350
369
  # To be removed when Rails 6.0 is no longer supported.
351
370
  if Rails::VERSION::STRING.match?(/^6.0/)
@@ -358,7 +377,7 @@ module MaintenanceTasks
358
377
  .map { |attribute, message| "#{attribute.inspect} #{message}" }
359
378
  errors.add(
360
379
  :arguments,
361
- "are invalid: #{error_messages.join("; ")}"
380
+ "are invalid: #{error_messages.join("; ")}",
362
381
  )
363
382
  end
364
383
  rescue Task::NotFoundError
@@ -374,7 +393,7 @@ module MaintenanceTasks
374
393
  .map { |error| "#{error.attribute.inspect} #{error.message}" }
375
394
  errors.add(
376
395
  :arguments,
377
- "are invalid: #{error_messages.join("; ")}"
396
+ "are invalid: #{error_messages.join("; ")}",
378
397
  )
379
398
  end
380
399
  rescue Task::NotFoundError