maintenance_tasks 1.1.2 → 1.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +44 -3
- data/Rakefile +16 -16
- data/app/controllers/maintenance_tasks/application_controller.rb +2 -4
- data/app/controllers/maintenance_tasks/tasks_controller.rb +1 -1
- data/app/helpers/maintenance_tasks/application_helper.rb +2 -13
- data/app/helpers/maintenance_tasks/tasks_helper.rb +17 -16
- data/app/jobs/maintenance_tasks/task_job.rb +13 -8
- data/app/models/maintenance_tasks/csv_collection.rb +1 -1
- data/app/models/maintenance_tasks/progress.rb +13 -11
- data/app/models/maintenance_tasks/run.rb +3 -3
- data/app/models/maintenance_tasks/runner.rb +2 -2
- data/app/models/maintenance_tasks/runs_page.rb +55 -0
- data/app/models/maintenance_tasks/task_data.rb +2 -2
- data/app/validators/maintenance_tasks/run_status_validator.rb +11 -11
- data/app/views/maintenance_tasks/runs/info/_running.html.erb +0 -2
- data/app/views/maintenance_tasks/tasks/show.html.erb +3 -3
- data/config/routes.rb +4 -4
- data/db/migrate/20210225152418_remove_index_on_task_name.rb +1 -1
- data/exe/maintenance_tasks +2 -2
- data/lib/generators/maintenance_tasks/install_generator.rb +4 -4
- data/lib/generators/maintenance_tasks/task_generator.rb +10 -10
- data/lib/maintenance_tasks.rb +17 -13
- data/lib/maintenance_tasks/cli.rb +3 -3
- data/lib/maintenance_tasks/engine.rb +3 -3
- metadata +5 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ed5e344271f9c3342d90720aef36f06a0d62634eb646318afa0bb18e29b0edcc
|
4
|
+
data.tar.gz: 3a32052fb7148cdf403939cbfc154c816285897dd58396db3e4b39d6bb9aa9b0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 34d930320936fb1286687ec4268c1af2ba7cdd3e7a1d704ce45dc68de76ff3f47dece9ad714848b76ad6d33407dbe9204b6d2eb172cc2f1040831ab9a728bfb0
|
7
|
+
data.tar.gz: 612b1a57750e4a3c250d914e915b2389a8931c1b5de19fcd3bf8a3cc2be1bb5f9cc98881380294455c035d71717065fb6b6728a1b839a98cd71c2aaedc890070
|
data/README.md
CHANGED
@@ -22,6 +22,7 @@ A Rails engine for queuing and managing maintenance tasks.
|
|
22
22
|
* [Customizing the maintenance tasks module](#customizing-the-maintenance-tasks-module)
|
23
23
|
* [Customizing the underlying job class](#customizing-the-underlying-job-class)
|
24
24
|
* [Customizing the rate at which task progress gets updated](#customizing-the-rate-at-which-task-progress-gets-updated)
|
25
|
+
* [Customizing which Active Storage service to use](#customizing-which-active-storage-service-to-use)
|
25
26
|
* [Upgrading](#upgrading)
|
26
27
|
* [Contributing](#contributing)
|
27
28
|
* [Releasing new versions](#releasing-new-versions)
|
@@ -338,12 +339,15 @@ end
|
|
338
339
|
|
339
340
|
The error handler should be a lambda that accepts three arguments:
|
340
341
|
|
341
|
-
* `error`: The
|
342
|
+
* `error`: The exception that was raised.
|
342
343
|
* `task_context`: A hash with additional information about the Task and the
|
343
344
|
error:
|
344
345
|
* `task_name`: The name of the Task that errored
|
345
346
|
* `started_at`: The time the Task started
|
346
347
|
* `ended_at`: The time the Task errored
|
348
|
+
Note that `task_context` may be empty if the Task produced an error before any
|
349
|
+
context could be gathered (for example, if deserializing the job to process
|
350
|
+
your Task failed).
|
347
351
|
* `errored_element`: The element, if any, that was being processed when the
|
348
352
|
Task raised an exception. If you would like to pass this object to your
|
349
353
|
exception monitoring service, make sure you **sanitize the object** to avoid
|
@@ -400,6 +404,36 @@ MaintenanceTasks.ticker_delay = 2.seconds
|
|
400
404
|
|
401
405
|
If no value is specified, it will default to 1 second.
|
402
406
|
|
407
|
+
#### Customizing which Active Storage service to use
|
408
|
+
|
409
|
+
The Active Storage framework in Rails 6.1 and up supports multiple storage
|
410
|
+
services per environment. To specify which service to use,
|
411
|
+
`MaintenanceTasks.active_storage_service` can be configured with the service's
|
412
|
+
key, as specified in your application's `config/storage.yml`:
|
413
|
+
|
414
|
+
```yaml
|
415
|
+
# config/storage.yml
|
416
|
+
user_data:
|
417
|
+
service: GCS
|
418
|
+
credentials: <%= Rails.root.join("path/to/user/data/keyfile.json") %>
|
419
|
+
project: "my-project"
|
420
|
+
bucket: "user-data-bucket"
|
421
|
+
|
422
|
+
internal:
|
423
|
+
service: GCS
|
424
|
+
credentials: <%= Rails.root.join("path/to/internal/keyfile.json") %>
|
425
|
+
project: "my-project"
|
426
|
+
bucket: "internal-bucket"
|
427
|
+
```
|
428
|
+
|
429
|
+
```ruby
|
430
|
+
# config/initializers/maintenance_tasks.rb
|
431
|
+
MaintenanceTasks.active_storage_service = :internal
|
432
|
+
```
|
433
|
+
|
434
|
+
There is no need to configure this option if your application uses only one
|
435
|
+
storage service per environment.
|
436
|
+
|
403
437
|
## Upgrading
|
404
438
|
|
405
439
|
Use bundler to check for and upgrade to newer versions. After installing a new
|
@@ -421,13 +455,20 @@ pull requests. You can find the contribution guidelines on
|
|
421
455
|
|
422
456
|
## Releasing new versions
|
423
457
|
|
458
|
+
Updates should be added to the latest draft release on GitHub as Pull Requests
|
459
|
+
are merged.
|
460
|
+
|
461
|
+
Once a release is ready, follow these steps:
|
462
|
+
|
424
463
|
* Update `spec.version` in `maintenance_tasks.gemspec`.
|
425
464
|
* Run `bundle install` to bump the `Gemfile.lock` version of the gem.
|
426
465
|
* Open a PR and merge on approval.
|
427
|
-
* Create a [release on GitHub][release] with a version number that matches the
|
428
|
-
version defined in the gemspec.
|
429
466
|
* Deploy via [Shipit][shipit] and see the new version on
|
430
467
|
<https://rubygems.org/gems/maintenance_tasks>.
|
468
|
+
* Ensure the release has documented all changes and publish it.
|
469
|
+
* Create a new [draft release on GitHub][release] with the title
|
470
|
+
'Upcoming Release'. The tag version can be left blank. This will be the
|
471
|
+
starting point for documenting changes related to the next release.
|
431
472
|
|
432
473
|
[release]: https://help.github.com/articles/creating-releases/
|
433
474
|
[shipit]: https://shipit.shopify.io/shopify/maintenance_tasks/rubygems
|
data/Rakefile
CHANGED
@@ -1,29 +1,29 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
begin
|
3
|
-
require
|
3
|
+
require "bundler/setup"
|
4
4
|
rescue LoadError
|
5
|
-
puts
|
5
|
+
puts "You must `gem install bundler` and `bundle install` to run rake tasks"
|
6
6
|
end
|
7
7
|
|
8
|
-
require
|
8
|
+
require "rdoc/task"
|
9
9
|
RDoc::Task.new(:rdoc) do |rdoc|
|
10
|
-
rdoc.rdoc_dir =
|
11
|
-
rdoc.title =
|
12
|
-
rdoc.options <<
|
13
|
-
rdoc.rdoc_files.include(
|
14
|
-
rdoc.rdoc_files.include(
|
10
|
+
rdoc.rdoc_dir = "rdoc"
|
11
|
+
rdoc.title = "MaintenanceTasks"
|
12
|
+
rdoc.options << "--line-numbers"
|
13
|
+
rdoc.rdoc_files.include("README.md")
|
14
|
+
rdoc.rdoc_files.include("lib/**/*.rb")
|
15
15
|
end
|
16
16
|
|
17
|
-
APP_RAKEFILE = File.expand_path(
|
18
|
-
load(
|
17
|
+
APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
|
18
|
+
load("rails/tasks/engine.rake")
|
19
19
|
|
20
|
-
load(
|
20
|
+
load("rails/tasks/statistics.rake")
|
21
21
|
|
22
|
-
require
|
22
|
+
require "bundler/gem_tasks"
|
23
23
|
|
24
|
-
require
|
24
|
+
require "rubocop/rake_task"
|
25
25
|
RuboCop::RakeTask.new
|
26
26
|
|
27
|
-
task(test:
|
28
|
-
task(
|
29
|
-
task(default: [
|
27
|
+
task(test: "app:test")
|
28
|
+
task("test:system" => "app:test:system")
|
29
|
+
task(default: ["db:test:prepare", "test", "test:system", "rubocop"])
|
@@ -5,9 +5,7 @@ module MaintenanceTasks
|
|
5
5
|
#
|
6
6
|
# Can be extended to add different authentication and authorization code.
|
7
7
|
class ApplicationController < ActionController::Base
|
8
|
-
|
9
|
-
|
10
|
-
BULMA_CDN = 'https://cdn.jsdelivr.net'
|
8
|
+
BULMA_CDN = "https://cdn.jsdelivr.net"
|
11
9
|
|
12
10
|
content_security_policy do |policy|
|
13
11
|
policy.style_src(BULMA_CDN)
|
@@ -17,7 +15,7 @@ module MaintenanceTasks
|
|
17
15
|
before_action do
|
18
16
|
request.content_security_policy_nonce_generator ||=
|
19
17
|
->(_request) { SecureRandom.base64(16) }
|
20
|
-
request.content_security_policy_nonce_directives = [
|
18
|
+
request.content_security_policy_nonce_directives = ["style-src"]
|
21
19
|
end
|
22
20
|
|
23
21
|
protect_from_forgery with: :exception
|
@@ -20,7 +20,7 @@ module MaintenanceTasks
|
|
20
20
|
def show
|
21
21
|
@task = TaskData.find(params.fetch(:id))
|
22
22
|
set_refresh if @task.last_run&.active?
|
23
|
-
@
|
23
|
+
@runs_page = RunsPage.new(@task.previous_runs, params[:cursor])
|
24
24
|
end
|
25
25
|
|
26
26
|
# Runs a given Task and redirects to the Task page.
|
@@ -4,17 +4,6 @@ module MaintenanceTasks
|
|
4
4
|
#
|
5
5
|
# @api private
|
6
6
|
module ApplicationHelper
|
7
|
-
include Pagy::Frontend
|
8
|
-
|
9
|
-
# Renders pagination for the page, if there is more than one page present.
|
10
|
-
#
|
11
|
-
# @param pagy [Pagy] the pagy instance containing pagination details,
|
12
|
-
# including the number of pages the results are spread across.
|
13
|
-
# @return [String] the HTML to render for pagination.
|
14
|
-
def pagination(pagy)
|
15
|
-
raw(pagy_bulma_nav(pagy)) if pagy.pages > 1
|
16
|
-
end
|
17
|
-
|
18
7
|
# Renders a time element with the given datetime, worded as relative to the
|
19
8
|
# current time.
|
20
9
|
#
|
@@ -24,8 +13,8 @@ module MaintenanceTasks
|
|
24
13
|
# @param datetime [ActiveSupport::TimeWithZone] the time to be presented.
|
25
14
|
# @return [String] the HTML to render with the relative datetime in words.
|
26
15
|
def time_ago(datetime)
|
27
|
-
time_tag(datetime, title: datetime.utc.iso8601, class:
|
28
|
-
time_ago_in_words(datetime) +
|
16
|
+
time_tag(datetime, title: datetime.utc.iso8601, class: "is-clickable") do
|
17
|
+
time_ago_in_words(datetime) + " ago"
|
29
18
|
end
|
30
19
|
end
|
31
20
|
end
|
@@ -1,6 +1,6 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
require
|
3
|
+
require "ripper"
|
4
4
|
|
5
5
|
module MaintenanceTasks
|
6
6
|
# Helpers for formatting data in the maintenance_tasks views.
|
@@ -8,16 +8,16 @@ module MaintenanceTasks
|
|
8
8
|
# @api private
|
9
9
|
module TasksHelper
|
10
10
|
STATUS_COLOURS = {
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
11
|
+
"new" => ["is-primary"],
|
12
|
+
"enqueued" => ["is-primary is-light"],
|
13
|
+
"running" => ["is-info"],
|
14
|
+
"interrupted" => ["is-info", "is-light"],
|
15
|
+
"pausing" => ["is-warning", "is-light"],
|
16
|
+
"paused" => ["is-warning"],
|
17
|
+
"succeeded" => ["is-success"],
|
18
|
+
"cancelling" => ["is-light"],
|
19
|
+
"cancelled" => ["is-dark"],
|
20
|
+
"errored" => ["is-danger"],
|
21
21
|
}
|
22
22
|
|
23
23
|
# Formats a run backtrace.
|
@@ -44,12 +44,13 @@ module MaintenanceTasks
|
|
44
44
|
|
45
45
|
progress = Progress.new(run)
|
46
46
|
|
47
|
-
tag.progress(
|
47
|
+
progress_bar = tag.progress(
|
48
48
|
value: progress.value,
|
49
49
|
max: progress.max,
|
50
|
-
|
51
|
-
class: ['progress'] + STATUS_COLOURS.fetch(run.status)
|
50
|
+
class: ["progress"] + STATUS_COLOURS.fetch(run.status)
|
52
51
|
)
|
52
|
+
progress_text = tag.p(tag.i(progress.text))
|
53
|
+
tag.div(progress_bar + progress_text, class: "block")
|
53
54
|
end
|
54
55
|
|
55
56
|
# Renders a span with a Run's status, with the corresponding tag class
|
@@ -59,7 +60,7 @@ module MaintenanceTasks
|
|
59
60
|
# @return [String] the span element containing the status, with the
|
60
61
|
# appropriate tag class attached.
|
61
62
|
def status_tag(status)
|
62
|
-
tag.span(status.capitalize, class: [
|
63
|
+
tag.span(status.capitalize, class: ["tag"] + STATUS_COLOURS.fetch(status))
|
63
64
|
end
|
64
65
|
|
65
66
|
# Returns the distance between now and the Run's expected completion time,
|
@@ -100,7 +101,7 @@ module MaintenanceTasks
|
|
100
101
|
when :on_nl, :on_sp, :on_ignored_nl
|
101
102
|
content
|
102
103
|
else
|
103
|
-
tag.span(content, class: type.to_s.sub(
|
104
|
+
tag.span(content, class: type.to_s.sub("on_", "ruby-").dasherize)
|
104
105
|
end
|
105
106
|
end
|
106
107
|
safe_join(tokens)
|
@@ -19,7 +19,7 @@ module MaintenanceTasks
|
|
19
19
|
# Overrides ActiveJob::Exceptions.retry_on to declare it unsupported.
|
20
20
|
# The use of rescue_from prevents retry_on from being usable.
|
21
21
|
def retry_on(*, **)
|
22
|
-
raise NotImplementedError,
|
22
|
+
raise NotImplementedError, "retry_on is not supported"
|
23
23
|
end
|
24
24
|
end
|
25
25
|
|
@@ -38,7 +38,7 @@ module MaintenanceTasks
|
|
38
38
|
JobIteration::CsvEnumerator.new(collection).rows(cursor: cursor)
|
39
39
|
else
|
40
40
|
raise ArgumentError, "#{@task.class.name}#collection must be either "\
|
41
|
-
|
41
|
+
"an Active Record Relation, Array, or CSV."
|
42
42
|
end
|
43
43
|
end
|
44
44
|
|
@@ -122,13 +122,18 @@ module MaintenanceTasks
|
|
122
122
|
|
123
123
|
def on_error(error)
|
124
124
|
@ticker.persist if defined?(@ticker)
|
125
|
-
@run.persist_error(error)
|
126
125
|
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
|
131
|
-
|
126
|
+
if defined?(@run)
|
127
|
+
@run.persist_error(error)
|
128
|
+
|
129
|
+
task_context = {
|
130
|
+
task_name: @run.task_name,
|
131
|
+
started_at: @run.started_at,
|
132
|
+
ended_at: @run.ended_at,
|
133
|
+
}
|
134
|
+
else
|
135
|
+
task_context = {}
|
136
|
+
end
|
132
137
|
errored_element = @errored_element if defined?(@errored_element)
|
133
138
|
MaintenanceTasks.error_handler.call(error, task_context, errored_element)
|
134
139
|
end
|
@@ -4,6 +4,7 @@ module MaintenanceTasks
|
|
4
4
|
# This class generates progress information about a Run.
|
5
5
|
class Progress
|
6
6
|
include ActiveSupport::NumberHelper
|
7
|
+
include ActionView::Helpers::TextHelper
|
7
8
|
|
8
9
|
# Sets the Progress initial state with a Run.
|
9
10
|
#
|
@@ -41,22 +42,23 @@ module MaintenanceTasks
|
|
41
42
|
estimatable? ? @run.tick_total : @run.tick_count
|
42
43
|
end
|
43
44
|
|
44
|
-
# The
|
45
|
-
#
|
46
|
-
#
|
45
|
+
# The text containing progress information. This describes the progress of
|
46
|
+
# the Run so far. It includes the percentage done out of the maximum, if an
|
47
|
+
# estimate is possible.
|
47
48
|
#
|
48
|
-
# @return [String] the
|
49
|
-
def
|
49
|
+
# @return [String] the text for the Run progress.
|
50
|
+
def text
|
51
|
+
count = @run.tick_count
|
52
|
+
total = @run.tick_total
|
50
53
|
if !total?
|
51
|
-
"Processed #{
|
54
|
+
"Processed #{pluralize(count, 'item')}."
|
52
55
|
elsif over_total?
|
53
|
-
"Processed #{
|
54
|
-
"(expected #{@run.tick_total})."
|
56
|
+
"Processed #{pluralize(count, 'item')} (expected #{total})."
|
55
57
|
else
|
56
|
-
percentage = 100.0 *
|
58
|
+
percentage = 100.0 * count / total
|
57
59
|
|
58
|
-
"Processed #{
|
59
|
-
"(#{number_to_percentage(percentage, precision: 0)})"
|
60
|
+
"Processed #{count} out of #{pluralize(total, 'item')} "\
|
61
|
+
"(#{number_to_percentage(percentage, precision: 0)})."
|
60
62
|
end
|
61
63
|
end
|
62
64
|
|
@@ -48,7 +48,7 @@ module MaintenanceTasks
|
|
48
48
|
|
49
49
|
validates_with RunStatusValidator, on: :update
|
50
50
|
|
51
|
-
has_one_attached :csv_file
|
51
|
+
has_one_attached :csv_file, service: MaintenanceTasks.active_storage_service
|
52
52
|
|
53
53
|
# Sets the run status to enqueued, making sure the transition is validated
|
54
54
|
# in case it's already enqueued.
|
@@ -192,9 +192,9 @@ module MaintenanceTasks
|
|
192
192
|
# if the Run does not meet the above criteria.
|
193
193
|
def csv_attachment_presence
|
194
194
|
if Task.named(task_name) < CsvCollection && !csv_file.attached?
|
195
|
-
errors.add(:csv_file,
|
195
|
+
errors.add(:csv_file, "must be attached to CSV Task.")
|
196
196
|
elsif !(Task.named(task_name) < CsvCollection) && csv_file.present?
|
197
|
-
errors.add(:csv_file,
|
197
|
+
errors.add(:csv_file, "should not be attached to non-CSV Task.")
|
198
198
|
end
|
199
199
|
rescue Task::NotFoundError
|
200
200
|
nil
|
@@ -8,7 +8,7 @@ module MaintenanceTasks
|
|
8
8
|
# @deprecated Use {Runner} directly instead.
|
9
9
|
def new
|
10
10
|
ActiveSupport::Deprecation.warn(
|
11
|
-
|
11
|
+
"Use Runner.run instead of Runner.new.run"
|
12
12
|
)
|
13
13
|
self
|
14
14
|
end
|
@@ -57,7 +57,7 @@ module MaintenanceTasks
|
|
57
57
|
def enqueue(run)
|
58
58
|
unless MaintenanceTasks.job.constantize.perform_later(run)
|
59
59
|
raise "The job to perform #{run.task_name} could not be enqueued. "\
|
60
|
-
|
60
|
+
"Enqueuing has been prevented by a callback."
|
61
61
|
end
|
62
62
|
rescue => error
|
63
63
|
run.persist_error(error)
|
@@ -0,0 +1,55 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
module MaintenanceTasks
|
3
|
+
# This class is responsible for handling cursor-based pagination for Run
|
4
|
+
# records.
|
5
|
+
#
|
6
|
+
# @api private
|
7
|
+
class RunsPage
|
8
|
+
# The number of Runs to show on a single Task page.
|
9
|
+
RUNS_PER_PAGE = 20
|
10
|
+
|
11
|
+
# Initializes a Runs Page with a Runs relation and a cursor. This page is
|
12
|
+
# used by the views to render a set of Runs.
|
13
|
+
# @param runs [ActiveRecord::Relation<MaintenanceTasks::Run>] the relation
|
14
|
+
# of Run records to be paginated.
|
15
|
+
# @param cursor [String, nil] the id that serves as the cursor when
|
16
|
+
# querying the Runs dataset to produce a page of Runs. If nil, the first
|
17
|
+
# Runs in the relation are used.
|
18
|
+
def initialize(runs, cursor)
|
19
|
+
@runs = runs
|
20
|
+
@cursor = cursor
|
21
|
+
end
|
22
|
+
|
23
|
+
# Returns the records for a Page, taking into account the cursor if one is
|
24
|
+
# present. Limits the number of records to 20.
|
25
|
+
#
|
26
|
+
# @return [ActiveRecord::Relation<MaintenanceTasks::Run>] a limited amount
|
27
|
+
# of Run records.
|
28
|
+
def records
|
29
|
+
@records ||= begin
|
30
|
+
runs_after_cursor = if @cursor.present?
|
31
|
+
@runs.where("id < ?", @cursor)
|
32
|
+
else
|
33
|
+
@runs
|
34
|
+
end
|
35
|
+
runs_after_cursor.limit(RUNS_PER_PAGE)
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
# Returns the cursor to use for the next Page of Runs. It is the id of the
|
40
|
+
# last record on the current Page.
|
41
|
+
#
|
42
|
+
# @return [Integer] the id of the last record for the Page.
|
43
|
+
def next_cursor
|
44
|
+
records.last.id
|
45
|
+
end
|
46
|
+
|
47
|
+
# Returns whether this Page is the last one.
|
48
|
+
#
|
49
|
+
# @return [Boolean] whether this Page contains the last Run record in the
|
50
|
+
# Runs dataset that is being paginated.
|
51
|
+
def last?
|
52
|
+
@runs.unscope(:includes).pluck(:id).last == next_cursor
|
53
|
+
end
|
54
|
+
end
|
55
|
+
end
|
@@ -41,7 +41,7 @@ module MaintenanceTasks
|
|
41
41
|
task_names = Task.available_tasks.map(&:name)
|
42
42
|
available_task_runs = Run.where(task_name: task_names)
|
43
43
|
last_runs = Run.where(
|
44
|
-
id: available_task_runs.select(
|
44
|
+
id: available_task_runs.select("MAX(id) as id").group(:task_name)
|
45
45
|
)
|
46
46
|
|
47
47
|
task_names.map do |task_name|
|
@@ -111,7 +111,7 @@ module MaintenanceTasks
|
|
111
111
|
#
|
112
112
|
# @return [String] the Task status.
|
113
113
|
def status
|
114
|
-
last_run&.status ||
|
114
|
+
last_run&.status || "new"
|
115
115
|
end
|
116
116
|
|
117
117
|
# Retrieves the Task's category, which is one of active, new, or completed.
|
@@ -13,7 +13,7 @@ module MaintenanceTasks
|
|
13
13
|
# before starting.
|
14
14
|
# enqueued -> errored occurs when the task job fails to be enqueued, or
|
15
15
|
# if the Task is deleted before is starts running.
|
16
|
-
|
16
|
+
"enqueued" => ["running", "pausing", "cancelling", "errored"],
|
17
17
|
# pausing -> paused occurs when the task actually halts performing and
|
18
18
|
# occupies a status of paused.
|
19
19
|
# pausing -> cancelling occurs when the user cancels a task immediately
|
@@ -24,14 +24,14 @@ module MaintenanceTasks
|
|
24
24
|
# nothing in its collection to process.
|
25
25
|
# pausing -> errored occurs when the job raises an exception after the
|
26
26
|
# user has paused it.
|
27
|
-
|
27
|
+
"pausing" => ["paused", "cancelling", "succeeded", "errored"],
|
28
28
|
# cancelling -> cancelled occurs when the task actually halts performing
|
29
29
|
# and occupies a status of cancelled.
|
30
30
|
# cancelling -> succeeded occurs when the task completes immediately after
|
31
31
|
# being cancelled. See description for pausing -> succeeded.
|
32
32
|
# cancelling -> errored occurs when the job raises an exception after the
|
33
33
|
# user has cancelled it.
|
34
|
-
|
34
|
+
"cancelling" => ["cancelled", "succeeded", "errored"],
|
35
35
|
# running -> succeeded occurs when the task completes successfully.
|
36
36
|
# running -> pausing occurs when a user pauses the task as
|
37
37
|
# it's performing.
|
@@ -40,17 +40,17 @@ module MaintenanceTasks
|
|
40
40
|
# running -> interrupted occurs when the job infra shuts down the task as
|
41
41
|
# it's performing.
|
42
42
|
# running -> errored occurs when the job raises an exception when running.
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
43
|
+
"running" => [
|
44
|
+
"succeeded",
|
45
|
+
"pausing",
|
46
|
+
"cancelling",
|
47
|
+
"interrupted",
|
48
|
+
"errored",
|
49
49
|
],
|
50
50
|
# paused -> enqueued occurs when the task is resumed after being paused.
|
51
51
|
# paused -> cancelling when the user cancels the task after it is paused.
|
52
52
|
# paused -> cancelled when the user cancels the task after it is paused.
|
53
|
-
|
53
|
+
"paused" => ["enqueued", "cancelling", "cancelled"],
|
54
54
|
# interrupted -> running occurs when the task is resumed after being
|
55
55
|
# interrupted by the job infrastructure.
|
56
56
|
# interrupted -> pausing occurs when the task is paused by the user while
|
@@ -59,7 +59,7 @@ module MaintenanceTasks
|
|
59
59
|
# while it is interrupted.
|
60
60
|
# interrupted -> errored occurs when the task is deleted while it is
|
61
61
|
# interrupted.
|
62
|
-
|
62
|
+
"interrupted" => ["running", "pausing", "cancelling", "errored"],
|
63
63
|
}
|
64
64
|
|
65
65
|
# Validate whether a transition from one Run status
|
@@ -41,12 +41,12 @@
|
|
41
41
|
<pre><code><%= highlight_code(code) %></code></pre>
|
42
42
|
<% end %>
|
43
43
|
|
44
|
-
<% if @
|
44
|
+
<% if @runs_page.records.present? %>
|
45
45
|
<hr/>
|
46
46
|
|
47
47
|
<h4 class="title is-4">Previous Runs</h4>
|
48
48
|
|
49
|
-
<%= render @
|
49
|
+
<%= render @runs_page.records %>
|
50
50
|
|
51
|
-
<%=
|
51
|
+
<%= link_to "Next page", task_path(@task, cursor: @runs_page.next_cursor) unless @runs_page.last? %>
|
52
52
|
<% end %>
|
data/config/routes.rb
CHANGED
@@ -2,16 +2,16 @@
|
|
2
2
|
MaintenanceTasks::Engine.routes.draw do
|
3
3
|
resources :tasks, only: [:index, :show], format: false do
|
4
4
|
member do
|
5
|
-
put
|
5
|
+
put "run"
|
6
6
|
end
|
7
7
|
|
8
8
|
resources :runs, only: [], format: false do
|
9
9
|
member do
|
10
|
-
put
|
11
|
-
put
|
10
|
+
put "pause"
|
11
|
+
put "cancel"
|
12
12
|
end
|
13
13
|
end
|
14
14
|
end
|
15
15
|
|
16
|
-
root to:
|
16
|
+
root to: "tasks#index"
|
17
17
|
end
|
data/exe/maintenance_tasks
CHANGED
@@ -2,11 +2,11 @@
|
|
2
2
|
|
3
3
|
# frozen_string_literal: true
|
4
4
|
|
5
|
-
require File.expand_path(
|
5
|
+
require File.expand_path("config/application", Dir.pwd)
|
6
6
|
|
7
7
|
Rails.application.require_environment!
|
8
8
|
|
9
|
-
require
|
9
|
+
require "maintenance_tasks/cli"
|
10
10
|
|
11
11
|
module MaintenanceTasks
|
12
12
|
CLI.start(ARGV)
|
@@ -5,17 +5,17 @@ module MaintenanceTasks
|
|
5
5
|
#
|
6
6
|
# @api private
|
7
7
|
class InstallGenerator < Rails::Generators::Base
|
8
|
-
source_root File.expand_path(
|
8
|
+
source_root File.expand_path("templates", __dir__)
|
9
9
|
|
10
10
|
# Mounts the engine in the host application's config/routes.rb
|
11
11
|
def mount_engine
|
12
|
-
route("mount MaintenanceTasks::Engine =>
|
12
|
+
route("mount MaintenanceTasks::Engine => \"/maintenance_tasks\"")
|
13
13
|
end
|
14
14
|
|
15
15
|
# Copies engine migrations to host application and migrates the database
|
16
16
|
def install_migrations
|
17
|
-
rake(
|
18
|
-
rake(
|
17
|
+
rake("maintenance_tasks:install:migrations")
|
18
|
+
rake("db:migrate")
|
19
19
|
end
|
20
20
|
end
|
21
21
|
end
|
@@ -5,14 +5,14 @@ module MaintenanceTasks
|
|
5
5
|
#
|
6
6
|
# @api private
|
7
7
|
class TaskGenerator < Rails::Generators::NamedBase
|
8
|
-
source_root File.expand_path(
|
9
|
-
desc
|
10
|
-
|
8
|
+
source_root File.expand_path("templates", __dir__)
|
9
|
+
desc "This generator creates a task file at app/tasks and a corresponding "\
|
10
|
+
"test."
|
11
11
|
|
12
12
|
class_option :csv, type: :boolean, default: false,
|
13
|
-
desc:
|
13
|
+
desc: "Generate a CSV Task."
|
14
14
|
|
15
|
-
check_class_collision suffix:
|
15
|
+
check_class_collision suffix: "Task"
|
16
16
|
|
17
17
|
# Creates the Task file.
|
18
18
|
def create_task_file
|
@@ -22,9 +22,9 @@ module MaintenanceTasks
|
|
22
22
|
"#{file_name}_task.rb"
|
23
23
|
)
|
24
24
|
if options[:csv]
|
25
|
-
template(
|
25
|
+
template("csv_task.rb", template_file)
|
26
26
|
else
|
27
|
-
template(
|
27
|
+
template("task.rb", template_file)
|
28
28
|
end
|
29
29
|
end
|
30
30
|
|
@@ -49,7 +49,7 @@ module MaintenanceTasks
|
|
49
49
|
class_path,
|
50
50
|
"#{file_name}_task_test.rb"
|
51
51
|
)
|
52
|
-
template(
|
52
|
+
template("task_test.rb", template_file)
|
53
53
|
end
|
54
54
|
|
55
55
|
def create_task_spec_file
|
@@ -58,11 +58,11 @@ module MaintenanceTasks
|
|
58
58
|
class_path,
|
59
59
|
"#{file_name}_task_spec.rb"
|
60
60
|
)
|
61
|
-
template(
|
61
|
+
template("task_spec.rb", template_file)
|
62
62
|
end
|
63
63
|
|
64
64
|
def file_name
|
65
|
-
super.sub(/_task\z/i,
|
65
|
+
super.sub(/_task\z/i, "")
|
66
66
|
end
|
67
67
|
|
68
68
|
def tasks_module
|
data/lib/maintenance_tasks.rb
CHANGED
@@ -1,17 +1,15 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
|
-
require
|
3
|
-
require
|
4
|
-
require
|
5
|
-
require
|
2
|
+
require "action_controller"
|
3
|
+
require "action_view"
|
4
|
+
require "active_job"
|
5
|
+
require "active_record"
|
6
6
|
|
7
|
-
require
|
8
|
-
require
|
9
|
-
require 'pagy'
|
10
|
-
require 'pagy/extras/bulma'
|
7
|
+
require "job-iteration"
|
8
|
+
require "maintenance_tasks/engine"
|
11
9
|
|
12
10
|
# Force the TaskJob class to load so we can verify upstream compatibility with
|
13
11
|
# the JobIteration gem
|
14
|
-
require_relative
|
12
|
+
require_relative "../app/jobs/maintenance_tasks/task_job"
|
15
13
|
|
16
14
|
# The engine's namespace module. It provides isolation between the host
|
17
15
|
# application's code and the engine-specific code. Top-level engine constants
|
@@ -19,13 +17,13 @@ require_relative '../app/jobs/maintenance_tasks/task_job'
|
|
19
17
|
module MaintenanceTasks
|
20
18
|
# The module to namespace Tasks in, as a String. Defaults to 'Maintenance'.
|
21
19
|
# @param [String] the tasks_module value.
|
22
|
-
mattr_accessor :tasks_module, default:
|
20
|
+
mattr_accessor :tasks_module, default: "Maintenance"
|
23
21
|
|
24
22
|
# Defines the job to be used to perform Tasks. This job must be either
|
25
23
|
# `MaintenanceTasks::TaskJob` or a class that inherits from it.
|
26
24
|
#
|
27
25
|
# @param [String] the name of the job class.
|
28
|
-
mattr_accessor :job, default:
|
26
|
+
mattr_accessor :job, default: "MaintenanceTasks::TaskJob"
|
29
27
|
|
30
28
|
# After each iteration, the progress of the task may be updated. This duration
|
31
29
|
# in seconds limits these updates, skipping if the duration since the last
|
@@ -36,6 +34,12 @@ module MaintenanceTasks
|
|
36
34
|
# the ticker during Task iterations.
|
37
35
|
mattr_accessor :ticker_delay, default: 1.second
|
38
36
|
|
37
|
+
# Specifies which Active Storage service to use for uploading CSV file blobs.
|
38
|
+
#
|
39
|
+
# @param [Symbol] the key for the storage service, as specified in the app's
|
40
|
+
# config/storage.yml.
|
41
|
+
mattr_accessor :active_storage_service
|
42
|
+
|
39
43
|
# Retrieves the callback to be performed when an error occurs in the task.
|
40
44
|
def self.error_handler
|
41
45
|
return @error_handler if defined?(@error_handler)
|
@@ -46,8 +50,8 @@ module MaintenanceTasks
|
|
46
50
|
def self.error_handler=(error_handler)
|
47
51
|
unless error_handler.arity == 3
|
48
52
|
ActiveSupport::Deprecation.warn(
|
49
|
-
|
50
|
-
|
53
|
+
"MaintenanceTasks.error_handler should be a lambda that takes three "\
|
54
|
+
"arguments: error, task_context, and errored_element."
|
51
55
|
)
|
52
56
|
@error_handler = ->(error, _task_context, _errored_element) do
|
53
57
|
error_handler.call(error)
|
@@ -1,6 +1,6 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
require
|
3
|
+
require "thor"
|
4
4
|
|
5
5
|
module MaintenanceTasks
|
6
6
|
# Defines the command line interface commands exposed by Maintenance Tasks in
|
@@ -13,7 +13,7 @@ module MaintenanceTasks
|
|
13
13
|
end
|
14
14
|
end
|
15
15
|
|
16
|
-
desc
|
16
|
+
desc "perform [TASK NAME]", "Runs the given Maintenance Task"
|
17
17
|
|
18
18
|
long_desc <<-LONGDESC
|
19
19
|
`maintenance_tasks perform` will run the Maintenance Task specified by the
|
@@ -25,7 +25,7 @@ module MaintenanceTasks
|
|
25
25
|
LONGDESC
|
26
26
|
|
27
27
|
# Specify the CSV file to process for CSV Tasks
|
28
|
-
option :csv, desc:
|
28
|
+
option :csv, desc: "Supply a CSV file to be processed by a CSV Task, "\
|
29
29
|
'--csv "path/to/csv/file.csv"'
|
30
30
|
|
31
31
|
# Command to run a Task.
|
@@ -1,5 +1,5 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
|
-
require
|
2
|
+
require "active_record/railtie"
|
3
3
|
|
4
4
|
module MaintenanceTasks
|
5
5
|
# The engine's main class, which defines its namespace. The engine is mounted
|
@@ -7,7 +7,7 @@ module MaintenanceTasks
|
|
7
7
|
class Engine < ::Rails::Engine
|
8
8
|
isolate_namespace MaintenanceTasks
|
9
9
|
|
10
|
-
initializer
|
10
|
+
initializer "eager_load_for_classic_autoloader" do
|
11
11
|
eager_load! unless Rails.autoloaders.zeitwerk_enabled?
|
12
12
|
end
|
13
13
|
|
@@ -25,7 +25,7 @@ module MaintenanceTasks
|
|
25
25
|
end
|
26
26
|
|
27
27
|
config.action_dispatch.rescue_responses.merge!(
|
28
|
-
|
28
|
+
"MaintenanceTasks::Task::NotFoundError" => :not_found,
|
29
29
|
)
|
30
30
|
end
|
31
31
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: maintenance_tasks
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Shopify Engineering
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-03-
|
11
|
+
date: 2021-03-29 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: actionpack
|
@@ -66,20 +66,6 @@ dependencies:
|
|
66
66
|
- - "~>"
|
67
67
|
- !ruby/object:Gem::Version
|
68
68
|
version: '1.1'
|
69
|
-
- !ruby/object:Gem::Dependency
|
70
|
-
name: pagy
|
71
|
-
requirement: !ruby/object:Gem::Requirement
|
72
|
-
requirements:
|
73
|
-
- - "~>"
|
74
|
-
- !ruby/object:Gem::Version
|
75
|
-
version: '3.9'
|
76
|
-
type: :runtime
|
77
|
-
prerelease: false
|
78
|
-
version_requirements: !ruby/object:Gem::Requirement
|
79
|
-
requirements:
|
80
|
-
- - "~>"
|
81
|
-
- !ruby/object:Gem::Version
|
82
|
-
version: '3.9'
|
83
69
|
- !ruby/object:Gem::Dependency
|
84
70
|
name: railties
|
85
71
|
requirement: !ruby/object:Gem::Requirement
|
@@ -114,6 +100,7 @@ files:
|
|
114
100
|
- app/models/maintenance_tasks/progress.rb
|
115
101
|
- app/models/maintenance_tasks/run.rb
|
116
102
|
- app/models/maintenance_tasks/runner.rb
|
103
|
+
- app/models/maintenance_tasks/runs_page.rb
|
117
104
|
- app/models/maintenance_tasks/task_data.rb
|
118
105
|
- app/models/maintenance_tasks/ticker.rb
|
119
106
|
- app/tasks/maintenance_tasks/task.rb
|
@@ -151,10 +138,10 @@ files:
|
|
151
138
|
homepage: https://github.com/Shopify/maintenance_tasks
|
152
139
|
licenses: []
|
153
140
|
metadata:
|
154
|
-
source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v1.
|
141
|
+
source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v1.2.0
|
155
142
|
allowed_push_host: https://rubygems.org
|
156
143
|
post_install_message: |-
|
157
|
-
Thank you for installing Maintenance Tasks 1.
|
144
|
+
Thank you for installing Maintenance Tasks 1.2.0. To complete, please run:
|
158
145
|
|
159
146
|
rails generate maintenance_tasks:install
|
160
147
|
rdoc_options: []
|