maintenance_tasks 1.1.2 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +44 -3
- data/Rakefile +16 -16
- data/app/controllers/maintenance_tasks/application_controller.rb +2 -4
- data/app/controllers/maintenance_tasks/tasks_controller.rb +1 -1
- data/app/helpers/maintenance_tasks/application_helper.rb +2 -13
- data/app/helpers/maintenance_tasks/tasks_helper.rb +17 -16
- data/app/jobs/maintenance_tasks/task_job.rb +13 -8
- data/app/models/maintenance_tasks/csv_collection.rb +1 -1
- data/app/models/maintenance_tasks/progress.rb +13 -11
- data/app/models/maintenance_tasks/run.rb +3 -3
- data/app/models/maintenance_tasks/runner.rb +2 -2
- data/app/models/maintenance_tasks/runs_page.rb +55 -0
- data/app/models/maintenance_tasks/task_data.rb +2 -2
- data/app/validators/maintenance_tasks/run_status_validator.rb +11 -11
- data/app/views/maintenance_tasks/runs/info/_running.html.erb +0 -2
- data/app/views/maintenance_tasks/tasks/show.html.erb +3 -3
- data/config/routes.rb +4 -4
- data/db/migrate/20210225152418_remove_index_on_task_name.rb +1 -1
- data/exe/maintenance_tasks +2 -2
- data/lib/generators/maintenance_tasks/install_generator.rb +4 -4
- data/lib/generators/maintenance_tasks/task_generator.rb +10 -10
- data/lib/maintenance_tasks.rb +17 -13
- data/lib/maintenance_tasks/cli.rb +3 -3
- data/lib/maintenance_tasks/engine.rb +3 -3
- metadata +5 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ed5e344271f9c3342d90720aef36f06a0d62634eb646318afa0bb18e29b0edcc
|
4
|
+
data.tar.gz: 3a32052fb7148cdf403939cbfc154c816285897dd58396db3e4b39d6bb9aa9b0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 34d930320936fb1286687ec4268c1af2ba7cdd3e7a1d704ce45dc68de76ff3f47dece9ad714848b76ad6d33407dbe9204b6d2eb172cc2f1040831ab9a728bfb0
|
7
|
+
data.tar.gz: 612b1a57750e4a3c250d914e915b2389a8931c1b5de19fcd3bf8a3cc2be1bb5f9cc98881380294455c035d71717065fb6b6728a1b839a98cd71c2aaedc890070
|
data/README.md
CHANGED
@@ -22,6 +22,7 @@ A Rails engine for queuing and managing maintenance tasks.
|
|
22
22
|
* [Customizing the maintenance tasks module](#customizing-the-maintenance-tasks-module)
|
23
23
|
* [Customizing the underlying job class](#customizing-the-underlying-job-class)
|
24
24
|
* [Customizing the rate at which task progress gets updated](#customizing-the-rate-at-which-task-progress-gets-updated)
|
25
|
+
* [Customizing which Active Storage service to use](#customizing-which-active-storage-service-to-use)
|
25
26
|
* [Upgrading](#upgrading)
|
26
27
|
* [Contributing](#contributing)
|
27
28
|
* [Releasing new versions](#releasing-new-versions)
|
@@ -338,12 +339,15 @@ end
|
|
338
339
|
|
339
340
|
The error handler should be a lambda that accepts three arguments:
|
340
341
|
|
341
|
-
* `error`: The
|
342
|
+
* `error`: The exception that was raised.
|
342
343
|
* `task_context`: A hash with additional information about the Task and the
|
343
344
|
error:
|
344
345
|
* `task_name`: The name of the Task that errored
|
345
346
|
* `started_at`: The time the Task started
|
346
347
|
* `ended_at`: The time the Task errored
|
348
|
+
Note that `task_context` may be empty if the Task produced an error before any
|
349
|
+
context could be gathered (for example, if deserializing the job to process
|
350
|
+
your Task failed).
|
347
351
|
* `errored_element`: The element, if any, that was being processed when the
|
348
352
|
Task raised an exception. If you would like to pass this object to your
|
349
353
|
exception monitoring service, make sure you **sanitize the object** to avoid
|
@@ -400,6 +404,36 @@ MaintenanceTasks.ticker_delay = 2.seconds
|
|
400
404
|
|
401
405
|
If no value is specified, it will default to 1 second.
|
402
406
|
|
407
|
+
#### Customizing which Active Storage service to use
|
408
|
+
|
409
|
+
The Active Storage framework in Rails 6.1 and up supports multiple storage
|
410
|
+
services per environment. To specify which service to use,
|
411
|
+
`MaintenanceTasks.active_storage_service` can be configured with the service's
|
412
|
+
key, as specified in your application's `config/storage.yml`:
|
413
|
+
|
414
|
+
```yaml
|
415
|
+
# config/storage.yml
|
416
|
+
user_data:
|
417
|
+
service: GCS
|
418
|
+
credentials: <%= Rails.root.join("path/to/user/data/keyfile.json") %>
|
419
|
+
project: "my-project"
|
420
|
+
bucket: "user-data-bucket"
|
421
|
+
|
422
|
+
internal:
|
423
|
+
service: GCS
|
424
|
+
credentials: <%= Rails.root.join("path/to/internal/keyfile.json") %>
|
425
|
+
project: "my-project"
|
426
|
+
bucket: "internal-bucket"
|
427
|
+
```
|
428
|
+
|
429
|
+
```ruby
|
430
|
+
# config/initializers/maintenance_tasks.rb
|
431
|
+
MaintenanceTasks.active_storage_service = :internal
|
432
|
+
```
|
433
|
+
|
434
|
+
There is no need to configure this option if your application uses only one
|
435
|
+
storage service per environment.
|
436
|
+
|
403
437
|
## Upgrading
|
404
438
|
|
405
439
|
Use bundler to check for and upgrade to newer versions. After installing a new
|
@@ -421,13 +455,20 @@ pull requests. You can find the contribution guidelines on
|
|
421
455
|
|
422
456
|
## Releasing new versions
|
423
457
|
|
458
|
+
Updates should be added to the latest draft release on GitHub as Pull Requests
|
459
|
+
are merged.
|
460
|
+
|
461
|
+
Once a release is ready, follow these steps:
|
462
|
+
|
424
463
|
* Update `spec.version` in `maintenance_tasks.gemspec`.
|
425
464
|
* Run `bundle install` to bump the `Gemfile.lock` version of the gem.
|
426
465
|
* Open a PR and merge on approval.
|
427
|
-
* Create a [release on GitHub][release] with a version number that matches the
|
428
|
-
version defined in the gemspec.
|
429
466
|
* Deploy via [Shipit][shipit] and see the new version on
|
430
467
|
<https://rubygems.org/gems/maintenance_tasks>.
|
468
|
+
* Ensure the release has documented all changes and publish it.
|
469
|
+
* Create a new [draft release on GitHub][release] with the title
|
470
|
+
'Upcoming Release'. The tag version can be left blank. This will be the
|
471
|
+
starting point for documenting changes related to the next release.
|
431
472
|
|
432
473
|
[release]: https://help.github.com/articles/creating-releases/
|
433
474
|
[shipit]: https://shipit.shopify.io/shopify/maintenance_tasks/rubygems
|
data/Rakefile
CHANGED
@@ -1,29 +1,29 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
begin
|
3
|
-
require
|
3
|
+
require "bundler/setup"
|
4
4
|
rescue LoadError
|
5
|
-
puts
|
5
|
+
puts "You must `gem install bundler` and `bundle install` to run rake tasks"
|
6
6
|
end
|
7
7
|
|
8
|
-
require
|
8
|
+
require "rdoc/task"
|
9
9
|
RDoc::Task.new(:rdoc) do |rdoc|
|
10
|
-
rdoc.rdoc_dir =
|
11
|
-
rdoc.title =
|
12
|
-
rdoc.options <<
|
13
|
-
rdoc.rdoc_files.include(
|
14
|
-
rdoc.rdoc_files.include(
|
10
|
+
rdoc.rdoc_dir = "rdoc"
|
11
|
+
rdoc.title = "MaintenanceTasks"
|
12
|
+
rdoc.options << "--line-numbers"
|
13
|
+
rdoc.rdoc_files.include("README.md")
|
14
|
+
rdoc.rdoc_files.include("lib/**/*.rb")
|
15
15
|
end
|
16
16
|
|
17
|
-
APP_RAKEFILE = File.expand_path(
|
18
|
-
load(
|
17
|
+
APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
|
18
|
+
load("rails/tasks/engine.rake")
|
19
19
|
|
20
|
-
load(
|
20
|
+
load("rails/tasks/statistics.rake")
|
21
21
|
|
22
|
-
require
|
22
|
+
require "bundler/gem_tasks"
|
23
23
|
|
24
|
-
require
|
24
|
+
require "rubocop/rake_task"
|
25
25
|
RuboCop::RakeTask.new
|
26
26
|
|
27
|
-
task(test:
|
28
|
-
task(
|
29
|
-
task(default: [
|
27
|
+
task(test: "app:test")
|
28
|
+
task("test:system" => "app:test:system")
|
29
|
+
task(default: ["db:test:prepare", "test", "test:system", "rubocop"])
|
@@ -5,9 +5,7 @@ module MaintenanceTasks
|
|
5
5
|
#
|
6
6
|
# Can be extended to add different authentication and authorization code.
|
7
7
|
class ApplicationController < ActionController::Base
|
8
|
-
|
9
|
-
|
10
|
-
BULMA_CDN = 'https://cdn.jsdelivr.net'
|
8
|
+
BULMA_CDN = "https://cdn.jsdelivr.net"
|
11
9
|
|
12
10
|
content_security_policy do |policy|
|
13
11
|
policy.style_src(BULMA_CDN)
|
@@ -17,7 +15,7 @@ module MaintenanceTasks
|
|
17
15
|
before_action do
|
18
16
|
request.content_security_policy_nonce_generator ||=
|
19
17
|
->(_request) { SecureRandom.base64(16) }
|
20
|
-
request.content_security_policy_nonce_directives = [
|
18
|
+
request.content_security_policy_nonce_directives = ["style-src"]
|
21
19
|
end
|
22
20
|
|
23
21
|
protect_from_forgery with: :exception
|
@@ -20,7 +20,7 @@ module MaintenanceTasks
|
|
20
20
|
def show
|
21
21
|
@task = TaskData.find(params.fetch(:id))
|
22
22
|
set_refresh if @task.last_run&.active?
|
23
|
-
@
|
23
|
+
@runs_page = RunsPage.new(@task.previous_runs, params[:cursor])
|
24
24
|
end
|
25
25
|
|
26
26
|
# Runs a given Task and redirects to the Task page.
|
@@ -4,17 +4,6 @@ module MaintenanceTasks
|
|
4
4
|
#
|
5
5
|
# @api private
|
6
6
|
module ApplicationHelper
|
7
|
-
include Pagy::Frontend
|
8
|
-
|
9
|
-
# Renders pagination for the page, if there is more than one page present.
|
10
|
-
#
|
11
|
-
# @param pagy [Pagy] the pagy instance containing pagination details,
|
12
|
-
# including the number of pages the results are spread across.
|
13
|
-
# @return [String] the HTML to render for pagination.
|
14
|
-
def pagination(pagy)
|
15
|
-
raw(pagy_bulma_nav(pagy)) if pagy.pages > 1
|
16
|
-
end
|
17
|
-
|
18
7
|
# Renders a time element with the given datetime, worded as relative to the
|
19
8
|
# current time.
|
20
9
|
#
|
@@ -24,8 +13,8 @@ module MaintenanceTasks
|
|
24
13
|
# @param datetime [ActiveSupport::TimeWithZone] the time to be presented.
|
25
14
|
# @return [String] the HTML to render with the relative datetime in words.
|
26
15
|
def time_ago(datetime)
|
27
|
-
time_tag(datetime, title: datetime.utc.iso8601, class:
|
28
|
-
time_ago_in_words(datetime) +
|
16
|
+
time_tag(datetime, title: datetime.utc.iso8601, class: "is-clickable") do
|
17
|
+
time_ago_in_words(datetime) + " ago"
|
29
18
|
end
|
30
19
|
end
|
31
20
|
end
|
@@ -1,6 +1,6 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
require
|
3
|
+
require "ripper"
|
4
4
|
|
5
5
|
module MaintenanceTasks
|
6
6
|
# Helpers for formatting data in the maintenance_tasks views.
|
@@ -8,16 +8,16 @@ module MaintenanceTasks
|
|
8
8
|
# @api private
|
9
9
|
module TasksHelper
|
10
10
|
STATUS_COLOURS = {
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
11
|
+
"new" => ["is-primary"],
|
12
|
+
"enqueued" => ["is-primary is-light"],
|
13
|
+
"running" => ["is-info"],
|
14
|
+
"interrupted" => ["is-info", "is-light"],
|
15
|
+
"pausing" => ["is-warning", "is-light"],
|
16
|
+
"paused" => ["is-warning"],
|
17
|
+
"succeeded" => ["is-success"],
|
18
|
+
"cancelling" => ["is-light"],
|
19
|
+
"cancelled" => ["is-dark"],
|
20
|
+
"errored" => ["is-danger"],
|
21
21
|
}
|
22
22
|
|
23
23
|
# Formats a run backtrace.
|
@@ -44,12 +44,13 @@ module MaintenanceTasks
|
|
44
44
|
|
45
45
|
progress = Progress.new(run)
|
46
46
|
|
47
|
-
tag.progress(
|
47
|
+
progress_bar = tag.progress(
|
48
48
|
value: progress.value,
|
49
49
|
max: progress.max,
|
50
|
-
|
51
|
-
class: ['progress'] + STATUS_COLOURS.fetch(run.status)
|
50
|
+
class: ["progress"] + STATUS_COLOURS.fetch(run.status)
|
52
51
|
)
|
52
|
+
progress_text = tag.p(tag.i(progress.text))
|
53
|
+
tag.div(progress_bar + progress_text, class: "block")
|
53
54
|
end
|
54
55
|
|
55
56
|
# Renders a span with a Run's status, with the corresponding tag class
|
@@ -59,7 +60,7 @@ module MaintenanceTasks
|
|
59
60
|
# @return [String] the span element containing the status, with the
|
60
61
|
# appropriate tag class attached.
|
61
62
|
def status_tag(status)
|
62
|
-
tag.span(status.capitalize, class: [
|
63
|
+
tag.span(status.capitalize, class: ["tag"] + STATUS_COLOURS.fetch(status))
|
63
64
|
end
|
64
65
|
|
65
66
|
# Returns the distance between now and the Run's expected completion time,
|
@@ -100,7 +101,7 @@ module MaintenanceTasks
|
|
100
101
|
when :on_nl, :on_sp, :on_ignored_nl
|
101
102
|
content
|
102
103
|
else
|
103
|
-
tag.span(content, class: type.to_s.sub(
|
104
|
+
tag.span(content, class: type.to_s.sub("on_", "ruby-").dasherize)
|
104
105
|
end
|
105
106
|
end
|
106
107
|
safe_join(tokens)
|
@@ -19,7 +19,7 @@ module MaintenanceTasks
|
|
19
19
|
# Overrides ActiveJob::Exceptions.retry_on to declare it unsupported.
|
20
20
|
# The use of rescue_from prevents retry_on from being usable.
|
21
21
|
def retry_on(*, **)
|
22
|
-
raise NotImplementedError,
|
22
|
+
raise NotImplementedError, "retry_on is not supported"
|
23
23
|
end
|
24
24
|
end
|
25
25
|
|
@@ -38,7 +38,7 @@ module MaintenanceTasks
|
|
38
38
|
JobIteration::CsvEnumerator.new(collection).rows(cursor: cursor)
|
39
39
|
else
|
40
40
|
raise ArgumentError, "#{@task.class.name}#collection must be either "\
|
41
|
-
|
41
|
+
"an Active Record Relation, Array, or CSV."
|
42
42
|
end
|
43
43
|
end
|
44
44
|
|
@@ -122,13 +122,18 @@ module MaintenanceTasks
|
|
122
122
|
|
123
123
|
def on_error(error)
|
124
124
|
@ticker.persist if defined?(@ticker)
|
125
|
-
@run.persist_error(error)
|
126
125
|
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
|
131
|
-
|
126
|
+
if defined?(@run)
|
127
|
+
@run.persist_error(error)
|
128
|
+
|
129
|
+
task_context = {
|
130
|
+
task_name: @run.task_name,
|
131
|
+
started_at: @run.started_at,
|
132
|
+
ended_at: @run.ended_at,
|
133
|
+
}
|
134
|
+
else
|
135
|
+
task_context = {}
|
136
|
+
end
|
132
137
|
errored_element = @errored_element if defined?(@errored_element)
|
133
138
|
MaintenanceTasks.error_handler.call(error, task_context, errored_element)
|
134
139
|
end
|
@@ -4,6 +4,7 @@ module MaintenanceTasks
|
|
4
4
|
# This class generates progress information about a Run.
|
5
5
|
class Progress
|
6
6
|
include ActiveSupport::NumberHelper
|
7
|
+
include ActionView::Helpers::TextHelper
|
7
8
|
|
8
9
|
# Sets the Progress initial state with a Run.
|
9
10
|
#
|
@@ -41,22 +42,23 @@ module MaintenanceTasks
|
|
41
42
|
estimatable? ? @run.tick_total : @run.tick_count
|
42
43
|
end
|
43
44
|
|
44
|
-
# The
|
45
|
-
#
|
46
|
-
#
|
45
|
+
# The text containing progress information. This describes the progress of
|
46
|
+
# the Run so far. It includes the percentage done out of the maximum, if an
|
47
|
+
# estimate is possible.
|
47
48
|
#
|
48
|
-
# @return [String] the
|
49
|
-
def
|
49
|
+
# @return [String] the text for the Run progress.
|
50
|
+
def text
|
51
|
+
count = @run.tick_count
|
52
|
+
total = @run.tick_total
|
50
53
|
if !total?
|
51
|
-
"Processed #{
|
54
|
+
"Processed #{pluralize(count, 'item')}."
|
52
55
|
elsif over_total?
|
53
|
-
"Processed #{
|
54
|
-
"(expected #{@run.tick_total})."
|
56
|
+
"Processed #{pluralize(count, 'item')} (expected #{total})."
|
55
57
|
else
|
56
|
-
percentage = 100.0 *
|
58
|
+
percentage = 100.0 * count / total
|
57
59
|
|
58
|
-
"Processed #{
|
59
|
-
"(#{number_to_percentage(percentage, precision: 0)})"
|
60
|
+
"Processed #{count} out of #{pluralize(total, 'item')} "\
|
61
|
+
"(#{number_to_percentage(percentage, precision: 0)})."
|
60
62
|
end
|
61
63
|
end
|
62
64
|
|
@@ -48,7 +48,7 @@ module MaintenanceTasks
|
|
48
48
|
|
49
49
|
validates_with RunStatusValidator, on: :update
|
50
50
|
|
51
|
-
has_one_attached :csv_file
|
51
|
+
has_one_attached :csv_file, service: MaintenanceTasks.active_storage_service
|
52
52
|
|
53
53
|
# Sets the run status to enqueued, making sure the transition is validated
|
54
54
|
# in case it's already enqueued.
|
@@ -192,9 +192,9 @@ module MaintenanceTasks
|
|
192
192
|
# if the Run does not meet the above criteria.
|
193
193
|
def csv_attachment_presence
|
194
194
|
if Task.named(task_name) < CsvCollection && !csv_file.attached?
|
195
|
-
errors.add(:csv_file,
|
195
|
+
errors.add(:csv_file, "must be attached to CSV Task.")
|
196
196
|
elsif !(Task.named(task_name) < CsvCollection) && csv_file.present?
|
197
|
-
errors.add(:csv_file,
|
197
|
+
errors.add(:csv_file, "should not be attached to non-CSV Task.")
|
198
198
|
end
|
199
199
|
rescue Task::NotFoundError
|
200
200
|
nil
|
@@ -8,7 +8,7 @@ module MaintenanceTasks
|
|
8
8
|
# @deprecated Use {Runner} directly instead.
|
9
9
|
def new
|
10
10
|
ActiveSupport::Deprecation.warn(
|
11
|
-
|
11
|
+
"Use Runner.run instead of Runner.new.run"
|
12
12
|
)
|
13
13
|
self
|
14
14
|
end
|
@@ -57,7 +57,7 @@ module MaintenanceTasks
|
|
57
57
|
def enqueue(run)
|
58
58
|
unless MaintenanceTasks.job.constantize.perform_later(run)
|
59
59
|
raise "The job to perform #{run.task_name} could not be enqueued. "\
|
60
|
-
|
60
|
+
"Enqueuing has been prevented by a callback."
|
61
61
|
end
|
62
62
|
rescue => error
|
63
63
|
run.persist_error(error)
|
@@ -0,0 +1,55 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
module MaintenanceTasks
|
3
|
+
# This class is responsible for handling cursor-based pagination for Run
|
4
|
+
# records.
|
5
|
+
#
|
6
|
+
# @api private
|
7
|
+
class RunsPage
|
8
|
+
# The number of Runs to show on a single Task page.
|
9
|
+
RUNS_PER_PAGE = 20
|
10
|
+
|
11
|
+
# Initializes a Runs Page with a Runs relation and a cursor. This page is
|
12
|
+
# used by the views to render a set of Runs.
|
13
|
+
# @param runs [ActiveRecord::Relation<MaintenanceTasks::Run>] the relation
|
14
|
+
# of Run records to be paginated.
|
15
|
+
# @param cursor [String, nil] the id that serves as the cursor when
|
16
|
+
# querying the Runs dataset to produce a page of Runs. If nil, the first
|
17
|
+
# Runs in the relation are used.
|
18
|
+
def initialize(runs, cursor)
|
19
|
+
@runs = runs
|
20
|
+
@cursor = cursor
|
21
|
+
end
|
22
|
+
|
23
|
+
# Returns the records for a Page, taking into account the cursor if one is
|
24
|
+
# present. Limits the number of records to 20.
|
25
|
+
#
|
26
|
+
# @return [ActiveRecord::Relation<MaintenanceTasks::Run>] a limited amount
|
27
|
+
# of Run records.
|
28
|
+
def records
|
29
|
+
@records ||= begin
|
30
|
+
runs_after_cursor = if @cursor.present?
|
31
|
+
@runs.where("id < ?", @cursor)
|
32
|
+
else
|
33
|
+
@runs
|
34
|
+
end
|
35
|
+
runs_after_cursor.limit(RUNS_PER_PAGE)
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
# Returns the cursor to use for the next Page of Runs. It is the id of the
|
40
|
+
# last record on the current Page.
|
41
|
+
#
|
42
|
+
# @return [Integer] the id of the last record for the Page.
|
43
|
+
def next_cursor
|
44
|
+
records.last.id
|
45
|
+
end
|
46
|
+
|
47
|
+
# Returns whether this Page is the last one.
|
48
|
+
#
|
49
|
+
# @return [Boolean] whether this Page contains the last Run record in the
|
50
|
+
# Runs dataset that is being paginated.
|
51
|
+
def last?
|
52
|
+
@runs.unscope(:includes).pluck(:id).last == next_cursor
|
53
|
+
end
|
54
|
+
end
|
55
|
+
end
|
@@ -41,7 +41,7 @@ module MaintenanceTasks
|
|
41
41
|
task_names = Task.available_tasks.map(&:name)
|
42
42
|
available_task_runs = Run.where(task_name: task_names)
|
43
43
|
last_runs = Run.where(
|
44
|
-
id: available_task_runs.select(
|
44
|
+
id: available_task_runs.select("MAX(id) as id").group(:task_name)
|
45
45
|
)
|
46
46
|
|
47
47
|
task_names.map do |task_name|
|
@@ -111,7 +111,7 @@ module MaintenanceTasks
|
|
111
111
|
#
|
112
112
|
# @return [String] the Task status.
|
113
113
|
def status
|
114
|
-
last_run&.status ||
|
114
|
+
last_run&.status || "new"
|
115
115
|
end
|
116
116
|
|
117
117
|
# Retrieves the Task's category, which is one of active, new, or completed.
|
@@ -13,7 +13,7 @@ module MaintenanceTasks
|
|
13
13
|
# before starting.
|
14
14
|
# enqueued -> errored occurs when the task job fails to be enqueued, or
|
15
15
|
# if the Task is deleted before is starts running.
|
16
|
-
|
16
|
+
"enqueued" => ["running", "pausing", "cancelling", "errored"],
|
17
17
|
# pausing -> paused occurs when the task actually halts performing and
|
18
18
|
# occupies a status of paused.
|
19
19
|
# pausing -> cancelling occurs when the user cancels a task immediately
|
@@ -24,14 +24,14 @@ module MaintenanceTasks
|
|
24
24
|
# nothing in its collection to process.
|
25
25
|
# pausing -> errored occurs when the job raises an exception after the
|
26
26
|
# user has paused it.
|
27
|
-
|
27
|
+
"pausing" => ["paused", "cancelling", "succeeded", "errored"],
|
28
28
|
# cancelling -> cancelled occurs when the task actually halts performing
|
29
29
|
# and occupies a status of cancelled.
|
30
30
|
# cancelling -> succeeded occurs when the task completes immediately after
|
31
31
|
# being cancelled. See description for pausing -> succeeded.
|
32
32
|
# cancelling -> errored occurs when the job raises an exception after the
|
33
33
|
# user has cancelled it.
|
34
|
-
|
34
|
+
"cancelling" => ["cancelled", "succeeded", "errored"],
|
35
35
|
# running -> succeeded occurs when the task completes successfully.
|
36
36
|
# running -> pausing occurs when a user pauses the task as
|
37
37
|
# it's performing.
|
@@ -40,17 +40,17 @@ module MaintenanceTasks
|
|
40
40
|
# running -> interrupted occurs when the job infra shuts down the task as
|
41
41
|
# it's performing.
|
42
42
|
# running -> errored occurs when the job raises an exception when running.
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
43
|
+
"running" => [
|
44
|
+
"succeeded",
|
45
|
+
"pausing",
|
46
|
+
"cancelling",
|
47
|
+
"interrupted",
|
48
|
+
"errored",
|
49
49
|
],
|
50
50
|
# paused -> enqueued occurs when the task is resumed after being paused.
|
51
51
|
# paused -> cancelling when the user cancels the task after it is paused.
|
52
52
|
# paused -> cancelled when the user cancels the task after it is paused.
|
53
|
-
|
53
|
+
"paused" => ["enqueued", "cancelling", "cancelled"],
|
54
54
|
# interrupted -> running occurs when the task is resumed after being
|
55
55
|
# interrupted by the job infrastructure.
|
56
56
|
# interrupted -> pausing occurs when the task is paused by the user while
|
@@ -59,7 +59,7 @@ module MaintenanceTasks
|
|
59
59
|
# while it is interrupted.
|
60
60
|
# interrupted -> errored occurs when the task is deleted while it is
|
61
61
|
# interrupted.
|
62
|
-
|
62
|
+
"interrupted" => ["running", "pausing", "cancelling", "errored"],
|
63
63
|
}
|
64
64
|
|
65
65
|
# Validate whether a transition from one Run status
|
@@ -41,12 +41,12 @@
|
|
41
41
|
<pre><code><%= highlight_code(code) %></code></pre>
|
42
42
|
<% end %>
|
43
43
|
|
44
|
-
<% if @
|
44
|
+
<% if @runs_page.records.present? %>
|
45
45
|
<hr/>
|
46
46
|
|
47
47
|
<h4 class="title is-4">Previous Runs</h4>
|
48
48
|
|
49
|
-
<%= render @
|
49
|
+
<%= render @runs_page.records %>
|
50
50
|
|
51
|
-
<%=
|
51
|
+
<%= link_to "Next page", task_path(@task, cursor: @runs_page.next_cursor) unless @runs_page.last? %>
|
52
52
|
<% end %>
|
data/config/routes.rb
CHANGED
@@ -2,16 +2,16 @@
|
|
2
2
|
MaintenanceTasks::Engine.routes.draw do
|
3
3
|
resources :tasks, only: [:index, :show], format: false do
|
4
4
|
member do
|
5
|
-
put
|
5
|
+
put "run"
|
6
6
|
end
|
7
7
|
|
8
8
|
resources :runs, only: [], format: false do
|
9
9
|
member do
|
10
|
-
put
|
11
|
-
put
|
10
|
+
put "pause"
|
11
|
+
put "cancel"
|
12
12
|
end
|
13
13
|
end
|
14
14
|
end
|
15
15
|
|
16
|
-
root to:
|
16
|
+
root to: "tasks#index"
|
17
17
|
end
|
data/exe/maintenance_tasks
CHANGED
@@ -2,11 +2,11 @@
|
|
2
2
|
|
3
3
|
# frozen_string_literal: true
|
4
4
|
|
5
|
-
require File.expand_path(
|
5
|
+
require File.expand_path("config/application", Dir.pwd)
|
6
6
|
|
7
7
|
Rails.application.require_environment!
|
8
8
|
|
9
|
-
require
|
9
|
+
require "maintenance_tasks/cli"
|
10
10
|
|
11
11
|
module MaintenanceTasks
|
12
12
|
CLI.start(ARGV)
|
@@ -5,17 +5,17 @@ module MaintenanceTasks
|
|
5
5
|
#
|
6
6
|
# @api private
|
7
7
|
class InstallGenerator < Rails::Generators::Base
|
8
|
-
source_root File.expand_path(
|
8
|
+
source_root File.expand_path("templates", __dir__)
|
9
9
|
|
10
10
|
# Mounts the engine in the host application's config/routes.rb
|
11
11
|
def mount_engine
|
12
|
-
route("mount MaintenanceTasks::Engine =>
|
12
|
+
route("mount MaintenanceTasks::Engine => \"/maintenance_tasks\"")
|
13
13
|
end
|
14
14
|
|
15
15
|
# Copies engine migrations to host application and migrates the database
|
16
16
|
def install_migrations
|
17
|
-
rake(
|
18
|
-
rake(
|
17
|
+
rake("maintenance_tasks:install:migrations")
|
18
|
+
rake("db:migrate")
|
19
19
|
end
|
20
20
|
end
|
21
21
|
end
|
@@ -5,14 +5,14 @@ module MaintenanceTasks
|
|
5
5
|
#
|
6
6
|
# @api private
|
7
7
|
class TaskGenerator < Rails::Generators::NamedBase
|
8
|
-
source_root File.expand_path(
|
9
|
-
desc
|
10
|
-
|
8
|
+
source_root File.expand_path("templates", __dir__)
|
9
|
+
desc "This generator creates a task file at app/tasks and a corresponding "\
|
10
|
+
"test."
|
11
11
|
|
12
12
|
class_option :csv, type: :boolean, default: false,
|
13
|
-
desc:
|
13
|
+
desc: "Generate a CSV Task."
|
14
14
|
|
15
|
-
check_class_collision suffix:
|
15
|
+
check_class_collision suffix: "Task"
|
16
16
|
|
17
17
|
# Creates the Task file.
|
18
18
|
def create_task_file
|
@@ -22,9 +22,9 @@ module MaintenanceTasks
|
|
22
22
|
"#{file_name}_task.rb"
|
23
23
|
)
|
24
24
|
if options[:csv]
|
25
|
-
template(
|
25
|
+
template("csv_task.rb", template_file)
|
26
26
|
else
|
27
|
-
template(
|
27
|
+
template("task.rb", template_file)
|
28
28
|
end
|
29
29
|
end
|
30
30
|
|
@@ -49,7 +49,7 @@ module MaintenanceTasks
|
|
49
49
|
class_path,
|
50
50
|
"#{file_name}_task_test.rb"
|
51
51
|
)
|
52
|
-
template(
|
52
|
+
template("task_test.rb", template_file)
|
53
53
|
end
|
54
54
|
|
55
55
|
def create_task_spec_file
|
@@ -58,11 +58,11 @@ module MaintenanceTasks
|
|
58
58
|
class_path,
|
59
59
|
"#{file_name}_task_spec.rb"
|
60
60
|
)
|
61
|
-
template(
|
61
|
+
template("task_spec.rb", template_file)
|
62
62
|
end
|
63
63
|
|
64
64
|
def file_name
|
65
|
-
super.sub(/_task\z/i,
|
65
|
+
super.sub(/_task\z/i, "")
|
66
66
|
end
|
67
67
|
|
68
68
|
def tasks_module
|
data/lib/maintenance_tasks.rb
CHANGED
@@ -1,17 +1,15 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
|
-
require
|
3
|
-
require
|
4
|
-
require
|
5
|
-
require
|
2
|
+
require "action_controller"
|
3
|
+
require "action_view"
|
4
|
+
require "active_job"
|
5
|
+
require "active_record"
|
6
6
|
|
7
|
-
require
|
8
|
-
require
|
9
|
-
require 'pagy'
|
10
|
-
require 'pagy/extras/bulma'
|
7
|
+
require "job-iteration"
|
8
|
+
require "maintenance_tasks/engine"
|
11
9
|
|
12
10
|
# Force the TaskJob class to load so we can verify upstream compatibility with
|
13
11
|
# the JobIteration gem
|
14
|
-
require_relative
|
12
|
+
require_relative "../app/jobs/maintenance_tasks/task_job"
|
15
13
|
|
16
14
|
# The engine's namespace module. It provides isolation between the host
|
17
15
|
# application's code and the engine-specific code. Top-level engine constants
|
@@ -19,13 +17,13 @@ require_relative '../app/jobs/maintenance_tasks/task_job'
|
|
19
17
|
module MaintenanceTasks
|
20
18
|
# The module to namespace Tasks in, as a String. Defaults to 'Maintenance'.
|
21
19
|
# @param [String] the tasks_module value.
|
22
|
-
mattr_accessor :tasks_module, default:
|
20
|
+
mattr_accessor :tasks_module, default: "Maintenance"
|
23
21
|
|
24
22
|
# Defines the job to be used to perform Tasks. This job must be either
|
25
23
|
# `MaintenanceTasks::TaskJob` or a class that inherits from it.
|
26
24
|
#
|
27
25
|
# @param [String] the name of the job class.
|
28
|
-
mattr_accessor :job, default:
|
26
|
+
mattr_accessor :job, default: "MaintenanceTasks::TaskJob"
|
29
27
|
|
30
28
|
# After each iteration, the progress of the task may be updated. This duration
|
31
29
|
# in seconds limits these updates, skipping if the duration since the last
|
@@ -36,6 +34,12 @@ module MaintenanceTasks
|
|
36
34
|
# the ticker during Task iterations.
|
37
35
|
mattr_accessor :ticker_delay, default: 1.second
|
38
36
|
|
37
|
+
# Specifies which Active Storage service to use for uploading CSV file blobs.
|
38
|
+
#
|
39
|
+
# @param [Symbol] the key for the storage service, as specified in the app's
|
40
|
+
# config/storage.yml.
|
41
|
+
mattr_accessor :active_storage_service
|
42
|
+
|
39
43
|
# Retrieves the callback to be performed when an error occurs in the task.
|
40
44
|
def self.error_handler
|
41
45
|
return @error_handler if defined?(@error_handler)
|
@@ -46,8 +50,8 @@ module MaintenanceTasks
|
|
46
50
|
def self.error_handler=(error_handler)
|
47
51
|
unless error_handler.arity == 3
|
48
52
|
ActiveSupport::Deprecation.warn(
|
49
|
-
|
50
|
-
|
53
|
+
"MaintenanceTasks.error_handler should be a lambda that takes three "\
|
54
|
+
"arguments: error, task_context, and errored_element."
|
51
55
|
)
|
52
56
|
@error_handler = ->(error, _task_context, _errored_element) do
|
53
57
|
error_handler.call(error)
|
@@ -1,6 +1,6 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
require
|
3
|
+
require "thor"
|
4
4
|
|
5
5
|
module MaintenanceTasks
|
6
6
|
# Defines the command line interface commands exposed by Maintenance Tasks in
|
@@ -13,7 +13,7 @@ module MaintenanceTasks
|
|
13
13
|
end
|
14
14
|
end
|
15
15
|
|
16
|
-
desc
|
16
|
+
desc "perform [TASK NAME]", "Runs the given Maintenance Task"
|
17
17
|
|
18
18
|
long_desc <<-LONGDESC
|
19
19
|
`maintenance_tasks perform` will run the Maintenance Task specified by the
|
@@ -25,7 +25,7 @@ module MaintenanceTasks
|
|
25
25
|
LONGDESC
|
26
26
|
|
27
27
|
# Specify the CSV file to process for CSV Tasks
|
28
|
-
option :csv, desc:
|
28
|
+
option :csv, desc: "Supply a CSV file to be processed by a CSV Task, "\
|
29
29
|
'--csv "path/to/csv/file.csv"'
|
30
30
|
|
31
31
|
# Command to run a Task.
|
@@ -1,5 +1,5 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
|
-
require
|
2
|
+
require "active_record/railtie"
|
3
3
|
|
4
4
|
module MaintenanceTasks
|
5
5
|
# The engine's main class, which defines its namespace. The engine is mounted
|
@@ -7,7 +7,7 @@ module MaintenanceTasks
|
|
7
7
|
class Engine < ::Rails::Engine
|
8
8
|
isolate_namespace MaintenanceTasks
|
9
9
|
|
10
|
-
initializer
|
10
|
+
initializer "eager_load_for_classic_autoloader" do
|
11
11
|
eager_load! unless Rails.autoloaders.zeitwerk_enabled?
|
12
12
|
end
|
13
13
|
|
@@ -25,7 +25,7 @@ module MaintenanceTasks
|
|
25
25
|
end
|
26
26
|
|
27
27
|
config.action_dispatch.rescue_responses.merge!(
|
28
|
-
|
28
|
+
"MaintenanceTasks::Task::NotFoundError" => :not_found,
|
29
29
|
)
|
30
30
|
end
|
31
31
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: maintenance_tasks
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Shopify Engineering
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-03-
|
11
|
+
date: 2021-03-29 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: actionpack
|
@@ -66,20 +66,6 @@ dependencies:
|
|
66
66
|
- - "~>"
|
67
67
|
- !ruby/object:Gem::Version
|
68
68
|
version: '1.1'
|
69
|
-
- !ruby/object:Gem::Dependency
|
70
|
-
name: pagy
|
71
|
-
requirement: !ruby/object:Gem::Requirement
|
72
|
-
requirements:
|
73
|
-
- - "~>"
|
74
|
-
- !ruby/object:Gem::Version
|
75
|
-
version: '3.9'
|
76
|
-
type: :runtime
|
77
|
-
prerelease: false
|
78
|
-
version_requirements: !ruby/object:Gem::Requirement
|
79
|
-
requirements:
|
80
|
-
- - "~>"
|
81
|
-
- !ruby/object:Gem::Version
|
82
|
-
version: '3.9'
|
83
69
|
- !ruby/object:Gem::Dependency
|
84
70
|
name: railties
|
85
71
|
requirement: !ruby/object:Gem::Requirement
|
@@ -114,6 +100,7 @@ files:
|
|
114
100
|
- app/models/maintenance_tasks/progress.rb
|
115
101
|
- app/models/maintenance_tasks/run.rb
|
116
102
|
- app/models/maintenance_tasks/runner.rb
|
103
|
+
- app/models/maintenance_tasks/runs_page.rb
|
117
104
|
- app/models/maintenance_tasks/task_data.rb
|
118
105
|
- app/models/maintenance_tasks/ticker.rb
|
119
106
|
- app/tasks/maintenance_tasks/task.rb
|
@@ -151,10 +138,10 @@ files:
|
|
151
138
|
homepage: https://github.com/Shopify/maintenance_tasks
|
152
139
|
licenses: []
|
153
140
|
metadata:
|
154
|
-
source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v1.
|
141
|
+
source_code_uri: https://github.com/Shopify/maintenance_tasks/tree/v1.2.0
|
155
142
|
allowed_push_host: https://rubygems.org
|
156
143
|
post_install_message: |-
|
157
|
-
Thank you for installing Maintenance Tasks 1.
|
144
|
+
Thank you for installing Maintenance Tasks 1.2.0. To complete, please run:
|
158
145
|
|
159
146
|
rails generate maintenance_tasks:install
|
160
147
|
rdoc_options: []
|