maintenance_on_steroids 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +267 -0
- data/LICENSE.txt +21 -0
- data/README.md +948 -0
- data/app/controllers/maintenance_on_steroids/application_controller.rb +66 -0
- data/app/controllers/maintenance_on_steroids/dashboard_controller.rb +18 -0
- data/app/controllers/maintenance_on_steroids/jobs_controller.rb +59 -0
- data/app/controllers/maintenance_on_steroids/runs_controller.rb +223 -0
- data/app/jobs/maintenance_on_steroids/run_job.rb +292 -0
- data/app/models/maintenance_on_steroids/application_record.rb +6 -0
- data/app/models/maintenance_on_steroids/artifact.rb +255 -0
- data/app/models/maintenance_on_steroids/run.rb +310 -0
- data/app/views/layouts/maintenance_on_steroids/application.html.erb +48 -0
- data/app/views/maintenance_on_steroids/dashboard/index.html.erb +163 -0
- data/app/views/maintenance_on_steroids/jobs/index.html.erb +35 -0
- data/app/views/maintenance_on_steroids/jobs/show.html.erb +107 -0
- data/app/views/maintenance_on_steroids/jobs/source.html.erb +79 -0
- data/app/views/maintenance_on_steroids/runs/new.html.erb +98 -0
- data/app/views/maintenance_on_steroids/runs/show.html.erb +410 -0
- data/app/views/maintenance_on_steroids/shared/_auto_refresh.html.erb +85 -0
- data/app/views/maintenance_on_steroids/shared/_javascript.html.erb +16 -0
- data/app/views/maintenance_on_steroids/shared/_pager.html.erb +20 -0
- data/app/views/maintenance_on_steroids/shared/_styles.html.erb +649 -0
- data/app/views/maintenance_on_steroids/shared/_task_list_item.html.erb +31 -0
- data/config/routes.rb +22 -0
- data/lib/generators/maintenance_on_steroids/install/install_generator.rb +56 -0
- data/lib/generators/maintenance_on_steroids/install/templates/create_maintenance_on_steroids_tables.rb.erb +56 -0
- data/lib/generators/maintenance_on_steroids/install/templates/initializer.rb +42 -0
- data/lib/generators/maintenance_on_steroids/job/job_generator.rb +17 -0
- data/lib/generators/maintenance_on_steroids/job/templates/job.rb.erb +30 -0
- data/lib/maintenance_on_steroids/about_dsl.rb +51 -0
- data/lib/maintenance_on_steroids/artifact_dsl.rb +82 -0
- data/lib/maintenance_on_steroids/artifacts_proxy.rb +205 -0
- data/lib/maintenance_on_steroids/callbacks_dsl.rb +61 -0
- data/lib/maintenance_on_steroids/csv_artifact.rb +88 -0
- data/lib/maintenance_on_steroids/engine.rb +26 -0
- data/lib/maintenance_on_steroids/form_dsl.rb +63 -0
- data/lib/maintenance_on_steroids/instrumentation.rb +33 -0
- data/lib/maintenance_on_steroids/job_dsl.rb +61 -0
- data/lib/maintenance_on_steroids/job_registry.rb +83 -0
- data/lib/maintenance_on_steroids/jsonb_artifact.rb +39 -0
- data/lib/maintenance_on_steroids/params_proxy.rb +93 -0
- data/lib/maintenance_on_steroids/task.rb +109 -0
- data/lib/maintenance_on_steroids/text_artifact.rb +74 -0
- data/lib/maintenance_on_steroids/version.rb +3 -0
- data/lib/maintenance_on_steroids.rb +187 -0
- metadata +125 -0
data/README.md
ADDED
|
@@ -0,0 +1,948 @@
|
|
|
1
|
+
# Maintenance on Steroids
|
|
2
|
+
|
|
3
|
+

|
|
4
|
+
|
|
5
|
+
A powerful maintenance task runner for **Rails 8.1+** that leverages `ActiveJob::Continuable` for safe, resumable background processing. Think of it as a batteries-included toolkit for one-off data migrations, batch updates, CSV exports, and any maintenance work your app needs.
|
|
6
|
+
|
|
7
|
+
**Built-in web dashboard** with dark/light themes, live progress tracking, pause/resume/cancel controls, source code viewer, and artifact downloads.
|
|
8
|
+
|
|
9
|
+
> [!IMPORTANT]
|
|
10
|
+
> **Requires Rails >= 8.1.3.1, < 9 and Ruby >= 3.2.** Resumption uses `ActiveJob::Continuable`. The minimum Rails patch includes security fixes; keep your host application's full bundle patched too.
|
|
11
|
+
|
|
12
|
+
## Features
|
|
13
|
+
|
|
14
|
+
- **Collection & callable tasks** -- iterate over ActiveRecord relations or run one-off jobs
|
|
15
|
+
- **Automatic resumption** -- cursor-based progress tracking resumes from the last committed checkpoint; tasks must tolerate retries
|
|
16
|
+
- **Rich DSL** -- typed form inputs, named artifacts, lifecycle callbacks, queue configuration, task metadata
|
|
17
|
+
- **Live dashboard** -- auto-refreshing stats and active runs, real-time progress bars, status badges, run history, and source code viewer
|
|
18
|
+
- **Estimated time remaining** -- live ETA for pending records, extrapolated from the current processing rate
|
|
19
|
+
- **Artifacts** -- store JSON results, export CSV/binary files, downloadable from the UI, with per-run indicators on the dashboard
|
|
20
|
+
- **Pause / Resume / Cancel** -- safely interrupt long-running tasks mid-execution
|
|
21
|
+
- **Instrumentation** -- `ActiveSupport::Notifications` events for every run lifecycle transition
|
|
22
|
+
- **User tracking** -- records who triggered each run with configurable display
|
|
23
|
+
- **Authentication & access control** -- layered, off by default: HTTP Basic (constant-time compare), a custom auth hook for Devise/Warden/etc., and a `verify_access_proc` for role- or IP-based authorization ([jump to setup](#authentication--access-control))
|
|
24
|
+
- **Dark & light themes** -- toggle with one click, persisted in localStorage
|
|
25
|
+
|
|
26
|
+
## Requirements
|
|
27
|
+
|
|
28
|
+
- **Rails >= 8.1.3.1, < 9** — resumption depends on `ActiveJob::Continuable`
|
|
29
|
+
- **Ruby >= 3.2**
|
|
30
|
+
|
|
31
|
+
## Installation
|
|
32
|
+
|
|
33
|
+
Add the gem to your Gemfile:
|
|
34
|
+
|
|
35
|
+
```ruby
|
|
36
|
+
gem "maintenance_on_steroids"
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Run the install generator:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
bundle install
|
|
43
|
+
rails generate maintenance_on_steroids:install
|
|
44
|
+
rails db:migrate
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
This will:
|
|
48
|
+
1. Create the database migration for runs and artifacts tables
|
|
49
|
+
2. Create the `app/maintenance/` directory for your task classes
|
|
50
|
+
3. Mount the engine at `/maintenance` in your routes
|
|
51
|
+
|
|
52
|
+
Update and audit the host application's complete bundle, including Rack and
|
|
53
|
+
other transitive dependencies. This gem does not ship a lockfile; the security
|
|
54
|
+
scans in this repository check its development bundle, not your host's versions.
|
|
55
|
+
|
|
56
|
+
## Quick Start
|
|
57
|
+
|
|
58
|
+
### Generate a task
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
rails generate maintenance_on_steroids:job BackfillUserNames
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
This creates `app/maintenance/backfill_user_names.rb`:
|
|
65
|
+
|
|
66
|
+
```ruby
|
|
67
|
+
class BackfillUserNames < MaintenanceOnSteroids::Task
|
|
68
|
+
about do
|
|
69
|
+
title "Backfill User Names"
|
|
70
|
+
description "TODO: Add description"
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def collection
|
|
74
|
+
User.where(name: nil)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def process(record)
|
|
78
|
+
record.update!(name: record.email.split("@").first)
|
|
79
|
+
end
|
|
80
|
+
end
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### Generate a callable (one-off) task
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
rails generate maintenance_on_steroids:job ClearExpiredTokens --collection=false
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
```ruby
|
|
90
|
+
class ClearExpiredTokens < MaintenanceOnSteroids::Task
|
|
91
|
+
about do
|
|
92
|
+
title "Clear Expired Tokens"
|
|
93
|
+
description "TODO: Add description"
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
def call
|
|
97
|
+
Token.where("expires_at < ?", Time.current).delete_all
|
|
98
|
+
end
|
|
99
|
+
end
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
### Run it
|
|
103
|
+
|
|
104
|
+
Start your Rails server and visit **`/maintenance`**. You'll see the dashboard with your tasks listed. Click a task, then "New Run" to start it.
|
|
105
|
+
|
|
106
|
+
## Task DSL
|
|
107
|
+
|
|
108
|
+
### About
|
|
109
|
+
|
|
110
|
+
Describe your task for the dashboard:
|
|
111
|
+
|
|
112
|
+
```ruby
|
|
113
|
+
class MyTask < MaintenanceOnSteroids::Task
|
|
114
|
+
about do
|
|
115
|
+
title "My Task"
|
|
116
|
+
description "Does something useful"
|
|
117
|
+
owner "Backend Team"
|
|
118
|
+
end
|
|
119
|
+
end
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
If `title` is omitted, the class name is used (e.g. `MyTask` becomes "My Task").
|
|
123
|
+
|
|
124
|
+
### Collection tasks
|
|
125
|
+
|
|
126
|
+
Define `collection` (returns an `ActiveRecord::Relation`) and `process` (handles one record):
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
class DeactivateOldUsers < MaintenanceOnSteroids::Task
|
|
130
|
+
about do
|
|
131
|
+
title "Deactivate Old Users"
|
|
132
|
+
description "Deactivates users who haven't logged in for a year"
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
def collection
|
|
136
|
+
User.where("last_sign_in_at < ?", 1.year.ago).where(active: true)
|
|
137
|
+
end
|
|
138
|
+
|
|
139
|
+
def process(user)
|
|
140
|
+
user.update!(active: false)
|
|
141
|
+
end
|
|
142
|
+
end
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
The engine iterates using `find_each` and commits a cursor and accumulated output after each record. Resumption starts after that checkpoint. Processing is **at least once**: a crash after a task side effect but before its checkpoint can repeat that record, so make task effects idempotent.
|
|
146
|
+
|
|
147
|
+
### Callable tasks
|
|
148
|
+
|
|
149
|
+
For one-off jobs that don't iterate over a collection, define `call`:
|
|
150
|
+
|
|
151
|
+
```ruby
|
|
152
|
+
class RecalculateStats < MaintenanceOnSteroids::Task
|
|
153
|
+
about do
|
|
154
|
+
title "Recalculate Stats"
|
|
155
|
+
description "Rebuilds all cached statistics"
|
|
156
|
+
end
|
|
157
|
+
|
|
158
|
+
def call
|
|
159
|
+
StatsService.rebuild_all
|
|
160
|
+
end
|
|
161
|
+
end
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### Form Inputs
|
|
165
|
+
|
|
166
|
+
Accept parameters from the UI with typed inputs:
|
|
167
|
+
|
|
168
|
+
```ruby
|
|
169
|
+
class UpdateUserAges < MaintenanceOnSteroids::Task
|
|
170
|
+
form do
|
|
171
|
+
input :name, type: :string, required: true, placeholder: "Filter by name"
|
|
172
|
+
input :new_age, type: :integer, required: true, default: 25
|
|
173
|
+
end
|
|
174
|
+
|
|
175
|
+
def collection
|
|
176
|
+
User.where(name: params[:name])
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
def process(user)
|
|
180
|
+
user.update!(age: params[:new_age])
|
|
181
|
+
end
|
|
182
|
+
end
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
The `params` proxy provides typed access -- `:integer` values are cast to `Integer`, `:boolean` to `true/false`, etc.
|
|
186
|
+
|
|
187
|
+
**Supported input types:**
|
|
188
|
+
|
|
189
|
+
| Type | HTML Element | Cast |
|
|
190
|
+
|------|-------------|------|
|
|
191
|
+
| `:string` | `<input type="text">` | String |
|
|
192
|
+
| `:integer` | `<input type="number">` | Integer |
|
|
193
|
+
| `:float` | `<input type="number" step="any">` | Float |
|
|
194
|
+
| `:boolean` | `<input type="checkbox">` | Boolean |
|
|
195
|
+
| `:text` | `<textarea>` | String |
|
|
196
|
+
| `:date` | `<input type="date">` | Date |
|
|
197
|
+
| `:datetime` | `<input type="datetime-local">` | Time |
|
|
198
|
+
| `:blob` | `<input type="file">` | Binary |
|
|
199
|
+
| `:select` | `<select>` | String |
|
|
200
|
+
|
|
201
|
+
**Input options:**
|
|
202
|
+
|
|
203
|
+
```ruby
|
|
204
|
+
input :role,
|
|
205
|
+
type: :select,
|
|
206
|
+
options: %w[admin user moderator],
|
|
207
|
+
required: true,
|
|
208
|
+
default: "user",
|
|
209
|
+
label: "User Role",
|
|
210
|
+
placeholder: "Choose a role",
|
|
211
|
+
help_text: "The role to assign to matched users"
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
**File uploads (`type: :blob`):** the uploaded file is stored with the run (as an `input` artifact) and read through `params`:
|
|
215
|
+
|
|
216
|
+
- `params[:name]` -- the raw file **bytes** (a `String`), or `nil` if nothing was uploaded
|
|
217
|
+
- `params.file_name(:name)` -- the original filename
|
|
218
|
+
- `params.content_type(:name)` -- the uploaded MIME type
|
|
219
|
+
|
|
220
|
+
```ruby
|
|
221
|
+
class CountLetterATask < MaintenanceOnSteroids::Task
|
|
222
|
+
form do
|
|
223
|
+
input :file, type: :blob, required: true, help_text: "CSV/text file to scan"
|
|
224
|
+
end
|
|
225
|
+
|
|
226
|
+
artifact :result, type: :jsonb, default: {}
|
|
227
|
+
|
|
228
|
+
def call
|
|
229
|
+
content = params[:file].to_s # raw bytes of the upload
|
|
230
|
+
|
|
231
|
+
artifacts.save(:result, {
|
|
232
|
+
"file_name" => params.file_name(:file),
|
|
233
|
+
"content_type" => params.content_type(:file),
|
|
234
|
+
"bytes" => content.bytesize,
|
|
235
|
+
"a_count" => content.count("aA") # letter "a", case-insensitive
|
|
236
|
+
})
|
|
237
|
+
end
|
|
238
|
+
end
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
Uploads are read into memory and capped by `config.max_upload_size` (see Configuration). Need a CSV as rows? `CSV.parse(params[:file])`.
|
|
242
|
+
|
|
243
|
+
### Artifacts
|
|
244
|
+
|
|
245
|
+
Store output data (JSON, files, text) that persists with the run:
|
|
246
|
+
|
|
247
|
+
```ruby
|
|
248
|
+
class ExportUsers < MaintenanceOnSteroids::Task
|
|
249
|
+
about do
|
|
250
|
+
title "Export Users to CSV"
|
|
251
|
+
end
|
|
252
|
+
|
|
253
|
+
artifact :csv_file, type: :file, file_name: "users.csv"
|
|
254
|
+
|
|
255
|
+
def call
|
|
256
|
+
csv_data = CSV.generate do |csv|
|
|
257
|
+
csv << %w[id name email]
|
|
258
|
+
User.find_each do |user|
|
|
259
|
+
csv << [user.id, user.name, user.email]
|
|
260
|
+
end
|
|
261
|
+
end
|
|
262
|
+
|
|
263
|
+
artifacts[:csv_file] = csv_data
|
|
264
|
+
end
|
|
265
|
+
end
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
The generated file is downloadable from the run's detail page in the UI.
|
|
269
|
+
|
|
270
|
+
**Artifact types:**
|
|
271
|
+
|
|
272
|
+
| Type | Storage | Use case |
|
|
273
|
+
|------|---------|----------|
|
|
274
|
+
| `:jsonb` | JSON column | Structured results, counters, logs |
|
|
275
|
+
| `:file` | Binary blob | PDFs, images, pre-rendered files |
|
|
276
|
+
| `:text` | Text column | Plain text output |
|
|
277
|
+
| `:csv` | Binary blob | Row-oriented exports, table-previewed in the UI |
|
|
278
|
+
|
|
279
|
+
An unknown `type:` raises `ArgumentError` at load time, so typos surface immediately.
|
|
280
|
+
|
|
281
|
+
**Declaration options** (all types): `label:` (human name shown in the UI, defaults to the humanized artifact name), `description:` (shown under the artifact on the run page), `content_type:` (download MIME -- otherwise inferred from `file_name`), plus `default:`, `file_name:`, and `headers:` (CSV).
|
|
282
|
+
|
|
283
|
+
**Writing artifacts -- two styles:**
|
|
284
|
+
|
|
285
|
+
1. **Explicit (`save`)** -- persist a whole value immediately. The type comes from the declaration, so one call covers every kind. This is the simplest path and never relies on auto-flush:
|
|
286
|
+
|
|
287
|
+
```ruby
|
|
288
|
+
artifacts.save(:json_data, { name: "Igor", age: 40 }) # jsonb
|
|
289
|
+
artifacts.save(:export, rows) # csv (array of rows, or a String)
|
|
290
|
+
artifacts.save(:log, "done") # text
|
|
291
|
+
# artifacts[:json_data] = { ... } is an alias of save
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
2. **Accumulator (`<<` / in-place)** -- build the value incrementally across records; the job auto-flushes it (see Auto-flush below). Best for resumable tasks that accrue output row by row:
|
|
295
|
+
|
|
296
|
+
```ruby
|
|
297
|
+
artifacts.export << [user.id, user.name] # append a CSV row
|
|
298
|
+
artifacts.result[user.id.to_s] = { ok: true } # mutate a JSONB hash in place
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
**Access** -- `artifacts[:name]` and method style are equivalent:
|
|
302
|
+
|
|
303
|
+
```ruby
|
|
304
|
+
artifacts[:result]["k"] = v
|
|
305
|
+
artifacts.result["k"] = v # same thing, reads nicer
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
**JSONB artifacts** behave like a hash:
|
|
309
|
+
|
|
310
|
+
```ruby
|
|
311
|
+
artifact :result, type: :jsonb, default: {}
|
|
312
|
+
|
|
313
|
+
def process(user)
|
|
314
|
+
user.update!(active: false)
|
|
315
|
+
artifacts.result[user.id.to_s] = { deactivated: true }
|
|
316
|
+
end
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
**CSV artifacts** are append-oriented and render to a downloadable `.csv`:
|
|
320
|
+
|
|
321
|
+
```ruby
|
|
322
|
+
artifact :export, type: :csv, headers: %w[id name email], description: "All users"
|
|
323
|
+
|
|
324
|
+
def call
|
|
325
|
+
User.find_each { |u| artifacts.export << [u.id, u.name, u.email] }
|
|
326
|
+
# auto-flushed on completion; previewed as a table on the run page
|
|
327
|
+
end
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
**Auto-flush:** collection tasks commit mutated accumulators together with each record's cursor. Callable tasks flush at `checkpoint!` and on completion or interruption. Completion callbacks can produce final output; it is saved before the run becomes `completed` and before the success notification. Pause/cancel callbacks likewise run before the final state is committed. Persistence failures mark the run `errored`; failed collection-record buffers are discarded. Reads alone never create a row. Explicit `artifacts.save` / `save!` writes happen immediately and are not coupled to the collection cursor, so use accumulators for checkpointed exports.
|
|
331
|
+
|
|
332
|
+
**Metadata:** every artifact records lightweight stats on save -- entry/line count, byte size, and a generated-at timestamp -- shown on the run page (`12 entries · 3.4 KB · 2 minutes ago`) without loading the full payload. Available on the model via `artifact.summary`, `artifact.byte_size`, and `artifact.generated_at`.
|
|
333
|
+
|
|
334
|
+
**File artifacts** -- if `file_name` is omitted, it defaults to `task_class_name_YYYYMMDD_HHMMSS`:
|
|
335
|
+
|
|
336
|
+
```ruby
|
|
337
|
+
artifact :export, type: :file
|
|
338
|
+
# File name auto-generated: "export_users_20260218_143022"
|
|
339
|
+
|
|
340
|
+
artifact :report, type: :file, file_name: "report.pdf"
|
|
341
|
+
# Explicit file name: "report.pdf"
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
### Callbacks
|
|
345
|
+
|
|
346
|
+
Hook into the task lifecycle:
|
|
347
|
+
|
|
348
|
+
```ruby
|
|
349
|
+
class ImportUsers < MaintenanceOnSteroids::Task
|
|
350
|
+
after_start :notify_started
|
|
351
|
+
after_complete :notify_completed
|
|
352
|
+
after_error :notify_failed
|
|
353
|
+
after_pause :log_pause
|
|
354
|
+
after_cancel :cleanup
|
|
355
|
+
after_interrupt :save_progress
|
|
356
|
+
|
|
357
|
+
def collection
|
|
358
|
+
User.where(imported: false)
|
|
359
|
+
end
|
|
360
|
+
|
|
361
|
+
def process(user)
|
|
362
|
+
user.update!(imported: true)
|
|
363
|
+
end
|
|
364
|
+
|
|
365
|
+
private
|
|
366
|
+
|
|
367
|
+
def notify_started
|
|
368
|
+
Slack.notify("#imports", "User import started")
|
|
369
|
+
end
|
|
370
|
+
|
|
371
|
+
def notify_completed
|
|
372
|
+
Slack.notify("#imports", "User import completed!")
|
|
373
|
+
end
|
|
374
|
+
|
|
375
|
+
def notify_failed
|
|
376
|
+
Slack.notify("#imports", "User import failed!")
|
|
377
|
+
end
|
|
378
|
+
|
|
379
|
+
def log_pause
|
|
380
|
+
Rails.logger.info "Import paused by user"
|
|
381
|
+
end
|
|
382
|
+
|
|
383
|
+
def cleanup
|
|
384
|
+
TempFile.cleanup
|
|
385
|
+
end
|
|
386
|
+
|
|
387
|
+
def save_progress
|
|
388
|
+
# Called on both pause and cancel, before the specific callback
|
|
389
|
+
end
|
|
390
|
+
end
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
**Available callbacks:**
|
|
394
|
+
|
|
395
|
+
| Callback | When it fires |
|
|
396
|
+
|----------|--------------|
|
|
397
|
+
| `after_start` | Task begins execution |
|
|
398
|
+
| `after_complete` | Task finishes successfully |
|
|
399
|
+
| `after_error` | Task raises an exception |
|
|
400
|
+
| `after_pause` | Task is paused |
|
|
401
|
+
| `after_cancel` | Task is cancelled |
|
|
402
|
+
| `after_interrupt` | Task is interrupted (fires before pause or cancel) |
|
|
403
|
+
|
|
404
|
+
### Job Configuration
|
|
405
|
+
|
|
406
|
+
Set a custom queue and/or priority:
|
|
407
|
+
|
|
408
|
+
```ruby
|
|
409
|
+
class HeavyExport < MaintenanceOnSteroids::Task
|
|
410
|
+
job do
|
|
411
|
+
queue "exports"
|
|
412
|
+
priority 10
|
|
413
|
+
end
|
|
414
|
+
|
|
415
|
+
def call
|
|
416
|
+
# ...
|
|
417
|
+
end
|
|
418
|
+
end
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
### Database Role Switching
|
|
422
|
+
|
|
423
|
+
To scan a collection against a replica, declare the role. The job holds it open
|
|
424
|
+
for the whole scan:
|
|
425
|
+
|
|
426
|
+
```ruby
|
|
427
|
+
class BackfillTask < MaintenanceOnSteroids::Task
|
|
428
|
+
job do
|
|
429
|
+
database_role :reading # :read is accepted too
|
|
430
|
+
end
|
|
431
|
+
|
|
432
|
+
def collection
|
|
433
|
+
User.where(active: true) # every batch query runs on the replica
|
|
434
|
+
end
|
|
435
|
+
|
|
436
|
+
def process(user)
|
|
437
|
+
user.update!(...) # writes run on the primary
|
|
438
|
+
end
|
|
439
|
+
end
|
|
440
|
+
```
|
|
441
|
+
|
|
442
|
+
`process` and the run's own cursor/progress bookkeeping step back to `:writing`
|
|
443
|
+
automatically, so a declared read role never blocks the task's writes.
|
|
444
|
+
|
|
445
|
+
For a one-off read inside `call` or `process`, `with_database_role` wraps a
|
|
446
|
+
block:
|
|
447
|
+
|
|
448
|
+
```ruby
|
|
449
|
+
def call
|
|
450
|
+
stale = with_database_role(:read) { User.where(active: false).count }
|
|
451
|
+
artifacts.save(:summary, { stale: stale })
|
|
452
|
+
end
|
|
453
|
+
```
|
|
454
|
+
|
|
455
|
+
> **The block must force whatever it reads.** Returning a lazy `Relation` from
|
|
456
|
+
> `with_database_role` does nothing — the role is restored on the way out and
|
|
457
|
+
> the query runs later, on the primary. That is why `collection` uses the
|
|
458
|
+
> declarative `database_role` above instead.
|
|
459
|
+
|
|
460
|
+
## Configuration
|
|
461
|
+
|
|
462
|
+
`rails g maintenance_on_steroids:install` writes
|
|
463
|
+
`config/initializers/maintenance_on_steroids.rb` for you, with every access
|
|
464
|
+
control commented out and ready to fill in. The full set of options:
|
|
465
|
+
|
|
466
|
+
```ruby
|
|
467
|
+
MaintenanceOnSteroids.configure do |config|
|
|
468
|
+
# --- Authentication ---
|
|
469
|
+
|
|
470
|
+
# HTTP Basic auth (simplest option)
|
|
471
|
+
config.http_basic_authentication_enabled = true
|
|
472
|
+
config.http_basic_authentication_user_name = "admin"
|
|
473
|
+
config.http_basic_authentication_password = Rails.application.credentials.maintenance_password
|
|
474
|
+
|
|
475
|
+
# Or use a custom authentication hook (e.g. Devise)
|
|
476
|
+
config.authentication = -> {
|
|
477
|
+
authenticate_user! # Devise method
|
|
478
|
+
}
|
|
479
|
+
|
|
480
|
+
# Role-based access control
|
|
481
|
+
config.verify_access_proc = ->(controller) {
|
|
482
|
+
controller.current_user&.admin?
|
|
483
|
+
}
|
|
484
|
+
|
|
485
|
+
# --- User Tracking ---
|
|
486
|
+
|
|
487
|
+
# Track who triggers each run
|
|
488
|
+
config.current_user_resolver = -> {
|
|
489
|
+
Current.user # Or any way to get the current user
|
|
490
|
+
}
|
|
491
|
+
|
|
492
|
+
# Customize how user names are displayed
|
|
493
|
+
config.user_display_formatter = ->(run) {
|
|
494
|
+
User.find(run.user_id).full_name rescue run.user_email
|
|
495
|
+
}
|
|
496
|
+
end
|
|
497
|
+
```
|
|
498
|
+
|
|
499
|
+
### Configuration Options
|
|
500
|
+
|
|
501
|
+
| Option | Default | Description |
|
|
502
|
+
|--------|---------|-------------|
|
|
503
|
+
| `parent_controller` | `"ActionController::Base"` | Base controller class for the engine |
|
|
504
|
+
| `max_upload_size` | `50.megabytes` | Maximum size (bytes) for file inputs uploaded when starting a run |
|
|
505
|
+
| `max_artifact_size` | `64.megabytes` | Hard ceiling on a single stored artifact; exceeding it fails the run |
|
|
506
|
+
| `http_basic_authentication_enabled` | `false` | Enable HTTP Basic auth |
|
|
507
|
+
| `http_basic_authentication_user_name` | `"admin"` | HTTP Basic username |
|
|
508
|
+
| `http_basic_authentication_password` | `"secret"` | HTTP Basic password |
|
|
509
|
+
| `authentication` | `nil` | Proc executed in controller context for auth |
|
|
510
|
+
| `verify_access_proc` | `nil` | Proc receiving controller, return true/false |
|
|
511
|
+
| `current_user_resolver` | `nil` | Proc returning the current user object |
|
|
512
|
+
| `user_display_formatter` | `nil` | Proc receiving Run, returning display string |
|
|
513
|
+
| `allow_insecure_dashboard` | `false` | Boot in production with no access-control layer configured |
|
|
514
|
+
|
|
515
|
+
### Authentication & Access Control
|
|
516
|
+
|
|
517
|
+
The dashboard is mounted in **your** app, so it inherits your app's middleware — but it ships with **no access control of its own until you configure it**. Three independent layers run as `before_action`s on every engine request, in order. Use any combination; each layer is skipped when its config is left at the default.
|
|
518
|
+
|
|
519
|
+
| Order | Layer | Config | When it runs | On failure |
|
|
520
|
+
|-------|-------|--------|--------------|------------|
|
|
521
|
+
| 1 | **HTTP Basic** | `http_basic_authentication_enabled` | Always (when enabled) | `401 Unauthorized` (browser credential prompt) |
|
|
522
|
+
| 2 | **Auth hook** | `authentication` | After Basic passes | Whatever the proc does (usually `redirect_to` sign-in) |
|
|
523
|
+
| 3 | **Access check** | `verify_access_proc` | After the hook | `403 Forbidden` (`"Access denied"`) |
|
|
524
|
+
|
|
525
|
+
> [!WARNING]
|
|
526
|
+
> **In production the engine refuses to boot until one layer is configured.** With none of them set, anyone who can reach the mounted path can run, pause, and cancel maintenance tasks against your database.
|
|
527
|
+
>
|
|
528
|
+
> The shipped HTTP Basic defaults (`admin` / `secret`) are **placeholders**, and leaving the password at `"secret"` counts as *unconfigured* — a dashboard behind a published password is not protected, and treating it as configured would make the most dangerous setup the quietest one. Override it, and prefer HTTPS since Basic credentials travel in every request.
|
|
529
|
+
>
|
|
530
|
+
> If the dashboard is already gated somewhere this gem cannot see — a reverse proxy, a VPN, your own middleware — say so explicitly rather than leaving the layers empty:
|
|
531
|
+
>
|
|
532
|
+
> ```ruby
|
|
533
|
+
> config.allow_insecure_dashboard = true
|
|
534
|
+
> ```
|
|
535
|
+
>
|
|
536
|
+
> Outside production, an unconfigured dashboard logs a warning instead of raising.
|
|
537
|
+
|
|
538
|
+
#### Recipe: HTTP Basic (quickest)
|
|
539
|
+
|
|
540
|
+
Good for a staging box or a small team. Credentials are compared in constant time (`ActiveSupport::SecurityUtils.secure_compare`), so it's not vulnerable to timing attacks.
|
|
541
|
+
|
|
542
|
+
```ruby
|
|
543
|
+
# config/initializers/maintenance_on_steroids.rb
|
|
544
|
+
MaintenanceOnSteroids.configure do |config|
|
|
545
|
+
config.http_basic_authentication_enabled = true
|
|
546
|
+
config.http_basic_authentication_user_name = "admin"
|
|
547
|
+
config.http_basic_authentication_password = Rails.application.credentials.maintenance_password
|
|
548
|
+
end
|
|
549
|
+
```
|
|
550
|
+
|
|
551
|
+
#### Recipe: Devise + admin role (most common)
|
|
552
|
+
|
|
553
|
+
Reuse your app's existing login, then restrict to admins. Point `parent_controller` at your `ApplicationController` so Devise's helpers (`current_user`, `authenticate_user!`) are in scope.
|
|
554
|
+
|
|
555
|
+
```ruby
|
|
556
|
+
MaintenanceOnSteroids.configure do |config|
|
|
557
|
+
config.parent_controller = "ApplicationController"
|
|
558
|
+
|
|
559
|
+
# Layer 2: bounce anyone who isn't signed in
|
|
560
|
+
config.authentication = -> {
|
|
561
|
+
redirect_to(main_app.new_user_session_path, alert: "Please sign in.") unless current_user
|
|
562
|
+
}
|
|
563
|
+
|
|
564
|
+
# Layer 3: of the signed-in users, only admins get in (renders 403 otherwise)
|
|
565
|
+
config.verify_access_proc = ->(controller) { controller.current_user&.admin? }
|
|
566
|
+
|
|
567
|
+
# Stamp each run with who triggered it (shown in the UI)
|
|
568
|
+
config.current_user_resolver = -> { current_user }
|
|
569
|
+
end
|
|
570
|
+
```
|
|
571
|
+
|
|
572
|
+
#### Recipe: IP allowlist
|
|
573
|
+
|
|
574
|
+
`verify_access_proc` receives the controller, so any request attribute is fair game — e.g. lock the dashboard to your office/VPN range:
|
|
575
|
+
|
|
576
|
+
```ruby
|
|
577
|
+
ALLOWED_IPS = %w[203.0.113.4 198.51.100.0/24].map { |ip| IPAddr.new(ip) }
|
|
578
|
+
|
|
579
|
+
config.verify_access_proc = ->(controller) {
|
|
580
|
+
ip = IPAddr.new(controller.request.remote_ip)
|
|
581
|
+
ALLOWED_IPS.any? { |range| range.include?(ip) }
|
|
582
|
+
}
|
|
583
|
+
```
|
|
584
|
+
|
|
585
|
+
Combine layers freely — e.g. HTTP Basic **and** an IP check both have to pass. Each returns independently, so the first failing layer short-circuits the request.
|
|
586
|
+
|
|
587
|
+
## Dashboard
|
|
588
|
+
|
|
589
|
+
The engine provides a full-featured web UI at your mounted path (default: `/maintenance`).
|
|
590
|
+
|
|
591
|
+
### Pages
|
|
592
|
+
|
|
593
|
+
- **Dashboard** -- stats overview, active runs, and recent history; runs with output artifacts show a 📎 indicator with the artifact count
|
|
594
|
+
- **Tasks** -- all registered task classes, sortable by name (default) or last execution time; each row shows the last run's status and age, and never-executed tasks carry a "New" badge
|
|
595
|
+
- **Task detail** -- task metadata, run history, "New Run" and "Source" buttons
|
|
596
|
+
- **Source viewer** -- view the Ruby source code of any task class
|
|
597
|
+
- **New Run** -- form with typed inputs to start a task
|
|
598
|
+
- **Run detail** -- live progress bar, status, duration, estimated time remaining, parameters, artifacts, pause/resume/cancel controls, and a "View Source" shortcut
|
|
599
|
+
|
|
600
|
+
### Live Progress
|
|
601
|
+
|
|
602
|
+
Active runs poll for status updates every 2 seconds. The progress bar, percentage, status badge, duration, and the **Estimated (pending)** time remaining (`2d 4h 12m 30s`-style, leading zero units omitted) update in real-time without page refresh.
|
|
603
|
+
|
|
604
|
+
The dashboard auto-refreshes its stats and active-runs table every 2 seconds in the background (paused while the tab is hidden) -- no flash, no scroll jumps.
|
|
605
|
+
|
|
606
|
+
While a run is in a transitional state (`pausing`, `cancelling`), the run page shows a visible auto-refresh countdown next to the status badge and reloads every few seconds until the worker settles the state.
|
|
607
|
+
|
|
608
|
+
### Theme
|
|
609
|
+
|
|
610
|
+
The UI supports dark and light themes. Click the sun/moon toggle in the top bar. Your preference is saved in localStorage.
|
|
611
|
+
|
|
612
|
+
## How Resumption Works
|
|
613
|
+
|
|
614
|
+
This gem uses Rails 8.1's `ActiveJob::Continuable` for safe background processing:
|
|
615
|
+
|
|
616
|
+
1. **Collection tasks** iterate over records using `find_each`, ordered by primary key
|
|
617
|
+
2. After processing each record, accumulated output and the database cursor commit in one transaction, then `ActiveJob::Continuable`'s step cursor advances
|
|
618
|
+
3. If Sidekiq restarts mid-job, `ActiveJob::Continuable` resumes from its last cursor
|
|
619
|
+
4. If you pause and resume, a new job is enqueued that reads the cursor from the database
|
|
620
|
+
5. In both cases, the query uses `WHERE id > cursor` to skip already-processed records
|
|
621
|
+
|
|
622
|
+
**Task side effects are at least once.** A record whose checkpoint committed is skipped on resume. A crash after a database/API side effect but before the checkpoint can repeat that side effect. Use idempotent updates or a durable deduplication key for external operations. The engine cannot make another database or service commit atomically with its cursor.
|
|
623
|
+
|
|
624
|
+
Collection accumulator output and its cursor survive together, including hard
|
|
625
|
+
worker death. Output from an unfinished record is discarded and that record is
|
|
626
|
+
retried. Callable tasks restart `call` on manual resume; `checkpoint!` saves output
|
|
627
|
+
and updates the heartbeat, but does not remember a position inside your method.
|
|
628
|
+
Persist your own callable work position and make retries safe. Hard termination
|
|
629
|
+
can lose callable output since its last checkpoint.
|
|
630
|
+
|
|
631
|
+
Each worker atomically claims a run with an execution token. Duplicate deliveries
|
|
632
|
+
and stale job IDs cannot claim it. Reaping revokes that token; an old worker stops
|
|
633
|
+
at its next checkpoint and cannot overwrite engine progress or artifacts. A
|
|
634
|
+
currently executing external side effect cannot be recalled, so set reaper
|
|
635
|
+
thresholds conservatively.
|
|
636
|
+
|
|
637
|
+
> **Note:** cursor-based resumption relies on monotonically increasing primary keys. Collections with UUID/string primary keys can skip or repeat records on resume -- the job logs a warning when it detects one.
|
|
638
|
+
|
|
639
|
+
## Operations
|
|
640
|
+
|
|
641
|
+
### Recovering from worker crashes
|
|
642
|
+
|
|
643
|
+
If a worker process dies hard (OOM kill, `kill -9`, node failure), its run can be left in `running` forever. `Run.reap_stale!` transitions in-flight runs whose row hasn't been touched recently to `errored` (the job updates the row at least once per processed record, so `updated_at` acts as a heartbeat):
|
|
644
|
+
|
|
645
|
+
```ruby
|
|
646
|
+
# Run periodically (cron, recurring job, e.g. solid_queue recurring task):
|
|
647
|
+
MaintenanceOnSteroids::Run.reap_stale!(threshold: 30.minutes)
|
|
648
|
+
```
|
|
649
|
+
|
|
650
|
+
Pick a threshold comfortably larger than the time your slowest record or callable
|
|
651
|
+
checkpoint interval takes. `checkpoint!` refreshes a callable's heartbeat.
|
|
652
|
+
Inside a collection's `process`, it refreshes the heartbeat and checks ownership;
|
|
653
|
+
pause/cancel waits until that record finishes, keeping output and cursor together.
|
|
654
|
+
`enqueued` and `paused` runs are excluded by default. To recover a process crash
|
|
655
|
+
between saving a run and dispatching its job, opt into an enqueue timeout longer
|
|
656
|
+
than your maximum queue delay:
|
|
657
|
+
|
|
658
|
+
```ruby
|
|
659
|
+
MaintenanceOnSteroids::Run.reap_stale!(threshold: 30.minutes, enqueued_threshold: 1.day)
|
|
660
|
+
```
|
|
661
|
+
|
|
662
|
+
An expired queued job is ignored unless an operator resumes the run, which
|
|
663
|
+
creates a new job identity. Queue submission failures, including refused
|
|
664
|
+
Continuable retries, are surfaced as `errored`.
|
|
665
|
+
|
|
666
|
+
### Recovering from a failed run
|
|
667
|
+
|
|
668
|
+
A run that raises is marked `errored` and stops, with the message and backtrace on the run page. Fix the cause and hit **Resume** to restart after the last committed checkpoint. Uncheckpointed effects can be repeated. Resuming prepares fresh metadata before dispatch, preserving any new worker error even if that worker finishes immediately.
|
|
669
|
+
|
|
670
|
+
### Pruning old runs
|
|
671
|
+
|
|
672
|
+
Nothing expires runs on its own, so a long-lived app keeps every run, backtrace
|
|
673
|
+
and stored artifact forever. `Run.prune!` deletes finished runs (and their
|
|
674
|
+
artifacts) past a cutoff; schedule it like `reap_stale!`:
|
|
675
|
+
|
|
676
|
+
```ruby
|
|
677
|
+
MaintenanceOnSteroids::Run.prune!(older_than: 90.days)
|
|
678
|
+
```
|
|
679
|
+
|
|
680
|
+
Only `completed`, `cancelled` and `errored` runs are eligible — anything active
|
|
681
|
+
or paused is left alone regardless of age. Returns the number deleted.
|
|
682
|
+
|
|
683
|
+
### Artifact size limits
|
|
684
|
+
|
|
685
|
+
**Artifacts are buffered in the worker's memory and stored in a single database
|
|
686
|
+
row.** That makes them ideal for maintenance *results* — a summary, a log, a
|
|
687
|
+
few thousand rows of exceptions — and unsuitable as a bulk-export pipeline.
|
|
688
|
+
A million-row CSV will exhaust the worker before it ever reaches the database.
|
|
689
|
+
|
|
690
|
+
`MaintenanceOnSteroids.max_artifact_size` (default 64 MB) rejects oversized
|
|
691
|
+
persistent output and checks text/CSV appends before adding them to the buffer.
|
|
692
|
+
It is not a process memory limit: arbitrary task allocations, nested JSON edits,
|
|
693
|
+
serialization, and multiple artifacts can use more memory. Each collection
|
|
694
|
+
checkpoint rewrites dirty artifacts, so keep them small. Write large output to
|
|
695
|
+
object storage and keep only a reference:
|
|
696
|
+
|
|
697
|
+
```ruby
|
|
698
|
+
def process(record)
|
|
699
|
+
# ... build rows ...
|
|
700
|
+
end
|
|
701
|
+
|
|
702
|
+
after_complete do
|
|
703
|
+
key = S3Uploader.call(big_file)
|
|
704
|
+
artifacts.save(:location, { bucket: "exports", key: key })
|
|
705
|
+
end
|
|
706
|
+
```
|
|
707
|
+
|
|
708
|
+
Inline preview rendering is capped (256 KB, or 200 top-level JSON entries), but
|
|
709
|
+
the database still returns complete payloads. Large or numerous artifacts can
|
|
710
|
+
consume substantial web memory. File and CSV artifacts have a Download action.
|
|
711
|
+
|
|
712
|
+
### Pausing a long-running callable task
|
|
713
|
+
|
|
714
|
+
Collection tasks check for a pending pause or cancel between records. A callable
|
|
715
|
+
task is a single unit of work, so a long `call` won't notice Pause until it
|
|
716
|
+
returns. Call `checkpoint!` at the points where stopping is safe:
|
|
717
|
+
|
|
718
|
+
```ruby
|
|
719
|
+
def call
|
|
720
|
+
Account.find_each do |account|
|
|
721
|
+
checkpoint! # honours a pending Pause/Cancel here
|
|
722
|
+
account.recalculate!
|
|
723
|
+
end
|
|
724
|
+
end
|
|
725
|
+
```
|
|
726
|
+
|
|
727
|
+
When a stop is pending, `checkpoint!` does not return — the job unwinds, buffered
|
|
728
|
+
artifacts are flushed, and the run lands in `paused` or `cancelled`.
|
|
729
|
+
|
|
730
|
+
### Preventing concurrent runs
|
|
731
|
+
|
|
732
|
+
Nothing stops an operator from starting the same task twice — a double-clicked
|
|
733
|
+
**New Run** on a destructive task runs it twice. Declare a limit:
|
|
734
|
+
|
|
735
|
+
```ruby
|
|
736
|
+
class BackfillTask < MaintenanceOnSteroids::Task
|
|
737
|
+
job do
|
|
738
|
+
concurrency 1
|
|
739
|
+
end
|
|
740
|
+
end
|
|
741
|
+
```
|
|
742
|
+
|
|
743
|
+
Further runs are refused while that many are still active. The check is advisory
|
|
744
|
+
(two simultaneous submissions can still race); it exists to catch the double-click,
|
|
745
|
+
not to provide a distributed lock. For a hard guarantee, take an advisory lock
|
|
746
|
+
inside the task itself.
|
|
747
|
+
|
|
748
|
+
### Content Security Policy
|
|
749
|
+
|
|
750
|
+
The dashboard's live updates use small inline `<script>` blocks and no build
|
|
751
|
+
step, so a strict CSP without `'unsafe-inline'` will block polling (the pages
|
|
752
|
+
still render and work; they just stop refreshing themselves). If your app sets
|
|
753
|
+
a strict policy, scope an exception to the mounted path:
|
|
754
|
+
|
|
755
|
+
```ruby
|
|
756
|
+
# config/initializers/content_security_policy.rb
|
|
757
|
+
Rails.application.config.content_security_policy_nonce_directives = %w[script-src]
|
|
758
|
+
```
|
|
759
|
+
|
|
760
|
+
or exclude the engine's path from the policy entirely.
|
|
761
|
+
|
|
762
|
+
### Collections with UUID or string primary keys
|
|
763
|
+
|
|
764
|
+
Resumption works by remembering the last processed primary key and continuing
|
|
765
|
+
with `WHERE id > cursor`, which assumes keys increase over time. That holds for
|
|
766
|
+
integer and bigint keys, and for time-ordered UUIDs (UUIDv7, ULID). It does
|
|
767
|
+
**not** hold for random UUIDv4: after an interruption such a run can skip
|
|
768
|
+
records it never processed and reprocess others.
|
|
769
|
+
|
|
770
|
+
The job logs a warning when it detects a non-integer primary key. If your
|
|
771
|
+
collection uses random UUIDs, either avoid pause/resume for it or scope the
|
|
772
|
+
collection so each run is complete in itself.
|
|
773
|
+
|
|
774
|
+
### Running on SQLite
|
|
775
|
+
|
|
776
|
+
A running task writes to the runs table after every processed record, so the dashboard's own writes (pause, cancel, resume) compete with the worker for SQLite's single writer. Make sure your `database.yml` sets a busy timeout, or those clicks will fail with `SQLite3::BusyException: database is locked`:
|
|
777
|
+
|
|
778
|
+
```yaml
|
|
779
|
+
production:
|
|
780
|
+
adapter: sqlite3
|
|
781
|
+
database: storage/production.sqlite3
|
|
782
|
+
timeout: 5000 # ms to wait for the write lock instead of failing instantly
|
|
783
|
+
```
|
|
784
|
+
|
|
785
|
+
The default is `0` -- no waiting at all. Postgres and MySQL need no equivalent setting.
|
|
786
|
+
|
|
787
|
+
## Instrumentation
|
|
788
|
+
|
|
789
|
+
Every run lifecycle transition emits an `ActiveSupport::Notifications` event in the `maintenance_on_steroids` namespace -- the same pattern as the `maintenance_tasks` gem:
|
|
790
|
+
|
|
791
|
+
```ruby
|
|
792
|
+
ActiveSupport::Notifications.subscribe("enqueued.maintenance_on_steroids") do |event|
|
|
793
|
+
run = event.payload[:run]
|
|
794
|
+
Rails.logger.info "Enqueued #{event.payload[:task_name]} (run ##{run.id})"
|
|
795
|
+
end
|
|
796
|
+
|
|
797
|
+
# Or subscribe to all events at once:
|
|
798
|
+
ActiveSupport::Notifications.subscribe(/\.maintenance_on_steroids\z/) do |event|
|
|
799
|
+
StatsD.increment("maintenance.#{event.name.split('.').first}")
|
|
800
|
+
end
|
|
801
|
+
```
|
|
802
|
+
|
|
803
|
+
| Event | Fired when |
|
|
804
|
+
|-------|-----------|
|
|
805
|
+
| `enqueued.maintenance_on_steroids` | A run is enqueued (initial enqueue and re-enqueue on resume) |
|
|
806
|
+
| `started.maintenance_on_steroids` | The job starts executing for the first time (not on resumptions/retries) |
|
|
807
|
+
| `paused.maintenance_on_steroids` | A pause request is honored by the worker |
|
|
808
|
+
| `resumed.maintenance_on_steroids` | A paused run is resumed |
|
|
809
|
+
| `cancelled.maintenance_on_steroids` | A run is cancelled (directly, or honored by the worker mid-run) |
|
|
810
|
+
| `succeeded.maintenance_on_steroids` | A run completes successfully |
|
|
811
|
+
| `errored.maintenance_on_steroids` | A run raises -- payload includes `error:` with the exception |
|
|
812
|
+
|
|
813
|
+
Payload: `{ run:, task_name: }` (plus `error:` for `errored`).
|
|
814
|
+
|
|
815
|
+
## Development
|
|
816
|
+
|
|
817
|
+
### Setup
|
|
818
|
+
|
|
819
|
+
```bash
|
|
820
|
+
git clone https://github.com/igorkasyanchuk/maintenance_on_steroids.git
|
|
821
|
+
cd maintenance_on_steroids
|
|
822
|
+
bundle install
|
|
823
|
+
```
|
|
824
|
+
|
|
825
|
+
### Run the dummy app
|
|
826
|
+
|
|
827
|
+
```bash
|
|
828
|
+
cd spec/dummy
|
|
829
|
+
bin/rails db:schema:load
|
|
830
|
+
bin/rails db:seed
|
|
831
|
+
bin/rails server
|
|
832
|
+
```
|
|
833
|
+
|
|
834
|
+
Visit `http://localhost:3000/maintenance` to see the dashboard. The seed creates 12 users with different roles for testing.
|
|
835
|
+
|
|
836
|
+
### Run tests
|
|
837
|
+
|
|
838
|
+
```bash
|
|
839
|
+
bundle exec rspec
|
|
840
|
+
```
|
|
841
|
+
|
|
842
|
+
### Project structure
|
|
843
|
+
|
|
844
|
+
```
|
|
845
|
+
app/
|
|
846
|
+
controllers/maintenance_on_steroids/
|
|
847
|
+
application_controller.rb # Auth chain
|
|
848
|
+
dashboard_controller.rb # Dashboard page
|
|
849
|
+
jobs_controller.rb # Task list, detail, source
|
|
850
|
+
runs_controller.rb # Run CRUD, pause/resume/cancel, status API
|
|
851
|
+
models/maintenance_on_steroids/
|
|
852
|
+
run.rb # Run record (status, progress, user tracking)
|
|
853
|
+
artifact.rb # Artifact record (jsonb, blob, text)
|
|
854
|
+
views/maintenance_on_steroids/
|
|
855
|
+
dashboard/index.html.erb # Dashboard
|
|
856
|
+
jobs/index.html.erb # Task list
|
|
857
|
+
jobs/show.html.erb # Task detail
|
|
858
|
+
jobs/source.html.erb # Source code viewer
|
|
859
|
+
runs/new.html.erb # New run form
|
|
860
|
+
runs/show.html.erb # Run detail with live progress
|
|
861
|
+
jobs/maintenance_on_steroids/
|
|
862
|
+
run_job.rb # ActiveJob::Continuable job
|
|
863
|
+
lib/
|
|
864
|
+
maintenance_on_steroids/
|
|
865
|
+
task.rb # Base task class
|
|
866
|
+
form_dsl.rb # Form input DSL
|
|
867
|
+
artifact_dsl.rb # Artifact DSL
|
|
868
|
+
about_dsl.rb # Task metadata DSL
|
|
869
|
+
job_dsl.rb # Queue configuration DSL
|
|
870
|
+
callbacks_dsl.rb # Lifecycle callbacks DSL
|
|
871
|
+
params_proxy.rb # Typed parameter access
|
|
872
|
+
artifacts_proxy.rb # Artifact read/write
|
|
873
|
+
jsonb_artifact.rb # Hash-like JSONB wrapper
|
|
874
|
+
job_registry.rb # Task class discovery
|
|
875
|
+
engine.rb # Rails engine setup
|
|
876
|
+
generators/maintenance_on_steroids/
|
|
877
|
+
install/ # Install generator
|
|
878
|
+
job/ # Task generator
|
|
879
|
+
```
|
|
880
|
+
|
|
881
|
+
## Full Example
|
|
882
|
+
|
|
883
|
+
A complete task using most features:
|
|
884
|
+
|
|
885
|
+
```ruby
|
|
886
|
+
class MigrateUserProfiles < MaintenanceOnSteroids::Task
|
|
887
|
+
about do
|
|
888
|
+
title "Migrate User Profiles"
|
|
889
|
+
description "Migrates legacy profile data to the new format"
|
|
890
|
+
owner "Backend Team"
|
|
891
|
+
end
|
|
892
|
+
|
|
893
|
+
job do
|
|
894
|
+
queue "maintenance"
|
|
895
|
+
end
|
|
896
|
+
|
|
897
|
+
form do
|
|
898
|
+
input :batch_label, type: :string, default: "migration-v2", help_text: "Label for tracking"
|
|
899
|
+
input :dry_run, type: :boolean, default: true, help_text: "Preview changes without saving"
|
|
900
|
+
end
|
|
901
|
+
|
|
902
|
+
artifact :results, type: :jsonb, default: {}
|
|
903
|
+
artifact :error_log, type: :file, file_name: "migration_errors.csv"
|
|
904
|
+
|
|
905
|
+
after_start :log_start
|
|
906
|
+
after_complete :send_summary
|
|
907
|
+
after_error :alert_team
|
|
908
|
+
|
|
909
|
+
def collection
|
|
910
|
+
User.where(profile_version: 1)
|
|
911
|
+
end
|
|
912
|
+
|
|
913
|
+
def process(user)
|
|
914
|
+
new_data = ProfileMigrator.transform(user.profile_data)
|
|
915
|
+
|
|
916
|
+
if params[:dry_run]
|
|
917
|
+
artifacts[:results][user.id.to_s] = { status: "preview", changes: new_data }
|
|
918
|
+
else
|
|
919
|
+
user.update!(profile_data: new_data, profile_version: 2)
|
|
920
|
+
artifacts[:results][user.id.to_s] = { status: "migrated" }
|
|
921
|
+
end
|
|
922
|
+
artifacts[:results].save!
|
|
923
|
+
end
|
|
924
|
+
|
|
925
|
+
private
|
|
926
|
+
|
|
927
|
+
def log_start
|
|
928
|
+
Rails.logger.info "[MigrateUserProfiles] Started: #{params[:batch_label]}"
|
|
929
|
+
end
|
|
930
|
+
|
|
931
|
+
def send_summary
|
|
932
|
+
count = artifacts[:results].size
|
|
933
|
+
AdminMailer.migration_complete(count, params[:batch_label]).deliver_later
|
|
934
|
+
end
|
|
935
|
+
|
|
936
|
+
def alert_team
|
|
937
|
+
Slack.notify("#alerts", "Profile migration failed! Check run ##{run.id}")
|
|
938
|
+
end
|
|
939
|
+
end
|
|
940
|
+
```
|
|
941
|
+
|
|
942
|
+
## Alternatives
|
|
943
|
+
|
|
944
|
+
https://github.com/Shopify/maintenance_tasks - a gem from Shopify. This gem actually inspired me to build my version, because the gem from Shopify is missing many features I need.
|
|
945
|
+
|
|
946
|
+
## License
|
|
947
|
+
|
|
948
|
+
MIT License. See [LICENSE.txt](LICENSE.txt).
|