maintenance_on_steroids 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. checksums.yaml +7 -0
  2. data/CHANGELOG.md +267 -0
  3. data/LICENSE.txt +21 -0
  4. data/README.md +948 -0
  5. data/app/controllers/maintenance_on_steroids/application_controller.rb +66 -0
  6. data/app/controllers/maintenance_on_steroids/dashboard_controller.rb +18 -0
  7. data/app/controllers/maintenance_on_steroids/jobs_controller.rb +59 -0
  8. data/app/controllers/maintenance_on_steroids/runs_controller.rb +223 -0
  9. data/app/jobs/maintenance_on_steroids/run_job.rb +292 -0
  10. data/app/models/maintenance_on_steroids/application_record.rb +6 -0
  11. data/app/models/maintenance_on_steroids/artifact.rb +255 -0
  12. data/app/models/maintenance_on_steroids/run.rb +310 -0
  13. data/app/views/layouts/maintenance_on_steroids/application.html.erb +48 -0
  14. data/app/views/maintenance_on_steroids/dashboard/index.html.erb +163 -0
  15. data/app/views/maintenance_on_steroids/jobs/index.html.erb +35 -0
  16. data/app/views/maintenance_on_steroids/jobs/show.html.erb +107 -0
  17. data/app/views/maintenance_on_steroids/jobs/source.html.erb +79 -0
  18. data/app/views/maintenance_on_steroids/runs/new.html.erb +98 -0
  19. data/app/views/maintenance_on_steroids/runs/show.html.erb +410 -0
  20. data/app/views/maintenance_on_steroids/shared/_auto_refresh.html.erb +85 -0
  21. data/app/views/maintenance_on_steroids/shared/_javascript.html.erb +16 -0
  22. data/app/views/maintenance_on_steroids/shared/_pager.html.erb +20 -0
  23. data/app/views/maintenance_on_steroids/shared/_styles.html.erb +649 -0
  24. data/app/views/maintenance_on_steroids/shared/_task_list_item.html.erb +31 -0
  25. data/config/routes.rb +22 -0
  26. data/lib/generators/maintenance_on_steroids/install/install_generator.rb +56 -0
  27. data/lib/generators/maintenance_on_steroids/install/templates/create_maintenance_on_steroids_tables.rb.erb +56 -0
  28. data/lib/generators/maintenance_on_steroids/install/templates/initializer.rb +42 -0
  29. data/lib/generators/maintenance_on_steroids/job/job_generator.rb +17 -0
  30. data/lib/generators/maintenance_on_steroids/job/templates/job.rb.erb +30 -0
  31. data/lib/maintenance_on_steroids/about_dsl.rb +51 -0
  32. data/lib/maintenance_on_steroids/artifact_dsl.rb +82 -0
  33. data/lib/maintenance_on_steroids/artifacts_proxy.rb +205 -0
  34. data/lib/maintenance_on_steroids/callbacks_dsl.rb +61 -0
  35. data/lib/maintenance_on_steroids/csv_artifact.rb +88 -0
  36. data/lib/maintenance_on_steroids/engine.rb +26 -0
  37. data/lib/maintenance_on_steroids/form_dsl.rb +63 -0
  38. data/lib/maintenance_on_steroids/instrumentation.rb +33 -0
  39. data/lib/maintenance_on_steroids/job_dsl.rb +61 -0
  40. data/lib/maintenance_on_steroids/job_registry.rb +83 -0
  41. data/lib/maintenance_on_steroids/jsonb_artifact.rb +39 -0
  42. data/lib/maintenance_on_steroids/params_proxy.rb +93 -0
  43. data/lib/maintenance_on_steroids/task.rb +109 -0
  44. data/lib/maintenance_on_steroids/text_artifact.rb +74 -0
  45. data/lib/maintenance_on_steroids/version.rb +3 -0
  46. data/lib/maintenance_on_steroids.rb +187 -0
  47. metadata +125 -0
data/README.md ADDED
@@ -0,0 +1,948 @@
1
+ # Maintenance on Steroids
2
+
3
+ ![Maintenance on Steroids dashboard walkthrough](docs/demo.gif)
4
+
5
+ A powerful maintenance task runner for **Rails 8.1+** that leverages `ActiveJob::Continuable` for safe, resumable background processing. Think of it as a batteries-included toolkit for one-off data migrations, batch updates, CSV exports, and any maintenance work your app needs.
6
+
7
+ **Built-in web dashboard** with dark/light themes, live progress tracking, pause/resume/cancel controls, source code viewer, and artifact downloads.
8
+
9
+ > [!IMPORTANT]
10
+ > **Requires Rails >= 8.1.3.1, < 9 and Ruby >= 3.2.** Resumption uses `ActiveJob::Continuable`. The minimum Rails patch includes security fixes; keep your host application's full bundle patched too.
11
+
12
+ ## Features
13
+
14
+ - **Collection & callable tasks** -- iterate over ActiveRecord relations or run one-off jobs
15
+ - **Automatic resumption** -- cursor-based progress tracking resumes from the last committed checkpoint; tasks must tolerate retries
16
+ - **Rich DSL** -- typed form inputs, named artifacts, lifecycle callbacks, queue configuration, task metadata
17
+ - **Live dashboard** -- auto-refreshing stats and active runs, real-time progress bars, status badges, run history, and source code viewer
18
+ - **Estimated time remaining** -- live ETA for pending records, extrapolated from the current processing rate
19
+ - **Artifacts** -- store JSON results, export CSV/binary files, downloadable from the UI, with per-run indicators on the dashboard
20
+ - **Pause / Resume / Cancel** -- safely interrupt long-running tasks mid-execution
21
+ - **Instrumentation** -- `ActiveSupport::Notifications` events for every run lifecycle transition
22
+ - **User tracking** -- records who triggered each run with configurable display
23
+ - **Authentication & access control** -- layered, off by default: HTTP Basic (constant-time compare), a custom auth hook for Devise/Warden/etc., and a `verify_access_proc` for role- or IP-based authorization ([jump to setup](#authentication--access-control))
24
+ - **Dark & light themes** -- toggle with one click, persisted in localStorage
25
+
26
+ ## Requirements
27
+
28
+ - **Rails >= 8.1.3.1, < 9** — resumption depends on `ActiveJob::Continuable`
29
+ - **Ruby >= 3.2**
30
+
31
+ ## Installation
32
+
33
+ Add the gem to your Gemfile:
34
+
35
+ ```ruby
36
+ gem "maintenance_on_steroids"
37
+ ```
38
+
39
+ Run the install generator:
40
+
41
+ ```bash
42
+ bundle install
43
+ rails generate maintenance_on_steroids:install
44
+ rails db:migrate
45
+ ```
46
+
47
+ This will:
48
+ 1. Create the database migration for runs and artifacts tables
49
+ 2. Create the `app/maintenance/` directory for your task classes
50
+ 3. Mount the engine at `/maintenance` in your routes
51
+
52
+ Update and audit the host application's complete bundle, including Rack and
53
+ other transitive dependencies. This gem does not ship a lockfile; the security
54
+ scans in this repository check its development bundle, not your host's versions.
55
+
56
+ ## Quick Start
57
+
58
+ ### Generate a task
59
+
60
+ ```bash
61
+ rails generate maintenance_on_steroids:job BackfillUserNames
62
+ ```
63
+
64
+ This creates `app/maintenance/backfill_user_names.rb`:
65
+
66
+ ```ruby
67
+ class BackfillUserNames < MaintenanceOnSteroids::Task
68
+ about do
69
+ title "Backfill User Names"
70
+ description "TODO: Add description"
71
+ end
72
+
73
+ def collection
74
+ User.where(name: nil)
75
+ end
76
+
77
+ def process(record)
78
+ record.update!(name: record.email.split("@").first)
79
+ end
80
+ end
81
+ ```
82
+
83
+ ### Generate a callable (one-off) task
84
+
85
+ ```bash
86
+ rails generate maintenance_on_steroids:job ClearExpiredTokens --collection=false
87
+ ```
88
+
89
+ ```ruby
90
+ class ClearExpiredTokens < MaintenanceOnSteroids::Task
91
+ about do
92
+ title "Clear Expired Tokens"
93
+ description "TODO: Add description"
94
+ end
95
+
96
+ def call
97
+ Token.where("expires_at < ?", Time.current).delete_all
98
+ end
99
+ end
100
+ ```
101
+
102
+ ### Run it
103
+
104
+ Start your Rails server and visit **`/maintenance`**. You'll see the dashboard with your tasks listed. Click a task, then "New Run" to start it.
105
+
106
+ ## Task DSL
107
+
108
+ ### About
109
+
110
+ Describe your task for the dashboard:
111
+
112
+ ```ruby
113
+ class MyTask < MaintenanceOnSteroids::Task
114
+ about do
115
+ title "My Task"
116
+ description "Does something useful"
117
+ owner "Backend Team"
118
+ end
119
+ end
120
+ ```
121
+
122
+ If `title` is omitted, the class name is used (e.g. `MyTask` becomes "My Task").
123
+
124
+ ### Collection tasks
125
+
126
+ Define `collection` (returns an `ActiveRecord::Relation`) and `process` (handles one record):
127
+
128
+ ```ruby
129
+ class DeactivateOldUsers < MaintenanceOnSteroids::Task
130
+ about do
131
+ title "Deactivate Old Users"
132
+ description "Deactivates users who haven't logged in for a year"
133
+ end
134
+
135
+ def collection
136
+ User.where("last_sign_in_at < ?", 1.year.ago).where(active: true)
137
+ end
138
+
139
+ def process(user)
140
+ user.update!(active: false)
141
+ end
142
+ end
143
+ ```
144
+
145
+ The engine iterates using `find_each` and commits a cursor and accumulated output after each record. Resumption starts after that checkpoint. Processing is **at least once**: a crash after a task side effect but before its checkpoint can repeat that record, so make task effects idempotent.
146
+
147
+ ### Callable tasks
148
+
149
+ For one-off jobs that don't iterate over a collection, define `call`:
150
+
151
+ ```ruby
152
+ class RecalculateStats < MaintenanceOnSteroids::Task
153
+ about do
154
+ title "Recalculate Stats"
155
+ description "Rebuilds all cached statistics"
156
+ end
157
+
158
+ def call
159
+ StatsService.rebuild_all
160
+ end
161
+ end
162
+ ```
163
+
164
+ ### Form Inputs
165
+
166
+ Accept parameters from the UI with typed inputs:
167
+
168
+ ```ruby
169
+ class UpdateUserAges < MaintenanceOnSteroids::Task
170
+ form do
171
+ input :name, type: :string, required: true, placeholder: "Filter by name"
172
+ input :new_age, type: :integer, required: true, default: 25
173
+ end
174
+
175
+ def collection
176
+ User.where(name: params[:name])
177
+ end
178
+
179
+ def process(user)
180
+ user.update!(age: params[:new_age])
181
+ end
182
+ end
183
+ ```
184
+
185
+ The `params` proxy provides typed access -- `:integer` values are cast to `Integer`, `:boolean` to `true/false`, etc.
186
+
187
+ **Supported input types:**
188
+
189
+ | Type | HTML Element | Cast |
190
+ |------|-------------|------|
191
+ | `:string` | `<input type="text">` | String |
192
+ | `:integer` | `<input type="number">` | Integer |
193
+ | `:float` | `<input type="number" step="any">` | Float |
194
+ | `:boolean` | `<input type="checkbox">` | Boolean |
195
+ | `:text` | `<textarea>` | String |
196
+ | `:date` | `<input type="date">` | Date |
197
+ | `:datetime` | `<input type="datetime-local">` | Time |
198
+ | `:blob` | `<input type="file">` | Binary |
199
+ | `:select` | `<select>` | String |
200
+
201
+ **Input options:**
202
+
203
+ ```ruby
204
+ input :role,
205
+ type: :select,
206
+ options: %w[admin user moderator],
207
+ required: true,
208
+ default: "user",
209
+ label: "User Role",
210
+ placeholder: "Choose a role",
211
+ help_text: "The role to assign to matched users"
212
+ ```
213
+
214
+ **File uploads (`type: :blob`):** the uploaded file is stored with the run (as an `input` artifact) and read through `params`:
215
+
216
+ - `params[:name]` -- the raw file **bytes** (a `String`), or `nil` if nothing was uploaded
217
+ - `params.file_name(:name)` -- the original filename
218
+ - `params.content_type(:name)` -- the uploaded MIME type
219
+
220
+ ```ruby
221
+ class CountLetterATask < MaintenanceOnSteroids::Task
222
+ form do
223
+ input :file, type: :blob, required: true, help_text: "CSV/text file to scan"
224
+ end
225
+
226
+ artifact :result, type: :jsonb, default: {}
227
+
228
+ def call
229
+ content = params[:file].to_s # raw bytes of the upload
230
+
231
+ artifacts.save(:result, {
232
+ "file_name" => params.file_name(:file),
233
+ "content_type" => params.content_type(:file),
234
+ "bytes" => content.bytesize,
235
+ "a_count" => content.count("aA") # letter "a", case-insensitive
236
+ })
237
+ end
238
+ end
239
+ ```
240
+
241
+ Uploads are read into memory and capped by `config.max_upload_size` (see Configuration). Need a CSV as rows? `CSV.parse(params[:file])`.
242
+
243
+ ### Artifacts
244
+
245
+ Store output data (JSON, files, text) that persists with the run:
246
+
247
+ ```ruby
248
+ class ExportUsers < MaintenanceOnSteroids::Task
249
+ about do
250
+ title "Export Users to CSV"
251
+ end
252
+
253
+ artifact :csv_file, type: :file, file_name: "users.csv"
254
+
255
+ def call
256
+ csv_data = CSV.generate do |csv|
257
+ csv << %w[id name email]
258
+ User.find_each do |user|
259
+ csv << [user.id, user.name, user.email]
260
+ end
261
+ end
262
+
263
+ artifacts[:csv_file] = csv_data
264
+ end
265
+ end
266
+ ```
267
+
268
+ The generated file is downloadable from the run's detail page in the UI.
269
+
270
+ **Artifact types:**
271
+
272
+ | Type | Storage | Use case |
273
+ |------|---------|----------|
274
+ | `:jsonb` | JSON column | Structured results, counters, logs |
275
+ | `:file` | Binary blob | PDFs, images, pre-rendered files |
276
+ | `:text` | Text column | Plain text output |
277
+ | `:csv` | Binary blob | Row-oriented exports, table-previewed in the UI |
278
+
279
+ An unknown `type:` raises `ArgumentError` at load time, so typos surface immediately.
280
+
281
+ **Declaration options** (all types): `label:` (human name shown in the UI, defaults to the humanized artifact name), `description:` (shown under the artifact on the run page), `content_type:` (download MIME -- otherwise inferred from `file_name`), plus `default:`, `file_name:`, and `headers:` (CSV).
282
+
283
+ **Writing artifacts -- two styles:**
284
+
285
+ 1. **Explicit (`save`)** -- persist a whole value immediately. The type comes from the declaration, so one call covers every kind. This is the simplest path and never relies on auto-flush:
286
+
287
+ ```ruby
288
+ artifacts.save(:json_data, { name: "Igor", age: 40 }) # jsonb
289
+ artifacts.save(:export, rows) # csv (array of rows, or a String)
290
+ artifacts.save(:log, "done") # text
291
+ # artifacts[:json_data] = { ... } is an alias of save
292
+ ```
293
+
294
+ 2. **Accumulator (`<<` / in-place)** -- build the value incrementally across records; the job auto-flushes it (see Auto-flush below). Best for resumable tasks that accrue output row by row:
295
+
296
+ ```ruby
297
+ artifacts.export << [user.id, user.name] # append a CSV row
298
+ artifacts.result[user.id.to_s] = { ok: true } # mutate a JSONB hash in place
299
+ ```
300
+
301
+ **Access** -- `artifacts[:name]` and method style are equivalent:
302
+
303
+ ```ruby
304
+ artifacts[:result]["k"] = v
305
+ artifacts.result["k"] = v # same thing, reads nicer
306
+ ```
307
+
308
+ **JSONB artifacts** behave like a hash:
309
+
310
+ ```ruby
311
+ artifact :result, type: :jsonb, default: {}
312
+
313
+ def process(user)
314
+ user.update!(active: false)
315
+ artifacts.result[user.id.to_s] = { deactivated: true }
316
+ end
317
+ ```
318
+
319
+ **CSV artifacts** are append-oriented and render to a downloadable `.csv`:
320
+
321
+ ```ruby
322
+ artifact :export, type: :csv, headers: %w[id name email], description: "All users"
323
+
324
+ def call
325
+ User.find_each { |u| artifacts.export << [u.id, u.name, u.email] }
326
+ # auto-flushed on completion; previewed as a table on the run page
327
+ end
328
+ ```
329
+
330
+ **Auto-flush:** collection tasks commit mutated accumulators together with each record's cursor. Callable tasks flush at `checkpoint!` and on completion or interruption. Completion callbacks can produce final output; it is saved before the run becomes `completed` and before the success notification. Pause/cancel callbacks likewise run before the final state is committed. Persistence failures mark the run `errored`; failed collection-record buffers are discarded. Reads alone never create a row. Explicit `artifacts.save` / `save!` writes happen immediately and are not coupled to the collection cursor, so use accumulators for checkpointed exports.
331
+
332
+ **Metadata:** every artifact records lightweight stats on save -- entry/line count, byte size, and a generated-at timestamp -- shown on the run page (`12 entries · 3.4 KB · 2 minutes ago`) without loading the full payload. Available on the model via `artifact.summary`, `artifact.byte_size`, and `artifact.generated_at`.
333
+
334
+ **File artifacts** -- if `file_name` is omitted, it defaults to `task_class_name_YYYYMMDD_HHMMSS`:
335
+
336
+ ```ruby
337
+ artifact :export, type: :file
338
+ # File name auto-generated: "export_users_20260218_143022"
339
+
340
+ artifact :report, type: :file, file_name: "report.pdf"
341
+ # Explicit file name: "report.pdf"
342
+ ```
343
+
344
+ ### Callbacks
345
+
346
+ Hook into the task lifecycle:
347
+
348
+ ```ruby
349
+ class ImportUsers < MaintenanceOnSteroids::Task
350
+ after_start :notify_started
351
+ after_complete :notify_completed
352
+ after_error :notify_failed
353
+ after_pause :log_pause
354
+ after_cancel :cleanup
355
+ after_interrupt :save_progress
356
+
357
+ def collection
358
+ User.where(imported: false)
359
+ end
360
+
361
+ def process(user)
362
+ user.update!(imported: true)
363
+ end
364
+
365
+ private
366
+
367
+ def notify_started
368
+ Slack.notify("#imports", "User import started")
369
+ end
370
+
371
+ def notify_completed
372
+ Slack.notify("#imports", "User import completed!")
373
+ end
374
+
375
+ def notify_failed
376
+ Slack.notify("#imports", "User import failed!")
377
+ end
378
+
379
+ def log_pause
380
+ Rails.logger.info "Import paused by user"
381
+ end
382
+
383
+ def cleanup
384
+ TempFile.cleanup
385
+ end
386
+
387
+ def save_progress
388
+ # Called on both pause and cancel, before the specific callback
389
+ end
390
+ end
391
+ ```
392
+
393
+ **Available callbacks:**
394
+
395
+ | Callback | When it fires |
396
+ |----------|--------------|
397
+ | `after_start` | Task begins execution |
398
+ | `after_complete` | Task finishes successfully |
399
+ | `after_error` | Task raises an exception |
400
+ | `after_pause` | Task is paused |
401
+ | `after_cancel` | Task is cancelled |
402
+ | `after_interrupt` | Task is interrupted (fires before pause or cancel) |
403
+
404
+ ### Job Configuration
405
+
406
+ Set a custom queue and/or priority:
407
+
408
+ ```ruby
409
+ class HeavyExport < MaintenanceOnSteroids::Task
410
+ job do
411
+ queue "exports"
412
+ priority 10
413
+ end
414
+
415
+ def call
416
+ # ...
417
+ end
418
+ end
419
+ ```
420
+
421
+ ### Database Role Switching
422
+
423
+ To scan a collection against a replica, declare the role. The job holds it open
424
+ for the whole scan:
425
+
426
+ ```ruby
427
+ class BackfillTask < MaintenanceOnSteroids::Task
428
+ job do
429
+ database_role :reading # :read is accepted too
430
+ end
431
+
432
+ def collection
433
+ User.where(active: true) # every batch query runs on the replica
434
+ end
435
+
436
+ def process(user)
437
+ user.update!(...) # writes run on the primary
438
+ end
439
+ end
440
+ ```
441
+
442
+ `process` and the run's own cursor/progress bookkeeping step back to `:writing`
443
+ automatically, so a declared read role never blocks the task's writes.
444
+
445
+ For a one-off read inside `call` or `process`, `with_database_role` wraps a
446
+ block:
447
+
448
+ ```ruby
449
+ def call
450
+ stale = with_database_role(:read) { User.where(active: false).count }
451
+ artifacts.save(:summary, { stale: stale })
452
+ end
453
+ ```
454
+
455
+ > **The block must force whatever it reads.** Returning a lazy `Relation` from
456
+ > `with_database_role` does nothing — the role is restored on the way out and
457
+ > the query runs later, on the primary. That is why `collection` uses the
458
+ > declarative `database_role` above instead.
459
+
460
+ ## Configuration
461
+
462
+ `rails g maintenance_on_steroids:install` writes
463
+ `config/initializers/maintenance_on_steroids.rb` for you, with every access
464
+ control commented out and ready to fill in. The full set of options:
465
+
466
+ ```ruby
467
+ MaintenanceOnSteroids.configure do |config|
468
+ # --- Authentication ---
469
+
470
+ # HTTP Basic auth (simplest option)
471
+ config.http_basic_authentication_enabled = true
472
+ config.http_basic_authentication_user_name = "admin"
473
+ config.http_basic_authentication_password = Rails.application.credentials.maintenance_password
474
+
475
+ # Or use a custom authentication hook (e.g. Devise)
476
+ config.authentication = -> {
477
+ authenticate_user! # Devise method
478
+ }
479
+
480
+ # Role-based access control
481
+ config.verify_access_proc = ->(controller) {
482
+ controller.current_user&.admin?
483
+ }
484
+
485
+ # --- User Tracking ---
486
+
487
+ # Track who triggers each run
488
+ config.current_user_resolver = -> {
489
+ Current.user # Or any way to get the current user
490
+ }
491
+
492
+ # Customize how user names are displayed
493
+ config.user_display_formatter = ->(run) {
494
+ User.find(run.user_id).full_name rescue run.user_email
495
+ }
496
+ end
497
+ ```
498
+
499
+ ### Configuration Options
500
+
501
+ | Option | Default | Description |
502
+ |--------|---------|-------------|
503
+ | `parent_controller` | `"ActionController::Base"` | Base controller class for the engine |
504
+ | `max_upload_size` | `50.megabytes` | Maximum size (bytes) for file inputs uploaded when starting a run |
505
+ | `max_artifact_size` | `64.megabytes` | Hard ceiling on a single stored artifact; exceeding it fails the run |
506
+ | `http_basic_authentication_enabled` | `false` | Enable HTTP Basic auth |
507
+ | `http_basic_authentication_user_name` | `"admin"` | HTTP Basic username |
508
+ | `http_basic_authentication_password` | `"secret"` | HTTP Basic password |
509
+ | `authentication` | `nil` | Proc executed in controller context for auth |
510
+ | `verify_access_proc` | `nil` | Proc receiving controller, return true/false |
511
+ | `current_user_resolver` | `nil` | Proc returning the current user object |
512
+ | `user_display_formatter` | `nil` | Proc receiving Run, returning display string |
513
+ | `allow_insecure_dashboard` | `false` | Boot in production with no access-control layer configured |
514
+
515
+ ### Authentication & Access Control
516
+
517
+ The dashboard is mounted in **your** app, so it inherits your app's middleware — but it ships with **no access control of its own until you configure it**. Three independent layers run as `before_action`s on every engine request, in order. Use any combination; each layer is skipped when its config is left at the default.
518
+
519
+ | Order | Layer | Config | When it runs | On failure |
520
+ |-------|-------|--------|--------------|------------|
521
+ | 1 | **HTTP Basic** | `http_basic_authentication_enabled` | Always (when enabled) | `401 Unauthorized` (browser credential prompt) |
522
+ | 2 | **Auth hook** | `authentication` | After Basic passes | Whatever the proc does (usually `redirect_to` sign-in) |
523
+ | 3 | **Access check** | `verify_access_proc` | After the hook | `403 Forbidden` (`"Access denied"`) |
524
+
525
+ > [!WARNING]
526
+ > **In production the engine refuses to boot until one layer is configured.** With none of them set, anyone who can reach the mounted path can run, pause, and cancel maintenance tasks against your database.
527
+ >
528
+ > The shipped HTTP Basic defaults (`admin` / `secret`) are **placeholders**, and leaving the password at `"secret"` counts as *unconfigured* — a dashboard behind a published password is not protected, and treating it as configured would make the most dangerous setup the quietest one. Override it, and prefer HTTPS since Basic credentials travel in every request.
529
+ >
530
+ > If the dashboard is already gated somewhere this gem cannot see — a reverse proxy, a VPN, your own middleware — say so explicitly rather than leaving the layers empty:
531
+ >
532
+ > ```ruby
533
+ > config.allow_insecure_dashboard = true
534
+ > ```
535
+ >
536
+ > Outside production, an unconfigured dashboard logs a warning instead of raising.
537
+
538
+ #### Recipe: HTTP Basic (quickest)
539
+
540
+ Good for a staging box or a small team. Credentials are compared in constant time (`ActiveSupport::SecurityUtils.secure_compare`), so it's not vulnerable to timing attacks.
541
+
542
+ ```ruby
543
+ # config/initializers/maintenance_on_steroids.rb
544
+ MaintenanceOnSteroids.configure do |config|
545
+ config.http_basic_authentication_enabled = true
546
+ config.http_basic_authentication_user_name = "admin"
547
+ config.http_basic_authentication_password = Rails.application.credentials.maintenance_password
548
+ end
549
+ ```
550
+
551
+ #### Recipe: Devise + admin role (most common)
552
+
553
+ Reuse your app's existing login, then restrict to admins. Point `parent_controller` at your `ApplicationController` so Devise's helpers (`current_user`, `authenticate_user!`) are in scope.
554
+
555
+ ```ruby
556
+ MaintenanceOnSteroids.configure do |config|
557
+ config.parent_controller = "ApplicationController"
558
+
559
+ # Layer 2: bounce anyone who isn't signed in
560
+ config.authentication = -> {
561
+ redirect_to(main_app.new_user_session_path, alert: "Please sign in.") unless current_user
562
+ }
563
+
564
+ # Layer 3: of the signed-in users, only admins get in (renders 403 otherwise)
565
+ config.verify_access_proc = ->(controller) { controller.current_user&.admin? }
566
+
567
+ # Stamp each run with who triggered it (shown in the UI)
568
+ config.current_user_resolver = -> { current_user }
569
+ end
570
+ ```
571
+
572
+ #### Recipe: IP allowlist
573
+
574
+ `verify_access_proc` receives the controller, so any request attribute is fair game — e.g. lock the dashboard to your office/VPN range:
575
+
576
+ ```ruby
577
+ ALLOWED_IPS = %w[203.0.113.4 198.51.100.0/24].map { |ip| IPAddr.new(ip) }
578
+
579
+ config.verify_access_proc = ->(controller) {
580
+ ip = IPAddr.new(controller.request.remote_ip)
581
+ ALLOWED_IPS.any? { |range| range.include?(ip) }
582
+ }
583
+ ```
584
+
585
+ Combine layers freely — e.g. HTTP Basic **and** an IP check both have to pass. Each returns independently, so the first failing layer short-circuits the request.
586
+
587
+ ## Dashboard
588
+
589
+ The engine provides a full-featured web UI at your mounted path (default: `/maintenance`).
590
+
591
+ ### Pages
592
+
593
+ - **Dashboard** -- stats overview, active runs, and recent history; runs with output artifacts show a 📎 indicator with the artifact count
594
+ - **Tasks** -- all registered task classes, sortable by name (default) or last execution time; each row shows the last run's status and age, and never-executed tasks carry a "New" badge
595
+ - **Task detail** -- task metadata, run history, "New Run" and "Source" buttons
596
+ - **Source viewer** -- view the Ruby source code of any task class
597
+ - **New Run** -- form with typed inputs to start a task
598
+ - **Run detail** -- live progress bar, status, duration, estimated time remaining, parameters, artifacts, pause/resume/cancel controls, and a "View Source" shortcut
599
+
600
+ ### Live Progress
601
+
602
+ Active runs poll for status updates every 2 seconds. The progress bar, percentage, status badge, duration, and the **Estimated (pending)** time remaining (`2d 4h 12m 30s`-style, leading zero units omitted) update in real-time without page refresh.
603
+
604
+ The dashboard auto-refreshes its stats and active-runs table every 2 seconds in the background (paused while the tab is hidden) -- no flash, no scroll jumps.
605
+
606
+ While a run is in a transitional state (`pausing`, `cancelling`), the run page shows a visible auto-refresh countdown next to the status badge and reloads every few seconds until the worker settles the state.
607
+
608
+ ### Theme
609
+
610
+ The UI supports dark and light themes. Click the sun/moon toggle in the top bar. Your preference is saved in localStorage.
611
+
612
+ ## How Resumption Works
613
+
614
+ This gem uses Rails 8.1's `ActiveJob::Continuable` for safe background processing:
615
+
616
+ 1. **Collection tasks** iterate over records using `find_each`, ordered by primary key
617
+ 2. After processing each record, accumulated output and the database cursor commit in one transaction, then `ActiveJob::Continuable`'s step cursor advances
618
+ 3. If Sidekiq restarts mid-job, `ActiveJob::Continuable` resumes from its last cursor
619
+ 4. If you pause and resume, a new job is enqueued that reads the cursor from the database
620
+ 5. In both cases, the query uses `WHERE id > cursor` to skip already-processed records
621
+
622
+ **Task side effects are at least once.** A record whose checkpoint committed is skipped on resume. A crash after a database/API side effect but before the checkpoint can repeat that side effect. Use idempotent updates or a durable deduplication key for external operations. The engine cannot make another database or service commit atomically with its cursor.
623
+
624
+ Collection accumulator output and its cursor survive together, including hard
625
+ worker death. Output from an unfinished record is discarded and that record is
626
+ retried. Callable tasks restart `call` on manual resume; `checkpoint!` saves output
627
+ and updates the heartbeat, but does not remember a position inside your method.
628
+ Persist your own callable work position and make retries safe. Hard termination
629
+ can lose callable output since its last checkpoint.
630
+
631
+ Each worker atomically claims a run with an execution token. Duplicate deliveries
632
+ and stale job IDs cannot claim it. Reaping revokes that token; an old worker stops
633
+ at its next checkpoint and cannot overwrite engine progress or artifacts. A
634
+ currently executing external side effect cannot be recalled, so set reaper
635
+ thresholds conservatively.
636
+
637
+ > **Note:** cursor-based resumption relies on monotonically increasing primary keys. Collections with UUID/string primary keys can skip or repeat records on resume -- the job logs a warning when it detects one.
638
+
639
+ ## Operations
640
+
641
+ ### Recovering from worker crashes
642
+
643
+ If a worker process dies hard (OOM kill, `kill -9`, node failure), its run can be left in `running` forever. `Run.reap_stale!` transitions in-flight runs whose row hasn't been touched recently to `errored` (the job updates the row at least once per processed record, so `updated_at` acts as a heartbeat):
644
+
645
+ ```ruby
646
+ # Run periodically (cron, recurring job, e.g. solid_queue recurring task):
647
+ MaintenanceOnSteroids::Run.reap_stale!(threshold: 30.minutes)
648
+ ```
649
+
650
+ Pick a threshold comfortably larger than the time your slowest record or callable
651
+ checkpoint interval takes. `checkpoint!` refreshes a callable's heartbeat.
652
+ Inside a collection's `process`, it refreshes the heartbeat and checks ownership;
653
+ pause/cancel waits until that record finishes, keeping output and cursor together.
654
+ `enqueued` and `paused` runs are excluded by default. To recover a process crash
655
+ between saving a run and dispatching its job, opt into an enqueue timeout longer
656
+ than your maximum queue delay:
657
+
658
+ ```ruby
659
+ MaintenanceOnSteroids::Run.reap_stale!(threshold: 30.minutes, enqueued_threshold: 1.day)
660
+ ```
661
+
662
+ An expired queued job is ignored unless an operator resumes the run, which
663
+ creates a new job identity. Queue submission failures, including refused
664
+ Continuable retries, are surfaced as `errored`.
665
+
666
+ ### Recovering from a failed run
667
+
668
+ A run that raises is marked `errored` and stops, with the message and backtrace on the run page. Fix the cause and hit **Resume** to restart after the last committed checkpoint. Uncheckpointed effects can be repeated. Resuming prepares fresh metadata before dispatch, preserving any new worker error even if that worker finishes immediately.
669
+
670
+ ### Pruning old runs
671
+
672
+ Nothing expires runs on its own, so a long-lived app keeps every run, backtrace
673
+ and stored artifact forever. `Run.prune!` deletes finished runs (and their
674
+ artifacts) past a cutoff; schedule it like `reap_stale!`:
675
+
676
+ ```ruby
677
+ MaintenanceOnSteroids::Run.prune!(older_than: 90.days)
678
+ ```
679
+
680
+ Only `completed`, `cancelled` and `errored` runs are eligible — anything active
681
+ or paused is left alone regardless of age. Returns the number deleted.
682
+
683
+ ### Artifact size limits
684
+
685
+ **Artifacts are buffered in the worker's memory and stored in a single database
686
+ row.** That makes them ideal for maintenance *results* — a summary, a log, a
687
+ few thousand rows of exceptions — and unsuitable as a bulk-export pipeline.
688
+ A million-row CSV will exhaust the worker before it ever reaches the database.
689
+
690
+ `MaintenanceOnSteroids.max_artifact_size` (default 64 MB) rejects oversized
691
+ persistent output and checks text/CSV appends before adding them to the buffer.
692
+ It is not a process memory limit: arbitrary task allocations, nested JSON edits,
693
+ serialization, and multiple artifacts can use more memory. Each collection
694
+ checkpoint rewrites dirty artifacts, so keep them small. Write large output to
695
+ object storage and keep only a reference:
696
+
697
+ ```ruby
698
+ def process(record)
699
+ # ... build rows ...
700
+ end
701
+
702
+ after_complete do
703
+ key = S3Uploader.call(big_file)
704
+ artifacts.save(:location, { bucket: "exports", key: key })
705
+ end
706
+ ```
707
+
708
+ Inline preview rendering is capped (256 KB, or 200 top-level JSON entries), but
709
+ the database still returns complete payloads. Large or numerous artifacts can
710
+ consume substantial web memory. File and CSV artifacts have a Download action.
711
+
712
+ ### Pausing a long-running callable task
713
+
714
+ Collection tasks check for a pending pause or cancel between records. A callable
715
+ task is a single unit of work, so a long `call` won't notice Pause until it
716
+ returns. Call `checkpoint!` at the points where stopping is safe:
717
+
718
+ ```ruby
719
+ def call
720
+ Account.find_each do |account|
721
+ checkpoint! # honours a pending Pause/Cancel here
722
+ account.recalculate!
723
+ end
724
+ end
725
+ ```
726
+
727
+ When a stop is pending, `checkpoint!` does not return — the job unwinds, buffered
728
+ artifacts are flushed, and the run lands in `paused` or `cancelled`.
729
+
730
+ ### Preventing concurrent runs
731
+
732
+ Nothing stops an operator from starting the same task twice — a double-clicked
733
+ **New Run** on a destructive task runs it twice. Declare a limit:
734
+
735
+ ```ruby
736
+ class BackfillTask < MaintenanceOnSteroids::Task
737
+ job do
738
+ concurrency 1
739
+ end
740
+ end
741
+ ```
742
+
743
+ Further runs are refused while that many are still active. The check is advisory
744
+ (two simultaneous submissions can still race); it exists to catch the double-click,
745
+ not to provide a distributed lock. For a hard guarantee, take an advisory lock
746
+ inside the task itself.
747
+
748
+ ### Content Security Policy
749
+
750
+ The dashboard's live updates use small inline `<script>` blocks and no build
751
+ step, so a strict CSP without `'unsafe-inline'` will block polling (the pages
752
+ still render and work; they just stop refreshing themselves). If your app sets
753
+ a strict policy, scope an exception to the mounted path:
754
+
755
+ ```ruby
756
+ # config/initializers/content_security_policy.rb
757
+ Rails.application.config.content_security_policy_nonce_directives = %w[script-src]
758
+ ```
759
+
760
+ or exclude the engine's path from the policy entirely.
761
+
762
+ ### Collections with UUID or string primary keys
763
+
764
+ Resumption works by remembering the last processed primary key and continuing
765
+ with `WHERE id > cursor`, which assumes keys increase over time. That holds for
766
+ integer and bigint keys, and for time-ordered UUIDs (UUIDv7, ULID). It does
767
+ **not** hold for random UUIDv4: after an interruption such a run can skip
768
+ records it never processed and reprocess others.
769
+
770
+ The job logs a warning when it detects a non-integer primary key. If your
771
+ collection uses random UUIDs, either avoid pause/resume for it or scope the
772
+ collection so each run is complete in itself.
773
+
774
+ ### Running on SQLite
775
+
776
+ A running task writes to the runs table after every processed record, so the dashboard's own writes (pause, cancel, resume) compete with the worker for SQLite's single writer. Make sure your `database.yml` sets a busy timeout, or those clicks will fail with `SQLite3::BusyException: database is locked`:
777
+
778
+ ```yaml
779
+ production:
780
+ adapter: sqlite3
781
+ database: storage/production.sqlite3
782
+ timeout: 5000 # ms to wait for the write lock instead of failing instantly
783
+ ```
784
+
785
+ The default is `0` -- no waiting at all. Postgres and MySQL need no equivalent setting.
786
+
787
+ ## Instrumentation
788
+
789
+ Every run lifecycle transition emits an `ActiveSupport::Notifications` event in the `maintenance_on_steroids` namespace -- the same pattern as the `maintenance_tasks` gem:
790
+
791
+ ```ruby
792
+ ActiveSupport::Notifications.subscribe("enqueued.maintenance_on_steroids") do |event|
793
+ run = event.payload[:run]
794
+ Rails.logger.info "Enqueued #{event.payload[:task_name]} (run ##{run.id})"
795
+ end
796
+
797
+ # Or subscribe to all events at once:
798
+ ActiveSupport::Notifications.subscribe(/\.maintenance_on_steroids\z/) do |event|
799
+ StatsD.increment("maintenance.#{event.name.split('.').first}")
800
+ end
801
+ ```
802
+
803
+ | Event | Fired when |
804
+ |-------|-----------|
805
+ | `enqueued.maintenance_on_steroids` | A run is enqueued (initial enqueue and re-enqueue on resume) |
806
+ | `started.maintenance_on_steroids` | The job starts executing for the first time (not on resumptions/retries) |
807
+ | `paused.maintenance_on_steroids` | A pause request is honored by the worker |
808
+ | `resumed.maintenance_on_steroids` | A paused run is resumed |
809
+ | `cancelled.maintenance_on_steroids` | A run is cancelled (directly, or honored by the worker mid-run) |
810
+ | `succeeded.maintenance_on_steroids` | A run completes successfully |
811
+ | `errored.maintenance_on_steroids` | A run raises -- payload includes `error:` with the exception |
812
+
813
+ Payload: `{ run:, task_name: }` (plus `error:` for `errored`).
814
+
815
+ ## Development
816
+
817
+ ### Setup
818
+
819
+ ```bash
820
+ git clone https://github.com/igorkasyanchuk/maintenance_on_steroids.git
821
+ cd maintenance_on_steroids
822
+ bundle install
823
+ ```
824
+
825
+ ### Run the dummy app
826
+
827
+ ```bash
828
+ cd spec/dummy
829
+ bin/rails db:schema:load
830
+ bin/rails db:seed
831
+ bin/rails server
832
+ ```
833
+
834
+ Visit `http://localhost:3000/maintenance` to see the dashboard. The seed creates 12 users with different roles for testing.
835
+
836
+ ### Run tests
837
+
838
+ ```bash
839
+ bundle exec rspec
840
+ ```
841
+
842
+ ### Project structure
843
+
844
+ ```
845
+ app/
846
+ controllers/maintenance_on_steroids/
847
+ application_controller.rb # Auth chain
848
+ dashboard_controller.rb # Dashboard page
849
+ jobs_controller.rb # Task list, detail, source
850
+ runs_controller.rb # Run CRUD, pause/resume/cancel, status API
851
+ models/maintenance_on_steroids/
852
+ run.rb # Run record (status, progress, user tracking)
853
+ artifact.rb # Artifact record (jsonb, blob, text)
854
+ views/maintenance_on_steroids/
855
+ dashboard/index.html.erb # Dashboard
856
+ jobs/index.html.erb # Task list
857
+ jobs/show.html.erb # Task detail
858
+ jobs/source.html.erb # Source code viewer
859
+ runs/new.html.erb # New run form
860
+ runs/show.html.erb # Run detail with live progress
861
+ jobs/maintenance_on_steroids/
862
+ run_job.rb # ActiveJob::Continuable job
863
+ lib/
864
+ maintenance_on_steroids/
865
+ task.rb # Base task class
866
+ form_dsl.rb # Form input DSL
867
+ artifact_dsl.rb # Artifact DSL
868
+ about_dsl.rb # Task metadata DSL
869
+ job_dsl.rb # Queue configuration DSL
870
+ callbacks_dsl.rb # Lifecycle callbacks DSL
871
+ params_proxy.rb # Typed parameter access
872
+ artifacts_proxy.rb # Artifact read/write
873
+ jsonb_artifact.rb # Hash-like JSONB wrapper
874
+ job_registry.rb # Task class discovery
875
+ engine.rb # Rails engine setup
876
+ generators/maintenance_on_steroids/
877
+ install/ # Install generator
878
+ job/ # Task generator
879
+ ```
880
+
881
+ ## Full Example
882
+
883
+ A complete task using most features:
884
+
885
+ ```ruby
886
+ class MigrateUserProfiles < MaintenanceOnSteroids::Task
887
+ about do
888
+ title "Migrate User Profiles"
889
+ description "Migrates legacy profile data to the new format"
890
+ owner "Backend Team"
891
+ end
892
+
893
+ job do
894
+ queue "maintenance"
895
+ end
896
+
897
+ form do
898
+ input :batch_label, type: :string, default: "migration-v2", help_text: "Label for tracking"
899
+ input :dry_run, type: :boolean, default: true, help_text: "Preview changes without saving"
900
+ end
901
+
902
+ artifact :results, type: :jsonb, default: {}
903
+ artifact :error_log, type: :file, file_name: "migration_errors.csv"
904
+
905
+ after_start :log_start
906
+ after_complete :send_summary
907
+ after_error :alert_team
908
+
909
+ def collection
910
+ User.where(profile_version: 1)
911
+ end
912
+
913
+ def process(user)
914
+ new_data = ProfileMigrator.transform(user.profile_data)
915
+
916
+ if params[:dry_run]
917
+ artifacts[:results][user.id.to_s] = { status: "preview", changes: new_data }
918
+ else
919
+ user.update!(profile_data: new_data, profile_version: 2)
920
+ artifacts[:results][user.id.to_s] = { status: "migrated" }
921
+ end
922
+ artifacts[:results].save!
923
+ end
924
+
925
+ private
926
+
927
+ def log_start
928
+ Rails.logger.info "[MigrateUserProfiles] Started: #{params[:batch_label]}"
929
+ end
930
+
931
+ def send_summary
932
+ count = artifacts[:results].size
933
+ AdminMailer.migration_complete(count, params[:batch_label]).deliver_later
934
+ end
935
+
936
+ def alert_team
937
+ Slack.notify("#alerts", "Profile migration failed! Check run ##{run.id}")
938
+ end
939
+ end
940
+ ```
941
+
942
+ ## Alternatives
943
+
944
+ https://github.com/Shopify/maintenance_tasks - a gem from Shopify. This gem actually inspired me to build my version, because the gem from Shopify is missing many features I need.
945
+
946
+ ## License
947
+
948
+ MIT License. See [LICENSE.txt](LICENSE.txt).