solid_queue_web 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +43 -16
- data/app/assets/stylesheets/solid_queue_web/_07_forms.css +24 -0
- data/app/controllers/solid_queue_web/dashboard_controller.rb +1 -0
- data/app/controllers/solid_queue_web/failed_jobs/arguments_controller.rb +15 -0
- data/app/services/solid_queue_web/alert_webhook.rb +7 -2
- data/app/services/solid_queue_web/queue_depth_alert.rb +74 -0
- data/app/views/solid_queue_web/jobs/show.html.erb +13 -1
- data/config/routes.rb +1 -0
- data/lib/solid_queue_web/version.rb +1 -1
- data/lib/solid_queue_web.rb +5 -1
- metadata +9 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e28c1cc2c32722a5b876083166f98549f60c2788cb5e225d6d6fa1122a2964ae
|
|
4
|
+
data.tar.gz: b68d4c0cf42091242b1816956e217ae64f471142df5ce974f71772ae8b454d83
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: ba9a31d372ecf83a97b0ed708ce8209ea5b24e9e042d11f7ff61ce54925bef988d0a6e12a7337c9757051268210279bd46eb6e011b6008d602832d340448b0db
|
|
7
|
+
data.tar.gz: 0c41966b6dab2b47e4049b0afbf884d016d62fb78feb424934738900d4f676978c05c0129c7b02b31cc2eb47caf7770e090ad2921f8fec56e8854bc307b7f1df
|
data/README.md
CHANGED
|
@@ -8,6 +8,8 @@
|
|
|
8
8
|
|
|
9
9
|
A monitoring and management dashboard for [Solid Queue](https://github.com/rails/solid_queue), mountable as a Rails engine in any app.
|
|
10
10
|
|
|
11
|
+

|
|
12
|
+
|
|
11
13
|
## The problem
|
|
12
14
|
|
|
13
15
|
Solid Queue ships without a web interface. When jobs fail, queues back up, or workers go silent in production, the only options are `rails console` or raw SQL queries. SolidQueueWeb gives your team a real-time dashboard to inspect, retry, and discard jobs without leaving the browser — and without standing up any additional infrastructure.
|
|
@@ -38,7 +40,7 @@ SolidQueueWeb surfaces all of this in a browser UI available at any route you ch
|
|
|
38
40
|
- **Jobs** — filterable by status (ready, scheduled, claimed, blocked, failed), queue, and priority; search by job class name with dynamic auto-submit; time-based period filter (1 h / 24 h / 7 d); discard individual or all jobs; Turbo Frame navigation so only the table updates on filter or search; auto-refreshes every 10 seconds
|
|
39
41
|
- **Scheduled job management** — reschedule a scheduled job to run immediately ("Run Now") or push its `scheduled_at` forward by 1 h, 24 h, or 7 d; Turbo Stream responses update the row in place; "Run All Now" bulk action promotes every scheduled job in the current filtered view in a single operation
|
|
40
42
|
- **Failed jobs** — list of failed executions with error details; search by class name; filter by queue; time-based period filter; retry or discard individually or in bulk; bulk retry with configurable stagger (+5s / +10s / +30s / +1m) to avoid thundering herd on recovery
|
|
41
|
-
- **Job detail** — full arguments, timestamps, blocked-until date, and error backtrace; action buttons based on job status
|
|
43
|
+
- **Job detail** — full arguments, timestamps, blocked-until date, and error backtrace; action buttons based on job status; failed jobs show an editable arguments textarea so you can correct a bad payload and retry in one step without redeploying
|
|
42
44
|
- **Queue management** — pause and resume individual queues; queue-scoped job list with status filter, search, and discard
|
|
43
45
|
- **Recurring tasks** — all configured recurring tasks with cron schedule, next run time, last run time, and static/dynamic classification; "Run Now" button enqueues a task immediately without waiting for its next scheduled run
|
|
44
46
|
- **Processes** — workers, dispatchers, and supervisors with heartbeat health status; auto-refreshes every 10 seconds
|
|
@@ -53,10 +55,6 @@ SolidQueueWeb surfaces all of this in a browser UI available at any route you ch
|
|
|
53
55
|
- **Performance analytics** — per-job-class statistics at `/jobs/performance` showing run count, average, p50, p95, min, and max duration; sorted by p95 descending so the slowest classes surface first; period filter scopes to 1h / 24h / 7d or all time; each class name links to the filtered History view
|
|
54
56
|
- **Metrics / health endpoint** — `GET /jobs/metrics.json` returns a machine-readable JSON document with job counts, throughput, per-queue depth and pause state, and process health summary; suitable for Prometheus scraping, uptime monitors, or external dashboards; `slow_jobs` count included when `slow_job_threshold` is configured
|
|
55
57
|
|
|
56
|
-
## Screenshots
|
|
57
|
-
|
|
58
|
-

|
|
59
|
-
|
|
60
58
|
## Compatibility
|
|
61
59
|
|
|
62
60
|
| Dependency | Version |
|
|
@@ -102,9 +100,10 @@ SolidQueueWeb.configure do |config|
|
|
|
102
100
|
config.default_refresh_interval = 30_000 # jobs/processes/history auto-refresh in ms (default: 10_000)
|
|
103
101
|
config.search_results_limit = 10 # max results per status in global search (default: 25)
|
|
104
102
|
config.slow_job_threshold = 5.minutes # flag claimed jobs running longer than this (default: nil = disabled)
|
|
105
|
-
config.alert_webhook_url = "https://hooks.example.com/solid-queue" # POST target (default: nil = disabled)
|
|
103
|
+
config.alert_webhook_url = "https://hooks.example.com/solid-queue" # POST target — string or array (default: nil = disabled)
|
|
106
104
|
config.alert_failure_threshold = 10 # fire when failed count >= this (default: nil = disabled)
|
|
107
|
-
config.
|
|
105
|
+
config.alert_queue_thresholds = { "critical" => 50, "default" => 200 } # fire when queue depth >= threshold (default: {})
|
|
106
|
+
config.alert_webhook_cooldown = 1800 # seconds between repeated alerts per alert type (default: 3600)
|
|
108
107
|
config.connects_to = { reading: :reading, writing: :writing } # read replica (default: nil)
|
|
109
108
|
end
|
|
110
109
|
|
|
@@ -129,6 +128,17 @@ SolidQueueWeb.configure do |config|
|
|
|
129
128
|
end
|
|
130
129
|
```
|
|
131
130
|
|
|
131
|
+
To fan out to multiple endpoints (e.g. Slack and PagerDuty simultaneously), pass an array:
|
|
132
|
+
|
|
133
|
+
```ruby
|
|
134
|
+
config.alert_webhook_url = [
|
|
135
|
+
"https://hooks.slack.com/services/...",
|
|
136
|
+
"https://events.pagerduty.com/..."
|
|
137
|
+
]
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
All configured URLs receive the same payload. A failure posting to one URL is logged and skipped without blocking the remaining targets.
|
|
141
|
+
|
|
132
142
|
The request body is JSON:
|
|
133
143
|
|
|
134
144
|
```json
|
|
@@ -142,6 +152,31 @@ The request body is JSON:
|
|
|
142
152
|
|
|
143
153
|
The webhook fires asynchronously in a background thread so dashboard page loads are never delayed. HTTP errors are logged to `Rails.logger` and swallowed. The cooldown window prevents repeated alerts while the count stays elevated — the clock resets on each app restart.
|
|
144
154
|
|
|
155
|
+
## Queue depth alerts
|
|
156
|
+
|
|
157
|
+
Set `alert_queue_thresholds` to fire a webhook when any queue's ready job count meets or exceeds a per-queue limit:
|
|
158
|
+
|
|
159
|
+
```ruby
|
|
160
|
+
SolidQueueWeb.configure do |config|
|
|
161
|
+
config.alert_webhook_url = "https://hooks.example.com/solid-queue"
|
|
162
|
+
config.alert_queue_thresholds = { "critical" => 50, "default" => 200 }
|
|
163
|
+
end
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
The same `alert_webhook_url` endpoint(s) receive the payload, with a distinct event type so you can route it differently:
|
|
167
|
+
|
|
168
|
+
```json
|
|
169
|
+
{
|
|
170
|
+
"event": "queue_depth_threshold_exceeded",
|
|
171
|
+
"queue_name": "critical",
|
|
172
|
+
"depth": 63,
|
|
173
|
+
"threshold": 50,
|
|
174
|
+
"fired_at": "2026-05-21T12:34:56Z"
|
|
175
|
+
}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Cooldown is tracked independently per queue, so a persistently deep "critical" queue does not suppress alerts for "default". The shared `alert_webhook_cooldown` setting applies to each queue separately.
|
|
179
|
+
|
|
145
180
|
## Metrics endpoint
|
|
146
181
|
|
|
147
182
|
`GET /jobs/metrics.json` returns a machine-readable JSON document suitable for Prometheus scraping, uptime monitors, or external dashboards. No configuration is required — the endpoint is available as soon as the engine is mounted.
|
|
@@ -205,15 +240,7 @@ When `connects_to` is `nil` (the default), no connection switching occurs and si
|
|
|
205
240
|
|
|
206
241
|
## Roadmap
|
|
207
242
|
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
**Operations**
|
|
211
|
-
- Admin audit log — record who retried or discarded which jobs and when (requires host-app user identity)
|
|
212
|
-
- Failed job retry with modified arguments — edit the arguments JSON from the job detail page before retrying; useful for correcting bad payloads without redeploying
|
|
213
|
-
|
|
214
|
-
**Notifications**
|
|
215
|
-
- Multiple webhook targets — support an array of `alert_webhook_url` values so alerts can fan out to Slack, PagerDuty, and custom endpoints simultaneously
|
|
216
|
-
- Queue depth alert — fire a webhook when a queue's ready job count exceeds a configurable threshold (complements the existing failure-count alert)
|
|
243
|
+
See [ROADMAP.md](ROADMAP.md) for the full post-1.0 feature plan, organized by release milestone.
|
|
217
244
|
|
|
218
245
|
Pull requests for any of these are welcome. See [Contributing](#contributing) below.
|
|
219
246
|
|
|
@@ -117,4 +117,28 @@
|
|
|
117
117
|
background: var(--muted);
|
|
118
118
|
border-color: var(--muted);
|
|
119
119
|
color: #fff;
|
|
120
|
+
}
|
|
121
|
+
|
|
122
|
+
.sqd-textarea {
|
|
123
|
+
width: 100%;
|
|
124
|
+
padding: 0.5rem 0.75rem;
|
|
125
|
+
border: 1px solid var(--border);
|
|
126
|
+
border-radius: 5px;
|
|
127
|
+
font-size: 13px;
|
|
128
|
+
background: var(--surface);
|
|
129
|
+
color: var(--text);
|
|
130
|
+
line-height: 1.6;
|
|
131
|
+
resize: vertical;
|
|
132
|
+
box-sizing: border-box;
|
|
133
|
+
display: block;
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
.sqd-textarea:focus {
|
|
137
|
+
outline: 2px solid var(--primary);
|
|
138
|
+
outline-offset: -1px;
|
|
139
|
+
border-color: var(--primary);
|
|
140
|
+
}
|
|
141
|
+
|
|
142
|
+
.sqd-args-form__submit {
|
|
143
|
+
margin-top: 0.75rem;
|
|
120
144
|
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
module SolidQueueWeb
|
|
2
|
+
class FailedJobs::ArgumentsController < ApplicationController
|
|
3
|
+
def update
|
|
4
|
+
execution = SolidQueue::FailedExecution.find(params[:failed_job_id])
|
|
5
|
+
new_arguments = JSON.parse(params[:arguments])
|
|
6
|
+
execution.job.update!(arguments: new_arguments)
|
|
7
|
+
execution.retry
|
|
8
|
+
redirect_to failed_jobs_path, notice: "Job arguments updated and queued for retry."
|
|
9
|
+
rescue JSON::ParserError
|
|
10
|
+
redirect_to job_path(execution.job), alert: "Invalid JSON: could not parse arguments."
|
|
11
|
+
rescue => e
|
|
12
|
+
redirect_to failed_jobs_path, alert: "Could not update job: #{e.message}"
|
|
13
|
+
end
|
|
14
|
+
end
|
|
15
|
+
end
|
|
@@ -12,7 +12,8 @@ module SolidQueueWeb
|
|
|
12
12
|
return if failure_count < SolidQueueWeb.alert_failure_threshold
|
|
13
13
|
return unless should_fire?
|
|
14
14
|
|
|
15
|
-
|
|
15
|
+
urls = webhook_urls
|
|
16
|
+
Thread.new { urls.each { |url| post(url, failure_count) } }
|
|
16
17
|
end
|
|
17
18
|
|
|
18
19
|
def reset!
|
|
@@ -22,7 +23,11 @@ module SolidQueueWeb
|
|
|
22
23
|
private
|
|
23
24
|
|
|
24
25
|
def configured?
|
|
25
|
-
|
|
26
|
+
webhook_urls.any? && SolidQueueWeb.alert_failure_threshold.present?
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
def webhook_urls
|
|
30
|
+
Array(SolidQueueWeb.alert_webhook_url).flatten.compact.select(&:present?)
|
|
26
31
|
end
|
|
27
32
|
|
|
28
33
|
def should_fire?
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
require "net/http"
|
|
2
|
+
require "json"
|
|
3
|
+
require "uri"
|
|
4
|
+
|
|
5
|
+
module SolidQueueWeb
|
|
6
|
+
class QueueDepthAlert
|
|
7
|
+
MUTEX = Mutex.new
|
|
8
|
+
|
|
9
|
+
class << self
|
|
10
|
+
def call
|
|
11
|
+
return unless configured?
|
|
12
|
+
|
|
13
|
+
queue_depths = SolidQueue::ReadyExecution
|
|
14
|
+
.joins(:job)
|
|
15
|
+
.group("solid_queue_jobs.queue_name")
|
|
16
|
+
.count
|
|
17
|
+
|
|
18
|
+
queue_depths.each do |queue_name, depth|
|
|
19
|
+
threshold = SolidQueueWeb.alert_queue_thresholds[queue_name.to_s]
|
|
20
|
+
next unless threshold && depth >= threshold
|
|
21
|
+
next unless should_fire?(queue_name)
|
|
22
|
+
|
|
23
|
+
urls = webhook_urls
|
|
24
|
+
Thread.new { urls.each { |url| post(url, queue_name, depth, threshold) } }
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def reset!
|
|
29
|
+
MUTEX.synchronize { @last_fired_at = {} }
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
private
|
|
33
|
+
|
|
34
|
+
def configured?
|
|
35
|
+
SolidQueueWeb.alert_queue_thresholds.any? && webhook_urls.any?
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def webhook_urls
|
|
39
|
+
Array(SolidQueueWeb.alert_webhook_url).flatten.compact.select(&:present?)
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def should_fire?(queue_name)
|
|
43
|
+
MUTEX.synchronize do
|
|
44
|
+
@last_fired_at ||= {}
|
|
45
|
+
cooldown = SolidQueueWeb.alert_webhook_cooldown
|
|
46
|
+
return false if @last_fired_at[queue_name] && Time.current - @last_fired_at[queue_name] < cooldown
|
|
47
|
+
|
|
48
|
+
@last_fired_at[queue_name] = Time.current
|
|
49
|
+
true
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def post(url_string, queue_name, depth, threshold)
|
|
54
|
+
uri = URI.parse(url_string)
|
|
55
|
+
payload = JSON.generate(
|
|
56
|
+
event: "queue_depth_threshold_exceeded",
|
|
57
|
+
queue_name: queue_name,
|
|
58
|
+
depth: depth,
|
|
59
|
+
threshold: threshold,
|
|
60
|
+
fired_at: Time.current.iso8601
|
|
61
|
+
)
|
|
62
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
|
63
|
+
http.use_ssl = uri.scheme == "https"
|
|
64
|
+
http.open_timeout = 5
|
|
65
|
+
http.read_timeout = 10
|
|
66
|
+
request = Net::HTTP::Post.new(uri.path.presence || "/", "Content-Type" => "application/json")
|
|
67
|
+
request.body = payload
|
|
68
|
+
http.request(request)
|
|
69
|
+
rescue => e
|
|
70
|
+
Rails.logger.error("[SolidQueueWeb] Queue depth alert webhook failed: #{e.message}")
|
|
71
|
+
end
|
|
72
|
+
end
|
|
73
|
+
end
|
|
74
|
+
end
|
|
@@ -63,7 +63,19 @@
|
|
|
63
63
|
|
|
64
64
|
<div class="sqd-card sqd-detail-section">
|
|
65
65
|
<h2 class="sqd-section-title">Arguments</h2>
|
|
66
|
-
|
|
66
|
+
<% if @execution_status == "failed" && @job.failed_execution %>
|
|
67
|
+
<% args_json = begin; JSON.pretty_generate(@job.arguments); rescue; @job.arguments.inspect; end %>
|
|
68
|
+
<%= form_with url: failed_job_arguments_path(@job.failed_execution), method: :patch do |f| %>
|
|
69
|
+
<%= f.text_area :arguments,
|
|
70
|
+
value: args_json,
|
|
71
|
+
class: "sqd-textarea sqd-mono",
|
|
72
|
+
rows: [args_json.lines.count + 1, 6].max,
|
|
73
|
+
"aria-label": "Job arguments JSON" %>
|
|
74
|
+
<%= f.submit "Retry with these arguments", class: "sqd-btn sqd-btn--primary sqd-args-form__submit" %>
|
|
75
|
+
<% end %>
|
|
76
|
+
<% else %>
|
|
77
|
+
<pre class="sqd-pre"><%= JSON.pretty_generate(@job.arguments) rescue @job.arguments.inspect %></pre>
|
|
78
|
+
<% end %>
|
|
67
79
|
</div>
|
|
68
80
|
</div>
|
|
69
81
|
|
data/config/routes.rb
CHANGED
|
@@ -38,6 +38,7 @@ SolidQueueWeb::Engine.routes.draw do
|
|
|
38
38
|
resource :failed_job_selection, path: "failed_jobs/selection", only: [:create, :destroy],
|
|
39
39
|
controller: "failed_jobs/selections"
|
|
40
40
|
resources :failed_jobs, only: [:index, :destroy] do
|
|
41
|
+
resource :arguments, only: [:update], controller: "failed_jobs/arguments"
|
|
41
42
|
collection do
|
|
42
43
|
post :retry_all, to: "retry_failed_jobs#create"
|
|
43
44
|
post :discard_all, action: :destroy
|
data/lib/solid_queue_web.rb
CHANGED
|
@@ -6,7 +6,7 @@ module SolidQueueWeb
|
|
|
6
6
|
class << self
|
|
7
7
|
attr_writer :page_size, :dashboard_refresh_interval, :default_refresh_interval, :search_results_limit,
|
|
8
8
|
:slow_job_threshold, :alert_webhook_url, :alert_failure_threshold, :alert_webhook_cooldown,
|
|
9
|
-
:connects_to
|
|
9
|
+
:alert_queue_thresholds, :connects_to
|
|
10
10
|
|
|
11
11
|
def page_size
|
|
12
12
|
@page_size || 25
|
|
@@ -40,6 +40,10 @@ module SolidQueueWeb
|
|
|
40
40
|
@alert_webhook_cooldown || 3600
|
|
41
41
|
end
|
|
42
42
|
|
|
43
|
+
def alert_queue_thresholds
|
|
44
|
+
@alert_queue_thresholds || {}
|
|
45
|
+
end
|
|
46
|
+
|
|
43
47
|
def connects_to
|
|
44
48
|
@connects_to
|
|
45
49
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: solid_queue_web
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.1.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Chuck Smith
|
|
@@ -93,9 +93,12 @@ dependencies:
|
|
|
93
93
|
- - ">="
|
|
94
94
|
- !ruby/object:Gem::Version
|
|
95
95
|
version: '1.2'
|
|
96
|
-
description: Mount SolidQueueWeb in any Rails app using Solid Queue to get a
|
|
97
|
-
|
|
98
|
-
|
|
96
|
+
description: 'Mount SolidQueueWeb in any Rails app using Solid Queue to get a full-featured
|
|
97
|
+
job dashboard: inspect jobs by status (ready, scheduled, running, blocked, failed),
|
|
98
|
+
retry or discard failed jobs, reschedule or run scheduled jobs immediately, manage
|
|
99
|
+
recurring tasks, filter by queue/priority/period, export to CSV, detect slow jobs,
|
|
100
|
+
view queue depth sparklines, track job performance (p50/p95), and scrape a /metrics
|
|
101
|
+
JSON endpoint for external monitoring — all without leaving your app.'
|
|
99
102
|
email:
|
|
100
103
|
- eclectic-coding@users.noreply.github.com
|
|
101
104
|
executables: []
|
|
@@ -121,6 +124,7 @@ files:
|
|
|
121
124
|
- app/controllers/solid_queue_web/application_controller.rb
|
|
122
125
|
- app/controllers/solid_queue_web/blocked_jobs_controller.rb
|
|
123
126
|
- app/controllers/solid_queue_web/dashboard_controller.rb
|
|
127
|
+
- app/controllers/solid_queue_web/failed_jobs/arguments_controller.rb
|
|
124
128
|
- app/controllers/solid_queue_web/failed_jobs/selections_controller.rb
|
|
125
129
|
- app/controllers/solid_queue_web/failed_jobs_controller.rb
|
|
126
130
|
- app/controllers/solid_queue_web/history_controller.rb
|
|
@@ -150,6 +154,7 @@ files:
|
|
|
150
154
|
- app/services/solid_queue_web/dashboard_stats.rb
|
|
151
155
|
- app/services/solid_queue_web/job_performance_stats.rb
|
|
152
156
|
- app/services/solid_queue_web/metrics_payload.rb
|
|
157
|
+
- app/services/solid_queue_web/queue_depth_alert.rb
|
|
153
158
|
- app/services/solid_queue_web/queue_stats.rb
|
|
154
159
|
- app/views/layouts/solid_queue_web/application.html.erb
|
|
155
160
|
- app/views/solid_queue_web/dashboard/index.html.erb
|