mission_control-jobs 0.2.0 → 0.2.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +10 -4
- data/app/controllers/mission_control/jobs/application_controller.rb +6 -0
- data/app/controllers/mission_control/jobs/queues_controller.rb +1 -9
- data/app/views/layouts/mission_control/jobs/application.html.erb +1 -0
- data/app/views/mission_control/jobs/shared/_job.html.erb +1 -0
- data/lib/active_job/queue_adapters/resque_ext.rb +5 -1
- data/lib/mission_control/jobs/console/connect_to.rb +15 -0
- data/lib/mission_control/jobs/console/jobs_help.rb +23 -0
- data/lib/mission_control/jobs/engine.rb +4 -2
- data/lib/mission_control/jobs/version.rb +1 -1
- data/lib/mission_control/jobs.rb +1 -0
- metadata +71 -14
- data/lib/mission_control/jobs/console/helpers.rb +0 -26
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: '0516069c5660225d8cc418be4c2898ae247004ff150e9bf491b74fa7b1111a79'
|
4
|
+
data.tar.gz: 16b68f33818169e31b3701d84e3e53d8b5069e07c4b5e84303c0917212bc45fc
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9290598911e316cb5e70f6cb23757704b1966c291d4e928fc3f94f0c50b03191d00fb8a8170ee0b154a36235c43b16b686b05b0629467d3575807e28ef57331c
|
7
|
+
data.tar.gz: 2136cc09bcaa59e3a8dbfd6d19984bcb2d82d1b44c8870cf7019330af08b0b26a112079013eaa5df86e705efc50161851b0cc6094a173e778e672961426ac0e6
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# Mission Control — Jobs
|
2
2
|
|
3
|
-
This gem provides a Rails-based frontend to Active Job adapters. It currently supports [Resque](https://github.com/resque/resque/) and [Solid Queue](https://github.com/
|
3
|
+
This gem provides a Rails-based frontend to Active Job adapters. It currently supports [Resque](https://github.com/resque/resque/) and [Solid Queue](https://github.com/rails/solid_queue). Its features depend on those offered by the adapter itself. At a minimum, it allows you to inspect job queues and jobs currently waiting in those queues and inspect and retry or discard failed jobs.
|
4
4
|
|
5
5
|
## Installation
|
6
6
|
Add this line to your application's Gemfile:
|
@@ -56,10 +56,16 @@ Besides `base_controller_class`, you can also set the following for `MissionCont
|
|
56
56
|
- `adapters`: a list of adapters that you want Mission Control to use and extend. By default this will be the adapter you have set for `active_job.queue_adapter`.
|
57
57
|
- `internal_query_count_limit`: in count queries, the maximum number of records that will be counted if the adapter needs to limit these queries. True counts above this number will be returned as `INFINITY`. This keeps count queries fast—defaults to `500,000`
|
58
58
|
- `scheduled_job_delay_threshold`: the time duration before a scheduled job is considered delayed. Defaults to `1.minute` (a job is considered delayed if it hasn't transitioned from the `scheduled` status 1 minute after the scheduled time).
|
59
|
+
- `show_console_help`: whether to show the console help. If you don't want the console help message, set this to `false`—defaults to `true`.
|
59
60
|
|
60
61
|
This library extends Active Job with a querying interface and the following setting:
|
61
62
|
- `config.active_job.default_page_size`: the internal batch size that Active Job will use when sending queries to the underlying adapter and the batch size for the bulk operations defined above—defaults to `1000`.
|
62
63
|
|
64
|
+
|
65
|
+
## Adapter Specifics
|
66
|
+
|
67
|
+
- **Resque**: Queue pausing is supported only if you have `resque-pause` installed in your project
|
68
|
+
|
63
69
|
## Advanced configuration
|
64
70
|
|
65
71
|
When we built Mission Control Jobs, we did it with the idea of managing multiple apps' job backends from a single, centralized app that we used for monitoring, alerts and other tools that related to all our apps. Some of our apps run in more than one datacenter, and we run different Resque instances with different Redis configurations in each. Because of this, we added support for multiple apps and multiple adapters per app. Even when running Mission Control Job within the app it manages, and a single DC, as we migrated from Resque to Solid Queue, we needed to manage both adapters from Mission Control.
|
@@ -80,7 +86,7 @@ require "resque_pause_helper"
|
|
80
86
|
|
81
87
|
require "solid_queue"
|
82
88
|
|
83
|
-
Resque.redis = Redis::Namespace.new "#{Rails.env}", redis: Redis.new(host: "localhost", port: 6379
|
89
|
+
Resque.redis = Redis::Namespace.new "#{Rails.env}", redis: Redis.new(host: "localhost", port: 6379)
|
84
90
|
|
85
91
|
SERVERS_BY_APP = {
|
86
92
|
BC4: %w[ resque_ashburn resque_chicago ],
|
@@ -109,7 +115,7 @@ end
|
|
109
115
|
|
110
116
|
This is an example for two different apps, BC4 and HEY, each one with two servers. BC4 has two Resque servers with two different configurations, and HEY has one Resque server and one Solid Queue server.
|
111
117
|
|
112
|
-
Currently, only one Solid Queue configuration is supported, but support for several Solid Queue backends (with different databases) [is planned](https://github.com/
|
118
|
+
Currently, only one Solid Queue configuration is supported, but support for several Solid Queue backends (with different databases) [is planned](https://github.com/rails/mission_control-jobs/issues/35).
|
113
119
|
|
114
120
|
This is how we set Resque and Solid Queue together when we migrated from one to the other:
|
115
121
|
|
@@ -128,7 +134,7 @@ When you have multiple apps and servers configured, you can choose between them
|
|
128
134
|
|
129
135
|
## Basic UI usage
|
130
136
|
|
131
|
-
As mentioned, the features available in Mission Control depend on the adapter you're using, as each adapter supports different features. Besides inspecting the queues and the jobs in them, and discarding and retrying failed jobs, you can inspect jobs in different statuses supported by each adapter, filter them by _queue name_ and _job class name_ (with the idea of [adding more filters in the future](https://github.com/
|
137
|
+
As mentioned, the features available in Mission Control depend on the adapter you're using, as each adapter supports different features. Besides inspecting the queues and the jobs in them, and discarding and retrying failed jobs, you can inspect jobs in different statuses supported by each adapter, filter them by _queue name_ and _job class name_ (with the idea of [adding more filters in the future](https://github.com/rails/mission_control-jobs/issues/30)), pause and un-pause queues (if the adapter allows that), inspect workers, know which jobs are being run by what worker, checking a specific job or a specific worker...
|
132
138
|
|
133
139
|
![Default queue tab](docs/images/default-queue.png)
|
134
140
|
|
@@ -4,8 +4,14 @@ class MissionControl::Jobs::ApplicationController < MissionControl::Jobs.base_co
|
|
4
4
|
include MissionControl::Jobs::ApplicationScoped, MissionControl::Jobs::NotFoundRedirections
|
5
5
|
include MissionControl::Jobs::AdapterFeatures
|
6
6
|
|
7
|
+
around_action :set_current_locale
|
8
|
+
|
7
9
|
private
|
8
10
|
def default_url_options
|
9
11
|
{ server_id: MissionControl::Jobs::Current.server }
|
10
12
|
end
|
13
|
+
|
14
|
+
def set_current_locale(&block)
|
15
|
+
I18n.with_locale(:en, &block)
|
16
|
+
end
|
11
17
|
end
|
@@ -2,7 +2,7 @@ class MissionControl::Jobs::QueuesController < MissionControl::Jobs::Application
|
|
2
2
|
before_action :set_queue, only: :show
|
3
3
|
|
4
4
|
def index
|
5
|
-
@queues =
|
5
|
+
@queues = ActiveJob.queues.sort_by(&:name)
|
6
6
|
end
|
7
7
|
|
8
8
|
def show
|
@@ -13,12 +13,4 @@ class MissionControl::Jobs::QueuesController < MissionControl::Jobs::Application
|
|
13
13
|
def set_queue
|
14
14
|
@queue = ActiveJob.queues[params[:id]]
|
15
15
|
end
|
16
|
-
|
17
|
-
def filtered_queues
|
18
|
-
if prefix = ActiveJob::Base.queue_name_prefix
|
19
|
-
ActiveJob.queues.select { |queue| queue.name.start_with?(prefix) }
|
20
|
-
else
|
21
|
-
ActiveJob.queues
|
22
|
-
end
|
23
|
-
end
|
24
16
|
end
|
@@ -6,6 +6,7 @@
|
|
6
6
|
<%= csp_meta_tag %>
|
7
7
|
|
8
8
|
<meta name="viewport" content="width=device-width,initial-scale=1">
|
9
|
+
<meta name="turbo-cache-control" content="no-cache">
|
9
10
|
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bulma@0.9.4/css/bulma.min.css">
|
10
11
|
<%= stylesheet_link_tag "mission_control/jobs/application", "data-turbo-track": "reload" %>
|
11
12
|
<%= javascript_importmap_tags "application-mcj" %>
|
@@ -81,6 +81,10 @@ module ActiveJob::QueueAdapters::ResqueExt
|
|
81
81
|
resque_jobs_for(jobs_relation).find_job(job_id)
|
82
82
|
end
|
83
83
|
|
84
|
+
def supports_queue_pausing?
|
85
|
+
defined?(ResquePauseHelper)
|
86
|
+
end
|
87
|
+
|
84
88
|
private
|
85
89
|
attr_reader :redis
|
86
90
|
|
@@ -261,7 +265,7 @@ module ActiveJob::QueueAdapters::ResqueExt
|
|
261
265
|
resque_job = job.raw_data
|
262
266
|
resque_job["retried_at"] = Time.now.strftime("%Y/%m/%d %H:%M:%S")
|
263
267
|
|
264
|
-
redis.lset(queue_redis_key, job.position, resque_job)
|
268
|
+
redis.lset(queue_redis_key, job.position, Resque.encode(resque_job))
|
265
269
|
Resque::Job.create(resque_job["queue"], resque_job["payload"]["class"], *resque_job["payload"]["args"])
|
266
270
|
rescue Redis::CommandError => error
|
267
271
|
handle_resque_job_error(job, error)
|
@@ -0,0 +1,15 @@
|
|
1
|
+
require "irb/command"
|
2
|
+
|
3
|
+
module MissionControl::Jobs::Console
|
4
|
+
class ConnectTo < IRB::Command::Base
|
5
|
+
category "Mission control jobs"
|
6
|
+
description "Connect to a job server"
|
7
|
+
|
8
|
+
def execute(server_locator)
|
9
|
+
server = MissionControl::Jobs::Server.from_global_id(server_locator)
|
10
|
+
MissionControl::Jobs::Current.server = server
|
11
|
+
|
12
|
+
puts "Connected to #{server_locator}"
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
@@ -0,0 +1,23 @@
|
|
1
|
+
require "irb/command"
|
2
|
+
|
3
|
+
module MissionControl::Jobs::Console
|
4
|
+
class JobsHelp < IRB::Command::Base
|
5
|
+
category "Mission control jobs"
|
6
|
+
description "Show help for managing jobs"
|
7
|
+
|
8
|
+
def execute(*)
|
9
|
+
puts "You are currently connected to #{MissionControl::Jobs::Current.server}" if MissionControl::Jobs::Current.server
|
10
|
+
|
11
|
+
puts "You can connect to a job server with"
|
12
|
+
puts %( connect_to <app_id>:<server_id>\n\n)
|
13
|
+
|
14
|
+
puts "Available job servers:\n"
|
15
|
+
|
16
|
+
MissionControl::Jobs.applications.each do |application|
|
17
|
+
application.servers.each do |server|
|
18
|
+
puts " * #{server.to_global_id}"
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
@@ -36,6 +36,7 @@ module MissionControl
|
|
36
36
|
|
37
37
|
config.before_initialize do
|
38
38
|
if MissionControl::Jobs.adapters.include?(:resque)
|
39
|
+
require "resque/thread_safe_redis"
|
39
40
|
ActiveJob::QueueAdapters::ResqueAdapter.prepend ActiveJob::QueueAdapters::ResqueExt
|
40
41
|
Resque.prepend Resque::ThreadSafeRedis
|
41
42
|
end
|
@@ -66,10 +67,11 @@ module MissionControl
|
|
66
67
|
|
67
68
|
console do
|
68
69
|
require "irb"
|
69
|
-
|
70
|
+
|
71
|
+
IRB::Command.register :connect_to, Console::ConnectTo
|
72
|
+
IRB::Command.register :jobs_help, Console::JobsHelp
|
70
73
|
|
71
74
|
IRB::Context.prepend(MissionControl::Jobs::Console::Context)
|
72
|
-
Rails::ConsoleMethods.include(MissionControl::Jobs::Console::Helpers)
|
73
75
|
|
74
76
|
MissionControl::Jobs.delay_between_bulk_operation_batches = 2
|
75
77
|
MissionControl::Jobs.logger = ActiveSupport::Logger.new(STDOUT)
|
data/lib/mission_control/jobs.rb
CHANGED
metadata
CHANGED
@@ -1,27 +1,27 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: mission_control-jobs
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
4
|
+
version: 0.2.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Jorge Manrubia
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2024-
|
11
|
+
date: 2024-08-06 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rails
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
|
-
- - "
|
17
|
+
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
19
|
version: '7.1'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
|
-
- - "
|
24
|
+
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: '7.1'
|
27
27
|
- !ruby/object:Gem::Dependency
|
@@ -66,6 +66,20 @@ dependencies:
|
|
66
66
|
- - ">="
|
67
67
|
- !ruby/object:Gem::Version
|
68
68
|
version: '0'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: irb
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - "~>"
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '1.13'
|
76
|
+
type: :runtime
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '1.13'
|
69
83
|
- !ruby/object:Gem::Dependency
|
70
84
|
name: resque
|
71
85
|
requirement: !ruby/object:Gem::Requirement
|
@@ -154,16 +168,16 @@ dependencies:
|
|
154
168
|
name: redis
|
155
169
|
requirement: !ruby/object:Gem::Requirement
|
156
170
|
requirements:
|
157
|
-
- - "
|
171
|
+
- - ">="
|
158
172
|
- !ruby/object:Gem::Version
|
159
|
-
version:
|
173
|
+
version: '0'
|
160
174
|
type: :development
|
161
175
|
prerelease: false
|
162
176
|
version_requirements: !ruby/object:Gem::Requirement
|
163
177
|
requirements:
|
164
|
-
- - "
|
178
|
+
- - ">="
|
165
179
|
- !ruby/object:Gem::Version
|
166
|
-
version:
|
180
|
+
version: '0'
|
167
181
|
- !ruby/object:Gem::Dependency
|
168
182
|
name: redis-namespace
|
169
183
|
requirement: !ruby/object:Gem::Requirement
|
@@ -207,7 +221,49 @@ dependencies:
|
|
207
221
|
- !ruby/object:Gem::Version
|
208
222
|
version: '0'
|
209
223
|
- !ruby/object:Gem::Dependency
|
210
|
-
name: rubocop-rails
|
224
|
+
name: rubocop-rails-omakase
|
225
|
+
requirement: !ruby/object:Gem::Requirement
|
226
|
+
requirements:
|
227
|
+
- - ">="
|
228
|
+
- !ruby/object:Gem::Version
|
229
|
+
version: '0'
|
230
|
+
type: :development
|
231
|
+
prerelease: false
|
232
|
+
version_requirements: !ruby/object:Gem::Requirement
|
233
|
+
requirements:
|
234
|
+
- - ">="
|
235
|
+
- !ruby/object:Gem::Version
|
236
|
+
version: '0'
|
237
|
+
- !ruby/object:Gem::Dependency
|
238
|
+
name: sprockets-rails
|
239
|
+
requirement: !ruby/object:Gem::Requirement
|
240
|
+
requirements:
|
241
|
+
- - ">="
|
242
|
+
- !ruby/object:Gem::Version
|
243
|
+
version: '0'
|
244
|
+
type: :development
|
245
|
+
prerelease: false
|
246
|
+
version_requirements: !ruby/object:Gem::Requirement
|
247
|
+
requirements:
|
248
|
+
- - ">="
|
249
|
+
- !ruby/object:Gem::Version
|
250
|
+
version: '0'
|
251
|
+
- !ruby/object:Gem::Dependency
|
252
|
+
name: sqlite3
|
253
|
+
requirement: !ruby/object:Gem::Requirement
|
254
|
+
requirements:
|
255
|
+
- - ">="
|
256
|
+
- !ruby/object:Gem::Version
|
257
|
+
version: '0'
|
258
|
+
type: :development
|
259
|
+
prerelease: false
|
260
|
+
version_requirements: !ruby/object:Gem::Requirement
|
261
|
+
requirements:
|
262
|
+
- - ">="
|
263
|
+
- !ruby/object:Gem::Version
|
264
|
+
version: '0'
|
265
|
+
- !ruby/object:Gem::Dependency
|
266
|
+
name: puma
|
211
267
|
requirement: !ruby/object:Gem::Requirement
|
212
268
|
requirements:
|
213
269
|
- - ">="
|
@@ -334,8 +390,9 @@ files:
|
|
334
390
|
- lib/mission_control/jobs/adapter.rb
|
335
391
|
- lib/mission_control/jobs/application.rb
|
336
392
|
- lib/mission_control/jobs/applications.rb
|
393
|
+
- lib/mission_control/jobs/console/connect_to.rb
|
337
394
|
- lib/mission_control/jobs/console/context.rb
|
338
|
-
- lib/mission_control/jobs/console/
|
395
|
+
- lib/mission_control/jobs/console/jobs_help.rb
|
339
396
|
- lib/mission_control/jobs/engine.rb
|
340
397
|
- lib/mission_control/jobs/errors/incompatible_adapter.rb
|
341
398
|
- lib/mission_control/jobs/errors/resource_not_found.rb
|
@@ -349,12 +406,12 @@ files:
|
|
349
406
|
- lib/mission_control/jobs/workers_relation.rb
|
350
407
|
- lib/resque/thread_safe_redis.rb
|
351
408
|
- lib/tasks/mission_control/jobs_tasks.rake
|
352
|
-
homepage: https://github.com/
|
409
|
+
homepage: https://github.com/rails/mission_control-jobs
|
353
410
|
licenses:
|
354
411
|
- MIT
|
355
412
|
metadata:
|
356
|
-
homepage_uri: https://github.com/
|
357
|
-
source_code_uri: https://github.com/
|
413
|
+
homepage_uri: https://github.com/rails/mission_control-jobs
|
414
|
+
source_code_uri: https://github.com/rails/mission_control-jobs
|
358
415
|
post_install_message:
|
359
416
|
rdoc_options: []
|
360
417
|
require_paths:
|
@@ -370,7 +427,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
370
427
|
- !ruby/object:Gem::Version
|
371
428
|
version: '0'
|
372
429
|
requirements: []
|
373
|
-
rubygems_version: 3.
|
430
|
+
rubygems_version: 3.5.9
|
374
431
|
signing_key:
|
375
432
|
specification_version: 4
|
376
433
|
summary: Operational controls for Active Job
|
@@ -1,26 +0,0 @@
|
|
1
|
-
module MissionControl::Jobs::Console::Helpers
|
2
|
-
def connect_to(server_locator)
|
3
|
-
server = MissionControl::Jobs::Server.from_global_id(server_locator)
|
4
|
-
MissionControl::Jobs::Current.server = server
|
5
|
-
|
6
|
-
puts "Connected to #{server_locator}"
|
7
|
-
nil
|
8
|
-
end
|
9
|
-
|
10
|
-
def jobs_help
|
11
|
-
puts "You are currently connected to #{MissionControl::Jobs::Current.server}" if MissionControl::Jobs::Current.server
|
12
|
-
|
13
|
-
puts "You can connect to a job server with"
|
14
|
-
puts %( connect_to "<app_id>:<server_id>"\n\n)
|
15
|
-
|
16
|
-
puts "Available job servers:\n"
|
17
|
-
|
18
|
-
MissionControl::Jobs.applications.each do |application|
|
19
|
-
application.servers.each do |server|
|
20
|
-
puts " * #{server.to_global_id}"
|
21
|
-
end
|
22
|
-
end
|
23
|
-
|
24
|
-
nil
|
25
|
-
end
|
26
|
-
end
|