good_job 3.20.0 → 3.21.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +71 -8
- data/README.md +6 -5
- data/app/controllers/good_job/jobs_controller.rb +1 -1
- data/app/models/good_job/batch_record.rb +2 -0
- data/app/models/good_job/execution.rb +8 -9
- data/app/models/good_job/process.rb +8 -2
- data/app/views/good_job/processes/index.html.erb +2 -0
- data/config/locales/de.yml +4 -0
- data/config/locales/en.yml +4 -0
- data/config/locales/es.yml +4 -0
- data/config/locales/fr.yml +4 -0
- data/config/locales/ja.yml +4 -0
- data/config/locales/nl.yml +4 -0
- data/config/locales/ru.yml +4 -0
- data/config/locales/tr.yml +4 -0
- data/config/locales/uk.yml +4 -0
- data/lib/good_job/adapter.rb +2 -2
- data/lib/good_job/capsule.rb +11 -11
- data/lib/good_job/cli.rb +8 -4
- data/lib/good_job/job_performer/metrics.rb +76 -0
- data/lib/good_job/job_performer.rb +31 -3
- data/lib/good_job/multi_scheduler.rb +38 -1
- data/lib/good_job/scheduler.rb +13 -47
- data/lib/good_job/version.rb +1 -1
- data/lib/good_job.rb +12 -1
- metadata +3 -3
- data/lib/good_job/metrics.rb +0 -57
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: dd97ba03dea3561d77db9ebb2f534d03aafed6e39b2327429c5e5eea3f971d29
|
4
|
+
data.tar.gz: b83947c459532713022c32d2845e8e139113cad8dc91012a4eeb45ed20e08f3b
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 349b6201b1add40178d726aaaf62887b0e89a36fed4f2546838d2251e7e593ce60e24ca4f08eac8c0067fd57c095f8ed1620bf2987805c5eac9fc7f327129b5d
|
7
|
+
data.tar.gz: b29eaf8018d61233b4245bbdf3fdc8204d8390995a583044166e25d824abd5db47f0d60c30b3ab9c1782387f2d9114d0eebf94d7aee9076bc819e719743ea659
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,68 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v3.21.2](https://github.com/bensheldon/good_job/tree/v3.21.2) (2023-11-24)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.21.1...v3.21.2)
|
6
|
+
|
7
|
+
**Fixed bugs:**
|
8
|
+
|
9
|
+
- Skip `RecordAlreadyAdvisoryLockedError` during mass-update action [\#1158](https://github.com/bensheldon/good_job/pull/1158) ([jmarsh24](https://github.com/jmarsh24))
|
10
|
+
|
11
|
+
**Closed issues:**
|
12
|
+
|
13
|
+
- Processes are created for puma workers after version v3.12.5 with execution\_mode==:external [\#1156](https://github.com/bensheldon/good_job/issues/1156)
|
14
|
+
|
15
|
+
**Merged pull requests:**
|
16
|
+
|
17
|
+
- Update README.md [\#1152](https://github.com/bensheldon/good_job/pull/1152) ([LucasKendi](https://github.com/LucasKendi))
|
18
|
+
|
19
|
+
## [v3.21.1](https://github.com/bensheldon/good_job/tree/v3.21.1) (2023-11-14)
|
20
|
+
|
21
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.21.0...v3.21.1)
|
22
|
+
|
23
|
+
**Fixed bugs:**
|
24
|
+
|
25
|
+
- Explicitly require `active_job/arguments` in `GoodJob::BatchRecord` [\#1150](https://github.com/bensheldon/good_job/pull/1150) ([hidenba](https://github.com/hidenba))
|
26
|
+
- Bug: Polling only activates single thread, should eagerly create additional threads when jobs exist [\#1148](https://github.com/bensheldon/good_job/pull/1148) ([bensheldon](https://github.com/bensheldon))
|
27
|
+
|
28
|
+
**Closed issues:**
|
29
|
+
|
30
|
+
- Error when executing ActiveJob::Batch.new in Rails 7.1.x [\#1149](https://github.com/bensheldon/good_job/issues/1149)
|
31
|
+
- Show whether or not cron scheduler is enable in dashboard \(UI\) [\#1117](https://github.com/bensheldon/good_job/issues/1117)
|
32
|
+
- ActiveRecord::ConnectionNotEstablished For rails multi DB [\#1103](https://github.com/bensheldon/good_job/issues/1103)
|
33
|
+
- Rails API dies when using latest good\_job version [\#952](https://github.com/bensheldon/good_job/issues/952)
|
34
|
+
- config.good\_job.preserve\_job\_records = false not working with CRON [\#927](https://github.com/bensheldon/good_job/issues/927)
|
35
|
+
- Pundit::NotDefinedError \(unable to find policy `Admin::GoodJob::JobPolicy` for \) - version 2.13.0 and above [\#618](https://github.com/bensheldon/good_job/issues/618)
|
36
|
+
- Running CLI under foreman doesn't display log output until exit [\#490](https://github.com/bensheldon/good_job/issues/490)
|
37
|
+
|
38
|
+
**Merged pull requests:**
|
39
|
+
|
40
|
+
- Always instantiate MultiScheduler; delegate Scheduler Metrics to JobPerformer [\#1147](https://github.com/bensheldon/good_job/pull/1147) ([bensheldon](https://github.com/bensheldon))
|
41
|
+
- Clarify on concurrency uniqueness constraints [\#1144](https://github.com/bensheldon/good_job/pull/1144) ([Earlopain](https://github.com/Earlopain))
|
42
|
+
|
43
|
+
## [v3.21.0](https://github.com/bensheldon/good_job/tree/v3.21.0) (2023-11-06)
|
44
|
+
|
45
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.20.0...v3.21.0)
|
46
|
+
|
47
|
+
**Implemented enhancements:**
|
48
|
+
|
49
|
+
- Add "cron enabled" column to processes index page [\#1127](https://github.com/bensheldon/good_job/pull/1127) ([bforma](https://github.com/bforma))
|
50
|
+
- Add `limit:` kwarg to `GoodJob.perform_inline` [\#1126](https://github.com/bensheldon/good_job/pull/1126) ([bensheldon](https://github.com/bensheldon))
|
51
|
+
|
52
|
+
**Closed issues:**
|
53
|
+
|
54
|
+
- Cron scheduler and multiple processes [\#1128](https://github.com/bensheldon/good_job/issues/1128)
|
55
|
+
- `GoodJob.on_thread_error` not called in tests [\#1102](https://github.com/bensheldon/good_job/issues/1102)
|
56
|
+
|
57
|
+
**Merged pull requests:**
|
58
|
+
|
59
|
+
- Use a Concurrent::Event for CLI signal-trapping loop [\#1141](https://github.com/bensheldon/good_job/pull/1141) ([bensheldon](https://github.com/bensheldon))
|
60
|
+
- Update README's optimize queue explanation [\#1138](https://github.com/bensheldon/good_job/pull/1138) ([maestromac](https://github.com/maestromac))
|
61
|
+
- Update development dependencies and light Rubocop'ing [\#1136](https://github.com/bensheldon/good_job/pull/1136) ([bensheldon](https://github.com/bensheldon))
|
62
|
+
- Move the Rails app harness from `spec/test_app` to `demo` [\#1135](https://github.com/bensheldon/good_job/pull/1135) ([bensheldon](https://github.com/bensheldon))
|
63
|
+
- In test, shutdown schedulers/capsules before doing assertions because of race conditions; store CI logs for Dev Env tests [\#1129](https://github.com/bensheldon/good_job/pull/1129) ([bensheldon](https://github.com/bensheldon))
|
64
|
+
- Use a constant to represent `None` for default/blank memoizable values [\#1125](https://github.com/bensheldon/good_job/pull/1125) ([bensheldon](https://github.com/bensheldon))
|
65
|
+
|
3
66
|
## [v3.20.0](https://github.com/bensheldon/good_job/tree/v3.20.0) (2023-10-23)
|
4
67
|
|
5
68
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v3.19.4...v3.20.0)
|
@@ -1962,7 +2025,7 @@
|
|
1962
2025
|
- Warn in Readme that configuration should not go into `config/initializers/*.rb` [\#418](https://github.com/bensheldon/good_job/pull/418) ([bensheldon](https://github.com/bensheldon))
|
1963
2026
|
- Replace worker wording [\#409](https://github.com/bensheldon/good_job/pull/409) ([Hugo-Hache](https://github.com/Hugo-Hache))
|
1964
2027
|
- Improve Readme's "Optimize queues, threads, processes" section [\#405](https://github.com/bensheldon/good_job/pull/405) ([Hugo-Hache](https://github.com/Hugo-Hache))
|
1965
|
-
- Update GH Test Matrix with more PG versions [\#401](https://github.com/bensheldon/good_job/pull/401) ([
|
2028
|
+
- Update GH Test Matrix with more PG versions [\#401](https://github.com/bensheldon/good_job/pull/401) ([tedt10e](https://github.com/tedt10e))
|
1966
2029
|
- Extract cron configuration hash into CronEntry ActiveModel objects [\#400](https://github.com/bensheldon/good_job/pull/400) ([bensheldon](https://github.com/bensheldon))
|
1967
2030
|
- Remove errant copy-paste from app.json [\#397](https://github.com/bensheldon/good_job/pull/397) ([morgoth](https://github.com/morgoth))
|
1968
2031
|
|
@@ -2002,7 +2065,7 @@
|
|
2002
2065
|
|
2003
2066
|
**Merged pull requests:**
|
2004
2067
|
|
2005
|
-
- Update GH Test Matrix with latest JRuby 9.3.0.0 [\#387](https://github.com/bensheldon/good_job/pull/387) ([
|
2068
|
+
- Update GH Test Matrix with latest JRuby 9.3.0.0 [\#387](https://github.com/bensheldon/good_job/pull/387) ([tedt10e](https://github.com/tedt10e))
|
2006
2069
|
- Improve test support's ShellOut command's process termination and add test logs [\#385](https://github.com/bensheldon/good_job/pull/385) ([bensheldon](https://github.com/bensheldon))
|
2007
2070
|
- @bensheldon Add Rails 7 alpha to Appraisal; update development dependencies [\#384](https://github.com/bensheldon/good_job/pull/384) ([bensheldon](https://github.com/bensheldon))
|
2008
2071
|
|
@@ -2261,7 +2324,7 @@
|
|
2261
2324
|
|
2262
2325
|
- Have prettier Dashboard asset urls e.g. `bootstrap.css` instead of `bootstrap_css.css` [\#306](https://github.com/bensheldon/good_job/pull/306) ([bensheldon](https://github.com/bensheldon))
|
2263
2326
|
- Create dashboard demo app on Heroku [\#305](https://github.com/bensheldon/good_job/pull/305) ([bensheldon](https://github.com/bensheldon))
|
2264
|
-
- Add Frozen String Literal to all files [\#302](https://github.com/bensheldon/good_job/pull/302) ([
|
2327
|
+
- Add Frozen String Literal to all files [\#302](https://github.com/bensheldon/good_job/pull/302) ([tedt10e](https://github.com/tedt10e))
|
2265
2328
|
|
2266
2329
|
## [v1.11.2](https://github.com/bensheldon/good_job/tree/v1.11.2) (2021-07-20)
|
2267
2330
|
|
@@ -2319,7 +2382,7 @@
|
|
2319
2382
|
|
2320
2383
|
**Merged pull requests:**
|
2321
2384
|
|
2322
|
-
- Update GH Test Matrix with latest JRuby 9.2.19.0 [\#283](https://github.com/bensheldon/good_job/pull/283) ([
|
2385
|
+
- Update GH Test Matrix with latest JRuby 9.2.19.0 [\#283](https://github.com/bensheldon/good_job/pull/283) ([tedt10e](https://github.com/tedt10e))
|
2323
2386
|
|
2324
2387
|
## [v1.10.0](https://github.com/bensheldon/good_job/tree/v1.10.0) (2021-06-29)
|
2325
2388
|
|
@@ -2469,11 +2532,11 @@
|
|
2469
2532
|
- Move executable flags from constants to accessors on GoodJob::CLI [\#234](https://github.com/bensheldon/good_job/pull/234) ([bensheldon](https://github.com/bensheldon))
|
2470
2533
|
- Add custom Scheduler::TimerSet [\#232](https://github.com/bensheldon/good_job/pull/232) ([bensheldon](https://github.com/bensheldon))
|
2471
2534
|
- Fix assorted constant references in YARD documentation [\#231](https://github.com/bensheldon/good_job/pull/231) ([bensheldon](https://github.com/bensheldon))
|
2472
|
-
- Update GH Test Matrix with latest JRuby 9.2.17.0 [\#228](https://github.com/bensheldon/good_job/pull/228) ([
|
2535
|
+
- Update GH Test Matrix with latest JRuby 9.2.17.0 [\#228](https://github.com/bensheldon/good_job/pull/228) ([tedt10e](https://github.com/tedt10e))
|
2473
2536
|
- Update gem dependencies [\#227](https://github.com/bensheldon/good_job/pull/227) ([bensheldon](https://github.com/bensheldon))
|
2474
2537
|
- Remove leftover text from Readme [\#226](https://github.com/bensheldon/good_job/pull/226) ([weh](https://github.com/weh))
|
2475
2538
|
- Fix appraisal and bundler version CI conflicts [\#224](https://github.com/bensheldon/good_job/pull/224) ([bensheldon](https://github.com/bensheldon))
|
2476
|
-
- Update GH Test Matrix with latest JRuby [\#223](https://github.com/bensheldon/good_job/pull/223) ([
|
2539
|
+
- Update GH Test Matrix with latest JRuby [\#223](https://github.com/bensheldon/good_job/pull/223) ([tedt10e](https://github.com/tedt10e))
|
2477
2540
|
|
2478
2541
|
## [v1.8.0](https://github.com/bensheldon/good_job/tree/v1.8.0) (2021-03-04)
|
2479
2542
|
|
@@ -2556,8 +2619,8 @@
|
|
2556
2619
|
**Merged pull requests:**
|
2557
2620
|
|
2558
2621
|
- Update bundler version to 2.2.5 [\#200](https://github.com/bensheldon/good_job/pull/200) ([bensheldon](https://github.com/bensheldon))
|
2559
|
-
- Update GH Test Matrix with minimum & latest JRuby version [\#197](https://github.com/bensheldon/good_job/pull/197) ([
|
2560
|
-
- Fix JRuby version number [\#193](https://github.com/bensheldon/good_job/pull/193) ([
|
2622
|
+
- Update GH Test Matrix with minimum & latest JRuby version [\#197](https://github.com/bensheldon/good_job/pull/197) ([tedt10e](https://github.com/tedt10e))
|
2623
|
+
- Fix JRuby version number [\#193](https://github.com/bensheldon/good_job/pull/193) ([tedt10e](https://github.com/tedt10e))
|
2561
2624
|
|
2562
2625
|
## [v1.4.1](https://github.com/bensheldon/good_job/tree/v1.4.1) (2021-01-09)
|
2563
2626
|
|
data/README.md
CHANGED
@@ -470,7 +470,7 @@ class MyJob < ApplicationJob
|
|
470
470
|
# Can be String or Lambda/Proc that is invoked in the context of the job.
|
471
471
|
# Note: Arguments passed to #perform_later can be accessed through ActiveJob's `arguments` method
|
472
472
|
# which is an array containing positional arguments and, optionally, a kwarg hash.
|
473
|
-
key: -> { "
|
473
|
+
key: -> { "MyJob-#{arguments.first}-#{arguments.last[:version]}" } # MyJob.perform_later("Alice", version: 'v2') => "MyJob-Alice-v2"
|
474
474
|
)
|
475
475
|
|
476
476
|
def perform(first_name)
|
@@ -483,7 +483,7 @@ When testing, the resulting concurrency key value can be inspected:
|
|
483
483
|
|
484
484
|
```ruby
|
485
485
|
job = MyJob.perform_later("Alice")
|
486
|
-
job.good_job_concurrency_key #=> "
|
486
|
+
job.good_job_concurrency_key #=> "MyJob-Alice"
|
487
487
|
```
|
488
488
|
|
489
489
|
#### How concurrency controls work
|
@@ -585,9 +585,10 @@ Batches track a set of jobs, and enqueue an optional callback job when all of th
|
|
585
585
|
|
586
586
|
```ruby
|
587
587
|
batch = GoodJob::Batch.new
|
588
|
-
batch
|
588
|
+
batch.add do
|
589
589
|
10.times { MyJob.perform_later }
|
590
590
|
end
|
591
|
+
|
591
592
|
batch.add do
|
592
593
|
10.times { OtherJob.perform_later }
|
593
594
|
end
|
@@ -939,7 +940,7 @@ By default, GoodJob creates a single thread execution pool that will execute job
|
|
939
940
|
|
940
941
|
- `transactional_messages:2`: execute jobs enqueued on `transactional_messages`, with up to 2 threads.
|
941
942
|
- `batch_processing:1` execute jobs enqueued on `batch_processing`, with a single thread.
|
942
|
-
- `-transactional_messages,batch_processing`: execute jobs enqueued on _any_ queue _excluding_ `transactional_messages` or `batch_processing`, with up to 2 threads.
|
943
|
+
- `-transactional_messages,batch_processing:2`: execute jobs enqueued on _any_ queue _excluding_ `transactional_messages` or `batch_processing`, with up to 2 threads.
|
943
944
|
- `*`: execute jobs on any queue, with up to 5 threads (as configured by `--max-threads=5`).
|
944
945
|
|
945
946
|
When a pool is performing jobs from multiple queues, jobs will be performed from specified queues, ordered by priority and creation time. To perform jobs from queues in the queues' given order, use the `+` modifier. In this example, jobs in `batch_processing` will be performed only when there are no jobs in `transactional_messages`:
|
@@ -1370,7 +1371,7 @@ bin/setup
|
|
1370
1371
|
|
1371
1372
|
#### Rails development harness
|
1372
1373
|
|
1373
|
-
A Rails application exists within `
|
1374
|
+
A Rails application exists within `demo` that is used for development, test, and GoodJob Demo environments.
|
1374
1375
|
|
1375
1376
|
```bash
|
1376
1377
|
# Run a local development webserver
|
@@ -248,6 +248,7 @@ module GoodJob
|
|
248
248
|
|
249
249
|
# Finds the next eligible Execution, acquire an advisory lock related to it, and
|
250
250
|
# executes the job.
|
251
|
+
# @yield [Execution, nil] The next eligible Execution, or +nil+ if none found, before it is performed.
|
251
252
|
# @return [ExecutionResult, nil]
|
252
253
|
# If a job was executed, returns an array with the {Execution} record, the
|
253
254
|
# return value for the job's +#perform+ method, and the exception the job
|
@@ -256,21 +257,19 @@ module GoodJob
|
|
256
257
|
def self.perform_with_advisory_lock(parsed_queues: nil, queue_select_limit: nil)
|
257
258
|
execution = nil
|
258
259
|
result = nil
|
260
|
+
|
259
261
|
unfinished.dequeueing_ordered(parsed_queues).only_scheduled.limit(1).with_advisory_lock(select_limit: queue_select_limit) do |executions|
|
260
262
|
execution = executions.first
|
261
|
-
|
262
|
-
|
263
|
-
|
264
|
-
|
263
|
+
if execution&.executable?
|
264
|
+
yield(execution) if block_given?
|
265
|
+
result = execution.perform
|
266
|
+
else
|
265
267
|
execution = nil
|
266
|
-
|
268
|
+
yield(nil) if block_given?
|
267
269
|
end
|
268
|
-
|
269
|
-
yield(execution) if block_given?
|
270
|
-
result = execution.perform
|
271
270
|
end
|
272
|
-
execution&.run_callbacks(:perform_unlocked)
|
273
271
|
|
272
|
+
execution&.run_callbacks(:perform_unlocked)
|
274
273
|
result
|
275
274
|
end
|
276
275
|
|
@@ -52,6 +52,10 @@ module GoodJob # :nodoc:
|
|
52
52
|
end
|
53
53
|
|
54
54
|
def self.ns_current_state
|
55
|
+
total_succeeded_executions_count = GoodJob::Scheduler.instances.sum { |scheduler| scheduler.stats.fetch(:succeeded_executions_count, 0) }
|
56
|
+
total_errored_executions_count = GoodJob::Scheduler.instances.sum { |scheduler| scheduler.stats.fetch(:errored_executions_count, 0) }
|
57
|
+
total_empty_executions_count = GoodJob::Scheduler.instances.sum { |scheduler| scheduler.stats.fetch(:empty_executions_count, 0) }
|
58
|
+
|
55
59
|
{
|
56
60
|
id: ns_current_id,
|
57
61
|
hostname: Socket.gethostname,
|
@@ -61,8 +65,10 @@ module GoodJob # :nodoc:
|
|
61
65
|
retry_on_unhandled_error: GoodJob.retry_on_unhandled_error,
|
62
66
|
schedulers: GoodJob::Scheduler.instances.map(&:stats),
|
63
67
|
cron_enabled: GoodJob.configuration.enable_cron?,
|
64
|
-
total_succeeded_executions_count:
|
65
|
-
total_errored_executions_count:
|
68
|
+
total_succeeded_executions_count: total_succeeded_executions_count,
|
69
|
+
total_errored_executions_count: total_errored_executions_count,
|
70
|
+
total_executions_count: total_succeeded_executions_count + total_errored_executions_count,
|
71
|
+
total_empty_executions_count: total_empty_executions_count,
|
66
72
|
database_connection_pool: {
|
67
73
|
size: connection_pool.size,
|
68
74
|
active: connection_pool.connections.count(&:in_use?),
|
@@ -8,6 +8,7 @@
|
|
8
8
|
<div class="row small text-muted text-uppercase align-items-center">
|
9
9
|
<div class="col"><%= t ".process" %></div>
|
10
10
|
<div class="col"><%= t ".schedulers" %></div>
|
11
|
+
<div class="col-2 d-flex gap-2"><%= t ".cron_enabled" %></div>
|
11
12
|
<div class="col-2 d-flex gap-2">
|
12
13
|
<span><%= t ".started" %></span>
|
13
14
|
</div>
|
@@ -44,6 +45,7 @@
|
|
44
45
|
<pre class="mb-0"><%= scheduler.is_a?(Hash) ? scheduler['name'] : scheduler %></pre>
|
45
46
|
<% end %>
|
46
47
|
</div>
|
48
|
+
<div class="col-2 small"><%= t(ActiveModel::Type::Boolean.new.cast(process.state["cron_enabled"]), scope: "good_job.shared.boolean") %></div>
|
47
49
|
<div class="col-2 small"><%= relative_time(process.created_at) %></div>
|
48
50
|
<div class="col-2 small"><%= relative_time(process.updated_at) %></div>
|
49
51
|
<div class="col-auto">
|
data/config/locales/de.yml
CHANGED
@@ -198,6 +198,7 @@ de:
|
|
198
198
|
unit: ''
|
199
199
|
processes:
|
200
200
|
index:
|
201
|
+
cron_enabled: Cron aktiviert
|
201
202
|
no_good_job_processes_found: Keine GoodJob-Prozesse gefunden.
|
202
203
|
process: Verfahren
|
203
204
|
schedulers: Planer
|
@@ -205,6 +206,9 @@ de:
|
|
205
206
|
title: Prozesse
|
206
207
|
updated: Aktualisiert
|
207
208
|
shared:
|
209
|
+
boolean:
|
210
|
+
'false': Nein
|
211
|
+
'true': Ja
|
208
212
|
error: Fehler
|
209
213
|
filter:
|
210
214
|
all: Alle
|
data/config/locales/en.yml
CHANGED
@@ -198,6 +198,7 @@ en:
|
|
198
198
|
unit: ''
|
199
199
|
processes:
|
200
200
|
index:
|
201
|
+
cron_enabled: Cron enabled
|
201
202
|
no_good_job_processes_found: No GoodJob processes found.
|
202
203
|
process: Process
|
203
204
|
schedulers: Schedulers
|
@@ -205,6 +206,9 @@ en:
|
|
205
206
|
title: Processes
|
206
207
|
updated: Updated
|
207
208
|
shared:
|
209
|
+
boolean:
|
210
|
+
'false': 'No'
|
211
|
+
'true': 'Yes'
|
208
212
|
error: Error
|
209
213
|
filter:
|
210
214
|
all: All
|
data/config/locales/es.yml
CHANGED
@@ -196,6 +196,7 @@ es:
|
|
196
196
|
unit: ''
|
197
197
|
processes:
|
198
198
|
index:
|
199
|
+
cron_enabled: Cron habilitado
|
199
200
|
no_good_job_processes_found: No hay procesos de GoodJob.
|
200
201
|
process: Proceso
|
201
202
|
schedulers: Schedulers
|
@@ -203,6 +204,9 @@ es:
|
|
203
204
|
title: Procesos
|
204
205
|
updated: Actualizado
|
205
206
|
shared:
|
207
|
+
boolean:
|
208
|
+
'false': 'No'
|
209
|
+
'true': Sí
|
206
210
|
error: Error
|
207
211
|
filter:
|
208
212
|
all: Todas
|
data/config/locales/fr.yml
CHANGED
@@ -198,6 +198,7 @@ fr:
|
|
198
198
|
unit: ''
|
199
199
|
processes:
|
200
200
|
index:
|
201
|
+
cron_enabled: Cron activé
|
201
202
|
no_good_job_processes_found: Aucun processus GoodJob trouvé.
|
202
203
|
process: Processus
|
203
204
|
schedulers: Schedulers
|
@@ -205,6 +206,9 @@ fr:
|
|
205
206
|
title: Processus
|
206
207
|
updated: Mis à jour
|
207
208
|
shared:
|
209
|
+
boolean:
|
210
|
+
'false': Non
|
211
|
+
'true': Oui
|
208
212
|
error: Erreur
|
209
213
|
filter:
|
210
214
|
all: Tous
|
data/config/locales/ja.yml
CHANGED
@@ -198,6 +198,7 @@ ja:
|
|
198
198
|
unit: ''
|
199
199
|
processes:
|
200
200
|
index:
|
201
|
+
cron_enabled: Cron が有効になっている
|
201
202
|
no_good_job_processes_found: GoodJobのプロセスが見つかりませんでした。
|
202
203
|
process: プロセス
|
203
204
|
schedulers: スケジューラー
|
@@ -205,6 +206,9 @@ ja:
|
|
205
206
|
title: プロセス
|
206
207
|
updated: 更新された
|
207
208
|
shared:
|
209
|
+
boolean:
|
210
|
+
'false': いいえ
|
211
|
+
'true': はい
|
208
212
|
error: エラー
|
209
213
|
filter:
|
210
214
|
all: 全て
|
data/config/locales/nl.yml
CHANGED
@@ -198,6 +198,7 @@ nl:
|
|
198
198
|
unit: ''
|
199
199
|
processes:
|
200
200
|
index:
|
201
|
+
cron_enabled: Cron ingeschakeld
|
201
202
|
no_good_job_processes_found: Geen GoodJob-processen gevonden.
|
202
203
|
process: Proces
|
203
204
|
schedulers: Planners
|
@@ -205,6 +206,9 @@ nl:
|
|
205
206
|
title: Processen
|
206
207
|
updated: Bijgewerkt
|
207
208
|
shared:
|
209
|
+
boolean:
|
210
|
+
'false': Nee
|
211
|
+
'true': Ja
|
208
212
|
error: Fout
|
209
213
|
filter:
|
210
214
|
all: Alle
|
data/config/locales/ru.yml
CHANGED
@@ -224,6 +224,7 @@ ru:
|
|
224
224
|
unit: ''
|
225
225
|
processes:
|
226
226
|
index:
|
227
|
+
cron_enabled: Крон включен
|
227
228
|
no_good_job_processes_found: Процессы GoodJob не найдены.
|
228
229
|
process: Процесс
|
229
230
|
schedulers: Планировщики
|
@@ -231,6 +232,9 @@ ru:
|
|
231
232
|
title: Процессы
|
232
233
|
updated: Обновлено
|
233
234
|
shared:
|
235
|
+
boolean:
|
236
|
+
'false': Нет
|
237
|
+
'true': Да
|
234
238
|
error: Ошибка
|
235
239
|
filter:
|
236
240
|
all: Все
|
data/config/locales/tr.yml
CHANGED
@@ -198,6 +198,7 @@ tr:
|
|
198
198
|
unit: ''
|
199
199
|
processes:
|
200
200
|
index:
|
201
|
+
cron_enabled: Cron etkin
|
201
202
|
no_good_job_processes_found: GoodJob süreci bulunamadı.
|
202
203
|
process: Süreç
|
203
204
|
schedulers: Planlayıcılar
|
@@ -205,6 +206,9 @@ tr:
|
|
205
206
|
title: Süreçler
|
206
207
|
updated: Güncellenmiş
|
207
208
|
shared:
|
209
|
+
boolean:
|
210
|
+
'false': Hayır
|
211
|
+
'true': Evet
|
208
212
|
error: Hata
|
209
213
|
filter:
|
210
214
|
all: Tümü
|
data/config/locales/uk.yml
CHANGED
@@ -224,6 +224,7 @@ uk:
|
|
224
224
|
unit: ''
|
225
225
|
processes:
|
226
226
|
index:
|
227
|
+
cron_enabled: Cron увімкнено
|
227
228
|
no_good_job_processes_found: Процеси GoodJob не знайдені.
|
228
229
|
process: Процес
|
229
230
|
schedulers: Планувальники
|
@@ -231,6 +232,9 @@ uk:
|
|
231
232
|
title: Процеси
|
232
233
|
updated: Оновлено
|
233
234
|
shared:
|
235
|
+
boolean:
|
236
|
+
'false': Ні
|
237
|
+
'true': Так
|
234
238
|
error: Помилка
|
235
239
|
filter:
|
236
240
|
all: Всі
|
data/lib/good_job/adapter.rb
CHANGED
@@ -168,13 +168,13 @@ module GoodJob
|
|
168
168
|
end
|
169
169
|
|
170
170
|
# Shut down the thread pool executors.
|
171
|
-
# @param timeout [nil, Numeric,
|
171
|
+
# @param timeout [nil, Numeric, NONE] Seconds to wait for active threads.
|
172
172
|
# * +nil+ trigger a shutdown but not wait for it to complete.
|
173
173
|
# * +-1+ wait until the shutdown is complete.
|
174
174
|
# * +0+ immediately shutdown and stop any threads.
|
175
175
|
# * A positive number will wait that many seconds before stopping any remaining active threads.
|
176
176
|
# @return [void]
|
177
|
-
def shutdown(timeout:
|
177
|
+
def shutdown(timeout: NONE)
|
178
178
|
@capsule&.shutdown(timeout: timeout)
|
179
179
|
@_async_started = false
|
180
180
|
end
|
data/lib/good_job/capsule.rb
CHANGED
@@ -32,9 +32,9 @@ module GoodJob
|
|
32
32
|
@shared_executor = GoodJob::SharedExecutor.new
|
33
33
|
@notifier = GoodJob::Notifier.new(enable_listening: @configuration.enable_listen_notify, executor: @shared_executor.executor)
|
34
34
|
@poller = GoodJob::Poller.new(poll_interval: @configuration.poll_interval)
|
35
|
-
@
|
36
|
-
@notifier.recipients
|
37
|
-
@poller.recipients
|
35
|
+
@multi_scheduler = GoodJob::MultiScheduler.from_configuration(@configuration, warm_cache_on_initialize: true)
|
36
|
+
@notifier.recipients.push([@multi_scheduler, :create_thread])
|
37
|
+
@poller.recipients.push(-> { @multi_scheduler.create_thread({ fanout: true }) })
|
38
38
|
|
39
39
|
@cron_manager = GoodJob::CronManager.new(@configuration.cron_entries, start_on_initialize: true, executor: @shared_executor.executor) if @configuration.enable_cron?
|
40
40
|
|
@@ -44,23 +44,23 @@ module GoodJob
|
|
44
44
|
end
|
45
45
|
|
46
46
|
# Shut down the thread pool executors.
|
47
|
-
# @param timeout [nil, Numeric,
|
47
|
+
# @param timeout [nil, Numeric, NONE] Seconds to wait for active threads.
|
48
48
|
# * +-1+ will wait for all active threads to complete.
|
49
49
|
# * +0+ will interrupt active threads.
|
50
50
|
# * +N+ will wait at most N seconds and then interrupt active threads.
|
51
51
|
# * +nil+ will trigger a shutdown but not wait for it to complete.
|
52
52
|
# @return [void]
|
53
|
-
def shutdown(timeout:
|
54
|
-
timeout = @configuration.shutdown_timeout if timeout ==
|
55
|
-
GoodJob._shutdown_all([@shared_executor, @notifier, @poller, @
|
53
|
+
def shutdown(timeout: NONE)
|
54
|
+
timeout = @configuration.shutdown_timeout if timeout == NONE
|
55
|
+
GoodJob._shutdown_all([@shared_executor, @notifier, @poller, @multi_scheduler, @cron_manager].compact, timeout: timeout)
|
56
56
|
@startable = false
|
57
57
|
@running = false
|
58
58
|
end
|
59
59
|
|
60
60
|
# Shutdown and then start the capsule again.
|
61
|
-
# @param timeout [Numeric,
|
61
|
+
# @param timeout [Numeric, NONE] Seconds to wait for active threads.
|
62
62
|
# @return [void]
|
63
|
-
def restart(timeout:
|
63
|
+
def restart(timeout: NONE)
|
64
64
|
raise ArgumentError, "Capsule#restart cannot be called with a timeout of nil" if timeout.nil?
|
65
65
|
|
66
66
|
shutdown(timeout: timeout)
|
@@ -74,7 +74,7 @@ module GoodJob
|
|
74
74
|
|
75
75
|
# @return [Boolean] Whether the capsule has been shutdown.
|
76
76
|
def shutdown?
|
77
|
-
[@shared_executor, @notifier, @poller, @
|
77
|
+
[@shared_executor, @notifier, @poller, @multi_scheduler, @cron_manager].compact.all?(&:shutdown?)
|
78
78
|
end
|
79
79
|
|
80
80
|
# Creates an execution thread(s) with the given attributes.
|
@@ -82,7 +82,7 @@ module GoodJob
|
|
82
82
|
# @return [Boolean, nil] Whether the thread was created.
|
83
83
|
def create_thread(job_state = nil)
|
84
84
|
start if startable?
|
85
|
-
@
|
85
|
+
@multi_scheduler&.create_thread(job_state)
|
86
86
|
end
|
87
87
|
|
88
88
|
private
|
data/lib/good_job/cli.rb
CHANGED
@@ -17,6 +17,9 @@ module GoodJob
|
|
17
17
|
# Requiring this loads the application's configuration and classes.
|
18
18
|
RAILS_ENVIRONMENT_RB = File.expand_path("config/environment.rb")
|
19
19
|
|
20
|
+
# Number of seconds between checking shutdown conditions
|
21
|
+
SHUTDOWN_EVENT_TIMEOUT = 10
|
22
|
+
|
20
23
|
class << self
|
21
24
|
# Whether the CLI is running from the executable
|
22
25
|
# @return [Boolean, nil]
|
@@ -106,14 +109,15 @@ module GoodJob
|
|
106
109
|
probe_server.start
|
107
110
|
end
|
108
111
|
|
109
|
-
|
112
|
+
require 'concurrent/atomic/event'
|
113
|
+
@stop_good_job_executable = Concurrent::Event.new
|
110
114
|
%w[INT TERM].each do |signal|
|
111
|
-
trap(signal) { @stop_good_job_executable
|
115
|
+
trap(signal) { Thread.new { @stop_good_job_executable.set }.join }
|
112
116
|
end
|
113
117
|
|
114
118
|
Kernel.loop do
|
115
|
-
|
116
|
-
break if @stop_good_job_executable || capsule.shutdown?
|
119
|
+
@stop_good_job_executable.wait(SHUTDOWN_EVENT_TIMEOUT)
|
120
|
+
break if @stop_good_job_executable.set? || capsule.shutdown?
|
117
121
|
end
|
118
122
|
|
119
123
|
systemd.stop do
|
@@ -0,0 +1,76 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'concurrent/atomic/atomic_fixnum'
|
4
|
+
|
5
|
+
module GoodJob # :nodoc:
|
6
|
+
class JobPerformer
|
7
|
+
# Metrics for the scheduler.
|
8
|
+
class Metrics
|
9
|
+
def initialize
|
10
|
+
@mutex = Mutex.new
|
11
|
+
@empty_executions = Concurrent::AtomicFixnum.new
|
12
|
+
@errored_executions = Concurrent::AtomicFixnum.new
|
13
|
+
@succeeded_executions = Concurrent::AtomicFixnum.new
|
14
|
+
@execution_at = nil
|
15
|
+
@check_queue_at = nil
|
16
|
+
end
|
17
|
+
|
18
|
+
# Increments number of failed executions.
|
19
|
+
# @return [Integer]
|
20
|
+
def increment_errored_executions
|
21
|
+
@execution_at = Time.current
|
22
|
+
@errored_executions.increment
|
23
|
+
end
|
24
|
+
|
25
|
+
# Increments number of succeeded executions.
|
26
|
+
# @return [Integer]
|
27
|
+
def increment_succeeded_executions
|
28
|
+
@execution_at = Time.current
|
29
|
+
@succeeded_executions.increment
|
30
|
+
end
|
31
|
+
|
32
|
+
# Increments number of dequeue attempts with no executions.
|
33
|
+
# @return [Integer]
|
34
|
+
def increment_empty_executions
|
35
|
+
@execution_at = Time.current
|
36
|
+
@empty_executions.increment
|
37
|
+
end
|
38
|
+
|
39
|
+
# Last time a job was executed (started or finished).
|
40
|
+
# @return [Time, nil]
|
41
|
+
def touch_execution_at
|
42
|
+
@execution_at = Time.current
|
43
|
+
end
|
44
|
+
|
45
|
+
# Last time the queue was checked for jobs.
|
46
|
+
# @return [Time, nil]
|
47
|
+
def touch_check_queue_at
|
48
|
+
@check_queue_at = Time.current
|
49
|
+
end
|
50
|
+
|
51
|
+
# All metrics in a Hash.
|
52
|
+
# @return [Hash]
|
53
|
+
def to_h
|
54
|
+
{
|
55
|
+
empty_executions_count: @empty_executions.value,
|
56
|
+
errored_executions_count: @errored_executions.value,
|
57
|
+
succeeded_executions_count: @succeeded_executions.value,
|
58
|
+
}.tap do |values|
|
59
|
+
values[:total_executions_count] = values[:succeeded_executions_count] + values[:errored_executions_count]
|
60
|
+
values[:execution_at] = @execution_at
|
61
|
+
values[:check_queue_at] = @check_queue_at
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
65
|
+
# Reset counters.
|
66
|
+
# @return [void]
|
67
|
+
def reset
|
68
|
+
@empty_executions.value = 0
|
69
|
+
@errored_executions.value = 0
|
70
|
+
@succeeded_executions.value = 0
|
71
|
+
@execution_at = nil
|
72
|
+
@check_queue_at = nil
|
73
|
+
end
|
74
|
+
end
|
75
|
+
end
|
76
|
+
end
|
@@ -16,6 +16,7 @@ module GoodJob
|
|
16
16
|
# @param queue_string [String] Queues to execute jobs from
|
17
17
|
def initialize(queue_string)
|
18
18
|
@queue_string = queue_string
|
19
|
+
@metrics = Metrics.new
|
19
20
|
end
|
20
21
|
|
21
22
|
# A meaningful name to identify the performer in logs and for debugging.
|
@@ -25,15 +26,28 @@ module GoodJob
|
|
25
26
|
end
|
26
27
|
|
27
28
|
# Perform the next eligible job
|
29
|
+
# @yield [Execution] Yields the execution, if one is dequeued
|
28
30
|
# @return [Object, nil] Returns job result or +nil+ if no job was found
|
29
31
|
def next
|
30
32
|
active_job_id = nil
|
31
33
|
job_query.perform_with_advisory_lock(parsed_queues: parsed_queues, queue_select_limit: GoodJob.configuration.queue_select_limit) do |execution|
|
32
|
-
|
33
|
-
|
34
|
+
@metrics.touch_check_queue_at
|
35
|
+
|
36
|
+
if execution
|
37
|
+
active_job_id = execution.active_job_id
|
38
|
+
performing_active_job_ids << active_job_id
|
39
|
+
@metrics.touch_execution_at
|
40
|
+
yield(execution) if block_given?
|
41
|
+
else
|
42
|
+
@metrics.increment_empty_executions
|
43
|
+
end
|
44
|
+
end.tap do |result|
|
45
|
+
if result
|
46
|
+
result.succeeded? ? @metrics.increment_succeeded_executions : @metrics.increment_errored_executions
|
47
|
+
end
|
34
48
|
end
|
35
49
|
ensure
|
36
|
-
performing_active_job_ids.delete(active_job_id)
|
50
|
+
performing_active_job_ids.delete(active_job_id) if active_job_id
|
37
51
|
end
|
38
52
|
|
39
53
|
# Tests whether this performer should be used in GoodJob's current state.
|
@@ -72,6 +86,20 @@ module GoodJob
|
|
72
86
|
GoodJob.cleanup_preserved_jobs
|
73
87
|
end
|
74
88
|
|
89
|
+
# Metrics about this performer
|
90
|
+
# @return [Hash]
|
91
|
+
def stats
|
92
|
+
{
|
93
|
+
name: name,
|
94
|
+
}.merge(@metrics.to_h)
|
95
|
+
end
|
96
|
+
|
97
|
+
# Reset metrics about this performer
|
98
|
+
# @return [void]
|
99
|
+
def reset_stats
|
100
|
+
@metrics.reset
|
101
|
+
end
|
102
|
+
|
75
103
|
private
|
76
104
|
|
77
105
|
attr_reader :queue_string
|
@@ -3,6 +3,29 @@
|
|
3
3
|
module GoodJob
|
4
4
|
# Delegates the interface of a single {Scheduler} to multiple Schedulers.
|
5
5
|
class MultiScheduler
|
6
|
+
# Creates MultiScheduler from a GoodJob::Configuration instance.
|
7
|
+
# @param configuration [GoodJob::Configuration]
|
8
|
+
# @param warm_cache_on_initialize [Boolean]
|
9
|
+
# @return [GoodJob::MultiScheduler]
|
10
|
+
def self.from_configuration(configuration, warm_cache_on_initialize: false)
|
11
|
+
schedulers = configuration.queue_string.split(';').map do |queue_string_and_max_threads|
|
12
|
+
queue_string, max_threads = queue_string_and_max_threads.split(':')
|
13
|
+
max_threads = (max_threads || configuration.max_threads).to_i
|
14
|
+
|
15
|
+
job_performer = GoodJob::JobPerformer.new(queue_string)
|
16
|
+
GoodJob::Scheduler.new(
|
17
|
+
job_performer,
|
18
|
+
max_threads: max_threads,
|
19
|
+
max_cache: configuration.max_cache,
|
20
|
+
warm_cache_on_initialize: warm_cache_on_initialize,
|
21
|
+
cleanup_interval_seconds: configuration.cleanup_interval_seconds,
|
22
|
+
cleanup_interval_jobs: configuration.cleanup_interval_jobs
|
23
|
+
)
|
24
|
+
end
|
25
|
+
|
26
|
+
new(schedulers)
|
27
|
+
end
|
28
|
+
|
6
29
|
# @return [Array<Scheduler>] List of the scheduler delegates
|
7
30
|
attr_reader :schedulers
|
8
31
|
|
@@ -43,7 +66,7 @@ module GoodJob
|
|
43
66
|
def create_thread(state = nil)
|
44
67
|
results = []
|
45
68
|
|
46
|
-
if state
|
69
|
+
if state && !state[:fanout]
|
47
70
|
schedulers.any? do |scheduler|
|
48
71
|
scheduler.create_thread(state).tap { |result| results << result }
|
49
72
|
end
|
@@ -61,5 +84,19 @@ module GoodJob
|
|
61
84
|
nil
|
62
85
|
end
|
63
86
|
end
|
87
|
+
|
88
|
+
def stats
|
89
|
+
scheduler_stats = schedulers.map(&:stats)
|
90
|
+
|
91
|
+
{
|
92
|
+
schedulers: scheduler_stats,
|
93
|
+
empty_executions_count: scheduler_stats.sum { |stats| stats.fetch(:empty_executions_count, 0) },
|
94
|
+
errored_executions_count: scheduler_stats.sum { |stats| stats.fetch(:errored_executions_count, 0) },
|
95
|
+
succeeded_executions_count: scheduler_stats.sum { |stats| stats.fetch(:succeeded_executions_count, 0) },
|
96
|
+
total_executions_count: scheduler_stats.sum { |stats| stats.fetch(:total_executions_count, 0) },
|
97
|
+
execution_at: scheduler_stats.map { |stats| stats.fetch(:execution_at, nil) }.compact.max,
|
98
|
+
check_queue_at: scheduler_stats.map { |stats| stats.fetch(:check_queue_at, nil) }.compact.max,
|
99
|
+
}
|
100
|
+
end
|
64
101
|
end
|
65
102
|
end
|
data/lib/good_job/scheduler.rb
CHANGED
@@ -4,7 +4,6 @@ require "concurrent/executor/thread_pool_executor"
|
|
4
4
|
require "concurrent/executor/timer_set"
|
5
5
|
require "concurrent/scheduled_task"
|
6
6
|
require "concurrent/utility/processor_counter"
|
7
|
-
require 'good_job/metrics'
|
8
7
|
|
9
8
|
module GoodJob # :nodoc:
|
10
9
|
#
|
@@ -36,33 +35,6 @@ module GoodJob # :nodoc:
|
|
36
35
|
# @return [Array<GoodJob::Scheduler>, nil]
|
37
36
|
cattr_reader :instances, default: Concurrent::Array.new, instance_reader: false
|
38
37
|
|
39
|
-
# Creates GoodJob::Scheduler(s) and Performers from a GoodJob::Configuration instance.
|
40
|
-
# @param configuration [GoodJob::Configuration]
|
41
|
-
# @param warm_cache_on_initialize [Boolean]
|
42
|
-
# @return [GoodJob::Scheduler, GoodJob::MultiScheduler]
|
43
|
-
def self.from_configuration(configuration, warm_cache_on_initialize: false)
|
44
|
-
schedulers = configuration.queue_string.split(';').map do |queue_string_and_max_threads|
|
45
|
-
queue_string, max_threads = queue_string_and_max_threads.split(':')
|
46
|
-
max_threads = (max_threads || configuration.max_threads).to_i
|
47
|
-
|
48
|
-
job_performer = GoodJob::JobPerformer.new(queue_string)
|
49
|
-
GoodJob::Scheduler.new(
|
50
|
-
job_performer,
|
51
|
-
max_threads: max_threads,
|
52
|
-
max_cache: configuration.max_cache,
|
53
|
-
warm_cache_on_initialize: warm_cache_on_initialize,
|
54
|
-
cleanup_interval_seconds: configuration.cleanup_interval_seconds,
|
55
|
-
cleanup_interval_jobs: configuration.cleanup_interval_jobs
|
56
|
-
)
|
57
|
-
end
|
58
|
-
|
59
|
-
if schedulers.size > 1
|
60
|
-
GoodJob::MultiScheduler.new(schedulers)
|
61
|
-
else
|
62
|
-
schedulers.first
|
63
|
-
end
|
64
|
-
end
|
65
|
-
|
66
38
|
# Human readable name of the scheduler that includes configuration values.
|
67
39
|
# @return [String]
|
68
40
|
attr_reader :name
|
@@ -88,7 +60,6 @@ module GoodJob # :nodoc:
|
|
88
60
|
@executor_options[:name] = name
|
89
61
|
|
90
62
|
@cleanup_tracker = CleanupTracker.new(cleanup_interval_seconds: cleanup_interval_seconds, cleanup_interval_jobs: cleanup_interval_jobs)
|
91
|
-
@metrics = ::GoodJob::Metrics.new
|
92
63
|
@executor_options[:name] = name
|
93
64
|
|
94
65
|
create_executor
|
@@ -143,7 +114,7 @@ module GoodJob # :nodoc:
|
|
143
114
|
|
144
115
|
instrument("scheduler_restart_pools") do
|
145
116
|
shutdown(timeout: timeout)
|
146
|
-
@
|
117
|
+
@performer.reset_stats
|
147
118
|
create_executor
|
148
119
|
warm_cache
|
149
120
|
end
|
@@ -152,16 +123,17 @@ module GoodJob # :nodoc:
|
|
152
123
|
# Wakes a thread to allow the performer to execute a task.
|
153
124
|
# @param state [Hash, nil] Contextual information for the performer. See {JobPerformer#next?}.
|
154
125
|
# @return [Boolean, nil] Whether work was started.
|
155
|
-
#
|
156
126
|
# * +nil+ if the scheduler is unable to take new work, for example if the thread pool is shut down or at capacity.
|
157
127
|
# * +true+ if the performer started executing work.
|
158
128
|
# * +false+ if the performer decides not to attempt to execute a task based on the +state+ that is passed to it.
|
159
129
|
def create_thread(state = nil)
|
160
130
|
return nil unless executor.running?
|
161
131
|
|
162
|
-
if state
|
132
|
+
if state.present?
|
163
133
|
return false unless performer.next?(state)
|
164
134
|
|
135
|
+
fanout = state&.fetch(:fanout, nil)
|
136
|
+
|
165
137
|
if state[:count]
|
166
138
|
# When given state for multiple jobs, try to create a thread for each one.
|
167
139
|
# Return true if a thread can be created for all of them, nil if partial or none.
|
@@ -193,7 +165,7 @@ module GoodJob # :nodoc:
|
|
193
165
|
return nil unless remaining_cache_count.positive?
|
194
166
|
end
|
195
167
|
|
196
|
-
create_task(delay)
|
168
|
+
create_task(delay, fanout: fanout)
|
197
169
|
|
198
170
|
run_now ? true : nil
|
199
171
|
end
|
@@ -207,16 +179,6 @@ module GoodJob # :nodoc:
|
|
207
179
|
unhandled_error = thread_error || result&.unhandled_error
|
208
180
|
GoodJob._on_thread_error(unhandled_error) if unhandled_error
|
209
181
|
|
210
|
-
if unhandled_error || result&.handled_error
|
211
|
-
@metrics.increment_errored_executions
|
212
|
-
elsif result&.unexecutable
|
213
|
-
@metrics.increment_unexecutable_executions
|
214
|
-
elsif result
|
215
|
-
@metrics.increment_succeeded_executions
|
216
|
-
else
|
217
|
-
@metrics.increment_empty_executions
|
218
|
-
end
|
219
|
-
|
220
182
|
instrument("finished_job_task", { result: output, error: thread_error, time: time })
|
221
183
|
return unless output
|
222
184
|
|
@@ -232,6 +194,7 @@ module GoodJob # :nodoc:
|
|
232
194
|
# @return [Hash]
|
233
195
|
def stats
|
234
196
|
available_threads = executor.ready_worker_count
|
197
|
+
|
235
198
|
{
|
236
199
|
name: name,
|
237
200
|
queues: performer.name,
|
@@ -241,7 +204,7 @@ module GoodJob # :nodoc:
|
|
241
204
|
max_cache: @max_cache,
|
242
205
|
active_cache: cache_count,
|
243
206
|
available_cache: remaining_cache_count,
|
244
|
-
}.merge!(@
|
207
|
+
}.merge!(@performer.stats.without(:name))
|
245
208
|
end
|
246
209
|
|
247
210
|
# Preload existing runnable and future-scheduled jobs
|
@@ -300,12 +263,15 @@ module GoodJob # :nodoc:
|
|
300
263
|
end
|
301
264
|
|
302
265
|
# @param delay [Integer]
|
266
|
+
# @param fanout [Boolean] Whether to eagerly create a 2nd execution thread if a job is found.
|
303
267
|
# @return [void]
|
304
|
-
def create_task(delay = 0)
|
305
|
-
future = Concurrent::ScheduledTask.new(delay, args: [performer], executor: executor, timer_set: timer_set) do |thr_performer|
|
268
|
+
def create_task(delay = 0, fanout: false)
|
269
|
+
future = Concurrent::ScheduledTask.new(delay, args: [self, performer], executor: executor, timer_set: timer_set) do |thr_scheduler, thr_performer|
|
306
270
|
Thread.current.name = Thread.current.name.sub("-worker-", "-thread-") if Thread.current.name
|
307
271
|
Rails.application.reloader.wrap do
|
308
|
-
thr_performer.next
|
272
|
+
thr_performer.next do |found|
|
273
|
+
thr_scheduler.create_thread({ fanout: fanout }) if found && fanout
|
274
|
+
end
|
309
275
|
end
|
310
276
|
end
|
311
277
|
future.add_observer(self, :task_observer)
|
data/lib/good_job/version.rb
CHANGED
data/lib/good_job.rb
CHANGED
@@ -26,6 +26,7 @@ require 'good_job/current_thread'
|
|
26
26
|
require "good_job/daemon"
|
27
27
|
require "good_job/dependencies"
|
28
28
|
require "good_job/job_performer"
|
29
|
+
require "good_job/job_performer/metrics"
|
29
30
|
require "good_job/log_subscriber"
|
30
31
|
require "good_job/multi_scheduler"
|
31
32
|
require "good_job/notifier"
|
@@ -42,6 +43,10 @@ require "good_job/systemd_service"
|
|
42
43
|
module GoodJob
|
43
44
|
include GoodJob::Dependencies
|
44
45
|
|
46
|
+
# Default, null, blank value placeholder.
|
47
|
+
NONE = Module.new.freeze
|
48
|
+
|
49
|
+
# Default logger for GoodJob; overridden by Rails.logger in Railtie.
|
45
50
|
DEFAULT_LOGGER = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new($stdout))
|
46
51
|
|
47
52
|
# @!attribute [rw] active_record_parent_class
|
@@ -233,13 +238,19 @@ module GoodJob
|
|
233
238
|
# This is primarily intended for usage in a test environment.
|
234
239
|
# Unhandled job errors will be raised.
|
235
240
|
# @param queue_string [String] Queues to execute jobs from
|
241
|
+
# @param limit [Integer, nil] Maximum number of iterations for the loop
|
236
242
|
# @return [void]
|
237
|
-
def self.perform_inline(queue_string = "*")
|
243
|
+
def self.perform_inline(queue_string = "*", limit: nil)
|
238
244
|
job_performer = JobPerformer.new(queue_string)
|
245
|
+
iteration = 0
|
239
246
|
loop do
|
247
|
+
break if limit && iteration >= limit
|
248
|
+
|
240
249
|
result = Rails.application.reloader.wrap { job_performer.next }
|
241
250
|
break unless result
|
242
251
|
raise result.unhandled_error if result.unhandled_error
|
252
|
+
|
253
|
+
iteration += 1
|
243
254
|
end
|
244
255
|
end
|
245
256
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 3.
|
4
|
+
version: 3.21.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-
|
11
|
+
date: 2023-11-24 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -366,8 +366,8 @@ files:
|
|
366
366
|
- lib/good_job/http_server.rb
|
367
367
|
- lib/good_job/interrupt_error.rb
|
368
368
|
- lib/good_job/job_performer.rb
|
369
|
+
- lib/good_job/job_performer/metrics.rb
|
369
370
|
- lib/good_job/log_subscriber.rb
|
370
|
-
- lib/good_job/metrics.rb
|
371
371
|
- lib/good_job/multi_scheduler.rb
|
372
372
|
- lib/good_job/notifier.rb
|
373
373
|
- lib/good_job/notifier/process_heartbeat.rb
|
data/lib/good_job/metrics.rb
DELETED
@@ -1,57 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
module GoodJob # :nodoc:
|
4
|
-
# Metrics for the scheduler.
|
5
|
-
class Metrics
|
6
|
-
def initialize
|
7
|
-
@empty_executions = Concurrent::AtomicFixnum.new
|
8
|
-
@errored_executions = Concurrent::AtomicFixnum.new
|
9
|
-
@succeeded_executions = Concurrent::AtomicFixnum.new
|
10
|
-
@unexecutable_executions = Concurrent::AtomicFixnum.new
|
11
|
-
end
|
12
|
-
|
13
|
-
# Increments number of empty queried executions.
|
14
|
-
# @return [Integer]
|
15
|
-
def increment_empty_executions
|
16
|
-
@empty_executions.increment
|
17
|
-
end
|
18
|
-
|
19
|
-
# Increments number of failed executions.
|
20
|
-
# @return [Integer]
|
21
|
-
def increment_errored_executions
|
22
|
-
@errored_executions.increment
|
23
|
-
end
|
24
|
-
|
25
|
-
# Increments number of succeeded executions.
|
26
|
-
# @return [Integer]
|
27
|
-
def increment_succeeded_executions
|
28
|
-
@succeeded_executions.increment
|
29
|
-
end
|
30
|
-
|
31
|
-
# Increments number of unlocked executions.
|
32
|
-
# @return [Integer]
|
33
|
-
def increment_unexecutable_executions
|
34
|
-
@unexecutable_executions.increment
|
35
|
-
end
|
36
|
-
|
37
|
-
def to_h
|
38
|
-
{
|
39
|
-
empty_executions_count: @empty_executions.value,
|
40
|
-
errored_executions_count: @errored_executions.value,
|
41
|
-
succeeded_executions_count: @succeeded_executions.value,
|
42
|
-
unexecutable_executions_count: @unexecutable_executions.value,
|
43
|
-
}.tap do |values|
|
44
|
-
values[:total_executions_count] = values.values.sum
|
45
|
-
end
|
46
|
-
end
|
47
|
-
|
48
|
-
# Reset counters.
|
49
|
-
# @return [void]
|
50
|
-
def reset
|
51
|
-
@empty_executions.value = 0
|
52
|
-
@errored_executions.value = 0
|
53
|
-
@succeeded_executions.value = 0
|
54
|
-
@unexecutable_executions.value = 0
|
55
|
-
end
|
56
|
-
end
|
57
|
-
end
|