good_job 2.11.1 → 2.11.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +15 -0
- data/README.md +28 -2
- data/engine/app/charts/good_job/scheduled_by_queue_chart.rb +2 -1
- data/engine/app/filters/good_job/jobs_filter.rb +2 -2
- data/lib/good_job/active_job_job.rb +8 -1
- data/lib/good_job/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 44ffd48ff0a352758f67aeaaf60eeb7242fdf95c71cf16fcb7089689d72709a6
|
4
|
+
data.tar.gz: da29ba7f5b8f212d989d24fec8c7e5672166d3950bae04f3f29a2ca6e8cb035a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c72947a7302d52b2b4f81a6092fbaca28dc900095fa477e0f7abd6d3419c5825c9693063fcdf6081d00ed93b082be705cef2beb2d1088c90de54737098103df5
|
7
|
+
data.tar.gz: 4d9b6c34665cd02c840841460a2a162592d494fb27bd79e12987d4cda50a93d8959f130f29ccbad7e04b4be79ef5878dd13ae98ff87c6d8c87aad514b8937cd9
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,20 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v2.11.2](https://github.com/bensheldon/good_job/tree/v2.11.2) (2022-03-03)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.11.1...v2.11.2)
|
6
|
+
|
7
|
+
**Closed issues:**
|
8
|
+
|
9
|
+
- Best practices in deploying and monitoring a queue [\#523](https://github.com/bensheldon/good_job/issues/523)
|
10
|
+
|
11
|
+
**Merged pull requests:**
|
12
|
+
|
13
|
+
- Wrap Rspec before and example blocks with a mutex for JRuby [\#537](https://github.com/bensheldon/good_job/pull/537) ([bensheldon](https://github.com/bensheldon))
|
14
|
+
- Delegate `ActiveJobJob.table_name` to `Execution` and prevent it from being directly assignable [\#536](https://github.com/bensheldon/good_job/pull/536) ([bensheldon](https://github.com/bensheldon))
|
15
|
+
- Enable DB table names customization [\#535](https://github.com/bensheldon/good_job/pull/535) ([dimvic](https://github.com/dimvic))
|
16
|
+
- Added a chapter on how to prepare for production. [\#525](https://github.com/bensheldon/good_job/pull/525) ([stas](https://github.com/stas))
|
17
|
+
|
3
18
|
## [v2.11.1](https://github.com/bensheldon/good_job/tree/v2.11.1) (2022-03-01)
|
4
19
|
|
5
20
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.11.0...v2.11.1)
|
data/README.md
CHANGED
@@ -52,6 +52,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
52
52
|
- [Timeouts](#timeouts)
|
53
53
|
- [Optimize queues, threads, and processes](#optimize-queues-threads-and-processes)
|
54
54
|
- [Database connections](#database-connections)
|
55
|
+
- [Production setup](#production-setup)
|
55
56
|
- [Execute jobs async / in-process](#execute-jobs-async--in-process)
|
56
57
|
- [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
|
57
58
|
- [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
|
@@ -674,13 +675,38 @@ Keep in mind, queue operations and management is an advanced discipline. This st
|
|
674
675
|
|
675
676
|
### Database connections
|
676
677
|
|
677
|
-
Each GoodJob execution thread requires its own database connection that is automatically checked out from Rails’ connection pool.
|
678
|
+
Each GoodJob execution thread requires its own database connection that is automatically checked out from Rails’ connection pool. For example:
|
678
679
|
|
679
680
|
```yaml
|
680
681
|
# config/database.yml
|
681
|
-
pool: <%=
|
682
|
+
pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5).to_i + (ENV.fetch("GOOD_JOB_MAX_THREADS", 4).to_i %>
|
682
683
|
```
|
683
684
|
|
685
|
+
To calculate the total number of the database connections you'll need:
|
686
|
+
|
687
|
+
- 1 connection dedicated to the scheduler aka `LISTEN/NOTIFY`
|
688
|
+
- 1 connection per query pool thread e.g. `--queues=mice:2;elephants:1` is 3 threads. Pool thread size defaults to `--max-threads`
|
689
|
+
- (optional) 2 connections for Cron scheduler if you're running it
|
690
|
+
- (optional) 1 connection per subthread, if your application makes multithreaded database queries within a job
|
691
|
+
- When running `:async`, you must also add the number of threads by the webserver
|
692
|
+
|
693
|
+
The queue process will not crash if the connections pool is exhausted, instead it will report an exception (eg. `ActiveRecord::ConnectionTimeoutError`).
|
694
|
+
|
695
|
+
#### Production setup
|
696
|
+
|
697
|
+
When running GoodJob in a production environment, you should be mindful of:
|
698
|
+
|
699
|
+
- [Execution mode](execute-jobs-async--in-process)
|
700
|
+
- [Database connection pool size](#database-connections)
|
701
|
+
- [Health check probes](#cli-http-health-check-probes) and potentially the [instrumentation support](#monitor-and-preserve-worked-jobs)
|
702
|
+
|
703
|
+
The recommended way to monitor the queue in production is:
|
704
|
+
|
705
|
+
- have an exception notifier callback (see `on_thread_error`)
|
706
|
+
- if possible, run the queue as a dedicated instance and use available HTTP health check probes instead of pid-based monitoring
|
707
|
+
- keep an eye on the number of jobs in the queue (abnormal high number of unscheduled jobs means the queue could be underperforming)
|
708
|
+
- consider performance monitoring services which support the built-in Rails instrumentation (eg. Sentry, Skylight, etc.)
|
709
|
+
|
684
710
|
### Execute jobs async / in-process
|
685
711
|
|
686
712
|
GoodJob can execute jobs "async" in the same process as the web server (e.g. `bin/rails s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
|
@@ -9,6 +9,7 @@ module GoodJob
|
|
9
9
|
def data
|
10
10
|
end_time = Time.current
|
11
11
|
start_time = end_time - 1.day
|
12
|
+
table_name = GoodJob::ActiveJobJob.table_name
|
12
13
|
|
13
14
|
count_query = Arel.sql(GoodJob::Execution.pg_or_jdbc_query(<<~SQL.squish))
|
14
15
|
SELECT *
|
@@ -23,7 +24,7 @@ module GoodJob
|
|
23
24
|
queue_name,
|
24
25
|
count(*) AS count
|
25
26
|
FROM (
|
26
|
-
#{@filter.filtered_query.except(:select, :order).select('queue_name',
|
27
|
+
#{@filter.filtered_query.except(:select, :order).select('queue_name', "COALESCE(#{table_name}.scheduled_at, #{table_name}.created_at)::timestamp AS scheduled_at").to_sql}
|
27
28
|
) sources
|
28
29
|
GROUP BY date_trunc('hour', scheduled_at), queue_name
|
29
30
|
) sources ON sources.scheduled_at = timestamp
|
@@ -14,7 +14,7 @@ module GoodJob
|
|
14
14
|
|
15
15
|
def filtered_query
|
16
16
|
query = base_query.includes(:executions)
|
17
|
-
.joins_advisory_locks.select(
|
17
|
+
.joins_advisory_locks.select("#{GoodJob::ActiveJobJob.table_name}.*", 'pg_locks.locktype AS locktype')
|
18
18
|
|
19
19
|
query = query.job_class(params[:job_class]) if params[:job_class].present?
|
20
20
|
query = query.where(queue_name: params[:queue_name]) if params[:queue_name].present?
|
@@ -31,7 +31,7 @@ module GoodJob
|
|
31
31
|
when 'scheduled'
|
32
32
|
query = query.scheduled
|
33
33
|
when 'running'
|
34
|
-
query = query.running.select(
|
34
|
+
query = query.running.select("#{GoodJob::ActiveJobJob.table_name}.*", 'pg_locks.locktype')
|
35
35
|
when 'queued'
|
36
36
|
query = query.queued
|
37
37
|
end
|
@@ -15,7 +15,14 @@ module GoodJob
|
|
15
15
|
# Attached to a Job's Execution when the Job is discarded.
|
16
16
|
DiscardJobError = Class.new(StandardError)
|
17
17
|
|
18
|
-
|
18
|
+
class << self
|
19
|
+
delegate :table_name, to: Execution
|
20
|
+
|
21
|
+
def table_name=(_value)
|
22
|
+
raise NotImplementedError, 'Assign GoodJob::Execution.table_name directly'
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
19
26
|
self.primary_key = 'active_job_id'
|
20
27
|
self.advisory_lockable_column = 'active_job_id'
|
21
28
|
|
data/lib/good_job/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.11.
|
4
|
+
version: 2.11.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2022-03-
|
11
|
+
date: 2022-03-03 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|