good_job 0.2.0 → 0.2.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +39 -0
- data/README.md +2 -2
- data/{bin → exe}/good_job +0 -0
- data/lib/good_job/adapter.rb +6 -4
- data/lib/good_job/cli.rb +2 -1
- data/lib/good_job/lockable.rb +16 -11
- data/lib/good_job/logging.rb +33 -35
- data/lib/good_job/scheduler.rb +10 -4
- data/lib/good_job/version.rb +1 -1
- metadata +75 -17
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 66c19e40862f09e5c7816911c0a21e8cb88f036980a9c33df62cba058b3993e5
|
4
|
+
data.tar.gz: 218153476e92ca847e7279d9979c4396ed02f36a41918b32b58bf1fb17b8cd92
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d5b0305c889392d2879ccd818eb98b09a867d85a5b853dc8ec35b5a17fee3490cc082f032827f631b15e0a8e1143ea676eea05738d901f97ec4737adf0a514d0
|
7
|
+
data.tar.gz: 96a8537b85de48b07f676746642bb43ff18369db559b0c058c21d101a7740a0269a772a3b8629fefe3242d2e8a1df0d34a0f12b85b5928caf6272572fcff0d67
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,39 @@
|
|
1
|
+
# Changelog
|
2
|
+
|
3
|
+
## [Unreleased](https://github.com/bensheldon/good_job/tree/HEAD)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.2.0...HEAD)
|
6
|
+
|
7
|
+
**Merged pull requests:**
|
8
|
+
|
9
|
+
- Clean up Gemspec [\#15](https://github.com/bensheldon/good_job/pull/15) ([bensheldon](https://github.com/bensheldon))
|
10
|
+
- Set up Rubocop [\#14](https://github.com/bensheldon/good_job/pull/14) ([bensheldon](https://github.com/bensheldon))
|
11
|
+
- Add pg gem as explicit dependency [\#13](https://github.com/bensheldon/good_job/pull/13) ([bensheldon](https://github.com/bensheldon))
|
12
|
+
- Bump nokogiri from 1.10.7 to 1.10.9 [\#12](https://github.com/bensheldon/good_job/pull/12) ([dependabot[bot]](https://github.com/apps/dependabot))
|
13
|
+
- Add Appraisal with tests for Rails 5.1, 5.2, 6.0 [\#11](https://github.com/bensheldon/good_job/pull/11) ([bensheldon](https://github.com/bensheldon))
|
14
|
+
|
15
|
+
## [v0.2.0](https://github.com/bensheldon/good_job/tree/v0.2.0) (2020-03-06)
|
16
|
+
|
17
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.1.0...v0.2.0)
|
18
|
+
|
19
|
+
**Merged pull requests:**
|
20
|
+
|
21
|
+
- Use Rails.logger and ActiveSupport::Notifications for logging instead of puts [\#10](https://github.com/bensheldon/good_job/pull/10) ([bensheldon](https://github.com/bensheldon))
|
22
|
+
- Remove minitest files [\#9](https://github.com/bensheldon/good_job/pull/9) ([bensheldon](https://github.com/bensheldon))
|
23
|
+
- Use scheduled\_at and priority for scheduling [\#8](https://github.com/bensheldon/good_job/pull/8) ([bensheldon](https://github.com/bensheldon))
|
24
|
+
- Create Github Action workflow for PRs and Issues [\#7](https://github.com/bensheldon/good_job/pull/7) ([bensheldon](https://github.com/bensheldon))
|
25
|
+
|
26
|
+
## [v0.1.0](https://github.com/bensheldon/good_job/tree/v0.1.0) (2020-03-03)
|
27
|
+
|
28
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/6866006239f1a6b7fcb7103f5df60d904952fb84...v0.1.0)
|
29
|
+
|
30
|
+
**Merged pull requests:**
|
31
|
+
|
32
|
+
- Add executable with Thor [\#4](https://github.com/bensheldon/good_job/pull/4) ([bensheldon](https://github.com/bensheldon))
|
33
|
+
- Refactor adapter enqueing methods; expand Readme, tests, editorconfig [\#3](https://github.com/bensheldon/good_job/pull/3) ([bensheldon](https://github.com/bensheldon))
|
34
|
+
- Fetch new jobs within the worker thread itself; incrementally grow worker threads [\#2](https://github.com/bensheldon/good_job/pull/2) ([bensheldon](https://github.com/bensheldon))
|
35
|
+
- Set up Github Workflows for tests [\#1](https://github.com/bensheldon/good_job/pull/1) ([bensheldon](https://github.com/bensheldon))
|
36
|
+
|
37
|
+
|
38
|
+
|
39
|
+
\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
|
data/README.md
CHANGED
@@ -107,10 +107,10 @@ $ gem signin
|
|
107
107
|
$ gem bump -v minor --no-commit
|
108
108
|
|
109
109
|
# Update the changelog
|
110
|
-
$ bundle exec changelog
|
110
|
+
$ bundle exec rake changelog
|
111
111
|
|
112
112
|
# Commit the version and changelog to git
|
113
|
-
$ bundle commit_version
|
113
|
+
$ bundle exec rake commit_version
|
114
114
|
|
115
115
|
# Push to rubygems.org
|
116
116
|
$ gem release
|
data/{bin → exe}/good_job
RENAMED
File without changes
|
data/lib/good_job/adapter.rb
CHANGED
@@ -5,10 +5,6 @@ module GoodJob
|
|
5
5
|
@scheduler = InlineScheduler.new if inline?
|
6
6
|
end
|
7
7
|
|
8
|
-
def inline?
|
9
|
-
@options.fetch(:inline, false)
|
10
|
-
end
|
11
|
-
|
12
8
|
def enqueue(job)
|
13
9
|
enqueue_at(job, nil)
|
14
10
|
end
|
@@ -35,5 +31,11 @@ module GoodJob
|
|
35
31
|
def shutdown(wait: true)
|
36
32
|
@scheduler&.shutdown(wait: wait)
|
37
33
|
end
|
34
|
+
|
35
|
+
private
|
36
|
+
|
37
|
+
def inline?
|
38
|
+
@options.fetch(:inline, false)
|
39
|
+
end
|
38
40
|
end
|
39
41
|
end
|
data/lib/good_job/cli.rb
CHANGED
data/lib/good_job/lockable.rb
CHANGED
@@ -17,7 +17,7 @@ module GoodJob
|
|
17
17
|
scope :advisory_unlocked, -> { joins_advisory_locks.where(pg_locks: { locktype: nil }) }
|
18
18
|
scope :with_advisory_lock, (lambda do
|
19
19
|
where(<<~SQL)
|
20
|
-
|
20
|
+
pg_try_advisory_lock(('x'||substr(md5(id::text), 1, 16))::bit(64)::bigint)
|
21
21
|
SQL
|
22
22
|
end)
|
23
23
|
|
@@ -42,7 +42,7 @@ module GoodJob
|
|
42
42
|
end
|
43
43
|
|
44
44
|
def advisory_lock
|
45
|
-
self.class.connection.execute(
|
45
|
+
self.class.connection.execute(sanitize_sql_for_conditions(["SELECT 1 as one WHERE pg_try_advisory_lock(('x'||substr(md5(?), 1, 16))::bit(64)::bigint)", id])).ntuples.positive?
|
46
46
|
end
|
47
47
|
|
48
48
|
def advisory_lock!
|
@@ -51,17 +51,15 @@ module GoodJob
|
|
51
51
|
end
|
52
52
|
|
53
53
|
def with_advisory_lock
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
raise
|
60
|
-
end
|
54
|
+
advisory_lock!
|
55
|
+
yield
|
56
|
+
rescue StandardError => e
|
57
|
+
advisory_unlock unless e.is_a? RecordAlreadyAdvisoryLockedError
|
58
|
+
raise
|
61
59
|
end
|
62
60
|
|
63
61
|
def advisory_locked?
|
64
|
-
self.class.connection.execute(<<~SQL).ntuples
|
62
|
+
self.class.connection.execute(<<~SQL).ntuples.positive?
|
65
63
|
SELECT 1 as one
|
66
64
|
FROM pg_locks
|
67
65
|
WHERE
|
@@ -73,7 +71,7 @@ module GoodJob
|
|
73
71
|
end
|
74
72
|
|
75
73
|
def owns_advisory_lock?
|
76
|
-
self.class.connection.execute(<<~SQL).ntuples
|
74
|
+
self.class.connection.execute(<<~SQL).ntuples.positive?
|
77
75
|
SELECT 1 as one
|
78
76
|
FROM pg_locks
|
79
77
|
WHERE
|
@@ -92,6 +90,13 @@ module GoodJob
|
|
92
90
|
def advisory_unlock!
|
93
91
|
advisory_unlock while advisory_locked?
|
94
92
|
end
|
93
|
+
|
94
|
+
private
|
95
|
+
|
96
|
+
def sanitize_sql_for_conditions(*args)
|
97
|
+
# Made public in Rails 5.2
|
98
|
+
self.class.send(:sanitize_sql_for_conditions, *args)
|
99
|
+
end
|
95
100
|
end
|
96
101
|
end
|
97
102
|
end
|
data/lib/good_job/logging.rb
CHANGED
@@ -1,58 +1,56 @@
|
|
1
|
-
module GoodJob
|
2
|
-
|
3
|
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
1
|
+
module GoodJob
|
2
|
+
module Logging
|
3
|
+
extend ActiveSupport::Concern
|
4
|
+
|
5
|
+
included do
|
6
|
+
cattr_accessor :logger, default: ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
|
7
|
+
|
8
|
+
def self.tag_logger(*tags)
|
9
|
+
if logger.respond_to?(:tagged)
|
10
|
+
tags.unshift "GoodJob" unless logger.formatter.current_tags.include?("GoodJob")
|
11
|
+
logger.tagged(*tags) { yield }
|
12
|
+
else
|
13
|
+
yield
|
14
|
+
end
|
13
15
|
end
|
14
16
|
end
|
15
|
-
end
|
16
17
|
|
17
|
-
|
18
|
-
|
19
|
-
|
18
|
+
class LogSubscriber < ActiveSupport::LogSubscriber
|
19
|
+
def create(event)
|
20
|
+
good_job = event.payload[:good_job]
|
20
21
|
|
21
|
-
|
22
|
-
|
22
|
+
info do
|
23
|
+
"Created GoodJob resource with id #{good_job.id}"
|
24
|
+
end
|
23
25
|
end
|
24
|
-
end
|
25
26
|
|
26
|
-
|
27
|
-
|
28
|
-
|
27
|
+
def timer_task_finished(event)
|
28
|
+
exception = event.payload[:error]
|
29
|
+
return unless exception
|
29
30
|
|
30
|
-
if exception
|
31
31
|
error do
|
32
32
|
"ERROR: #{exception}\n #{exception.backtrace}"
|
33
33
|
end
|
34
34
|
end
|
35
|
-
end
|
36
35
|
|
37
|
-
|
38
|
-
|
39
|
-
|
36
|
+
def job_finished(event)
|
37
|
+
exception = event.payload[:error]
|
38
|
+
return unless exception
|
40
39
|
|
41
|
-
if exception
|
42
40
|
error do
|
43
41
|
"ERROR: #{exception}\n #{exception.backtrace}"
|
44
42
|
end
|
45
43
|
end
|
46
|
-
end
|
47
44
|
|
48
|
-
|
45
|
+
private
|
49
46
|
|
50
|
-
|
51
|
-
|
52
|
-
|
47
|
+
def logger
|
48
|
+
GoodJob.logger
|
49
|
+
end
|
53
50
|
|
54
|
-
|
55
|
-
|
51
|
+
def thread_name
|
52
|
+
Thread.current.name || Thread.current.object_id
|
53
|
+
end
|
56
54
|
end
|
57
55
|
end
|
58
56
|
end
|
data/lib/good_job/scheduler.rb
CHANGED
@@ -9,7 +9,7 @@ module GoodJob
|
|
9
9
|
DEFAULT_TIMER_OPTIONS = {
|
10
10
|
execution_interval: 1,
|
11
11
|
timeout_interval: 1,
|
12
|
-
run_now: true
|
12
|
+
run_now: true,
|
13
13
|
}.freeze
|
14
14
|
|
15
15
|
DEFAULT_POOL_OPTIONS = {
|
@@ -19,10 +19,10 @@ module GoodJob
|
|
19
19
|
auto_terminate: true,
|
20
20
|
idletime: 0,
|
21
21
|
max_queue: 0,
|
22
|
-
fallback_policy: :abort # shouldn't matter -- 0 max queue
|
22
|
+
fallback_policy: :abort, # shouldn't matter -- 0 max queue
|
23
23
|
}.freeze
|
24
24
|
|
25
|
-
def initialize(query = GoodJob::Job.all, **
|
25
|
+
def initialize(query = GoodJob::Job.all, **_options)
|
26
26
|
@query = query
|
27
27
|
|
28
28
|
@pool = Concurrent::ThreadPoolExecutor.new(DEFAULT_POOL_OPTIONS)
|
@@ -51,12 +51,17 @@ module GoodJob
|
|
51
51
|
@pool.shutdown
|
52
52
|
@pool.wait_for_termination if wait
|
53
53
|
end
|
54
|
+
|
55
|
+
true
|
54
56
|
end
|
55
57
|
|
56
58
|
def create_thread
|
57
59
|
future = Concurrent::Future.new(args: [ordered_query], executor: @pool) do |query|
|
58
60
|
Rails.application.executor.wrap do
|
59
|
-
|
61
|
+
loop do
|
62
|
+
good_job = query.with_advisory_lock.first
|
63
|
+
break unless good_job
|
64
|
+
|
60
65
|
ActiveSupport::Notifications.instrument("job_started.good_job", { good_job: good_job })
|
61
66
|
|
62
67
|
JobWrapper.new(good_job).perform
|
@@ -64,6 +69,7 @@ module GoodJob
|
|
64
69
|
good_job.advisory_unlock
|
65
70
|
end
|
66
71
|
end
|
72
|
+
|
67
73
|
true
|
68
74
|
end
|
69
75
|
future.add_observer(TaskObserver.new)
|
data/lib/good_job/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
4
|
+
version: 0.2.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
|
-
bindir:
|
9
|
+
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-03-
|
11
|
+
date: 2020-03-07 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: concurrent-ruby
|
@@ -16,37 +16,65 @@ dependencies:
|
|
16
16
|
requirements:
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version:
|
19
|
+
version: 1.0.2
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version:
|
26
|
+
version: 1.0.2
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: pg
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - ">="
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: 1.0.0
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - ">="
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: 1.0.0
|
27
41
|
- !ruby/object:Gem::Dependency
|
28
42
|
name: rails
|
29
43
|
requirement: !ruby/object:Gem::Requirement
|
30
44
|
requirements:
|
31
45
|
- - ">="
|
32
46
|
- !ruby/object:Gem::Version
|
33
|
-
version:
|
47
|
+
version: 5.1.0
|
34
48
|
type: :runtime
|
35
49
|
prerelease: false
|
36
50
|
version_requirements: !ruby/object:Gem::Requirement
|
37
51
|
requirements:
|
38
52
|
- - ">="
|
39
53
|
- !ruby/object:Gem::Version
|
40
|
-
version:
|
54
|
+
version: 5.1.0
|
41
55
|
- !ruby/object:Gem::Dependency
|
42
56
|
name: thor
|
43
57
|
requirement: !ruby/object:Gem::Requirement
|
44
58
|
requirements:
|
45
59
|
- - ">="
|
46
60
|
- !ruby/object:Gem::Version
|
47
|
-
version:
|
61
|
+
version: 0.14.1
|
48
62
|
type: :runtime
|
49
63
|
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - ">="
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: 0.14.1
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: appraisal
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - ">="
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '0'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
50
78
|
version_requirements: !ruby/object:Gem::Requirement
|
51
79
|
requirements:
|
52
80
|
- - ">="
|
@@ -95,7 +123,7 @@ dependencies:
|
|
95
123
|
- !ruby/object:Gem::Version
|
96
124
|
version: '0'
|
97
125
|
- !ruby/object:Gem::Dependency
|
98
|
-
name:
|
126
|
+
name: rspec-rails
|
99
127
|
requirement: !ruby/object:Gem::Requirement
|
100
128
|
requirements:
|
101
129
|
- - ">="
|
@@ -109,7 +137,21 @@ dependencies:
|
|
109
137
|
- !ruby/object:Gem::Version
|
110
138
|
version: '0'
|
111
139
|
- !ruby/object:Gem::Dependency
|
112
|
-
name:
|
140
|
+
name: rubocop
|
141
|
+
requirement: !ruby/object:Gem::Requirement
|
142
|
+
requirements:
|
143
|
+
- - ">="
|
144
|
+
- !ruby/object:Gem::Version
|
145
|
+
version: '0'
|
146
|
+
type: :development
|
147
|
+
prerelease: false
|
148
|
+
version_requirements: !ruby/object:Gem::Requirement
|
149
|
+
requirements:
|
150
|
+
- - ">="
|
151
|
+
- !ruby/object:Gem::Version
|
152
|
+
version: '0'
|
153
|
+
- !ruby/object:Gem::Dependency
|
154
|
+
name: rubocop-rspec
|
113
155
|
requirement: !ruby/object:Gem::Requirement
|
114
156
|
requirements:
|
115
157
|
- - ">="
|
@@ -122,17 +164,21 @@ dependencies:
|
|
122
164
|
- - ">="
|
123
165
|
- !ruby/object:Gem::Version
|
124
166
|
version: '0'
|
125
|
-
description:
|
167
|
+
description: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails
|
126
168
|
email:
|
127
169
|
- bensheldon@gmail.com
|
128
170
|
executables:
|
129
171
|
- good_job
|
130
172
|
extensions: []
|
131
|
-
extra_rdoc_files:
|
173
|
+
extra_rdoc_files:
|
174
|
+
- README.md
|
175
|
+
- CHANGELOG.md
|
176
|
+
- LICENSE.txt
|
132
177
|
files:
|
178
|
+
- CHANGELOG.md
|
133
179
|
- LICENSE.txt
|
134
180
|
- README.md
|
135
|
-
-
|
181
|
+
- exe/good_job
|
136
182
|
- lib/good_job.rb
|
137
183
|
- lib/good_job/adapter.rb
|
138
184
|
- lib/good_job/cli.rb
|
@@ -147,16 +193,28 @@ files:
|
|
147
193
|
homepage: https://github.com/benheldon/good_job
|
148
194
|
licenses:
|
149
195
|
- MIT
|
150
|
-
metadata:
|
196
|
+
metadata:
|
197
|
+
bug_tracker_uri: https://github.com/bensheldon/good_job/issues
|
198
|
+
changelog_uri: https://github.com/bensheldon/good_job/blob/master/CHANGELOG.md
|
199
|
+
documentation_uri: https://rdoc.info/github/bensheldon/good_job
|
200
|
+
homepage_uri: https://github.com/benheldon/good_job
|
201
|
+
source_code_uri: https://github.com/bensheldon/good_job
|
151
202
|
post_install_message:
|
152
|
-
rdoc_options:
|
203
|
+
rdoc_options:
|
204
|
+
- "--title"
|
205
|
+
- GoodJob - a multithreaded, Postgres-based ActiveJob backend for Ruby on Rails
|
206
|
+
- "--main"
|
207
|
+
- README.md
|
208
|
+
- "--line-numbers"
|
209
|
+
- "--inline-source"
|
210
|
+
- "--quiet"
|
153
211
|
require_paths:
|
154
212
|
- lib
|
155
213
|
required_ruby_version: !ruby/object:Gem::Requirement
|
156
214
|
requirements:
|
157
215
|
- - ">="
|
158
216
|
- !ruby/object:Gem::Version
|
159
|
-
version:
|
217
|
+
version: 2.4.0
|
160
218
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
161
219
|
requirements:
|
162
220
|
- - ">="
|
@@ -166,5 +224,5 @@ requirements: []
|
|
166
224
|
rubygems_version: 3.1.2
|
167
225
|
signing_key:
|
168
226
|
specification_version: 4
|
169
|
-
summary:
|
227
|
+
summary: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails
|
170
228
|
test_files: []
|