solid_queue 0.2.0 → 0.2.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/MIT-LICENSE +1 -1
- data/README.md +4 -4
- data/app/models/solid_queue/blocked_execution.rb +2 -2
- data/app/models/solid_queue/claimed_execution.rb +13 -1
- data/app/models/solid_queue/execution/dispatching.rb +20 -0
- data/app/models/solid_queue/execution/job_attributes.rb +4 -4
- data/app/models/solid_queue/execution.rb +44 -2
- data/app/models/solid_queue/failed_execution.rb +26 -16
- data/app/models/solid_queue/job/clearable.rb +4 -1
- data/app/models/solid_queue/job/concurrency_controls.rb +17 -1
- data/app/models/solid_queue/job/executable.rb +20 -9
- data/app/models/solid_queue/job/schedulable.rb +9 -1
- data/app/models/solid_queue/job.rb +3 -0
- data/app/models/solid_queue/queue.rb +1 -1
- data/app/models/solid_queue/ready_execution.rb +10 -0
- data/app/models/solid_queue/scheduled_execution.rb +4 -12
- data/app/models/solid_queue/semaphore.rb +58 -46
- data/db/migrate/20240110143450_add_missing_index_to_blocked_executions.rb +5 -0
- data/lib/generators/solid_queue/install/USAGE +2 -2
- data/lib/generators/solid_queue/install/templates/config.yml +1 -1
- data/lib/solid_queue/processes/registrable.rb +1 -1
- data/lib/solid_queue/processes/runnable.rb +2 -2
- data/lib/solid_queue/version.rb +1 -1
- data/lib/solid_queue/worker.rb +1 -1
- data/lib/solid_queue.rb +1 -1
- metadata +7 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: d907fb9133f2c72a61b586038f05b248a29e7c55f506aa541ca39f35a15d40ff
|
4
|
+
data.tar.gz: 887ecd7d0a15159ae10845466993699165a531367dbb21151e3ae704e56fa8e9
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 665bb353c9cc8c557952ca5e2f63120b0541eef7a20008b6c288147e0350564200be2e6272f9924448867b9b2d09a7e48dc5b2267ec35b91f5dcd38bc409fff1
|
7
|
+
data.tar.gz: 9e911e1a270b5da57f75011e40ad6949480a8b87a2eb391934697159701273e1f05da9ac5c114e8d9dac492bb7685f6884eacac592ed44a460f513fd7349f85a
|
data/MIT-LICENSE
CHANGED
data/README.md
CHANGED
@@ -31,7 +31,7 @@ $ bin/rails generate solid_queue:install
|
|
31
31
|
|
32
32
|
This will set `solid_queue` as the Active Job's adapter in production, and will copy the required migration over to your app.
|
33
33
|
|
34
|
-
Alternatively, you can add
|
34
|
+
Alternatively, you can add only the migration to your app:
|
35
35
|
```bash
|
36
36
|
$ bin/rails solid_queue:install:migrations
|
37
37
|
```
|
@@ -142,6 +142,7 @@ When receiving a `QUIT` signal, if workers still have jobs in-flight, these will
|
|
142
142
|
If processes have no chance of cleaning up before exiting (e.g. if someone pulls a cable somewhere), in-flight jobs might remain claimed by the processes executing them. Processes send heartbeats, and the supervisor checks and prunes processes with expired heartbeats, which will release any claimed jobs back to their queues. You can configure both the frequency of heartbeats and the threshold to consider a process dead. See the section below for this.
|
143
143
|
|
144
144
|
### Other configuration settings
|
145
|
+
_Note_: The settings in this section should be set in your `config/application.rb` or your environment config like this: `config.solid_queue.silence_polling = true`
|
145
146
|
|
146
147
|
There are several settings that control how Solid Queue works that you can set as well:
|
147
148
|
- `logger`: the logger you want Solid Queue to use. Defaults to the app logger.
|
@@ -161,7 +162,7 @@ There are several settings that control how Solid Queue works that you can set a
|
|
161
162
|
- `process_heartbeat_interval`: the heartbeat interval that all processes will follow—defaults to 60 seconds.
|
162
163
|
- `process_alive_threshold`: how long to wait until a process is considered dead after its last heartbeat—defaults to 5 minutes.
|
163
164
|
- `shutdown_timeout`: time the supervisor will wait since it sent the `TERM` signal to its supervised processes before sending a `QUIT` version to them requesting immediate termination—defaults to 5 seconds.
|
164
|
-
- `silence_polling`: whether to silence Active Record logs emitted when polling for both workers and dispatchers—defaults to `
|
165
|
+
- `silence_polling`: whether to silence Active Record logs emitted when polling for both workers and dispatchers—defaults to `true`.
|
165
166
|
- `supervisor_pidfile`: path to a pidfile that the supervisor will create when booting to prevent running more than one supervisor in the same host, or in case you want to use it for a health check. It's `nil` by default.
|
166
167
|
- `preserve_finished_jobs`: whether to keep finished jobs in the `solid_queue_jobs` table—defaults to `true`.
|
167
168
|
- `clear_finished_jobs_after`: period to keep finished jobs around, in case `preserve_finished_jobs` is true—defaults to 1 day. **Note:** Right now, there's no automatic cleanup of finished jobs. You'd need to do this by periodically invoking `SolidQueue::Job.clear_finished_in_batches`, but this will happen automatically in the near future.
|
@@ -228,8 +229,7 @@ failed_execution.retry # This will re-enqueue the job as if it was enqueued for
|
|
228
229
|
failed_execution.discard # This will delete the job from the system
|
229
230
|
```
|
230
231
|
|
231
|
-
|
232
|
-
|
232
|
+
However, we recommend taking a look at [mission_control-jobs](https://github.com/basecamp/mission_control-jobs), a dashboard where, among other things, you can examine and retry/discard failed jobs.
|
233
233
|
|
234
234
|
## Puma plugin
|
235
235
|
We provide a Puma plugin if you want to run the Solid Queue's supervisor together with Puma and have Puma monitor and manage it. You just need to add
|
@@ -30,10 +30,10 @@ module SolidQueue
|
|
30
30
|
|
31
31
|
private
|
32
32
|
def releasable(concurrency_keys)
|
33
|
-
semaphores = Semaphore.where(key: concurrency_keys).
|
33
|
+
semaphores = Semaphore.where(key: concurrency_keys).pluck(:key, :value).to_h
|
34
34
|
|
35
35
|
# Concurrency keys without semaphore + concurrency keys with open semaphore
|
36
|
-
(concurrency_keys - semaphores.keys) | semaphores.select { |
|
36
|
+
(concurrency_keys - semaphores.keys) | semaphores.select { |_key, value| value > 0 }.keys
|
37
37
|
end
|
38
38
|
end
|
39
39
|
|
@@ -23,6 +23,14 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
|
|
23
23
|
def release_all
|
24
24
|
includes(:job).each(&:release)
|
25
25
|
end
|
26
|
+
|
27
|
+
def discard_all_in_batches(*)
|
28
|
+
raise UndiscardableError, "Can't discard jobs in progress"
|
29
|
+
end
|
30
|
+
|
31
|
+
def discard_all_from_jobs(*)
|
32
|
+
raise UndiscardableError, "Can't discard jobs in progress"
|
33
|
+
end
|
26
34
|
end
|
27
35
|
|
28
36
|
def perform
|
@@ -39,11 +47,15 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
|
|
39
47
|
|
40
48
|
def release
|
41
49
|
transaction do
|
42
|
-
job.
|
50
|
+
job.dispatch_bypassing_concurrency_limits
|
43
51
|
destroy!
|
44
52
|
end
|
45
53
|
end
|
46
54
|
|
55
|
+
def discard
|
56
|
+
raise UndiscardableError, "Can't discard a job in progress"
|
57
|
+
end
|
58
|
+
|
47
59
|
private
|
48
60
|
def execute
|
49
61
|
ActiveJob::Base.execute(job.arguments)
|
@@ -0,0 +1,20 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module SolidQueue
|
4
|
+
class Execution
|
5
|
+
module Dispatching
|
6
|
+
extend ActiveSupport::Concern
|
7
|
+
|
8
|
+
class_methods do
|
9
|
+
def dispatch_jobs(job_ids)
|
10
|
+
jobs = Job.where(id: job_ids)
|
11
|
+
|
12
|
+
Job.dispatch_all(jobs).map(&:id).tap do |dispatched_job_ids|
|
13
|
+
where(job_id: dispatched_job_ids).order(:job_id).delete_all
|
14
|
+
SolidQueue.logger.info("[SolidQueue] Dispatched #{dispatched_job_ids.size} jobs")
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
20
|
+
end
|
@@ -6,23 +6,23 @@ module SolidQueue
|
|
6
6
|
extend ActiveSupport::Concern
|
7
7
|
|
8
8
|
included do
|
9
|
-
class_attribute :
|
9
|
+
class_attribute :assumable_attributes_from_job, instance_accessor: false, default: %i[ queue_name priority ]
|
10
10
|
end
|
11
11
|
|
12
12
|
class_methods do
|
13
13
|
def assumes_attributes_from_job(*attribute_names)
|
14
|
-
self.
|
14
|
+
self.assumable_attributes_from_job |= attribute_names
|
15
15
|
before_create -> { assume_attributes_from_job }
|
16
16
|
end
|
17
17
|
|
18
18
|
def attributes_from_job(job)
|
19
|
-
job.attributes.symbolize_keys.slice(*
|
19
|
+
job.attributes.symbolize_keys.slice(*assumable_attributes_from_job)
|
20
20
|
end
|
21
21
|
end
|
22
22
|
|
23
23
|
private
|
24
24
|
def assume_attributes_from_job
|
25
|
-
self.class.
|
25
|
+
self.class.assumable_attributes_from_job.each do |attribute|
|
26
26
|
send("#{attribute}=", job.send(attribute))
|
27
27
|
end
|
28
28
|
end
|
@@ -2,6 +2,8 @@
|
|
2
2
|
|
3
3
|
module SolidQueue
|
4
4
|
class Execution < Record
|
5
|
+
class UndiscardableError < StandardError; end
|
6
|
+
|
5
7
|
include JobAttributes
|
6
8
|
|
7
9
|
self.abstract_class = true
|
@@ -10,8 +12,6 @@ module SolidQueue
|
|
10
12
|
|
11
13
|
belongs_to :job
|
12
14
|
|
13
|
-
alias_method :discard, :destroy
|
14
|
-
|
15
15
|
class << self
|
16
16
|
def create_all_from_jobs(jobs)
|
17
17
|
insert_all execution_data_from_jobs(jobs)
|
@@ -22,6 +22,48 @@ module SolidQueue
|
|
22
22
|
attributes_from_job(job).merge(job_id: job.id)
|
23
23
|
end
|
24
24
|
end
|
25
|
+
|
26
|
+
def discard_all_in_batches(batch_size: 500)
|
27
|
+
pending = count
|
28
|
+
discarded = 0
|
29
|
+
|
30
|
+
loop do
|
31
|
+
transaction do
|
32
|
+
job_ids = limit(batch_size).order(:job_id).lock.pluck(:job_id)
|
33
|
+
|
34
|
+
discard_jobs job_ids
|
35
|
+
discarded = where(job_id: job_ids).delete_all
|
36
|
+
pending -= discarded
|
37
|
+
end
|
38
|
+
|
39
|
+
break if pending <= 0 || discarded == 0
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
def discard_all_from_jobs(jobs)
|
44
|
+
transaction do
|
45
|
+
job_ids = lock_all_from_jobs(jobs)
|
46
|
+
|
47
|
+
discard_jobs job_ids
|
48
|
+
where(job_id: job_ids).delete_all
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
private
|
53
|
+
def lock_all_from_jobs(jobs)
|
54
|
+
where(job_id: jobs.map(&:id)).order(:job_id).lock.pluck(:job_id)
|
55
|
+
end
|
56
|
+
|
57
|
+
def discard_jobs(job_ids)
|
58
|
+
Job.where(id: job_ids).delete_all
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
def discard
|
63
|
+
with_lock do
|
64
|
+
job.destroy
|
65
|
+
destroy
|
66
|
+
end
|
25
67
|
end
|
26
68
|
end
|
27
69
|
end
|
@@ -1,27 +1,37 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
|
4
|
-
|
3
|
+
module SolidQueue
|
4
|
+
class FailedExecution < Execution
|
5
|
+
include Dispatching
|
5
6
|
|
6
|
-
|
7
|
+
serialize :error, coder: JSON
|
7
8
|
|
8
|
-
|
9
|
+
before_create :expand_error_details_from_exception
|
9
10
|
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
11
|
+
attr_accessor :exception
|
12
|
+
|
13
|
+
def self.retry_all(jobs)
|
14
|
+
transaction do
|
15
|
+
dispatch_jobs lock_all_from_jobs(jobs)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
def retry
|
20
|
+
with_lock do
|
21
|
+
job.prepare_for_execution
|
22
|
+
destroy!
|
23
|
+
end
|
14
24
|
end
|
15
|
-
end
|
16
25
|
|
17
|
-
|
18
|
-
|
19
|
-
|
26
|
+
%i[ exception_class message backtrace ].each do |attribute|
|
27
|
+
define_method(attribute) { error.with_indifferent_access[attribute] }
|
28
|
+
end
|
20
29
|
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
30
|
+
private
|
31
|
+
def expand_error_details_from_exception
|
32
|
+
if exception
|
33
|
+
self.error = { exception_class: exception.class.name, message: exception.message, backtrace: exception.backtrace }
|
34
|
+
end
|
25
35
|
end
|
26
36
|
end
|
27
37
|
end
|
@@ -11,7 +11,10 @@ module SolidQueue
|
|
11
11
|
|
12
12
|
class_methods do
|
13
13
|
def clear_finished_in_batches(batch_size: 500, finished_before: SolidQueue.clear_finished_jobs_after.ago)
|
14
|
-
|
14
|
+
loop do
|
15
|
+
records_deleted = clearable(finished_before: finished_before).limit(batch_size).delete_all
|
16
|
+
break if records_deleted == 0
|
17
|
+
end
|
15
18
|
end
|
16
19
|
end
|
17
20
|
end
|
@@ -6,9 +6,17 @@ module SolidQueue
|
|
6
6
|
extend ActiveSupport::Concern
|
7
7
|
|
8
8
|
included do
|
9
|
-
has_one :blocked_execution
|
9
|
+
has_one :blocked_execution
|
10
10
|
|
11
11
|
delegate :concurrency_limit, :concurrency_duration, to: :job_class
|
12
|
+
|
13
|
+
before_destroy :unblock_next_blocked_job, if: -> { concurrency_limited? && ready? }
|
14
|
+
end
|
15
|
+
|
16
|
+
class_methods do
|
17
|
+
def release_all_concurrency_locks(jobs)
|
18
|
+
Semaphore.signal_all(jobs.select(&:concurrency_limited?))
|
19
|
+
end
|
12
20
|
end
|
13
21
|
|
14
22
|
def unblock_next_blocked_job
|
@@ -21,6 +29,10 @@ module SolidQueue
|
|
21
29
|
concurrency_key.present?
|
22
30
|
end
|
23
31
|
|
32
|
+
def blocked?
|
33
|
+
blocked_execution.present?
|
34
|
+
end
|
35
|
+
|
24
36
|
private
|
25
37
|
def acquire_concurrency_lock
|
26
38
|
return true unless concurrency_limited?
|
@@ -45,6 +57,10 @@ module SolidQueue
|
|
45
57
|
def job_class
|
46
58
|
@job_class ||= class_name.safe_constantize
|
47
59
|
end
|
60
|
+
|
61
|
+
def execution
|
62
|
+
super || blocked_execution
|
63
|
+
end
|
48
64
|
end
|
49
65
|
end
|
50
66
|
end
|
@@ -8,9 +8,9 @@ module SolidQueue
|
|
8
8
|
included do
|
9
9
|
include Clearable, ConcurrencyControls, Schedulable
|
10
10
|
|
11
|
-
has_one :ready_execution
|
12
|
-
has_one :claimed_execution
|
13
|
-
has_one :failed_execution
|
11
|
+
has_one :ready_execution
|
12
|
+
has_one :claimed_execution
|
13
|
+
has_one :failed_execution
|
14
14
|
|
15
15
|
after_create :prepare_for_execution
|
16
16
|
|
@@ -73,6 +73,10 @@ module SolidQueue
|
|
73
73
|
end
|
74
74
|
end
|
75
75
|
|
76
|
+
def dispatch_bypassing_concurrency_limits
|
77
|
+
ready
|
78
|
+
end
|
79
|
+
|
76
80
|
def finished!
|
77
81
|
if preserve_finished_jobs?
|
78
82
|
touch(:finished_at)
|
@@ -85,18 +89,22 @@ module SolidQueue
|
|
85
89
|
finished_at.present?
|
86
90
|
end
|
87
91
|
|
88
|
-
def
|
89
|
-
|
90
|
-
|
91
|
-
|
92
|
-
|
93
|
-
|
92
|
+
def status
|
93
|
+
if finished?
|
94
|
+
:finished
|
95
|
+
elsif execution.present?
|
96
|
+
execution.model_name.element.sub("_execution", "").to_sym
|
97
|
+
end
|
94
98
|
end
|
95
99
|
|
96
100
|
def retry
|
97
101
|
failed_execution&.retry
|
98
102
|
end
|
99
103
|
|
104
|
+
def discard
|
105
|
+
execution&.discard
|
106
|
+
end
|
107
|
+
|
100
108
|
def failed_with(exception)
|
101
109
|
FailedExecution.create_or_find_by!(job_id: id, exception: exception)
|
102
110
|
end
|
@@ -106,6 +114,9 @@ module SolidQueue
|
|
106
114
|
ReadyExecution.create_or_find_by!(job_id: id)
|
107
115
|
end
|
108
116
|
|
117
|
+
def execution
|
118
|
+
%w[ ready claimed failed ].reduce(nil) { |acc, status| acc || public_send("#{status}_execution") }
|
119
|
+
end
|
109
120
|
|
110
121
|
def preserve_finished_jobs?
|
111
122
|
SolidQueue.preserve_finished_jobs
|
@@ -6,7 +6,7 @@ module SolidQueue
|
|
6
6
|
extend ActiveSupport::Concern
|
7
7
|
|
8
8
|
included do
|
9
|
-
has_one :scheduled_execution
|
9
|
+
has_one :scheduled_execution
|
10
10
|
|
11
11
|
scope :scheduled, -> { where.not(finished_at: nil) }
|
12
12
|
end
|
@@ -31,10 +31,18 @@ module SolidQueue
|
|
31
31
|
scheduled_at.nil? || scheduled_at <= Time.current
|
32
32
|
end
|
33
33
|
|
34
|
+
def scheduled?
|
35
|
+
scheduled_execution.present?
|
36
|
+
end
|
37
|
+
|
34
38
|
private
|
35
39
|
def schedule
|
36
40
|
ScheduledExecution.create_or_find_by!(job_id: id)
|
37
41
|
end
|
42
|
+
|
43
|
+
def execution
|
44
|
+
super || scheduled_execution
|
45
|
+
end
|
38
46
|
end
|
39
47
|
end
|
40
48
|
end
|
@@ -15,9 +15,12 @@ module SolidQueue
|
|
15
15
|
prepare_all_for_execution(jobs).tap do |enqueued_jobs|
|
16
16
|
enqueued_jobs.each do |enqueued_job|
|
17
17
|
active_jobs_by_job_id[enqueued_job.active_job_id].provider_job_id = enqueued_job.id
|
18
|
+
active_jobs_by_job_id[enqueued_job.active_job_id].successfully_enqueued = true
|
18
19
|
end
|
19
20
|
end
|
20
21
|
end
|
22
|
+
|
23
|
+
active_jobs.count(&:successfully_enqueued?)
|
21
24
|
end
|
22
25
|
|
23
26
|
def enqueue(active_job, scheduled_at: Time.current)
|
@@ -15,6 +15,10 @@ module SolidQueue
|
|
15
15
|
end
|
16
16
|
end
|
17
17
|
|
18
|
+
def aggregated_count_across(queue_list)
|
19
|
+
QueueSelector.new(queue_list, self).scoped_relations.map(&:count).sum
|
20
|
+
end
|
21
|
+
|
18
22
|
private
|
19
23
|
def select_and_lock(queue_relation, process_id, limit)
|
20
24
|
return [] if limit <= 0
|
@@ -36,6 +40,12 @@ module SolidQueue
|
|
36
40
|
where(job_id: claimed.pluck(:job_id)).delete_all
|
37
41
|
end
|
38
42
|
end
|
43
|
+
|
44
|
+
|
45
|
+
def discard_jobs(job_ids)
|
46
|
+
Job.release_all_concurrency_locks Job.where(id: job_ids)
|
47
|
+
super
|
48
|
+
end
|
39
49
|
end
|
40
50
|
end
|
41
51
|
end
|
@@ -2,8 +2,10 @@
|
|
2
2
|
|
3
3
|
module SolidQueue
|
4
4
|
class ScheduledExecution < Execution
|
5
|
+
include Dispatching
|
6
|
+
|
5
7
|
scope :due, -> { where(scheduled_at: ..Time.current) }
|
6
|
-
scope :ordered, -> { order(scheduled_at: :asc, priority: :asc) }
|
8
|
+
scope :ordered, -> { order(scheduled_at: :asc, priority: :asc, job_id: :asc) }
|
7
9
|
scope :next_batch, ->(batch_size) { due.ordered.limit(batch_size) }
|
8
10
|
|
9
11
|
assumes_attributes_from_job :scheduled_at
|
@@ -14,20 +16,10 @@ module SolidQueue
|
|
14
16
|
job_ids = next_batch(batch_size).non_blocking_lock.pluck(:job_id)
|
15
17
|
if job_ids.empty? then []
|
16
18
|
else
|
17
|
-
|
19
|
+
dispatch_jobs(job_ids)
|
18
20
|
end
|
19
21
|
end
|
20
22
|
end
|
21
|
-
|
22
|
-
private
|
23
|
-
def dispatch_batch(job_ids)
|
24
|
-
jobs = Job.where(id: job_ids)
|
25
|
-
|
26
|
-
Job.dispatch_all(jobs).map(&:id).tap do |dispatched_job_ids|
|
27
|
-
where(job_id: dispatched_job_ids).delete_all
|
28
|
-
SolidQueue.logger.info("[SolidQueue] Dispatched scheduled batch with #{dispatched_job_ids.size} jobs")
|
29
|
-
end
|
30
|
-
end
|
31
23
|
end
|
32
24
|
end
|
33
25
|
end
|
@@ -1,65 +1,77 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
|
4
|
-
|
5
|
-
|
3
|
+
module SolidQueue
|
4
|
+
class Semaphore < Record
|
5
|
+
scope :available, -> { where("value > 0") }
|
6
|
+
scope :expired, -> { where(expires_at: ...Time.current) }
|
6
7
|
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
def signal(job)
|
13
|
-
Proxy.new(job, self).signal
|
14
|
-
end
|
15
|
-
end
|
16
|
-
|
17
|
-
class Proxy
|
18
|
-
def initialize(job, proxied_class)
|
19
|
-
@job = job
|
20
|
-
@proxied_class = proxied_class
|
21
|
-
end
|
8
|
+
class << self
|
9
|
+
def wait(job)
|
10
|
+
Proxy.new(job).wait
|
11
|
+
end
|
22
12
|
|
23
|
-
|
24
|
-
|
25
|
-
semaphore.value > 0 && attempt_decrement
|
26
|
-
else
|
27
|
-
attempt_creation
|
13
|
+
def signal(job)
|
14
|
+
Proxy.new(job).signal
|
28
15
|
end
|
29
|
-
end
|
30
16
|
|
31
|
-
|
32
|
-
|
17
|
+
def signal_all(jobs)
|
18
|
+
Proxy.signal_all(jobs)
|
19
|
+
end
|
33
20
|
end
|
34
21
|
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
def attempt_creation
|
39
|
-
proxied_class.create!(key: key, value: limit - 1, expires_at: expires_at)
|
40
|
-
true
|
41
|
-
rescue ActiveRecord::RecordNotUnique
|
42
|
-
attempt_decrement
|
22
|
+
class Proxy
|
23
|
+
def self.signal_all(jobs)
|
24
|
+
Semaphore.where(key: jobs.map(&:concurrency_key)).update_all("value = value + 1")
|
43
25
|
end
|
44
26
|
|
45
|
-
def
|
46
|
-
|
27
|
+
def initialize(job)
|
28
|
+
@job = job
|
47
29
|
end
|
48
30
|
|
49
|
-
def
|
50
|
-
|
31
|
+
def wait
|
32
|
+
if semaphore = Semaphore.find_by(key: key)
|
33
|
+
semaphore.value > 0 && attempt_decrement
|
34
|
+
else
|
35
|
+
attempt_creation
|
36
|
+
end
|
51
37
|
end
|
52
38
|
|
53
|
-
def
|
54
|
-
|
39
|
+
def signal
|
40
|
+
attempt_increment
|
55
41
|
end
|
56
42
|
|
57
|
-
|
58
|
-
job
|
59
|
-
end
|
43
|
+
private
|
44
|
+
attr_accessor :job
|
60
45
|
|
61
|
-
|
62
|
-
|
63
|
-
|
46
|
+
def attempt_creation
|
47
|
+
Semaphore.create!(key: key, value: limit - 1, expires_at: expires_at)
|
48
|
+
true
|
49
|
+
rescue ActiveRecord::RecordNotUnique
|
50
|
+
if limit == 1 then false
|
51
|
+
else
|
52
|
+
attempt_decrement
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
def attempt_decrement
|
57
|
+
Semaphore.available.where(key: key).update_all([ "value = value - 1, expires_at = ?", expires_at ]) > 0
|
58
|
+
end
|
59
|
+
|
60
|
+
def attempt_increment
|
61
|
+
Semaphore.where(key: key, value: ...limit).update_all([ "value = value + 1, expires_at = ?", expires_at ]) > 0
|
62
|
+
end
|
63
|
+
|
64
|
+
def key
|
65
|
+
job.concurrency_key
|
66
|
+
end
|
67
|
+
|
68
|
+
def expires_at
|
69
|
+
job.concurrency_duration.from_now
|
70
|
+
end
|
71
|
+
|
72
|
+
def limit
|
73
|
+
job.concurrency_limit
|
74
|
+
end
|
75
|
+
end
|
64
76
|
end
|
65
77
|
end
|
@@ -4,6 +4,6 @@ Description:
|
|
4
4
|
Example:
|
5
5
|
bin/rails generate solid_queue:install
|
6
6
|
|
7
|
-
This will
|
7
|
+
This will perform the following:
|
8
8
|
Installs solid_queue migrations
|
9
|
-
Replaces Active Job's adapter in
|
9
|
+
Replaces Active Job's adapter in environment configuration
|
@@ -4,6 +4,8 @@ module SolidQueue::Processes
|
|
4
4
|
module Runnable
|
5
5
|
include Supervised
|
6
6
|
|
7
|
+
attr_writer :mode
|
8
|
+
|
7
9
|
def start
|
8
10
|
@stopping = false
|
9
11
|
|
@@ -19,8 +21,6 @@ module SolidQueue::Processes
|
|
19
21
|
end
|
20
22
|
|
21
23
|
private
|
22
|
-
attr_writer :mode
|
23
|
-
|
24
24
|
DEFAULT_MODE = :async
|
25
25
|
|
26
26
|
def mode
|
data/lib/solid_queue/version.rb
CHANGED
data/lib/solid_queue/worker.rb
CHANGED
data/lib/solid_queue.rb
CHANGED
@@ -33,7 +33,7 @@ module SolidQueue
|
|
33
33
|
|
34
34
|
mattr_accessor :shutdown_timeout, default: 5.seconds
|
35
35
|
|
36
|
-
mattr_accessor :silence_polling, default:
|
36
|
+
mattr_accessor :silence_polling, default: true
|
37
37
|
|
38
38
|
mattr_accessor :supervisor_pidfile
|
39
39
|
mattr_accessor :supervisor, default: false
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: solid_queue
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
4
|
+
version: 0.2.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Rosa Gutierrez
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2024-03-05 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rails
|
@@ -79,6 +79,7 @@ files:
|
|
79
79
|
- app/models/solid_queue/blocked_execution.rb
|
80
80
|
- app/models/solid_queue/claimed_execution.rb
|
81
81
|
- app/models/solid_queue/execution.rb
|
82
|
+
- app/models/solid_queue/execution/dispatching.rb
|
82
83
|
- app/models/solid_queue/execution/job_attributes.rb
|
83
84
|
- app/models/solid_queue/failed_execution.rb
|
84
85
|
- app/models/solid_queue/job.rb
|
@@ -97,6 +98,7 @@ files:
|
|
97
98
|
- app/models/solid_queue/semaphore.rb
|
98
99
|
- config/routes.rb
|
99
100
|
- db/migrate/20231211200639_create_solid_queue_tables.rb
|
101
|
+
- db/migrate/20240110143450_add_missing_index_to_blocked_executions.rb
|
100
102
|
- lib/active_job/concurrency_controls.rb
|
101
103
|
- lib/active_job/queue_adapters/solid_queue_adapter.rb
|
102
104
|
- lib/generators/solid_queue/install/USAGE
|
@@ -122,12 +124,12 @@ files:
|
|
122
124
|
- lib/solid_queue/tasks.rb
|
123
125
|
- lib/solid_queue/version.rb
|
124
126
|
- lib/solid_queue/worker.rb
|
125
|
-
homepage:
|
127
|
+
homepage: https://github.com/basecamp/solid_queue
|
126
128
|
licenses:
|
127
129
|
- MIT
|
128
130
|
metadata:
|
129
|
-
homepage_uri:
|
130
|
-
source_code_uri:
|
131
|
+
homepage_uri: https://github.com/basecamp/solid_queue
|
132
|
+
source_code_uri: https://github.com/basecamp/solid_queue
|
131
133
|
post_install_message:
|
132
134
|
rdoc_options: []
|
133
135
|
require_paths:
|