philiprehberger-task_queue 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -0
- data/README.md +23 -3
- data/lib/philiprehberger/task_queue/queue.rb +39 -3
- data/lib/philiprehberger/task_queue/version.rb +1 -1
- data/lib/philiprehberger/task_queue.rb +4 -0
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 3b052683a824db4cfaab12c8216cb66a99d40580d085ae630a3dfda88879b82d
|
|
4
|
+
data.tar.gz: de60b9e402664d5688461aa1e5659a3a686646de142ce2123728c61f5da7ff35
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 76e39073b3e7a723836810427fc10a7e7777831e4f5473ecd5cb4cd0a9424eb9b91c4f303d5d9bc8e33f293e8d6342d77d6e8f9f45e75a9bfeedfa6c551779e1
|
|
7
|
+
data.tar.gz: 287f3097f118a996bd0dc0b10edb86c9c904fdf62cfd8d4113b72e1808582309cae98409a520dd0d3edd39e7320f4af5b845848d318a2f0ea11d5be2d78ba2cc
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.9.0] - 2026-10-30
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
- Bounded queue — `max_size:` constructor option caps the number of pending tasks (`nil`, the default, keeps the queue unbounded); the capacity check and enqueue happen atomically under the queue mutex
|
|
14
|
+
- `Philiprehberger::TaskQueue::QueueFullError` (subclass of `Error`) raised by `push`/`<<` when a bounded queue is full
|
|
15
|
+
- `Queue#full?` — whether the pending task count has reached `max_size` (always `false` when unbounded)
|
|
16
|
+
- `Queue#max_size` — reader for the configured limit
|
|
17
|
+
|
|
18
|
+
### Changed
|
|
19
|
+
- `.gitignore` aligned with the gem guide template
|
|
20
|
+
- CI workflow aligned with the gem guide template, publishing via a RubyGems credentials file
|
|
21
|
+
- Feature request issue template now includes a proposed-API code placeholder
|
|
22
|
+
|
|
10
23
|
## [0.8.0] - 2026-07-15
|
|
11
24
|
|
|
12
25
|
### Added
|
data/README.md
CHANGED
|
@@ -200,6 +200,23 @@ queue.drain(timeout: 5)
|
|
|
200
200
|
# low
|
|
201
201
|
```
|
|
202
202
|
|
|
203
|
+
### Bounded queue
|
|
204
|
+
|
|
205
|
+
Pass `max_size:` to cap the number of pending (not yet started) tasks. Once the limit is reached, `push` and `<<` raise `Philiprehberger::TaskQueue::QueueFullError` (a subclass of `Philiprehberger::TaskQueue::Error`) instead of growing without bound. The capacity check and the enqueue happen atomically, so concurrent producers can never overfill the queue. Tasks requeued for a retry are not counted against the limit. The default, `nil`, leaves the queue unbounded.
|
|
206
|
+
|
|
207
|
+
```ruby
|
|
208
|
+
queue = Philiprehberger::TaskQueue.new(concurrency: 2, max_size: 100)
|
|
209
|
+
puts queue.max_size # => 100
|
|
210
|
+
|
|
211
|
+
begin
|
|
212
|
+
queue.push { process(job) }
|
|
213
|
+
rescue Philiprehberger::TaskQueue::QueueFullError
|
|
214
|
+
# apply backpressure: drop, retry later, or slow the producer
|
|
215
|
+
end
|
|
216
|
+
|
|
217
|
+
puts queue.full? # => true once 100 tasks are pending
|
|
218
|
+
```
|
|
219
|
+
|
|
203
220
|
### FIFO ordering guarantees
|
|
204
221
|
|
|
205
222
|
Tasks are stored in an internal array and dequeued in priority-then-FIFO order. With the default priority of `0`, ordering is pure FIFO. When `concurrency` is `1`, equal-priority tasks execute strictly in the order they were pushed. With higher concurrency, dequeue order is still priority-then-FIFO but tasks may complete out of order depending on individual execution time.
|
|
@@ -248,9 +265,9 @@ queue.shutdown(timeout: 5)
|
|
|
248
265
|
|
|
249
266
|
| Method | Parameters | Returns | Description |
|
|
250
267
|
|---|---|---|---|
|
|
251
|
-
| `.new(concurrency:, max_retries:, retry_backoff:, retry_base_delay:)` | `concurrency` — max worker threads (Integer, default `4`); `max_retries` — retries before a task counts as failed (Integer, default `0`); `retry_backoff` — `:none`/`:fixed`/`:exponential` (default `:none`); `retry_base_delay` — base delay in seconds (Numeric, default `0.1`) | `Queue` | Create a new queue; raises `ArgumentError` on a negative `max_retries` or
|
|
252
|
-
| `#push(&block)` | `priority` — dequeue priority, higher runs first (Integer, default `0`); `&block` — the task to execute | `self` | Enqueue a block for async execution; raises `ArgumentError` if no block given, raises `RuntimeError` if the queue is shut down |
|
|
253
|
-
| `#<<(callable)` | `callable` — any object responding to `#call` | `self` | Alias for `#push`; convenient for lambdas and procs |
|
|
268
|
+
| `.new(concurrency:, max_retries:, retry_backoff:, retry_base_delay:, max_size:)` | `concurrency` — max worker threads (Integer, default `4`); `max_retries` — retries before a task counts as failed (Integer, default `0`); `retry_backoff` — `:none`/`:fixed`/`:exponential` (default `:none`); `retry_base_delay` — base delay in seconds (Numeric, default `0.1`); `max_size` — max pending tasks (Integer or `nil`, default `nil` for unbounded) | `Queue` | Create a new queue; raises `ArgumentError` on a negative `max_retries`, unknown `retry_backoff`, or a `max_size` that is not a positive Integer |
|
|
269
|
+
| `#push(&block)` | `priority` — dequeue priority, higher runs first (Integer, default `0`); `&block` — the task to execute | `self` | Enqueue a block for async execution; raises `ArgumentError` if no block given, raises `RuntimeError` if the queue is shut down, raises `QueueFullError` if the queue holds `max_size` pending tasks |
|
|
270
|
+
| `#<<(callable)` | `callable` — any object responding to `#call` | `self` | Alias for `#push`; convenient for lambdas and procs; raises `QueueFullError` when full |
|
|
254
271
|
| `#size` | _(none)_ | `Integer` | Number of pending (not yet started) tasks |
|
|
255
272
|
| `#empty?` | _(none)_ | `Boolean` | Whether there are no pending tasks waiting to be started |
|
|
256
273
|
| `#busy?` | _(none)_ | `Boolean` | Whether the queue has any pending tasks or in-flight tasks |
|
|
@@ -266,6 +283,9 @@ queue.shutdown(timeout: 5)
|
|
|
266
283
|
| `#clear` | _(none)_ | `Integer` | Remove all pending tasks and return the number cleared |
|
|
267
284
|
| `#stats_reset!` | _(none)_ | `self` | Atomically zero the `completed` and `failed` counters while leaving pending, in-flight, workers, and callbacks untouched |
|
|
268
285
|
| `#concurrency` | _(none)_ | `Integer` | Returns the configured maximum number of concurrent worker threads |
|
|
286
|
+
| `#max_size` | _(none)_ | `Integer`, `nil` | Returns the configured pending-task limit, or `nil` when unbounded |
|
|
287
|
+
| `#full?` | _(none)_ | `Boolean` | Whether the pending task count has reached `max_size`; always `false` when unbounded |
|
|
288
|
+
| `QueueFullError` | _(class)_ | — | Raised by `#push`/`#<<` on a full bounded queue; subclass of `Philiprehberger::TaskQueue::Error` |
|
|
269
289
|
|
|
270
290
|
## Development
|
|
271
291
|
|
|
@@ -19,6 +19,10 @@ module Philiprehberger
|
|
|
19
19
|
# @return [Integer] the maximum number of concurrent worker threads
|
|
20
20
|
attr_reader :concurrency
|
|
21
21
|
|
|
22
|
+
# @return [Integer, nil] the maximum number of pending tasks, or +nil+
|
|
23
|
+
# when the queue is unbounded
|
|
24
|
+
attr_reader :max_size
|
|
25
|
+
|
|
22
26
|
# @param concurrency [Integer] maximum number of concurrent worker threads
|
|
23
27
|
# @param max_retries [Integer] how many times a failing task is requeued
|
|
24
28
|
# before it is counted as failed (default +0+, i.e. no retries)
|
|
@@ -26,10 +30,14 @@ module Philiprehberger
|
|
|
26
30
|
# +:none+, +:fixed+, or +:exponential+
|
|
27
31
|
# @param retry_base_delay [Numeric] base delay in seconds used by the
|
|
28
32
|
# +:fixed+ and +:exponential+ backoff policies
|
|
29
|
-
# @
|
|
30
|
-
#
|
|
31
|
-
|
|
33
|
+
# @param max_size [Integer, nil] maximum number of pending (not yet
|
|
34
|
+
# started) tasks; +nil+ (the default) leaves the queue unbounded
|
|
35
|
+
# @raise [ArgumentError] if +max_retries+ is negative, +retry_backoff+
|
|
36
|
+
# is not a recognized policy, or +max_size+ is not a positive Integer
|
|
37
|
+
def initialize(concurrency: 4, max_retries: 0, retry_backoff: :none, retry_base_delay: 0.1, max_size: nil)
|
|
32
38
|
@concurrency = concurrency
|
|
39
|
+
@max_size = max_size
|
|
40
|
+
validate_max_size!
|
|
33
41
|
@max_retries = max_retries
|
|
34
42
|
@retry_backoff = retry_backoff
|
|
35
43
|
@retry_base_delay = retry_base_delay
|
|
@@ -166,16 +174,23 @@ module Philiprehberger
|
|
|
166
174
|
# that share a priority run in FIFO (insertion) order. The +<<+ alias
|
|
167
175
|
# always enqueues at the default priority of +0+.
|
|
168
176
|
#
|
|
177
|
+
# When the queue was created with +max_size+, the capacity check and the
|
|
178
|
+
# enqueue happen atomically, so concurrent pushers can never overfill it.
|
|
179
|
+
# Tasks requeued for a retry are not subject to the limit.
|
|
180
|
+
#
|
|
169
181
|
# @param callable [#call, nil] a callable object (used by +<<+)
|
|
170
182
|
# @param priority [Integer] dequeue priority; higher runs first (default +0+)
|
|
171
183
|
# @yield the block to execute (takes precedence over +callable+)
|
|
172
184
|
# @return [self]
|
|
185
|
+
# @raise [ArgumentError] if neither a block nor a callable is given
|
|
186
|
+
# @raise [QueueFullError] if the queue already holds +max_size+ pending tasks
|
|
173
187
|
def push(callable = nil, priority: 0, &block)
|
|
174
188
|
task = block || callable
|
|
175
189
|
raise ArgumentError, 'a block is required' unless task
|
|
176
190
|
|
|
177
191
|
@mutex.synchronize do
|
|
178
192
|
raise 'queue is shut down' unless @running
|
|
193
|
+
raise QueueFullError, "queue is full (max_size: #{@max_size})" if at_capacity?
|
|
179
194
|
|
|
180
195
|
start_workers unless @started
|
|
181
196
|
@tasks << Entry.new(task, priority, 0)
|
|
@@ -203,6 +218,15 @@ module Philiprehberger
|
|
|
203
218
|
@mutex.synchronize { @tasks.empty? }
|
|
204
219
|
end
|
|
205
220
|
|
|
221
|
+
# Whether the pending task count has reached +max_size+.
|
|
222
|
+
#
|
|
223
|
+
# Always +false+ for an unbounded queue.
|
|
224
|
+
#
|
|
225
|
+
# @return [Boolean]
|
|
226
|
+
def full?
|
|
227
|
+
@mutex.synchronize { at_capacity? }
|
|
228
|
+
end
|
|
229
|
+
|
|
206
230
|
# Whether the queue has any pending tasks or any in-flight tasks.
|
|
207
231
|
#
|
|
208
232
|
# Convenient inverse of "idle" for callers polling without going
|
|
@@ -235,6 +259,18 @@ module Philiprehberger
|
|
|
235
259
|
|
|
236
260
|
private
|
|
237
261
|
|
|
262
|
+
# Must be called while holding +@mutex+.
|
|
263
|
+
def at_capacity?
|
|
264
|
+
!@max_size.nil? && @tasks.size >= @max_size
|
|
265
|
+
end
|
|
266
|
+
|
|
267
|
+
def validate_max_size!
|
|
268
|
+
return if @max_size.nil?
|
|
269
|
+
return if @max_size.is_a?(Integer) && @max_size.positive?
|
|
270
|
+
|
|
271
|
+
raise ArgumentError, "max_size must be a positive Integer or nil, got #{@max_size.inspect}"
|
|
272
|
+
end
|
|
273
|
+
|
|
238
274
|
def signal_shutdown
|
|
239
275
|
@mutex.synchronize do
|
|
240
276
|
return unless @running
|
|
@@ -8,6 +8,10 @@ module Philiprehberger
|
|
|
8
8
|
# Base error class for all TaskQueue-specific errors.
|
|
9
9
|
class Error < StandardError; end
|
|
10
10
|
|
|
11
|
+
# Raised by {Queue#push} when a bounded queue already holds +max_size+
|
|
12
|
+
# pending tasks.
|
|
13
|
+
class QueueFullError < Error; end
|
|
14
|
+
|
|
11
15
|
# Convenience constructor.
|
|
12
16
|
#
|
|
13
17
|
# @param options [Hash] forwarded to {Queue#initialize}
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: philiprehberger-task_queue
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.9.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Philip Rehberger
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-10-30 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: A lightweight, zero-dependency, thread-safe in-process async job queue
|
|
14
14
|
with configurable concurrency for Ruby applications.
|