async-background 1.0.2 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 233f46d3617b40d9a8a27f83689050b0127603679342ddfa80a18eea0137651c
-  data.tar.gz: 5d70fe357702dff97b51734c1fddf64178f3ea0d9d9d88449fe3af9b32fc59d3
+  metadata.gz: 1da83cfa99bb2a77a32b8bd72dd5bee692f94f87b453706f4443cc015e065ac4
+  data.tar.gz: b0c60214b0e13a7a0c346f5e681d78b1701a7de95e5e816f84193d6593f2c282
 SHA512:
-  metadata.gz: 148770cb2522083b5944a5ee0f5830a1e1453f9c06647feaf0b92842e6008e2654aefd7e8c4a4a97509fc823702d64176a7bf4e30e8a08d409308f57792bbb1d
-  data.tar.gz: 99e64f1f1066528429617270d54ae64109896d1318634f5b4b001a9d10e4ed19b88239dc4b7a089c885d4f481d2799b22b70495b763d64df51702c82b15865f8
+  metadata.gz: 6cdc11bc1e0bbfb5d76e8d26bb80a71d498c3a04a0fbd1a7e3212640c3f0ae1513fb8bb18784e1dfdd7b96aafc1771ed8cf0db00ac1fedde8e63579943df1e2b
+  data.tar.gz: ed97b4b602abb304db44199247b533d2fce410ab8a3a80182e9a08ac65eff680d0df198f529ef53b87e947ccf8114b4e89e333f5695928fc1b0137b6dff90c53

data/CHANGELOG.md

@@ -1,5 +1,72 @@
 # Changelog
 
+## 1.1.0
+
+The gem no longer depends on `async`. The same worker runs under any
+`Fiber::Scheduler` the host installs — Falcon/Async, Itsi, or another
+implementation.
+
+### Breaking
+
+- `Runner#run` no longer starts a reactor. It needs an active
+  `Fiber.scheduler` and raises `Runtime::SchedulerRequired` without one:
+
+  ```ruby
+  Async { runner.run }                             # Falcon / Async
+  Async::Background::Scheduler.run { runner.run }  # async or itsi via ENV
+  ```
+
+- Job timeouts raise `Async::Background::Runtime::TimeoutError` instead of
+  `Async::TimeoutError`. Update rescue sites and any stored `error_class`.
+- `Runner#drain_jobs` is bounded (`drain_timeout:` default 30s). In-flight jobs
+  that outlive it are cancelled. Pass `drain_timeout: nil` for an unbounded wait.
+- `TaskGroup#stop_all(grace = nil)` returns whether the group drained, not a
+  count of cancelled tasks.
+- `Async::Background::Error` is the base class for `Runtime::Error` and
+  `ConfigError`. Both are still `StandardError`.
+
+### Added
+
+- `Async::Background::Runtime` — `Task`, `TaskGroup`, `Semaphore`,
+  `Notification`, `with_timeout`, `native_timeouts?`, `with_error_handler`.
+- `Async::Background::Scheduler` — optional bootstrap
+  (`require "async/background/scheduler"`) that installs `async` or
+  `itsi-scheduler` from `ASYNC_BACKGROUND_SCHEDULER`. `Scheduler.preload!`
+  loads the gem before fork. `Scheduler.run` on Itsi uses the current thread;
+  `ASYNC_BACKGROUND_SCHEDULER_THREAD=1` restores a dedicated thread.
+
+### Fixed
+
+- The saturated-queue wake-up fires from `TaskGroup#on_release`, after the job
+  has left `@jobs`. Signalling from the task's `ensure` left the listener
+  seeing a full group on any scheduler that resumes `unblock` immediately.
+- `Runtime.with_timeout` calls `scheduler.timeout_after` when the hook exists.
+  The `::Timeout.timeout` fallback uses `Thread#raise` and can hit the wrong
+  fiber; a missing hook now warns once per scheduler class.
+- `SocketWaker#close` stops the accept loop before the self-connect that
+  unblocks it, and hangs up tracked client sockets. A fiber parked in
+  `IO#wait_readable` is not cancellable via `Task#stop` on a scheduler without
+  `#fiber_interrupt`.
+- `Runtime.error_handler` is scoped per runner (`on_error:` /
+  `with_error_handler`). A process-global handler was overwritten by the next
+  runner and never restored.
+- A failure awaited via `Task#wait` is no longer also sent to the error handler.
+- `Semaphore.new(0)` raises `ArgumentError` instead of deadlocking on acquire.
+
+### Changed
+
+- `async` is a development dependency. Runtime gems are `console`, `fugit`
+  and `base64`.
+- `SocketWaker#start_accept_loop` no longer needs a parent task (the
+  argument is still accepted and ignored).
+- Shutdown closes the waker before the store, so the listener cannot be inside
+  `Store#fetch` when the connection disappears.
+
+### Unchanged
+
+- Dashboard, `perform_async` / `perform_in` / `perform_at`, SQLite schema,
+  and the cross-process wake protocol.
+
 ## 1.0.2
 
 Queue maintenance bug fix plus profiler-driven work on the hot paths that

data/README.md

@@ -1,8 +1,8 @@
 # Async::Background
 
-A lightweight cron, interval, and job-queue scheduler for Ruby's
-[Async](https://github.com/socketry/async) ecosystem. Built for
-[Falcon](https://github.com/socketry/falcon), works with any Async app.
+A lightweight cron, interval, and job-queue scheduler for Ruby fibers.
+Works with any `Fiber::Scheduler` — [Falcon](https://github.com/socketry/falcon) /
+[Async](https://github.com/socketry/async), [Itsi](https://itsi.fyi), or your own.
 
 - **Cron & interval scheduling** on a single event loop with a min-heap.
 - **Dynamic job queue** backed by SQLite, with delayed jobs
@@ -15,11 +15,11 @@ A lightweight cron, interval, and job-queue scheduler for Ruby's
 
 ---
 
-## Why Async? Why fibers?
+## Why fibers?
 
-The whole gem is built around the assumption that Falcon's reactor schedules
-many fibers on top of one OS thread per process — so the dashboard's SSE
-stream, the cron scheduler, and the queue worker all share that one thread
+The whole gem is built around the assumption that the host schedules many
+fibers on top of one OS thread per process — so the dashboard's SSE stream,
+the cron scheduler, and the queue worker all share that one thread
 cooperatively. A blocked fiber yields; a blocked thread doesn't.
 
 ![Threads vs fibers under different Ruby web servers](docs/fibers-vs-threads.svg)
@@ -30,13 +30,31 @@ open tab.
 
 ---
 
+## Fiber scheduler
+
+`Runner#run` does not start a reactor. The host must install a
+`Fiber.scheduler` first:
+
+```ruby
+Async { runner.run }  # Falcon / Async
+
+require "async/background/scheduler"
+Async::Background::Scheduler.run { runner.run }  # ASYNC_BACKGROUND_SCHEDULER=async|itsi
+```
+
+Without a scheduler it raises `Runtime::SchedulerRequired`.
+Worker setup for Falcon and Itsi is in [Get started](docs/GET_STARTED.md).
+
+---
+
 ## Requirements
 
 | Dependency           | Version          | Required?            |
 | -------------------- | ---------------- | -------------------- |
 | Ruby                 | `>= 3.3`         | yes                  |
-| `async`              | `~> 2.0`         | yes                  |
+| a `Fiber::Scheduler` | `async ~> 2.0`, `itsi-scheduler`, … | yes — installed by the host, not by this gem |
 | `fugit`              | `~> 1.0`         | yes                  |
+| `console`            | `~> 1.0`         | yes                  |
 | `sqlite3`            | `~> 2.0`         | for the queue & dashboard |
 | `async-utilization`  | `>= 0.3, < 0.5`  | for metrics          |
 
@@ -48,16 +66,16 @@ open tab.
 # Gemfile
 gem "async-background"
 
-gem "sqlite3",           "~> 2.0"          # if you use the queue or dashboard
-gem "async-utilization", ">= 0.3", "< 0.5" # if you want worker metrics
+gem "async", "~> 2.0"          # or `itsi-scheduler`
+gem "sqlite3", "~> 2.0"        # queue / dashboard
+gem "async-utilization", ">= 0.3", "< 0.5"  # optional metrics
 ```
 
 ---
 
 ## ➡️ [Get started](docs/GET_STARTED.md)
 
-A four-step walkthrough: schedule config, Falcon integration, Docker, queue,
-delayed jobs.
+Schedule config, Falcon or Itsi workers, Docker, queue, delayed jobs.
 
 ---
 
@@ -120,9 +138,12 @@ jobs use wall-clock time, because "every day at 3am" needs to mean 3am.
 
 ## How it works
 
-A single Async task sleeps until the next entry is due, then dispatches it
-under a semaphore that caps concurrency. Overlapping ticks are skipped and
-rescheduled.
+A single fiber sleeps until the next entry is due, then dispatches it under a
+semaphore that caps concurrency. Overlapping ticks are skipped and rescheduled.
+Every dispatched job is tracked in a `Runtime::TaskGroup`, so shutdown drains
+in-flight work before the SQLite store and the wake-up socket are closed. The
+drain is bounded by `drain_timeout:` (30s by default; pass `nil` for the old
+unbounded wait) so one wedged job cannot turn SIGTERM into SIGKILL.
 
 ```
 schedule.yml → build_heap → MinHeap<Entry> → scheduler loop → Semaphore → run_job

data/async-background.gemspec

@@ -8,12 +8,13 @@ Gem::Specification.new do |spec|
   spec.authors = ['Roman Hajdarov']
   spec.email   = ['romnhajdarov@gmail.com']
 
-  spec.summary     = 'Lightweight heap-based cron/interval scheduler for Async.'
-  spec.description = 'A production-grade lightweight scheduler built on top of Async. ' \
-                     'Single event loop with min-heap timer, skip-overlapping execution, ' \
-                     'jitter, monotonic clock intervals, semaphore concurrency control, ' \
-                     'and deterministic worker sharding. Designed for Falcon but works ' \
-                     'with any Async-based application.'
+  spec.summary     = 'Lightweight heap-based cron/interval scheduler for any Fiber scheduler.'
+  spec.description = 'A production-grade lightweight background job scheduler built on the ' \
+                     'Fiber::Scheduler interface itself, not on any one implementation: it ' \
+                     'runs under Async/Falcon, Itsi, or anything else that installs a ' \
+                     'scheduler. Single event loop with min-heap timer, skip-overlapping ' \
+                     'execution, jitter, monotonic clock intervals, semaphore concurrency ' \
+                     'control, and deterministic worker sharding.'
 
   spec.homepage = 'https://github.com/roman-haidarov/async-background'
   spec.license  = 'MIT'
@@ -30,15 +31,20 @@ Gem::Specification.new do |spec|
 
   spec.required_ruby_version = '>= 3.3'
 
-  spec.add_dependency 'async',   '~> 2.0'
+  # No scheduler dependency on purpose. The gem uses the Fiber::Scheduler
+  # interface (Fiber.schedule, #block/#unblock, Timeout) and requires the host
+  # to install an implementation.
   spec.add_dependency 'console', '~> 1.0'
   spec.add_dependency 'fugit',   '~> 1.0'
   spec.add_dependency 'base64',  '~> 0.2'
 
   # Optional: add to your own Gemfile if you need these features
+  #   gem 'async', '~> 2.0'                      # or any other Fiber scheduler
+  #   gem 'itsi-scheduler'                       # ... such as this one
   #   gem 'sqlite3', '~> 2.0'
   #   gem 'async-utilization', '>= 0.3', '< 0.5' # shared-memory worker metrics
 
+  spec.add_development_dependency 'async', '~> 2.0'
   spec.add_development_dependency 'rake',  '~> 13.0'
   spec.add_development_dependency 'rspec', '~> 3.12'
   spec.add_development_dependency 'rack',  '~> 3.0'

data/lib/async/background/queue/schema.rb

@@ -29,11 +29,9 @@ module Async
 
           enable_incremental_vacuum!(db) unless jobs_table?(db)
 
-          with_migration_timeout(db) do
-            immediate_transaction(db) do
-              reject_future_version!(db)
-              upgrade!(db) unless current?(db)
-            end
+          synchronized_change(db) do
+            reject_future_version!(db)
+            upgrade!(db) unless current?(db)
           end
         end
 
@@ -41,20 +39,20 @@ module Async
           migrate!(db)
           return if dashboard_indexes_current?(db)
 
-          with_migration_timeout(db) do
-            immediate_transaction(db) do
-              create_dashboard_indexes!(db) unless dashboard_indexes_current?(db)
-            end
+          synchronized_change(db) do
+            create_dashboard_indexes!(db) unless dashboard_indexes_current?(db)
           end
         end
 
+        def synchronized_change(db, &change)
+          with_migration_timeout(db) { immediate_transaction(db, &change) }
+        end
+
         def current?(db)
           jobs_table?(db) && version(db) == VERSION && core_indexes_current?(db)
         end
 
-        def dashboard_indexes_current?(db)
-          (DASHBOARD_INDEXES - index_names(db)).empty?
-        end
+        def dashboard_indexes_current?(db) = indexes_present?(db, DASHBOARD_INDEXES)
 
         def version(db)
           db.get_first_value(SQL::USER_VERSION).to_i
@@ -101,8 +99,10 @@ module Async
           SQL::CREATE_DASHBOARD_INDEXES.each { |statement| db.execute(statement) }
         end
 
-        def core_indexes_current?(db)
-          (CORE_INDEXES - index_names(db)).empty?
+        def core_indexes_current?(db) = indexes_present?(db, CORE_INDEXES)
+
+        def indexes_present?(db, names)
+          (names - index_names(db)).empty?
         end
 
         def jobs_table?(db)

data/lib/async/background/queue/socket_waker.rb

@@ -1,104 +1,167 @@
 # frozen_string_literal: true
 
 require 'socket'
-require 'async/notification'
 require 'fileutils'
+require_relative '../runtime'
 
 module Async
   module Background
     module Queue
       class SocketWaker
+        CLOSE_GRACE = 2
+
         attr_reader :path
 
         def initialize(path)
           @path = path
           @server = nil
-          @notification = ::Async::Notification.new
+          @notification = Runtime::Notification.new
           @running = false
           @accept_task = nil
+          @clients = Runtime::TaskGroup.new
+          @sockets = {}
         end
 
         def open!
-          cleanup_stale_socket
           ensure_directory
+          cleanup_stale_socket
           @server = UNIXServer.new(@path)
-          File.chmod(0600, @path)
           @running = true
+          self
         rescue Errno::EADDRINUSE
           raise "Socket #{@path} is already in use by another process"
         end
 
-        def start_accept_loop(parent_task)
-          @accept_task = parent_task.async do |task|
-            while @running
-              begin
-                client = @server.accept_nonblock
-                handle_client(task, client)
-              rescue IO::WaitReadable
-                @server.wait_readable
-              rescue Errno::EBADF, IOError
-                break
-              rescue => e
-                Console.logger.error(self) { "SocketWaker accept error: #{e.class} #{e.message}" }
-              end
-            end
-          rescue => e
-            Console.logger.error(self) { "SocketWaker loop crashed: #{e.class} #{e.message}\n#{e.backtrace.join("\n")}" }
-          ensure
-            @accept_task = nil
-          end
+        def start_accept_loop(_parent_task = nil)
+          @accept_task = Runtime.spawn(name: 'socket-waker-accept') { accept_loop }
         end
 
         def wait(timeout: nil)
-          if timeout
-            ::Async::Task.current.with_timeout(timeout) { @notification.wait }
-          else
-            @notification.wait
-          end
-        rescue ::Async::TimeoutError
-          # Timeout is normal - listener will fall back to polling
+          @notification.wait(timeout)
         end
 
         def signal
-          @notification.signal
+          @notification.signal_all
+          true
         end
 
         def close
+          return unless @running || @server
+
           @running = false
-          if @accept_task && !@accept_task.finished?
-            @accept_task.stop rescue nil
-          end
+          stop_accept_loop
+          @notification.signal_all
+          hang_up_clients
+          stop_clients
 
           @server&.close rescue nil
           @server = nil
+          @accept_task = nil
           File.unlink(@path) rescue nil
         end
 
         private
 
-        def handle_client(parent_task, client)
-          parent_task.async do
+        def accept_loop
+          while @running
             begin
-              loop do
-                begin
-                  client.read_nonblock(256)
-                  @notification.signal
-                rescue IO::WaitReadable
-                  client.wait_readable
-                  retry
-                rescue EOFError, Errno::ECONNRESET
-                  break
-                end
-              end
-            rescue => e
-              Console.logger.warn(self) { "SocketWaker client handler error: #{e.class} #{e.message}" }
-            ensure
-              client.close rescue nil
-              @notification.signal
+              client = @server.accept_nonblock
+            rescue IO::WaitReadable
+              @server.wait_readable
+              next
+            rescue Errno::EBADF, IOError
+              break
+            rescue StandardError => e
+              Console.logger.error(self) { "SocketWaker accept error: #{e.class} #{e.message}" }
+              next
             end
+
+            break unless @running
+
+            handle_client(client)
+          end
+        rescue StandardError => e
+          Console.logger.error(self) { "SocketWaker loop crashed: #{e.class} #{e.message}\n#{e.backtrace.join("\n")}" }
+        end
+
+        def handle_client(client)
+          @sockets[client] = true
+
+          @clients.spawn(name: 'socket-waker-client') do
+            loop do
+              client.read_nonblock(256)
+              @notification.signal_all
+            rescue IO::WaitReadable
+              client.wait_readable
+              retry
+            rescue EOFError, Errno::ECONNRESET, Errno::EBADF, IOError
+              break
+            end
+          rescue StandardError => e
+            Console.logger.warn(self) { "SocketWaker client handler error: #{e.class} #{e.message}" }
+          ensure
+            @sockets.delete(client)
+            client.close rescue nil
+            @notification.signal_all
+          end
+        end
+
+        def stop_accept_loop
+          task = @accept_task or return
+
+          wake_accept_loop
+          return if await(task, CLOSE_GRACE)
+
+          task.stop
+          await(task, CLOSE_GRACE)
+        end
+
+        def wake_accept_loop
+          return unless @server
+
+          UNIXSocket.open(@path) { |s| s.write_nonblock("\x00") rescue nil }
+        rescue StandardError
+          nil
+        end
+
+        def hang_up_clients
+          @sockets.keys.each do |socket|
+            socket.close
+          rescue StandardError
+            nil
           end
         end
 
+        def stop_clients
+          return if @clients.empty?
+
+          return if await_group(@clients, CLOSE_GRACE)
+
+          @clients.stop_all(CLOSE_GRACE)
+        end
+
+        def await(task, grace)
+          return true unless Runtime.scheduler
+
+          task.wait(grace)
+          true
+        rescue Runtime::TimeoutError
+          false
+        rescue Exception # rubocop:disable Lint/RescueException
+          true
+        end
+
+        def await_group(group, grace)
+          return true unless Runtime.scheduler
+
+          group.wait(grace)
+          true
+        rescue Runtime::TimeoutError
+          false
+        rescue StandardError
+          true
+        end
+
         def cleanup_stale_socket
           return unless File.exist?(@path)
 

data/lib/async/background/queue/store.rb

@@ -360,22 +360,26 @@ module Async
           Job::Options.new(**options)
         end
 
+        STATEMENTS = {
+          :@enqueue_stmt => SQL::INSERT_JOB,
+          :@fetch_stmt => SQL::FETCH_NEXT_JOB,
+          :@mark_started_stmt => SQL::MARK_STARTED,
+          :@complete_stmt => SQL::COMPLETE_JOB,
+          :@fail_stmt => SQL::FAIL_JOB,
+          :@retry_state_stmt => SQL::RETRY_STATE,
+          :@lease_check_stmt => SQL::LEASE_ALIVE,
+          :@retry_stmt => SQL::RETRY_JOB,
+          :@requeue_stmt => SQL::RECOVER_WORKER,
+          :@cleanup_done_stmt => SQL::CLEANUP_DONE,
+          :@cleanup_failed_stmt => SQL::CLEANUP_FAILED,
+          :@next_pending_stmt => SQL::NEXT_PENDING_RUN_AT,
+          :@begin_stmt => SQL::BEGIN_IMMEDIATE,
+          :@commit_stmt => SQL::COMMIT,
+          :@rollback_stmt => SQL::ROLLBACK
+        }.freeze
+
         def prepare_statements
-          @enqueue_stmt = @db.prepare(SQL::INSERT_JOB)
-          @fetch_stmt = @db.prepare(SQL::FETCH_NEXT_JOB)
-          @mark_started_stmt = @db.prepare(SQL::MARK_STARTED)
-          @complete_stmt = @db.prepare(SQL::COMPLETE_JOB)
-          @fail_stmt = @db.prepare(SQL::FAIL_JOB)
-          @retry_state_stmt = @db.prepare(SQL::RETRY_STATE)
-          @lease_check_stmt = @db.prepare(SQL::LEASE_ALIVE)
-          @retry_stmt = @db.prepare(SQL::RETRY_JOB)
-          @requeue_stmt = @db.prepare(SQL::RECOVER_WORKER)
-          @cleanup_done_stmt = @db.prepare(SQL::CLEANUP_DONE)
-          @cleanup_failed_stmt = @db.prepare(SQL::CLEANUP_FAILED)
-          @next_pending_stmt = @db.prepare(SQL::NEXT_PENDING_RUN_AT)
-          @begin_stmt = @db.prepare(SQL::BEGIN_IMMEDIATE)
-          @commit_stmt = @db.prepare(SQL::COMMIT)
-          @rollback_stmt = @db.prepare(SQL::ROLLBACK)
+          STATEMENTS.each { |name, sql| instance_variable_set(name, @db.prepare(sql)) }
         end
 
         def finalize_statements
@@ -384,32 +388,11 @@ module Async
         end
 
         def statements
-          [
-            @enqueue_stmt,
-            @fetch_stmt,
-            @mark_started_stmt,
-            @complete_stmt,
-            @fail_stmt,
-            @retry_state_stmt,
-            @lease_check_stmt,
-            @retry_stmt,
-            @requeue_stmt,
-            @cleanup_done_stmt,
-            @cleanup_failed_stmt,
-            @next_pending_stmt,
-            @begin_stmt,
-            @commit_stmt,
-            @rollback_stmt
-          ]
+          STATEMENTS.keys.map { |name| instance_variable_get(name) }
         end
 
         def clear_statements
-          @enqueue_stmt = @fetch_stmt = @mark_started_stmt = nil
-          @complete_stmt = @fail_stmt = @retry_state_stmt = @lease_check_stmt = nil
-          @retry_stmt = @requeue_stmt = nil
-          @cleanup_done_stmt = @cleanup_failed_stmt = nil
-          @next_pending_stmt = nil
-          @begin_stmt = @commit_stmt = @rollback_stmt = nil
+          STATEMENTS.each_key { |name| instance_variable_set(name, nil) }
         end
 
         def maybe_cleanup

data/lib/async/background/runner/queue_execution.rb

@@ -9,6 +9,7 @@ module Async
         private
 
         def setup_queue(queue_socket_dir, queue_db_path, queue_mmap)
+          @queue_saturated = false
           @listen_queue = !!queue_socket_dir && !isolated_worker?
           return unless @listen_queue
 
@@ -26,20 +27,36 @@ module Async
           recover_queue_jobs
         end
 
-        def start_queue_listener(task)
-          @queue_waker.start_accept_loop(task)
+        def start_queue_listener
+          @queue_waker.start_accept_loop
 
-          task.async do
+          @services.spawn(name: 'queue-listener') do
             logger.info { "Async::Background queue: listening on worker #{worker_index}" }
 
+            failures = 0
+
             while running?
-              @queue_waker.wait(timeout: next_wait_timeout)
-              dispatch_available_queue_jobs
+              begin
+                @queue_waker.wait(timeout: next_wait_timeout)
+                break unless running?
+
+                dispatch_available_queue_jobs
+                failures = 0
+              rescue StandardError => error
+                failures += 1
+                backoff = [QUEUE_ERROR_BACKOFF * failures, QUEUE_POLL_INTERVAL].min
+                logger.error('Async::Background') do
+                  "queue listener: #{error.class} #{error.message}; retrying in #{backoff}s"
+                end
+                shutdown.wait(backoff)
+              end
             end
           end
         end
 
         def next_wait_timeout
+          return QUEUE_POLL_INTERVAL if @queue_saturated
+
           next_due = @queue_store.next_pending_run_at
           return QUEUE_POLL_INTERVAL unless next_due
 
@@ -68,7 +85,7 @@ module Async
           complete_queue_job!(job, class_name, claim_token, started_at)
         rescue ConfigError => error
           record_invalid_queue_job!(job, class_name, claim_token, error)
-        rescue ::Async::TimeoutError => error
+        rescue Runtime::TimeoutError => error
           handle_queue_failure(
             job,
             options,
@@ -129,11 +146,18 @@ module Async
         end
 
         def dispatch_available_queue_jobs
+          @queue_saturated = false
+
           while running?
+            if @jobs.size >= @semaphore.limit
+              @queue_saturated = true
+              return
+            end
+
             job = @queue_store.fetch(worker_index)
-            break unless job
+            return if job.nil?
 
-            semaphore.async { |job_task| run_queue_job(job_task, job) }
+            spawn_job { |job_task| run_queue_job(job_task, job) }
           end
         end