solid-jobs 0.1.1 → 0.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ce7c73e437bf4a78efbf9422e4a5cb7cf05635b5f3c9cbca4cd44a9baa0ee2d9
-  data.tar.gz: f0b4513d2c4d75bc9f871ba2c64c7848789747cf797e32044d56eefdab961647
+  metadata.gz: 1a9ddd8000ed07b176d4e291ba7e6b5c1b6a3d909efe282ea6ace542d676e142
+  data.tar.gz: 242f4734ef254188dee64f1ce73df43cbb83f6d39840d7e8f59c6017a499a5f3
 SHA512:
-  metadata.gz: e6d245f323c4cdf117f2bb53584eed2e2c697a72fdc170792954d25feccb15b32d2cae7f3d8a337e7b29c3fedb071ab4ad9b773ac414fe893e389754532ca5af
-  data.tar.gz: 9abce08d1c4fb800a5f39e56fcebc266a7cc1dd52acd891fdaccf9fd1f2840ead4a3c4ce0a5649acc6c27a8f26ad5554d0599a5eb55f8e742afc568a307ddc09
+  metadata.gz: 8d2bb684bb107820e189f12d4ac709ce8ad43e546bdf2da5933fcd781627655a8897030ed53010861a74619d65afb33fd89466a1c12993d8ab51e4de85f9c2f8
+  data.tar.gz: 22fb65c94e80dc84dc94f5e6ad3fa9a5047378f45fe29263c5a98817c6a0aeda71f9d54f597a69b0407332e8052f1a5d5da097b97fbd22be912543c56015f78d

data/CHANGELOG.md

@@ -4,6 +4,25 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+## [0.1.2] - 2026-10-01
+
+### Added
+
+- `Server#start` logs a warning on Ruby < 4 when `concurrency > 1`, pointing
+  to the Ruby 3.4 Ractor GC-barrier deadlock and the recommended setups.
+- `Server#stop` is bounded: components get `shutdown_timeout + STOP_GRACE`
+  to return, `:stop` is re-sent up to `STOP_RESENDS` times, then the
+  component is abandoned with an error log instead of hanging shutdown.
+- `test/support/ractor_barrier_repro.rb`: standalone reproducer of the Ruby
+  3.4 Ractor GC-barrier deadlock (no SolidJobs, no Redis).
+
+### Changed
+
+- Server-based stress tests are skipped on Ruby < 4: Ruby 3.4's Ractor
+  scheduler deadlocks the VM on a GC barrier under cross-Ractor `move:`
+  traffic. Multi-Ractor servers are recommended on Ruby ≥ 4.0 (see
+  `docs/reliability.md`). CI jobs now time out after 20 minutes.
+
 ## [0.1.1] - 2026-10-01
 
 ### Fixed

data/README.md

@@ -149,11 +149,15 @@ also uses more RSS, but reaches 13.77 jobs/s/MiB versus 3.41 for Sidekiq.
 These are local synthetic measurements, not application-capacity claims.
 Queue p95/p99 values in this run use sparse sampling and are excluded from the
 summary until the final latency campaign increases the sample count. Ruby
-3.4.4 eight-Ractor results are also excluded: concurrent TCP/RESP
-initialization triggered a reproducible native crash on the tested Apple
-Silicon environment. Ruby 4.0.1 passed the equivalent reproducer 100/100
-times, and `StartupBarrier` serializes component initialization before
-releasing normal parallel processing.
+3.4.4 eight-Ractor results are also excluded: Ruby 3.4's Ractor scheduler
+crashed (concurrent TCP/RESP initialization, reproduced on macOS arm64 and
+Linux x86_64) or deadlocked VM-wide on a GC barrier under cross-Ractor
+message traffic (`test/support/ractor_barrier_repro.rb` reproduces it without
+SolidJobs).
+Ruby 4.0.1 passed the equivalent reproducers, and `StartupBarrier` serializes
+component initialization before releasing normal parallel processing.
+**Multi-Ractor servers are recommended on Ruby ≥ 4.0**; see
+`docs/reliability.md`.
 
 The project is under active development. The Web UI and commercial Sidekiq
 features are not part of the initial scope.

data/docs/reliability.md

@@ -71,6 +71,33 @@ initialization paths known to crash Ruby 3.4 (reproduced on 3.4.4 macOS arm64
 and 3.4.11 Linux x86_64; `rake startup_torture` is the reproducer); normal
 processing remains parallel after `RUNNING`.
 
+### Ruby 3.4 Ractor caveat
+
+Ruby 3.4's Ractor scheduler can deadlock the whole VM (main thread included)
+when a GC-triggered `rb_ractor_sched_barrier_start` runs while several
+Ractors exchange `move: true` messages: every thread parks in
+`ractor_sched_barrier_join_wait_locked` and the barrier never completes.
+`test/support/ractor_barrier_repro.rb` reproduces it **without SolidJobs or
+Redis** (one receiver, four senders, 24k moved messages per iteration):
+Ruby 3.4.4 freezes within the first iterations, Ruby 4.0.1 completes 30/30.
+Inside SolidJobs the same traffic pattern is the Processor → Heartbeat
+`:work`/`:done`/`:stats` channel, so any multi-Processor server on Ruby 3.4
+is exposed; once frozen, neither `Timeout` nor process exit
+(`rb_ractor_terminate_all`) can recover.
+
+Recommendation: **run multi-Ractor SolidJobs servers on Ruby ≥ 4.0**. On
+Ruby 3.4 use the client/API side freely, and prefer one process per
+Processor (`concurrency: 1`) for the server. Server-based stress tests are
+skipped on Ruby < 4 for this reason, and `Server#start` logs a warning when
+it detects `concurrency > 1` on Ruby < 4.
+
+### Bounded shutdown
+
+`Server#stop` never waits forever for a component. Each Ractor gets
+`shutdown_timeout + Server::STOP_GRACE` to return after `:stop`; the signal
+is re-sent up to `STOP_RESENDS` times, then the component is abandoned with
+an error log so the process can proceed with shutdown.
+
 ## Configuration scope
 
 `SolidJobs.config` and the testing mode are Ractor-local, not thread-local.

data/lib/solid_jobs/server.rb

@@ -2,13 +2,20 @@
 
 require "securerandom"
 require "socket"
+require "timeout"
 
 module SolidJobs
   class Server
+    # Extra time granted beyond shutdown_timeout before a component is
+    # considered stuck, and how many times :stop is re-sent before giving up.
+    STOP_GRACE = 5.0
+    STOP_RESENDS = 3
+
     attr_reader :config, :identity, :ractors
 
-    def initialize(config: SolidJobs.config)
+    def initialize(config: SolidJobs.config, stop_grace: STOP_GRACE)
       @config = config
+      @stop_grace = stop_grace
       @ractors = []
       @scheduler = nil
       @heartbeat = nil
@@ -20,6 +27,7 @@ module SolidJobs
     def start
       raise Error, "Server is already running" if @started
 
+      warn_ruby34_multi_ractor
       snapshot = Utilities.shareable_copy(
         config.ractor_snapshot.merge(identity: identity, started_at: @started_at),
       )
@@ -197,10 +205,12 @@ module SolidJobs
 
       @ractors.each { |ractor| ractor.send(:stop) }
       @scheduler&.send(:stop)
-      results = @ractors.map { |ractor| RactorSupport.value(ractor) }
-      RactorSupport.value(@scheduler) if @scheduler
+      results = @ractors.each_with_index.map do |ractor, index|
+        await_termination(ractor, "processor #{index}")
+      end
+      await_termination(@scheduler, "scheduler") if @scheduler
       @heartbeat&.send(:stop)
-      RactorSupport.value(@heartbeat) if @heartbeat
+      await_termination(@heartbeat, "heartbeat") if @heartbeat
       @ractors = []
       @scheduler = nil
       @heartbeat = nil
@@ -213,6 +223,47 @@ module SolidJobs
       @started
     end
 
+    private
+
+    RUBY34_MULTI_RACTOR_WARNING =
+      "SolidJobs: Ruby %s can deadlock the whole VM on a Ractor GC barrier " \
+      "under multi-Processor load (see docs/reliability.md). Run multi-Ractor " \
+      "servers on Ruby >= 4.0, or use concurrency: 1 (one process per Processor)."
+
+    def warn_ruby34_multi_ractor
+      return if RUBY_VERSION >= "4" || config.concurrency <= 1
+
+      config.logger.warn(format(RUBY34_MULTI_RACTOR_WARNING, RUBY_VERSION))
+    end
+
+    # Waits for a component Ractor to return after :stop. Ruby 3.4 can lose
+    # the wakeup of a Ractor.receive running in a secondary thread while the
+    # inbox is busy; a fresh :stop message re-triggers it. If the component
+    # still does not return, it is abandoned so shutdown never hangs forever.
+    def await_termination(ractor, name)
+      deadline = config.shutdown_timeout + @stop_grace
+      resends = 0
+      begin
+        Timeout.timeout(deadline) { RactorSupport.value(ractor) }
+      rescue Timeout::Error
+        if resends < STOP_RESENDS
+          resends += 1
+          config.logger.warn(
+            "SolidJobs #{name} did not stop within #{deadline}s, " \
+            "re-sending :stop (#{resends}/#{STOP_RESENDS})",
+          )
+          begin
+            ractor.send(:stop)
+          rescue Ractor::ClosedError
+            nil
+          end
+          retry
+        end
+        config.logger.error("SolidJobs #{name} did not stop; abandoning it")
+        nil
+      end
+    end
+
     def remote_signal
       config.redis_pool.call("RPOP", "#{identity}-signals")
     end

data/lib/solid_jobs/version.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
 module SolidJobs
-  VERSION = "0.1.1"
+  VERSION = "0.1.2"
 end
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: solid-jobs
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.1.2
 platform: ruby
 authors:
 - Nicolas Vandenbogaerde