carbon_fiber 0.1.3 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e4912cee9b33c19f921ee9ac02bfb3b7522295d4036a026c4e01756f827268b1
-  data.tar.gz: a19ab53436bd10394f12ba6f0f0a91e0b62aac470865b86af932ba98922fb0fa
+  metadata.gz: 3906713d933ac5b29349ed47160f5e2704060f0a3cdd101474eea9e59cf74dcf
+  data.tar.gz: 78da45d46027261788f4c01333ae68cb90ece1992350b7aabb52e59564b32b9b
 SHA512:
-  metadata.gz: c59ef2c0819bc66a3ba7f4b19474d1f2b38404ac99e0dc018e1bdf4247008627aea99203b063817f746702be501d2b357ac67048ebd74ef4f059448e67e20d1e
-  data.tar.gz: a4703832a539939964ed9dcb535ee1813038a23fe94837457c8ddf5a8fea5e808552b40661bad6ae83091d2cbefc8c55f513a69a3d9671ffb8e6dee73bbfa821
+  metadata.gz: cb0ae60a2ef8f3463d437b79cc4fd9d36f092060504eb2bcaea31e17b96cf1e137784cd3b01f1f1950d53fd7cb2737f6cb15cbc0c952fb3156578d3ac7b8b31e
+  data.tar.gz: df4857f35926d78a04e75f908f03c423d0d275a4a2bb3a4d7fd03eab3f84f46b16feafffc352b6898860aeda02a9f8f91fa51156afbf8d82ef43056899990200

data/CHANGELOG.md

@@ -1,3 +1,19 @@
+## [Unreleased]
+
+- Fix the pure-Ruby fallback loop burning CPU while a fiber waits on a background thread; inside a non-main Ractor that spin starved the thread it was waiting for and hung the loop.
+
+## [0.2.0] - 2026-09-05
+
+- Work inside Ractors: the native extension is declared Ractor-safe, so each Ractor can install its own `CarbonFiber::Scheduler` (several at once need Ruby 4.0 or later). See the README for the supported messaging pattern.
+- Resolve hostnames in non-main Ractors through the platform resolver on a background thread, since Resolv cannot run there.
+- Fix fibers parked in `Queue#pop`, `Mutex#lock`, or any `block` without a timeout being garbage collected while waiting; once unblocked this raised `TypeError: wrong argument type false (expected fiber)` or hung the loop.
+- Fix a race between garbage collection and I/O completions arriving while the loop waits with the GVL released.
+- Support the Ruby 4.1 buffered I/O scheduler contract: `io_read`/`io_write` take `(io, buffer, offset, length)` and perform a single transfer; Ruby 3.4 through 4.0 keep the legacy behavior. (Thanks to [Samuel Williams](https://github.com/samuel-williams-shopify)).
+- The native extension now loads on Ruby development builds: a C shim compiled against the target headers provides the `ruby_abi_version` their ABI check requires.
+- Match Ruby 4.1's updated `rb_data_type_t` layout when registering the native selector, and verify the layout against the target Ruby's headers when the extension loads.
+- Under the Ruby 4.1 contract, regular-file reads and writes run on the background I/O path instead of risking blocking the scheduler thread.
+- `Scheduler#io_close` and `Async::Selector#io_close` now take a file descriptor (Integer), matching Ruby's Fiber::Scheduler protocol. (Thanks to [Samuel Williams](https://github.com/samuel-williams-shopify)).
+
 ## [0.1.3] - 2026-05-08
 
 - Update to Zig 0.16.0; update libxev and zig.rb to the latest versions.

data/README.md

@@ -240,6 +240,47 @@ end
 
 ---
 
+### Inside Ractors
+
+A fiber scheduler belongs to a thread, and every Ractor runs its own threads, so each Ractor installs its own `CarbonFiber::Scheduler`. Construct it inside the Ractor that runs it; never share one between Ractors. Running several such Ractors at once needs Ruby 4.0 or later; Ruby 3.4 aborts or hangs under that load, with the pure-Ruby selector as well. On Windows a garbage collection inside a non-main Ractor pauses for several seconds while the main Ractor waits, so Ractor workloads there are slow. Every hook works there as on the main Ractor, including DNS: `address_resolve` uses Resolv on the main Ractor and the platform resolver on a background thread elsewhere, because Resolv keeps state that Ractor isolation forbids.
+
+`Ractor.receive` (and `Ractor::Port#receive`) block the calling thread without going through the scheduler, so a worker must not wait for messages on the thread that runs its event loop: every fiber in the Ractor would stall until the next message. Receive on a dedicated thread instead and hand messages to a scheduled fiber through a `Thread::Queue`. A push from the receiving thread wakes the parked fiber through the scheduler's cross-thread `unblock` path, the same path background operations use.
+
+```ruby
+worker = Ractor.new do
+  scheduler = CarbonFiber::Scheduler.new
+  Fiber.set_scheduler(scheduler)
+
+  jobs = Thread::Queue.new
+  receiver = Thread.new do
+    while (message = Ractor.receive) != :stop
+      jobs.push(message)
+    end
+    jobs.close
+  end
+
+  results = []
+  Fiber.schedule do
+    loop do
+      job = jobs.pop or break # a fresh binding per job for the fiber below
+      Fiber.schedule { results << handle(job) } # any I/O the scheduler intercepts
+    end
+  end
+
+  Fiber.set_scheduler(nil) # runs the loop until the queue closes and every job is done
+  receiver.join
+  results
+end
+
+inputs.each { |job| worker.send(job) }
+worker.send(:stop)
+worker.value # Ractor#take on Ruby 3.4
+```
+
+A pipe works too: the receiving thread writes a byte per message and the dispatching fiber reads it before popping the queue, so the wake-up arrives through `io_wait`. The queue alone is simpler and is what `spec/carbon_fiber/ractor_spec.rb` exercises.
+
+---
+
 ## How It Works
 
 The scheduler has two layers:

data/lib/carbon_fiber/async.rb

@@ -2,6 +2,7 @@
 
 # Please note that this code is heavily AI-assisted.
 
+require_relative "io_contract"
 require_relative "native"
 
 module CarbonFiber
@@ -26,8 +27,12 @@ module CarbonFiber
       attr_reader :loop
 
       # @param loop [Fiber] the Async event loop fiber
-      def initialize(loop)
-        super
+      # @param io_contract_v4 [Boolean] use the Ruby 4.1+ single-transfer
+      #   contract in the native I/O paths (defaults to what the running
+      #   Ruby speaks; override only for testing)
+      def initialize(loop, io_contract_v4: CarbonFiber.io_contract_v4?)
+        super(loop)
+        self.io_contract_v4 = io_contract_v4
         @loop = loop
         @idle_duration = 0.0
 
@@ -117,37 +122,84 @@ module CarbonFiber
       EAGAIN = -Errno::EAGAIN::Errno
       EWOULDBLOCK = -Errno::EWOULDBLOCK::Errno
 
+      # Legacy contract (Ruby 3.4 through 4.0): length before offset,
+      # minimum-progress semantics.
       # @param fiber [Fiber]
       # @param io [IO]
       # @param buffer [IO::Buffer]
       # @param length [Integer]
       # @param offset [Integer]
       # @return [Integer] bytes read, or negative errno
-      def io_read(fiber, io, buffer, length, offset = 0)
+      def io_read_v3(fiber, io, buffer, length, offset = 0)
         result = native_io_read(io.fileno, buffer, length, offset)
         return result unless result.nil?
 
         ruby_io_read(fiber, io, buffer, length, offset)
       end
 
+      # Ruby 4.1+ contract: offset before length, single transfer. One
+      # nonblocking attempt; short results and -EAGAIN are returned
+      # directly, and the caller composes retries via io_wait.
+      # @param fiber [Fiber]
+      # @param io [IO]
+      # @param buffer [IO::Buffer]
+      # @param offset [Integer]
+      # @param length [Integer] maximum bytes for this transfer
+      # @return [Integer] bytes read, or negative errno
+      def io_read_v4(fiber, io, buffer, offset, length)
+        return 0 if length.zero?
+
+        result = native_io_read(io.fileno, buffer, length, offset)
+        return result unless result.nil?
+
+        ruby_io_read_v4(io, buffer, offset, length)
+      end
+
+      # Legacy contract (Ruby 3.4 through 4.0): length before offset,
+      # minimum-progress semantics.
       # @param fiber [Fiber]
       # @param io [IO]
       # @param buffer [IO::Buffer]
       # @param length [Integer]
       # @param offset [Integer]
       # @return [Integer] bytes written, or negative errno
-      def io_write(fiber, io, buffer, length, offset = 0)
+      def io_write_v3(fiber, io, buffer, length, offset = 0)
         result = native_io_write(io.fileno, buffer, length, offset)
         return result unless result.nil?
 
         ruby_io_write(fiber, io, buffer, length, offset)
       end
 
-      # Cancel pending waiters and close the descriptor.
+      # Ruby 4.1+ contract: offset before length, single transfer.
+      # @param fiber [Fiber]
       # @param io [IO]
-      def io_close(io)
-        fd = io.respond_to?(:fileno) ? io.fileno : io.to_i
-        super(fd, IOError.new("stream closed while waiting"))
+      # @param buffer [IO::Buffer]
+      # @param offset [Integer]
+      # @param length [Integer] maximum bytes for this transfer
+      # @return [Integer] bytes written, or negative errno
+      def io_write_v4(fiber, io, buffer, offset, length)
+        return 0 if length.zero?
+
+        result = native_io_write(io.fileno, buffer, length, offset)
+        return result unless result.nil?
+
+        ruby_io_write_v4(io, buffer, offset, length)
+      end
+
+      # The public hook names follow the contract of the running Ruby;
+      # both generations stay defined for direct testing.
+      if CarbonFiber.io_contract_v4?
+        alias_method :io_read, :io_read_v4
+        alias_method :io_write, :io_write_v4
+      else
+        alias_method :io_read, :io_read_v3
+        alias_method :io_write, :io_write_v3
+      end
+
+      # Cancel pending waiters on the descriptor.
+      # @param descriptor [Integer]
+      def io_close(descriptor)
+        super(descriptor, IOError.new("stream closed while waiting"))
       end
 
       # Wait for a child process on a background thread.
@@ -230,6 +282,22 @@ module CarbonFiber
         total
       end
 
+      # Ruby-level single-transfer io_read/io_write for non-socket fds under
+      # the Ruby 4.1+ contract: one nonblocking attempt, -EAGAIN and short
+      # results returned directly. No io_wait, no minimum-progress loop;
+      # Ruby's own read loop composes retries.
+      def ruby_io_read_v4(io, buffer, offset, length)
+        IO::Event::Selector.nonblock(io) do
+          Fiber.blocking { buffer.read(io, offset, length) }
+        end
+      end
+
+      def ruby_io_write_v4(io, buffer, offset, length)
+        IO::Event::Selector.nonblock(io) do
+          Fiber.blocking { buffer.write(io, offset, length) }
+        end
+      end
+
       def fallback_io_wait(io, events)
         Thread.new do
           Thread.current.report_on_exception = false

data/lib/carbon_fiber/io_contract.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module CarbonFiber
+  # Ruby 4.1 (ruby/ruby#18483) reordered the buffered scheduler hooks to
+  # +io_read(io, buffer, offset, length)+ and made them single-transfer:
+  # one nonblocking attempt, short results returned directly, -EAGAIN
+  # instead of waiting, and zero length returns 0 without I/O.
+  # +IO::Buffer::VERSION >= 3+ signals the new contract; Ruby 3.4 through
+  # 4.0 define no such constant.
+  #
+  # @return [Boolean] whether this Ruby uses the v4 scheduler I/O contract
+  def self.io_contract_v4?
+    defined?(IO::Buffer::VERSION) ? IO::Buffer::VERSION.to_i >= 3 : false
+  end
+end

data/lib/carbon_fiber/native/fallback.rb

@@ -33,6 +33,10 @@ module CarbonFiber
         true
       end
 
+      # Accepted for interface parity with the native selector; the fallback
+      # has no native I/O paths, so the contract generation changes nothing.
+      attr_writer :io_contract_v4
+
       # @return [Boolean] whether there is pending work
       def pending?
         @mutex.synchronize { @ready.any? || @timers.any? || @read_waits.any? }
@@ -100,7 +104,7 @@ module CarbonFiber
       # Run one event loop iteration.
       def select(timeout = nil)
         flush_ready
-        return 0 unless pending?
+        return 0 unless pending? || parked?
 
         deadline = next_wait_deadline(timeout)
 
@@ -124,6 +128,19 @@ module CarbonFiber
         flush_ready
       end
 
+      # True while a fiber is parked in block() or do_io_wait. Nothing in the
+      # ready list, timers, or read waits refers to it, so pending? is false,
+      # yet select must sleep on the condition variable rather than return:
+      # the wake-up comes from another thread (a background operation's
+      # resume, an unblock, or wakeup), and returning at once makes
+      # Scheduler#run spin with the GVL held. On the main Ractor the timer
+      # thread preempts that spin; inside a non-main Ractor Ruby's M:N
+      # scheduler does not, and the thread that would wake the fiber never
+      # runs.
+      def parked?
+        @mutex.synchronize { @blocked_fibers.any? }
+      end
+
       # Mirrors `Selector#kernel_sleep` on the native side so
       # `Scheduler#kernel_sleep` can delegate to `@selector.kernel_sleep`
       # in both paths. Branches on the duration: nil parks the fiber on

data/lib/carbon_fiber/scheduler.rb

@@ -2,9 +2,9 @@
 
 # Please note that this code is heavily AI-assisted.
 
-require "resolv"
 require "socket"
 require "timeout"
+require_relative "io_contract"
 require_relative "native"
 
 # High-performance Ruby Fiber Scheduler backed by Zig and libxev.
@@ -43,10 +43,20 @@ module CarbonFiber
   class Scheduler
     # @param root_fiber [Fiber] the event loop fiber (defaults to current)
     # @param selector [Class] native selector class to instantiate
-    def initialize(root_fiber = Fiber.current, selector: CarbonFiber::Native::Selector)
+    # @param io_contract_v4 [Boolean] use the Ruby 4.1+ single-transfer
+    #   contract in the native I/O paths (defaults to what the running
+    #   Ruby speaks; override only for testing)
+    def initialize(root_fiber = Fiber.current, selector: CarbonFiber::Native::Selector,
+      io_contract_v4: CarbonFiber.io_contract_v4?)
       @root_fiber = root_fiber
       @scheduler_thread = Thread.current
+      @main_ractor = !defined?(Ractor) || Ractor.current == Ractor.main
+      # Resolv keeps state Ractor isolation forbids, and only the main
+      # Ractor resolves through it (see #address_resolve), so load it here
+      # rather than at require time; a worker Ractor never loads it.
+      require "resolv" if @main_ractor
       @selector = selector.new(root_fiber)
+      @selector.io_contract_v4 = io_contract_v4
       @active_fibers = 0
       @background_count = 0
       @closed = false
@@ -189,14 +199,15 @@ module CarbonFiber
       await_background_operation { io_select_readiness(io, events, timeout) }
     end
 
-    # Read from an IO into a buffer via the native selector.
+    # Read from an IO into a buffer via the native selector (legacy contract,
+    # Ruby 3.4 through 4.0: length before offset, minimum-progress semantics).
     # Falls back to a background thread for non-socket descriptors.
     # @param io [IO]
     # @param buffer [IO::Buffer]
     # @param length [Integer]
     # @param offset [Integer]
     # @return [Integer] bytes read, or negative errno
-    def io_read(io, buffer, length, offset = 0)
+    def io_read_v3(io, buffer, length, offset = 0)
       # Native io_read_object extracts the descriptor in Zig, skipping a
       # `respond_to?(:fileno)` + `io.fileno` method-send pair per call.
       native_result = @selector.io_read_object(io, buffer, length, offset)
@@ -211,14 +222,39 @@ module CarbonFiber
       end
     end
 
-    # Write from a buffer to an IO via the native selector.
+    # Read from an IO into a buffer via the native selector (Ruby 4.1+
+    # contract: offset before length, single transfer). One nonblocking
+    # attempt; short results and -EAGAIN are returned directly, and Ruby's
+    # own read loop composes retries via io_wait.
+    # @param io [IO]
+    # @param buffer [IO::Buffer]
+    # @param offset [Integer]
+    # @param length [Integer] maximum bytes for this transfer
+    # @return [Integer] bytes read, or negative errno
+    def io_read_v4(io, buffer, offset, length)
+      return 0 if length.zero?
+
+      native_result = @selector.io_read_object(io, buffer, length, offset)
+      return native_result unless native_result.nil?
+
+      await_background_operation do
+        Fiber.blocking { buffer.read(io, offset, length) }
+      end
+    rescue NoMethodError, TypeError
+      await_background_operation do
+        Fiber.blocking { buffer.read(io, offset, length) }
+      end
+    end
+
+    # Write from a buffer to an IO via the native selector (legacy contract,
+    # Ruby 3.4 through 4.0: length before offset, minimum-progress semantics).
     # Falls back to a background thread for non-socket descriptors.
     # @param io [IO]
     # @param buffer [IO::Buffer]
     # @param length [Integer]
     # @param offset [Integer]
     # @return [Integer] bytes written, or negative errno
-    def io_write(io, buffer, length, offset = 0)
+    def io_write_v3(io, buffer, length, offset = 0)
       native_result = @selector.io_write_object(io, buffer, length, offset)
       return native_result unless native_result.nil?
 
@@ -231,6 +267,38 @@ module CarbonFiber
       end
     end
 
+    # Write from a buffer to an IO via the native selector (Ruby 4.1+
+    # contract: offset before length, single transfer).
+    # @param io [IO]
+    # @param buffer [IO::Buffer]
+    # @param offset [Integer]
+    # @param length [Integer] maximum bytes for this transfer
+    # @return [Integer] bytes written, or negative errno
+    def io_write_v4(io, buffer, offset, length)
+      return 0 if length.zero?
+
+      native_result = @selector.io_write_object(io, buffer, length, offset)
+      return native_result unless native_result.nil?
+
+      await_background_operation do
+        Fiber.blocking { buffer.write(io, offset, length) }
+      end
+    rescue NoMethodError, TypeError
+      await_background_operation do
+        Fiber.blocking { buffer.write(io, offset, length) }
+      end
+    end
+
+    # The public hook names follow the contract of the running Ruby;
+    # both generations stay defined for direct testing.
+    if CarbonFiber.io_contract_v4?
+      alias_method :io_read, :io_read_v4
+      alias_method :io_write, :io_write_v4
+    else
+      alias_method :io_read, :io_read_v3
+      alias_method :io_write, :io_write_v3
+    end
+
     # Blocking IO.select on a background thread.
     def io_select(...)
       await_background_operation do
@@ -238,15 +306,14 @@ module CarbonFiber
       end
     end
 
-    # Cancel pending waiters on an IO and close the descriptor.
-    # @param io [IO]
-    def io_close(io)
-      descriptor = io.respond_to?(:to_i) ? io.to_i : io
+    # Cancel pending waiters on a descriptor and close it.
+    # @param descriptor [Integer]
+    def io_close(descriptor)
       @selector.io_close(descriptor, IOError.new("stream closed while waiting"))
 
       Fiber.blocking do
-        target = io.is_a?(IO) ? io : IO.for_fd(descriptor.to_i)
-        target.close unless target.closed?
+        io = IO.for_fd(descriptor)
+        io.close unless io.closed?
       end
 
       true
@@ -270,14 +337,28 @@ module CarbonFiber
       end
     end
 
-    # Resolve a hostname to addresses via Resolv.
+    # Resolve a hostname to addresses.
+    #
+    # The main Ractor uses Resolv: it speaks DNS over sockets the scheduler
+    # already multiplexes, so lookups never block the loop. Resolv keeps
+    # class-level state that Ractor isolation forbids, so a scheduler running
+    # in another Ractor asks the platform resolver instead. That call blocks
+    # its thread, and from a scheduled fiber it would re-enter this very
+    # hook, so it runs on a background thread like process_wait does.
+    # Unknown hosts yield an empty list on both paths.
     # @param hostname [String]
     # @return [Array<String>]
     def address_resolve(hostname)
       if hostname.include?("%")
         hostname = hostname.split("%", 2).first
       end
-      Resolv.getaddresses(hostname)
+      return Resolv.getaddresses(hostname) if @main_ractor
+
+      await_background_operation do
+        Addrinfo.getaddrinfo(hostname, nil, nil, :STREAM).map(&:ip_address)
+      rescue SocketError
+        []
+      end
     end
 
     # Run an arbitrary callable on a background thread.

data/lib/carbon_fiber/version.rb

@@ -2,5 +2,5 @@
 
 module CarbonFiber
   # @return [String] the current gem version
-  VERSION = "0.1.3"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: carbon_fiber
 version: !ruby/object:Gem::Version
-  version: 0.1.3
+  version: 0.2.0
 platform: ruby
 authors:
 - Yaroslav Markin
@@ -121,6 +121,7 @@ files:
 - README.md
 - lib/carbon_fiber.rb
 - lib/carbon_fiber/async.rb
+- lib/carbon_fiber/io_contract.rb
 - lib/carbon_fiber/native.rb
 - lib/carbon_fiber/native/fallback.rb
 - lib/carbon_fiber/scheduler.rb
@@ -148,7 +149,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.6
+rubygems_version: 4.0.16
 specification_version: 4
 summary: High-performance Ruby Fiber Scheduler backed by Zig with libxev. Pure Ruby
   and gem async.