carbon_fiber 0.1.0-aarch64-linux

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: b2168303a8c978a7e889d8767d9df9e1380d58777ab15e07bdb1964a9635ed09
+  data.tar.gz: f693ad61709cc53fee8f9b302eb5869d5253edd76d035457c089422e065f9b60
+SHA512:
+  metadata.gz: f8b12b3107a2fee454ccd5d421484b28d7ee916796aab57c9428d3c4a912fc95718f83b5d3409dbdd386f0b3686900178947ce66481aaa1c8ef441d2b923aefe
+  data.tar.gz: fd9ad628d5545082aa9e2eb6700952c4fc9cd0f8a7bbc7250e83055813a0c56200a219d0d7f235f449d006e59c20a928a9a86074c1e798a943b52cf9c8bcf39a

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Yaroslav Markin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,320 @@
+# Carbon Fiber
+
+**High performance Ruby Fiber Scheduler powered by Zig and libxev.**
+
+<div align="center">
+  <img src="https://raw.githubusercontent.com/yaroslav/carbon_fiber/refs/heads/main/assets/images/carbon_fiber.jpg" width="600" height="400" alt="Carbon Fiber">
+</div>
+
+[![GitHub Release](https://img.shields.io/github/v/release/yaroslav/carbon_fiber)](https://github.com/yaroslav/carbon_fiber/releases)
+[![Docs](https://img.shields.io/badge/yard-docs-blue.svg)](https://rubydoc.info/gems/carbon_fiber)
+
+**Carbon Fiber is a Ruby Fiber Scheduler** with a native event loop (io_uring on Linux, kqueue on macOS). Install it, and your `Net::HTTP`, `TCPSocket`, `Mutex`, `Queue`, `Process.spawn` code becomes concurrent without any changes. Carbon Fiber also **supports the Async framework** ([async gem](https://github.com/socketry/async)) working as a plug and play backend.
+
+By my benchmarks (follows), Carbon Fiber ends up being the **fastest pure Ruby Fiber Scheduler** currently available.
+
+Carbon Fiber is implemented using the **Zig** programming language and is powered by **[libxev](https://github.com/mitchellh/libxev)** by Mitchell Hashimoto, used in his **Ghostty** terminal emulator. It is one of the first Ruby native extensions written in Zig.
+
+## Features
+
+- **Very fast.** My benchmarks place it as overall the fastest pure Ruby Fiber Scheduler. Uses **io_uring on Linux** and **kqueue on macOS**. 
+- **Plug and play with plain Ruby**, thanks to the Fiber Scheduler API—`Net::HTTP`, `TCPSocket`, `Process.spawn`, `Mutex`, `Queue`, DNS—all transparently concurrent under the scheduler; no gem-specific wrappers required.
+- **Async (gem async) support** as a swappable backend. One call swaps the [Async](https://github.com/socketry/async) event loop for ours.
+- **Pure-Ruby fallback**—when the native extension isn't available (Windows, for instance), drops to pure Ruby 4.0+ code behind the same API.
+
+## Example
+
+```ruby
+require "carbon_fiber"
+Fiber.set_scheduler(CarbonFiber::Scheduler.new)
+
+urls.each do |url|
+  Fiber.schedule { Net::HTTP.get(URI(url)) }  # plain Net::HTTP, runs concurrently
+end
+```
+
+---
+
+## Contents
+
+- [Performance](#performance)
+- [Requirements](#requirements)
+- [Installation](#installation)
+- [Usage](#usage)
+- [How It Works](#how-it-works)
+- [Benchmarks](#benchmarks)
+- [Acknowledgements](#acknowledgements)
+- [License](#license)
+
+---
+
+## Performance
+
+AWS EC2 c7a.2xlarge, 8 dedicated vCPUs, Ubuntu 24.04 LTS, kernel 6.17, Ruby 4.0.2 + YJIT, io_uring. 5-run median.
+
+Some benchmarks:
+
+| Workload | Carbon Fiber | Async | Itsi | Carbon Fiber vs. Async |
+|---|---|---|---|---|
+| `http_server` | **48.175k req/s** | 37.409k req/s | 29.708k req/s | +29% |
+| `http_client_api` | **15.528k req/s** | 13.373k req/s | timeout | +16% |
+| `http_client_download` | **6.747k dl/s** | 5.920k dl/s | timeout | +14% |
+| `tcp_echo` | **50.392k ops/s** | 38.907k ops/s | 30.660k ops/s | +29% |
+| `cascading_timeout` | **4.659k ops/s** | 4.488k ops/s | error | +4% |
+| `connection_pool` | **4.989k co/s** | 4.912k co/s | 4.968k co/s | +2% |
+
+On my workloads, wins almost all workloads across Async, Itsi, fiber_scheduler, io-event, and libev. [See detailed benchmarks →](#benchmarks)
+
+---
+
+## Requirements
+
+- Ruby 3.4 or 4.0.
+- Linux (amd64 or arm64), macOS (Apple Silicon, x64 uses fallback*), Windows (fallback*).
+
+Distributed as aleady compiled and ready to use: Zig compiler _not_ needed.
+
+_\* Fallback means pure Ruby code, worse performance._
+
+## Installation
+
+```
+bundle add carbon_fiber
+```
+
+## Usage
+
+### As a standalone fiber scheduler
+
+Ruby's fiber scheduler protocol is built into the standard library. Install Carbon Fiber as the thread's scheduler and every blocking I/O call (`sleep`, `Net::HTTP.get`, `TCPSocket.new`, `Process::Status.wait`) yields automatically—no callbacks, no special wrappers.
+
+See also: [Ruby Fiber Scheduler protocol](https://ruby-doc.org/core/Fiber/Scheduler.html).
+
+#### Example: Parallel file downloads
+
+Fetch a batch of files concurrently. Each fiber yields to the event loop while waiting for data; the scheduler runs all of them on a single thread.
+
+```ruby
+require "carbon_fiber"
+
+require "net/http"
+require "uri"
+
+urls = %w[
+  http://files.example.net/archive-1.tar.gz
+  http://files.example.net/archive-2.tar.gz
+  http://files.example.net/archive-3.tar.gz
+]
+
+scheduler = CarbonFiber::Scheduler.new
+Fiber.set_scheduler(scheduler)
+
+results = {}
+urls.each do |url|
+  Fiber.schedule do
+    results[url] = Net::HTTP.get(URI(url))
+  end
+end
+
+scheduler.run
+Fiber.set_scheduler(nil)
+
+puts "Downloaded #{results.size} files (#{results.values.sum(&:bytesize)} bytes total)"
+```
+
+The `Net::HTTP` code is unchanged from a sequential version. The scheduler intercepts every socket read and write, parks the fiber, and resumes it when the kernel signals readiness—all three downloads proceed in parallel.
+
+#### Example: Parallel subprocess processing
+
+Transcode a batch of video files using multiple `ffmpeg` processes, all monitored concurrently. `Process::Status.wait` yields to the scheduler while the child runs; no polling required.
+
+```ruby
+require "carbon_fiber"
+
+jobs = {
+  "output-1080p.mp4" => ["ffmpeg", "-i", "input.mov", "-vf", "scale=1920:1080", "output-1080p.mp4"],
+  "output-720p.mp4"  => ["ffmpeg", "-i", "input.mov", "-vf", "scale=1280:720",  "output-720p.mp4"],
+  "output-480p.mp4"  => ["ffmpeg", "-i", "input.mov", "-vf", "scale=854:480",   "output-480p.mp4"],
+}
+
+scheduler = CarbonFiber::Scheduler.new
+Fiber.set_scheduler(scheduler)
+
+statuses = {}
+jobs.each do |label, cmd|
+  Fiber.schedule do
+    pid = Process.spawn(*cmd, out: File::NULL, err: File::NULL)
+    statuses[label] = Process::Status.wait(pid)
+  end
+end
+
+scheduler.run
+Fiber.set_scheduler(nil)
+
+statuses.each do |label, status|
+  puts "#{label}: #{status.success? ? "ok" : "exit #{status.exitstatus}"}"
+end
+```
+
+All three `ffmpeg` processes run in parallel. The scheduler uses a background thread per `process_wait` call and parks the fiber until the process exits.
+
+---
+
+### With the Async framework
+
+Carbon Fiber implements the `IO::Event::Selector` interface, so it can replace Async's built-in event loop. Call `CarbonFiber::Async.default!` once at startup; every subsequent `Async { }` block uses our backend.
+
+```ruby
+require "async"
+require "carbon_fiber/async"
+CarbonFiber::Async.default!
+```
+
+Alternatively, set the environment variable: `IO_EVENT_SELECTOR=CarbonFiberSelector ruby app.rb`.
+
+#### Example: Fan-out API calls
+
+Call multiple upstream endpoints in parallel using `Async::Barrier`, then wait for all results. 
+
+```ruby
+require "async"
+require "async/barrier"
+require "carbon_fiber/async"
+
+require "net/http"
+require "json"
+
+CarbonFiber::Async.default!
+
+ENDPOINTS = %w[/users /orders /products /inventory /settings]
+
+Async do
+  barrier = Async::Barrier.new
+  results = {}
+
+  ENDPOINTS.each do |path|
+    barrier.async do
+      response = Net::HTTP.get_response(URI("http://api.example.com#{path}"))
+      results[path] = JSON.parse(response.body)
+    end
+  end
+
+  barrier.wait
+
+  puts "Loaded #{results.size} resources"
+  results
+end
+```
+
+#### Example: Rate-limited concurrent crawl
+
+Fetch many pages concurrently while keeping no more than 10 connections open at once. `Async::Semaphore` limits in-flight requests; `Async::Barrier` tracks completion.
+
+```ruby
+require "async"
+require "async/barrier"
+require "async/semaphore"
+require "carbon_fiber/async"
+
+require "net/http"
+
+CarbonFiber::Async.default!
+
+pages = (1..200).map { |i| "http://data.example.com/records?page=#{i}" }
+
+Async do
+  barrier   = Async::Barrier.new
+  semaphore = Async::Semaphore.new(10, parent: barrier)
+  results   = []
+
+  pages.each do |url|
+    semaphore.async do
+      results << Net::HTTP.get(URI(url))
+    end
+  end
+
+  barrier.wait
+  puts "Fetched #{results.size} pages"
+end
+```
+
+---
+
+## How It Works
+
+The scheduler has two layers:
+
+1. **Zig native core** (`ext/carbon_fiber_native/`) owns the event loop (libxev), ready queue, timer heap, and fiber chaining. Handles `io_wait`, `io_read`, `io_write`, `block`/`unblock`, timer-based sleep, and `timeout_after` directly in native code.
+
+2. **Ruby shell** (`lib/carbon_fiber/scheduler.rb`) implements the Ruby Fiber Scheduler protocol, delegates to the native Selector, and falls back to a background thread for operations the native core doesn't cover (DNS, `process_wait`).
+
+**Fiber chaining:** when a fiber parks, the native event loop calls `rb_fiber_transfer` directly to the next ready fiber, skipping a round-trip through the root fiber. This removes one context switch per scheduling decision on every `sleep`, `read`, and `write`.
+
+If the native extension can't be loaded (on Windows, for example), a pure-Ruby fallback (`lib/carbon_fiber/native/fallback.rb`) provides the same Selector API using threads and condition variables.
+
+---
+
+## Benchmarks
+
+AWS EC2 c7a.2xlarge, 8 dedicated vCPUs,Ubuntu 24.04 LTS, kernel 6.17, Ruby 4.0.2 + YJIT, io_uring. 5-run median.
+
+### Ruby Fiber Schedulers (leading ones): Carbon Fiber vs. Async vs. Itsi
+
+Measuring pure Ruby Fiber Scheduler performance (`Fiber.set_scheduler`).
+
+| Workload | Unit | Carbon Fiber | Async | Itsi | Carbon Fiber vs. Async |
+|---|---|---|---|---|---|
+| `http_client_api` | req/s | **15,528** | 13,373 | timeout | +16% |
+| `http_client_download` | dl/s | **6,747** | 5,920 | timeout | +14% |
+| `http_server` | req/s | **48,175** | 37,409 | 29,708 | +29% |
+| `tcp_echo` | ops/s | **50,392** | 38,907 | 30,660 | +29% |
+| `connection_pool` | co/s | **4,989** | 4,912 | 4,968 | +2% |
+| `fan_out_gather` | cyc/s | 2,024 | 2,046 | **2,104** | −1% |
+| `db_query_mix` | qry/s | 1,660 | 1,652 | **1,662** | +0.5% |
+| `cascading_timeout` | ops/s | **4,659** | 4,488 | error | +4% |
+
+Enabling YJIT turned out to be very beneficial for Async as well—numbers here are with `--yjit` on both sides.
+
+### Async framework: stock Async vs. Async + Carbon Fiber backend
+
+Swapped the io-event selector for Carbon Fiber's native backend. Same Async code.
+
+| Workload | Unit | Stock Async | Carbon Fiber | Delta |
+|---|---|---|---|---|
+| `http_client_api` | req/s | 13,375 | **14,331** | +7.1% |
+| `http_client_download` | dl/s | 3,893 | **3,956** | +1.6% |
+| `task_churn` | task/s | **87,883** | 85,027 | −3.3% |
+| `condition_signal` | sig/s | 337,282 | **361,089** | +7.1% |
+| `cascading_timeout` | ops/s | 4,497 | **4,511** | +0.3% |
+| `tcp_throughput` | ops/s | 42,292 | **51,930** | +22.8% |
+
+### Examples of how to run benchmarks
+
+See [README](benchmarks/README.md) in the `benchmarks/` directory.
+
+```bash
+# macOS smoke test
+benchmarks/bench -t carbon,async -w http_client_api,tcp_echo
+
+# Linux via Docker (io_uring)
+benchmarks/core_docker -t carbon,async,itsi -r 5
+
+# Async framework comparison
+benchmarks/async_docker -r 5
+```
+
+Note that your results will differ depending on the operating system (io_uring or kqueue), system load and hardware, and there will be a lot of variance with certain workloads anyway. Try to benchmark on dedicated hardware, with setup (operating system, kernel) close to your production environment.
+
+---
+
+## Acknowledgements
+
+Thanks to Mitchell Hashimoto for [libxev](https://github.com/mitchellh/libxev) (and Ghostty).
+
+Thanks to furunkel for [zig.rb](https://github.com/furunkel/zig.rb).
+
+Thanks to Samuel Williams for the [Async](https://github.com/socketry/async) framework and [ecosystem](https://github.com/socketry).
+
+## License
+
+MIT

data/lib/carbon_fiber/async.rb

@@ -0,0 +1,256 @@
+# frozen_string_literal: true
+
+# Please note that this code is heavily AI-assisted.
+
+require_relative "native"
+
+module CarbonFiber
+  module Async
+    # IO::Event::Selector-compatible adapter backed by our native Zig selector.
+    # Subclasses Native::Selector so hot-path methods (transfer, yield, wakeup)
+    # dispatch directly to native code without an extra Ruby method frame.
+    #
+    # Registration:
+    #   require "async"
+    #   require "carbon_fiber/async"
+    #   CarbonFiber::Async.default!
+    #
+    # Or via environment:
+    #   IO_EVENT_SELECTOR=CarbonFiberSelector ruby app.rb
+    #
+    class Selector < CarbonFiber::Native::Selector
+      # @return [Float] seconds spent idle in the last {#select} call
+      attr_reader :idle_duration
+
+      # @return [Fiber] the event loop fiber
+      attr_reader :loop
+
+      # @param loop [Fiber] the Async event loop fiber
+      def initialize(loop)
+        super
+        @loop = loop
+        @idle_duration = 0.0
+
+        # Auxiliary ready queue for non-Fiber pushables (e.g. FiberInterrupt)
+        # and cross-thread pushes of non-Fiber objects. Thread-safe via mutex.
+        @auxiliary = []
+        @auxiliary_mutex = Mutex.new
+      end
+
+      # transfer:  inherited from native (no override needed)
+      # yield:     inherited from native (no override needed)
+      # wakeup:    inherited from native (no override needed)
+
+      # Release native resources.
+      def close
+        destroy
+      end
+
+      # Enqueue a fiber or fiber-like object into the ready queue.
+      # @param fiber [Fiber, Object]
+      def push(fiber)
+        if fiber.is_a?(Fiber)
+          super
+        else
+          @auxiliary_mutex.synchronize { @auxiliary << fiber }
+        end
+      end
+
+      # Re-enqueue the current fiber and transfer to +fiber+ with arguments.
+      # @param fiber [Fiber]
+      # @param arguments [Array]
+      def resume(fiber, *arguments)
+        current = Fiber.current
+        native_push(current) unless current.equal?(@loop)
+        fiber.transfer(*arguments)
+      end
+
+      # Re-enqueue the current fiber and raise on +fiber+.
+      # @param fiber [Fiber]
+      def raise(fiber, *arguments, **options)
+        current = Fiber.current
+        native_push(current) unless current.equal?(@loop)
+        fiber.raise(*arguments, **options)
+      end
+
+      # @return [Boolean] whether there is pending work
+      def ready?
+        !@auxiliary.empty? || pending?
+      end
+
+      # --- Event loop ---
+
+      # Run one event loop iteration, draining the auxiliary queue before
+      # and after the native select.
+      #
+      # Note: +idle_duration+ is not actually measured—it stays at 0.0.
+      # Async uses it for load stats only (not correctness), and the two
+      # +Process.clock_gettime+ calls plus Float allocation cost ~1-2%
+      # on select-heavy workloads.
+      # @param duration [Float, nil] maximum seconds to wait
+      def select(duration = nil)
+        drain_auxiliary
+        super
+        drain_auxiliary
+      end
+
+      # --- IO operations ---
+
+      # Wait for I/O readiness. Falls back to IO.select on a background
+      # thread when the native path returns nil (kqueue WRITE bypass,
+      # closed fd, duplicate waiter).
+      # @param fiber [Fiber]
+      # @param io [IO]
+      # @param events [Integer] bitmask of +IO::READABLE+, +IO::WRITABLE+
+      # @return [Integer, false] readiness bitmask, or false on timeout
+      def io_wait(fiber, io, events)
+        result = native_io_wait(fiber, io.fileno, events)
+        return result unless result.nil?
+
+        fallback_io_wait(io, events)
+      end
+
+      # Native Zig io_read/io_write use recv/send with kernel buffer
+      # draining. Falls back to Ruby-level nonblock+io_wait for non-socket
+      # fds (pipes, files) where native returns nil.
+
+      EAGAIN = -Errno::EAGAIN::Errno
+      EWOULDBLOCK = -Errno::EWOULDBLOCK::Errno
+
+      # @param fiber [Fiber]
+      # @param io [IO]
+      # @param buffer [IO::Buffer]
+      # @param length [Integer]
+      # @param offset [Integer]
+      # @return [Integer] bytes read, or negative errno
+      def io_read(fiber, io, buffer, length, offset = 0)
+        result = native_io_read(io.fileno, buffer, length, offset)
+        return result unless result.nil?
+
+        ruby_io_read(fiber, io, buffer, length, offset)
+      end
+
+      # @param fiber [Fiber]
+      # @param io [IO]
+      # @param buffer [IO::Buffer]
+      # @param length [Integer]
+      # @param offset [Integer]
+      # @return [Integer] bytes written, or negative errno
+      def io_write(fiber, io, buffer, length, offset = 0)
+        result = native_io_write(io.fileno, buffer, length, offset)
+        return result unless result.nil?
+
+        ruby_io_write(fiber, io, buffer, length, offset)
+      end
+
+      # Cancel pending waiters and close the descriptor.
+      # @param io [IO]
+      def io_close(io)
+        fd = io.respond_to?(:fileno) ? io.fileno : io.to_i
+        super(fd, IOError.new("stream closed while waiting"))
+      end
+
+      # Wait for a child process on a background thread.
+      # @param fiber [Fiber]
+      # @param pid [Integer]
+      # @param flags [Integer]
+      # @return [Process::Status]
+      def process_wait(fiber, pid, flags)
+        Thread.new do
+          Thread.current.report_on_exception = false
+          Process::Status.wait(pid, flags)
+        end.value
+      end
+
+      private
+
+      def drain_auxiliary
+        return if @auxiliary.empty?
+
+        items = @auxiliary_mutex.synchronize do
+          batch = @auxiliary.dup
+          @auxiliary.clear
+          batch
+        end
+
+        items.each { |item| item.transfer if item.alive? }
+      end
+
+      # Ruby-level io_read/io_write for non-socket fds (pipes, files).
+      # Mirrors io-event's Select fallback selector.
+      def ruby_io_read(fiber, io, buffer, length, offset = 0)
+        total = 0
+
+        IO::Event::Selector.nonblock(io) do
+          while true
+            result = Fiber.blocking { buffer.read(io, 0, offset) }
+
+            if result < 0
+              if result == EAGAIN || result == EWOULDBLOCK
+                io_wait(fiber, io, IO::READABLE)
+              else
+                return result
+              end
+            elsif result == 0
+              break
+            else
+              total += result
+              break if total >= length
+              offset += result
+            end
+          end
+        end
+
+        total
+      end
+
+      def ruby_io_write(fiber, io, buffer, length, offset = 0)
+        total = 0
+
+        IO::Event::Selector.nonblock(io) do
+          while true
+            result = Fiber.blocking { buffer.write(io, 0, offset) }
+
+            if result < 0
+              if result == EAGAIN || result == EWOULDBLOCK
+                io_wait(fiber, io, IO::WRITABLE)
+              else
+                return result
+              end
+            elsif result == 0
+              break
+            else
+              total += result
+              break if total >= length
+              offset += result
+            end
+          end
+        end
+
+        total
+      end
+
+      def fallback_io_wait(io, events)
+        Thread.new do
+          Thread.current.report_on_exception = false
+          readers = (events & IO::READABLE).zero? ? nil : [io]
+          writers = (events & IO::WRITABLE).zero? ? nil : [io]
+          ready = ::IO.select(readers, writers, nil, nil)
+          return false unless ready
+
+          readiness = 0
+          readiness |= IO::READABLE if ready[0]&.include?(io)
+          readiness |= IO::WRITABLE if ready[1]&.include?(io)
+          readiness.zero? ? false : readiness
+        end.value
+      end
+    end
+
+    # Register as the default IO::Event selector for Async.
+    # Call after +require "async"+ so IO::Event::Selector is available.
+    def self.default!
+      IO::Event::Selector.const_set(:CarbonFiberSelector, CarbonFiber::Async::Selector) unless IO::Event::Selector.const_defined?(:CarbonFiberSelector, false)
+      ENV["IO_EVENT_SELECTOR"] = "CarbonFiberSelector"
+    end
+  end
+end