uringmachine 0.23.1 → 0.25.0

data/grant-2025/journal.md

@@ -455,16 +455,180 @@ Ruby I/O layer. Some interesting warts in the Ruby `IO` implementation:
   [Extralite](https://github.com/digital-fabric/extralite/)): normally, using an
   actor interface, or protected by a mutex. I'll try to follow up with a
   benchmark measuring concurrent access to SQLite DBs, similar to the PG one.
-  
+
   Another interesting benchmark I found was one for resolving DNS addresses
   using Ruby's builtin `Addrinfo` API, the bundled `resolv` gem, and a basic DNS
   resolver included in UringMachine (I totally forgot I made one). Here too, I'd
   like to add a benchmark to measure how these different solutions do in a
   highly concurrent scenario.
-  
+
 - Thanks to one of these old benchmarks I made a change that more than doubled
   the performance of `UM#snooze`. What this method does is it adds the current
   fiber to the end of the runqueue, and yields control to the next fiber in the
   runqueue, or to process available CQE's. This method is useful for testing,
   but also for yielding control periodically when performing CPU-bound work, in
   order to keep the application responsive and improve latency.
+
+# 2025-12-14
+
+- Changed how `struct um_op`s are allocated. This struct is used to represent
+  any io_uring operation. It is also used to represent runqueue entries. Now,
+  for most I/O operations, this struct is stack-allocated. But when a new fiber
+  is scheduled, or when using the `#timeout` or any of the `#xxx_async` methods,
+  like `#close_async` or `#write_async`, we need to use a heap-allocated
+  `um_op`, because we don't control its lifetime. In order to minimize
+  allocations, once a `um_op` is done with (it's been pulled out of the
+  runqueue, or its corresponding CQE has been processed), it is put on a
+  freelist in order to be reused when needed. Previously, when the freelist was
+  empty, UringMachine would just allocate a new one using `malloc`. Now
+  UringMachine allocates a array of 256 structs at once and puts all of them on
+  the freelist.
+- Implemented the vectorized versions of `#write` and `#send`, so now one can
+  use `#writev` and `#sendv` to send multiple buffers at once. This could be
+  very useful for situations like sending an HTTP response, which is made of a
+  headers part and a body part. Also, `#writev` and `#sendv` are guaranteed to
+  write/send the entirety of the given buffers, unlike `#write` and `#send`
+  which can do partial write/send (for `#send` you can specify the
+  `UM::MSG_WAITALL` flag) to guarantee a complete send.
+- With the new built-in `Set` class and its new [C
+  API](https://github.com/ruby/ruby/pull/13735), I've switched the internal
+  `pending_fibers` holding fibers waiting for an operation to complete, from a
+  hash to a set.
+
+# 2025-12-15
+
+- Working more with benchmarks, it has occurred to me that with the current
+  design of UringMachine, whenever we check for I/O completions (which is also
+  the moment when we make I/O submissions to the kernel), we leave some
+  performance on the table. This is because when we call `io_uring_submit` or
+  `io_uring_wait_cqes`, we make a blocking system call (namely,
+  `io_uring_enter`), and correspondigly we release the GVL.
+
+  What this means is that while we're waiting for the system call to return, the
+  GVL is available for another Ruby thread to do CPU-bound work. Normally when
+  there's a discussion about concurrency in Ruby, there's this dichotomy: it's
+  either threads or fibers. But as described above, even when using fibers and
+  io_uring for concurrent I/O, we still need to enter the kernel periodically in
+  order to submit operations and process completions. So this is an opportunity
+  to yield the GVL to a different thread, which can run some Ruby code while the
+  first thread is waiting for the system call to return.
+
+  With that in mind, I modified the benchmark code to see what would happen if
+  we run two UringMachine instances on two separate threads. The results are
+  quite interesting: splitting the work load between two UringMachine instances
+  running on separate threads, we get a marked improvement in performance.
+  Depending on the benchmark, we get even better performance if we increase the
+  thread count to 4.
+
+  But, as we increase the thread count, we eventually hit diminishing returns
+  and risk actually having worse performance than with just a single thread. So,
+  at least for the workloads I tested (including a very primitive HTTP/1.1
+  server), the sweet spot is between 2 and 4 threads.
+
+  One thing I have noticed though, is that while the pure UM version (i.e. using
+  the UM low-level API) gets a boost from running on multiple threads, the UM
+  fiber scheduler actually can perform worse. This is also the case for the
+  Async fiber scheduler, so this might have to do with the fact that the Ruby IO
+  class does a lot of work behind the scenes, including locking write mutexes
+  and other stuff that's done when the IO is closed. This is still to be
+  investigated...
+
+# 2025-12-16
+
+- Added `UM#accept_into_queue`, which accepts incoming socket connections in a
+  loop and pushes them to the given queue.
+
+- Improved error handling in the fiber scheduler, and added more tests. There
+  are now about 4.2KLoC of test code, with 255 test cases and 780 assertions. And
+  that's without all the tests that depend on the
+  [`rb_process_new`](https://github.com/ruby/ruby/pull/15213) API, the PR for
+  which is currently still not merged.
+
+- Added a test mode to UringMachine that affects runqueue processing, without
+  impacting performance under normal conditions.
+
+# 2025-12-17
+
+- I noticed that the fiber scheduler `#io_write` was not being called on
+  `IO#flush` or when closing an IO with buffered writes. So any time the IO
+  write buffer needs to be flushed, instead of calling the `#io_write` hook, the
+  Ruby I/O layer would just run this on a worker thread by calling the
+  `#blocking_operation_wait` hook. I've made a
+  [PR](https://github.com/ruby/ruby/pull/15609) to fix this.
+
+# 2025-12-18
+
+- Added a [PR](https://github.com/ruby/ruby/pull/15629) to update Ruby NEWS with
+  changes to the FiberScheduler interface.
+
+- I did some more verification work on the fiber scheduler implementation. I
+  added more tests and improved error handling in read/write hooks.
+
+- Made some small changes to fiber scheduling. I added a test mode which peeks
+  at CQEs on each snooze, in order to facilitate testing.
+
+# 2025-12-20
+
+- Did some more work on benchmarks, and added provisory GVL time measurement.
+
+- Implemented sidecar mode - the basic idea is that UringMachine starts an
+  auxiliary thread that loops entering the kernel with a call to
+  `io_uring_enter` in order to make CQEs available. On return from the system
+  call, it signals through a futex that ready CQEs can be processed.
+
+  On fiber switch, the next fiber to run is shifted from the runqueue. If the
+  runqueue is empty, the UringMachine will wait for the signal, and then process
+  all CQEs. The idea is that in a single threaded environment, under high enough
+  I/O load,  we don't need to release the GVL in order to process ready CQEs,
+  and thus we can better saturate the CPU.
+
+# 2025-12-26
+
+- Finished up the sidecar mode implementation. I did some preliminary benchmarks
+  and this mode does provide a small performance benefit, depending on the
+  context. But for the moment, I consider this mode experimental.
+
+# 2026-01-07
+
+- In the last week I've been working on implementing a buffer pool
+  with automatic buffer manangement. I've been contemplating the design for a
+  few weeks already, and after the vacation has decided the idea is solid enough
+  for me to start writing some code. But let me back up and explain what I'm
+  trying to achieve.
+
+  The io_uring interface includes a facility for setting up buffer rings. The
+  idea is that the application provides buffers to the kernel, which uses those
+  buffers for reading or receiving repeatedly from an fd, letting the
+  application know with each CQE which buffer was used and with how much data.
+  This is particularly useful when dealing with bursts of incoming data.
+
+  The application initiates multishot read/recv operations on each connection,
+  and the kernel has at its disposition a pool of application-provided buffers
+  it can use whenever a chunk of data is read / received. So the kernel consumes
+  those buffers as needed, and fills them with data when it becomes available.
+  Those data will be processed by the application at some later time when it's
+  ready to process CQEs. The application will then add the consumed buffers back
+  to the buffer ring, making them available to the kernel again.
+
+  Multiple buffer rings may be registered by the application, each with a set
+  maxmimum number of buffers and with a buffer group id (`bgid`). The buffers
+  added to a buffer ring may be of any size. Each buffer in a buffer ring also
+  has an id (`bid`). So buffers are identified by the tuple `[bgid, bid]`. When
+  submitting a multishot read/recv operation, we indicate the buffer group id
+  (`bgid`), letting the kernel know which buffer ring to use. The kernel then
+  generates CQEs (completion queue entries) which contain the id of the buffer
+  that contains the data (`bid`). Crucially, a single buffer ring may be used in
+  multiple concurrent multishot read/recv operations on different fd's.
+
+  In addition,on recent kernels io_uring is capable of partially consuming
+  buffers, which prevents wasting buffer space. When a buffer ring is set up for
+  [partial buffer
+  consumption](https://www.man7.org/linux/man-pages/man3/io_uring_setup_buf_ring.3.html),
+  each CQE relating to a multishot read/recv operation will also have a flag
+  telling the application [whether the buffer will be further
+  used](https://www.man7.org/linux/man-pages/man3/io_uring_prep_recv.3.html)
+  beyond the amount of data readily available. Each completion of a given buffer
+  ID will continue where the previous one left off. So it's great that buffer
+  space can be used fully by the kernel, but the application is required to keep
+  track of a "cursor" for each buffer.
+

data/grant-2025/tasks.md

@@ -12,29 +12,39 @@
         https://unixism.net/loti/tutorial/sq_poll.html
   - [v] Add `UM.socketpair`
 
-  - [ ] Add more metrics
+  - [v] Add more metrics
     - [v] runqueue depth
     - [v] number of pending fibers
     - [v] ops: transient count, free count
     - [v] total fiber switches, total waiting for CQEs
-    - [ ] watermark: ops_pending, ops_unsubmitted, ops_runqueue, ops_free, ops_transient
-          (only in profile mode)
-  - [ ] Performance tuning parameters
-    - [ ] max fiber switches before processing CQEs
-    - [ ] max fiber switches before submitting unsubmitted SQEs
-    - [ ] measure switches since last submitting / last CQE processing
-
-  - [ ] Better buffer management buffer rings
+
+  - [v] Make writev automatically complete partial writes
+
+  - [v] Add inotify API
+
+    https://www.man7.org/linux/man-pages/man7/inotify.7.html
+
+  - [ ] Better buffer management
     - [v] Add `UM#sendv` method (see below)
     - [v] Benchmark `#sendv` vs `#send_bundle` (in concurrent situation)
+    - [v] Support for `IO::Buffer`?
     - [ ] Benchmark `#read_each` vs `#read` (in concurrent situation)
-    - [ ] Support for `IO::Buffer`? How's the API gonna look like?
-    - [ ] Some higher-level abstraction for managing a *pool* of buffer rings
-
-  - [ ] Add some way to measure fiber CPU time.
-        https://github.com/socketry/async/issues/428
-
-- [ ] UringMachine Fiber::Scheduler implementation
+    - [ ] Implement automatic buffer pool:
+      - [ ] Automatic buffer allocation,registration and management.
+      - [ ] Support for partial buffer consumption.
+      - [ ] Data processing through a rewritten stream implementation.
+
+  - [v] Sidecar mode
+    - [v] Convert `UM#initialize` to take kwargs
+      - [v] `:size` - SQ entries
+      - [v] `:sqpoll` - sqpoll mode
+      - [v] `:sidecar` - sidecar mode
+    - [v] Sidecar implementation
+      - [v] sidecar thread
+      - [v] futex handling
+      - [v] submission logic
+
+- [v] UringMachine Fiber::Scheduler implementation
   - [v] Check how scheduler interacts with `fork`.
   - [v] Implement `process_wait` (with `rb_process_status_new`)
   - [v] Implement `fiber_interrupt` hook
@@ -97,7 +107,7 @@
     - [v] pipes: multiple pairs of fibers - reader / writer
     - [v] sockets: echo server + many clients
 
-  - [ ] Benchmarks
+  - [v] Benchmarks
     - [v] UM queue / Ruby queue (threads) / Ruby queue with UM fiber scheduler
 
           N groups where each group has M producers and O consumers accessing the same queue.
@@ -124,10 +134,12 @@
   - [v] hook for close
   - [ ] hooks for send/recv/sendmsg/recvmsg
 
-- [ ] SSL
-  - [ ] openssl gem: custom BIO?
+- [v] SSL
+  - [v] setup custom BIO
+  - [v] SSL read/write methods
 
-    - curl: https://github.com/curl/curl/blob/5f4cd4c689c822ce957bb415076f0c78e5f474b5/lib/vtls/openssl.c#L786-L803
+- [v] RDoc
+  - [v] Rewrite README to show some examples
 
 - [ ] UringMachine website
   - [ ] domain: uringmachine.dev

data/lib/uringmachine/dns_resolver.rb

@@ -3,7 +3,13 @@
 require 'resolv'
 
 class UringMachine
+  # A basic DNS resolver implementation.
   class DNSResolver
+
+    # Initializes the DNS resolver.
+    #
+    # @param machine [UringMachine] UringMachine instance
+    # @return [void]
     def initialize(machine)
       @machine = machine
       @requests = UM::Queue.new
@@ -12,11 +18,19 @@ class UringMachine
       @last_id = 0
     end
 
+    # Resolves the given hostname.
+    #
+    # @param hostname [String] hostname
+    # @param type [Symbol] DNS record type
+    # @return [String] IP address
     def resolve(hostname, type)
       @machine.push(@requests, [hostname, type, Fiber.current])
       @machine.yield
     end
 
+    private
+
+    # Handles resolve requests as they come.
     def handle_requests_loop
       while true
         hostname, type, fiber = @machine.shift(@requests)
@@ -25,6 +39,9 @@ class UringMachine
       end
     end
 
+    # Returns an array of nameservers.
+    #
+    # @return [Array<String>] name servers
     def get_nameservers
       nameservers = []
       IO.readlines('/etc/resolv.conf').each do |line|
@@ -35,10 +52,16 @@ class UringMachine
       nameservers
     end
 
+    # Returns a DNS socket fd connected to a name server.
+    #
+    # @return [Integer] fd
     def socket_fd
       @socket_fd ||= prepare_socket
     end
 
+    # Prepares a socket fd connected to a name server.
+    #
+    # @return [Integer] fd
     def prepare_socket
       fd = @machine.socket(UM::AF_INET, UM::SOCK_DGRAM, 0, 0)
       @machine.bind(fd, '0.0.0.0', 0)
@@ -46,6 +69,12 @@ class UringMachine
       fd
     end
 
+    # Resolves a DNS query.
+    #
+    # @param hostname [String] hostname
+    # @param type [Symbol] DNS record type
+    # @param try_count [Integer] retry counter
+    # @return [Array<String>] array of addresses
     def do_resolve(hostname, type, try_count = 0)
       fd = socket_fd
       req = prepare_request_packet(hostname, type)
@@ -67,6 +96,11 @@ class UringMachine
       addrs
     end
 
+    # Prepares a request packet.
+    #
+    # @param hostname [String] hostname
+    # @param type [Symbol] DNS record type
+    # @return [Resolv::DNS::Message]
     def prepare_request_packet(hostname, type)
       msg = Resolv::DNS::Message.new
       msg.id = (@last_id += 1)
@@ -75,6 +109,10 @@ class UringMachine
       msg
     end
 
+    # Returns the message type class.
+    #
+    # @param type [Symbol] DNS record type
+    # @return [Class] message type class
     def msg_type(type)
       # TODO: add support for other types
       Resolv::DNS::Resource::IN::A

data/lib/uringmachine/fiber_scheduler.rb

@@ -5,6 +5,7 @@ require 'etc'
 require 'uringmachine'
 
 class UringMachine
+
   # Implements a worker thread pool for running blocking operations. Worker
   # threads are started as needed. Worker thread count is limited to the number
   # of CPU cores available.
@@ -12,11 +13,13 @@ class UringMachine
 
     # Initializes a new worker pool.
     #
+    # @param max_workers [Integer] maximum worker thread count
     # @return [void]
-    def initialize
+    def initialize(max_workers = Etc.nprocessors)
+      @max_workers = max_workers
       @pending_count = 0
       @worker_count = 0
-      @max_workers = Etc.nprocessors
+
       @worker_mutex = UM::Mutex.new
       @job_queue = UM::Queue.new
       @workers = []
@@ -52,7 +55,7 @@ class UringMachine
 
     # @return [void]
     def run_worker_thread
-      machine = UM.new(4)
+      machine = UM.new(size: 4)
       loop do
         q, op = machine.shift(@job_queue)
         @pending_count += 1
@@ -70,13 +73,13 @@ class UringMachine
     end
   end
 
-  # UringMachine::FiberScheduler implements the Fiber::Scheduler interface for
-  # creating fiber-based concurrent applications in Ruby, in tight integration
-  # with the standard Ruby I/O and locking APIs.
+  # Implements the `Fiber::Scheduler` interface for creating fiber-based
+  # concurrent applications in Ruby, in tight integration with the standard Ruby
+  # I/O and locking APIs.
   class FiberScheduler
 
     # The blocking operation thread pool is shared by all fiber schedulers.
-    @@blocking_operation_thread_pool = BlockingOperationThreadPool.new
+    DEFAULT_THREAD_POOL = BlockingOperationThreadPool.new
 
     # UringMachine instance associated with scheduler.
     attr_reader :machine
@@ -92,8 +95,9 @@ class UringMachine
     #
     # @param machine [UringMachine, nil] UringMachine instance
     # @return [void]
-    def initialize(machine = nil)
+    def initialize(machine = nil, thread_pool = DEFAULT_THREAD_POOL)
       @machine = machine || UM.new
+      @thread_pool = thread_pool
       @fiber_map = ObjectSpace::WeakMap.new
       @thread = Thread.current
     end
@@ -107,7 +111,8 @@ class UringMachine
     # the fiber map, scheduled on the scheduler machine, and started before this
     # method returns (by calling snooze).
     #
-    # @param block [Proc] fiber block @return [Fiber]
+    # @param block [Proc] fiber block
+    # @return [Fiber]
     def fiber(&block)
       fiber = Fiber.new(blocking: false) { @machine.run(fiber, &block) }
 
@@ -145,7 +150,7 @@ class UringMachine
     # @param op [callable] blocking operation
     # @return [void]
     def blocking_operation_wait(op)
-      @@blocking_operation_thread_pool.process(@machine, op)
+      @thread_pool.process(@machine, op)
     end
 
     # Blocks the current fiber by yielding to the machine. This hook is called
@@ -188,7 +193,6 @@ class UringMachine
     # Yields to the next runnable fiber.
     def yield
       @machine.snooze
-      # @machine.yield
     end
 
     # Waits for the given io to become ready.
@@ -196,9 +200,8 @@ class UringMachine
     # @param io [IO] IO object
     # @param events [Number] readiness bitmask
     # @param timeout [Number, nil] optional timeout
-    # @param return
+    # @return [void]
     def io_wait(io, events, timeout = nil)
-      # p(io_wait: io, events:)
       timeout ||= io.timeout
       if timeout
         @machine.timeout(timeout, Timeout::Error) {
@@ -243,7 +246,7 @@ class UringMachine
       length = buffer.size if length == 0
 
       if (timeout = io.timeout)
-				@machine.timeout(timeout, Timeout::Error) do
+        @machine.timeout(timeout, Timeout::Error) do
           @machine.read(io.fileno, buffer, length, offset)
         rescue Errno::EINTR
           retry
@@ -253,6 +256,8 @@ class UringMachine
       end
     rescue Errno::EINTR
       retry
+    rescue Errno => e
+      -e.errno
     end
 
     # Reads from the given IO at the given file offset
@@ -267,7 +272,7 @@ class UringMachine
       length = buffer.size if length == 0
 
       if (timeout = io.timeout)
-				@machine.timeout(timeout, Timeout::Error) do
+        @machine.timeout(timeout, Timeout::Error) do
           @machine.read(io.fileno, buffer, length, offset, from)
         rescue Errno::EINTR
           retry
@@ -277,6 +282,8 @@ class UringMachine
       end
     rescue Errno::EINTR
       retry
+    rescue Errno => e
+      -e.errno
     end
 
     # Writes to the given IO.
@@ -287,12 +294,11 @@ class UringMachine
     # @param offset [Integer] write offset
     # @return [Integer] bytes written
     def io_write(io, buffer, length, offset)
-      # p(io_write: io, length:, offset:, timeout: io.timeout)
       length = buffer.size if length == 0
       buffer = buffer.slice(offset) if offset > 0
 
       if (timeout = io.timeout)
-				@machine.timeout(timeout, Timeout::Error) do
+        @machine.timeout(timeout, Timeout::Error) do
           @machine.write(io.fileno, buffer, length)
         rescue Errno::EINTR
           retry
@@ -302,23 +308,24 @@ class UringMachine
       end
     rescue Errno::EINTR
       retry
+    rescue Errno => e
+      -e.errno
     end
 
     # Writes to the given IO at the given file offset.
     #
     # @param io [IO] IO object
     # @param buffer [IO::Buffer] write buffer
-    # @param length [Integer] file offset
+    # @param from [Integer] file offset
     # @param length [Integer] write length
     # @param offset [Integer] buffer offset
     # @return [Integer] bytes written
     def io_pwrite(io, buffer, from, length, offset)
-      # p(io_pwrite: io, from:, length:, offset:, timeout: io.timeout)
       length = buffer.size if length == 0
       buffer = buffer.slice(offset) if offset > 0
 
       if (timeout = io.timeout)
-				@machine.timeout(timeout, Timeout::Error) do
+        @machine.timeout(timeout, Timeout::Error) do
           @machine.write(io.fileno, buffer, length, from)
         rescue Errno::EINTR
           retry
@@ -328,6 +335,8 @@ class UringMachine
       end
     rescue Errno::EINTR
       retry
+    rescue Errno => e
+      -e.errno
     end
 
     # Closes the given fd.
@@ -335,8 +344,9 @@ class UringMachine
     # @param fd [Integer] file descriptor
     # @return [Integer] file descriptor
     def io_close(fd)
-      # p(io_close: fd)
       @machine.close_async(fd)
+    rescue Errno => e
+      -e.errno
     end
 
     if UM.method_defined?(:waitid_status)
@@ -366,17 +376,17 @@ class UringMachine
     #
     # @param hostname [String] hostname to resolve
     # @return [Array<Addrinfo>] array of resolved addresses
-		def address_resolve(hostname)
-			Resolv.getaddresses(hostname)
-		end
-
-		# Run the given block with a timeout.
-		#
-		# @param duration [Number] timeout duration
-		# @param exception [Class] exception Class
-		# @param message [String] exception message
-		# @param block [Proc] block to run
-		# @return [any] block return value
+    def address_resolve(hostname)
+      Resolv.getaddresses(hostname)
+    end
+
+    # Run the given block with a timeout.
+    #
+    # @param duration [Number] timeout duration
+    # @param exception [Class] exception Class
+    # @param message [String] exception message
+    # @param block [Proc] block to run
+    # @return [any] block return value
     def timeout_after(duration, exception, message, &block)
       @machine.timeout(duration, exception, &block)
     end

data/lib/uringmachine/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 class UringMachine
-  VERSION = '0.23.1'
+  VERSION = '0.25.0'
 end