hyperion-rb 1.2.0 → 1.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4174d7143559b6bd05bdc78acf4377add8aca32f885e933786c50f31c956e9ba
-  data.tar.gz: f163a7f5bd2b363f37205e1f1ba845fb0324c329cc15b4c1144e6d519a1bc60a
+  metadata.gz: 53235f97fb1e384507f62373cd12180ade1eecc8df6f9ce75145ba60403a983e
+  data.tar.gz: 1c58ede296c54a098d26cae2b69837f23bf60fb5ce4239c033446f583f12a4df
 SHA512:
-  metadata.gz: ea61b5e3298ae50b9b6530d51e1f9a5299b0ccfea3b99248230a601a96ebaf764b5d7978215e09a7d73ed7e85ee3f8b5f7d13d40a830ca5c4482a9d192b2919a
-  data.tar.gz: ed8e125b2ff0c9aab53f3178d0f31d1b0db028f8ebf3a40d09ba11e86c3a62756a3c15c7eb3b288faf8dee6f1062159d372cb0c08997a76fedcb97d485d87283
+  metadata.gz: 8484e7168d8ba27312edece5c86af770ef0604bf85d13cdde37c5f4c87b9de0417216f241e073340bedeabef292fc4ec032a7379a76a836e236f6f129c97bcd3
+  data.tar.gz: 89ac23881d0ddd4beff79d08551fa6f7e8399948c1607a3a32475202780170dc9c036f1ef93bd1f17dc5a34e1d91d9901bf3e4119e9efea05d5aa528b22271ff

data/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## [1.3.0] - 2026-04-27
+
+Adds the structural moat for fiber-cooperative I/O. No breaking changes.
+
+### Added
+- **`async_io: true` config flag** (also `--async-io` CLI flag) — when enabled, the plain HTTP/1.1 accept loop runs each connection on a fiber under `Async::Scheduler` instead of handing it to a worker thread. This is what makes [hyperion-async-pg](https://github.com/andrew-woblavobla/hyperion-async-pg) (and other Async-aware libraries) actually cooperate: each fiber yields the OS thread on socket waits, so one thread can serve N concurrent in-flight DB queries instead of 1. **Default off** to keep the 1.2.0 raw-loop perf for fiber-unaware apps. Trade-off: ~5% throughput hit on hello-world; 5–10× throughput on PG-bound workloads when paired with hyperion-async-pg + a fiber-aware connection pool.
+- **Bench validation (macOS, 50ms PG round-trip, 200 concurrent wrk conns):**
+
+  | | r/s | p99 |
+  |---|---:|---:|
+  | Puma 7.2 `-t 5` + plain pg (pool=5) | 88.9 | 2.31 s |
+  | **Hyperion 1.3.0 `--async-io -t 5` + hyperion-async-pg (FiberPool=64)** | **1,103.7** | **237 ms** |
+
+  **12.4× throughput, 9.7× lower p99.** Theoretical ceiling at pool=64 + 50ms query is ~1280 r/s; achieved 86% of it. Linux numbers will land in a follow-up bench section.
+
+### Changed
+- TLS / HTTP/2 paths still always use the Async accept loop (unchanged); they ignore the `async_io` flag because they need the scheduler for ALPN handshake yields and per-stream fiber dispatch anyway.
+- When `async_io: true`, plain HTTP/1.1 dispatch bypasses the thread pool and serves the connection inline on the calling fiber. The pool stays in use for the TLS path's `app.call` hops on each h2 stream.
+
 ## [1.2.0] - 2026-04-27
 
 Production hardening + perf round 2. No breaking changes.

data/README.md

@@ -29,26 +29,27 @@ All numbers are real wrk runs against published Hyperion configs. Hyperion ships
 
 ### Hello-world Rack app
 
-`bench/hello.ru`, single worker, parity threads (`-t 16` vs Puma `-t 16:16`), 4 wrk threads / 50 connections / 10s, macOS arm64 / Ruby 3.3.3:
+`bench/hello.ru`, single worker, parity threads (`-t 5` vs Puma `-t 5:5`), 4 wrk threads / 100 connections / 15s, macOS arm64 / Ruby 3.3.3, Hyperion 1.2.0:
 
-| | r/s | p99 |
-|---|---:|---:|
-| **Hyperion default (logs ON)** | **23,885** | **1.05 ms** |
-| Hyperion `--no-log-requests` | 24,222 | 1.00 ms |
-| Puma `-t 16:16` | 18,794 | 30.89 ms |
+| | r/s | p99 | tail vs Hyperion |
+|---|---:|---:|---:|
+| **Hyperion 1.2.0** (default, logs ON) | **22,496** | **502 µs** | **1×** |
+| Falcon 0.55.3 `--count 1` | 22,199 | 5.36 ms | 11× worse |
+| Puma 7.1.0 `-t 5:5` | 20,400 | 422.85 ms | 845× worse |
 
-**1.27× Puma throughput, ~30× lower p99 — while emitting structured JSON access logs Puma doesn't.**
+**Hyperion: 1.10× Puma throughput, parity with Falcon on throughput, ~10× lower p99 than Falcon and ~845× lower than Puma — while emitting structured JSON access logs the others don't.**
 
 ### Production cluster config (`-w 4`)
 
-Same bench app, `-w 4` cluster, parity threads. macOS arm64:
+Same bench app, `-w 4` cluster, parity threads (`-t 5` everywhere), 4 wrk threads / 200 connections / 15s, macOS arm64:
 
-| | r/s | p99 |
-|---|---:|---:|
-| **Hyperion `-w 4 -t 10`** | **44,221** | **1.15 ms** |
-| Puma `-w 4 -t 10:10` | 37,929 | 17.06 ms |
+| | r/s | p99 | tail vs Hyperion |
+|---|---:|---:|---:|
+| Falcon `--count 4` | 48,197 | 4.84 ms | 5.9× worse |
+| **Hyperion `-w 4 -t 5`** | **40,137** | **825 µs** | **1×** |
+| Puma `-w 4 -t 5:5` | 34,793 | 177.76 ms | 215× worse (1 timeout) |
 
-**1.17× Puma throughput, ~15× lower p99.**
+Falcon edges Hyperion ~20% on raw rps at `-w 4` on macOS hello-world. **Hyperion still leads on tail latency by 5.9× over Falcon and 215× over Puma**, and beats Puma on throughput by 1.15×. On Linux production-config and DB-backed workloads (below) Hyperion takes the rps lead too — the macOS hello-world advantage to Falcon disappears once the workload includes any actual work or the kernel is Linux.
 
 ### Linux production-config (DB-backed Rack)
 
@@ -60,7 +61,37 @@ Same bench app, `-w 4` cluster, parity threads. macOS arm64:
 | Hyperion `--no-log-requests` | 6,364 | 1.114× |
 | Puma `-w 4 -t 10:10` (no per-req logs) | 5,715 | 1.000× |
 
-Bench is network-bound (~3-4 ms median is the PG + Redis round-trip). Hyperion's lead comes from cheaper per-request CPU: lock-free per-thread metrics, per-thread cached iso8601 timestamps in the access log, hand-rolled single-interpolation log line builder, no logger mutex (POSIX `write(2)` atomicity), C-extension response-head builder.
+Bench is **wait-bound** — ~3-4 ms median is the PG + Redis round-trip, dwarfing the per-request CPU work where Hyperion's optimisations live. With a synchronous `pg` driver, fibers don't help: every in-flight DB call still parks an OS thread, and both servers max out at `workers × threads` concurrent queries. To widen this gap requires either an async PG driver — see [hyperion-async-pg](https://github.com/andrew-woblavobla/hyperion-async-pg) (companion gem; pair with `--async-io` and a fiber-aware pool, see "Async I/O — fiber concurrency on PG-bound apps" below) — or a CPU-bound workload, where Hyperion's lead becomes visible (next section).
+
+### Async I/O — fiber concurrency on PG-bound apps
+
+`bench/pg_concurrent.ru` (50 ms PG query per request, pool sized for the server's concurrency model). macOS, Postgres over WAN, wrk `-t4 -c200 -d20s`:
+
+| | r/s | p99 |
+|---|---:|---:|
+| Puma 7.2 `-t 5` + plain pg (pool=5) | 88.9 | 2.31 s |
+| **Hyperion 1.3.0 `--async-io -t 5` + hyperion-async-pg (FiberPool=64)** | **1,103.7** | **237 ms** |
+
+**12.4× throughput, 9.7× lower p99.** Puma is bottlenecked at `threads × 1 in-flight query` because plain `pg` blocks the OS thread on `recv()`. Hyperion + async-pg + a fiber-aware pool decouples concurrency from threads: 5 OS threads serve 64 concurrent in-flight queries via fiber cooperation. Theoretical ceiling at pool=64 + 50 ms query = 1280 r/s; achieved 1103 r/s = 86% of it.
+
+Three things must all be true to get this win:
+1. **`async_io: true`** in your Hyperion config (or `--async-io` CLI flag). Default is off to keep 1.2.0's raw-loop perf for fiber-unaware apps.
+2. **`hyperion-async-pg`** installed: `gem 'hyperion-async-pg', require: 'hyperion/async_pg'` + `Hyperion::AsyncPg.install!` at boot.
+3. **Fiber-aware connection pool.** The popular `connection_pool` gem is NOT — its Mutex blocks the OS thread. Use [`async-pool`](https://github.com/socketry/async-pool), `Async::Semaphore`, or hand-roll one (see `bench/pg_concurrent.ru` for a 30-line FiberPool example).
+
+Skip any of these and you get parity with Puma at the same `-t`. Run the bench yourself: `MODE=async DATABASE_URL=... PG_POOL_SIZE=64 bundle exec hyperion --async-io -t 5 bench/pg_concurrent.ru` (in the [hyperion-async-pg](https://github.com/andrew-woblavobla/hyperion-async-pg) repo).
+
+### CPU-bound JSON workload
+
+`bench/work.ru` — handler builds a 50-key fixture, JSON-encodes a fresh response per request (~8 KB body), processes a 6-cookie header chain. wrk `-t4 -c200 -d15s`, macOS arm64 / Ruby 3.3.3, 1.2.0:
+
+| | r/s | p99 | tail vs Hyperion |
+|---|---:|---:|---:|
+| Falcon `--count 4` | 46,166 | 20.17 ms | 24× worse |
+| **Hyperion `-w 4 -t 5`** | **43,924** | **824 µs** | **1×** |
+| Puma `-w 4 -t 5:5` | 36,383 | 166.30 ms (47 socket errors) | 200× worse |
+
+**1.21× Puma throughput, 200× lower p99.** This is the gap that hides behind PG-round-trip noise on the DB bench. Hyperion's per-request CPU savings (lock-free per-thread metrics, frozen header keys in the Rack adapter, C-ext response head builder, cached iso8601 timestamps, cached HTTP Date header) land on the wire when the workload is CPU-bound. Falcon edges us 5% on raw r/s but with 24× worse tail — a different tradeoff curve. Reproduce: `bundle exec bin/hyperion -w 4 -t 5 -p 9292 bench/work.ru`.
 
 ### Real Rails 8.1 app (single worker, parity threads `-t 16`)
 
@@ -77,6 +108,25 @@ Health endpoint that traverses the full middleware chain (rack-attack, locale re
 
 On Grape and Rails-controller workloads Puma hits wrk's 2 s timeout cap on ~⅔ of requests — its real p99 is censored above 2 s. Hyperion serves all of its requests under 1.2 s with 0 to 16 timeouts. **1.14–1.48× Puma throughput** depending on endpoint.
 
+### Static-asset serving (sendfile zero-copy path, 1.2.0+)
+
+`bench/static.ru` (`Rack::Files` over a 1 MiB asset), `-w 1`, `wrk -t4 -c100 -d15s`, macOS arm64 / Ruby 3.3.3:
+
+| | r/s | p99 | transferred | tail vs winner |
+|---|---:|---:|---:|---:|
+| **Hyperion (sendfile path)** | **2,069** | **3.10 ms** | 30.4 GB | **1×** |
+| Puma `-w 1 -t 5:5` | 2,109 | 566.16 ms | 31.0 GB | 183× worse |
+| Falcon `--count 1` | 1,269 | 801.01 ms | 18.7 GB | 258× worse (28 timeouts) |
+
+Throughput is bandwidth-bound on localhost (≈2 GB/s = the loopback memory ceiling), so the throughput column looks like parity. The actual win is in the **tail latency** column: Hyperion's `IO.copy_stream` → `sendfile(2)` path skips userspace entirely, while Puma allocates a String per response and Falcon serializes more aggressively. On real network paths sendfile widens the gap further (kernel-to-NIC zero-copy).
+
+Reproduce:
+```sh
+ruby -e 'File.binwrite("/tmp/hyperion_bench_asset_1m.bin", "x" * (1024*1024))'
+bundle exec bin/hyperion -p 9292 bench/static.ru
+wrk --latency -t4 -c100 -d15s http://127.0.0.1:9292/hyperion_bench_asset_1m.bin
+```
+
 ### Concurrency at scale (architectural advantages)
 
 These workloads demonstrate structural differences between Hyperion's fiber-per-connection / fiber-per-stream model and Puma's thread-pool model. Numbers are illustrative; the architecture is what matters. Run on Ubuntu 24.04 / Ruby 3.3.3, single worker, h2load `-c <conns> -n 100000 --rps 1000 --h1`.

data/lib/hyperion/cli.rb

@@ -57,6 +57,10 @@ module Hyperion
              'Enable Ruby YJIT (default: auto on RAILS_ENV/RACK_ENV=production/staging)') do |v|
           cli_opts[:yjit] = v
         end
+        o.on('--[no-]async-io',
+             'Run plain HTTP/1.1 connections under Async::Scheduler (required for hyperion-async-pg and other fiber-cooperative I/O; default off)') do |v|
+          cli_opts[:async_io] = v
+        end
         o.on('-h', '--help', 'show help') do
           puts o
           exit 0
@@ -114,7 +118,8 @@ module Hyperion
                           read_timeout: config.read_timeout,
                           max_pending: config.max_pending,
                           max_request_read_seconds: config.max_request_read_seconds,
-                          h2_settings: Master.build_h2_settings(config))
+                          h2_settings: Master.build_h2_settings(config),
+                          async_io: config.async_io)
       server.listen
       scheme = tls ? 'https' : 'http'
       Hyperion.logger.info { { message: 'listening', url: "#{scheme}://#{server.host}:#{server.port}" } }

data/lib/hyperion/config.rb

@@ -31,6 +31,7 @@ module Hyperion
       admin_token: nil, # String. When set, exposes admin endpoints (POST /-/quit triggers graceful drain; GET /-/metrics returns Prometheus-format Hyperion.stats). Same token guards both. nil disables admin entirely (paths fall through to the app).
       max_pending: nil, # Integer, e.g. 256. When the per-worker accept inbox has this many queued connections, additional accepts are rejected with HTTP 503 + Retry-After:1 instead of being queued. nil disables (current behaviour: unbounded queue).
       max_request_read_seconds: 60, # Numeric. Total wallclock budget (seconds) for reading the request line + headers + body for ONE request. Defends against slowloris-style drips that satisfy the per-recv read_timeout but never finish the request. Resets between requests on a keep-alive connection. nil disables.
+      async_io: false, # When true, the plain HTTP/1.1 accept loop runs each connection on a fiber under Async::Scheduler instead of handing it to a worker thread. Required for fiber-cooperative I/O (e.g. hyperion-async-pg). Costs ~5% throughput on hello-world; in exchange one OS thread can serve N concurrent in-flight DB queries on wait-bound workloads. TLS / HTTP/2 paths always use the async loop and ignore this flag.
       h2_max_concurrent_streams: 128, # HTTP/2 SETTINGS_MAX_CONCURRENT_STREAMS — cap on simultaneously-open streams per connection. Falcon: 64. nil leaves protocol-http2 default (0xFFFFFFFF).
       h2_initial_window_size: 1_048_576, # HTTP/2 SETTINGS_INITIAL_WINDOW_SIZE (octets) — flow-control window per stream at open. Bigger = fewer WINDOW_UPDATE round-trips on large bodies. Spec default is 65535. nil → leave protocol default.
       h2_max_frame_size: 1_048_576, # HTTP/2 SETTINGS_MAX_FRAME_SIZE (octets) — biggest DATA/HEADERS frame we'll accept. Spec floor 16384, ceiling 16777215. We pick 1 MiB to match common CDNs without unbounded buffer growth. nil → leave protocol default (16384).

data/lib/hyperion/master.rb

@@ -166,7 +166,8 @@ module Hyperion
           worker_index: worker_index,
           max_pending: @config.max_pending,
           max_request_read_seconds: @config.max_request_read_seconds,
-          h2_settings: Master.build_h2_settings(@config)
+          h2_settings: Master.build_h2_settings(@config),
+          async_io: @config.async_io
         }
         # Hand the inherited socket to the worker in :share mode. In
         # :reuseport mode the worker binds its own with SO_REUSEPORT.

data/lib/hyperion/server.rb

@@ -42,7 +42,7 @@ module Hyperion
 
     def initialize(app:, host: '127.0.0.1', port: 9292, read_timeout: DEFAULT_READ_TIMEOUT_SECONDS,
                    tls: nil, thread_count: DEFAULT_THREAD_COUNT, max_pending: nil,
-                   max_request_read_seconds: 60, h2_settings: nil)
+                   max_request_read_seconds: 60, h2_settings: nil, async_io: false)
       @host                     = host
       @port                     = port
       @app                      = app
@@ -52,6 +52,7 @@ module Hyperion
       @max_pending              = max_pending
       @max_request_read_seconds = max_request_read_seconds
       @h2_settings              = h2_settings
+      @async_io                 = async_io
       @thread_pool              = nil
       @stopped                  = false
     end
@@ -107,16 +108,23 @@ module Hyperion
       listen unless @server
       @thread_pool = ThreadPool.new(size: @thread_count, max_pending: @max_pending) if @thread_count.positive?
 
-      if @tls
+      if @tls || @async_io
         # TLS path: ALPN may pick `h2`, and h2 spawns one fiber per stream
         # inside Http2Handler. Keep the Async wrapper so the scheduler is
         # available for those fibers and for handshake yields.
+        #
+        # async_io: true: operator opt-in for plain HTTP/1.1. The Async wrap
+        # is required when callers want fiber cooperative I/O — e.g.
+        # `hyperion-async-pg` yielding while a Postgres query is in flight.
+        # Pays ~5% throughput vs the raw-loop fast path; in exchange one
+        # OS thread can serve N concurrent in-flight DB queries instead of 1.
         start_async_loop
       else
-        # Plain HTTP/1.1: the worker thread owns each connection for its
-        # lifetime, so the Async wrapper adds zero value (no fibers ever
-        # run on this loop's task). Skip it — pure IO.select + accept_nonblock
-        # shaves measurable overhead off the accept hot path.
+        # Plain HTTP/1.1, async_io: false (default): the worker thread owns
+        # each connection for its lifetime, so the Async wrapper adds zero
+        # value (no fibers ever run on this loop's task). Skip it — pure
+        # IO.select + accept_nonblock shaves measurable overhead off the
+        # accept hot path.
         start_raw_loop
       end
     ensure
@@ -174,6 +182,15 @@ module Hyperion
         # handler still uses the pool's `#call` for app.call hops on each
         # stream (one per stream, not one per connection).
         Http2Handler.new(app: @app, thread_pool: @thread_pool, h2_settings: @h2_settings).serve(socket)
+      elsif @async_io
+        # async_io plain HTTP/1.1: serve inline on the calling fiber so the
+        # request runs *under* Async::Scheduler. This is what makes
+        # hyperion-async-pg (and other Async-aware libraries) actually
+        # cooperate — each fiber yields the OS thread on socket waits, so
+        # one thread can serve N concurrent in-flight DB queries. The
+        # thread pool is intentionally bypassed here: handing the socket
+        # to a worker thread strips the scheduler context.
+        Connection.new.serve(socket, @app, max_request_read_seconds: @max_request_read_seconds)
       elsif @thread_pool
         # HTTP/1.1 (e.g. TLS-wrapped after ALPN picked http/1.1): hand the
         # connection to a worker thread. The fiber that called dispatch

data/lib/hyperion/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Hyperion
-  VERSION = '1.2.0'
+  VERSION = '1.3.0'
 end

data/lib/hyperion/worker.rb

@@ -20,7 +20,7 @@ module Hyperion
                    thread_count: Server::DEFAULT_THREAD_COUNT,
                    config: nil, worker_index: 0, listener: nil,
                    max_pending: nil, max_request_read_seconds: 60,
-                   h2_settings: nil)
+                   h2_settings: nil, async_io: false)
       @host                     = host
       @port                     = port
       @app                      = app
@@ -33,6 +33,7 @@ module Hyperion
       @max_pending              = max_pending
       @max_request_read_seconds = max_request_read_seconds
       @h2_settings              = h2_settings
+      @async_io                 = async_io
     end
 
     def run
@@ -51,7 +52,8 @@ module Hyperion
                           thread_count: @thread_count,
                           max_pending: @max_pending,
                           max_request_read_seconds: @max_request_read_seconds,
-                          h2_settings: @h2_settings)
+                          h2_settings: @h2_settings,
+                          async_io: @async_io)
       tcp_server = @listener || build_reuseport_listener
       server.adopt_listener(tcp_server)
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: hyperion-rb
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.3.0
 platform: ruby
 authors:
 - Andrey Lobanov