hyperion-rb 1.4.0 → 1.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 94ad9628c5683d6b9c9d2d8359066e4bda37d29c7ec2e4fb4e48ae726f8ff939
-  data.tar.gz: 7ebf3c87d10a384e388cf7f2c28d276c0975044a49fe09e1174ffa331cab76b4
+  metadata.gz: 64221d424c994e0757262dca6984cd2b65bf9ec3da6c85094daa717c716da906
+  data.tar.gz: 236188ff1777b49178bb4b2bfe75fbea9309ec6f5d56ec01329943dfa1afbb36
 SHA512:
-  metadata.gz: 9fb844690d5a06aa55335211bdaf3a5276b48f7cfd8a248bb714258ae02549a7e595bc1088987ae9c77d3f28aa1efec91fcd8cc401c26264620f48d78d85faf3
-  data.tar.gz: d050bffe3c8fed2ae1ce434fb7ccf8904a418e3fa18c6afd3e4314b841215ca0dec621edb7bc5f80350e4807e04503b1e785786fb57503b716e52afaf2e30a20
+  metadata.gz: 9b196f8d046c828546f8f5fbac91e0300e8704a70f11b47096a9b0fb05caf2ec34f63e2f33768ab53b41413aaa4e957ac499aa4779ecd54b6a987deef7fd464e
+  data.tar.gz: cb3d64f757736bcbc2632492e24b0794a67c98bfdcae7a1e53d89f583cf2eecc404e060fd3ec10e441ddfdcd11273c41e0d46fa9b80be904b29a7718fe0258ba

data/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## [1.4.1] - 2026-04-27
+
+### Fixed
+- **`Hyperion::Metrics` fiber-key bug** — pre-1.4.1 the metrics module stored counters via `Thread.current[:key]`, which is FIBER-local in Ruby 1.9+. Under an `Async::Scheduler` (TLS / h2 / `--async-io` plain HTTP/1.1) every handler fiber got its own private counters Hash that `Hyperion.stats` could never see — increments were stranded, the dispatch counters and `:bytes_written` etc. read as zero from any non-handler-fiber observer (including the Prometheus `/-/metrics` exporter when scraped from a different fiber). Switched to `Thread#thread_variable_*` (truly thread-local across fibers) plus direct counter-Hash list storage so snapshots also survive thread death. Verified via 4 new specs: cross-fiber on same thread, cross-thread, cross-fiber-on-different-thread, many-fibers-on-same-thread (210 increments aggregated correctly). Surfaced by hyperion-async-pg 0.4.0's bench round, which couldn't read `:requests_async_dispatched` from spec assertions even though the increments were firing.
+
 ## [1.4.0] - 2026-04-27
 
 Default-behaviour change for TLS users: HTTP/1.1-over-TLS now dispatches inline on the calling fiber instead of hopping through the worker thread pool. Fiber-cooperative libraries (`hyperion-async-pg`, `async-redis`) work on the TLS h1 path without `--async-io`. No code-path changes for plain HTTP/1.1 default behaviour.

data/README.md

@@ -95,7 +95,7 @@ Ubuntu 24.04 / 16 vCPU / Ruby 3.3.3, Postgres 17 over WAN, `wrk -t4 -c200 -d20s`
 1. **Linear scaling with pool size** under `--async-io` — `r/s ≈ pool × 12` on this WAN bench. Single-worker pool=200 hits 2381 r/s, **42× Puma `-t 5`** and **5.9× Puma's best** (`-t 30`).
 2. **Mixed workload doesn't kill the win** — Hyperion `--async-io` pool=128 actually goes *up* on mixed (1740 vs 1344 r/s) because CPU work overlaps other fibers' PG-wait windows. This is the honest "what happens to a real Rails handler" answer.
 3. **Hyperion ≈ Falcon within 3-7%** across pool sizes; both fiber-native architectures extract similar value from `hyperion-async-pg`.
-4. **RSS at single-worker scale isn't the architectural moat** — Linux thread stacks are demand-paged; PG connection buffers dominate RSS at pool sizes ≤ 200. The MB-vs-GB story shows up at **idle keep-alive connection scale** (10k+ conns), not in this PG-bound throughput bench. See [Concurrency at scale](#concurrency-at-scale-architectural-advantages) for the connection-count win.
+4. **RSS at single-worker scale isn't the architectural moat** — Linux thread stacks are demand-paged; PG connection buffers dominate RSS at pool sizes ≤ 200. The architectural win is **handler concurrency under load**, not idle memory: Hyperion's fiber path runs thousands of in-flight handler invocations per OS thread, so wait-bound handlers don't queue at `max_threads`. See [Concurrency at scale](#concurrency-at-scale-architectural-advantages) for both the throughput-under-load row and a measured 10k-idle-keepalive RSS sweep against Puma and Falcon.
 5. **`-w 4` cold-start caveat** — multi-worker p99 inflates because the bench rackup uses lazy per-process pool init (each worker pays full pool fill on its first request). Production apps avoid this with `on_worker_boot { Hyperion::AsyncPg::FiberPool.new(...).fill }`.
 
 Three things must all be true to get this win:
@@ -176,7 +176,21 @@ These workloads demonstrate structural differences between Hyperion's fiber-per-
 | Hyperion `-w 1 -t 10` | 93,090 | 6,910 | 3,446 | 27.01 s |
 | Puma `-w 1 -t 10:10`  | 77,340 | 22,660 | 706 | 109.59 s |
 
-Hyperion holds each connection in a ~1 KB fiber stack; Puma needs an OS thread (~1–8 MB each, capped at `max_threads`). At 10k concurrent connections Hyperion serves **~5× the throughput** of Puma with **~20% fewer dropped requests**, while the per-connection bookkeeping cost is bounded by fiber size, not by `max_threads`.
+At 10k concurrent connections under load Hyperion serves **~5× the throughput** of Puma with **~20% fewer dropped requests**. The per-connection bookkeeping cost is bounded by fiber size, not by `max_threads` — workers don't get pinned to long-lived sockets, so a slow handler doesn't starve other connections.
+
+**Memory at idle keep-alive scale — 10,000 idle HTTP/1.1 keep-alive connections:**
+
+Each client opens a TCP connection, sends one keep-alive GET, drains the response, then holds the socket open without sending a follow-up request. RSS is sampled once a second across a 30s idle hold. Same hello-world rackup, single worker, no TLS. Hyperion runs with `async_io true` (fiber-per-connection on the plain HTTP/1.1 path).
+
+| | held | dropped | peak RSS | RSS after drain |
+|---|---:|---:|---:|---:|
+| Hyperion `-w 1 -t 5 --async-io` | 10,000 / 10,000 | 0 | 173 MB | 155 MB |
+| Puma `-w 0 -t 100`               | 10,000 / 10,000 | 0 | 101 MB | 104 MB |
+| Falcon `--count 1`               | 10,000 / 10,000 | 0 | 429 MB | 440 MB |
+
+All three hold 10k idle conns without OOMing or dropping — the "MB-per-thread" intuition that thread-based servers can't reach this scale doesn't survive contact with Linux's demand-paged thread stacks plus Puma's reactor-based keep-alive handling. Per-conn RSS lands at ~14 KB (Hyperion fiber + parser state), ~7 KB (Puma reactor entry + tiny thread share), ~36 KB (Falcon Async::Task + protocol-http stack). Bounded, not unbounded — for all three.
+
+The architectural difference shows up under **load**, not at idle: Puma can only run `max_threads` handler invocations concurrently, so wait-bound handlers (DB, HTTP, Redis) starve at higher request concurrency than `max_threads`. Hyperion's fiber-per-connection model + `--async-io` gives one OS thread thousands of in-flight handler executions, paired with [hyperion-async-pg](https://github.com/exodusgaming-io/hyperion-async-pg) for non-blocking DB. The 10k-conn throughput row above (5× Puma) is the consequence — same idle RSS shape, very different behaviour once the handlers actually do work.
 
 **HTTP/2 multiplexing — 1 connection × 100 concurrent streams (handler sleeps 50 ms):**
 
@@ -194,6 +208,9 @@ Hyperion fans 100 in-flight streams across separate fibers within a single TCP c
 bundle exec ruby bench/compare.rb
 HYPERION_WORKERS=4 PUMA_WORKERS=4 FALCON_COUNT=4 bundle exec ruby bench/compare.rb
 
+# Idle keep-alive RSS sweep (1k / 5k / 10k conns, 30s hold per server)
+./bench/keepalive_memory.sh
+
 # Real Rails / Grape: see bench/db.ru for the schema
 ```
 

data/lib/hyperion/metrics.rb

@@ -7,6 +7,22 @@ module Hyperion
   # all threads that have ever incremented (one short mutex section, only
   # taken when the operator asks for stats).
   #
+  # Storage: counters live behind `Thread#thread_variable_*`, which is the
+  # only TRUE thread-local in Ruby 1.9+ — `Thread.current[:key]` is in fact
+  # FIBER-local, so under an `Async::Scheduler` (TLS path, h2 streams, the
+  # 1.3.0+ `--async-io` plain HTTP/1.1 path) every handler fiber would get
+  # its own private counters Hash that `snapshot` could never find.
+  # Verified with hyperion-async-pg 0.4.0's bench round; before the fix
+  # the dispatch counters dropped requests entirely under `--async-io` and
+  # an external scrape (Prometheus exporter on a different fiber than the
+  # handler) saw the dispatch buckets at zero.
+  #
+  # Cross-fiber races on the same OS thread: the `+=` is technically read-
+  # modify-write, but Ruby's fiber scheduler only preempts at IO boundaries
+  # (Fiber.scheduler-aware system calls), and `Hash#[]=` is purely Ruby —
+  # no preemption mid-increment, no torn writes. Two fibers cannot
+  # interleave a single `+=` on the same OS thread.
+  #
   # Reset semantics: counters monotonically increase. Operators that want
   # rate-of-change should snapshot, sleep, snapshot, diff.
   #
@@ -14,16 +30,40 @@ module Hyperion
   #   Hyperion.stats -> Hash with all current values across all threads.
   class Metrics
     def initialize
-      @threads = Set.new
-      @threads_mutex = Mutex.new
-      # Each Metrics instance has its own thread-local key so spec runs that
-      # build fresh Metrics objects don't share state across examples.
+      # Direct list of every per-thread counters Hash ever allocated through
+      # this Metrics instance. We hold the Hash refs ourselves (instead of
+      # holding Thread refs and looking the Hash up via thread-local
+      # storage) so snapshot survives thread death — counters from a
+      # short-lived worker that already exited still aggregate. Tiny per-
+      # thread footprint (one Hash + one slot in this Array).
+      @thread_counters = []
+      @counters_mutex = Mutex.new
+      # Per-instance thread-local key so spec runs that build fresh Metrics
+      # objects don't share state across examples.
       @thread_key = :"__hyperion_metrics_#{object_id}__"
     end
 
-    # Hot path: one TLS lookup + one hash op. No mutex.
+    # Hot path: one thread-variable lookup + one hash op. No mutex on the
+    # increment fast path; the mutex is taken only on first allocation per
+    # OS thread (very rare) and on snapshot.
+    #
+    # Storage uses Thread#thread_variable_*, which is the only TRUE thread-
+    # local in Ruby 1.9+ — Thread.current[:key] is in fact FIBER-local, so
+    # under an Async::Scheduler (TLS path, h2 streams, the 1.3.0+ --async-io
+    # plain HTTP/1.1 path) every handler fiber would get its own private
+    # counters Hash that snapshot could never aggregate. Verified with
+    # hyperion-async-pg 0.4.0's bench round; before the fix the dispatch
+    # counters dropped requests under --async-io.
+    #
+    # Cross-fiber races on the same OS thread: the `+=` is read-modify-write,
+    # but Ruby's fiber scheduler only preempts at IO boundaries (Fiber-
+    # scheduler-aware system calls). Hash#[]= is purely Ruby — no
+    # preemption mid-increment, no torn writes. Two fibers cannot
+    # interleave a single `+=` on the same OS thread.
     def increment(key, by = 1)
-      counters = Thread.current[@thread_key] ||= register_thread_counters
+      thread = Thread.current
+      counters = thread.thread_variable_get(@thread_key)
+      counters = register_thread_counters(thread) if counters.nil?
       counters[key] += by
     end
 
@@ -37,14 +77,9 @@ module Hyperion
 
     def snapshot
       result = Hash.new(0)
-      @threads_mutex.synchronize do
-        @threads.delete_if { |t| !t.alive? }
-        @threads.each do |t|
-          counters = t[@thread_key]
-          next unless counters
-
-          counters.each { |k, v| result[k] += v }
-        end
+      counters_snapshot = @counters_mutex.synchronize { @thread_counters.dup }
+      counters_snapshot.each do |counters|
+        counters.each { |k, v| result[k] += v }
       end
       result.default = nil
       result
@@ -52,16 +87,17 @@ module Hyperion
 
     # Tests can call .reset! between examples to avoid cross-spec leakage.
     def reset!
-      @threads_mutex.synchronize do
-        @threads.each { |t| t[@thread_key]&.clear }
+      @counters_mutex.synchronize do
+        @thread_counters.each(&:clear)
       end
     end
 
     private
 
-    def register_thread_counters
+    def register_thread_counters(thread)
       counters = Hash.new(0)
-      @threads_mutex.synchronize { @threads << Thread.current }
+      thread.thread_variable_set(@thread_key, counters)
+      @counters_mutex.synchronize { @thread_counters << counters }
       counters
     end
   end

data/lib/hyperion/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Hyperion
-  VERSION = '1.4.0'
+  VERSION = '1.4.1'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: hyperion-rb
 version: !ruby/object:Gem::Version
-  version: 1.4.0
+  version: 1.4.1
 platform: ruby
 authors:
 - Andrey Lobanov