hyperion-rb 2.11.0 → 2.13.0

data/CHANGELOG.md

@@ -1,5 +1,1084 @@
 # Changelog
 
+## 2.13.0 — 2026-05-01
+
+### 2.13-E — io_uring soak signal + default-ON decision
+
+**Background.** 2.12-D shipped the io_uring accept loop on Linux 5.x+
+(opt-in via `HYPERION_IO_URING_ACCEPT=1`, bench delta:
+15,685 → 134,084 r/s on `handle_static` hello — 8.6× over the 2.12-C
+accept4 fallback, 7× over Agoo's 19,024). The CHANGELOG explicitly
+deferred the default-ON decision to 2.13: "default off until 2.13
+production soak". 2.13-E is that soak — and the harness operators
+need to run their own soak in their own staging.
+
+**What 2.13-E ships.**
+
+1. **`bench/io_uring_soak.sh`** — bash-only soak harness.
+   - Boots Hyperion with `HYPERION_IO_URING_ACCEPT=1 -w 1 -t 32`
+     against `bench/hello_static.ru` (the 2.12-D fast path) on a
+     fixed port. `setsid nohup … & disown` so the master survives
+     SSH disconnect over a 24h run.
+   - 30s warm-up, then `wrk -t4 -c100 -d24h --latency` in the
+     foreground. `SOAK_DURATION` is operator-tunable (defaults to
+     `24h`; the bench-window proof-of-concept was `30m`).
+   - In parallel, every `SAMPLE_INTERVAL` (default 60s), samples
+     `/proc/$PID/status` (VmRSS, VmSize, Threads), `/proc/$PID/fd`
+     count, scrapes `hyperion_requests_dispatch_total` from
+     `/-/metrics`, and bucket-derives p50/p99 from
+     `hyperion_request_duration_seconds_bucket`. Appends one CSV
+     row per sample to `/tmp/io_uring_soak_<tag>_<ts>.csv`.
+   - On exit (24h elapsed OR Ctrl-C), prints summary:
+     min/max/mean/stddev RSS, fd_count peak, p99 stddev/mean, plus
+     wrk's HDR-precision p50/p99/p999 from `--latency`.
+   - **Verdict**:
+     - PASS if RSS variance < 10%, fd peak ≤ `WRK_CONNS + 50`, and
+       (when histogram has ≥ 3 distinct bucket values across the
+       window) p99 stddev/mean < 20%. Eligible for default-flip.
+     - SOAK FAIL on any breach. Defer the flip; the failed metric
+       is documented in the verdict notes.
+     - The histogram p99 check is bypassed when there are < 3
+       distinct bucket values across the soak window — Hyperion's
+       7-edge histogram (1 ms, 5 ms, 25 ms, …) quantizes a stable
+       hello-world p99 into bucket-boundary jumps, and the wrk
+       `--latency` p99 is the right tail-truth source there.
+   - `IO_URING=0` runs the same harness against the 2.12-C accept4
+     fallback so the operator can diff io_uring vs accept4 CSVs
+     apples-to-apples.
+
+2. **`spec/hyperion/io_uring_soak_smoke_spec.rb`** — durable CI
+   coverage. A 1000-request mini-soak over the io_uring loop with
+   a 200-request warm-up, asserts: RSS delta < 20 MB,
+   fd_count back to baseline ± 5, threads back to baseline ± 4.
+   Skipped on macOS / non-liburing builds via the same
+   `Hyperion::Http::PageCache.io_uring_loop_compiled?` predicate the
+   2.12-D wire-shape spec uses. Lives in its own file so a 2.13-E
+   leak signal regression is diagnosable without re-reading the
+   2.12-D wire-shape spec.
+
+   *Calibration note*: the 2.13-E ticket header proposed a 5 MB
+   delta bound. Bench-host measurements (3-run baseline, IO_URING=1,
+   1000 sequential GETs) put the delta at 7-9 MB — but the
+   dominant allocator is the test process itself
+   (`::TCPSocket.new`, `Timeout` threads, response Strings), not the
+   Hyperion server. The 20 MB threshold catches a real Hyperion
+   leak (1 KB/req of leakage = +1 MB at the assertion site) without
+   false-positiving on the test driver's own arena cost.
+
+3. **30-minute proof-of-concept soak** — see the companion
+   `[bench]` commit for the harness-vs-harness numbers across
+   IO_URING=1 / IO_URING=0 and the explicit default-ON decision.
+
+**Constraints respected.** No regression in spec count. macOS-host
+suite: 1124/0/14 → 1126/0/16 (+2 examples, +2 pending — the soak
+smoke is pending on macOS, the documentation example is active). Linux
+bench-host suite: 1124/0/14 → 1126/0/15 (+2 examples, +1 pending —
+the soak smoke runs, the documentation example is pending). The
+2.10-G TCP_NODELAY hunk, 2.10-E preload hooks, 2.10-F
+`rb_pc_serve_request`, 2.11-A dispatch pool warmup, 2.11-B cglue HPACK
+default, 2.12-C accept4 loop, 2.12-D io_uring loop, 2.12-E per-worker
+counter, 2.12-F gRPC unary trailers, 2.13-A metric shards, 2.13-B
+response head builder, 2.13-C flake fixes, 2.13-D gRPC streaming —
+all on master and untouched.
+
+### 2.13-D — gRPC streaming RPCs + ghz vs Falcon bench
+
+**Background.** 2.12-F shipped gRPC unary on h2 — trailers
+(`grpc-status` / `grpc-message` final HEADERS frame), `te: trailers`
+handling, and h2 request half-close semantics. The three remaining
+gRPC call shapes — server-streaming, client-streaming, and
+bidirectional — were not yet wired. 2.13-D closes that gap.
+
+**What was missing.** `dispatch_stream` gated on
+`RequestStream#request_complete` (i.e., END_STREAM on the request),
+which is correct for unary but blocks both streaming-input shapes:
+the app cannot read DATA frames until END_STREAM has already arrived.
+Likewise the response path materialised the full body into a single
+String before splitting it across DATA frames, which folded
+multi-message server-streaming responses into one logical write
+(verified: a 5-message body produced one DATA frame, not five).
+
+**What 2.13-D ships.**
+
+1. **Server-streaming.** When the response body responds to
+   `:trailers`, `dispatch_stream` now iterates `body#each` lazily and
+   emits one DATA frame per yielded chunk (no inter-chunk coalescing),
+   followed by the trailer HEADERS frame carrying END_STREAM=1. A
+   single oversize chunk still gets max-frame-size split inside the
+   per-chunk send path, but small messages stay one DATA frame each.
+   Plain HTTP/2 traffic (no `:trailers` method on the body) keeps the
+   pre-2.13-D buffered shape — no behaviour change for non-gRPC apps.
+
+2. **Streaming-input dispatch.** A new
+   `Hyperion::Http2Handler::StreamingInput` IO-shaped queue replaces
+   the buffered `@request_body` String for requests that look like
+   gRPC: `content-type: application/grpc*` AND `te: trailers` on a
+   POST. When promoted, `process_data` pushes each DATA frame's bytes
+   into the queue (and the END_STREAM frame closes the writer), and
+   the serve-loop dispatches the app on HEADERS arrival via a new
+   `RequestStream#dispatchable?` predicate. The Rack adapter detects
+   the non-String request body and sets `env['rack.input']` directly
+   to the queue (no StringIO wrap, so streaming-read semantics are
+   preserved). Reads block the calling fiber on `Async::Notification`
+   until either bytes arrive or the writer closes.
+
+3. **Bidirectional.** Falls out for free once 1 + 2 are in place —
+   each h2 stream already runs on its own fiber, so concurrent
+   read+write on the same stream is supported by the Async scheduler.
+
+**Tests.** `spec/hyperion/grpc_streaming_spec.rb` (5 examples):
+server-streaming wire shape (5 yielded chunks → 5 DATA frames + trailer
+HEADERS, END_STREAM ride placement asserted), client-streaming
+(5 spaced DATA frames decoded by the app via `rack.input.read`),
+bidirectional (5 round-trips with strict ordering), and 2 unit specs
+on `StreamingInput` (blocking reads + EOF handling, partial-chunk
+slicing). All run end-to-end via `Protocol::HTTP2::Client` over real
+TLS — same harness shape as the 2.12-F unary specs.
+
+**Constraints respected.** No regression in the 1176/0/15 baseline:
+post-2.13-D suite is 1181/0/15 (5 new streaming examples + 0
+regressions). The pre-existing
+`http2_empty_body_short_circuit_spec`'s `FakeStream` test double
+needed a `respond_to?(:streaming_input)` guard at the dispatch
+read-site — added defensively (no protocol change). The 2.10-G
+TCP_NODELAY hunk, 2.10-E preload hooks, 2.10-F `rb_pc_serve_request`,
+2.11-A dispatch pool warmup, 2.11-B cglue HPACK default, 2.12-C accept4
+loop, 2.12-D io_uring loop, 2.12-E per-worker counter, 2.12-F gRPC
+unary trailers, 2.13-A metric shards, 2.13-B response head builder
+C-rewrite, 2.13-C flake fixes are all on master and untouched by
+this change.
+
+**Bench.** See the companion `[bench]` commit for ghz numbers and
+the documented limits of the Hyperion-vs-Falcon comparison.
+
+### 2.13-C — Spec flake hunt
+
+Two flakes carried over from the 2.11/2.12 release cuts. Both are
+spec-side hermeticity issues — neither indicates a regression in
+`lib/` or `ext/`.
+
+**Flake 1 — `spec/hyperion/tls_ktls_spec.rb`, macOS, seed-dependent.**
+The two examples in the `Linux-only: kTLS engages with default :auto policy`
+describe block previously gated their bodies on
+`unless Hyperion::TLS.ktls_supported?`. That probe consults
+`Etc.uname[:sysname]` and `OpenSSL::OPENSSL_VERSION_NUMBER`. The same
+`Etc.uname` is stubbed elsewhere in the suite (`io_uring_spec`)
+to drive the io_uring platform matrix; under a particular full-
+suite seed those stubs and this spec's `before { reset_ktls_probe! }`
+overlapped in a way that let the probe report `true` on the actual
+Darwin host. The body then ran the kTLS-supported branch and failed.
+
+  *Fix shape:* hard `RUBY_PLATFORM.include?('linux')` guard at the
+  example-body top BEFORE the existing probe-based guard. Runtime
+  platform is unstubbable from another spec and matches the
+  example title's intent. Linux runs unaffected (the second guard
+  still protects Linux + old-OpenSSL hosts).
+
+  *Verification:* 10/10 green on macOS arm64; 1/1 green on the
+  Linux x86_64 bench (openclaw-vm, kernel 6.8); full suite holds
+  the 1175 baseline on macOS / 1118 on bench.
+
+**Flake 2 — `spec/hyperion/connection_loop_spec.rb:79`, Linux, deterministic.**
+Misdiagnosed previously as "port-9292-busy". The actual root cause
+is Linux ≥ 5.x's `close()`-doesn't-wake-other-thread-`accept(2)`
+behaviour: when the spec calls `listener.close` from one thread,
+the C accept loop parked in `accept(2)` on that fd in another thread
+stays blocked until the next connection arrives. The `stop_accept_loop`
+flag is checked BETWEEN accepts, not while parked, so flipping it
+without a wake is a no-op. `thread.join(5)` then exhausts its
+timeout and `result` (assigned inside the thread block) is still
+`nil`, breaking the `expect(result).to be_a(Integer)` assertion.
+Other examples in the file had the same teardown shape but happened
+to assert on side-effects populated BEFORE the join, so they passed
+despite the same 5 s thread leak — runtime was 46 s for 10 examples.
+
+  *Fix shape:* extract a `stop_loop_and_wake(listener, thread)`
+  helper that flips the stop flag, dials one throwaway TCP connection
+  at the listener so the parked `accept(2)` returns, then closes the
+  listener and joins. Replace the `stop_accept_loop` + `listener.close`
+  + `thread.join(5)` pattern at every callsite (8 in-file plus the
+  Server-level engagement example, which goes through `server.stop`).
+  Add a regression block — "teardown is hermetic across repeated
+  bring-ups" — that runs the bring-up + serve + teardown cycle 3
+  times in one process and asserts each teardown is < 1 s.
+
+  *Verification:* 0/10 failures on the bench (was 5/5 deterministic
+  failure pre-fix); spec runtime 46 s → 1.3 s; macOS 11/11 green;
+  full suite 1119 on bench / 1176 on macOS (regression spec adds 1).
+
+  *Out of scope:* the same wake-shape affects `Hyperion::Server#stop`
+  in production — `close()` on the listener fd from the signal-
+  handling thread won't reliably wake the worker's parked accept.
+  Flagging this as a follow-up rather than fixing in 2.13-C scope:
+  briefing was explicit ("Don't touch lib/ext code unless the flake
+  is a real bug there"), and the production failure mode (worker
+  hangs on shutdown) is operationally distinct from the spec flake
+  (test-suite stalls).
+
+### 2.13-B — CPU JSON gap
+
+**Background.** The 2.12-B re-bench surfaced one row that got *worse*
+relative to Agoo across the 2.10/2.11/2.12 streams: `bench/work.ru`
+(50-key JSON serialised per-request, no `handle_static` because the
+response varies per request). 2.10-B had Hyperion 3,450 / Agoo 6,374
+(1.85× behind); the 2.12-B re-bench had Hyperion 3,659 / Agoo 7,489
+(2.05× behind). Hyperion +6.0%, Agoo +17.5% over the same window —
+the gap *widened*. None of the 2.10/2.11/2.12 work touched this row,
+so it was the obvious 2.13 follow-on.
+
+**Profile.** `perf record -F 199 -g` on the worker pid while
+`wrk -t4 -c100 -d15s` ran (CPU-JSON workload, default config
+`-t 5 -w 1` = bench harness canonical):
+
+```
+13.15%  vm_exec_core                   (Ruby VM dispatch)
+13.01%  _raw_spin_unlock_irqrestore    (kernel; ~6% inside TCP write softirq)
+ 4.56%  raw_generate_json_string       (JSON.generate — app's own work)
+ 2.28%  generate_json_general          (ditto)
+ 1.75%  vm_call_cfunc_with_frame
+ 1.34%  rb_class_of
+ 1.24%  json_object_i                  (ditto)
+ 1.11%  rb_vm_opt_getconstant_path
+ 1.01%  BSD_vfprintf                   (sprintf — content-length builder + JSON floats)
+ 0.94%  generate_json_float            (ditto)
+ 0.70%  hash_foreach_call              (header iteration)
+ 0.57%  llhttp__internal__run          (Hyperion C ext request parser)
+```
+
+The honest read: ~10 % of CPU is `JSON.generate` (the app's per-request
+work — not removable from inside Hyperion); ~13 % is kernel TCP write
+softirq (one `write(2)` per response — already minimal); ~13 % is the
+Ruby VM dispatch loop, of which the adapter + writer Ruby path is a
+fraction. **The dominant tail is GVL serialisation under `-t 5 -w 1`**
+— a concurrency sweep shows c=1 → 5,800 r/s, c=5 → 3,563 r/s, c=100 →
+3,665 r/s. Hyperion's per-thread workload SCALES DOWN with concurrency
+because every request holds the GVL through `app.call` + `JSON.generate`
+(Ruby code, no I/O wait). Agoo (pure-C HTTP core) scales UP with
+concurrency: c=1 → 4,384 → c=5 → 6,182 → c=100 → 6,519. That structural
+gap cannot close inside Hyperion — `app.call(env)` IS Ruby. What 2.13-B
+*can* do is shrink Hyperion's GVL hold time per request so the ratio
+of "GVL held by Hyperion" to "GVL held by app.call" drops, leaving
+more room for the worker pool to interleave.
+
+### 2.13-B — CPU savings in `cbuild_response_head`
+
+The C-side response-head builder (called once per Rack response) had
+four removable per-request costs:
+
+  1. **Status line `snprintf`.** Every request ran
+     `snprintf("HTTP/1.1 %d ", status)` + `rb_str_cat(reason)` +
+     `rb_str_cat("\r\n", 2)` to build "HTTP/1.1 200 OK\r\n". The 23
+     status codes in `ResponseWriter::REASONS` are a fixed set with
+     fixed reason phrases — the entire status line is a constant per
+     `(status, reason)` pair. The 2.13-B builder switches on `status`
+     and emits the pre-baked line in ONE `rb_str_cat` when the reason
+     phrase matches; falls back to the snprintf path for unknown
+     statuses or operator-overridden reason phrases.
+
+  2. **Header iteration via `rb_funcall(:keys)`.** The legacy iterator
+     called `rb_funcall(rb_headers, :keys, 0)` to materialise a fresh
+     keys Array per request, then `rb_ary_entry(keys, i)` +
+     `rb_hash_aref(rb_headers, key)` per header. The 2.13-B builder
+     uses `rb_hash_foreach`, which walks the hash table directly with
+     no intermediate Array allocation and no per-key hash lookup.
+
+  3. **Per-key `String#downcase` allocation.** Header keys are nearly
+     always frozen-literal Strings in Rack apps (`'content-type'`,
+     `'cache-control'`, …) — same `VALUE` every request. The legacy
+     builder ran `rb_funcall(:downcase)` per key per call, allocating
+     a fresh lowercase String + crossing the FFI boundary. 2.13-B
+     keys an `st_table` on the input String's identity and stores
+     the cached lowercase form + the pre-built `"<lc>: "` prefix
+     buffer; the second-and-later requests for the same frozen-literal
+     key get one st-table hit. Capped at 64 entries — a misbehaving
+     app emitting `x-trace-<uuid>` per request can't grow the cache
+     without bound, just falls back to the per-call downcase.
+
+  4. **Per-(key, value) full-line cache.** When BOTH the key AND the
+     value are frozen-literal Strings (`'cache-control' => 'no-store'`
+     in `bench/work.ru`, `'content-type' => 'application/json'`),
+     the entire wire line `"<lc-key>: <value>\r\n"` is identical
+     every request. 2.13-B caches the prebuilt line keyed on
+     `(key.object_id, value.object_id)`; on hit the entire emit is
+     ONE `rb_str_cat`. Capped at 256 entries with the same fall-back
+     semantics. The CRLF-injection guard always re-runs (the cache
+     stores only validated lines; new (k, v) pairs go through the
+     full validator before the line populates).
+
+  5. **`itoa_positive_decimal` for content-length.** `snprintf("content-
+     length: %ld\r\n", body_size)` was 1 % of CPU on the profile.
+     `body_size` is always non-negative (bytesize of a buffered body)
+     so the sign branch + locale logic in `vfprintf` are pure
+     overhead. 2.13-B writes the digits backwards into a 24-byte
+     stack scratch then `rb_str_cat`s the populated suffix — no
+     heap, no locale, no format-string parser.
+
+**Bench impact.** Same bench host (openclaw-vm, Linux 6.8.0, Ruby
+3.3.3, loopback), each version compiled fresh from source on the
+host before its run.
+
+| Workload | Baseline (master adac63e) | 2.13-B | Δ |
+|---|---:|---:|---|
+| Single-thread synthetic (`Adapter::Rack.call → ResponseWriter#write` against a sink, 50,000 iters; 3-trial median r/s) | 18,018 | **19,404** | **+7.7%** |
+| Multi-thread loopback `wrk -t4 -c100 -d20s --latency`, two batches of 3-trial median r/s | 3,427; 3,550 | 3,440; 3,528 | **−0.1%** |
+| Multi-thread loopback p99 latency | 2.77ms; 2.64ms | 2.74ms; 2.67ms | tied |
+
+The +7.7 % single-thread win is the per-request CPU savings inside
+`cbuild_response_head`. The neutral multi-thread result is the
+GVL-contention floor: at `-t 5 -w 1` Hyperion's worker threads
+serialise on the GVL while running `JSON.generate` + `app.call`,
+so shaving 2-3 µs off Hyperion's slice of the hot path leaves the
+total throughput dominated by `JSON.generate` (~10 % CPU per the
+profile) and the kernel TCP write softirq (~6 %). For comparison
+the same bench host, same Ruby, with Hyperion `-w 4` SO_REUSEPORT
+on `bench/work.ru`: **14,200 r/s** — 2× over Agoo's single-process
+7,489 r/s baseline.
+
+**Honest assessment of the residual gap.** The 2.05× gap to Agoo
+on the canonical `-t 5 -w 1` row is a GVL-architecture gap, not a
+per-request CPU gap. Agoo's pure-C HTTP core lets 5 worker threads
+truly run in parallel; Hyperion's adapter + writer + `app.call`
+hold the GVL together because every step except the `read(2)` /
+`write(2)` syscalls is Ruby. Closing this row to ≥ Agoo would
+require either (a) running `-w 4` SO_REUSEPORT (the 2.12-E
+cluster work — Hyperion DOES exceed Agoo by 2× there), or (b) a
+2.14+ track that moves more of the per-request lifecycle into C
+(e.g. running `cbuild_response_head` from the C accept loop with
+the writer fully C-side). 2.13-B closes Hyperion's portion of the
+GVL hold; the rest is structural.
+
+### 2.13-A — Extend C-side wins to generic Rack apps
+
+**Background.** The 2.12 sprint shipped huge wins on the
+`Server.handle_static`-routed traffic shape: 5,502 r/s → 134,084 r/s on
+the static-route `hello` workload (24× over 2.11.0; 7× over Agoo). But
+those wins are gated on the C accept-loop's `route_table.lookup`
+returning a `RouteTable::StaticEntry`. Generic Rack apps — the vast
+majority of real-world deployments (Rails, Sinatra, Roda, Hanami,
+anything calling `body.each` to yield response chunks) — never engage
+the C loop; they go through `Hyperion::Adapter::Rack` + the Ruby
+accept loop + the thread pool. The 2.12-B re-bench confirmed a generic
+Rack `bench/hello.ru` ran at 4,477 r/s — 4.25× behind Agoo, and 30×
+behind the C-loop static path on the same machine. Most of the 2.12
+wins were not available to operators running real apps.
+
+**What 2.13-A targets.** Optimizations that *do* port to the generic
+Rack dispatch path without breaking semantics. Per-request we don't
+get to skip `app.call(env)` (that IS the dispatch) and we can't
+prebuild the response body (it's dynamic), but we can attack:
+syscall coalescing on accept+read, env hash + rack.input recycling,
+metrics-mutex contention under multi-thread workloads, and the
+keepalive-fast-path tail.
+
+### 2.13-A — Per-thread shard for hot-path metrics
+
+Pre-2.13-A, every `observe_histogram` and `increment_labeled_counter`
+took `@hg_mutex.synchronize`. The original commit comment claimed
+those paths were "low-rate", but that's no longer true:
+
+  * `tick_worker_request` is called once per dispatched request
+    (every `Connection#serve` iteration, every h2 stream, every
+    handed-off connection from the C loop).
+  * `observe_histogram` is called once per dispatched request via
+    the per-route request-duration histogram registered in
+    `Connection#register_request_duration_histogram!`.
+
+Under `-t 32` that single mutex serialised 32 worker threads on the
+request-completion tail — every `+= 1` waited behind the previous
+thread's release. That contention was invisible on the C accept loop
+(the loop bypasses Ruby metrics entirely and folds in its atomic
+counter at scrape time), but it was the dominant tail-latency term
+on the generic Rack workload.
+
+The new path keeps per-thread shards (`Thread#thread_variable_set`,
+true thread-local — NOT fiber-local, matching the unlabeled counter
+convention from 2.0.0) for both `@histograms` and `@labeled_counters`.
+Observations and increments hit the per-thread shard with zero
+contention; `histogram_snapshot` and `labeled_counter_snapshot` merge
+across shards under the mutex (a low-rate operation — once per
+`/-/metrics` scrape).
+
+**Public API stays identical.** `observe_histogram`, `register_histogram`,
+`set_gauge`, `histogram_snapshot`, `labeled_counter_snapshot`,
+`increment_labeled_counter` keep the same signatures and semantics.
+Registered-but-never-observed families still surface in the snapshot
+(pre-2.13-A behaviour). `reset!` now also clears the per-thread
+shards so cross-spec leakage stays prevented.
+
+**Edge cases covered by spec:**
+
+  * Multi-thread observe-then-snapshot: 8 threads × 1000 observations
+    on the same `(name, labels)` produce `count == 8000` and
+    cumulative bucket counts that match.
+  * 16 threads × 500 increments × distinct label values produce 16
+    series with count 500 each.
+  * Concurrent observe + 50 mid-run snapshots run without deadlock
+    or torn counts.
+  * Reset clears across threads.
+  * Unregistered observe is a silent no-op.
+  * Registered-but-never-observed families show up in the scrape
+    with an empty `:series` Hash.
+
+### 2.13-A — Cached worker-id label tuple + Rack-3 keepalive fast path
+
+Two micro-optimizations on the per-request hot path of
+`Connection#serve`, both targeting the steady-state -c1 single-
+keepalive profile (where 8000 r/s = the upper bound of single-thread
+Ruby work and every saved allocation / iteration shows up).
+
+**Cached worker-id label tuple.** `tick_worker_request(@worker_id)`
+went through a wrapper that called `worker_id.to_s` (worker_id is
+already a String) and built a fresh `[label]` Array per request. The
+wrapper also re-checked `@worker_request_family_registered` on every
+call. The new path pre-builds the frozen `[@worker_id]` tuple once in
+the Connection constructor, registers the family once at construction
+too, and the request loop calls `increment_labeled_counter` directly
+with the cached tuple — saving one Array allocation + one method
+dispatch + one early-return-checked branch per request.
+
+**Rack-3 keepalive fast path.** `should_keep_alive?` used to scan the
+entire response-headers Hash with `headers.find { |k,_| k.to_s.downcase
+== 'connection' }`. That ran `to_s.downcase` (one transient String
+allocation) PER iteration and walked to completion on every response
+that didn't carry a `Connection` header (which is most of them — the
+response writer adds its own). Rack 3 mandates lowercase Hash keys
+(spec §6.4), so the new path is a single `headers['connection']`
+lookup. Apps that violate Rack 3 by returning mixed-case keys lose
+the Connection-close response signal and stay on keep-alive — a
+benign degradation pinned by spec; the fix is to update the app to
+spec. Non-Hash header containers (legacy Array-of-pairs) still flow
+through a slow-scan fallback, also case-sensitive on the lowercase
+key.
+
+**Spec coverage:**
+
+  * Lowercase `connection: close` from app closes the connection.
+  * Lowercase `connection: keep-alive` keeps the conn alive across
+    pipelined requests.
+  * Mixed-case `Connection: close` (Rack-3 violation) is documented
+    as falling through to keep-alive — pinned so the behaviour is
+    stable.
+  * `@worker_id_label_tuple` is constructed once, frozen, and reused
+    by identity across requests on the same Connection.
+
+### 2.13-A — Bench result
+
+Bench host: openclaw-vm (Linux 6.8.0, x86_64). Hyperion `-t 32 -w 1`
+on `bench/hello.ru` (generic Rack hello). 5-trial median, wrk 4.x.
+
+**With access logging on (default — JSON access lines per request):**
+
+| Workload   | Master | 2.13-A | Δ |
+|------------|-------:|-------:|---:|
+| -c1 -t1    | 7,631  | 7,386  | -3.2% (within noise) |
+| -c100 -t4  | 4,004  | 4,031  | +0.7% (within noise) |
+
+**With `--no-log-requests` (logging disabled):**
+
+| Workload   | Master | 2.13-A | Δ |
+|------------|-------:|-------:|---:|
+| -c1 -t1    | 9,028  | 8,938  | -1.0% (within noise) |
+| -c100 -t4  | 4,804  | 4,979  | **+3.6%** |
+
+**Honest framing.** The +3.6% at `-c100 --no-log-requests` is the
+clearest signal that the metrics-mutex contention removal lands. At
+`-c1` the workload is single-thread CPU-bound (one wrk thread, one
+keepalive connection, one Hyperion worker thread serving it); the
+optimisations don't help and don't hurt. At `-c100` the
+optimisations reach a reasonable +3.6% on the no-log path; with
+logging on, the access-log path dominates the per-request CPU
+budget so the metrics savings are masked.
+
+**What the bench reveals.** The 4.25× gap to Agoo on the generic
+Rack workload that 2.12-B identified is not a metrics-contention
+problem and not an env-pool problem (env pooling was already in
+place since 1.6.x). It is a **single-thread Ruby work per request**
+ceiling — at `-c1` the bench tops out at ~9,000 r/s = 110 µs/req
+of single-thread Ruby work, which Agoo (Rack-shape C server) does
+in ~52 µs/req. Closing that gap requires moving meaningful chunks
+of the per-request Ruby surface (parser, env build, headers, log
+line) into C — work that's already 50% complete in
+`Hyperion::CParser` (`build_env`, `build_response_head`,
+`build_access_line`). The 2.13 sprint will continue moving the
+remaining Ruby-side pieces (`Connection#serve` request loop,
+ResponseWriter dispatch, `should_keep_alive?`) into C in subsequent
+phases.
+
+**Durable infrastructure.** The 2.13-A optimisations stay in the
+tree even though the headline-bench delta is small. The
+per-thread metrics shard removes a real mutex bottleneck that
+*will* compound under future workloads — Ractor-based dispatch,
+multi-process scrapers, observability-heavy apps that observe
+custom histograms per-request. The Rack-3 keepalive fast path and
+the cached worker-id tuple are pure-quality changes (less code,
+fewer allocations, no API surface change).
+
+## 2.12.0 — 2026-05-01
+
+### 2.12-F — gRPC support on h2
+
+**Background.** Hyperion has shipped a working HTTP/2 stack since 1.6
+(per-stream fiber multiplexing, WINDOW_UPDATE-aware flow control, ALPN
+auto-negotiation, native HPACK via the Rust v3/CGlue codec since 2.5-B).
+What was missing for gRPC over Hyperion was three small things:
+
+  1. **Trailing headers.** gRPC carries its protocol-level status
+     (`grpc-status: 0` / `grpc-message: OK`) as **trailers** — a
+     final HEADERS frame sent AFTER the body's DATA frames, with
+     END_STREAM=1. Hyperion's pre-2.12-F dispatch path always
+     folded END_STREAM onto the LAST DATA frame, so there was no
+     hook for emitting trailers.
+  2. **Binary-clean request body.** The h2 RequestStream initialised
+     `@request_body = +''` (UTF-8). Valid gRPC request bodies
+     (`[1-byte compressed flag][4-byte length-prefix][protobuf bytes]`)
+     contain non-UTF-8 byte sequences, which broke `.bytesize` /
+     `valid_encoding?` checks and corrupted bodies that downstream
+     code interpolated into a UTF-8 String.
+  3. **TE: trailers preservation.** Hyperion's RFC 7540 §8.1.2.2
+     validator already accepted `te: trailers` (any other TE value
+     is rejected as a protocol error per the spec). What was needed
+     was a non-regression spec confirming the header makes it to
+     `env['HTTP_TE']` for the Rack app.
+
+**What's new.** The h2 dispatch path (`Http2Handler#dispatch_stream`)
+now checks if the Rack response body responds to `:trailers` AFTER
+iterating the body. When it does, the wire shape becomes:
+
+```
+HEADERS (no END_STREAM)
+DATA (no END_STREAM on last DATA)
+HEADERS (END_STREAM=1)  ← trailers, e.g. grpc-status: 0
+```
+
+The trailer Hash is filtered defensively before encoding: pseudo-headers
+(`:status` / `:method` / etc.) and connection-specific names
+(`connection`, `transfer-encoding`, …) are stripped — same allow-list
+the regular response-header path uses. A misbehaving app that returns
+non-Hash trailers (or whose `body.trailers` raises) gets a warn log and
+falls back to the no-trailers path; the connection never crashes.
+
+`Http2Handler::RequestStream#@request_body` is now an
+`Encoding::ASCII_8BIT` String, so binary bytes survive verbatim through
+`<<` accumulation and `env['rack.input'].read`.
+
+**What's NOT changed.** The existing wire shape for non-gRPC h2 traffic
+is identical to 2.11.x. A Rack body that doesn't define `:trailers` (or
+where `body.trailers` is `nil` / empty) takes the pre-2.12-F path:
+HEADERS → DATA-with-END_STREAM-on-last. Verified by three non-regression
+specs in `spec/hyperion/grpc_trailers_spec.rb`.
+
+The HTTP/1.1 dispatch path is untouched. gRPC is HTTP/2-only by spec
+(the `Trailer:` mechanism on h1 has interop gaps with Go / Java clients;
+gRPC mandates h2 since v1.0). Hyperion follows suit.
+
+**What's NOT in scope for this stream.** gRPC server streaming /
+client streaming / bidi streaming require Rack 3 streaming bodies AND
+a way for the Rack app to read incoming DATA frames after sending
+response HEADERS. That's a separate stream-control problem (`rack.hijack`
+on h2 needs RFC 8441 Extended CONNECT, same blocker as WebSocket-over-h2).
+2.12-F lands the **unary** (request-response) and **server-side trailers**
+half — which covers ~80% of production gRPC traffic shapes by message
+volume per the public Google / Netflix talks. Streaming is a 2.13
+candidate.
+
+**Opt-in via Rack 3 `body.trailers`.** Apps don't need to know about
+Hyperion. Any Rack 3 body — hand-rolled, `grpc-server-rack`, a future
+`Rack::Trailers` middleware — that exposes `body.trailers` returning
+a Hash gets the trailing HEADERS frame on the wire. Bodies that don't
+expose it (the overwhelming majority of HTTP/2 traffic — Rails,
+Sinatra, asset servers, JSON APIs) take the legacy path with no
+behaviour change.
+
+**Spec coverage.** `spec/hyperion/grpc_trailers_spec.rb` — 11 specs
+covering:
+
+  - End-to-end TLS+h2 client driving a real Hyperion server through
+    `Protocol::HTTP2::Client`, asserting the trailing HEADERS frame
+    decodes to the expected `grpc-status` / `grpc-message` map.
+  - `te: trailers` request header reaches `env['HTTP_TE']`.
+  - Binary request body bytes (`\xFF\x00\x01\x02\xC3\x28\x80`)
+    round-trip verbatim through `env['rack.input'].read`.
+  - Non-regression: bodies without `:trailers` keep the
+    DATA-with-END_STREAM-on-last shape (no extra HEADERS frame).
+  - Defensive: `body.trailers = nil` and `body.trailers = {}` both
+    take the no-trailers path.
+  - Unit tests for `collect_response_trailers` covering nil /
+    raises / Hash-coercible / non-responding bodies.
+
+`spec/hyperion/grpc_smoke_spec.rb` — opt-in (`RUN_GRPC_SMOKE=1`)
+end-to-end smoke against the real `grpc` Ruby gem. Skipped by default
+because the gem pulls protobuf C bindings (~50 MB build); the unit
+specs above are the durable coverage.
+
+**Performance.** Zero hot-path cost when no app uses trailers — the
+`body.respond_to?(:trailers)` probe is one method dispatch, the
+trailers-Hash branch only runs when the probe returns truthy. The
+trailers-emitting path costs one extra encoder-mutex acquisition + one
+extra HEADERS frame per response — negligible against the existing
+HEADERS+DATA sequence. No bench numbers here because gRPC throughput
+on Hyperion is dominated by the application's protobuf marshalling,
+not the framing layer; a meaningful gRPC bench is a 2.13 follow-up
+once we have a streaming story to compare against Falcon's
+`async-grpc`.
+
+### 2.12-E — SO_REUSEPORT load balancing audit
+
+**Background.** Hyperion's cluster mode (`-w N`) uses two listener-FD
+models depending on platform:
+
+  - **Linux:** `SO_REUSEPORT`. Every worker calls `bind()` on the same
+    port; the kernel does in-kernel load balancing across connected
+    workers based on a 4-tuple hash. Documented as the
+    production-recommended setup since rc15.
+  - **Darwin / BSD:** master-bind + worker-fd-share. The master process
+    binds the listener and passes the fd to workers via `fork()`;
+    workers race to `accept()`. Darwin's `SO_REUSEPORT` doesn't
+    load-balance — it just lets multiple sockets bind without
+    `EADDRINUSE` — so we deliberately do NOT use it on Darwin.
+
+The Linux path was built on the assumption that the kernel hash
+distributes uniformly across workers under sustained load. We had no
+**measurement** of that — only theory. 2.12-E fixes the gap.
+
+**New per-worker request counter.** `Hyperion::Metrics#tick_worker_request`
+ticks the labeled counter family
+`hyperion_requests_dispatch_total{worker_id="<pid>"}` once per
+dispatched request, regardless of which dispatch shape served it:
+
+  - Rack-via-`Connection#serve` (the threadpool / inline / async-io
+    paths).
+  - HTTP/2 stream dispatch in `Http2Handler`.
+  - The 2.12-C `accept4` C accept loop.
+  - The 2.12-D io_uring accept loop.
+
+The C-loop variants tick a process-global atomic counter
+(`Hyperion::Http::PageCache.c_loop_requests_total`) that the
+`PrometheusExporter.render_full` pass folds into the
+per-worker series at scrape time — so operators see one consistent
+per-PID count even on the C-only fast path that bypasses
+`Connection#serve`. Hot-path cost on the C loops is one
+`__atomic_add_fetch` (relaxed) — negligible against the 134k r/s
+2.12-D bench peak.
+
+The label value is `Process.pid.to_s` — matches the 2.4-C
+`hyperion_io_uring_workers_active` and
+`hyperion_per_conn_rejections_total` labeling convention; lets
+operators correlate cluster-mode rows with `ps`/`/proc` data without
+a separate worker_id ↔ pid mapping table.
+
+**New bench harness.** `bench/cluster_distribution.sh` boots
+Hyperion `-w 4 -t 1 -p 9292 bench/hello_static.ru` (4 workers,
+sharp imbalance signal), runs `wrk -t8 -c200 -d30s` against `/`,
+then scrapes `/-/metrics` repeatedly until all 4 workers have
+responded. Reports the per-worker request distribution
+(pid + count + share-%), mean / stddev / max-vs-min ratio, and a
+verdict:
+
+  - `balanced` (max/min ≤ 1.10): kernel hash is doing its job.
+  - `mild` (1.10 < max/min ≤ 1.50): note here, no fix.
+  - `severe` (max/min > 1.50): file follow-up; do not ship as-is.
+
+Three runs back-to-back so noise is visible; aggregate verdict is
+the worst per-run verdict (one severe run is enough to fail the
+audit).
+
+**Bench result.** See `docs/BENCH_HYPERION_2_11.md`'s "Cluster
+distribution audit" section. Headline number lands in the
+`[bench][2.12.0] 2.12-E — bench result: …` follow-up commit.
+
+**Darwin caveat.** The Darwin master-bind/worker-fd-share path is
+documented as known-imbalanced and is NOT covered by this audit
+(no kernel SO_REUSEPORT distributor to measure). The metric
+infrastructure is platform-independent; any operator running on
+Darwin can read their own per-worker imbalance via `/-/metrics` and
+the same `hyperion_requests_dispatch_total{worker_id}` series.
+
+**Spec coverage.** `spec/hyperion/metrics_per_worker_request_count_spec.rb`
+asserts:
+
+  - `Metrics#tick_worker_request` registers the labeled family with
+    `worker_id` label and increments the right series.
+  - `Connection#serve` ticks the counter once per request from any
+    dispatch shape (regular Rack, direct dispatch, StaticEntry fast
+    path).
+  - `PrometheusExporter.render_full` emits the merged
+    `hyperion_requests_dispatch_total{worker_id="<pid>"}` line.
+  - The C accept4 loop's atomic counter ticks per request and is
+    reset on `run_static_accept_loop` entry. (Gated on the C ext
+    being available; same gating pattern 2.12-D specs use.)
+
+### 2.12-D — io_uring accept loop (Linux 5.x)
+
+The 2.12-C `accept4` loop closed most of the gap to Agoo on the
+`handle_static` hello row (5,502 → 15,685 r/s, 1.21× from parity).
+Looking at the remaining cost on Linux: the worker thread does
+**three** syscalls per request — `accept4` + `recv` + `write`. On a
+host that delivers ~150 ns of syscall entry/exit overhead each, that's
+~450 ns of pure kernel-mode bookkeeping per request, before any I/O
+actually happens.
+
+io_uring lets us collapse the train. We submit ACCEPT/RECV/WRITE/CLOSE
+SQEs to a per-worker ring; the worker thread does **one**
+`io_uring_enter` per cycle and reaps N completions in a single
+syscall round-trip. Steady-state cost goes from N×3 syscalls to
+~N÷K × 1 syscall (where K is the burst-batch size the kernel
+naturally delivers).
+
+**New C exports** on `Hyperion::Http::PageCache`:
+
+  - `run_static_io_uring_loop(listen_fd) -> Integer | :crashed | :unavailable`
+    drives the io_uring accept-and-serve loop. Same wire contract as
+    `run_static_accept_loop`: only `handle_static` routes, plain TCP,
+    GET/HEAD without a body, HTTP/1.1 only. Returns `:unavailable` if
+    the C ext was built without `liburing` OR the runtime
+    `io_uring_queue_init` probe failed (seccomp / locked-down
+    container / kernel < 5.5). The Ruby caller treats `:unavailable`
+    as "fall through to the 2.12-C `accept4` path" — operator gets a
+    one-line warn-level log, no surprise downtime.
+  - `io_uring_loop_compiled?` — boolean, true when the C ext was
+    built with `HAVE_LIBURING`. Cheap eligibility check that lets
+    `ConnectionLoop#io_uring_eligible?` skip the env-var read on
+    builds where the path can't engage anyway.
+
+**Build.** `extconf.rb` probes for `liburing` in two passes:
+
+  1. `pkg-config --exists liburing` — picks up Debian/Ubuntu's
+     pkg-config metadata and adds the right `-L`/`-l` flags.
+  2. `have_header('liburing.h')` + `have_library('uring', ...)` —
+     covers the no-pkg-config path.
+
+On success, `-DHAVE_LIBURING` lands and the io_uring loop compiles.
+On failure, the same `io_uring_loop.c` translation unit compiles to
+a thin stub that returns `:unavailable`. Soft-optional dependency:
+hosts without liburing-dev still build cleanly and ship the 2.12-C
+behaviour.
+
+**Wiring.** `Hyperion::Server::ConnectionLoop.io_uring_eligible?`
+returns true when ALL hold:
+
+  1. The C accept-loop path is available
+     (`ConnectionLoop.available?`).
+  2. The C ext was compiled with `HAVE_LIBURING`.
+  3. `HYPERION_IO_URING_ACCEPT=1` is set (default OFF in 2.12 — the
+     soak window before flipping the default to ON is 2.13 or later).
+
+When eligible, `Server#run_c_accept_loop` calls
+`run_static_io_uring_loop` first; on `:unavailable` (runtime probe
+fail) it falls through to `run_static_accept_loop`. A new
+`:c_accept_loop_io_uring_h1` `DispatchMode` symbol counts engagements
+under `requests_dispatch_c_accept_loop_io_uring_h1`.
+
+**Lifecycle hooks.** Same contract as 2.12-C: per-request
+`Runtime#fire_request_start` / `#fire_request_end` fire on every
+request the io_uring loop served. `env=nil`, minimal `Hyperion::Request`
+passed. The C-side `lifecycle_active?` flag gates the GVL re-acquisition
+(`rb_thread_call_with_gvl`); the no-hook hot path stays GVL-free for
+the whole `submit_and_wait` cycle.
+
+**Handoff.** Same set of "send to Ruby" triggers as 2.12-C: HTTP/1.0,
+`Content-Length` / `Transfer-Encoding`, `Upgrade`, `HTTP2-Settings`,
+`Connection: upgrade`, malformed framing, header section >64 KiB,
+unknown method, path miss. Ruby resumes ownership via the same
+`dispatch_handed_off` path the 2.12-C loop already uses.
+
+**TCP_NODELAY.** Applied per-accept via the shared
+`pc_internal_apply_tcp_nodelay` wrapper — preserves the 2.10-G hunk.
+
+**Tests.** `spec/hyperion/io_uring_loop_spec.rb` covers the Ruby
+surface (always exposed), the stub `:unavailable` return on
+non-Linux / no-liburing builds, eligibility-gate semantics
+(`HYPERION_IO_URING_ACCEPT` env var honoured, compile-time flag
+honoured), and — gated on `HAVE_LIBURING` — a smoke test (5 GETs,
+assert served count grows), lifecycle-hook firing parity with the
+2.12-C contract, and Server-level engagement (the
+`:c_accept_loop_io_uring_h1` dispatch mode is recorded).
+
+Total: 1065 specs / 0 failures / 15 pending on macOS — +10 specs and
++4 pending over the 2.12-C macOS baseline (1055 / 0 / 11). Linux:
+1065 / 1 / 14 (the one failure is a pre-existing flake on the 2.12-C
+smoke spec — reproduces on unmodified master at e526ef3 on the bench
+host, NOT a 2.12-D regression).
+
+**Bench (3 trials, median, openclaw-vm 16 vCPU Ubuntu 24.04 Linux 6.8.0,
+liburing 2.5, wrk -t4 -c100 -d20s, `bench/hello_static.ru`):**
+
+| Variant | r/s (median) | p99 |
+|---|---:|---:|
+| 2.12-C `handle_static` Hyperion (C accept loop, accept4) | 15,532 | 101 µs |
+| **2.12-D `handle_static` Hyperion (C accept loop, io_uring)** | **134,084** | **0.99 ms** |
+| Agoo 2.15.14 (reference, prior bench) | 19,024 | 10.47 ms |
+
+`handle_static` hello: **8.6× over the 2.12-C accept4 baseline**
+(15,532 → 134,084 r/s) and **7.0× over Agoo** (134,084 vs. 19,024).
+The accept4 path is unchanged and within bench noise of the prior
+2.12-C 15,685 r/s baseline (15,532 r/s today is -1%, well inside the
+±5% target).
+
+The p99 climb (101 µs accept4 → 0.99 ms io_uring) is the cost of
+batching: with one `io_uring_enter` reaping K completions, the
+last-enqueued request waits for the kernel to produce all K CQEs
+before our worker drains. At 134k r/s the per-request median is
+~7.4 µs; the p99 of ~990 µs is roughly the steady-state batch-tail
+(~130 requests' worth of completions). For the smoke-class workload
+this is a clear win — sub-millisecond p99 is below most real-world
+network jitter floors and well below Agoo's 10.47 ms p99.
+
+The Rack-style fallback (`bench/hello.ru`, no `handle_static`) is
+unaffected: io_uring engagement requires `Server.handle_static`
+registration, the regular dynamic-Rack path is unchanged.
+
+**Production rollout.** Default OFF for 2.12.0. Operators opt in
+with `HYPERION_IO_URING_ACCEPT=1` per worker. Soak window for
+flipping the default to ON is 2.13 — kernel io_uring has known
+sharp edges around fork-shared rings and `seccomp` policies that
+this default-off posture lets us discover under operator A/B
+before forcing the path on every install.
+
+### 2.12-C — Connection lifecycle in C
+
+The 2.12-B re-bench made it clear that Hyperion's `handle_static`
+hot path was already doing the **response side** in C
+(`PageCache.serve_request`, 2.10-F), but the **connection
+lifecycle around it** — accept loop, route lookup, per-`Connection`
+ivar init — was still in Ruby. The 2.10-D / 2.10-F bench analysis
+flagged that as the next dominant cost: `accept4` + `clone3`
+(worker thread wakeup) + `futex` (GVL handoff) + Ruby ivar init
+on every connection.
+
+This stream pushes the entire accept→read→route-lookup→write
+loop into C for `handle_static`-routed paths.
+
+**New C exports** on `Hyperion::Http::PageCache`:
+
+  - `run_static_accept_loop(listen_fd) -> Integer | :crashed`
+    drives the accept-and-serve loop. Releases the GVL during
+    `accept(2)`, `recv(2)`, and `write(2)`; re-acquires only for
+    the registered lifecycle / handoff Ruby callbacks. Returns
+    the count of requests served when the listener closes
+    cleanly, or `:crashed` if an unrecoverable accept error
+    happened (Ruby falls back to its own accept loop on this
+    sentinel).
+  - `set_lifecycle_callback(callable)` and
+    `set_lifecycle_active(bool)` — register / gate the per-request
+    lifecycle hook. Decoupled so flipping the gate at runtime
+    doesn't re-allocate the callback. The hot path checks a
+    single `int` flag; the no-hook path stays one syscall (recv
+    or write) per request.
+  - `set_handoff_callback(callable)` — register the callback the
+    C loop invokes when a connection's first request can't be
+    served from the static cache (path miss, malformed request,
+    body present, h2 upgrade requested, HTTP/1.0). Receives
+    `(fd_int, partial_buffer_or_nil)`; Ruby owns the fd from
+    that point on and resumes ownership via the regular
+    `Connection` path.
+  - `stop_accept_loop` and `lifecycle_active?` — operator /
+    spec helpers.
+  - `handoff_to_ruby(client_fd, partial_buffer, partial_len)` —
+    echo helper exposing the 2.12-C contract for spec-time
+    introspection (the actual handoff happens inside
+    `run_static_accept_loop` via the registered callback).
+
+**Wiring.** `Hyperion::Server::ConnectionLoop` (new module) holds
+the engagement check + callback factories. `Server#start_raw_loop`
+engages the C loop when:
+
+  1. The listener is plain TCP (no TLS, no h2 ALPN dance).
+  2. The route table has at least one
+     `RouteTable::StaticEntry` registration.
+  3. The route table has NO non-StaticEntry registrations
+     (any dynamic handler disables the C path; the C loop
+     only knows how to write prebuilt responses).
+  4. The C ext is available (the `Hyperion::Http::PageCache`
+     module responds to `:run_static_accept_loop`) and the
+     `HYPERION_C_ACCEPT_LOOP=0` escape hatch is not set.
+
+A new `:c_accept_loop_h1` `DispatchMode` symbol ships under
+`requests_dispatch_c_accept_loop_h1` (one bump per worker boot
+that engages the path) plus `c_accept_loop_requests` (count of
+requests the C loop served on this worker, bumped at loop exit).
+
+**Lifecycle hooks.** The 2.10-D contract holds: per-request
+`Runtime#fire_request_start` / `#fire_request_end` fire on every
+request the C loop served. `env=nil`, `path=` the matched path
+string — same surface as the Ruby direct-dispatch path. The
+Ruby-side bridge in `ConnectionLoop.build_lifecycle_callback`
+builds a minimal `Hyperion::Request` for the hooks to consume.
+The C-side `lifecycle_active?` flag gates the `rb_funcall`;
+when no hooks are registered, the no-hook hot path is one
+syscall (recv) + one syscall (write) per request, with **zero
+Ruby method invocations**.
+
+**Handoff.** When the C loop sees a request it can't serve from
+the static cache (POST, path miss, malformed framing,
+`Content-Length`, `Transfer-Encoding`, `Upgrade`,
+`HTTP2-Settings`, `Connection: upgrade`, HTTP/1.0), it invokes
+the handoff callback with `(fd_int, partial_buffer_str)` and
+continues to the next accept. Ruby resumes ownership of the fd
+via `dispatch_handed_off`, which wraps the fd in `Socket.for_fd`,
+applies the read timeout (matches the regular Ruby accept
+path), pre-loads the partial buffer onto the Connection's
+`@inbuf` ivar so the parser sees the bytes the C loop already
+consumed, and dispatches through the existing thread-pool /
+inline path.
+
+**Tests.** `spec/hyperion/connection_loop_spec.rb` covers
+smoke (registered route → C loop served-count grows), mixed
+registered + unregistered (C loop hands off to the Ruby callback
+spy with the partial buffer + fd), body-present requests
+(`POST /hello` hands off), lifecycle hooks
+(`set_lifecycle_active(true)` fires the registered callback
+once per served request; `false` is silent — no callback
+invocation, even with one set), GVL release (a slow C-loop
+parked on `accept(2)` doesn't block another Ruby thread doing
+arithmetic), keep-alive on a single connection (two pipelined
+requests on the same socket → two responses), Server-level
+engagement (only-static-routes engages the C loop;
+mixed-with-dynamic-handlers refuses to engage and falls back
+to the regular Ruby loop).
+
+Total: 1112 specs / 0 failures / 11 pending — +10 specs over
+the 2.12-B baseline (1102 / 0 / 11).
+
+**Bench (3 trials, median, openclaw-vm 16 vCPU Ubuntu 24.04 Linux 6.8.0,
+wrk -t4 -c100 -d20s, `bench/hello_static.ru`):**
+
+| Variant | r/s (median) | p99 |
+|---|---:|---:|
+| 2.12-B `handle_static` Hyperion (Ruby accept loop + C `serve_request`) | 5,502 | 1.59 ms |
+| **2.12-C `handle_static` Hyperion (C accept loop)** | **15,685** | **107 µs** |
+| Agoo 2.15.14 (reference) | 19,024 | 10.47 ms |
+
+`handle_static` hello: **2.85× over the 2.12-B baseline** (5,502 → 15,685).
+Gap to Agoo's 19,024 r/s closed from **3.46×** to **1.21×** — the
+2.12-C path is now within striking distance of Agoo on this row.
+Tail latency (p99) is now **107 µs**, an **15× improvement** over
+the 2.12-B 1.59 ms p99 and **97× better** than Agoo's 10.47 ms p99.
+
+The Rack-style fallback (`bench/hello.ru`, no `handle_static`) was
+re-checked: 4,648 r/s median (vs 4,477 r/s 2.12-B baseline — within
+bench noise, no regression). The C-loop path is only engaged when
+the operator opts in via `Server.handle_static` registration on
+every route; the regular dynamic-Rack path is unchanged.
+
+### 2.12-B — Fresh 4-way re-bench (post-2.10/2.11 wins)
+
+The 4-way head-to-head in `docs/BENCH_HYPERION_2_0.md`
+§ "4-way head-to-head (2.10-B baseline, 2026-05-01)" was
+captured before the 2.10-C / 2.10-D / 2.10-E / 2.10-F /
+2.10-G / 2.11-A / 2.11-B wins landed. Every Hyperion column
+in that doc was therefore stale; the Puma / Falcon / Agoo
+columns were unchanged (no version bumps on the bench host).
+This stream re-runs the entire 6-row matrix on the 2.11.0
+codebase — same host (`openclaw-vm`, 16 vCPU, Ubuntu 24.04,
+Linux 6.8.0), same tools (`wrk` + `perfer`), 3 trials per
+row, median reported — and writes the results to a new
+`docs/BENCH_HYPERION_2_11.md` (pointed at from the README's
+Benchmarks section; the 2.0 doc is now marked "historical
+baseline (pre-2.10/2.11 wins)").
+
+The harness at `bench/4way_compare.sh` gains a sixth server
+label, `hyperion_handle_static`, which boots Hyperion against
+an alternate rackup whose hot path is the
+`Hyperion::Server.handle_static` direct route + 2.10-F C-ext
+fast-path response writer + 2.10-C PageCache fold. This
+exposes two Hyperion columns on the rows where it applies
+(hello + small static): "Rack-style" (the legacy generic-Rack
+path most apps run unchanged) and "`handle_static`" (the peak
+path operators can opt into by registering one pre-built
+route at boot). New rackup `bench/static_handle_static.ru`
+preloads the 1 KB static asset bytes at boot and serves them
+through the same direct-route fast path; the existing
+`bench/hello_static.ru` (added in 2.10-F) plays the same role
+for the hello row.
+
+**Headline shifts (medians, vs the 2.10-B baseline at the
+same workload):**
+
+| # | Workload | 2.10-B Hyperion | 2.11.0 Rack-style | 2.11.0 `handle_static` | Gap-vs-leader shift |
+|---:|---|---:|---:|---:|---|
+| 1 | hello | 4,587 | 4,477 (-2.4%, noise) | **5,502** (+19.9%) | gap to Agoo: 4.22× → **3.46×** with `handle_static` |
+| 2 | static 1 KB | 1,380 | 1,687 (**+22.2%**, PageCache 2.10-C auto-engage) | **5,935** (+330%) | gap to Agoo: 1.89× behind → **flipped, Hyperion +127%** |
+| 3 | static 1 MiB | 1,378 | 1,513 (+9.8%) | n/a (handle_static buffers in memory; defeats sendfile) | Hyperion lead vs Agoo: 9.07× → 9.74× (held) |
+| 4 | CPU JSON 50-key | 3,450 | 3,659 (+6.0%) | n/a (per-request `JSON.generate`) | gap to Agoo: 1.85× → **2.05× (widened)**; Agoo +17.5% in same window |
+| 5 | PG-bound async (`pg_sleep 50ms`) | 1,564 | 1,565 (+0.1%) | n/a | Hyperion-only; identical |
+| 6 | SSE 1000 events × 50 B | 500 | 472 (-5.6%, noise) | n/a (single fixed response) | Hyperion lead vs Puma: 3.65× → 3.58× (flat) |
+
+**Reading the deltas.**
+
+- **The 2.10-D + 2.10-F + 2.10-C win-stack lands cleanly on
+  static 1 KB.** Hyperion's `handle_static` row at 5,935 r/s
+  wins the column outright by +127% (2.27×) over Agoo, +208%
+  over Falcon, +282% over Puma. The Rack-style row also moved
+  up +22.2% from the PageCache auto-engage even without
+  explicit `handle_static` registration. Operators who can
+  register one pre-built route via `handle_static` pick up a
+  3.5× lift over the generic Rack-style path on this exact
+  shape.
+- **Hello narrowed but did not close.** The gap to Agoo went
+  from 4.22× to 3.46× with `handle_static`; the Rack-style
+  row stayed flat (within bench noise) — meaning the 2.10-D
+  direct-route win **only manifests when the rackup actually
+  opts in via `handle_static`**.
+- **CPU JSON widened.** Agoo +17.5% in the same window
+  Hyperion +6.0% took the gap from 1.85× to 2.05×. This is
+  the one row the 2.10/2.11 streams didn't move; closing it
+  is the obvious 2.12 follow-on.
+- **Large static, PG-bound async, SSE — Hyperion's existing
+  wins held.** Sendfile (9.7× over Agoo on 1 MiB), fiber-
+  cooperative I/O (Hyperion-only on PG-bound, identical to
+  2.10-B), ChunkedCoalescer (3.58× over Puma, 16.7× over
+  Falcon on SSE) all stayed clean. Agoo's SSE behavior
+  regressed to a hard segfault at boot on `bench/sse_generic.ru`
+  (different shape from 2.10-B's "buffers entire response,
+  takes ~5 s to flush"; same conclusion either way — Agoo is
+  not a viable SSE server on this rackup at any throughput).
+- **Tail latency is still Hyperion's clean win** across every
+  row with a non-trivial p99: hello 1.73 ms vs Agoo 10.47 ms
+  / Puma 29 ms / Falcon 408 ms; 1 KB 1.69 ms vs 57–86 ms;
+  1 MiB 4.63 ms vs 82–720 ms; CPU JSON 2.60 ms vs 17–411 ms;
+  SSE 2.85 ms vs 11–42 ms.
+
+**Harness changes** (`bench/4way_compare.sh`):
+
+- New server label `hyperion_handle_static` — boots Hyperion
+  against an alternate rackup specified by the new
+  `HYPERION_STATIC_RACKUP` env var (falls back to the same
+  `RACKUP` as the legacy `hyperion` label if unset, which
+  becomes a harmless no-op duplicate). The four legacy
+  labels (`hyperion`, `puma`, `falcon`, `agoo`) keep their
+  byte-identical 2.10-B shape so the older doc stays
+  reproducible.
+
+**New rackup**: `bench/static_handle_static.ru` — preloads
+the 1 KB static asset (`/tmp/hyperion_bench_1k.bin`) at boot,
+registers it via `Hyperion::Server.handle_static`, and falls
+through to a 404 lambda for any other path. Mirrors the role
+`bench/hello_static.ru` plays for the hello row.
+
+**Spec coverage** (`spec/hyperion/bench_handle_static_rackups_spec.rb`,
+9 examples):
+
+- `bench/hello_static.ru` parses cleanly and registers a `/`
+  StaticEntry with the expected response bytes.
+- `bench/static_handle_static.ru` preloads the asset and
+  registers a route at boot; fails fast if the asset is
+  missing rather than booting empty (prevents a silently-
+  bound 404 from masking a misconfigured bench run).
+- `bench/4way_compare.sh` knows how to boot all five variant
+  labels (hyperion, hyperion_handle_static, puma, falcon,
+  agoo) — a typo in the harness's case statement or the
+  new env var name fails this spec at unit-level instead
+  of waiting for a 41-minute bench sweep to surface the
+  regression.
+
+Spec count: 1093 → 1102 (+9). 0 failures, 11 pending —
+invariant preserved.
+
+**Reproducing.** See
+`docs/BENCH_HYPERION_2_11.md` § "Reproducing 4-way" for the
+six per-row commands. Bench host: `openclaw-vm`, sweep dir
+`/home/ubuntu/bench-2.12-B/`. Total wall time ~41 min.
+
 ## 2.11.0 — 2026-05-01
 
 ### 2.11-B — HPACK FFI marshalling round-2 (cglue confirmed as firm default; +43% v3 vs v2 on Rails-shape h2)