kino 0.5.0 → 0.7.0

data/doc/benchmarks.md

@@ -17,9 +17,21 @@ the deployment most apps run today.
   9R14 (Genoa), 16 GB RAM, Amazon Linux 2023, kernel 6.18. A realistic
   app-server size, deliberately: nobody provisions a 32-core box per
   app process.
-- Toolchain built on the box via mise: Ruby 4.0.5 (**YJIT enabled**,
-  `RUBY_YJIT_ENABLE=1` for every server), Rust 1.96, Kino compiled in
+- Toolchain built on the box via mise: Ruby 4.0.6 (**YJIT enabled**,
+  `RUBY_YJIT_ENABLE=1` for every server), Rust stable, Kino compiled in
   the release profile.
+- **2026-09 full re-measurement** (a fresh c7a.2xlarge): every number in
+  this document and the README was re-run on Ruby 4.0.6 / kernel 6.18 /
+  Puma 8.0.2, adding the sharded-I/O and HTTP/2 studies. Kino's numbers
+  reproduced within 1-3% across the board—including the /io slot
+  ceiling and the arena balloon, both intact. The one real shift: Puma
+  8.0.2 is faster than 7.x (142k plaintext, 61k /cpu), narrowing
+  ractor mode's /cpu lead to +25% (was +34%). Reversed-boot-order
+  re-runs reproduced within ~1%. A methodology trap worth recording:
+  a 5-second single-endpoint warmup understates memory badly (81 MB
+  where the full battery shows 137 MB) and hides the arena balloon
+  entirely—memory is only comparable after the full endpoint battery,
+  and /io numbers are only comparable at equal slot counts.
 - Load generator: wrk 4.2 on the same host, 8-second windows, 64
   connections (`bench/run.sh 8 64`). Same-host load generation costs
   both sides CPU equally; we verified the generator was not the
@@ -249,14 +261,17 @@ visible.
 
 | config | RSS | PSS |
 |---|---:|---:|
-| Kino :ractor 8×1 (default) | 151 | **148** |
-| Kino lanes 8×1 | 137 | **135** |
-| Kino :ractor 8×3 | 171 | **169** |
+| Kino :ractor 8×1 (default) | 137 | **135** |
+| Kino lanes 8×1 | 128 | **126** |
+| Kino :ractor 8×3 | 170 | **168** |
 | Kino :threaded 8×3 (`MALLOC_ARENA_MAX=2`) | 109 | **107** |
-| Kino :threaded 8×3 (no arena cap) | 668 | **666**¹ |
-| Puma cluster 8×3 | 1,213 | **1,068** |
+| Kino :threaded 8×3 (no arena cap) | 672 | **670**¹ |
+| Puma cluster 8×3 | 1,216 | **1,072** |
 
-The tiny app is ~7× lighter than the cluster in ractor mode, ~10× in
+(2026-09 re-measure; the 2026-06 numbers reproduced within a few MB on
+every row—the arena-capped threaded row to the megabyte.)
+
+The tiny app is ~8× lighter than the cluster in ractor mode, ~10× in
 arena-capped threaded mode. RSS ≈ PSS for every Kino row (one process,
 nothing to share) and within ~12% for Puma here: a trivial app has almost
 no shared state, so Puma's footprint is ~1,051 MB of *private* per-worker
@@ -280,18 +295,19 @@ Here copy-on-write **does** matter, which is exactly why PSS is mandatory:
 
 | config | RSS | PSS |
 |---|---:|---:|
-| Kino :threaded (one process) |  97 |  **92** |
-| Puma cluster 8×3 (preload) | 794 | **389** |
+| Kino :threaded (one process) |  97 |  **95** |
+| Puma cluster 8×5 (preload) | 813 | **405** |
+| Puma cluster 8×5 (no preload) | 824 | **646** |
 
 Puma serves the same Rails framework from 8 forks that share it
-copy-on-write; RSS counts that shared framework once per worker (794 MB),
-PSS counts it once (389 MB). The fair ratio is **~4×**, not the ~8× a
+copy-on-write; RSS counts that shared framework once per worker (813 MB),
+PSS counts it once (405 MB). The fair ratio is **~4×**, not the ~8× a
 naive RSS sum reports—this is the correction that prompted the whole
-re-measure. Preload barely helps (389 vs 400 MB without): Ruby's GC
-dirties most heap pages, breaking copy-on-write, so even a preloaded
-cluster keeps a large private heap per worker. That is why "CoW should
-make a fork cluster nearly free" is only half true—it shares the code,
-not the live object heap.
+re-measure. The 2026-09 run also split out preload: it now saves a
+third (405 vs 646 MB PSS)—worth turning on—but even preloaded, Ruby's
+GC dirties heap pages and breaks copy-on-write, so each worker keeps a
+large private heap. That is why "CoW should make a fork cluster nearly
+free" is only half true—it shares the code, not the live object heap.
 
 ## Run-to-run variance (a.k.a. "is this a regression?")
 
@@ -376,27 +392,46 @@ crash semantics, stealing fairness, and drain behavior have spec
 coverage but not production mileage. (On loopback-bound macOS, lanes
 lose a few percent instead; see the secondary table below.)
 
+## Sharded I/O (`io_shards true`)
+
+`bench/studies.sh 8 64 shards`, ractor 8×3, 2026-09 reference box:
+
+| case (/plaintext) | req/s |
+|---|---:|
+| shared tokio runtime (baseline) | 193,461 |
+| io_shards, default shard count | 195,436 |
+| io_shards, `io_threads 2` | 170,509 |
+| io_shards, `io_threads 4` | 196,298 |
+| io_shards, `io_threads 8` | 199,657 |
+
+On 8 cores the shards buy +1-3%, best at `io_threads 8` (the
+half-the-cores default is close behind; 2 shards choke on accept
+handoff). The design removes work-stealing and cross-thread wakeups,
+so the win scales with scheduler contention—expect more on boxes with
+more cores and connections, and measure on your own core count.
+
 ## Logging costs
 
 Measured at full plaintext saturation (one log line per request—rates
 that no real deployment logs at; treat these as worst-case ceilings, not
 typical costs):
 
-| case (8×3, same session) | req/s |
+| case (8×3, same session, 2026-09) | req/s |
 |---|---:|
-| threaded, no logging | 219,168 |
-| threaded, `log_requests true` (native access log) | 193,998 (−11%) |
-| ractor, access log off / on | 197,596 / 181,050 (−8%) |
-| app logs 1 line/req via shared `::Logger` (file) | **62,961** |
-| app logs 1 line/req via `Kino::Logger` (file) | **149,519 (2.4×)** |
+| threaded, no logging | 213,166 |
+| threaded, `log_requests true` (native access log) | 173,501 (−19%) |
+| ractor, access log off / on | 191,152 / 163,777 (−14%) |
+| app logs 1 line/req via shared `::Logger` (file) | **90,175** |
+| app logs 1 line/req via `Kino::Logger` (file) | **154,653 (1.7×)** |
 
 The shared-`::Logger` cost is the mutex: 24 worker threads serialize
 through one lock plus a write syscall per line. `Kino::Logger` hands the
 formatted line to a lock-free channel and returns—the remaining cost vs
 not logging at all is Ruby-side formatting, which no device can remove.
-(In the Docker environment the same comparison showed 8.5×—overlay-fs
-write latency punished the synchronous logger far harder than this box's
-NVMe does. The ranking is environment-independent; the multiple is not.)
+(The multiple moves with the environment: 2.4× on the 2026-06 box,
+1.7× on the 2026-09 one, 8.5× under Docker, where overlay-fs write
+latency punished the synchronous logger hardest. The ranking is
+environment-independent; the multiple is not.)
 
 One trade-off worth knowing: the sink **never blocks** request threads,
 so at absurd rates against a slow disk it drops lines once its 8192-line
@@ -408,6 +443,65 @@ Puma comparison note: request logging is opt-in there too (`--quiet` is
 the default, `-v/--log-requests` enables it)—Kino's default-off
 `log_requests` matches the ecosystem's standard behavior.
 
+## HTTP/2
+
+`bench/h2.sh`, Linux only (on macOS run it under Docker; the 2026-09
+numbers below are from the c7a.2xlarge reference box). Every lane is
+measured with h2load so the generator is identical everywhere: h2
+lanes run 8 connections × 8 concurrent streams, h1 lanes 64
+connections—the same total in-flight. Servers without native h2 get
+the standard pattern instead: nginx terminating h2 and proxying
+HTTP/1.1 upstream over keep-alive. One labeled run (kino ractor 8×3,
+falcon `--count 8`, puma `-w 8 -t 3:3`, 5 s/lane); re-run the whole
+script for close calls, per the variance section.
+
+| target (h2 unless noted) | /plaintext | /10k | /big-cookie | /upload (64 KB) |
+|---|---:|---:|---:|---:|
+| kino h2c | 207,461 | 150,105 | 195,093 | 26,304 |
+| kino h1 cleartext (same boot) | 116,082 | 97,634 | 110,614 | 24,338 |
+| kino h2 TLS | 164,044 | 116,617 | 154,301 | 19,226 |
+| kino h1 TLS (same boot) | 80,789 | 69,942 | 76,654 | 17,210 |
+| falcon TLS (native h2) | 55,637 | 37,541 | 49,151 | 18,838 |
+| nginx h2 → puma h1 | 81,234 | 57,891 | 51,332 | 1,247 |
+| nginx h2 → kino h1 | 109,219 | 66,634 | 54,996 | 1,217 |
+
+What the numbers say:
+
+- **Native h2 beats h1 on the same server by +79% cleartext and +103%
+  over TLS** on /plaintext: the same 64 in-flight requests ride 8
+  connections instead of 64, so frames batch into fewer, larger
+  syscalls—and TLS amplifies it, since h1's 64 connections each pay
+  crypto per record. The `/big-cookie` lane (a ~2 KB cookie per
+  request) shows HPACK on top: the cookie crosses the wire once per
+  connection, not once per request, and holds 94% of bare-plaintext
+  throughput where h1 loses 5%.
+- **Native h2 beats proxied h2 by +50%** with the *same backend*: the
+  nginx→kino-h1 lane is the proxy-cost control, and the extra hop,
+  re-parse, and re-serialize cost ~55k req/s on /plaintext.
+- **Uploads run at h1 parity**—but only after a fix this lane
+  caught: h2 delivers bodies as 16 KB DATA frames, and `read_body`
+  originally crossed the GVL once per chunk, halving upload
+  throughput. It now drains every queued chunk per crossing
+  (doc/architecture.md), and a knob sweep over hyper's h2 codec
+  (frame size, adaptive/bigger windows) moved nothing afterwards.
+  nginx's h2 upload collapse (~1.5k) is its default-config
+  request-body flow control; tune `http2_body_preread_size`/buffering
+  before drawing conclusions there.
+- falcon lands at roughly a third of kino-h2-TLS on fast handlers and
+  slightly behind on uploads. Single-run caveat: the nginx lanes
+  showed ±20% swings between runs in the Docker environment; on the
+  reference box the kino-vs-kino and kino-vs-proxy ratios reproduced
+  across runs, the nginx lanes remain the noisiest.
+- **Header-value interning** (user-agent, accept-*, sec-ch-*: one
+  frozen string instead of a fresh allocation per request) was
+  measured with a realistic 11-header browser set on /plaintext,
+  using the within-boot header cost (bare vs with-headers) as the
+  drift-resistant metric: over h2 that cost fell from ~16% to ~12-13%
+  (~+3-5% throughput on the headers lane); h1's smaller header cost
+  stayed within noise. The effect the tiny app understates: 6-8 fewer
+  string allocations per request is GC pressure a real app feels more
+  than this one does.
+
 ## Hot-path notes
 
 For the curious, the dispatch-path work behind the numbers: a try-pop

data/ext/kino/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "kino"
-version = "0.5.0"
+version = "0.7.0"
 edition = "2021"
 authors = ["Yaroslav Markin <yaroslav@markin.net>"]
 license = "MIT"
@@ -21,8 +21,8 @@ smallvec = "1"
 lru = "0.18"
 mimalloc = { version = "0.1", default-features = false }
 tokio = { version = "1.45", features = ["rt-multi-thread", "net", "time", "sync", "io-util", "macros"] }
-hyper = { version = "1.6", features = ["http1", "server"] }
-hyper-util = { version = "0.1", features = ["server", "tokio", "http1"] }
+hyper = { version = "1.6", features = ["http1", "http2", "server"] }
+hyper-util = { version = "0.1", features = ["server", "server-auto", "tokio", "http1", "http2"] }
 http = "1"
 http-body-util = "0.1"
 bytes = "1"
@@ -49,3 +49,8 @@ rb-sys-env = "0.2.2"
 # undefined (the host process provides them at runtime); feature
 # unification turns this on only for `cargo test`.
 rb-sys = { version = "0.9", features = ["link-ruby"] }
+# The protocol tests drive the server end of a duplex pipe with hyper's
+# own client, and pause the clock to test timeouts; feature unification
+# enables both only for `cargo test`.
+hyper = { version = "1.6", features = ["client"] }
+tokio = { version = "1.45", features = ["test-util"] }

data/ext/kino/src/control.rs

@@ -18,6 +18,8 @@ pub struct WorkerStat {
     pub in_flight: usize,
     pub busy_ms: u64,
     pub quarantined: bool,
+    /// Sent home by the pool scaler; leaves at its next idle tick.
+    pub retired: bool,
 }
 
 /// Read every slot's per-worker sensors in one pass under the slots read
@@ -46,6 +48,7 @@ pub fn collect_worker_status(server: &ServerInner) -> Vec<WorkerStat> {
                     now.saturating_sub(started)
                 },
                 quarantined,
+                retired: slot.retired.load(Ordering::Relaxed),
             }
         })
         .collect()
@@ -70,6 +73,10 @@ pub struct StatsSnapshot {
     pub worker_status: Vec<WorkerStat>,
     pub quarantined_count: usize,
     pub quarantine_replacements: u64,
+    pub max_workers: usize,
+    pub active_workers: usize,
+    pub scale_ups: u64,
+    pub scale_downs: u64,
     pub queue_histogram: crate::registry::QueueHistogramSnapshot,
 }
 
@@ -94,6 +101,10 @@ impl StatsSnapshot {
             worker_status,
             quarantined_count,
             quarantine_replacements: server.quarantine_replacements.load(Ordering::Relaxed),
+            max_workers: server.topology.max_workers,
+            active_workers: server.active_workers.load(Ordering::Relaxed),
+            scale_ups: server.scale_ups.load(Ordering::Relaxed),
+            scale_downs: server.scale_downs.load(Ordering::Relaxed),
             queue_histogram: server.queue_histogram.snapshot(),
         }
     }
@@ -114,9 +125,10 @@ pub fn stats_json(s: &StatsSnapshot) -> String {
     let mut out = String::with_capacity(256);
     write!(
         out,
-        r#"{{"mode":"{}","lanes":{},"workers":{},"threads":{},"batch":{},"respawns":{},"queued":{},"in_flight":{},"served":{},"rejected":{},"timeouts":{}"#,
+        r#"{{"mode":"{}","lanes":{},"workers":{},"threads":{},"batch":{},"respawns":{},"queued":{},"in_flight":{},"served":{},"rejected":{},"timeouts":{},"max_workers":{},"active_workers":{},"scale_ups":{},"scale_downs":{}"#,
         s.mode, s.lanes, s.workers, s.threads, s.batch, s.respawns,
-        s.queued, s.in_flight, s.served, s.rejected, s.timeouts
+        s.queued, s.in_flight, s.served, s.rejected, s.timeouts,
+        s.max_workers, s.active_workers, s.scale_ups, s.scale_downs
     )
     .expect("writing to a String cannot fail");
     if let Some(depths) = &s.lane_depths {
@@ -134,8 +146,8 @@ pub fn stats_json(s: &StatsSnapshot) -> String {
         }
         write!(
             out,
-            r#"{{"index":{},"served":{},"in_flight":{},"busy_ms":{},"quarantined":{}}}"#,
-            w.index, w.served, w.in_flight, w.busy_ms, w.quarantined
+            r#"{{"index":{},"served":{},"in_flight":{},"busy_ms":{},"quarantined":{},"retired":{}}}"#,
+            w.index, w.served, w.in_flight, w.busy_ms, w.quarantined, w.retired
         )
         .expect("writing to a String cannot fail");
     }
@@ -247,6 +259,34 @@ pub fn metrics_text(s: &StatsSnapshot) -> String {
         "Configured threads per worker.",
         s.threads,
     );
+    metric(
+        &mut out,
+        "kino_max_workers",
+        "gauge",
+        "Worker pool ceiling (equals kino_workers for a fixed pool).",
+        s.max_workers,
+    );
+    metric(
+        &mut out,
+        "kino_active_workers",
+        "gauge",
+        "Workers currently alive and serving.",
+        s.active_workers,
+    );
+    metric(
+        &mut out,
+        "kino_scale_ups_total",
+        "counter",
+        "Workers added by the pool scaler.",
+        s.scale_ups,
+    );
+    metric(
+        &mut out,
+        "kino_scale_downs_total",
+        "counter",
+        "Idle workers retired by the pool scaler.",
+        s.scale_downs,
+    );
     metric(
         &mut out,
         "kino_ready",
@@ -641,6 +681,10 @@ mod tests {
             worker_status: vec![],
             quarantined_count: 0,
             quarantine_replacements: 0,
+            max_workers: 32,
+            active_workers: 12,
+            scale_ups: 3,
+            scale_downs: 1,
             queue_histogram: crate::registry::QueueHistogramSnapshot {
                 buckets: [0; crate::registry::QUEUE_BOUNDS_US.len()],
                 overflow: 0,
@@ -665,6 +709,10 @@ mod tests {
             r#""served":100"#,
             r#""rejected":5"#,
             r#""timeouts":6"#,
+            r#""max_workers":32"#,
+            r#""active_workers":12"#,
+            r#""scale_ups":3"#,
+            r#""scale_downs":1"#,
             r#""state":"ready""#,
             r#""version":""#,
         ] {
@@ -690,6 +738,49 @@ mod tests {
         assert!(draining.contains("kino_ready 0"));
     }
 
+    #[test]
+    fn metrics_text_reports_the_elastic_pool() {
+        let text = metrics_text(&snapshot(crate::registry::STATE_READY));
+        assert!(text.contains("# TYPE kino_max_workers gauge"));
+        assert!(text.contains("kino_max_workers 32"));
+        assert!(text.contains("# TYPE kino_active_workers gauge"));
+        assert!(text.contains("kino_active_workers 12"));
+        assert!(text.contains("# TYPE kino_scale_ups_total counter"));
+        assert!(text.contains("kino_scale_ups_total 3"));
+        assert!(text.contains("# TYPE kino_scale_downs_total counter"));
+        assert!(text.contains("kino_scale_downs_total 1"));
+    }
+
+    #[test]
+    fn worker_status_reports_retired_slots() {
+        let server = crate::registry::test_server(false, 4);
+        server.register_worker();
+        server.register_worker();
+        server.slots.read()[1].retire();
+
+        let status = collect_worker_status(&server);
+
+        assert_eq!(
+            status.iter().map(|w| w.retired).collect::<Vec<_>>(),
+            vec![false, true]
+        );
+        assert!(status.iter().all(|w| w.busy_ms == 0));
+    }
+
+    #[test]
+    fn snapshot_reads_the_pool_counters() {
+        let server = crate::registry::test_server(false, 4);
+        server.active_workers.store(3, Ordering::Relaxed);
+        server.scale_ups.fetch_add(2, Ordering::Relaxed);
+        server.scale_downs.fetch_add(1, Ordering::Relaxed);
+
+        let s = StatsSnapshot::take(&server);
+
+        assert_eq!(s.active_workers, 3);
+        assert_eq!(s.max_workers, server.topology.max_workers);
+        assert_eq!((s.scale_ups, s.scale_downs), (2, 1));
+    }
+
     #[test]
     fn metrics_text_includes_lane_depth_samples_when_lanes_are_on() {
         let mut s = snapshot(crate::registry::STATE_READY);
@@ -757,6 +848,7 @@ mod tests {
                 in_flight: 1,
                 busy_ms: 4,
                 quarantined: false,
+                retired: false,
             },
             WorkerStat {
                 index: 1,
@@ -764,10 +856,11 @@ mod tests {
                 in_flight: 0,
                 busy_ms: 0,
                 quarantined: false,
+                retired: true,
             },
         ];
         let json = stats_json(&s);
-        assert!(json.contains(r#""worker_status":[{"index":0,"served":10,"in_flight":1,"busy_ms":4,"quarantined":false},{"index":1,"served":7,"in_flight":0,"busy_ms":0,"quarantined":false}]"#), "got {json}");
+        assert!(json.contains(r#""worker_status":[{"index":0,"served":10,"in_flight":1,"busy_ms":4,"quarantined":false,"retired":false},{"index":1,"served":7,"in_flight":0,"busy_ms":0,"quarantined":false,"retired":true}]"#), "got {json}");
     }
 
     #[test]
@@ -786,6 +879,7 @@ mod tests {
                 in_flight: 1,
                 busy_ms: 4,
                 quarantined: false,
+                retired: false,
             },
             WorkerStat {
                 index: 1,
@@ -793,6 +887,7 @@ mod tests {
                 in_flight: 0,
                 busy_ms: 0,
                 quarantined: false,
+                retired: true,
             },
         ];
         let text = metrics_text(&s);
@@ -835,6 +930,7 @@ mod tests {
                 in_flight: 1,
                 busy_ms: 0,
                 quarantined: true,
+                retired: false,
             },
             WorkerStat {
                 index: 1,
@@ -842,6 +938,7 @@ mod tests {
                 in_flight: 1,
                 busy_ms: 5,
                 quarantined: false,
+                retired: false,
             },
         ];
         let json = stats_json(&s);
@@ -850,7 +947,7 @@ mod tests {
             "top-level count: {json}"
         );
         assert!(
-            json.contains(r#"{"index":0,"served":3,"in_flight":1,"busy_ms":0,"quarantined":true}"#),
+            json.contains(r#"{"index":0,"served":3,"in_flight":1,"busy_ms":0,"quarantined":true,"retired":false}"#),
             "{json}"
         );
         assert!(json.contains(r#""quarantined":false"#));