kino 0.5.0-aarch64-linux → 0.7.0-aarch64-linux

data/doc/benchmarks.md

@@ -17,9 +17,21 @@ the deployment most apps run today.
   9R14 (Genoa), 16 GB RAM, Amazon Linux 2023, kernel 6.18. A realistic
   app-server size, deliberately: nobody provisions a 32-core box per
   app process.
-- Toolchain built on the box via mise: Ruby 4.0.5 (**YJIT enabled**,
-  `RUBY_YJIT_ENABLE=1` for every server), Rust 1.96, Kino compiled in
+- Toolchain built on the box via mise: Ruby 4.0.6 (**YJIT enabled**,
+  `RUBY_YJIT_ENABLE=1` for every server), Rust stable, Kino compiled in
   the release profile.
+- **2026-09 full re-measurement** (a fresh c7a.2xlarge): every number in
+  this document and the README was re-run on Ruby 4.0.6 / kernel 6.18 /
+  Puma 8.0.2, adding the sharded-I/O and HTTP/2 studies. Kino's numbers
+  reproduced within 1-3% across the board—including the /io slot
+  ceiling and the arena balloon, both intact. The one real shift: Puma
+  8.0.2 is faster than 7.x (142k plaintext, 61k /cpu), narrowing
+  ractor mode's /cpu lead to +25% (was +34%). Reversed-boot-order
+  re-runs reproduced within ~1%. A methodology trap worth recording:
+  a 5-second single-endpoint warmup understates memory badly (81 MB
+  where the full battery shows 137 MB) and hides the arena balloon
+  entirely—memory is only comparable after the full endpoint battery,
+  and /io numbers are only comparable at equal slot counts.
 - Load generator: wrk 4.2 on the same host, 8-second windows, 64
   connections (`bench/run.sh 8 64`). Same-host load generation costs
   both sides CPU equally; we verified the generator was not the
@@ -249,14 +261,17 @@ visible.
 
 | config | RSS | PSS |
 |---|---:|---:|
-| Kino :ractor 8×1 (default) | 151 | **148** |
-| Kino lanes 8×1 | 137 | **135** |
-| Kino :ractor 8×3 | 171 | **169** |
+| Kino :ractor 8×1 (default) | 137 | **135** |
+| Kino lanes 8×1 | 128 | **126** |
+| Kino :ractor 8×3 | 170 | **168** |
 | Kino :threaded 8×3 (`MALLOC_ARENA_MAX=2`) | 109 | **107** |
-| Kino :threaded 8×3 (no arena cap) | 668 | **666**¹ |
-| Puma cluster 8×3 | 1,213 | **1,068** |
+| Kino :threaded 8×3 (no arena cap) | 672 | **670**¹ |
+| Puma cluster 8×3 | 1,216 | **1,072** |
 
-The tiny app is ~7× lighter than the cluster in ractor mode, ~10× in
+(2026-09 re-measure; the 2026-06 numbers reproduced within a few MB on
+every row—the arena-capped threaded row to the megabyte.)
+
+The tiny app is ~8× lighter than the cluster in ractor mode, ~10× in
 arena-capped threaded mode. RSS ≈ PSS for every Kino row (one process,
 nothing to share) and within ~12% for Puma here: a trivial app has almost
 no shared state, so Puma's footprint is ~1,051 MB of *private* per-worker
@@ -280,18 +295,19 @@ Here copy-on-write **does** matter, which is exactly why PSS is mandatory:
 
 | config | RSS | PSS |
 |---|---:|---:|
-| Kino :threaded (one process) |  97 |  **92** |
-| Puma cluster 8×3 (preload) | 794 | **389** |
+| Kino :threaded (one process) |  97 |  **95** |
+| Puma cluster 8×5 (preload) | 813 | **405** |
+| Puma cluster 8×5 (no preload) | 824 | **646** |
 
 Puma serves the same Rails framework from 8 forks that share it
-copy-on-write; RSS counts that shared framework once per worker (794 MB),
-PSS counts it once (389 MB). The fair ratio is **~4×**, not the ~8× a
+copy-on-write; RSS counts that shared framework once per worker (813 MB),
+PSS counts it once (405 MB). The fair ratio is **~4×**, not the ~8× a
 naive RSS sum reports—this is the correction that prompted the whole
-re-measure. Preload barely helps (389 vs 400 MB without): Ruby's GC
-dirties most heap pages, breaking copy-on-write, so even a preloaded
-cluster keeps a large private heap per worker. That is why "CoW should
-make a fork cluster nearly free" is only half true—it shares the code,
-not the live object heap.
+re-measure. The 2026-09 run also split out preload: it now saves a
+third (405 vs 646 MB PSS)—worth turning on—but even preloaded, Ruby's
+GC dirties heap pages and breaks copy-on-write, so each worker keeps a
+large private heap. That is why "CoW should make a fork cluster nearly
+free" is only half true—it shares the code, not the live object heap.
 
 ## Run-to-run variance (a.k.a. "is this a regression?")
 
@@ -376,27 +392,46 @@ crash semantics, stealing fairness, and drain behavior have spec
 coverage but not production mileage. (On loopback-bound macOS, lanes
 lose a few percent instead; see the secondary table below.)
 
+## Sharded I/O (`io_shards true`)
+
+`bench/studies.sh 8 64 shards`, ractor 8×3, 2026-09 reference box:
+
+| case (/plaintext) | req/s |
+|---|---:|
+| shared tokio runtime (baseline) | 193,461 |
+| io_shards, default shard count | 195,436 |
+| io_shards, `io_threads 2` | 170,509 |
+| io_shards, `io_threads 4` | 196,298 |
+| io_shards, `io_threads 8` | 199,657 |
+
+On 8 cores the shards buy +1-3%, best at `io_threads 8` (the
+half-the-cores default is close behind; 2 shards choke on accept
+handoff). The design removes work-stealing and cross-thread wakeups,
+so the win scales with scheduler contention—expect more on boxes with
+more cores and connections, and measure on your own core count.
+
 ## Logging costs
 
 Measured at full plaintext saturation (one log line per request—rates
 that no real deployment logs at; treat these as worst-case ceilings, not
 typical costs):
 
-| case (8×3, same session) | req/s |
+| case (8×3, same session, 2026-09) | req/s |
 |---|---:|
-| threaded, no logging | 219,168 |
-| threaded, `log_requests true` (native access log) | 193,998 (−11%) |
-| ractor, access log off / on | 197,596 / 181,050 (−8%) |
-| app logs 1 line/req via shared `::Logger` (file) | **62,961** |
-| app logs 1 line/req via `Kino::Logger` (file) | **149,519 (2.4×)** |
+| threaded, no logging | 213,166 |
+| threaded, `log_requests true` (native access log) | 173,501 (−19%) |
+| ractor, access log off / on | 191,152 / 163,777 (−14%) |
+| app logs 1 line/req via shared `::Logger` (file) | **90,175** |
+| app logs 1 line/req via `Kino::Logger` (file) | **154,653 (1.7×)** |
 
 The shared-`::Logger` cost is the mutex: 24 worker threads serialize
 through one lock plus a write syscall per line. `Kino::Logger` hands the
 formatted line to a lock-free channel and returns—the remaining cost vs
 not logging at all is Ruby-side formatting, which no device can remove.
-(In the Docker environment the same comparison showed 8.5×—overlay-fs
-write latency punished the synchronous logger far harder than this box's
-NVMe does. The ranking is environment-independent; the multiple is not.)
+(The multiple moves with the environment: 2.4× on the 2026-06 box,
+1.7× on the 2026-09 one, 8.5× under Docker, where overlay-fs write
+latency punished the synchronous logger hardest. The ranking is
+environment-independent; the multiple is not.)
 
 One trade-off worth knowing: the sink **never blocks** request threads,
 so at absurd rates against a slow disk it drops lines once its 8192-line
@@ -408,6 +443,65 @@ Puma comparison note: request logging is opt-in there too (`--quiet` is
 the default, `-v/--log-requests` enables it)—Kino's default-off
 `log_requests` matches the ecosystem's standard behavior.
 
+## HTTP/2
+
+`bench/h2.sh`, Linux only (on macOS run it under Docker; the 2026-09
+numbers below are from the c7a.2xlarge reference box). Every lane is
+measured with h2load so the generator is identical everywhere: h2
+lanes run 8 connections × 8 concurrent streams, h1 lanes 64
+connections—the same total in-flight. Servers without native h2 get
+the standard pattern instead: nginx terminating h2 and proxying
+HTTP/1.1 upstream over keep-alive. One labeled run (kino ractor 8×3,
+falcon `--count 8`, puma `-w 8 -t 3:3`, 5 s/lane); re-run the whole
+script for close calls, per the variance section.
+
+| target (h2 unless noted) | /plaintext | /10k | /big-cookie | /upload (64 KB) |
+|---|---:|---:|---:|---:|
+| kino h2c | 207,461 | 150,105 | 195,093 | 26,304 |
+| kino h1 cleartext (same boot) | 116,082 | 97,634 | 110,614 | 24,338 |
+| kino h2 TLS | 164,044 | 116,617 | 154,301 | 19,226 |
+| kino h1 TLS (same boot) | 80,789 | 69,942 | 76,654 | 17,210 |
+| falcon TLS (native h2) | 55,637 | 37,541 | 49,151 | 18,838 |
+| nginx h2 → puma h1 | 81,234 | 57,891 | 51,332 | 1,247 |
+| nginx h2 → kino h1 | 109,219 | 66,634 | 54,996 | 1,217 |
+
+What the numbers say:
+
+- **Native h2 beats h1 on the same server by +79% cleartext and +103%
+  over TLS** on /plaintext: the same 64 in-flight requests ride 8
+  connections instead of 64, so frames batch into fewer, larger
+  syscalls—and TLS amplifies it, since h1's 64 connections each pay
+  crypto per record. The `/big-cookie` lane (a ~2 KB cookie per
+  request) shows HPACK on top: the cookie crosses the wire once per
+  connection, not once per request, and holds 94% of bare-plaintext
+  throughput where h1 loses 5%.
+- **Native h2 beats proxied h2 by +50%** with the *same backend*: the
+  nginx→kino-h1 lane is the proxy-cost control, and the extra hop,
+  re-parse, and re-serialize cost ~55k req/s on /plaintext.
+- **Uploads run at h1 parity**—but only after a fix this lane
+  caught: h2 delivers bodies as 16 KB DATA frames, and `read_body`
+  originally crossed the GVL once per chunk, halving upload
+  throughput. It now drains every queued chunk per crossing
+  (doc/architecture.md), and a knob sweep over hyper's h2 codec
+  (frame size, adaptive/bigger windows) moved nothing afterwards.
+  nginx's h2 upload collapse (~1.5k) is its default-config
+  request-body flow control; tune `http2_body_preread_size`/buffering
+  before drawing conclusions there.
+- falcon lands at roughly a third of kino-h2-TLS on fast handlers and
+  slightly behind on uploads. Single-run caveat: the nginx lanes
+  showed ±20% swings between runs in the Docker environment; on the
+  reference box the kino-vs-kino and kino-vs-proxy ratios reproduced
+  across runs, the nginx lanes remain the noisiest.
+- **Header-value interning** (user-agent, accept-*, sec-ch-*: one
+  frozen string instead of a fresh allocation per request) was
+  measured with a realistic 11-header browser set on /plaintext,
+  using the within-boot header cost (bare vs with-headers) as the
+  drift-resistant metric: over h2 that cost fell from ~16% to ~12-13%
+  (~+3-5% throughput on the headers lane); h1's smaller header cost
+  stayed within noise. The effect the tiny app understates: 6-8 fewer
+  string allocations per request is GC pressure a real app feels more
+  than this one does.
+
 ## Hot-path notes
 
 For the curious, the dispatch-path work behind the numbers: a try-pop

data/lib/kino/cli.rb

@@ -127,7 +127,7 @@ module Kino
       stats = server.stats
       puts dim("- ruby:      #{RUBY_DESCRIPTION}")
       puts dim("- env:       #{ENV["RAILS_ENV"] || ENV["RACK_ENV"] || "development"}")
-      puts dim("- mode:      #{server.mode}, #{count(stats[:workers], "worker")} × #{count(stats[:threads], "thread")}")
+      puts dim("- mode:      #{server.mode}, #{workers_label(stats)} × #{count(stats[:threads], "thread")}")
       puts dim("- pid:       #{Process.pid}")
       puts dim("- listening: #{server.url}")
       puts dim("- control:   #{server.control_url}") if server.control_url
@@ -140,6 +140,14 @@ module Kino
       "#{number} #{noun}#{"s" unless number == 1}"
     end
 
+    # The pool as configured: "8 workers", or "8-32 workers" when it can
+    # grow.
+    def workers_label(stats)
+      return count(stats[:workers], "worker") if stats[:max_workers] == stats[:workers]
+
+      "#{stats[:workers]}-#{stats[:max_workers]} workers"
+    end
+
     # Roll credits when the process ends: normal exit or crash (at_exit
     # also runs after an uncaught exception; only a force-exit skips it).
     # @return [void]
@@ -234,7 +242,7 @@ module Kino
     end
 
     def write_sample(path)
-      require "kino"
+      require "kino" # audition:disable runtime-require
       Configuration.write_sample(path)
       puts "Kino: wrote sample config to #{path}"
       0
@@ -245,8 +253,8 @@ module Kino
 
     # Resolve the full configuration once: file + CLI flag overrides.
     def resolve_config(options)
-      require "kino"
-      require "rack"
+      require "kino" # audition:disable runtime-require
+      require "rack" # audition:disable runtime-require
 
       config_file = options[:config_file] || Configuration.default_path
 

data/lib/kino/configuration.rb

@@ -10,6 +10,8 @@ module Kino
       bind: "127.0.0.1",
       port: 0,
       workers: nil, # resolved to Kino.available_parallelism in #to_h
+      max_workers: nil, # nil = fixed pool; above workers = elastic pool
+      scale_down_after: nil, # resolved to 30 seconds in Server
       threads: nil, # resolved per mode in Server: 1 in :ractor, 3 in :threaded
       mode: :auto,
       queue_depth: 1024,
@@ -30,6 +32,7 @@ module Kino
       io_threads: nil,
       tokio_threads: nil,
       tls: nil,
+      http2: true,
       environment: nil,
       pidfile: nil,
       control_bind: nil,
@@ -43,7 +46,7 @@ module Kino
     SETTINGS = DEFAULTS.keys.freeze
 
     # Source template for {.sample}.
-    SAMPLE_TEMPLATE = File.expand_path("templates/kino.rb.tt", __dir__)
+    SAMPLE_TEMPLATE = File.expand_path("templates/kino.rb.tt", __dir__).freeze
 
     # Where the `kino` CLI and the Rack handler look for a config file when
     # none is named: the project root first, then the Rails-style config/.
@@ -145,6 +148,8 @@ module Kino
     #   bind "0.0.0.0"
     #   port 9292
     #   workers 8           # ractors (or thread groups in :threaded mode)
+    #   max_workers 32      # elastic pool ceiling; unset = fixed pool
+    #   scale_down_after 30 # seconds idle before an extra worker retires
     #   threads 3           # threads per worker
     #   mode :ractor        # :auto | :ractor | :threaded
     #   queue_depth 2048
@@ -171,9 +176,19 @@ module Kino
       # Port to listen on; 0 picks an ephemeral port.
       def port(port) = @config.set(:port, Integer(port))
 
-      # Worker count (ractors in :ractor mode); defaults to CPU cores.
+      # Worker count (ractors in :ractor mode); defaults to CPU cores. The
+      # pool floor when max_workers is set.
       def workers(count) = @config.set(:workers, Integer(count))
 
+      # Pool ceiling: under sustained queue pressure the pool grows past
+      # `workers`, one worker at a time, up to this many. Unset (the
+      # default) keeps the pool fixed at `workers`.
+      def max_workers(count) = @config.set(:max_workers, Integer(count))
+
+      # Seconds a worker above the floor must sit idle before it is
+      # retired (default 30). Only meaningful with max_workers.
+      def scale_down_after(seconds) = @config.set(:scale_down_after, seconds)
+
       # Threads per worker (I/O concurrency inside one ractor); default is
       # mode-dependent: 1 in :ractor mode, 3 in :threaded.
       def threads(count) = @config.set(:threads, Integer(count))
@@ -245,6 +260,11 @@ module Kino
       # TLS termination; file paths or inline PEM strings.
       def tls(cert:, key:) = @config.set(:tls, {cert: cert, key: key})
 
+      # HTTP/2: negotiated via ALPN on TLS binds, served to
+      # prior-knowledge clients on plaintext. On by default; set false
+      # to serve HTTP/1 only.
+      def http2(enabled = true) = @config.set(:http2, !!enabled)
+
       # Sets RACK_ENV (unless already set) before the CLI loads the app.
       def environment(env) = @config.set(:environment, env.to_s)
 

data/lib/kino/monitor.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+module Kino
+  # @private
+  # A thread on the main ractor that calls `scan` every `tick` seconds
+  # until stopped. Main-ractor so it stays responsive when worker ractors
+  # are wedged. A scan that raises is logged and skipped, never fatal:
+  # monitors keep the server healthy, they must not take it down.
+  class Monitor
+    def initialize(name:, tick:)
+      @name = name
+      @tick = tick
+      @running = false
+      @thread = nil
+    end
+
+    def start
+      @running = true
+      @thread = Thread.new do
+        Thread.current.name = @name
+        run
+      end
+      self
+    end
+
+    def stop
+      @running = false
+      @thread&.join(@tick * 2)
+    end
+
+    private
+
+    def run
+      tick while @running
+    rescue => e
+      Log.error("#{@name} crashed: #{e.class}: #{e.message}")
+    end
+
+    def tick
+      scan
+    rescue => e
+      Log.error("#{@name} tick error: #{e.class}: #{e.message}")
+    ensure
+      sleep @tick
+    end
+
+    # One poll; subclasses define it.
+    def scan
+      raise NotImplementedError, "#{self.class} must define scan"
+    end
+  end
+end

data/lib/kino/pool_scaler.rb

@@ -0,0 +1,103 @@
+# frozen_string_literal: true
+
+module Kino
+  # @private
+  # Grows and shrinks the worker pool between `floor` and `ceiling`. One
+  # thread on the main ractor polls the native queue and per-slot sensors
+  # every tick and drives a pool (RactorSupervisor or ThreadedPool) through
+  # three methods: `active_count`, `groups` (worker index => slot ids for
+  # every worker that may be retired), `grow`, and `retire(index)`.
+  #
+  # Policy: grow by one worker per tick once requests have been waiting in
+  # the queue on two consecutive ticks (a burst that clears within a tick
+  # is not pressure); retire one worker per tick, the one idle longest,
+  # once it has been idle for `scale_down_after`. "Idle" means no slot of
+  # the worker held a request at two consecutive samples and its served
+  # count did not move between them, so a worker serving short requests
+  # between samples is never mistaken for an idle one.
+  class PoolScaler < Monitor
+    TICK = 0.1
+    # Consecutive ticks with a non-empty queue before the pool grows.
+    PRESSURE_TICKS = 2
+
+    def initialize(server_id:, pool:, floor:, ceiling:, scale_down_after:, tick: TICK)
+      super(name: "pool scaler", tick: tick)
+      @server_id = server_id
+      @pool = pool
+      @floor = floor
+      @ceiling = ceiling
+      @scale_down_after = scale_down_after
+      @pressure = 0
+      @last_served = {}
+      @idle_since = {}
+    end
+
+    # One policy step over one set of observations: the monotonic time,
+    # the queue depth, and worker_stats rows ([slot, served, in_flight,
+    # busy_ms, quarantined, retired]). Public so the policy is testable
+    # without a server.
+    def step(now, queued, rows)
+      @pressure = queued.positive? ? @pressure + 1 : 0
+      if @pressure >= PRESSURE_TICKS
+        grow if @pool.active_count < @ceiling
+        return
+      end
+
+      observe_idle(now, rows)
+      return unless @pool.active_count > @floor
+
+      index, since = @idle_since.min_by { |_index, at| at }
+      return unless index && now - since >= @scale_down_after
+
+      retire(index)
+    end
+
+    private
+
+    def scan
+      queued, _in_flight = Native.queue_stats(@server_id)
+      step(Process.clock_gettime(Process::CLOCK_MONOTONIC), queued, Native.worker_stats(@server_id))
+    end
+
+    def grow
+      index = @pool.grow
+      return unless index
+
+      Native.record_scale_up(@server_id)
+      Log.info("pool grew to #{@pool.active_count} workers (queue pressure)")
+    end
+
+    def retire(index)
+      return unless @pool.retire(index)
+
+      forget(index)
+      Native.record_scale_down(@server_id)
+      Log.info("pool shrank to #{@pool.active_count} workers (worker-#{index} idle)")
+    end
+
+    # Per worker: idle when no slot holds a request at this sample and the
+    # served total did not move since the previous one. First sighting of
+    # a worker is never idle (it takes two samples to know).
+    def observe_idle(now, rows)
+      by_slot = rows.to_h { |row| [row[0], row] }
+      groups = @pool.groups
+      (@idle_since.keys - groups.keys).each { |index| forget(index) }
+      groups.each do |index, slot_ids|
+        served = slot_ids.sum { |id| by_slot.dig(id, 1) || 0 }
+        busy = slot_ids.any? { |id| (by_slot.dig(id, 2) || 0).positive? }
+        idle = !busy && @last_served[index] == served
+        @last_served[index] = served
+        if idle
+          @idle_since[index] ||= now
+        else
+          @idle_since.delete(index)
+        end
+      end
+    end
+
+    def forget(index)
+      @idle_since.delete(index)
+      @last_served.delete(index)
+    end
+  end
+end

data/lib/kino/quarantine_monitor.rb

@@ -3,54 +3,22 @@
 module Kino
   # @private
   # Polls per-slot busy_ms and, past the timeout, quarantines a wedged slot
-  # and asks the replacer to spawn a fresh worker. Runs one thread on the
-  # main ractor (uncontended by wedged worker ractors, so it stays
-  # responsive in :ractor mode). Never interrupts the wedged worker.
-  class QuarantineMonitor
+  # and asks the replacer to spawn a fresh worker. Never interrupts the
+  # wedged worker.
+  class QuarantineMonitor < Monitor
     def initialize(server_id:, timeout_ms:, max:, replacer:, tick: 0.5)
+      super(name: "quarantine monitor", tick: tick)
       @server_id = server_id
       @timeout_ms = timeout_ms
       @max = max
       @replacer = replacer
-      @tick = tick
       @outstanding = 0
       @at_cap_logged = false
-      @running = false
-      @thread = nil
-    end
-
-    def start
-      @running = true
-      @thread = Thread.new do
-        Thread.current.name = "quarantine"
-        run
-      end
-      self
-    end
-
-    def stop
-      @running = false
-      @thread&.join(@tick * 2)
     end
 
     private
 
-    def run
-      tick while @running
-    rescue => e
-      Log.error("quarantine monitor crashed: #{e.class}: #{e.message}")
-    end
-
-    def tick
-      scan_slots
-    rescue => e
-      # A bad tick must never kill the monitor.
-      Log.error("quarantine tick error: #{e.class}: #{e.message}")
-    ensure
-      sleep @tick
-    end
-
-    def scan_slots
+    def scan
       Native.worker_stats(@server_id).each do |index, _served, _in_flight, busy_ms, quarantined|
         next if quarantined || busy_ms <= @timeout_ms