kino 0.5.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 19a396c4f03ec64ac1f598ca8563c69122a0db0a543372c629a56e3ed88d8d06
4
- data.tar.gz: 57c185c402a0f89d0b177c0c017d377abc5fbe8effff999401ae3196426bd6c2
3
+ metadata.gz: 03f08b02b92df6b069d24b28d3abfba6147b2aa6ab854034a6e7619dd1eb771c
4
+ data.tar.gz: 945c95b779ea6c2729d514657b62e7a7014c30b28965122f6a62b7fb2f27ca20
5
5
  SHA512:
6
- metadata.gz: c208de3731186a58be3c4f0ca4f6795fae98405133f0d954ff105a3353892769842cc10bc11fe72dabcaa33c275218e2154eac839cef2fdf729d7959e886429f
7
- data.tar.gz: 549bd8c04aa00b3e2c799d717a8d5bbeb702530a81ea688948b04535b848f1eb03c8e0f52fc1dca362189a00b01269c06ff1d5993e6f2de8e6d0f1e322132f4b
6
+ metadata.gz: 631c65a77e3cae89796e2a7465ed11fbcec652b9bf78fe325d18f77b53f7c26d3922770cf1f42b6b6d3bf07e70ed53dbd639af4e71ba053e77b85508bc6825b4
7
+ data.tar.gz: 7cc673c49dcdefbb7e1f7343a9058426339d2b983507ced5e72aa573106c91219608761f84cc8dd06889292ca8761a88f85e901d49be70eb98afca6707340385
data/CHANGELOG.md CHANGED
@@ -1,3 +1,70 @@
1
+ ## [0.7.0] - 2026-09-08
2
+
3
+ - Experimental elastic worker pool. Set `max_workers` above `workers`
4
+ and the pool grows under load, one worker at a time, then shrinks
5
+ back to `workers` once the extra workers have sat idle for
6
+ `scale_down_after` seconds (default 30). Works in both modes; a
7
+ retiring worker finishes its request first. `stats`, `/stats` and
8
+ `/metrics` gain `max_workers`, `active_workers`, `scale_ups` and
9
+ `scale_downs`, and each `worker_status` row gains `retired`. Leave
10
+ `max_workers` unset and the pool is fixed, as before.
11
+ - Ractor mode warns at boot when `max_workers` (or `workers`) exceeds
12
+ `RUBY_MAX_CPU` (default 8), Ruby's cap on how many ractors run Ruby
13
+ code at once. Set the variable to your worker count to lift it.
14
+ - Ractor-readiness fixes, so external Ractor audits of Kino pass: the
15
+ env string caches root their strings through the lock-free pin slab
16
+ instead of per-value GC registration (unsynchronized across ractors
17
+ in Ruby 4.0, a crash) and no longer call into Ruby while locked (a
18
+ GC-barrier deadlock); the shared rack.errors and rack.input
19
+ singletons must be Ractor-shareable, not just frozen; the Rack
20
+ handler's option table is shareable. Throughput is unchanged.
21
+
22
+ ## [0.6.0] - 2026-09-01
23
+
24
+ - HTTP/2 support. Kino now speaks HTTP/2 natively, on by default:
25
+ browsers negotiate it over TLS (ALPN), h2-preferring balancers and
26
+ tools get prior-knowledge h2c on plaintext, and everyone else keeps
27
+ getting HTTP/1.1 on the same port. `http2 false` restores an
28
+ HTTP/1-only server. What ships with it:
29
+ - A spec-complete Rack env on h2: `SERVER_PROTOCOL` is `"HTTP/2"`,
30
+ `HTTP_HOST`/`SERVER_NAME`/`SERVER_PORT` come from the `:authority`
31
+ pseudo-header, split cookie headers are rejoined, and uploads
32
+ stream with the same backpressure as HTTP/1.
33
+ - Uploads at full speed: body reads now drain every arrived chunk in
34
+ one native call, so h2's 16 KB data frames don't pay a GVL
35
+ round-trip each—h2 uploads run at HTTP/1 parity, and chunked
36
+ HTTP/1 uploads got faster too.
37
+ - `MAX_CONCURRENT_STREAMS` advertised from worker-slot capacity, so
38
+ an h2-aware balancer sees the server's real admission and a single
39
+ connection cannot multiply into hundreds of queued requests.
40
+ - Graceful connection drains: on shutdown, every connection finishes
41
+ its in-flight request and then closes (`Connection: close` on
42
+ HTTP/1, GOAWAY on h2) instead of being cut mid-stream at the
43
+ deadline.
44
+ - Interned header values: low-cardinality headers (user-agent,
45
+ accept-*, sec-ch-*) reuse one frozen string instead of allocating
46
+ per request—the env-side analogue of HPACK's wire dedup, worth a
47
+ few percent on header-heavy traffic and 6-8 fewer allocations per
48
+ request, on HTTP/1 too. Cookies and authorization are deliberately
49
+ never cached.
50
+
51
+ Measured on the reference box: native h2 is +79% over HTTP/1.1
52
+ cleartext and 2× over TLS on the same server, +50% over
53
+ nginx-terminated h2 in front of Kino, ~3× falcon on fast handlers.
54
+ - Leaner request hot path: up to 2-4% more throughput on fast handlers
55
+ with `lanes` (measured on Linux), no change elsewhere.
56
+ - All benchmarks re-measured from scratch on a fresh reference box
57
+ (c7a.2xlarge, Ruby 4.0.6, Puma 8.0.2): the 2026-06 numbers
58
+ reproduced within 1-3% at matched configurations, memory to the
59
+ megabyte. The published tables carry the new run, with two additions
60
+ and one honest downgrade: new studies for sharded I/O (+1-3% on 8
61
+ cores, best at `io_threads 8`) and HTTP/2 (above), and a narrower
62
+ ractor /cpu lead over the cluster (+25%, was +34%) because Puma
63
+ 8.0.2 itself got faster. The bench harness gained the sharded-I/O
64
+ lanes and the Rails memory comparison, and the methodology notes now
65
+ record two measurement traps: memory is only comparable after the
66
+ full endpoint battery, and /io only at equal slot counts.
67
+
1
68
  ## [0.5.0] - 2026-08-29
2
69
 
3
70
  - Add opt-in `io_shards true`: accepted HTTP connections can run on
data/Cargo.lock CHANGED
@@ -184,6 +184,12 @@ dependencies = [
184
184
  "spin",
185
185
  ]
186
186
 
187
+ [[package]]
188
+ name = "fnv"
189
+ version = "1.0.7"
190
+ source = "registry+https://github.com/rust-lang/crates.io-index"
191
+ checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
192
+
187
193
  [[package]]
188
194
  name = "foldhash"
189
195
  version = "0.2.0"
@@ -253,6 +259,25 @@ version = "0.3.4"
253
259
  source = "registry+https://github.com/rust-lang/crates.io-index"
254
260
  checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b"
255
261
 
262
+ [[package]]
263
+ name = "h2"
264
+ version = "0.4.19"
265
+ source = "registry+https://github.com/rust-lang/crates.io-index"
266
+ checksum = "ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16"
267
+ dependencies = [
268
+ "atomic-waker",
269
+ "bytes",
270
+ "fnv",
271
+ "futures-core",
272
+ "futures-sink",
273
+ "http",
274
+ "indexmap",
275
+ "slab",
276
+ "tokio",
277
+ "tokio-util",
278
+ "tracing",
279
+ ]
280
+
256
281
  [[package]]
257
282
  name = "hashbrown"
258
283
  version = "0.17.1"
@@ -319,6 +344,7 @@ dependencies = [
319
344
  "bytes",
320
345
  "futures-channel",
321
346
  "futures-core",
347
+ "h2",
322
348
  "http",
323
349
  "http-body",
324
350
  "httparse",
@@ -327,6 +353,7 @@ dependencies = [
327
353
  "pin-project-lite",
328
354
  "smallvec",
329
355
  "tokio",
356
+ "want",
330
357
  ]
331
358
 
332
359
  [[package]]
@@ -343,6 +370,16 @@ dependencies = [
343
370
  "tokio",
344
371
  ]
345
372
 
373
+ [[package]]
374
+ name = "indexmap"
375
+ version = "2.14.1"
376
+ source = "registry+https://github.com/rust-lang/crates.io-index"
377
+ checksum = "07aa2048142242915a31d35844fb311e0e53fcca590c3a0a40dcf1b841fa09eb"
378
+ dependencies = [
379
+ "equivalent",
380
+ "hashbrown",
381
+ ]
382
+
346
383
  [[package]]
347
384
  name = "itertools"
348
385
  version = "0.13.0"
@@ -407,7 +444,7 @@ dependencies = [
407
444
 
408
445
  [[package]]
409
446
  name = "kino"
410
- version = "0.5.0"
447
+ version = "0.7.0"
411
448
  dependencies = [
412
449
  "ahash",
413
450
  "bytes",
@@ -817,6 +854,12 @@ version = "2.0.1"
817
854
  source = "registry+https://github.com/rust-lang/crates.io-index"
818
855
  checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
819
856
 
857
+ [[package]]
858
+ name = "slab"
859
+ version = "0.4.12"
860
+ source = "registry+https://github.com/rust-lang/crates.io-index"
861
+ checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
862
+
820
863
  [[package]]
821
864
  name = "smallvec"
822
865
  version = "1.15.2"
@@ -926,6 +969,45 @@ dependencies = [
926
969
  "tokio",
927
970
  ]
928
971
 
972
+ [[package]]
973
+ name = "tokio-util"
974
+ version = "0.7.19"
975
+ source = "registry+https://github.com/rust-lang/crates.io-index"
976
+ checksum = "494815d09bf52b5548659851081238f0ca39ff638363907596da739561c62c52"
977
+ dependencies = [
978
+ "bytes",
979
+ "futures-core",
980
+ "futures-sink",
981
+ "libc",
982
+ "pin-project-lite",
983
+ "tokio",
984
+ ]
985
+
986
+ [[package]]
987
+ name = "tracing"
988
+ version = "0.1.44"
989
+ source = "registry+https://github.com/rust-lang/crates.io-index"
990
+ checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
991
+ dependencies = [
992
+ "pin-project-lite",
993
+ "tracing-core",
994
+ ]
995
+
996
+ [[package]]
997
+ name = "tracing-core"
998
+ version = "0.1.36"
999
+ source = "registry+https://github.com/rust-lang/crates.io-index"
1000
+ checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
1001
+ dependencies = [
1002
+ "once_cell",
1003
+ ]
1004
+
1005
+ [[package]]
1006
+ name = "try-lock"
1007
+ version = "0.2.5"
1008
+ source = "registry+https://github.com/rust-lang/crates.io-index"
1009
+ checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
1010
+
929
1011
  [[package]]
930
1012
  name = "unicode-ident"
931
1013
  version = "1.0.24"
@@ -944,6 +1026,15 @@ version = "0.9.5"
944
1026
  source = "registry+https://github.com/rust-lang/crates.io-index"
945
1027
  checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
946
1028
 
1029
+ [[package]]
1030
+ name = "want"
1031
+ version = "0.3.1"
1032
+ source = "registry+https://github.com/rust-lang/crates.io-index"
1033
+ checksum = "bfa7760aed19e106de2c7c0b581b509f2f25d3dacaf737cb82ac61bc6d760b0e"
1034
+ dependencies = [
1035
+ "try-lock",
1036
+ ]
1037
+
947
1038
  [[package]]
948
1039
  name = "wasi"
949
1040
  version = "0.11.1+wasi-snapshot-preview1"
data/README.md CHANGED
@@ -11,11 +11,14 @@ on every core in **one small process**. A **Rust** (tokio + hyper)
11
11
  front-end owns the network, parallel **Ractors** run your Rack 3 app,
12
12
  and a threaded fallback mode runs everything else, Rails included.
13
13
 
14
- * **Fast.** On a real 8-core server, every Kino mode is **1.5-2×**
15
- ahead of a Puma fork cluster on I/O-light endpoints. Ractor mode also
16
- wins on pure CPU, **30%+**. [Benchmarks](#benchmarks) below.
17
- * **A fraction of the memory.** About **~7×** on the simplistic bench
18
- Ractor app, and about **4× less memory** than a Puma cluster serving Rails in fallback threaded mode.
14
+ * **Fast.** On a real 8-core server, every Kino mode is **1.5-1.7×**
15
+ ahead of a Puma fork cluster on I/O-light endpoints—with native
16
+ **HTTP/2** adding another **+79%** over HTTP/1.1 on the same server.
17
+ Ractor mode also wins on pure CPU, **+25%**.
18
+ [Benchmarks](#benchmarks) below.
19
+ * **A fraction of the memory.** About **~8×** on the simplistic bench
20
+ Ractor app, and about **4× less memory** than a Puma cluster serving
21
+ Rails in fallback threaded mode.
19
22
  * **Parallel without forking.** Ractor mode runs CPU work **more than
20
23
  5× faster** than Kino's own GVL-bound threaded mode, in the same
21
24
  small process.
@@ -73,32 +76,32 @@ rather than an app. It is Ractor-shareable, so Kino runs it in `:ractor`
73
76
  mode (and `:threaded` for comparison). **A real Rails app is a different
74
77
  story:** it is *not* Ractor-shareable, so it runs only in Kino's
75
78
  `:threaded` fallback, with its own numbers—see [Rails](#rails) below.
76
- Ruby 4.0.5 with YJIT, every server at its defaults: Puma forks 8 workers ×
77
- 3 threads, Kino stays in one process (8 workers; 1 thread each in ractor
78
- modes, 3 in threaded). Numbers are req/s by wrk (8-second windows, 64
79
- connections, same host). Methodology:
80
- [doc/benchmarks.md](doc/benchmarks.md).
79
+ Ruby 4.0.6 with YJIT (re-measured 2026-09), every server at its
80
+ defaults: Puma forks 8 workers × 3 threads, Kino stays in one process
81
+ (8 workers; 1 thread each in ractor modes, 3 in threaded). Numbers are
82
+ req/s by wrk (8-second windows, 64 connections, same host).
83
+ Methodology: [doc/benchmarks.md](doc/benchmarks.md).
81
84
 
82
85
  | endpoint | Kino :ractor | + lanes | :ractor, `workers 32`² | Kino :threaded | Puma (cluster) |
83
86
  |-------------|-------------:|--------:|-----------------------:|---------------:|---------------:|
84
- | /plaintext | 229,534 | **250,222** | 182,997 | 216,994 | 118,176 |
85
- | /10k | 178,083 | **189,862** | 151,034 | 160,400 | 106,768 |
86
- | /cpu (fib) | **77,999**¹| 70,885 | 66,100 | 13,429 | 58,006 |
87
- | /io (5 ms) | 1,552 | 1,551 | **5,888** | 4,709 | 4,693 |
88
- | /io_native | 1,570 | 1,571 | **6,274** | 4,695 | 4,691 |
87
+ | /plaintext | 222,980 | **244,652** | 115,553 | 213,104 | 142,094 |
88
+ | /10k | 175,182 | **188,365** | 103,147 | 132,009 | 125,116 |
89
+ | /cpu (fib) | **76,695**¹| 71,830 | 54,062 | 13,463 | 61,329 |
90
+ | /io (5 ms) | 1,548 | 1,556 | **5,388** | 4,722 | 4,699 |
91
+ | /io_native | 1,570 | 1,573 | **6,207** | 4,695 | 4,693 |
89
92
 
90
93
  Memory tells two different stories depending on the app, both by **PSS**
91
94
  (proportional set size; see note) after sustained load.
92
95
 
93
96
  **The tiny benchmark app** (Ractor-shareable, so Kino runs it in `:ractor`
94
- or `:threaded`). Kino is **~7× lighter in :ractor mode, ~10× in :threaded**
95
- than the Puma cluster the gap stays large because a trivial app is almost
97
+ or `:threaded`). Kino is **~8× lighter in :ractor mode, ~10× in :threaded**
98
+ than the Puma cluster—the gap stays large because a trivial app is almost
96
99
  all private per-worker heap, which copy-on-write can't share:
97
100
 
98
101
  | tiny app, Kino | Kino (one process) | Puma cluster (8 workers) | ratio |
99
102
  |-----------------|-------------------:|-------------------------:|------:|
100
- | :ractor (8×1) | **148 MB** | 1,068 MB | ~7× |
101
- | :threaded (8×3) | **107 MB**³| 1,068 MB | ~10× |
103
+ | :ractor (8×1) | **135 MB** | 1,072 MB | ~8× |
104
+ | :threaded (8×3) | **107 MB**³| 1,072 MB | ~10× |
102
105
 
103
106
  **A real Rails app** (not Ractor-shareable—Kino's `:threaded` fallback
104
107
  only, [below](#rails)). The gap is **~4×**, smaller because Rails' large
@@ -106,24 +109,25 @@ framework *is* shared copy-on-write across Puma's forks:
106
109
 
107
110
  | Rails hello-world | Kino :threaded | Puma cluster (8 workers) | ratio |
108
111
  |-------------------|---------------:|-------------------------:|------:|
109
- | **PSS** | **92 MB** | **389 MB** | ~4× |
112
+ | **PSS** | **95 MB** | **405 MB** | ~4× |
110
113
 
111
114
  "+ lanes" is the experimental per-worker-queue dispatcher (`lanes true`).
112
115
  It posts the fastest plaintext/10k of any configuration here. Details:
113
116
  [doc/benchmarks.md](doc/benchmarks.md#lane-dispatch-experimental-lanes-true).
114
117
 
115
118
  ¹ Stock settings, no tuning. Ractor mode beats the fork cluster on pure
116
- CPU by +34% (+22% with lanes). Threaded mode shows the GVL ceiling that
117
- every single-process Ruby server hits. The old CPU-tuning recipe is
118
- retired: its `threads 1` half **is** the default now, and its
119
- `tokio_threads 1` half costs −12% on real hardware; see
119
+ CPU by +25% (+17% with lanes; Puma 8.0.2 narrowed this from the +34%
120
+ measured against 7.x). Threaded mode shows the GVL ceiling that every
121
+ single-process Ruby server hits. The old CPU-tuning recipe stays
122
+ retired: its `threads 1` half **is** the default, and its
123
+ `tokio_threads 1` half still costs on real hardware; see
120
124
  [doc/benchmarks.md](doc/benchmarks.md#cpu-bound-tuning).
121
125
 
122
126
  ² Wait-bound throughput is slots ÷ wait, and the default columns bring
123
127
  8 single-thread workers against the cluster's 24 threads. Kino slots
124
128
  are threads, not processes—when your app waits a lot, raise `workers`.
125
- The `workers 32` column is that tuning: **+25% over the cluster on /io
126
- (+34% via `Kino.sleep`)** while still ahead of it on pure CPU, all in
129
+ The `workers 32` column is that tuning: **+15% over the cluster on /io
130
+ (+32% via `Kino.sleep`)** while still ahead of it on pure CPU, all in
127
131
  one small process. The cost is the CPU-light rows (32 ractors
128
132
  oversubscribe 8 cores); pick the topology your app's wait profile
129
133
  needs. See
@@ -131,7 +135,7 @@ needs. See
131
135
 
132
136
  ³ With `MALLOC_ARENA_MAX=2` (the standard Ruby deployment setting;
133
137
  Heroku's default). Without it, 24 threads churning 10 KB responses
134
- through one glibc heap balloon to ~670 MB—an arena-fragmentation
138
+ through one glibc heap balloon to ~647 MB—an arena-fragmentation
135
139
  footgun, not a leak, and ractor mode sidesteps it. See
136
140
  [doc/benchmarks.md](doc/benchmarks.md#memory-under-load-and-the-glibc-arena-footgun).
137
141
 
@@ -141,36 +145,64 @@ doc):
141
145
 
142
146
  | endpoint | Kino :ractor (8×3) | Puma + ractor wrapper | Falcon + ractor wrapper |
143
147
  |------------|-------------------:|----------------------:|------------------------:|
144
- | /plaintext | **193,826** | 19,480 | 99,776 |
145
- | /cpu (fib) | **68,061** | 17,755 | 48,721 |
146
- | /io (5 ms) | **4,530** | 1,454 | 1,549 |
148
+ | /plaintext | **190,206** | 22,055 | 107,203 |
149
+ | /cpu (fib) | **68,230** | 16,528 | 50,101 |
150
+ | /io (5 ms) | **4,477** | 1,482 | 1,545 |
151
+
152
+ (The Kino column here runs 8×3—the same 24 slots the wrappers get—so
153
+ the /io row is comparable; the main table's ractor column runs the 8×1
154
+ default.)
155
+
156
+ ### HTTP/2 head-to-head
157
+
158
+ Kino speaks HTTP/2 natively, so it skips the usual nginx-termination
159
+ hop. Same box as the tables above; all over TLS, h2load, 64 in-flight,
160
+ same app:
161
+
162
+ | /plaintext | req/s | /upload (64 KB) |
163
+ |-------------------------------|------------:|----------------:|
164
+ | Kino native h2 | **164,044** | **19,226** |
165
+ | Kino HTTP/1.1 (same boot) | 80,789 | 17,210 |
166
+ | nginx h2 → Kino HTTP/1.1 | 109,219 | 1,217¹ |
167
+ | Falcon (native h2) | 55,637 | 18,838 |
168
+
169
+ Native h2 **doubles** HTTP/1.1 throughput on the same TLS server
170
+ (+79% on cleartext h2c; fewer, larger socket operations, and HPACK
171
+ spares re-sending cookies) and is **+50% over fronting the same Kino
172
+ with nginx**—the proxy hop is pure cost. Uploads, h2's classic weak
173
+ spot, run at HTTP/1 parity. Full matrix, cleartext h2c lanes, and
174
+ methodology: [doc/benchmarks.md](doc/benchmarks.md#http2); reproduce
175
+ with `bench/h2.sh`.
176
+
177
+ ¹ nginx default-config h2 request-body flow control; see the doc.
147
178
 
148
179
  ### Rails
149
180
 
150
181
  Rails is not Ractor-shareable today, so Kino serves it in `:threaded`
151
- fallback one GVL-bound process. On the same box (`examples/rails-hello`,
182
+ fallback—one GVL-bound process. On the same box (`examples/rails-hello`,
152
183
  edge Rails, production, 8×5):
153
184
 
154
185
  | Rails hello-world | req/s | memory (PSS) |
155
186
  |------------------------------|-------:|-------------:|
156
- | Kino :threaded (one process) | 2,637 | **92 MB** |
157
- | Puma cluster (8 workers) | 12,138 | 389 MB |
187
+ | Kino :threaded (one process) | 2,731 | **95 MB** |
188
+ | Puma cluster (8 workers) | 15,422 | 405 MB |
158
189
 
159
190
  The honest trade-off: Puma's fork cluster uses all 8 cores, so it serves
160
- ~4.6× the throughput at ~4× the memory. Ractor-mode Rails would close
191
+ ~5.6× the throughput—at ~4× the memory. Ractor-mode Rails would close
161
192
  the throughput gap at one-process memory cost; the upstream blockers are
162
193
  tracked in [doc/rails-on-ractors.md](doc/rails-on-ractors.md).
163
194
 
164
- In short: on the tiny synthetic app, ractor mode beats fork-level CPU parallelism (**5.8×** Kino's
165
- own GVL-bound threaded mode, +34% over the cluster) in one process, at
166
- about 1/7th of the cluster's memory by PSS (~4× on a real Rails app).
167
- Every Kino mode is 1.5-2.1× ahead of the cluster on I/O-light endpoints. The macOS numbers
195
+ In short: on the tiny synthetic app, ractor mode beats fork-level CPU parallelism (**5.7×** Kino's
196
+ own GVL-bound threaded mode, +25% over the cluster) in one process, at
197
+ about 1/8th of the cluster's memory by PSS (~4× on a real Rails app).
198
+ Every Kino mode is 1.5-1.7× ahead of the cluster on I/O-light endpoints,
199
+ and native HTTP/2 adds +79% over HTTP/1.1 on top. The macOS numbers
168
200
  (secondary; everything there hits the loopback ceiling) and the
169
201
  YJIT × Ractors gotcha are in [doc/benchmarks.md](doc/benchmarks.md).
170
202
 
171
203
  Reproduce: `bench/run.sh [seconds] [concurrency]` for the main table,
172
204
  `bench/studies.sh` for the follow-ups (CPU recipe, topology, scaling,
173
- logging, memory).
205
+ sharded I/O, logging, memory), `bench/h2.sh` for the HTTP/2 matrix.
174
206
 
175
207
  ## Install
176
208
 
@@ -227,6 +259,8 @@ server = Kino::Server.new(app,
227
259
  bind: "127.0.0.1", # or "unix:///run/kino.sock" behind a proxy
228
260
  port: 9292, # 0 = ephemeral; read back via server.port
229
261
  workers: Kino.available_parallelism, # ractors (parallelism); the default
262
+ max_workers: nil, # experimental: grow past workers under load (see Elastic pool)
263
+ scale_down_after: 30, # seconds idle before an extra worker retires
230
264
  threads: 1, # per worker; ractor default 1, threaded default 3
231
265
  mode: :auto, # :auto | :ractor | :threaded
232
266
  queue_depth: 1024, # bounded queue; overflow → 503
@@ -239,6 +273,7 @@ server = Kino::Server.new(app,
239
273
  control_bind: "127.0.0.1:9293", # monitoring: /stats /metrics /ready /live; port 0 reads back via server.control_port
240
274
  control_token: ENV["KINO_CONTROL_TOKEN"], # optional Bearer auth for /stats + /metrics
241
275
  tls: { cert: "cert.pem", key: "key.pem" }, # file paths or inline PEM
276
+ http2: true, # ALPN h2 on TLS + plaintext h2c; false = HTTP/1 only
242
277
  )
243
278
  server.start
244
279
  server.shutdown # graceful: drain → deadline → abort stragglers
@@ -265,15 +300,39 @@ server.shutdown # graceful: drain → deadline → abort straggler
265
300
  multi-thread runtime onto current-thread shards: one thread accepts and
266
301
  hands each connection to the least-loaded shard, which then owns it for
267
302
  its lifetime—no work-stealing, no cross-thread wakeups on the hot path.
268
- Fast handlers gain double-digit throughput; Ruby-bound endpoints are
269
- unchanged. Orthogonal to `mode`: it reshapes the Rust side only.
303
+ On the 8-core reference box fast handlers gain +1-3% (best with
304
+ `io_threads 8`); the win grows with scheduler contention, so measure on
305
+ your own core count. Ruby-bound endpoints are unchanged. Orthogonal to
306
+ `mode`: it reshapes the Rust side only.
270
307
 
271
308
  ```ruby
272
309
  # kino.rb
273
310
  io_shards true
274
- io_threads 4 # optional; default: half the available CPUs
311
+ io_threads 8 # optional; default: half the available CPUs
275
312
  ```
276
313
 
314
+ ### HTTP/2
315
+
316
+ On by default, on both transports, with nothing to configure:
317
+
318
+ - **TLS binds** advertise `h2` via ALPN, so browsers and h2-capable
319
+ clients negotiate HTTP/2 and everything else stays on HTTP/1.1.
320
+ - **Plaintext binds** serve prior-knowledge h2c: a client that opens
321
+ with the HTTP/2 preface (an h2-preferring load balancer, a gRPC-style
322
+ backend hop, `curl --http2-prior-knowledge`) gets HTTP/2; ordinary
323
+ clients are HTTP/1.1 exactly as before. Browsers never do h2 on
324
+ plaintext, so a certificate-less kino behaves identically for them.
325
+
326
+ The Rack side is spec-complete on h2: `SERVER_PROTOCOL` is `"HTTP/2"`,
327
+ `HTTP_HOST`/`SERVER_NAME`/`SERVER_PORT` come from the `:authority`
328
+ pseudo-header (h2 requests carry no Host header), split cookie headers
329
+ are rejoined with `"; "`, and streamed uploads flow through the same
330
+ backpressured body channel as HTTP/1. Streams multiplex into the same
331
+ worker slots as keep-alive requests—`workers × threads` bounds
332
+ concurrency either way. (`rack.hijack` stays out on h2 as it is on h1;
333
+ the protocol has no 101 upgrade to hijack anyway.) `http2 false` pins
334
+ the server to HTTP/1 and drops `h2` from ALPN.
335
+
277
336
  ## Config file and CLI
278
337
 
279
338
  Settings can live in a Puma-style Ruby DSL file: `kino.rb` in the
@@ -361,6 +420,56 @@ Kino fires four lifecycle hooks alongside `on_error`, split by firing context.
361
420
 
362
421
  A raising hook is logged and never kills a worker.
363
422
 
423
+ ## Elastic pool (experimental)
424
+
425
+ Size the pool for the quiet hours and let it grow for the busy ones:
426
+
427
+ ```ruby
428
+ # kino.rb
429
+ workers 4 # always running
430
+ max_workers 16 # reached only under load
431
+ scale_down_after 30 # seconds idle before an extra worker retires
432
+ ```
433
+
434
+ Or `Kino::Server.new(app, workers: 4, max_workers: 16)`. Leave
435
+ `max_workers` unset and the pool is fixed at `workers`, as before.
436
+
437
+ While requests wait in the queue, Kino adds a worker every 100 ms until
438
+ the queue clears or the pool hits `max_workers`. When the load passes,
439
+ workers above `workers` retire one at a time after `scale_down_after`
440
+ seconds idle, each finishing its current request first. Same behavior
441
+ in `:ractor` and `:threaded` mode.
442
+
443
+ **Use it when your app waits**: on databases, upstream services, slow
444
+ clients. With `workers` at your core count, all workers can be blocked
445
+ on I/O while cores sit idle; a higher ceiling puts those cores to work,
446
+ and a ractor starts in microseconds, so the pool follows load closely.
447
+ Pure CPU work gains nothing past the core count. In `:ractor` mode Ruby
448
+ itself runs at most `RUBY_MAX_CPU` ractors' Ruby code at once (default
449
+ 8), and Kino warns at boot when the pool can exceed it. On a bigger box:
450
+
451
+ ```sh
452
+ RUBY_MAX_CPU=16 kino
453
+ ```
454
+
455
+ **Watch it breathe** in `server.stats`, `GET /stats` and `GET /metrics`:
456
+
457
+ ```sh
458
+ $ curl -s localhost:9293/stats | jq '{workers, max_workers, active_workers, scale_ups, scale_downs}'
459
+ {
460
+ "workers": 4,
461
+ "max_workers": 16,
462
+ "active_workers": 9,
463
+ "scale_ups": 12,
464
+ "scale_downs": 7
465
+ }
466
+ ```
467
+
468
+ Prometheus gets `kino_max_workers`, `kino_active_workers`,
469
+ `kino_scale_ups_total` and `kino_scale_downs_total`. `after_worker_boot`
470
+ fires for every worker the pool adds, `on_worker_exit` (with a nil
471
+ cause) for every one it retires.
472
+
364
473
  ## Stuck-worker quarantine
365
474
 
366
475
  `quarantine_timeout: seconds` (or `quarantine_timeout 60` in `kino.rb`)
@@ -439,8 +548,8 @@ not `queue_time`.
439
548
 
440
549
  ## Logging
441
550
 
442
- With one log line per request, `Kino::Logger` sustained **2.4× the
443
- throughput of a shared `::Logger`** (149k vs 63k req/s on the benchmark
551
+ With one log line per request, `Kino::Logger` sustained **1.7× the
552
+ throughput of a shared `::Logger`** (155k vs 90k req/s on the benchmark
444
553
  box). There are two native pieces. Both write through a lock-free
445
554
  channel to a Rust flusher thread, so request threads never take a log
446
555
  mutex and never make a write syscall:
data/doc/architecture.md CHANGED
@@ -31,6 +31,16 @@ Puma-style two-level: `workers × threads`.
31
31
  - Identical machinery either way: the flume queue is MPMC, a "worker slot"
32
32
  is per-thread, and the worker loop (`lib/kino/worker.rb`) is shared
33
33
  verbatim.
34
+ - Elastic pool (`max_workers`): a scaler thread on the main ractor
35
+ samples queue depth and the per-slot sensors every 100 ms, adds one
36
+ worker per sample while requests wait, and retires the longest-idle
37
+ worker above the floor after `scale_down_after`. Retirement is a
38
+ per-slot flag raised under the slot's lane lock: the lane dispatcher
39
+ skips the slot, the take loop honors the flag at its next idle tick (a
40
+ request already taken finishes first, a lane worker drains its own
41
+ lane), and the slot is reset and reused by the next worker, so the
42
+ slot table never grows with churn. Both pools (the ractor supervisor
43
+ and the threaded pool) expose the same grow/retire/groups seam.
34
44
  - Experimental `lanes true` replaces the one shared queue with a small
35
45
  private queue per worker slot (awake-preferring dispatch, work
36
46
  stealing); see [benchmarks](benchmarks.md#lane-dispatch-experimental-lanes-true).
@@ -109,13 +119,63 @@ plain decrement after an `.await` would never run).
109
119
 
110
120
  ## Graceful shutdown
111
121
 
112
- `stop_accepting` → drain until queue + in-flight reach zero or the
113
- deadline passes `close_queue` (idle workers see Disconnected and exit) →
122
+ `stop_accepting` → every live connection switches to graceful shutdown
123
+ (finish the in-flight request, answer with `Connection: close` on
124
+ HTTP/1 or GOAWAY on h2, take nothing new—so drains converge instead
125
+ of chasing chatty keep-alive clients) → drain until queue + in-flight
126
+ reach zero or the deadline passes → `close_queue` (idle workers see
127
+ Disconnected and exit) →
114
128
  join workers → past deadline: abort remaining clients (a 500, or a
115
129
  connection abort mid-stream), interrupt blocked workers, reap
116
130
  stragglers → tear down the tokio runtime. Idempotent;
117
131
  a second INT/TERM force-exits.
118
132
 
133
+ ## HTTP/2
134
+
135
+ One connection builder serves every protocol: hyper-util's auto builder
136
+ picks h2 by ALPN on TLS connections, by the 24-byte preface sniff on
137
+ plaintext (prior-knowledge h2c), and HTTP/1.x otherwise; `http2 false`
138
+ pins the HTTP/1 codec and skips the sniff entirely. Streams multiplex
139
+ into the same bounded queue as keep-alive requests, so h2 concurrency
140
+ is bounded by workers × threads exactly as h1's is, and the bounded
141
+ body channels give per-stream backpressure for free: while the
142
+ forwarder blocks, hyper withholds WINDOW_UPDATE and the client stalls.
143
+
144
+ The env bridge fills SERVER_NAME/SERVER_PORT/HTTP_HOST from the request
145
+ URI's `:authority` (h2 requests carry no Host header), through the same
146
+ host LRU the Host-header path uses, keyed by the authority bytes;
147
+ SERVER_PROTOCOL is the interned "HTTP/2". h2 trailer frames are dropped
148
+ (Rack has no trailer surface), and the h2 codec itself rejects
149
+ connection-ish headers before they can reach the env.
150
+
151
+ The upload path needed one h2-shaped fix: `read_body` drains every
152
+ already-queued chunk in a single native call (one GVL round-trip and
153
+ one Ruby string per 64 KB read), because a body arriving as 16 KB DATA
154
+ frames otherwise paid one crossing per frame—that alone took h2
155
+ uploads from half of h1's throughput to parity. A knob sweep over
156
+ hyper's h2 codec (frame size, adaptive windows, window sizes) moved
157
+ nothing after that, so hyper's defaults stay.
158
+
159
+ SETTINGS_MAX_CONCURRENT_STREAMS is derived from slot capacity
160
+ (workers × threads, clamped to [8, 1024]) rather than hyper's flat 200:
161
+ a smart balancer sees the server's true admission, and a hostile client
162
+ cannot multiply one connection into hundreds of queued requests—the
163
+ h2 analogue of h1's one-request-per-connection shape. The codec's own
164
+ abuse bounds ship as hyper/h2 defaults and were reviewed: 16 KB header
165
+ lists, 20 pending remote resets then GOAWAY (rapid reset), per-second
166
+ reset-churn and empty-frame budgets.
167
+
168
+ Low-cardinality header values (UA, accept-*, sec-ch-*, sec-fetch-*)
169
+ are interned in an LRU of frozen strings—the env-side analogue of
170
+ HPACK's wire dedup, and the practical form of it: hyper does not expose
171
+ HPACK table indices, so the cache keys on value bytes and works for
172
+ HTTP/1 too. Cookie and authorization are deliberately excluded
173
+ (per-user cardinality, secret lifetime).
174
+
175
+ Deferred until benchmarks justify it: slot-aware flow-control window
176
+ grants (a memory/abuse lever, not a throughput one—the knob sweep
177
+ showed windows don't gate upload throughput).
178
+
119
179
  ## Timer waits: `Kino.sleep`
120
180
 
121
181
  MRI's `sleep` parks the thread on the VM timer, whose wakeups inside
@@ -128,7 +188,8 @@ at the interrupt tick so `Thread#kill` and shutdown stay responsive.
128
188
 
129
189
  - **tokio + hyper**: the bottleneck is the Ruby dispatch boundary, not raw
130
190
  I/O throughput; what matters is HTTP correctness, keep-alive, TLS, and
131
- h2-later—hyper's territory. Cross-platform out of the box.
191
+ h2 (since shipped, via hyper-util's protocol-auto builder)—hyper's
192
+ territory. Cross-platform out of the box.
132
193
  - **monoio**: thread-per-core io_uring looks great in echo-server
133
194
  benchmarks, but hyper only works through its poll-io compat layer
134
195
  (forfeiting io_uring on the hot path), and the share-nothing advantage