kino 0.5.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +67 -0
- data/Cargo.lock +92 -1
- data/README.md +153 -44
- data/doc/architecture.md +64 -3
- data/doc/benchmarks.md +120 -26
- data/ext/kino/Cargo.toml +8 -3
- data/ext/kino/src/control.rs +103 -6
- data/ext/kino/src/env_strings.rs +350 -69
- data/ext/kino/src/lib.rs +22 -7
- data/ext/kino/src/pin.rs +21 -9
- data/ext/kino/src/queue.rs +158 -46
- data/ext/kino/src/registry.rs +145 -0
- data/ext/kino/src/request.rs +151 -70
- data/ext/kino/src/response.rs +39 -5
- data/ext/kino/src/server.rs +864 -33
- data/ext/kino/src/test_support.rs +44 -0
- data/ext/kino/src/tls.rs +119 -9
- data/lib/kino/cli.rb +12 -4
- data/lib/kino/configuration.rb +22 -2
- data/lib/kino/monitor.rb +52 -0
- data/lib/kino/pool_scaler.rb +103 -0
- data/lib/kino/quarantine_monitor.rb +5 -37
- data/lib/kino/ractor_supervisor.rb +102 -8
- data/lib/kino/server.rb +66 -85
- data/lib/kino/slot_bank.rb +31 -0
- data/lib/kino/templates/kino.rb.tt +20 -0
- data/lib/kino/threaded_pool.rb +186 -0
- data/lib/kino/version.rb +1 -1
- data/lib/kino/worker.rb +14 -11
- data/lib/kino.rb +4 -0
- data/lib/rackup/handler/kino.rb +4 -4
- data/sig/kino.rbs +3 -0
- metadata +19 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 03f08b02b92df6b069d24b28d3abfba6147b2aa6ab854034a6e7619dd1eb771c
|
|
4
|
+
data.tar.gz: 945c95b779ea6c2729d514657b62e7a7014c30b28965122f6a62b7fb2f27ca20
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 631c65a77e3cae89796e2a7465ed11fbcec652b9bf78fe325d18f77b53f7c26d3922770cf1f42b6b6d3bf07e70ed53dbd639af4e71ba053e77b85508bc6825b4
|
|
7
|
+
data.tar.gz: 7cc673c49dcdefbb7e1f7343a9058426339d2b983507ced5e72aa573106c91219608761f84cc8dd06889292ca8761a88f85e901d49be70eb98afca6707340385
|
data/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,70 @@
|
|
|
1
|
+
## [0.7.0] - 2026-09-08
|
|
2
|
+
|
|
3
|
+
- Experimental elastic worker pool. Set `max_workers` above `workers`
|
|
4
|
+
and the pool grows under load, one worker at a time, then shrinks
|
|
5
|
+
back to `workers` once the extra workers have sat idle for
|
|
6
|
+
`scale_down_after` seconds (default 30). Works in both modes; a
|
|
7
|
+
retiring worker finishes its request first. `stats`, `/stats` and
|
|
8
|
+
`/metrics` gain `max_workers`, `active_workers`, `scale_ups` and
|
|
9
|
+
`scale_downs`, and each `worker_status` row gains `retired`. Leave
|
|
10
|
+
`max_workers` unset and the pool is fixed, as before.
|
|
11
|
+
- Ractor mode warns at boot when `max_workers` (or `workers`) exceeds
|
|
12
|
+
`RUBY_MAX_CPU` (default 8), Ruby's cap on how many ractors run Ruby
|
|
13
|
+
code at once. Set the variable to your worker count to lift it.
|
|
14
|
+
- Ractor-readiness fixes, so external Ractor audits of Kino pass: the
|
|
15
|
+
env string caches root their strings through the lock-free pin slab
|
|
16
|
+
instead of per-value GC registration (unsynchronized across ractors
|
|
17
|
+
in Ruby 4.0, a crash) and no longer call into Ruby while locked (a
|
|
18
|
+
GC-barrier deadlock); the shared rack.errors and rack.input
|
|
19
|
+
singletons must be Ractor-shareable, not just frozen; the Rack
|
|
20
|
+
handler's option table is shareable. Throughput is unchanged.
|
|
21
|
+
|
|
22
|
+
## [0.6.0] - 2026-09-01
|
|
23
|
+
|
|
24
|
+
- HTTP/2 support. Kino now speaks HTTP/2 natively, on by default:
|
|
25
|
+
browsers negotiate it over TLS (ALPN), h2-preferring balancers and
|
|
26
|
+
tools get prior-knowledge h2c on plaintext, and everyone else keeps
|
|
27
|
+
getting HTTP/1.1 on the same port. `http2 false` restores an
|
|
28
|
+
HTTP/1-only server. What ships with it:
|
|
29
|
+
- A spec-complete Rack env on h2: `SERVER_PROTOCOL` is `"HTTP/2"`,
|
|
30
|
+
`HTTP_HOST`/`SERVER_NAME`/`SERVER_PORT` come from the `:authority`
|
|
31
|
+
pseudo-header, split cookie headers are rejoined, and uploads
|
|
32
|
+
stream with the same backpressure as HTTP/1.
|
|
33
|
+
- Uploads at full speed: body reads now drain every arrived chunk in
|
|
34
|
+
one native call, so h2's 16 KB data frames don't pay a GVL
|
|
35
|
+
round-trip each—h2 uploads run at HTTP/1 parity, and chunked
|
|
36
|
+
HTTP/1 uploads got faster too.
|
|
37
|
+
- `MAX_CONCURRENT_STREAMS` advertised from worker-slot capacity, so
|
|
38
|
+
an h2-aware balancer sees the server's real admission and a single
|
|
39
|
+
connection cannot multiply into hundreds of queued requests.
|
|
40
|
+
- Graceful connection drains: on shutdown, every connection finishes
|
|
41
|
+
its in-flight request and then closes (`Connection: close` on
|
|
42
|
+
HTTP/1, GOAWAY on h2) instead of being cut mid-stream at the
|
|
43
|
+
deadline.
|
|
44
|
+
- Interned header values: low-cardinality headers (user-agent,
|
|
45
|
+
accept-*, sec-ch-*) reuse one frozen string instead of allocating
|
|
46
|
+
per request—the env-side analogue of HPACK's wire dedup, worth a
|
|
47
|
+
few percent on header-heavy traffic and 6-8 fewer allocations per
|
|
48
|
+
request, on HTTP/1 too. Cookies and authorization are deliberately
|
|
49
|
+
never cached.
|
|
50
|
+
|
|
51
|
+
Measured on the reference box: native h2 is +79% over HTTP/1.1
|
|
52
|
+
cleartext and 2× over TLS on the same server, +50% over
|
|
53
|
+
nginx-terminated h2 in front of Kino, ~3× falcon on fast handlers.
|
|
54
|
+
- Leaner request hot path: up to 2-4% more throughput on fast handlers
|
|
55
|
+
with `lanes` (measured on Linux), no change elsewhere.
|
|
56
|
+
- All benchmarks re-measured from scratch on a fresh reference box
|
|
57
|
+
(c7a.2xlarge, Ruby 4.0.6, Puma 8.0.2): the 2026-06 numbers
|
|
58
|
+
reproduced within 1-3% at matched configurations, memory to the
|
|
59
|
+
megabyte. The published tables carry the new run, with two additions
|
|
60
|
+
and one honest downgrade: new studies for sharded I/O (+1-3% on 8
|
|
61
|
+
cores, best at `io_threads 8`) and HTTP/2 (above), and a narrower
|
|
62
|
+
ractor /cpu lead over the cluster (+25%, was +34%) because Puma
|
|
63
|
+
8.0.2 itself got faster. The bench harness gained the sharded-I/O
|
|
64
|
+
lanes and the Rails memory comparison, and the methodology notes now
|
|
65
|
+
record two measurement traps: memory is only comparable after the
|
|
66
|
+
full endpoint battery, and /io only at equal slot counts.
|
|
67
|
+
|
|
1
68
|
## [0.5.0] - 2026-08-29
|
|
2
69
|
|
|
3
70
|
- Add opt-in `io_shards true`: accepted HTTP connections can run on
|
data/Cargo.lock
CHANGED
|
@@ -184,6 +184,12 @@ dependencies = [
|
|
|
184
184
|
"spin",
|
|
185
185
|
]
|
|
186
186
|
|
|
187
|
+
[[package]]
|
|
188
|
+
name = "fnv"
|
|
189
|
+
version = "1.0.7"
|
|
190
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
191
|
+
checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
|
|
192
|
+
|
|
187
193
|
[[package]]
|
|
188
194
|
name = "foldhash"
|
|
189
195
|
version = "0.2.0"
|
|
@@ -253,6 +259,25 @@ version = "0.3.4"
|
|
|
253
259
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
254
260
|
checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b"
|
|
255
261
|
|
|
262
|
+
[[package]]
|
|
263
|
+
name = "h2"
|
|
264
|
+
version = "0.4.19"
|
|
265
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
266
|
+
checksum = "ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16"
|
|
267
|
+
dependencies = [
|
|
268
|
+
"atomic-waker",
|
|
269
|
+
"bytes",
|
|
270
|
+
"fnv",
|
|
271
|
+
"futures-core",
|
|
272
|
+
"futures-sink",
|
|
273
|
+
"http",
|
|
274
|
+
"indexmap",
|
|
275
|
+
"slab",
|
|
276
|
+
"tokio",
|
|
277
|
+
"tokio-util",
|
|
278
|
+
"tracing",
|
|
279
|
+
]
|
|
280
|
+
|
|
256
281
|
[[package]]
|
|
257
282
|
name = "hashbrown"
|
|
258
283
|
version = "0.17.1"
|
|
@@ -319,6 +344,7 @@ dependencies = [
|
|
|
319
344
|
"bytes",
|
|
320
345
|
"futures-channel",
|
|
321
346
|
"futures-core",
|
|
347
|
+
"h2",
|
|
322
348
|
"http",
|
|
323
349
|
"http-body",
|
|
324
350
|
"httparse",
|
|
@@ -327,6 +353,7 @@ dependencies = [
|
|
|
327
353
|
"pin-project-lite",
|
|
328
354
|
"smallvec",
|
|
329
355
|
"tokio",
|
|
356
|
+
"want",
|
|
330
357
|
]
|
|
331
358
|
|
|
332
359
|
[[package]]
|
|
@@ -343,6 +370,16 @@ dependencies = [
|
|
|
343
370
|
"tokio",
|
|
344
371
|
]
|
|
345
372
|
|
|
373
|
+
[[package]]
|
|
374
|
+
name = "indexmap"
|
|
375
|
+
version = "2.14.1"
|
|
376
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
377
|
+
checksum = "07aa2048142242915a31d35844fb311e0e53fcca590c3a0a40dcf1b841fa09eb"
|
|
378
|
+
dependencies = [
|
|
379
|
+
"equivalent",
|
|
380
|
+
"hashbrown",
|
|
381
|
+
]
|
|
382
|
+
|
|
346
383
|
[[package]]
|
|
347
384
|
name = "itertools"
|
|
348
385
|
version = "0.13.0"
|
|
@@ -407,7 +444,7 @@ dependencies = [
|
|
|
407
444
|
|
|
408
445
|
[[package]]
|
|
409
446
|
name = "kino"
|
|
410
|
-
version = "0.
|
|
447
|
+
version = "0.7.0"
|
|
411
448
|
dependencies = [
|
|
412
449
|
"ahash",
|
|
413
450
|
"bytes",
|
|
@@ -817,6 +854,12 @@ version = "2.0.1"
|
|
|
817
854
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
818
855
|
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
|
|
819
856
|
|
|
857
|
+
[[package]]
|
|
858
|
+
name = "slab"
|
|
859
|
+
version = "0.4.12"
|
|
860
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
861
|
+
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
|
|
862
|
+
|
|
820
863
|
[[package]]
|
|
821
864
|
name = "smallvec"
|
|
822
865
|
version = "1.15.2"
|
|
@@ -926,6 +969,45 @@ dependencies = [
|
|
|
926
969
|
"tokio",
|
|
927
970
|
]
|
|
928
971
|
|
|
972
|
+
[[package]]
|
|
973
|
+
name = "tokio-util"
|
|
974
|
+
version = "0.7.19"
|
|
975
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
976
|
+
checksum = "494815d09bf52b5548659851081238f0ca39ff638363907596da739561c62c52"
|
|
977
|
+
dependencies = [
|
|
978
|
+
"bytes",
|
|
979
|
+
"futures-core",
|
|
980
|
+
"futures-sink",
|
|
981
|
+
"libc",
|
|
982
|
+
"pin-project-lite",
|
|
983
|
+
"tokio",
|
|
984
|
+
]
|
|
985
|
+
|
|
986
|
+
[[package]]
|
|
987
|
+
name = "tracing"
|
|
988
|
+
version = "0.1.44"
|
|
989
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
990
|
+
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
|
|
991
|
+
dependencies = [
|
|
992
|
+
"pin-project-lite",
|
|
993
|
+
"tracing-core",
|
|
994
|
+
]
|
|
995
|
+
|
|
996
|
+
[[package]]
|
|
997
|
+
name = "tracing-core"
|
|
998
|
+
version = "0.1.36"
|
|
999
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1000
|
+
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
|
|
1001
|
+
dependencies = [
|
|
1002
|
+
"once_cell",
|
|
1003
|
+
]
|
|
1004
|
+
|
|
1005
|
+
[[package]]
|
|
1006
|
+
name = "try-lock"
|
|
1007
|
+
version = "0.2.5"
|
|
1008
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1009
|
+
checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
|
|
1010
|
+
|
|
929
1011
|
[[package]]
|
|
930
1012
|
name = "unicode-ident"
|
|
931
1013
|
version = "1.0.24"
|
|
@@ -944,6 +1026,15 @@ version = "0.9.5"
|
|
|
944
1026
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
945
1027
|
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
|
|
946
1028
|
|
|
1029
|
+
[[package]]
|
|
1030
|
+
name = "want"
|
|
1031
|
+
version = "0.3.1"
|
|
1032
|
+
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1033
|
+
checksum = "bfa7760aed19e106de2c7c0b581b509f2f25d3dacaf737cb82ac61bc6d760b0e"
|
|
1034
|
+
dependencies = [
|
|
1035
|
+
"try-lock",
|
|
1036
|
+
]
|
|
1037
|
+
|
|
947
1038
|
[[package]]
|
|
948
1039
|
name = "wasi"
|
|
949
1040
|
version = "0.11.1+wasi-snapshot-preview1"
|
data/README.md
CHANGED
|
@@ -11,11 +11,14 @@ on every core in **one small process**. A **Rust** (tokio + hyper)
|
|
|
11
11
|
front-end owns the network, parallel **Ractors** run your Rack 3 app,
|
|
12
12
|
and a threaded fallback mode runs everything else, Rails included.
|
|
13
13
|
|
|
14
|
-
* **Fast.** On a real 8-core server, every Kino mode is **1.5-
|
|
15
|
-
ahead of a Puma fork cluster on I/O-light endpoints
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
14
|
+
* **Fast.** On a real 8-core server, every Kino mode is **1.5-1.7×**
|
|
15
|
+
ahead of a Puma fork cluster on I/O-light endpoints—with native
|
|
16
|
+
**HTTP/2** adding another **+79%** over HTTP/1.1 on the same server.
|
|
17
|
+
Ractor mode also wins on pure CPU, **+25%**.
|
|
18
|
+
[Benchmarks](#benchmarks) below.
|
|
19
|
+
* **A fraction of the memory.** About **~8×** on the simplistic bench
|
|
20
|
+
Ractor app, and about **4× less memory** than a Puma cluster serving
|
|
21
|
+
Rails in fallback threaded mode.
|
|
19
22
|
* **Parallel without forking.** Ractor mode runs CPU work **more than
|
|
20
23
|
5× faster** than Kino's own GVL-bound threaded mode, in the same
|
|
21
24
|
small process.
|
|
@@ -73,32 +76,32 @@ rather than an app. It is Ractor-shareable, so Kino runs it in `:ractor`
|
|
|
73
76
|
mode (and `:threaded` for comparison). **A real Rails app is a different
|
|
74
77
|
story:** it is *not* Ractor-shareable, so it runs only in Kino's
|
|
75
78
|
`:threaded` fallback, with its own numbers—see [Rails](#rails) below.
|
|
76
|
-
Ruby 4.0.
|
|
77
|
-
3 threads, Kino stays in one process
|
|
78
|
-
modes, 3 in threaded). Numbers are
|
|
79
|
-
connections, same host).
|
|
80
|
-
[doc/benchmarks.md](doc/benchmarks.md).
|
|
79
|
+
Ruby 4.0.6 with YJIT (re-measured 2026-09), every server at its
|
|
80
|
+
defaults: Puma forks 8 workers × 3 threads, Kino stays in one process
|
|
81
|
+
(8 workers; 1 thread each in ractor modes, 3 in threaded). Numbers are
|
|
82
|
+
req/s by wrk (8-second windows, 64 connections, same host).
|
|
83
|
+
Methodology: [doc/benchmarks.md](doc/benchmarks.md).
|
|
81
84
|
|
|
82
85
|
| endpoint | Kino :ractor | + lanes | :ractor, `workers 32`² | Kino :threaded | Puma (cluster) |
|
|
83
86
|
|-------------|-------------:|--------:|-----------------------:|---------------:|---------------:|
|
|
84
|
-
| /plaintext |
|
|
85
|
-
| /10k |
|
|
86
|
-
| /cpu (fib) | **
|
|
87
|
-
| /io (5 ms) | 1,
|
|
88
|
-
| /io_native | 1,570 | 1,
|
|
87
|
+
| /plaintext | 222,980 | **244,652** | 115,553 | 213,104 | 142,094 |
|
|
88
|
+
| /10k | 175,182 | **188,365** | 103,147 | 132,009 | 125,116 |
|
|
89
|
+
| /cpu (fib) | **76,695**¹| 71,830 | 54,062 | 13,463 | 61,329 |
|
|
90
|
+
| /io (5 ms) | 1,548 | 1,556 | **5,388** | 4,722 | 4,699 |
|
|
91
|
+
| /io_native | 1,570 | 1,573 | **6,207** | 4,695 | 4,693 |
|
|
89
92
|
|
|
90
93
|
Memory tells two different stories depending on the app, both by **PSS**
|
|
91
94
|
(proportional set size; see note) after sustained load.
|
|
92
95
|
|
|
93
96
|
**The tiny benchmark app** (Ractor-shareable, so Kino runs it in `:ractor`
|
|
94
|
-
or `:threaded`). Kino is **~
|
|
95
|
-
than the Puma cluster
|
|
97
|
+
or `:threaded`). Kino is **~8× lighter in :ractor mode, ~10× in :threaded**
|
|
98
|
+
than the Puma cluster—the gap stays large because a trivial app is almost
|
|
96
99
|
all private per-worker heap, which copy-on-write can't share:
|
|
97
100
|
|
|
98
101
|
| tiny app, Kino | Kino (one process) | Puma cluster (8 workers) | ratio |
|
|
99
102
|
|-----------------|-------------------:|-------------------------:|------:|
|
|
100
|
-
| :ractor (8×1) | **
|
|
101
|
-
| :threaded (8×3) | **107 MB**³| 1,
|
|
103
|
+
| :ractor (8×1) | **135 MB** | 1,072 MB | ~8× |
|
|
104
|
+
| :threaded (8×3) | **107 MB**³| 1,072 MB | ~10× |
|
|
102
105
|
|
|
103
106
|
**A real Rails app** (not Ractor-shareable—Kino's `:threaded` fallback
|
|
104
107
|
only, [below](#rails)). The gap is **~4×**, smaller because Rails' large
|
|
@@ -106,24 +109,25 @@ framework *is* shared copy-on-write across Puma's forks:
|
|
|
106
109
|
|
|
107
110
|
| Rails hello-world | Kino :threaded | Puma cluster (8 workers) | ratio |
|
|
108
111
|
|-------------------|---------------:|-------------------------:|------:|
|
|
109
|
-
| **PSS** | **
|
|
112
|
+
| **PSS** | **95 MB** | **405 MB** | ~4× |
|
|
110
113
|
|
|
111
114
|
"+ lanes" is the experimental per-worker-queue dispatcher (`lanes true`).
|
|
112
115
|
It posts the fastest plaintext/10k of any configuration here. Details:
|
|
113
116
|
[doc/benchmarks.md](doc/benchmarks.md#lane-dispatch-experimental-lanes-true).
|
|
114
117
|
|
|
115
118
|
¹ Stock settings, no tuning. Ractor mode beats the fork cluster on pure
|
|
116
|
-
CPU by +
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
`
|
|
119
|
+
CPU by +25% (+17% with lanes; Puma 8.0.2 narrowed this from the +34%
|
|
120
|
+
measured against 7.x). Threaded mode shows the GVL ceiling that every
|
|
121
|
+
single-process Ruby server hits. The old CPU-tuning recipe stays
|
|
122
|
+
retired: its `threads 1` half **is** the default, and its
|
|
123
|
+
`tokio_threads 1` half still costs on real hardware; see
|
|
120
124
|
[doc/benchmarks.md](doc/benchmarks.md#cpu-bound-tuning).
|
|
121
125
|
|
|
122
126
|
² Wait-bound throughput is slots ÷ wait, and the default columns bring
|
|
123
127
|
8 single-thread workers against the cluster's 24 threads. Kino slots
|
|
124
128
|
are threads, not processes—when your app waits a lot, raise `workers`.
|
|
125
|
-
The `workers 32` column is that tuning: **+
|
|
126
|
-
(+
|
|
129
|
+
The `workers 32` column is that tuning: **+15% over the cluster on /io
|
|
130
|
+
(+32% via `Kino.sleep`)** while still ahead of it on pure CPU, all in
|
|
127
131
|
one small process. The cost is the CPU-light rows (32 ractors
|
|
128
132
|
oversubscribe 8 cores); pick the topology your app's wait profile
|
|
129
133
|
needs. See
|
|
@@ -131,7 +135,7 @@ needs. See
|
|
|
131
135
|
|
|
132
136
|
³ With `MALLOC_ARENA_MAX=2` (the standard Ruby deployment setting;
|
|
133
137
|
Heroku's default). Without it, 24 threads churning 10 KB responses
|
|
134
|
-
through one glibc heap balloon to ~
|
|
138
|
+
through one glibc heap balloon to ~647 MB—an arena-fragmentation
|
|
135
139
|
footgun, not a leak, and ractor mode sidesteps it. See
|
|
136
140
|
[doc/benchmarks.md](doc/benchmarks.md#memory-under-load-and-the-glibc-arena-footgun).
|
|
137
141
|
|
|
@@ -141,36 +145,64 @@ doc):
|
|
|
141
145
|
|
|
142
146
|
| endpoint | Kino :ractor (8×3) | Puma + ractor wrapper | Falcon + ractor wrapper |
|
|
143
147
|
|------------|-------------------:|----------------------:|------------------------:|
|
|
144
|
-
| /plaintext | **
|
|
145
|
-
| /cpu (fib) | **68,
|
|
146
|
-
| /io (5 ms) | **4,
|
|
148
|
+
| /plaintext | **190,206** | 22,055 | 107,203 |
|
|
149
|
+
| /cpu (fib) | **68,230** | 16,528 | 50,101 |
|
|
150
|
+
| /io (5 ms) | **4,477** | 1,482 | 1,545 |
|
|
151
|
+
|
|
152
|
+
(The Kino column here runs 8×3—the same 24 slots the wrappers get—so
|
|
153
|
+
the /io row is comparable; the main table's ractor column runs the 8×1
|
|
154
|
+
default.)
|
|
155
|
+
|
|
156
|
+
### HTTP/2 head-to-head
|
|
157
|
+
|
|
158
|
+
Kino speaks HTTP/2 natively, so it skips the usual nginx-termination
|
|
159
|
+
hop. Same box as the tables above; all over TLS, h2load, 64 in-flight,
|
|
160
|
+
same app:
|
|
161
|
+
|
|
162
|
+
| /plaintext | req/s | /upload (64 KB) |
|
|
163
|
+
|-------------------------------|------------:|----------------:|
|
|
164
|
+
| Kino native h2 | **164,044** | **19,226** |
|
|
165
|
+
| Kino HTTP/1.1 (same boot) | 80,789 | 17,210 |
|
|
166
|
+
| nginx h2 → Kino HTTP/1.1 | 109,219 | 1,217¹ |
|
|
167
|
+
| Falcon (native h2) | 55,637 | 18,838 |
|
|
168
|
+
|
|
169
|
+
Native h2 **doubles** HTTP/1.1 throughput on the same TLS server
|
|
170
|
+
(+79% on cleartext h2c; fewer, larger socket operations, and HPACK
|
|
171
|
+
spares re-sending cookies) and is **+50% over fronting the same Kino
|
|
172
|
+
with nginx**—the proxy hop is pure cost. Uploads, h2's classic weak
|
|
173
|
+
spot, run at HTTP/1 parity. Full matrix, cleartext h2c lanes, and
|
|
174
|
+
methodology: [doc/benchmarks.md](doc/benchmarks.md#http2); reproduce
|
|
175
|
+
with `bench/h2.sh`.
|
|
176
|
+
|
|
177
|
+
¹ nginx default-config h2 request-body flow control; see the doc.
|
|
147
178
|
|
|
148
179
|
### Rails
|
|
149
180
|
|
|
150
181
|
Rails is not Ractor-shareable today, so Kino serves it in `:threaded`
|
|
151
|
-
fallback
|
|
182
|
+
fallback—one GVL-bound process. On the same box (`examples/rails-hello`,
|
|
152
183
|
edge Rails, production, 8×5):
|
|
153
184
|
|
|
154
185
|
| Rails hello-world | req/s | memory (PSS) |
|
|
155
186
|
|------------------------------|-------:|-------------:|
|
|
156
|
-
| Kino :threaded (one process) | 2,
|
|
157
|
-
| Puma cluster (8 workers) |
|
|
187
|
+
| Kino :threaded (one process) | 2,731 | **95 MB** |
|
|
188
|
+
| Puma cluster (8 workers) | 15,422 | 405 MB |
|
|
158
189
|
|
|
159
190
|
The honest trade-off: Puma's fork cluster uses all 8 cores, so it serves
|
|
160
|
-
~
|
|
191
|
+
~5.6× the throughput—at ~4× the memory. Ractor-mode Rails would close
|
|
161
192
|
the throughput gap at one-process memory cost; the upstream blockers are
|
|
162
193
|
tracked in [doc/rails-on-ractors.md](doc/rails-on-ractors.md).
|
|
163
194
|
|
|
164
|
-
In short: on the tiny synthetic app, ractor mode beats fork-level CPU parallelism (**5.
|
|
165
|
-
own GVL-bound threaded mode, +
|
|
166
|
-
about 1/
|
|
167
|
-
Every Kino mode is 1.5-
|
|
195
|
+
In short: on the tiny synthetic app, ractor mode beats fork-level CPU parallelism (**5.7×** Kino's
|
|
196
|
+
own GVL-bound threaded mode, +25% over the cluster) in one process, at
|
|
197
|
+
about 1/8th of the cluster's memory by PSS (~4× on a real Rails app).
|
|
198
|
+
Every Kino mode is 1.5-1.7× ahead of the cluster on I/O-light endpoints,
|
|
199
|
+
and native HTTP/2 adds +79% over HTTP/1.1 on top. The macOS numbers
|
|
168
200
|
(secondary; everything there hits the loopback ceiling) and the
|
|
169
201
|
YJIT × Ractors gotcha are in [doc/benchmarks.md](doc/benchmarks.md).
|
|
170
202
|
|
|
171
203
|
Reproduce: `bench/run.sh [seconds] [concurrency]` for the main table,
|
|
172
204
|
`bench/studies.sh` for the follow-ups (CPU recipe, topology, scaling,
|
|
173
|
-
logging, memory).
|
|
205
|
+
sharded I/O, logging, memory), `bench/h2.sh` for the HTTP/2 matrix.
|
|
174
206
|
|
|
175
207
|
## Install
|
|
176
208
|
|
|
@@ -227,6 +259,8 @@ server = Kino::Server.new(app,
|
|
|
227
259
|
bind: "127.0.0.1", # or "unix:///run/kino.sock" behind a proxy
|
|
228
260
|
port: 9292, # 0 = ephemeral; read back via server.port
|
|
229
261
|
workers: Kino.available_parallelism, # ractors (parallelism); the default
|
|
262
|
+
max_workers: nil, # experimental: grow past workers under load (see Elastic pool)
|
|
263
|
+
scale_down_after: 30, # seconds idle before an extra worker retires
|
|
230
264
|
threads: 1, # per worker; ractor default 1, threaded default 3
|
|
231
265
|
mode: :auto, # :auto | :ractor | :threaded
|
|
232
266
|
queue_depth: 1024, # bounded queue; overflow → 503
|
|
@@ -239,6 +273,7 @@ server = Kino::Server.new(app,
|
|
|
239
273
|
control_bind: "127.0.0.1:9293", # monitoring: /stats /metrics /ready /live; port 0 reads back via server.control_port
|
|
240
274
|
control_token: ENV["KINO_CONTROL_TOKEN"], # optional Bearer auth for /stats + /metrics
|
|
241
275
|
tls: { cert: "cert.pem", key: "key.pem" }, # file paths or inline PEM
|
|
276
|
+
http2: true, # ALPN h2 on TLS + plaintext h2c; false = HTTP/1 only
|
|
242
277
|
)
|
|
243
278
|
server.start
|
|
244
279
|
server.shutdown # graceful: drain → deadline → abort stragglers
|
|
@@ -265,15 +300,39 @@ server.shutdown # graceful: drain → deadline → abort straggler
|
|
|
265
300
|
multi-thread runtime onto current-thread shards: one thread accepts and
|
|
266
301
|
hands each connection to the least-loaded shard, which then owns it for
|
|
267
302
|
its lifetime—no work-stealing, no cross-thread wakeups on the hot path.
|
|
268
|
-
|
|
269
|
-
|
|
303
|
+
On the 8-core reference box fast handlers gain +1-3% (best with
|
|
304
|
+
`io_threads 8`); the win grows with scheduler contention, so measure on
|
|
305
|
+
your own core count. Ruby-bound endpoints are unchanged. Orthogonal to
|
|
306
|
+
`mode`: it reshapes the Rust side only.
|
|
270
307
|
|
|
271
308
|
```ruby
|
|
272
309
|
# kino.rb
|
|
273
310
|
io_shards true
|
|
274
|
-
io_threads
|
|
311
|
+
io_threads 8 # optional; default: half the available CPUs
|
|
275
312
|
```
|
|
276
313
|
|
|
314
|
+
### HTTP/2
|
|
315
|
+
|
|
316
|
+
On by default, on both transports, with nothing to configure:
|
|
317
|
+
|
|
318
|
+
- **TLS binds** advertise `h2` via ALPN, so browsers and h2-capable
|
|
319
|
+
clients negotiate HTTP/2 and everything else stays on HTTP/1.1.
|
|
320
|
+
- **Plaintext binds** serve prior-knowledge h2c: a client that opens
|
|
321
|
+
with the HTTP/2 preface (an h2-preferring load balancer, a gRPC-style
|
|
322
|
+
backend hop, `curl --http2-prior-knowledge`) gets HTTP/2; ordinary
|
|
323
|
+
clients are HTTP/1.1 exactly as before. Browsers never do h2 on
|
|
324
|
+
plaintext, so a certificate-less kino behaves identically for them.
|
|
325
|
+
|
|
326
|
+
The Rack side is spec-complete on h2: `SERVER_PROTOCOL` is `"HTTP/2"`,
|
|
327
|
+
`HTTP_HOST`/`SERVER_NAME`/`SERVER_PORT` come from the `:authority`
|
|
328
|
+
pseudo-header (h2 requests carry no Host header), split cookie headers
|
|
329
|
+
are rejoined with `"; "`, and streamed uploads flow through the same
|
|
330
|
+
backpressured body channel as HTTP/1. Streams multiplex into the same
|
|
331
|
+
worker slots as keep-alive requests—`workers × threads` bounds
|
|
332
|
+
concurrency either way. (`rack.hijack` stays out on h2 as it is on h1;
|
|
333
|
+
the protocol has no 101 upgrade to hijack anyway.) `http2 false` pins
|
|
334
|
+
the server to HTTP/1 and drops `h2` from ALPN.
|
|
335
|
+
|
|
277
336
|
## Config file and CLI
|
|
278
337
|
|
|
279
338
|
Settings can live in a Puma-style Ruby DSL file: `kino.rb` in the
|
|
@@ -361,6 +420,56 @@ Kino fires four lifecycle hooks alongside `on_error`, split by firing context.
|
|
|
361
420
|
|
|
362
421
|
A raising hook is logged and never kills a worker.
|
|
363
422
|
|
|
423
|
+
## Elastic pool (experimental)
|
|
424
|
+
|
|
425
|
+
Size the pool for the quiet hours and let it grow for the busy ones:
|
|
426
|
+
|
|
427
|
+
```ruby
|
|
428
|
+
# kino.rb
|
|
429
|
+
workers 4 # always running
|
|
430
|
+
max_workers 16 # reached only under load
|
|
431
|
+
scale_down_after 30 # seconds idle before an extra worker retires
|
|
432
|
+
```
|
|
433
|
+
|
|
434
|
+
Or `Kino::Server.new(app, workers: 4, max_workers: 16)`. Leave
|
|
435
|
+
`max_workers` unset and the pool is fixed at `workers`, as before.
|
|
436
|
+
|
|
437
|
+
While requests wait in the queue, Kino adds a worker every 100 ms until
|
|
438
|
+
the queue clears or the pool hits `max_workers`. When the load passes,
|
|
439
|
+
workers above `workers` retire one at a time after `scale_down_after`
|
|
440
|
+
seconds idle, each finishing its current request first. Same behavior
|
|
441
|
+
in `:ractor` and `:threaded` mode.
|
|
442
|
+
|
|
443
|
+
**Use it when your app waits**: on databases, upstream services, slow
|
|
444
|
+
clients. With `workers` at your core count, all workers can be blocked
|
|
445
|
+
on I/O while cores sit idle; a higher ceiling puts those cores to work,
|
|
446
|
+
and a ractor starts in microseconds, so the pool follows load closely.
|
|
447
|
+
Pure CPU work gains nothing past the core count. In `:ractor` mode Ruby
|
|
448
|
+
itself runs at most `RUBY_MAX_CPU` ractors' Ruby code at once (default
|
|
449
|
+
8), and Kino warns at boot when the pool can exceed it. On a bigger box:
|
|
450
|
+
|
|
451
|
+
```sh
|
|
452
|
+
RUBY_MAX_CPU=16 kino
|
|
453
|
+
```
|
|
454
|
+
|
|
455
|
+
**Watch it breathe** in `server.stats`, `GET /stats` and `GET /metrics`:
|
|
456
|
+
|
|
457
|
+
```sh
|
|
458
|
+
$ curl -s localhost:9293/stats | jq '{workers, max_workers, active_workers, scale_ups, scale_downs}'
|
|
459
|
+
{
|
|
460
|
+
"workers": 4,
|
|
461
|
+
"max_workers": 16,
|
|
462
|
+
"active_workers": 9,
|
|
463
|
+
"scale_ups": 12,
|
|
464
|
+
"scale_downs": 7
|
|
465
|
+
}
|
|
466
|
+
```
|
|
467
|
+
|
|
468
|
+
Prometheus gets `kino_max_workers`, `kino_active_workers`,
|
|
469
|
+
`kino_scale_ups_total` and `kino_scale_downs_total`. `after_worker_boot`
|
|
470
|
+
fires for every worker the pool adds, `on_worker_exit` (with a nil
|
|
471
|
+
cause) for every one it retires.
|
|
472
|
+
|
|
364
473
|
## Stuck-worker quarantine
|
|
365
474
|
|
|
366
475
|
`quarantine_timeout: seconds` (or `quarantine_timeout 60` in `kino.rb`)
|
|
@@ -439,8 +548,8 @@ not `queue_time`.
|
|
|
439
548
|
|
|
440
549
|
## Logging
|
|
441
550
|
|
|
442
|
-
With one log line per request, `Kino::Logger` sustained **
|
|
443
|
-
throughput of a shared `::Logger`** (
|
|
551
|
+
With one log line per request, `Kino::Logger` sustained **1.7× the
|
|
552
|
+
throughput of a shared `::Logger`** (155k vs 90k req/s on the benchmark
|
|
444
553
|
box). There are two native pieces. Both write through a lock-free
|
|
445
554
|
channel to a Rust flusher thread, so request threads never take a log
|
|
446
555
|
mutex and never make a write syscall:
|
data/doc/architecture.md
CHANGED
|
@@ -31,6 +31,16 @@ Puma-style two-level: `workers × threads`.
|
|
|
31
31
|
- Identical machinery either way: the flume queue is MPMC, a "worker slot"
|
|
32
32
|
is per-thread, and the worker loop (`lib/kino/worker.rb`) is shared
|
|
33
33
|
verbatim.
|
|
34
|
+
- Elastic pool (`max_workers`): a scaler thread on the main ractor
|
|
35
|
+
samples queue depth and the per-slot sensors every 100 ms, adds one
|
|
36
|
+
worker per sample while requests wait, and retires the longest-idle
|
|
37
|
+
worker above the floor after `scale_down_after`. Retirement is a
|
|
38
|
+
per-slot flag raised under the slot's lane lock: the lane dispatcher
|
|
39
|
+
skips the slot, the take loop honors the flag at its next idle tick (a
|
|
40
|
+
request already taken finishes first, a lane worker drains its own
|
|
41
|
+
lane), and the slot is reset and reused by the next worker, so the
|
|
42
|
+
slot table never grows with churn. Both pools (the ractor supervisor
|
|
43
|
+
and the threaded pool) expose the same grow/retire/groups seam.
|
|
34
44
|
- Experimental `lanes true` replaces the one shared queue with a small
|
|
35
45
|
private queue per worker slot (awake-preferring dispatch, work
|
|
36
46
|
stealing); see [benchmarks](benchmarks.md#lane-dispatch-experimental-lanes-true).
|
|
@@ -109,13 +119,63 @@ plain decrement after an `.await` would never run).
|
|
|
109
119
|
|
|
110
120
|
## Graceful shutdown
|
|
111
121
|
|
|
112
|
-
`stop_accepting` →
|
|
113
|
-
|
|
122
|
+
`stop_accepting` → every live connection switches to graceful shutdown
|
|
123
|
+
(finish the in-flight request, answer with `Connection: close` on
|
|
124
|
+
HTTP/1 or GOAWAY on h2, take nothing new—so drains converge instead
|
|
125
|
+
of chasing chatty keep-alive clients) → drain until queue + in-flight
|
|
126
|
+
reach zero or the deadline passes → `close_queue` (idle workers see
|
|
127
|
+
Disconnected and exit) →
|
|
114
128
|
join workers → past deadline: abort remaining clients (a 500, or a
|
|
115
129
|
connection abort mid-stream), interrupt blocked workers, reap
|
|
116
130
|
stragglers → tear down the tokio runtime. Idempotent;
|
|
117
131
|
a second INT/TERM force-exits.
|
|
118
132
|
|
|
133
|
+
## HTTP/2
|
|
134
|
+
|
|
135
|
+
One connection builder serves every protocol: hyper-util's auto builder
|
|
136
|
+
picks h2 by ALPN on TLS connections, by the 24-byte preface sniff on
|
|
137
|
+
plaintext (prior-knowledge h2c), and HTTP/1.x otherwise; `http2 false`
|
|
138
|
+
pins the HTTP/1 codec and skips the sniff entirely. Streams multiplex
|
|
139
|
+
into the same bounded queue as keep-alive requests, so h2 concurrency
|
|
140
|
+
is bounded by workers × threads exactly as h1's is, and the bounded
|
|
141
|
+
body channels give per-stream backpressure for free: while the
|
|
142
|
+
forwarder blocks, hyper withholds WINDOW_UPDATE and the client stalls.
|
|
143
|
+
|
|
144
|
+
The env bridge fills SERVER_NAME/SERVER_PORT/HTTP_HOST from the request
|
|
145
|
+
URI's `:authority` (h2 requests carry no Host header), through the same
|
|
146
|
+
host LRU the Host-header path uses, keyed by the authority bytes;
|
|
147
|
+
SERVER_PROTOCOL is the interned "HTTP/2". h2 trailer frames are dropped
|
|
148
|
+
(Rack has no trailer surface), and the h2 codec itself rejects
|
|
149
|
+
connection-ish headers before they can reach the env.
|
|
150
|
+
|
|
151
|
+
The upload path needed one h2-shaped fix: `read_body` drains every
|
|
152
|
+
already-queued chunk in a single native call (one GVL round-trip and
|
|
153
|
+
one Ruby string per 64 KB read), because a body arriving as 16 KB DATA
|
|
154
|
+
frames otherwise paid one crossing per frame—that alone took h2
|
|
155
|
+
uploads from half of h1's throughput to parity. A knob sweep over
|
|
156
|
+
hyper's h2 codec (frame size, adaptive windows, window sizes) moved
|
|
157
|
+
nothing after that, so hyper's defaults stay.
|
|
158
|
+
|
|
159
|
+
SETTINGS_MAX_CONCURRENT_STREAMS is derived from slot capacity
|
|
160
|
+
(workers × threads, clamped to [8, 1024]) rather than hyper's flat 200:
|
|
161
|
+
a smart balancer sees the server's true admission, and a hostile client
|
|
162
|
+
cannot multiply one connection into hundreds of queued requests—the
|
|
163
|
+
h2 analogue of h1's one-request-per-connection shape. The codec's own
|
|
164
|
+
abuse bounds ship as hyper/h2 defaults and were reviewed: 16 KB header
|
|
165
|
+
lists, 20 pending remote resets then GOAWAY (rapid reset), per-second
|
|
166
|
+
reset-churn and empty-frame budgets.
|
|
167
|
+
|
|
168
|
+
Low-cardinality header values (UA, accept-*, sec-ch-*, sec-fetch-*)
|
|
169
|
+
are interned in an LRU of frozen strings—the env-side analogue of
|
|
170
|
+
HPACK's wire dedup, and the practical form of it: hyper does not expose
|
|
171
|
+
HPACK table indices, so the cache keys on value bytes and works for
|
|
172
|
+
HTTP/1 too. Cookie and authorization are deliberately excluded
|
|
173
|
+
(per-user cardinality, secret lifetime).
|
|
174
|
+
|
|
175
|
+
Deferred until benchmarks justify it: slot-aware flow-control window
|
|
176
|
+
grants (a memory/abuse lever, not a throughput one—the knob sweep
|
|
177
|
+
showed windows don't gate upload throughput).
|
|
178
|
+
|
|
119
179
|
## Timer waits: `Kino.sleep`
|
|
120
180
|
|
|
121
181
|
MRI's `sleep` parks the thread on the VM timer, whose wakeups inside
|
|
@@ -128,7 +188,8 @@ at the interrupt tick so `Thread#kill` and shutdown stay responsive.
|
|
|
128
188
|
|
|
129
189
|
- **tokio + hyper**: the bottleneck is the Ruby dispatch boundary, not raw
|
|
130
190
|
I/O throughput; what matters is HTTP correctness, keep-alive, TLS, and
|
|
131
|
-
h2-
|
|
191
|
+
h2 (since shipped, via hyper-util's protocol-auto builder)—hyper's
|
|
192
|
+
territory. Cross-platform out of the box.
|
|
132
193
|
- **monoio**: thread-per-core io_uring looks great in echo-server
|
|
133
194
|
benchmarks, but hyper only works through its poll-io compat layer
|
|
134
195
|
(forfeiting io_uring on the hot path), and the share-nothing advantage
|