raptor 0.22.3 → 0.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +30 -0
  3. data/Dockerfile +3 -1
  4. data/README.md +38 -31
  5. data/docs/raptor-vs-puma.md +44 -38
  6. data/docs/talk.md +1295 -0
  7. data/ext/raptor_bpf/reuseport_select.bpf.c +18 -9
  8. data/ext/raptor_http/extconf.rb +2 -0
  9. data/ext/raptor_http/raptor_http.c +91 -119
  10. data/ext/raptor_http2/raptor_http2.c +272 -223
  11. data/ext/raptor_native/raptor_native.c +38 -20
  12. data/lib/rackup/handler/raptor.rb +12 -11
  13. data/lib/raptor/binder.rb +12 -12
  14. data/lib/raptor/cli.rb +18 -18
  15. data/lib/raptor/cluster.rb +44 -44
  16. data/lib/raptor/control_server.rb +3 -2
  17. data/lib/raptor/detached_body.rb +6 -2
  18. data/lib/raptor/http.rb +7 -5
  19. data/lib/raptor/http1.rb +232 -207
  20. data/lib/raptor/http2.rb +312 -265
  21. data/lib/raptor/reactor.rb +138 -148
  22. data/lib/raptor/reuseport_bpf.rb +62 -38
  23. data/lib/raptor/server.rb +45 -34
  24. data/lib/raptor/stats.rb +15 -3
  25. data/lib/raptor/thread_locals.rb +4 -3
  26. data/lib/raptor/version.rb +1 -1
  27. data/sig/generated/rackup/handler/raptor.rbs +2 -1
  28. data/sig/generated/raptor/binder.rbs +9 -9
  29. data/sig/generated/raptor/cli.rbs +6 -6
  30. data/sig/generated/raptor/cluster.rbs +9 -9
  31. data/sig/generated/raptor/control_server.rbs +2 -1
  32. data/sig/generated/raptor/detached_body.rbs +5 -1
  33. data/sig/generated/raptor/http.rbs +3 -3
  34. data/sig/generated/raptor/http1.rbs +100 -91
  35. data/sig/generated/raptor/http2.rbs +200 -189
  36. data/sig/generated/raptor/reactor.rbs +107 -118
  37. data/sig/generated/raptor/reuseport_bpf.rbs +54 -17
  38. data/sig/generated/raptor/server.rbs +26 -26
  39. data/sig/generated/raptor/stats.rbs +3 -2
  40. data/sig/generated/raptor/thread_locals.rbs +4 -3
  41. metadata +2 -2
  42. data/docs/brisrails-talk.md +0 -2398
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 813fde7063bd9f7dd9e965f57ede9e1e207f4f0db558343e777fce9769a7ec0e
4
- data.tar.gz: faf2cdec407c5c8d36f4da222a50df4b47366d2a76204e22356a91bb24736879
3
+ metadata.gz: a1a18ea2df37b0221bea185c5c5a275a08eda61f059056b5f7599bf71be9661f
4
+ data.tar.gz: 7586e065cf01603b2fc78f04c75768dfcea43314e77fb5b6485a4dfabe7d01f0
5
5
  SHA512:
6
- metadata.gz: be5d3eb531ab184623787d4b18b06a0d51f908c92120fd46031c353518563da756183775155e250c9744fbc34400c82945204ba341f126be8aa3eab5b94ac8c3
7
- data.tar.gz: 4b125079b570a4e0f15a549d75ef09bcce749f1442fd18a20a27401cc9c6618797e61564e1ebc194fc93c93bf07eee669249a1e1df18064b306b09a2334b7f41
6
+ metadata.gz: 706c2c28172d8ca0613ed8d2d27dea007205af4142f96b548147d6aefeb4e9a3ddece60b9035cca7a72e0149408dfdadb3893cdfbfb7696f1d4b69b831b05966
7
+ data.tar.gz: e04215e61ac7ef2e306211615b17bf64a7c8341b2c701fb4b41e88ef75262310b65aedc310dfe758a5ed0cb8b7a5b442b2ffa49e6d5a9ddd55844ce539b91be5
data/CHANGELOG.md CHANGED
@@ -1,5 +1,35 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [0.23.0] - 2026-10-18
4
+
5
+ - Route connections through BPF dispatch on every TCP bind
6
+ - Reject non-integer `RAPTOR_WORKERS` and `RAPTOR_THREADS` in the Rackup handler
7
+ - Replace config file binds with `--bind` addresses instead of adding to them
8
+ - Stop BPF dispatch from routing connections to workers that stopped accepting
9
+ - Restart workers forked by the seed when they exit
10
+ - Keep draining HTTP/2 connections when another attaches during shutdown
11
+ - Close HTTP/1.1 connections on the reactor thread when a detached body fails to open
12
+ - Apply HTTP/2 initial window changes to responses waiting for flow control
13
+ - Default `SERVER_NAME` for HTTP/2 requests without a host name
14
+ - Join repeated HTTP/2 request headers instead of keeping only the last
15
+ - Ignore HTTP/2 DATA frames for streams that already closed
16
+ - Credit each HTTP/2 stream window for the data that stream receives
17
+ - Strip padding from HTTP/2 DATA and HEADERS frames
18
+ - Encode and decode long HTTP/2 header strings without overflowing the stack
19
+ - Limit HTTP/2 request header lists to 112 KiB
20
+ - Reject HTTP/2 frames larger than the maximum frame size
21
+ - Send GOAWAY when an HTTP/2 frame fails to decode
22
+ - Keep pipelined HTTP/1.1 requests out of the previous request's body
23
+ - Process chunked requests on kept-alive HTTP/1.1 connections without waiting for more data
24
+ - Validate request framing on kept-alive HTTP/1.1 connections
25
+ - Reject `Content-Length` values that overflow 64 bits
26
+ - Resume idle HTTP/1.1 keep-alive connections on application threads
27
+ - Defer accepting TCP connections until the client sends data
28
+ - Accept connections the BPF program cannot route to a worker
29
+ - Count connections waiting in accept queues towards BPF worker load
30
+ - Scale accept backpressure with the thread pool and yield at capacity
31
+ - Raise `Http2ParserError` for malformed HPACK Huffman data
32
+
3
33
  ## [0.22.3] - 2026-10-04
4
34
 
5
35
  - Keep workers accepting connections when clients reset during accept
data/Dockerfile CHANGED
@@ -1,7 +1,7 @@
1
1
  FROM golang:bookworm AS hey-builder
2
2
  RUN CGO_ENABLED=0 go install github.com/rakyll/hey@latest
3
3
 
4
- FROM ruby:latest
4
+ FROM rubylang/ruby:master-dev
5
5
 
6
6
  ENV DEBIAN_FRONTEND=noninteractive
7
7
  ENV LANG=C.UTF-8
@@ -12,6 +12,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
12
12
  clang \
13
13
  libbpf-dev \
14
14
  libelf-dev \
15
+ libssl-dev \
16
+ libyaml-dev \
15
17
  linux-libc-dev \
16
18
  nghttp2 \
17
19
  openssl \
data/README.md CHANGED
@@ -37,7 +37,7 @@ run proc { |_env| [200, { "content-type" => "text/plain" }, ["Hello, World!"]] }
37
37
  ```
38
38
  > bundle exec raptor -w 10 -t 3 hello_world.ru
39
39
  [Raptor 72876|Main|Main] Cluster initializing:
40
- [Raptor 72876|Main|Main] ├─ Version: 0.22.3
40
+ [Raptor 72876|Main|Main] ├─ Version: 0.23.0
41
41
  [Raptor 72876|Main|Main] ├─ Ruby Version: ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +YJIT +PRISM [arm64-darwin23]
42
42
  [Raptor 72876|Main|Main] ├─ Environment: development
43
43
  [Raptor 72876|Main|Main] ├─ Master PID: 72876
@@ -208,41 +208,48 @@ Set `control_url` to a Unix socket URL such as `unix:///tmp/raptor-control.sock`
208
208
 
209
209
  ## (Micro) Benchmarks
210
210
 
211
- Raptor 0.22.3 vs Puma 8.0.2 vs Falcon 0.57.0 across two workload profiles. **IO-bound** is a GET endpoint that
212
- interleaves 5-10 short sleeps (total 2.5-15ms) with small CPU work, simulating a read path that makes several DB or
213
- cache calls. **CPU-bound** is a POST endpoint that accepts a small JSON body, interleaves 3-5 chunks of JSON item
214
- building (total 450-1500 items) with sub-100µs sleeps, and returns the built array, simulating a write path that does
215
- most of its work in Ruby with a few near-zero-cost cache hits.
216
-
217
- Raptor is run in two modes: **Fixed** keeps 3 application threads per worker, matching Puma, while **Scaling** starts
218
- with 3 and may add threads without a fixed limit when queued work is blocked outside the GVL. Both modes are compared
219
- with both Puma and Falcon in the table below. Raptor request-local cleanup and Puma's Fiber-per-request mode are
220
- disabled, and both threaded servers allow 999 requests per HTTP/1.1 keep-alive connection.
211
+ Raptor 0.23.0 vs Puma 8.0.2 vs Falcon 0.57.0 across two workload profiles. **IO-bound** is a GET endpoint that spends
212
+ about 95% of each request waiting. It makes 5-10 simulated DB calls of 0.5-1.5ms each and renders a small JSON result
213
+ after each one. **CPU-bound** is a POST endpoint that spends about 85% of each request running Ruby. It accepts a small
214
+ JSON body, renders 2,100-7,000 JSON items in 3-5 chunks, each after a 25-50µs simulated cache hit, and returns a short
215
+ response. **IO + SSE** repeats the IO-bound keep-alive run while 2 server-sent event streams per worker stay open,
216
+ each receiving an event every second. Raptor serves them with `Raptor::DetachedBody`, which hands the stream to the
217
+ reactor and frees the application thread, while Puma and Falcon stream from an Enumerator that sleeps between events.
218
+ Puma keeps accepting connections while every thread in a worker is busy, so a stream or request it accepts can wait
219
+ behind streams that never finish. Streams that got no response within 2 seconds and requests that hey gave up on
220
+ after 20 seconds are counted in parentheses.
221
+
222
+ Raptor and Puma are run in two modes: **Fixed** keeps 3 application threads per worker, while **Scaling** starts with 3
223
+ and may grow to 16. Puma adds a thread whenever work is waiting, while Raptor only adds one when queued work is blocked
224
+ outside the GVL. Falcon is compared against both modes. Raptor request-local cleanup and Puma's Fiber-per-request mode
225
+ are disabled, and both threaded servers allow 999 requests per HTTP/1.1 keep-alive connection.
221
226
 
222
227
  Each cell reports the median throughput and median p95 latency independently across 3 runs, so the two numbers in a row
223
228
  may come from different runs. Every run starts a fresh server process so the samples are independent of each other;
224
229
  state accumulated in a previous run cannot bias the next. Across the whole table, the widest spread
225
- ((max - min) / 2 / median) between runs of a single cell was ±19.6% for throughput and ±13.6% for p95.
226
-
227
- | Protocol | Workload | Raptor mode | Raptor req/s | Raptor p95 | Puma req/s | Puma p95 | vs Puma req/s | vs Puma p95 | Falcon req/s | Falcon p95 | vs Falcon req/s | vs Falcon p95 |
228
- | --------------------- | -------- | ----------- | ------------ | ---------- | ----------- | --------- | ------------- | ------------ | ------------ | ---------- | --------------- | ------------- |
229
- | HTTP/1.1 | IO | Fixed | 2.95k req/s | 80.20 ms | 1.55k req/s | 122.60 ms | 90.1% higher | 34.6% lower | 12.24k req/s | 14.00 ms | 75.9% lower | 472.9% higher |
230
- | HTTP/1.1 | IO | Scaling | 7.05k req/s | 30.40 ms | 1.55k req/s | 122.60 ms | 354.1% higher | 75.2% lower | 12.24k req/s | 14.00 ms | 42.4% lower | 117.1% higher |
231
- | HTTP/1.1 | CPU | Fixed | 7.29k req/s | 35.00 ms | 8.73k req/s | 20.30 ms | 16.4% lower | 72.4% higher | 6.71k req/s | 26.80 ms | 8.7% higher | 30.6% higher |
232
- | HTTP/1.1 | CPU | Scaling | 6.73k req/s | 36.90 ms | 8.73k req/s | 20.30 ms | 22.8% lower | 81.8% higher | 6.71k req/s | 26.80 ms | 0.4% higher | 37.7% higher |
233
- | HTTP/1.1 (keep-alive) | IO | Fixed | 2.52k req/s | 70.70 ms | 1.50k req/s | 102.60 ms | 67.8% higher | 31.1% lower | 6.23k req/s | 28.20 ms | 59.5% lower | 150.7% higher |
234
- | HTTP/1.1 (keep-alive) | IO | Scaling | 8.19k req/s | 21.90 ms | 1.50k req/s | 102.60 ms | 445.8% higher | 78.7% lower | 6.23k req/s | 28.20 ms | 31.6% higher | 22.3% lower |
235
- | HTTP/1.1 (keep-alive) | CPU | Fixed | 7.04k req/s | 28.30 ms | 8.62k req/s | 21.50 ms | 18.3% lower | 31.6% higher | 7.07k req/s | 32.10 ms | 0.4% lower | 11.8% lower |
236
- | HTTP/1.1 (keep-alive) | CPU | Scaling | 7.78k req/s | 25.70 ms | 8.62k req/s | 21.50 ms | 9.8% lower | 19.5% higher | 7.07k req/s | 32.10 ms | 9.9% higher | 19.9% lower |
237
- | HTTP/2 | IO | Fixed | 1.50k req/s | 128.92 ms | N/A | N/A | - | - | 6.57k req/s | 27.29 ms | 77.2% lower | 372.4% higher |
238
- | HTTP/2 | IO | Scaling | 9.41k req/s | 19.30 ms | N/A | N/A | - | - | 6.57k req/s | 27.29 ms | 43.1% higher | 29.3% lower |
239
- | HTTP/2 | CPU | Fixed | 7.75k req/s | 26.55 ms | N/A | N/A | - | - | 7.24k req/s | 49.42 ms | 7.1% higher | 46.3% lower |
240
- | HTTP/2 | CPU | Scaling | 7.54k req/s | 26.97 ms | N/A | N/A | - | - | 7.24k req/s | 49.42 ms | 4.2% higher | 45.4% lower |
241
-
242
- > ruby 4.0.7 (2026-09-15 revision 229531a6cf) +YJIT +PRISM [aarch64-linux]
243
- > 10 worker processes; fixed Raptor and Puma run 3 threads per worker; scaling Raptor starts at 3 with no fixed limit;
230
+ ((max - min) / 2 / median) between runs of a single cell was ±48.9% for throughput and ±38.0% for p95.
231
+
232
+ | Protocol | Workload | Mode | Raptor req/s | Raptor p95 | Puma req/s | Puma p95 | vs Puma req/s | vs Puma p95 | Falcon req/s | Falcon p95 | vs Falcon req/s | vs Falcon p95 |
233
+ | --------------------- | -------- | ------- | ------------ | ---------- | ------------------------- | --------- | ------------- | ------------ | ------------ | ---------- | --------------- | ------------- |
234
+ | HTTP/1.1 | IO | Fixed | 3.41k req/s | 45.60 ms | 1.48k req/s | 127.40 ms | 130.1% higher | 64.2% lower | 10.72k req/s | 16.40 ms | 68.2% lower | 178.0% higher |
235
+ | HTTP/1.1 | IO | Scaling | 8.28k req/s | 21.90 ms | 8.06k req/s | 22.70 ms | 2.7% higher | 3.5% lower | 10.72k req/s | 16.40 ms | 22.8% lower | 33.5% higher |
236
+ | HTTP/1.1 | CPU | Fixed | 3.54k req/s | 62.40 ms | 3.44k req/s | 52.20 ms | 2.9% higher | 19.5% higher | 3.57k req/s | 45.90 ms | 1.0% lower | 35.9% higher |
237
+ | HTTP/1.1 | CPU | Scaling | 3.40k req/s | 63.40 ms | 3.14k req/s | 51.80 ms | 8.0% higher | 22.4% higher | 3.57k req/s | 45.90 ms | 4.9% lower | 38.1% higher |
238
+ | HTTP/1.1 (keep-alive) | IO | Fixed | 3.38k req/s | 37.50 ms | 1.46k req/s | 102.20 ms | 131.7% higher | 63.3% lower | 7.23k req/s | 25.20 ms | 53.2% lower | 48.8% higher |
239
+ | HTTP/1.1 (keep-alive) | IO | Scaling | 7.38k req/s | 24.70 ms | 7.40k req/s | 24.70 ms | 0.3% lower | 0.0% higher | 7.23k req/s | 25.20 ms | 2.0% higher | 2.0% lower |
240
+ | HTTP/1.1 (keep-alive) | CPU | Fixed | 3.34k req/s | 49.70 ms | 3.45k req/s | 53.90 ms | 3.3% lower | 7.8% lower | 3.71k req/s | 57.80 ms | 9.9% lower | 14.0% lower |
241
+ | HTTP/1.1 (keep-alive) | CPU | Scaling | 3.30k req/s | 49.70 ms | 3.06k req/s | 62.90 ms | 8.0% higher | 21.0% lower | 3.71k req/s | 57.80 ms | 10.9% lower | 14.0% lower |
242
+ | HTTP/1.1 (keep-alive) | IO + SSE | Fixed | 3.33k req/s | 41.90 ms | 0.35k req/s (21 timeouts) | 323.40 ms | 852.5% higher | 87.0% lower | 7.26k req/s | 25.10 ms | 54.2% lower | 66.9% higher |
243
+ | HTTP/1.1 (keep-alive) | IO + SSE | Scaling | 7.33k req/s | 24.80 ms | 7.16k req/s | 25.50 ms | 2.4% higher | 2.7% lower | 7.26k req/s | 25.10 ms | 1.0% higher | 1.2% lower |
244
+ | HTTP/2 | IO | Fixed | 1.23k req/s | 138.77 ms | N/A | N/A | - | - | 6.54k req/s | 34.34 ms | 81.2% lower | 304.2% higher |
245
+ | HTTP/2 | IO | Scaling | 6.87k req/s | 31.13 ms | N/A | N/A | - | - | 6.54k req/s | 34.34 ms | 5.2% higher | 9.3% lower |
246
+ | HTTP/2 | CPU | Fixed | 3.37k req/s | 68.35 ms | N/A | N/A | - | - | 1.75k req/s | 158.85 ms | 92.7% higher | 57.0% lower |
247
+ | HTTP/2 | CPU | Scaling | 3.28k req/s | 62.50 ms | N/A | N/A | - | - | 1.75k req/s | 158.85 ms | 87.5% higher | 60.7% lower |
248
+
249
+ > ruby 4.1.0dev (2026-10-17T18:56:13Z master 5b1c373522) +YJIT +PRISM [aarch64-linux]
250
+ > 10 worker processes; fixed Raptor and Puma run 3 threads per worker; scaling Raptor and Puma run 3 to 16;
244
251
  > Falcon runs unbounded fibers per worker; 120 concurrent HTTP/1.1 client connections; 40 concurrent HTTP/2 client
245
- > connections × 3 streams each
252
+ > connections × 3 streams each; 20 open SSE streams in IO + SSE runs
246
253
 
247
254
  See [bin/benchmark](bin/benchmark) for more details.
248
255
 
@@ -1,6 +1,6 @@
1
1
  # Raptor vs Puma: A Design Comparison
2
2
 
3
- Raptor is a Ruby web server built around Ractor-parallel protocol pipelines, a lock-free app thread pool, and an opinionated cluster architecture. Against the latest Puma release on the same hardware, it leads clearly on the IO-heavy HTTP/1.1 benchmark and lands within a few percent on CPU-heavy throughput, while Falcon's fibers remain the strongest fit for highly concurrent IO. Raptor also speaks HTTP/2 natively, which Puma does not. This document explains how those designs move requests and where each trade-off shows up.
3
+ Raptor is a Ruby web server built around Ractor-parallel protocol pipelines, a lock-free app thread pool, and an opinionated cluster architecture. It also speaks HTTP/2 natively, which Puma does not. This document explains how the two servers move requests, how they compare on the same hardware, and where each trade-off shows up.
4
4
 
5
5
  ## Why this document exists
6
6
 
@@ -20,17 +20,18 @@ This document focuses on Puma because Puma is the incumbent that any new Ruby we
20
20
 
21
21
  Raptor is a research project. It hasn't run production traffic. The numbers in the README come from a repeatable microbenchmark, not from a real deployment. They use controlled Rack workloads to measure the full server path: accepting connections, parsing, dispatching, running the app, and writing responses. A real Rails application adds database time, downstream services, middleware, and application-specific contention, so a percentage here will not transfer directly to production.
22
22
 
23
- The Raptor README carries the [current head-to-head numbers](../README.md#micro-benchmarks) against the latest Puma and Falcon releases, run on the same hardware with the same Rack app on a recent Ruby with YJIT enabled. All servers run one worker process per available CPU. Puma and Raptor's fixed mode use three app threads per worker. Raptor's scaling mode starts at three and has no fixed limit, while Falcon uses unbounded fibers. Load generators use four client connections per baseline app thread, so on the 10-core machine that produced the current numbers that's 120 concurrent HTTP/1.1 client connections and 40 h2 connections × 3 streams each.
23
+ The Raptor README carries the [current head-to-head numbers](../README.md#micro-benchmarks) against the latest Puma and Falcon releases, run on the same hardware with the same Rack app on Ruby head with YJIT enabled. Running on head means a Ruby regression shows up before it ships. All servers run one worker process per available CPU. Raptor and Puma are each run in two modes: **fixed** keeps three app threads per worker, and **scaling** starts at three and may grow to sixteen. Falcon uses unbounded fibers. Load generators use four client connections per baseline app thread, so on the 10-core machine that produced the current numbers that's 120 concurrent HTTP/1.1 client connections and 40 h2 connections × 3 streams each.
24
24
 
25
- Two workload profiles are measured. **IO-bound** is a GET endpoint that does 5 to 10 short sleeps interleaved with small CPU work per request, simulating a read path that makes several DB or cache calls throughout its lifetime. **CPU-bound** is a POST endpoint with a small JSON body that builds a JSON response in 3 to 5 chunks interleaved with sub-100µs sleeps, simulating a write path that does most of its work in Ruby with a few near-zero-cost cache hits. The workloads are interleaved rather than a single bulk sleep or single bulk serialise so a fiber-per-connection server like Falcon doesn't look artificially good from one-shot IO, and the CPU-bound workload is heavily CPU-dominated by design (roughly 95% CPU / 5% IO by wall time) so it actually measures CPU work rather than smuggling in enough IO for fibers to multiplex.
25
+ Two workload profiles are measured. **IO-bound** is a GET endpoint that spends about 95% of each request waiting: it makes 5 to 10 simulated DB calls of 0.5 to 1.5ms each and renders a small JSON result after each one. **CPU-bound** is a POST endpoint that spends about 85% of each request running Ruby: it accepts a small JSON body, renders 2,100 to 7,000 JSON items in 3 to 5 chunks, each after a 25 to 50µs simulated cache hit, and returns a short response. The work is interleaved rather than one bulk sleep or one bulk render, so a fiber-per-connection server like Falcon doesn't look artificially good from one-shot IO, and the CPU-bound profile measures Ruby execution rather than smuggling in enough IO for fibers to multiplex.
26
26
 
27
- Each cell in the table reports the median throughput and median p95 latency independently across 3 runs, and every run boots a fresh server process so state cannot accumulate across measurements. Both Raptor modes are compared with both servers: fixed mode is the like-for-like comparison with Puma, while scaling mode tests whether adaptive OS threads can close the IO-concurrency gap with Falcon's much cheaper fibers. Rather than pin every number into this document (they drift with Ruby versions and hardware), the shape of the result is what matters.
27
+ Each cell in the table reports the median throughput and median p95 latency independently across 3 runs, and every run boots a fresh server process so state cannot accumulate across measurements. The benchmark does not run external CPU spinners. Fixed mode compares the servers' request paths with the same number of threads. Scaling mode compares how each pool decides when to add threads: Puma adds one whenever work is waiting, while Raptor only adds one when queued work is blocked outside the GVL. Rather than pin every number into this document (they drift with Ruby versions and hardware), the shape of the result is what matters.
28
28
 
29
- - On IO-bound HTTP/1.1, fixed Raptor delivers a little over twice Puma's throughput with less than half its p95 latency. Scaling lifts Raptor to 8.63k req/s without keep-alive and 9.43k with it: still 29.3% behind Falcon on fresh connections, but 51.3% ahead with keep-alive.
30
- - On CPU-bound HTTP/1.1, fixed and scaling Raptor are effectively identical. Puma leads by 5.0% without keep-alive and 0.7% with it, while both Raptor modes lead Falcon. This is the intended result: the pool gets its IO gains without trading away CPU throughput.
31
- - On HTTP/2, Raptor and Falcon both implement it; Puma doesn't. Scaling raises Raptor's IO throughput from 1.14k to 4.46k req/s, though it remains 30.3% behind Falcon. On the CPU profile it narrows Falcon's throughput lead from 26.9% to 14.2% while keeping a lower p95. HTTP/2 also varies substantially more between runs in this benchmark, so those medians deserve less confidence than the stable HTTP/1.1 results.
29
+ - On IO-bound HTTP/1.1, fixed mode holds application concurrency equal, while scaling mode lets each server add threads up to the same limit. Raptor also polls for accept capacity under saturation rather than parking, trading CPU time for throughput and latency. Equal thread counts do not imply equal CPU use.
30
+ - On CPU-bound HTTP/1.1, extra threads cannot execute the Rack application in parallel within one worker's main Ractor. This profile checks whether adaptive scaling adds overhead when more threads cannot help.
31
+ - On HTTP/2, Raptor and Falcon both implement it; Puma doesn't. The samples vary substantially more between runs than HTTP/1.1, so the README reports the spread as well as the medians.
32
+ - The IO + SSE scenario repeats the keep-alive IO run while server-sent event streams stay open on every worker. Raptor hands each stream to the reactor with `Raptor::DetachedBody`, so the streams cost it no application threads, while a thread-per-request server parks one thread per open stream. The README table shows how much throughput each server keeps.
32
33
 
33
- The rest of this doc explains why the shape looks like that.
34
+ The rest of this doc explains the design differences behind those comparisons; the README carries the measured rankings.
34
35
 
35
36
  ## The two servers at a glance
36
37
 
@@ -209,9 +210,9 @@ Successive thresholds promote fresh seeds. When the next threshold trips, the ma
209
210
 
210
211
  Master and seed communicate over two pipes opened before the initial fork. The fork pipe (`@fork_r`, `@fork_w`) carries an 8-byte packed `[slot_index, phase]` from master to seed for each replacement to fork. The response pipe (`@resp_r`, `@resp_w`) carries a 4-byte packed pid back the other way, plus a one-shot 4-byte zero the seed writes when it enters its loop so the master knows drain is complete. The master polls the ready marker non-blockingly on the supervision loop; the phased restart that fills the vacated slot and cycles the remaining workers only starts once the marker arrives (or the drain deadline expires, in which case the master clears the seed reference and falls back to direct forks).
211
212
 
212
- Workers forked by the seed are not the master's direct children. Before spawning anything the master calls `PR_SET_CHILD_SUBREAPER` (via a small `prctl` wrapper, `Raptor::Subreaper`), so seed-forked children reparent to the master immediately and `Process.wait2` sees them like any other child. When a seed is retired on the next promotion its still-running descendants stay attached to the master through the same subreaper bit. Refork depends on this. On platforms without `PR_SET_CHILD_SUBREAPER` (everything except Linux), `refork_after` defaults to `nil` and is ignored with a warning if the user sets it anyway.
213
+ Workers forked by the seed are not the master's direct children. Before spawning anything the master calls `PR_SET_CHILD_SUBREAPER` (via a small `prctl` wrapper, `Raptor::Subreaper`). The seed then forks each worker through a short-lived intermediate process that exits as soon as the worker exists, so the worker reparents to the master immediately and `Process.wait2` sees it like any other child, including its real exit status when it dies. When a seed is retired on the next promotion its still-running descendants stay attached to the master through the same subreaper bit. Refork depends on this. On platforms without `PR_SET_CHILD_SUBREAPER` (everything except Linux), `refork_after` defaults to `nil` and is ignored with a warning if the user sets it anyway.
213
214
 
214
- The design is Pitchfork's, adapted for Raptor's process model. [Pitchfork](https://github.com/Shopify/pitchfork) was the first Ruby server built around `PR_SET_CHILD_SUBREAPER` and multi-generation mould promotion; Puma's shipping `fork_worker` keeps worker 0 serving traffic alongside its fork-server duties, and Instacart's [`mold_worker` PR #3643](https://github.com/puma/puma/pull/3643) is Puma catching up to the same shape Pitchfork established. Raptor's seed matches the Pitchfork shape: pure fork source, promoted from a warm worker, retired when the next generation takes over.
215
+ The design is Pitchfork's, adapted for Raptor's process model. [Pitchfork](https://github.com/Shopify/pitchfork) was the first Ruby server built around `PR_SET_CHILD_SUBREAPER` and multi-generation mould promotion; Puma's shipping `fork_worker` keeps worker 0 serving traffic alongside its fork-server duties, and its `mold_worker` mode, merged in [PR #4038](https://github.com/puma/puma/pull/4038) after starting life as Instacart's [PR #3643](https://github.com/puma/puma/pull/3643), adopts the same shape Pitchfork established. Puma's mold forks workers directly and reaps them itself, so the master detects the workers it did not fork by probing them with signals; Raptor's double fork lets the master wait on every worker directly. Raptor's seed matches the Pitchfork shape: pure fork source, promoted from a warm worker, retired when the next generation takes over.
215
216
 
216
217
  ### Threading model
217
218
 
@@ -224,7 +225,7 @@ This is where Raptor diverges dramatically. Inside a worker there are several di
224
225
  5. **A collector thread per Ractor pool** that receives parsed results via a `Ractor::Port`.
225
226
  6. **An `AtomicThreadPool` of T baseline app threads** running the Rack app and writing responses, with optional adaptive growth.
226
227
  7. **One stats thread** that writes the shared-memory slot every second.
227
- 8. **One load reporter thread when BPF dispatch is active** that publishes backlog to the kernel map.
228
+ 8. **One load reporter thread when BPF dispatch is active** that publishes the worker's load to the kernel map.
228
229
 
229
230
  That is a lot of moving parts. Let us go through why.
230
231
 
@@ -234,7 +235,7 @@ For a web server, this restriction turns out to be almost exactly right for HTTP
234
235
 
235
236
  The HTTP/1 parser also pre-interns the ~40 most common header keys (`HTTP_HOST`, `HTTP_USER_AGENT`, the `HTTP_ACCEPT_*` family, `CONTENT_LENGTH`, `HTTP_X_FORWARDED_*`, the `HTTP_SEC_FETCH_*` client hints, and so on) once at load time. During parsing, a `memcmp` lookup against that table returns the shared frozen `VALUE` for known keys and falls back to `rb_enc_interned_str` for the rest. Every request's env hash therefore reuses the same String object for its header names, which both skips per-request allocation and lets Ruby's hash lookup use the interned key's cached hash code.
236
237
 
237
- Your Rack app still runs on ordinary threads under one worker GVL, so the app does not need to be Ractor-safe. Requests that need the reactor pipeline can have their protocol work run in parallel in another Ractor. Complete requests found by the eager accept and keep-alive paths parse inline instead, avoiding a Ractor handoff when the bytes are already available.
238
+ Your Rack app still runs on ordinary threads under one worker GVL, so the app does not need to be Ractor-safe. Requests that need the reactor pipeline can have their protocol work run in parallel in another Ractor. Complete requests found by the eager accept and keep-alive paths parse inline instead, avoiding a Ractor handoff when the bytes are already available. The same applies when the next request arrives on an idle keep-alive connection.
238
239
 
239
240
  **How the Ractor pools actually work.** Raptor uses the `ractor-pool` gem, which is another one of my libraries. Each pool has one coordinator Ractor and M pipeline Ractors. When a pipeline Ractor is idle, it sends itself back to the coordinator via `coordinator.send(Ractor.current, move: true)`. When work arrives at the coordinator, it either forwards it to a waiting Ractor (if any) or queues it. This coordinator-dispatch pattern guarantees that no Ractor sits idle while there is work. Results flow back through a shared `Ractor::Port` (a many-to-one channel added in recent Ruby versions and stable in 4.0) to a Ruby-side collector thread. If `M == 1` the coordinator is skipped and work goes straight to the single pipeline Ractor.
240
241
 
@@ -242,7 +243,7 @@ Raptor always runs an HTTP/1.1 pool because TCP, Unix, and SSL bindings support
242
243
 
243
244
  **Why a custom thread pool.** The `AtomicThreadPool` in `atomic-ruby` (another one of my libraries) is backed by an `AtomicQueue`. The queue is a Michael-Scott multi-producer, multi-consumer FIFO: a singly linked list with a dummy sentinel and atomic head and tail pointers. Producers append nodes at the tail; consumers advance the head. Both operations are O(1) and make progress through compare-and-swap rather than a queue-wide mutex. Separate atoms track queue size and active app threads for backpressure.
244
245
 
245
- The pool starts at `threads` and scales without a fixed limit by default. Set `max_threads` to cap its growth, or set it to the same value as `threads` to keep the pool fixed. A native CRuby thread-event hook measures how much active workers spend running, blocked outside the GVL, and waiting to acquire it. The pool only adds a temporary thread after work has remained queued across several samples, every current worker is active, blocked time is above half of measured worker time, and GVL wait is below two percent. That last check matters: adding threads helps when existing threads are asleep in database or network calls, but hurts when CPU-bound Ruby threads are already fighting over the GVL. Temporary threads retire after the queue has remained empty for a second.
246
+ The pool starts at `threads` and scales without a fixed limit by default. Set `max_threads` to cap its growth, or set it to the same value as `threads` to keep the pool fixed. A native CRuby thread-event hook measures how much time workers spend running Ruby, blocked outside the GVL, and waiting to acquire it. While work stays queued and every worker is busy, the pool adds temporary threads when workers spend most of their time blocked and Ruby execution is using less than three quarters of a CPU. When Ruby execution becomes the bottleneck it retires them again. That distinction matters: adding threads helps when existing threads are asleep in database or network calls, but hurts when CPU-bound Ruby threads are already fighting over the GVL. Temporary threads also retire after the queue has stayed empty for five seconds.
246
247
 
247
248
  The pool still uses an `AtomicConditionVariable` under the hood to park idle threads (idle threads call `Thread.stop` and get woken with `Thread#wakeup`; there is no spinning), because idle spinning would waste CPU. The difference from Puma's pool is not "no locks anywhere" but rather "the hot path (enqueue and dequeue when the queue has items) is lock-free". Once every worker is busy the mechanics look similar; where things diverge is under contention when you have many threads all trying to push and pop.
248
249
 
@@ -252,26 +253,27 @@ The knock-on effect is that the server thread can read `pool.queue_size + pool.a
252
253
 
253
254
  ### I/O model
254
255
 
255
- The server accept loop is an `IO.select` + `accept_nonblock` loop, similar to Puma. What is different is the pair of checks right before `accept`:
256
+ The server accept loop is an `IO.select` + `accept_nonblock` loop, similar to Puma. What is different is the backpressure check right before `accept`:
256
257
 
257
258
  ```ruby
258
259
  backpressure_threshold = [(@thread_pool.size * BACKPRESSURE_THRESHOLD_MULTIPLIER).ceil, MIN_BACKPRESSURE_THRESHOLD].max
259
- # ...
260
- next if @reactor.backlog >= backpressure_threshold
261
-
262
- if @thread_pool.queue_size > @thread_pool.size
263
- Thread.pass
260
+ if @reactor.backlog >= backpressure_threshold || @thread_pool.queue_size > @thread_pool.size
261
+ CPU.yield
264
262
  next
265
263
  end
266
264
  ```
267
265
 
268
- The first is a hard skip. `@reactor.backlog` is `thread_pool.queue_size + thread_pool.active_count`; when the total load reaches the threshold, this worker stops accepting until it drains. `MIN_BACKPRESSURE_THRESHOLD` is 8, so a three-thread pool trips at 8 concurrent items rather than 4. The floor avoids overreacting to a few active requests while still bounding the amount of work a worker pulls from the kernel.
266
+ The server thread only accepts while two things hold. The worker's total load, `@reactor.backlog` (queued plus active work), must be below its threshold, and the queue alone must not exceed the pool size. Otherwise the thread yields and returns to polling the listeners. On Linux, the yield also gives native threads returning from I/O a chance to run without putting the server thread to sleep. Under saturation this consumes CPU to keep checking capacity rather than parking until an app thread signals a change; that is a deliberate throughput and latency tradeoff.
267
+
268
+ The threshold is 1.2 times the current pool size, with a floor of 8, so a three-thread pool trips at 8 concurrent items rather than 4. The floor avoids overreacting to a few active requests while still bounding the amount of work a worker pulls from the kernel. Because it is recalculated from the current size, a pool that has grown to handle blocking work accepts proportionally more.
269
269
 
270
- The second is a softer yield. When the queue alone exceeds the pool size — the app threads are all busy and there's a queue building on top of them — the accept loop yields the GVL via `Thread.pass` and re-checks the queue on the next iteration instead of accepting more work. This lets the app threads make progress before the server thread grabs another connection, and only fires under real pool pressure (queue depth greater than pool size), so IO-bound workloads where threads spend most of their time in `sleep` and rarely queue past the pool size aren't affected.
270
+ This limit applies to new connections, not to new requests on existing keep-alive connections. Existing connections can therefore keep a worker busy enough that it stops accepting new ones. Polling does not guarantee those new connections will make progress.
271
271
 
272
- On Linux, Raptor can replace the shared TCP listener with one `SO_REUSEPORT` listener per worker and attach a small BPF program to the group. A reporter thread publishes each worker's backlog into a kernel map every millisecond. For each new connection, the program hashes to two distinct workers and selects the one with the lower reported load: the power-of-two-choices strategy. It then atomically increments that worker's map slot before routing the connection, reserving capacity immediately rather than waiting for the next reporter tick. The accept path also publishes `reactor.backlog + 1` as soon as Ruby receives the socket. These reservations stop a burst of connections from repeatedly choosing the same stale minimum while retaining hash-based spread across the cluster.
272
+ On Linux, Raptor also sets `TCP_DEFER_ACCEPT` on its TCP listeners, so the kernel only hands over a connection once the client has sent data. Fresh connections therefore usually arrive with their request already readable, which is what the eager fast path below relies on.
273
273
 
274
- This path only wraps plain `tcp://` bindings. TLS listeners and non-TCP bindings remain shared listeners inherited from the master. If `libbpf-ruby` or the compiled BPF object is unavailable, Raptor silently uses those shared listeners for TCP too; if the prerequisites exist but the kernel refuses the program, startup raises. In either mode, a worker under Raptor's explicit backpressure stops competing for new accepts until its current work drains.
274
+ On Linux, Raptor can give each worker its own `SO_REUSEPORT` listener on every plain TCP address it binds and attach a small BPF program to each address's group. One program serves them all: its socket map holds a row of worker listeners per address, and it routes each connection to the chosen worker's listener in the group the connection arrived on. A reporter thread publishes each worker's load into a kernel map every millisecond: its queued and active work plus any connections still waiting in its accept queue, which the worker cannot see from Ruby until it accepts them. For each new connection, the program hashes to two distinct workers and selects the one with the lower reported load: the power-of-two-choices strategy. Once the connection is routed, it atomically increments that worker's map slot, reserving capacity immediately rather than waiting for the next reporter tick. The accept path also publishes its load as soon as Ruby receives the socket. These reservations stop a burst of connections from repeatedly choosing the same stale minimum while retaining hash-based spread across the cluster.
275
+
276
+ This path only wraps plain `tcp://` bindings. Every worker still accepts on the shared listeners inherited from the master, which carry TLS and non-TCP bindings as well as any TCP connection the BPF program cannot route, such as one arriving before every worker has booted. If `libbpf-ruby` or the compiled BPF object is unavailable, Raptor silently uses those shared listeners for TCP too; if the prerequisites exist but the kernel refuses the program, startup raises. In either mode, a worker under Raptor's explicit backpressure stops competing for new accepts until its current work drains.
275
277
 
276
278
  The BPF-based approach was inspired by [a comment](https://github.com/puma/puma/issues/3934#issuecomment-4356462590) by John Hawthorn ([@jhawthorn](https://github.com/jhawthorn)) on a Puma issue about `EPOLLEXCLUSIVE`, where he floated `SO_ATTACH_REUSEPORT_EBPF` as a way to route each connection to the least-busy worker.
277
279
 
@@ -324,7 +326,7 @@ The thread checks for bytes without waiting. If they are already available, it p
324
326
 
325
327
  Puma has a similar shape. It checks buffered back-to-back requests, then eagerly drains bytes already available on the socket. If a complete request is ready and the pool has a waiting thread, the current thread loops inline; otherwise Puma queues the client or returns it to the reactor.
326
328
 
327
- The `reactor.persist` call re-registers the socket with the reactor using `persistent_data_timeout` (65s) as the new deadline. When the next bytes arrive, the reactor treats the socket like any other partially-read connection.
329
+ The `reactor.persist` call re-registers the socket with the reactor using `persistent_data_timeout` (65s) as the new deadline. When the next bytes arrive, the reactor hands the socket back to the thread pool, and an app thread reads and parses the request inline. Only a request that is still incomplete takes the Ractor pipeline.
328
330
 
329
331
  A `Raptor::DetachedBody` returns the application thread while keeping the response open, chunked on HTTP/1.1 and ended by closing the connection on HTTP/1.0. The reactor owns the response until it closes, buffers later writes within fixed limits, and pauses the connection's next request without occupying an application thread.
330
332
 
@@ -365,7 +367,7 @@ flowchart TB
365
367
  end
366
368
 
367
369
  subgraph Worker["Raptor worker process, one of N"]
368
- SRV["Server thread<br/>IO.select + accept_nonblock<br/>backpressure check<br/>reactor.backlog vs pool_size"]
370
+ SRV["Server thread<br/>IO.select + accept_nonblock<br/>waits for pool capacity"]
369
371
 
370
372
  RCT["Reactor thread<br/>NIO::Selector<br/>read_nonblock 64KB<br/>Red-Black-Tree timeouts"]
371
373
 
@@ -409,6 +411,7 @@ flowchart TB
409
411
  KA -->|"yes"| EAG
410
412
  EAG -.->|"bytes ready, parse+dispatch on same thread"| ATP
411
413
  EAG -->|"no bytes, reactor.persist"| RCT
414
+ RCT -.->|"keep-alive bytes ready, push proc"| ATP
412
415
  RCT -->|"deadline expired"| TO["PING or close"]
413
416
 
414
417
  STA -.->|"writes slot"| SHM[("mmap shared memory")]
@@ -439,9 +442,9 @@ The critical structural difference from Puma is that Raptor has a separate proto
439
442
 
440
443
  **Puma.** Parsing happens on an app thread. The C parser callbacks build the env hash. A fresh client first enters the thread pool, where eager reads may complete it immediately; partial and idle keep-alive connections wait in the reactor before returning to the pool. Parsing shares the worker's GVL with the app.
441
444
 
442
- **Raptor.** Fresh and immediate keep-alive requests parse inline on the server or app thread. A connection that needs more bytes takes the longer path: reactor (I/O) → protocol Ractor pool (parse) → collector → app thread pool (Rack + write). Between keep-alive requests, the app thread checks for bytes without waiting before returning an idle connection to the reactor. Parsing in the Ractor pipeline has its own GVL; parsing on an eager path does not.
445
+ **Raptor.** Fresh and immediate keep-alive requests parse inline on the server or app thread. A connection that needs more bytes takes the longer path: reactor (I/O) → protocol Ractor pool (parse) → collector → app thread pool (Rack + write). Between keep-alive requests, the app thread checks for bytes without waiting before returning an idle connection to the reactor, which hands it back to an app thread once the next request arrives. Parsing in the Ractor pipeline has its own GVL; parsing on an eager path does not.
443
446
 
444
- In practice, Puma has one process-wide GVL per worker. Every Ruby thread inside that worker takes turns holding it. Raptor has the same main-Ractor GVL plus one GVL per protocol Ractor, so a pipeline Ractor can parse one connection while an app thread executes Rack for another. That parallelism is real, but so are the costs of making state shareable and crossing the Ractor and collector boundaries. Which side wins depends on how much work the request gives the protocol pipeline; the current CPU benchmark leaves Raptor and Puma close rather than proving a universal parsing advantage.
447
+ In practice, Puma has one process-wide GVL per worker. Every Ruby thread inside that worker takes turns holding it. Raptor has the same main-Ractor GVL plus one GVL per protocol Ractor, so a pipeline Ractor can parse one connection while an app thread executes Rack for another. That parallelism is real, but so are the costs of making state shareable and crossing the Ractor and collector boundaries. The current CPU-bound HTTP/1.1 throughput results favour Raptor in three of four configurations; Puma leads slightly with fixed threads and keep-alive. Raptor's p95 latency is higher on fresh connections and lower with keep-alive. These are full-server measurements, not an isolated comparison of the parsers.
445
448
 
446
449
  ### Timeout management
447
450
 
@@ -465,7 +468,7 @@ Under moderate load, queue mechanics are unlikely to dominate either server. Und
465
468
 
466
469
  **Puma.** After a response, if the connection is keep-alive and there are already buffered bytes for the next request (`has_back_to_back_requests?`) and there is a spare app thread, loop inline. Otherwise, if `eagerly_finish` (non-blocking reads while data is already buffered) returns true, either loop inline (if spare threads) or hand back to the thread pool (`@thread_pool << client`). Otherwise, back to the reactor with `@persistent_timeout`.
467
470
 
468
- **Raptor.** After a response, the app thread checks `socket.wait_readable(0)`. If bytes are already available, it parses the next request inline. If other work is waiting, the parsed request goes to the back of the pool queue; otherwise the same thread dispatches it inline. If no bytes are ready, `reactor.persist` and return.
471
+ **Raptor.** After a response, the app thread checks `socket.wait_readable(0)`. If bytes are already available, it parses the next request inline. If other work is waiting, the parsed request goes to the back of the pool queue; otherwise the same thread dispatches it inline. If no bytes are ready, `reactor.persist` and return. When the next request arrives, the reactor queues the connection for an app thread, which reads and parses it inline without a Ractor round trip.
469
472
 
470
473
  Both servers therefore keep a back-to-back request on an app thread when its bytes are already waiting, and return an idle connection to the reactor without deliberately holding an app thread open. The details of their parsing and pool handoff still differ, but neither widens the inline window by waiting for the client.
471
474
 
@@ -475,7 +478,7 @@ Raptor closes an HTTP/1.1 connection after 1,000 requests by default. The finite
475
478
 
476
479
  **Puma.** Cluster mode uses `accept_loop_delay` (sleep proportional to busy ratio) to prevent thundering herd across workers. Single-worker backpressure is implicit; if all threads are busy and the queue is growing, new accepts pile up in the kernel accept queue. Puma does have `queue_requests` (default true) which pushes partial requests into the reactor, freeing the accept loop, but there is no explicit "stop accepting" signal from the worker.
477
480
 
478
- **Raptor.** Explicit backpressure, read every iteration of the accept loop: hard skip when `backlog >= max(pool_size * 1.2, 8)`, and a softer `Thread.pass` yield when the queue alone exceeds the pool size. On supported Linux TCP bindings, the BPF reuseport program samples two workers, routes to the less loaded one, and reserves its map slot. TLS, Unix sockets, unsupported platforms, and installations without the BPF prerequisites use inherited shared listeners instead.
481
+ **Raptor.** Explicit backpressure before every accept: the server thread yields and polls again while `backlog >= max(pool_size * 1.2, 8)` or the queue alone exceeds the pool size. On supported Linux TCP bindings, the BPF reuseport program samples two workers, routes to the less loaded one, and reserves its map slot, counting connections still waiting in each worker's accept queue. TLS, Unix sockets, unsupported platforms, and installations without the BPF prerequisites use inherited shared listeners instead.
479
482
 
480
483
  ### Shared state (worker ↔ master)
481
484
 
@@ -571,6 +574,7 @@ sequenceDiagram
571
574
  RP->>RR: reactor.persist
572
575
  Client->>RR: request bytes
573
576
  RR->>RP: dispatch back to thread pool
577
+ RP->>RP: read + parse inline
574
578
  RP->>Client: response 3
575
579
  ```
576
580
 
@@ -580,21 +584,21 @@ In this timing, both servers keep Request 2 inline and return Request 3 to the r
580
584
 
581
585
  ### IO-bound work, where app concurrency matters
582
586
 
583
- On the IO-bound benchmark profile, each request does 5 to 10 short sleeps interleaved with small CPU work, simulating a request that makes several DB or cache calls throughout its lifetime. The bottleneck is how many requests a worker can keep in flight while they wait on IO. Fixed Raptor and Puma cap application execution at three threads per worker. Falcon spawns a fiber per connection and cooperatively yields on every sleep, so many more client connections can make progress while others wait. That advantage gives Falcon the clear lead over both fixed-thread servers, especially without keep-alive.
587
+ On the IO-bound benchmark profile, each request makes 5 to 10 simulated DB calls and renders a small JSON result after each one, spending about 95% of its time waiting. The bottleneck is how many requests a worker can keep in flight while they wait on IO. Fixed Raptor and Puma cap application execution at three threads per worker. Falcon spawns a fiber per connection and cooperatively yields on every sleep, so many more client connections can make progress while others wait. Raptor's accept-side polling also keeps CPUs active under saturation, which can affect the timing of these short sleeps. The fixed-thread comparison therefore measures more than parsing and dispatch overhead.
584
588
 
585
- Scaling Raptor starts with the same three threads, then adds temporary threads while work is queued and the active threads are mostly blocked outside the GVL. In the current results that raises throughput from 3.11k to 8.63k req/s without keep-alive, and from 3.21k to 9.43k with it. Falcon still leads the first by 29.3%; scaling Raptor leads the second by 51.3%. The benchmark reports fixed and scaling Raptor separately rather than hiding that difference in one result.
589
+ Scaling Raptor starts with the same three threads, then adds temporary threads while work is queued and the active threads are mostly blocked outside the GVL. Puma adds a thread whenever work is waiting. Raptor's stricter rule is intended to avoid adding threads that CPU-bound work cannot use. In the current HTTP/1.1 IO results, fixed Raptor delivers roughly 2.3 times Puma's throughput, while the two servers are approximately tied when both can scale. The benchmark reports the modes separately rather than hiding that distinction in one result.
586
590
 
587
- Between the thread-based servers, Raptor holds a clear lead over Puma on both throughput and p95. Its eager paths, explicit admission control, response batching, and app pool are all designed to reduce coordination, but the benchmark does not isolate enough variables to assign the result to one of them.
591
+ Long-lived responses change the picture. In the IO + SSE scenario, every worker also holds server-sent event streams open for the whole run. Raptor serves them with `Raptor::DetachedBody`, which hands the socket to the reactor and returns the application thread to the pool, so fixed Raptor keeps all three threads for ordinary requests. Puma streams each event from an application thread that sleeps between events, so every open stream takes a thread away from the rest of the traffic until the stream closes. Fixed Raptor delivers 9.5 times Puma's throughput in the current results, with Puma also reporting timeouts. With scaling threads, the gap narrows to a few percent. The README table includes requests the load generator abandoned after 20 seconds.
588
592
 
589
593
  Real applications that spend most of their time waiting on a database or an upstream service look like this. If your app is IO-heavy and you're free to adopt the `async` ecosystem, Falcon and scaling Raptor are the interesting comparison: fibers are cheaper, while Raptor can run ordinary blocking Rack code without requiring a fiber-aware stack.
590
594
 
591
- ### CPU-bound HTTP/1.1, where Puma and Raptor converge
595
+ ### CPU-bound HTTP/1.1, where extra threads cannot help
592
596
 
593
- On the CPU-bound benchmark profile, each POST request accepts a small JSON body and builds a JSON response in 3 to 5 chunks totalling 450 to 1500 items, with sub-100µs sleeps between chunks. It's roughly 95% CPU by wall time, so fibers can't multiplex their way to an advantage. The CPU work happens under a single Ruby VM regardless of concurrency model.
597
+ On the CPU-bound benchmark profile, each POST request accepts a small JSON body and renders 2,100 to 7,000 JSON items in 3 to 5 chunks, each after a 25 to 50µs simulated cache hit. It spends about 85% of its time running Ruby, so fibers can't multiplex their way to an advantage. The CPU work happens under a single Ruby VM regardless of concurrency model.
594
598
 
595
- **Without keep-alive**, every request opens a fresh TCP connection, gets parsed, dispatched, served, and closes. Puma leads fixed Raptor by 5.0% on throughput and has the lower p95; Raptor still leads Falcon. Fixed and scaling Raptor both deliver 8.22k req/s. The Puma/Raptor gap is small enough that neither architecture has overwhelmed the CPU cost of the Rack workload.
599
+ **Without keep-alive**, every request opens a fresh TCP connection, gets parsed, dispatched, served, and closes. This includes accept and connection setup costs alongside the Rack workload.
596
600
 
597
- **With keep-alive**, Puma leads fixed Raptor by 0.7% on throughput, while Raptor's p95 is 0.4ms lower; Raptor leads Falcon on both. Fixed and scaling Raptor both deliver 8.49k req/s, with scaling p95 matching Puma. The result is more useful as a guardrail than a victory claim: adaptive scaling provides the IO-bound gains without hurting this CPU-bound profile.
601
+ **With keep-alive**, subsequent requests reuse the connection, putting more emphasis on request handling and application execution. Comparing fixed and scaling modes checks whether adaptive scaling preserves CPU throughput while adding useful concurrency on the IO-bound profile.
598
602
 
599
603
  ### HTTP/2, when it matters
600
604
 
@@ -602,7 +606,7 @@ Puma doesn't implement HTTP/2, and most Rails production terminates HTTP/2 at ng
602
606
 
603
607
  Where Raptor's HTTP/2 support does matter is the all-Ruby stack: clients can speak h2 directly over TLS, or a trusted proxy can use cleartext HTTP/2 to an `h2c://` listener. In that setup, Puma uses HTTP/1.1 instead, so the app-server connection does not get HTTP/2 multiplexing or HPACK header compression.
604
608
 
605
- Falcon also speaks HTTP/2 natively, so it's the interesting comparison there rather than Puma. On the CPU profile, scaling narrows Raptor's throughput gap from 26.9% to 14.2% and improves its p95 advantage from 12.3% to 21.3%. On IO, scaling raises Raptor from 1.14k to 4.46k req/s, but remains 30.3% behind Falcon. The h2 samples vary substantially more than the h1 samples, so these results establish broad shape rather than a precise ranking.
609
+ Falcon also speaks HTTP/2 natively, so it's the interesting comparison there rather than Puma. Both workload profiles compare the full HTTP/2 path, including TLS, multiplexing, and flow control. The h2 samples vary substantially more than the h1 samples; the README gives the current throughput, p95, and spread rather than treating one run as a precise ranking.
606
610
 
607
611
  The benchmark's h2 listener uses TLS, while Raptor's BPF reuseport path only wraps plain TCP listeners. BPF dispatch therefore cannot explain the h2 variance. With 40 physical connections spread across 10 workers, each carrying three streams, placement and per-connection scheduling have coarse granularity; more instrumentation is needed before assigning the variance to a specific mechanism.
608
612
 
@@ -610,12 +614,14 @@ Raptor's HTTP/2 CPU-bound throughput remains in the same broad range as its HTTP
610
614
 
611
615
  ## Part V: What Raptor gives up
612
616
 
613
- No design comes free. Two disclosures matter most.
617
+ No design comes free. Three disclosures matter most.
614
618
 
615
619
  **Battle-tested.** Puma has been running production Rails apps since 2011. Raptor has been running my benchmarks since 2026. Those are not the same thing. Raptor is still young code. If your risk tolerance requires a decade of production hardening baked into the server itself, Puma is the answer today.
616
620
 
617
621
  **Ruby version.** Raptor requires Ruby 4.0 because it depends on `Ractor::Port` and on Ractor internals having stabilised. Puma works on 3.0 and up. If you need to support older Ruby, Puma wins by default.
618
622
 
623
+ **Ractor overhead.** Starting any Ractor puts the VM into multi-Ractor mode, and on Ruby 4.0 that makes ordinary Ruby in the same process measurably slower, even on threads that never touch a Ractor. On Ruby head, which becomes 4.1, that cost is gone in my measurements, and 4.1 also collects each Ractor's heap on its own instead of stopping every Ractor for each GC. Ractors are still experimental, so I expect this side of the trade-off to keep shrinking.
624
+
619
625
  A handful of smaller trade-offs are worth naming briefly. A request on Raptor's reactor pipeline pays for shareability and Ractor/collector handoffs; eager requests avoid them. Raptor's core dependencies (`ractor-pool`, `atomic-ruby`, `red-black-tree`, `mmap-ruby`, `libbpf-ruby`) are libraries I wrote specifically to make it work, which is either "purpose-built" or "narrower testing surface" depending on how you look at it. Debugging is harder because the slow path crosses Ractor and thread boundaries, so tracing it end-to-end means stitching several stack traces together. Raptor has no single-process mode, and on a single-CPU container that adds coordination without the possibility of parallel execution.
620
626
 
621
627
  ## Which server to choose