raptor 0.22.3 → 0.24.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +46 -0
  3. data/Dockerfile +10 -2
  4. data/README.md +183 -61
  5. data/docs/assets/profile_picture.png +0 -0
  6. data/docs/assets/startup_output.svg +37 -0
  7. data/docs/raptor-vs-puma.md +104 -424
  8. data/docs/talk.md +1388 -0
  9. data/ext/raptor_bpf/reuseport_select.bpf.c +18 -9
  10. data/ext/raptor_http/extconf.rb +2 -0
  11. data/ext/raptor_http/raptor_http.c +165 -234
  12. data/ext/raptor_http2/raptor_http2.c +272 -223
  13. data/ext/raptor_native/raptor_native.c +38 -20
  14. data/lib/rackup/handler/raptor.rb +14 -12
  15. data/lib/raptor/binder.rb +12 -12
  16. data/lib/raptor/cli.rb +19 -18
  17. data/lib/raptor/cluster.rb +120 -74
  18. data/lib/raptor/control_server.rb +5 -2
  19. data/lib/raptor/detached_body.rb +6 -2
  20. data/lib/raptor/http.rb +7 -6
  21. data/lib/raptor/http1.rb +281 -234
  22. data/lib/raptor/http2.rb +356 -276
  23. data/lib/raptor/plugin.rb +98 -0
  24. data/lib/raptor/reactor.rb +138 -148
  25. data/lib/raptor/reuseport_bpf.rb +62 -38
  26. data/lib/raptor/server.rb +45 -34
  27. data/lib/raptor/stats.rb +15 -3
  28. data/lib/raptor/thread_locals.rb +4 -3
  29. data/lib/raptor/version.rb +1 -1
  30. data/sig/generated/rackup/handler/raptor.rbs +2 -1
  31. data/sig/generated/raptor/binder.rbs +9 -9
  32. data/sig/generated/raptor/cli.rbs +6 -6
  33. data/sig/generated/raptor/cluster.rbs +18 -5
  34. data/sig/generated/raptor/control_server.rbs +2 -1
  35. data/sig/generated/raptor/detached_body.rbs +5 -1
  36. data/sig/generated/raptor/http.rbs +3 -5
  37. data/sig/generated/raptor/http1.rbs +119 -102
  38. data/sig/generated/raptor/http2.rbs +204 -190
  39. data/sig/generated/raptor/plugin.rbs +52 -0
  40. data/sig/generated/raptor/reactor.rbs +107 -118
  41. data/sig/generated/raptor/reuseport_bpf.rbs +54 -17
  42. data/sig/generated/raptor/server.rbs +26 -26
  43. data/sig/generated/raptor/stats.rbs +3 -2
  44. data/sig/generated/raptor/thread_locals.rbs +4 -3
  45. metadata +6 -2
  46. data/docs/brisrails-talk.md +0 -2398
@@ -1,116 +1,43 @@
1
- # Raptor vs Puma: A Design Comparison
1
+ # Raptor vs Puma
2
2
 
3
- Raptor is a Ruby web server built around Ractor-parallel protocol pipelines, a lock-free app thread pool, and an opinionated cluster architecture. Against the latest Puma release on the same hardware, it leads clearly on the IO-heavy HTTP/1.1 benchmark and lands within a few percent on CPU-heavy throughput, while Falcon's fibers remain the strongest fit for highly concurrent IO. Raptor also speaks HTTP/2 natively, which Puma does not. This document explains how those designs move requests and where each trade-off shows up.
3
+ Puma and Raptor both run Rack applications on threads inside worker processes. Their main differences are how they handle protocol work, admit connections, and support long-lived responses. Raptor also implements HTTP/2. Puma leaves that to a proxy.
4
4
 
5
- ## Why this document exists
5
+ Raptor began as an experiment with Ruby's Ractors and is still a young server. Puma has a long production history and a much larger operational ecosystem.
6
6
 
7
- Raptor began as a curiosity project. Ruby 4.0 was landing with a more polished Ractor implementation, and I wanted to see what a web server would look like if you actually leaned into parallel Ruby instead of pretending the GVL was not there. The initial goal was modest. Build something that could parse HTTP in parallel using Ractors, hook it up to Rack, and see if the numbers moved.
7
+ This guide compares their architecture and relates it to the [current README benchmarks](../README.md#micro-benchmarks). Those benchmarks measure the whole server, so architectural differences alone do not explain individual results.
8
8
 
9
- They did, but the more interesting result was structural. Committing to Ractor-based protocol pipelines made every boundary around them visible: which requests are worth sending through a Ractor, how partial requests return to the reactor, how completed requests reach ordinary app threads, and how concurrent HTTP/2 responses share one socket. It also pushed the rest of the server toward cheap timeout updates, a lock-free app work queue, explicit backpressure, and cluster dispatch that accounts for current load. No one of those choices explains the benchmark. Together they define the server.
9
+ ## At a glance
10
10
 
11
- This document walks through both servers at systems-design depth. By the end you should be able to describe how Puma and Raptor each move a request from `accept()` to `app.call(env)` and back, why the two architectures make the design decisions they do, and where the performance delta actually comes from. No source code required.
11
+ | Dimension | Puma | Raptor |
12
+ | --- | --- | --- |
13
+ | Ruby requirement | 3.0 and up | 4.0 and up |
14
+ | Process model | Single process or cluster | Preloaded, pre-fork cluster only |
15
+ | Application execution | Thread pool within each worker | Thread pool within each worker |
16
+ | Protocol processing | App threads and reactor share the worker's GVL | Eager parsing inline, separate protocol Ractors on the pipeline path |
17
+ | Pool coordination | Mutex and condition variables | Lock-free work queue with idle-thread parking |
18
+ | Pool growth | Adds threads when queued work exceeds waiting capacity | Adds threads when queued work and blocking time justify growth |
19
+ | Idle and partial connections | `nio4r` reactor | `nio4r` reactor |
20
+ | Cluster dispatch | Shared listeners with load-sensitive accept delay | Shared listeners, plus load-aware BPF dispatch on supported Linux TCP bindings |
21
+ | HTTP/2 | Not implemented | Native parsing, HPACK and stream flow control |
22
+ | Worker monitoring | Pipe messages | Shared-memory stats |
12
23
 
13
- ## A note on Falcon
24
+ The application does not need to be Ractor-safe to use Raptor. Rack runs on ordinary threads in the worker's main Ractor. It still needs to be thread-safe, just as it does with a multithreaded Puma configuration.
14
25
 
15
- Falcon is the other next-generation Ruby web server worth naming. It takes a different bet than either Puma or Raptor. Concurrency comes from fibers via the `async` gem instead of threads, HTTP/2 is native, and each request is a lightweight task rather than a slot in an app thread pool. Its strengths show up on workloads with lots of concurrent long-lived connections (WebSockets, SSE, streaming), applications built end-to-end on the `async` ecosystem, and HTTP/2-heavy traffic. On a traditional Rails workload where most of the request budget is a synchronous DB round-trip through ActiveRecord, its advantages over Puma are less pronounced because the fiber scheduler only helps when the underlying I/O is fiber-aware. If your app fits that sweet spot, Falcon belongs in your evaluation.
26
+ Falcon is a useful third comparison. It uses fibers through the `async` ecosystem and supports HTTP/2. Fibers can keep many requests in flight without assigning each one an OS thread. That depends on application I/O cooperating with the fiber scheduler. The README includes Falcon, but this is not a full comparison of its design.
16
27
 
17
- This document focuses on Puma because Puma is the incumbent that any new Ruby web server has to justify itself against; that is the comparison most readers actually need. Falcon appears in the benchmark table in the README as a third data point, but a full design comparison against Falcon would be its own document.
28
+ ## How Puma moves a request
18
29
 
19
- ## The shape of the benchmark
30
+ In cluster mode, a master supervises worker processes. Each worker accepts connections and has its own application thread pool. Puma can preload the application before forking so workers initially share memory through copy-on-write. It can also run without a cluster.
20
31
 
21
- Raptor is a research project. It hasn't run production traffic. The numbers in the README come from a repeatable microbenchmark, not from a real deployment. They use controlled Rack workloads to measure the full server path: accepting connections, parsing, dispatching, running the app, and writing responses. A real Rails application adds database time, downstream services, middleware, and application-specific contention, so a percentage here will not transfer directly to production.
32
+ With the default request queueing enabled, a newly accepted socket first goes to the thread pool. An app thread tries to read and parse a complete request without waiting for more bytes. If it succeeds, it calls Rack and writes the response. If the request is incomplete, it returns the socket to the reactor and becomes available for other work.
22
33
 
23
- The Raptor README carries the [current head-to-head numbers](../README.md#micro-benchmarks) against the latest Puma and Falcon releases, run on the same hardware with the same Rack app on a recent Ruby with YJIT enabled. All servers run one worker process per available CPU. Puma and Raptor's fixed mode use three app threads per worker. Raptor's scaling mode starts at three and has no fixed limit, while Falcon uses unbounded fibers. Load generators use four client connections per baseline app thread, so on the 10-core machine that produced the current numbers that's 120 concurrent HTTP/1.1 client connections and 40 h2 connections × 3 streams each.
34
+ The reactor watches partial requests and idle keep-alive connections. When bytes arrive, it can read and parse them through the client object. Once the request is complete, it queues the client for an app thread. Puma's native HTTP/1 parser therefore runs on both app threads and the reactor path. All of that Ruby coordination shares the worker's GVL with the application.
24
35
 
25
- Two workload profiles are measured. **IO-bound** is a GET endpoint that does 5 to 10 short sleeps interleaved with small CPU work per request, simulating a read path that makes several DB or cache calls throughout its lifetime. **CPU-bound** is a POST endpoint with a small JSON body that builds a JSON response in 3 to 5 chunks interleaved with sub-100µs sleeps, simulating a write path that does most of its work in Ruby with a few near-zero-cost cache hits. The workloads are interleaved rather than a single bulk sleep or single bulk serialise so a fiber-per-connection server like Falcon doesn't look artificially good from one-shot IO, and the CPU-bound workload is heavily CPU-dominated by design (roughly 95% CPU / 5% IO by wall time) so it actually measures CPU work rather than smuggling in enough IO for fibers to multiplex.
36
+ After a response, Puma can handle a ready follow-up request on the same app thread when pool capacity allows. Otherwise it queues the client again or returns an idle connection to the reactor. An idle keep-alive connection does not need to hold an application thread.
26
37
 
27
- Each cell in the table reports the median throughput and median p95 latency independently across 3 runs, and every run boots a fresh server process so state cannot accumulate across measurements. Both Raptor modes are compared with both servers: fixed mode is the like-for-like comparison with Puma, while scaling mode tests whether adaptive OS threads can close the IO-concurrency gap with Falcon's much cheaper fibers. Rather than pin every number into this document (they drift with Ruby versions and hardware), the shape of the result is what matters.
38
+ ### Puma request flow
28
39
 
29
- - On IO-bound HTTP/1.1, fixed Raptor delivers a little over twice Puma's throughput with less than half its p95 latency. Scaling lifts Raptor to 8.63k req/s without keep-alive and 9.43k with it: still 29.3% behind Falcon on fresh connections, but 51.3% ahead with keep-alive.
30
- - On CPU-bound HTTP/1.1, fixed and scaling Raptor are effectively identical. Puma leads by 5.0% without keep-alive and 0.7% with it, while both Raptor modes lead Falcon. This is the intended result: the pool gets its IO gains without trading away CPU throughput.
31
- - On HTTP/2, Raptor and Falcon both implement it; Puma doesn't. Scaling raises Raptor's IO throughput from 1.14k to 4.46k req/s, though it remains 30.3% behind Falcon. On the CPU profile it narrows Falcon's throughput lead from 26.9% to 14.2% while keeping a lower p95. HTTP/2 also varies substantially more between runs in this benchmark, so those medians deserve less confidence than the stable HTTP/1.1 results.
32
-
33
- The rest of this doc explains why the shape looks like that.
34
-
35
- ## The two servers at a glance
36
-
37
- | Dimension | Puma | Raptor |
38
- | ---------------------- | ------------------------------------------------------- | ---------------------------------------------------------------------------- |
39
- | Ruby requirement | 3.0 and up | 4.0 and up (needs `Ractor::Port`) |
40
- | Process model | Single, or cluster (pre-forks by default with 2+ workers); optional refork | Cluster only, always pre-forks |
41
- | Threading model | `ThreadPool` with `Mutex` + `ConditionVariable`; autoscaling min/max, or fixed when min == max | `AtomicThreadPool` with a CAS-based queue; GVL-aware adaptive scaling from a fixed baseline, with an optional limit |
42
- | True parallelism | One GVL inside each worker; parser runs on an app thread | Protocol Ractors can parse in parallel with the app on the pipeline path |
43
- | I/O multiplexing | `nio4r` reactor for keep-alive idle and slow reads | `nio4r` reactor for the same, plus a red-black tree for O(log n) timeouts |
44
- | Cluster dispatch | Workers race on inherited listeners with a load-proportional accept delay | Two-choice load-aware BPF dispatch for TCP on Linux; shared-listener fallback |
45
- | Work queue | Ruby `Queue` coordinated under the pool mutex | Lock-free Michael-Scott FIFO queue |
46
- | HTTP/2 | Not implemented | Native C parser + HPACK, reactor-owned frame scheduler |
47
- | Keep-alive fast path | Same-thread inline dispatch when spare threads exist | Same-thread inline dispatch for bytes that are already waiting |
48
- | Native extensions | 1 (Ragel HTTP/1 parser + MiniSSL) | 3, all Ractor-safe (Ragel HTTP/1 parser; HTTP/2 parser + HPACK; `writev`, `sched_setaffinity`, `prctl` wrappers) |
49
- | Shared state (worker↔master) | Pipes and signals | Anonymous shared-memory `mmap` region |
50
- | Restart primitives | Phased (USR1), hot (USR2 re-exec, inherits FDs via env), refork (SIGURG) | Phased (USR1), hot (USR2 re-exec, inherits FDs via env), refork (SIGURG) |
51
- | systemd integration | `sd_notify` via plugin, `LISTEN_FDS` via binder | Native `sd_notify` + `LISTEN_FDS` socket activation |
52
-
53
- The rest of the document expands on each row.
54
-
55
- ## Part I: Puma
56
-
57
- ### Process model
58
-
59
- Puma is bimodal. In single mode the CLI runs a single `Puma::Server` in the current process; in cluster mode the CLI runs a master process that forks N workers, each running its own `Puma::Server`. Cluster mode is the interesting one because it is what almost everyone uses in production, and since Puma 5 it pre-forks by default; with two or more workers, `preload_app` is enabled unless explicitly turned off. The app is loaded once in the master, then N workers are forked from it, which preserves copy-on-write memory. If your app boots threads during load, copy-on-write breaks; every mutation to a page in the child dirties it and forces a private copy. Puma tries to detect this and warns.
60
-
61
- Puma also has a `fork_worker` option, which is Puma's answer to CoW decay over time. Worker 0 becomes a "fork server", spawning new workers from itself rather than from the master, so state built up in worker 0 (JIT caches, initialised gems, warm-up allocations) is shared with siblings via CoW. This is triggered manually with SIGURG or automatically after a request threshold. It is a clever workaround for a real problem, but it also mixes the "master supervises workers" and "worker services requests" concerns.
62
-
63
- Workers write their vitals (pid, boot state, timestamp) into a per-worker pipe read by the master. If a worker misses its `worker_check_interval` for longer than `worker_timeout` the master kills it and forks a replacement. Booting workers get a separate `worker_boot_timeout`.
64
-
65
- Signals: INT and TERM start a graceful shutdown, USR1 does a phased restart (increment phase counter, replace workers one at a time waiting for each new worker to boot), USR2 does a hot restart (re-exec the master with the same argv), TTIN and TTOU adjust the worker count, HUP reopens logs, and URG triggers a fork-worker refork.
66
-
67
- ### Threading model
68
-
69
- Inside a worker, request work is handled by `Puma::ThreadPool`. The pool has a minimum and maximum thread count, and it autoscales; when work arrives and there are more items queued than there are waiting threads, it spawns a new thread up to the max. Work sits in Ruby's thread-safe `Queue`, while a pool-level `Mutex` protects enqueue/dequeue coordination, thread counts, autoscaling, trimming, and shutdown. A `ConditionVariable` parks idle threads. Adding work signals the condvar; a waiting thread wakes, dequeues an item, and processes it.
70
-
71
- This is a textbook thread pool. It works, and it has for a decade. But it has three characteristics worth noting for the comparison:
72
-
73
- 1. **Every enqueue and every dequeue takes the mutex.** Under heavy load, the mutex becomes a serialisation point. It is not the dominant cost, but it is a cost.
74
- 2. **The autoscaling is not free.** Spawning and reaping OS threads on demand means allocations, thread startup, and reaping accounting that all cost CPU.
75
- 3. **`busy_threads`, `waiting`, and `pool_capacity` are computed by reading multiple fields inside the mutex.** Fine, but again: mutex.
76
-
77
- The pool exposes `busy_threads`, `waiting`, and `pool_capacity` metrics that the accept loop reads to decide whether to keep accepting or apply back-off. In cluster mode there is also an `accept_loop_delay` that sleeps proportionally to the busy ratio, which prevents a thundering herd where every worker accepts every connection.
78
-
79
- ### I/O model
80
-
81
- Puma has two threads doing I/O per worker: a **server thread** running the accept loop, and a **reactor thread** running `NIO::Selector`.
82
-
83
- The server thread is straightforward. It calls `IO.select` on all listener sockets (no timeout by default, so it blocks until one is readable), then `accept_nonblock` on any that are, wraps the resulting socket in a `Puma::Client` object, and pushes the client into the thread pool. That is all it does. Every accepted client is handed off to the thread pool, even if the request would have been readable immediately.
84
-
85
- What happens next depends on `queue_requests`. `queue_requests` defaults to true and is the mode almost everyone runs in. When it is true, a thread-pool worker picks up the client and calls `client.eagerly_finish`, which loops non-blocking reads while bytes are already buffered, parsing after each. If the request is complete by the time the socket runs dry, the worker proceeds to invoke the Rack app inline. If it is not complete, the worker hands the client to the reactor and returns to the pool. When `queue_requests` is false, the worker skips the eagerly_finish attempt and does a blocking `client.finish(first_data_timeout)`. The `queue_requests: false` path is fine for fast trusted clients but does not scale; a slow client will hold a thread indefinitely.
86
-
87
- The reactor is a separate thread running `Puma::Reactor`, which wraps an `NIO::Selector` from `nio4r`. Its job is to babysit two kinds of sockets:
88
-
89
- 1. **Sockets in the middle of a request.** A client sent some headers, was not yet complete, and needs more data before the app can be called.
90
- 2. **Keep-alive sockets between requests.** The previous request finished, the connection is still alive, and we are waiting for the next request to start.
91
-
92
- The reactor stores clients in a data structure keyed by timeout, and every time through the loop it computes the deadline of the earliest-expiring client, calls `selector.select(timeout)`, and then dispatches. Any client whose socket is readable calls `try_to_finish`, which does a `read_nonblock` and attempts to parse whatever it got. If the request is now complete, the client is handed to the thread pool. If the socket is not ready or the request is still incomplete, the client goes back to sleep in the reactor.
93
-
94
- Puma stores its reactor timeouts in a Ruby array of `Client` objects. New clients are pushed onto the end and the array is re-sorted with `sort_by!(&:timeout_at)` after each batch of inserts. Deletion is a linear scan (`@timeouts.delete client`). That is fine for small numbers of clients but scales linearly in the number of connections. It is not a hot spot at moderate load, but the cost is there.
95
-
96
- Cross-thread waking uses `NIO::Selector#wakeup`. When work needs to enter the reactor from another thread (a Client being registered), the client is pushed onto a Ruby `Queue` and the selector is woken; the selector drains the queue and gets on with it.
97
-
98
- ### HTTP/1.1 request lifecycle
99
-
100
- Putting the pieces together, a request through Puma looks like this:
101
-
102
- 1. Server thread `IO.select` returns a readable listener.
103
- 2. Server thread `accept_nonblock` produces a new socket. Puma wraps it in a `Client` and pushes it to the thread pool.
104
- 3. A thread-pool worker picks up the client and calls `client.eagerly_finish`, which loops `read_nonblock` while bytes are already buffered and tries to parse after each.
105
- 4. If the request is complete after that read, the worker calls `handle_request(client)` inline, which invokes the Rack app, formats the response, writes it back.
106
- 5. If the request is not complete, the worker hands the client to the reactor and returns to the pool. The reactor waits for more bytes, retries the parse, and pushes the client back to the thread pool once complete.
107
- 6. After the response is written, the client is either closed or, if keep-alive, kept inline on the same thread (if `has_back_to_back_requests?` is true, or `eagerly_finish` on the next request returns true and there is a spare thread), pushed back to the thread pool, or re-added to the reactor with `@persistent_timeout` as the new deadline.
108
-
109
- Puma's parser is a Ragel-generated FSM in C. It is invoked from Ruby via `Puma::HttpParser#execute(env, buffer, offset)`. The parser calls back into Ruby via a set of C functions that set entries in the env hash: `REQUEST_METHOD`, `REQUEST_URI`, `PATH_INFO`, `QUERY_STRING`, `SERVER_PROTOCOL`, and one `HTTP_*` entry per header. It also detects the end of headers and reports where the body starts. Everything after that (body reading, chunked decoding, spooling large bodies to Tempfile) is pure Ruby in `Puma::Client`.
110
-
111
- Response writing is done from the app thread. Puma sets `TCP_CORK` on Linux, writes the status line and headers, writes the body (chunked, Content-Length, or via `IO.copy_stream` for File bodies to trigger sendfile), unsets `TCP_CORK`, and calls `Rack::BodyProxy#close` and any `rack.after_reply` hooks.
112
-
113
- ### Puma request flow diagram
40
+ This diagram shows the normal queued HTTP/1.1 path in cluster mode. The keep-alive branch is simplified. Ready requests may continue inline or be queued again.
114
41
 
115
42
  ```mermaid
116
43
  flowchart TB
@@ -135,10 +62,10 @@ flowchart TB
135
62
 
136
63
  STA["Stat thread<br/>writes to worker pipe<br/>on worker_check_interval"]
137
64
 
138
- C1{"Complete<br/>request<br/>on accept?"}
139
- C2{"Complete<br/>request<br/>after read?"}
65
+ C1{"Complete<br/>request<br/>after eager read?"}
66
+ C2{"Complete<br/>request<br/>after read and parse?"}
140
67
  KA{"Keep-alive?"}
141
- BB{"Back-to-back<br/>data in buffer?"}
68
+ BB{"Next request<br/>ready?"}
142
69
 
143
70
  SRV -->|"push new socket"| TP
144
71
  TP -->|"eagerly_finish"| C1
@@ -147,7 +74,7 @@ flowchart TB
147
74
  RCT -->|"socket readable"| C2
148
75
  C2 -->|"complete, push"| TP
149
76
  C2 -->|"incomplete, keep waiting"| RCT
150
- RCT -->|"timeout"| TO["write 408, close"]
77
+ RCT -->|"timeout"| TO["timeout handling, close"]
151
78
  APP -->|"write response"| KA
152
79
  KA -->|"no"| CLS["close socket"]
153
80
  KA -->|"yes"| BB
@@ -171,189 +98,24 @@ flowchart TB
171
98
  class PIPE storage
172
99
  ```
173
100
 
174
- Notably, the parser runs inside an app thread. Puma's concurrency model is "many app threads, each doing one full request (parse plus app plus write) at a time". Under MRI's GVL this is genuine concurrency but not parallelism; at any given moment at most one of those threads is executing Ruby, and the others are either blocked on I/O (which releases the GVL and lets another thread proceed) or waiting their turn. Puma leans on the fact that most Rack apps spend most of their time blocked on the database or an external API, where threads do effectively overlap.
175
-
176
- ## Part II: Raptor
177
-
178
- Raptor takes a different position on nearly every axis. It is opinionated in a way that Puma is not, largely because it does not have to support every deployment scenario Puma does.
179
-
180
- ### Process model
181
-
182
- There is no single mode. Raptor is always a cluster. A master forks N workers, monitors them, and restarts crashed workers. The Rack app is always loaded in the master before forking, so copy-on-write is preserved by default (no user-visible `preload_app` knob).
183
-
184
- The process and app-thread counts can be set with `RAPTOR_WORKERS`, `RAPTOR_THREADS`, and `RAPTOR_MAX_THREADS`, which lets a deployment tune them without generating a config file. Config files override the built-in defaults, environment variables override config files, and command-line options override both. `RAPTOR_MAX_THREADS=unlimited` leaves adaptive growth uncapped.
185
-
186
- The master is a supervisor. It never handles requests. It forks workers, watches them via a shared-memory region (more on that in a moment), traps signals, restarts crashed workers, and orchestrates restarts.
187
-
188
- Worker boot and shutdown hooks receive the worker's slot index, so setup and cleanup can identify the same slot across restarts. Hooks that do not take an argument continue to work.
189
-
190
- Two kinds of restart are supported:
191
-
192
- 1. **Phased restart on SIGUSR1.** Same idea as Puma. Kill each worker in sequence, wait for its replacement to boot, move on. Existing workers drain their connections while their replacements come up. This is cheap and safe when the change does not require a fresh master.
193
-
194
- 2. **Hot restart on SIGUSR2.** Also known as "zero-downtime restart with FD inheritance". The master clears the close-on-exec flag on every listener, JSON-encodes the map from bind URI to file descriptor into an environment variable (`RAPTOR_INHERITED_FDS`), and re-execs itself with the original command line. The new master reads the environment variable and rebuilds its `Binder` from the inherited FDs instead of binding fresh sockets. Not a single connection is dropped, and the new master runs its initialisation from scratch (loading a newer Rack app, applying new config, whatever).
195
-
196
- Systemd socket activation is a native feature and slots straight into this model. When the service unit is `Type=notify` and there is a socket unit, systemd passes listener FDs via `LISTEN_FDS`. Raptor detects this exactly the same way it detects a hot restart handoff: `Systemd.listen_fds` returns the FDs, the binder is built from them, and the master sends `READY=1` back to systemd once workers have booted. `STOPPING=1` and `RELOADING=1` fire on the corresponding lifecycle events.
197
-
198
- Routine worker monitoring does not use pipes. Every worker writes its stats (pid, request count, backlog, busy and available threads, last checkin timestamp, booted flag) into a fixed-size slot in an anonymous shared-memory region allocated with `mmap-ruby` before the fork. The master reads the region directly. There is no serialisation, pipe drain, or signal to trigger the read; it is 49 bytes per worker of native memory. `bundle exec raptor stats` prints a JSON snapshot. Refork coordination is separate and does use a pair of pipes between the master and seed.
199
-
200
- On Linux, `cpu_affinity: true` pins each worker to a distinct CPU via `sched_setaffinity` when the worker count fits within the process's allowed CPU set, so it stays on one core and its L1/L2 caches stay warm. It is off by default because an allowed CPU in a container is not necessarily dedicated to that container. When workers outnumber available CPUs the pin is skipped and the kernel scheduler manages placement.
201
-
202
- ### Refork
203
-
204
- Long-lived workers accumulate mutations. Every allocation, every YJIT-compiled block, every gem that lazy-loads on first use, every cache that populates over the first few thousand requests, all of it dirties pages that were shared copy-on-write with the master. After a few hours of traffic the CoW savings are mostly gone and each worker's RSS climbs toward its steady-state size. Reforking off a warmed source is the standard answer.
205
-
206
- Raptor's version is driven by `refork_after`, an `Integer`, `Array<Integer>`, or `nil`. When any worker's request count crosses the next threshold, the master picks the most-experienced booted worker in the current phase and sends it `SIGURG`. That worker breaks out of its main loop, runs any `before_refork` hooks (typically to close database connections and any other file descriptors the fresh workers should not inherit), drains in-flight requests, and then transitions into a seed loop instead of exiting. From that point on the seed's only job is to fork replacement workers on the master's request; it never accepts another connection. The master then rolls the remaining workers through a phased restart, forking each replacement from the seed.
207
-
208
- Successive thresholds promote fresh seeds. When the next threshold trips, the master picks a new candidate from the currently-serving generation, terminates the previous seed, and starts the same drain-and-promote cycle on the new candidate. Each generation's fork source is therefore a worker that has served enough traffic to warm its VM, its YJIT caches, and its lazy-loaded gems, and the previous generation's seed is retired the moment it is no longer needed. Passing an array to `refork_after` (e.g. `[1000, 5000, 20_000]`) sets successive thresholds; the last one repeats.
209
-
210
- Master and seed communicate over two pipes opened before the initial fork. The fork pipe (`@fork_r`, `@fork_w`) carries an 8-byte packed `[slot_index, phase]` from master to seed for each replacement to fork. The response pipe (`@resp_r`, `@resp_w`) carries a 4-byte packed pid back the other way, plus a one-shot 4-byte zero the seed writes when it enters its loop so the master knows drain is complete. The master polls the ready marker non-blockingly on the supervision loop; the phased restart that fills the vacated slot and cycles the remaining workers only starts once the marker arrives (or the drain deadline expires, in which case the master clears the seed reference and falls back to direct forks).
211
-
212
- Workers forked by the seed are not the master's direct children. Before spawning anything the master calls `PR_SET_CHILD_SUBREAPER` (via a small `prctl` wrapper, `Raptor::Subreaper`), so seed-forked children reparent to the master immediately and `Process.wait2` sees them like any other child. When a seed is retired on the next promotion its still-running descendants stay attached to the master through the same subreaper bit. Refork depends on this. On platforms without `PR_SET_CHILD_SUBREAPER` (everything except Linux), `refork_after` defaults to `nil` and is ignored with a warning if the user sets it anyway.
213
-
214
- The design is Pitchfork's, adapted for Raptor's process model. [Pitchfork](https://github.com/Shopify/pitchfork) was the first Ruby server built around `PR_SET_CHILD_SUBREAPER` and multi-generation mould promotion; Puma's shipping `fork_worker` keeps worker 0 serving traffic alongside its fork-server duties, and Instacart's [`mold_worker` PR #3643](https://github.com/puma/puma/pull/3643) is Puma catching up to the same shape Pitchfork established. Raptor's seed matches the Pitchfork shape: pure fork source, promoted from a warm worker, retired when the next generation takes over.
215
-
216
- ### Threading model
217
-
218
- This is where Raptor diverges dramatically. Inside a worker there are several distinct layers of concurrent activity:
219
-
220
- 1. **One server thread** running the accept loop.
221
- 2. **One reactor thread** running the NIO event loop plus timeout tree.
222
- 3. **A `RactorPool` for HTTP/1.1 parsing** sized by `http1.ractors`, defaulting to `round(cores / workers)` clamped to `[1, 3]`.
223
- 4. **A `RactorPool` for HTTP/2 parsing** sized by `http2.ractors`, defaulting to `round(cores / workers)` clamped to `[1, 2]`.
224
- 5. **A collector thread per Ractor pool** that receives parsed results via a `Ractor::Port`.
225
- 6. **An `AtomicThreadPool` of T baseline app threads** running the Rack app and writing responses, with optional adaptive growth.
226
- 7. **One stats thread** that writes the shared-memory slot every second.
227
- 8. **One load reporter thread when BPF dispatch is active** that publishes backlog to the kernel map.
228
-
229
- That is a lot of moving parts. Let us go through why.
230
-
231
- **Why Ractors for parsing.** Ractors are Ruby's answer to true parallelism. Multiple Ractors can execute Ruby code simultaneously on different OS threads, each with its own GVL. But Ractors are heavily restricted. Shared objects must satisfy Ractor's shareability rules, most global mutable state is inaccessible, and many existing gems assume shared-state semantics that are incompatible with isolation.
232
-
233
- For a web server, this restriction turns out to be almost exactly right for HTTP parsing. Parsing a request is CPU-bound (tokenising bytes, uppercasing header names, decoding chunked bodies), it does not need to touch any global state, and it produces a result (a hash) that can be safely frozen and handed off. The native HTTP/1 parser (`raptor_http.c`) is declared `rb_ext_ractor_safe(true)`; it holds no per-parser Ruby state in the extension itself, and it writes only into the caller-supplied env hash. Same for the HTTP/2 parser plus HPACK.
234
-
235
- The HTTP/1 parser also pre-interns the ~40 most common header keys (`HTTP_HOST`, `HTTP_USER_AGENT`, the `HTTP_ACCEPT_*` family, `CONTENT_LENGTH`, `HTTP_X_FORWARDED_*`, the `HTTP_SEC_FETCH_*` client hints, and so on) once at load time. During parsing, a `memcmp` lookup against that table returns the shared frozen `VALUE` for known keys and falls back to `rb_enc_interned_str` for the rest. Every request's env hash therefore reuses the same String object for its header names, which both skips per-request allocation and lets Ruby's hash lookup use the interned key's cached hash code.
236
-
237
- Your Rack app still runs on ordinary threads under one worker GVL, so the app does not need to be Ractor-safe. Requests that need the reactor pipeline can have their protocol work run in parallel in another Ractor. Complete requests found by the eager accept and keep-alive paths parse inline instead, avoiding a Ractor handoff when the bytes are already available.
238
-
239
- **How the Ractor pools actually work.** Raptor uses the `ractor-pool` gem, which is another one of my libraries. Each pool has one coordinator Ractor and M pipeline Ractors. When a pipeline Ractor is idle, it sends itself back to the coordinator via `coordinator.send(Ractor.current, move: true)`. When work arrives at the coordinator, it either forwards it to a waiting Ractor (if any) or queues it. This coordinator-dispatch pattern guarantees that no Ractor sits idle while there is work. Results flow back through a shared `Ractor::Port` (a many-to-one channel added in recent Ruby versions and stable in 4.0) to a Ruby-side collector thread. If `M == 1` the coordinator is skipped and work goes straight to the single pipeline Ractor.
240
-
241
- Raptor always runs an HTTP/1.1 pool because TCP, Unix, and SSL bindings support HTTP/1.1. It starts an HTTP/2 pool when an SSL or cleartext HTTP/2 binding is present. SSL bindings need both pools because ALPN allows each client to choose HTTP/2 or HTTP/1.1, while `h2c://` bindings accept only HTTP/2 clients using prior knowledge. Both pool sizes scale with headroom via `round(cores / workers)`, clamped to `[1, 3]` for `http1.ractors` and `[1, 2]` for `http2.ractors`. Keeping the pools separate means h1 and h2 parsing never share ractor slots, so a burst of small HTTP/1.1 requests cannot delay HTTP/2 frame handling on the same connection, and vice versa.
101
+ ## How Raptor moves a request
242
102
 
243
- **Why a custom thread pool.** The `AtomicThreadPool` in `atomic-ruby` (another one of my libraries) is backed by an `AtomicQueue`. The queue is a Michael-Scott multi-producer, multi-consumer FIFO: a singly linked list with a dummy sentinel and atomic head and tail pointers. Producers append nodes at the tail; consumers advance the head. Both operations are O(1) and make progress through compare-and-swap rather than a queue-wide mutex. Separate atoms track queue size and active app threads for backpressure.
103
+ Raptor always preloads the app and forks workers. Each worker has an accept thread, a reactor, protocol Ractors and their collector threads, an app thread pool, and a stats thread. HTTP/2 bindings add a separate HTTP/2 protocol pool.
244
104
 
245
- The pool starts at `threads` and scales without a fixed limit by default. Set `max_threads` to cap its growth, or set it to the same value as `threads` to keep the pool fixed. A native CRuby thread-event hook measures how much active workers spend running, blocked outside the GVL, and waiting to acquire it. The pool only adds a temporary thread after work has remained queued across several samples, every current worker is active, blocked time is above half of measured worker time, and GVL wait is below two percent. That last check matters: adding threads helps when existing threads are asleep in database or network calls, but hurts when CPU-bound Ruby threads are already fighting over the GVL. Temporary threads retire after the queue has remained empty for a second.
105
+ There are two important request paths.
246
106
 
247
- The pool still uses an `AtomicConditionVariable` under the hood to park idle threads (idle threads call `Thread.stop` and get woken with `Thread#wakeup`; there is no spinning), because idle spinning would waste CPU. The difference from Puma's pool is not "no locks anywhere" but rather "the hot path (enqueue and dequeue when the queue has items) is lock-free". Once every worker is busy the mechanics look similar; where things diverge is under contention when you have many threads all trying to push and pop.
107
+ 1. **Eager handling.** On a plain HTTP/1.1 connection, the accept thread tries to read and parse the request immediately. A complete request goes straight to the app pool. TLS connections perform their handshake and initial protocol handling on an app thread instead.
108
+ 2. **Protocol pipeline.** If more bytes are needed, the reactor watches the socket and reads available data. A protocol Ractor parses the buffered state. A collector routes the result back to the reactor for more data or to the app pool for Rack execution.
248
109
 
249
- Application thread locals are cleared when each request finishes. Running every request in a fresh Fiber is independently configurable, but opt-in. The pool keeps its own parser and response buffers in private thread variables so the server can reuse them without preserving application state between requests.
110
+ Protocol Ractors can execute in parallel with the main Ractor. The Rack application cannot use that parallelism for its own Ruby execution. Its threads still share the main Ractor's GVL within each worker. Multiple worker processes provide application-level Ruby parallelism in both servers.
250
111
 
251
- The knock-on effect is that the server thread can read `pool.queue_size + pool.active_count` on every accept-loop iteration without acquiring the queue's mutation lock. Those are still synchronised atomic reads, but they do not serialise producers and consumers behind one mutex.
112
+ Raptor does not send every request through a Ractor. After an HTTP/1.1 response, an app thread checks for the next request without waiting. It can parse ready bytes inline and yield to queued work when needed. If the connection is idle, it returns to the reactor. When that connection becomes readable, the reactor first sends it back to an app thread for another eager attempt. Only an incomplete request needs the longer parsing pipeline.
252
113
 
253
- ### I/O model
114
+ The distinction matters. Separate parsing can overlap protocol work with Rack execution, but crossing Ractor and collector boundaries also costs work. Eager paths avoid those handoffs. The benchmark does not isolate either path's contribution.
254
115
 
255
- The server accept loop is an `IO.select` + `accept_nonblock` loop, similar to Puma. What is different is the pair of checks right before `accept`:
116
+ ### Raptor request flow
256
117
 
257
- ```ruby
258
- backpressure_threshold = [(@thread_pool.size * BACKPRESSURE_THRESHOLD_MULTIPLIER).ceil, MIN_BACKPRESSURE_THRESHOLD].max
259
- # ...
260
- next if @reactor.backlog >= backpressure_threshold
261
-
262
- if @thread_pool.queue_size > @thread_pool.size
263
- Thread.pass
264
- next
265
- end
266
- ```
267
-
268
- The first is a hard skip. `@reactor.backlog` is `thread_pool.queue_size + thread_pool.active_count`; when the total load reaches the threshold, this worker stops accepting until it drains. `MIN_BACKPRESSURE_THRESHOLD` is 8, so a three-thread pool trips at 8 concurrent items rather than 4. The floor avoids overreacting to a few active requests while still bounding the amount of work a worker pulls from the kernel.
269
-
270
- The second is a softer yield. When the queue alone exceeds the pool size — the app threads are all busy and there's a queue building on top of them — the accept loop yields the GVL via `Thread.pass` and re-checks the queue on the next iteration instead of accepting more work. This lets the app threads make progress before the server thread grabs another connection, and only fires under real pool pressure (queue depth greater than pool size), so IO-bound workloads where threads spend most of their time in `sleep` and rarely queue past the pool size aren't affected.
271
-
272
- On Linux, Raptor can replace the shared TCP listener with one `SO_REUSEPORT` listener per worker and attach a small BPF program to the group. A reporter thread publishes each worker's backlog into a kernel map every millisecond. For each new connection, the program hashes to two distinct workers and selects the one with the lower reported load: the power-of-two-choices strategy. It then atomically increments that worker's map slot before routing the connection, reserving capacity immediately rather than waiting for the next reporter tick. The accept path also publishes `reactor.backlog + 1` as soon as Ruby receives the socket. These reservations stop a burst of connections from repeatedly choosing the same stale minimum while retaining hash-based spread across the cluster.
273
-
274
- This path only wraps plain `tcp://` bindings. TLS listeners and non-TCP bindings remain shared listeners inherited from the master. If `libbpf-ruby` or the compiled BPF object is unavailable, Raptor silently uses those shared listeners for TCP too; if the prerequisites exist but the kernel refuses the program, startup raises. In either mode, a worker under Raptor's explicit backpressure stops competing for new accepts until its current work drains.
275
-
276
- The BPF-based approach was inspired by [a comment](https://github.com/puma/puma/issues/3934#issuecomment-4356462590) by John Hawthorn ([@jhawthorn](https://github.com/jhawthorn)) on a Puma issue about `EPOLLEXCLUSIVE`, where he floated `SO_ATTACH_REUSEPORT_EBPF` as a way to route each connection to the least-busy worker.
277
-
278
- The reactor is again an `NIO::Selector` loop. Two things make it different from Puma's:
279
-
280
- 1. **Read strategy.** When a socket is readable, the reactor does one `read_nonblock(64KB)` in the reactor thread, updates the buffered state, makes that state shareable, and sends it to the protocol's Ractor pool. The Ractor decides whether the request or frame batch is complete. The collector then sends incomplete state back to the reactor or dispatches completed requests to the app pool. The reactor itself does I/O, not protocol parsing.
281
-
282
- 2. **Timeout data structure.** Instead of a sorted linked list, timeouts are stored in a red-black tree (`red-black-tree` gem, yes, also one of mine). Each connection is represented by a `TimeoutClient < RedBlackTree::Node` ordered by its `timeout_at` value. Insertion is O(log n), deletion by key (needed when a connection's timeout is updated mid-flight, which happens on every read) is O(log n), and in-order traversal is O(k) where k is the number of expired connections. After every selector poll, the reactor walks the tree in order and breaks on the first non-expired node.
283
-
284
- Why a tree instead of a heap? A min-heap gives you O(1) peek and O(log n) insertion, but deleting an arbitrary node (or updating one) is O(n) because you have to find the node first. Since every read on a connection resets its timeout deadline, and every response completion removes a connection from the reactor, you get a lot of "update this specific node" and "remove this specific node" operations. The tree makes those O(log n). It is a small thing, but the reactor manages hundreds to thousands of connections in a busy worker, and the difference between O(log n) and O(n) matters at that scale.
285
-
286
- Three timeout classes are tracked:
287
-
288
- - `first_data_timeout` (30s): applied to a fresh connection with no bytes read yet.
289
- - `chunk_data_timeout` (10s): applied once data has started arriving but the request is incomplete.
290
- - `persistent_data_timeout` (65s): applied to a keep-alive socket sitting idle between requests.
291
-
292
- An incomplete HTTP/1.1 request receives `408 Request Timeout` before the connection closes. Once an HTTP/2 connection has received its preface, its deadline instead sends a PING; an acknowledgement restores the idle deadline, while a missing acknowledgement closes the connection.
293
-
294
- ### HTTP/1.1 request lifecycle
295
-
296
- An HTTP/1.1 request takes one of two paths. The **fast path** fires when the first `read_nonblock` on the accepted socket produces the complete request. `eager_accept` on the server thread reads, parses the request inline, and pushes it straight to the thread pool, bypassing both the reactor and the Ractor pool. An app thread then builds the Rack env, calls the app, and writes the response.
297
-
298
- The **pipeline path** fires when the first read returns `WaitReadable` (bytes have not arrived yet) or leaves the request incomplete. It goes:
299
-
300
- 1. `eager_accept` calls `@reactor.add(state)` with the socket, an empty buffer, and metadata (remote address, URL scheme).
301
- 2. Reactor thread registers the socket with the NIO selector and adds a `TimeoutClient` node to the RBT with `first_data_timeout`.
302
- 3. Bytes arrive. Selector wakes. Reactor calls `read_nonblock(64KB)`, appends to the state's buffer.
303
- 4. Reactor calls `Ractor.make_shareable(state)` and pushes the state onto the Ractor pool via `ractor_pool << shareable_state`.
304
- 5. Pipeline Ractor parses the buffer using the C extension. If the request has a chunked body, chunk decoding runs inline in the pipeline Ractor as a pure function. The parser returns a result including whether the request is complete.
305
- 6. Result travels through `Ractor::Port` to the collector thread.
306
- 7. Collector thread receives the parsed result. If incomplete, state goes back into the reactor to await more data. If complete, the socket is deregistered from the reactor and a proc is pushed to the `AtomicThreadPool`.
307
- 8. An app thread pops the proc, builds a Rack env, calls `@app.call(env)`, and writes the response.
308
- 9. If the response signals keep-alive (HTTP/1.1 default without `Connection: close`), the app thread enters the **eager keep-alive loop**.
309
-
310
- The eager keep-alive loop is one of Raptor's more deliberate latency/occupancy trade-offs. Rather than immediately returning the connection to the reactor after a response, the app thread does:
311
-
312
- ```ruby
313
- loop do
314
- unless socket.wait_readable(0)
315
- reactor.persist(socket, id, request_count, ...)
316
- return
317
- end
318
- # There are bytes. Try to parse the next request inline.
319
- ...
320
- end
321
- ```
322
-
323
- The thread checks for bytes without waiting. If they are already available, it parses them inline and calls the Rack app again. Otherwise the connection returns to the reactor immediately. Pipelined requests avoid a reactor round-trip without letting an idle keep-alive connection occupy an app thread.
324
-
325
- Puma has a similar shape. It checks buffered back-to-back requests, then eagerly drains bytes already available on the socket. If a complete request is ready and the pool has a waiting thread, the current thread loops inline; otherwise Puma queues the client or returns it to the reactor.
326
-
327
- The `reactor.persist` call re-registers the socket with the reactor using `persistent_data_timeout` (65s) as the new deadline. When the next bytes arrive, the reactor treats the socket like any other partially-read connection.
328
-
329
- A `Raptor::DetachedBody` returns the application thread while keeping the response open, chunked on HTTP/1.1 and ended by closing the connection on HTTP/1.0. The reactor owns the response until it closes, buffers later writes within fixed limits, and pauses the connection's next request without occupying an application thread.
330
-
331
- ### HTTP/2 request lifecycle
332
-
333
- Raptor speaks HTTP/2 on TLS connections where the client negotiates it via ALPN and on `h2c://` listeners for cleartext clients using prior knowledge. The binder sets `alpn_protocols = ["h2", "http/1.1"]` on the SSL context and the ALPN callback picks h2 whenever the client offers it. Puma does not do this. Puma's SSL context does not advertise `h2` in ALPN, so clients transparently fall back to HTTP/1.1.
334
-
335
- Once ALPN selects h2 or an `h2c://` listener accepts a connection, the server calls `Http2#eager_accept`, which creates the per-connection `Writer` and `FlowControl`, attaches them to the reactor, writes the initial `SETTINGS` frame, and tries a non-blocking read on the socket. If bytes are already there, it parses the first frame batch inline via `Http2.process_frames` and dispatches completed streams to the thread pool without ever touching the ractor pool. If not, it hands the socket to the reactor to watch for the first bytes.
336
-
337
- From there the shape is similar to HTTP/1.1:
338
-
339
- 1. Reactor reads frames.
340
- 2. The HTTP/2 parser (native C, with an HPACK decoder using a static Huffman table) parses the frames in the HTTP/2 Ractor pool.
341
- 3. Completed requests (once `HEADERS` and `DATA` are complete for a stream) go to the thread pool as separate work items. **A single connection can be servicing many streams in parallel across the thread pool.**
342
- 4. Each stream's response frames are queued for the reactor, which owns every socket write after connection setup.
343
-
344
- Responses to `HEAD` requests and statuses that prohibit a message body end with the response `HEADERS` frame.
345
- Early hints and response-finished callbacks follow the same Rack lifecycle as HTTP/1.1.
346
- Trailing request `HEADERS` complete an open request. Rack has no standard request-trailer key, so Raptor validates them without adding them to the environment.
347
-
348
- The `Writer` hands serialized frames to the reactor, which writes as the socket becomes ready and closes clients that stop reading. Application threads never wait for socket writability, and the connection has a single I/O owner without a per-connection mutex.
349
-
350
- Flow control uses similar CAS-protected atoms. Ordinary Rack bodies wait for connection and stream capacity as they yield. A `Raptor::DetachedBody` instead returns its application thread immediately; the reactor schedules its bounded buffer as capacity becomes available and closes it when the client cancels. This keeps long-lived streams from consuming one application thread each.
351
-
352
- Frame processing also has an eager loop. After processing one batch of frames, the h2 handler checks `wait_readable(0)` and reads the next batch only if it has already arrived. Up to eight rounds are consumed inline before handing back to the reactor, and the loop bails out early once the app thread pool has more queued work than worker slots so one busy connection cannot starve the collector. This is the same principle as the HTTP/1.1 eager keep-alive: amortise the reactor round-trip when the client is actively sending, but back off under saturation.
353
-
354
- During worker shutdown, Raptor stops accepting connections and sends GOAWAY with the last stream handed to the Rack application. Later streams are refused while application work and detached bodies drain. The reactor remains active so in-flight responses can receive flow-control updates; detached bodies still open when the drain period expires are cancelled before their connections close.
355
-
356
- ### Raptor request flow diagram
118
+ The eager accept arrows below show plain connections. TLS setup takes an app-thread detour before protocol dispatch. HTTP/2 response writes return to the reactor. Ordinary HTTP/1.1 responses are written by app threads.
357
119
 
358
120
  ```mermaid
359
121
  flowchart TB
@@ -361,11 +123,11 @@ flowchart TB
361
123
  MM["Master loop<br/>Watches shared mmap for<br/>per-worker stats<br/>Handles signals"]
362
124
  MM -->|"fork"| WK1["Worker 1"]
363
125
  MM -->|"fork"| WKN["Worker N"]
364
- MM -.->|"SIGUSR2<br/>exec + LISTEN_FDS"| MM
126
+ MM -.->|"SIGUSR2<br/>exec + inherited listeners"| MM
365
127
  end
366
128
 
367
129
  subgraph Worker["Raptor worker process, one of N"]
368
- SRV["Server thread<br/>IO.select + accept_nonblock<br/>backpressure check<br/>reactor.backlog vs pool_size"]
130
+ SRV["Server thread<br/>IO.select + accept_nonblock<br/>polls for pool capacity"]
369
131
 
370
132
  RCT["Reactor thread<br/>NIO::Selector<br/>read_nonblock 64KB<br/>Red-Black-Tree timeouts"]
371
133
 
@@ -391,7 +153,7 @@ flowchart TB
391
153
 
392
154
  CHK{"Request<br/>complete?"}
393
155
  KA{"Keep-alive?"}
394
- EAG{"wait_readable(0)<br/>bytes ready?"}
156
+ EAG{"Buffered or<br/>readable bytes?"}
395
157
 
396
158
  SRV -->|"HTTP/1.1 eager_accept, parse inline, push proc"| ATP
397
159
  SRV -->|"HTTP/2 eager_accept, parse inline, push proc"| ATP
@@ -405,10 +167,12 @@ flowchart TB
405
167
  CHK -->|"yes, push proc"| ATP
406
168
  ATP -->|"HTTP/2 response frames"| RCT
407
169
  ATP -->|"HTTP/1.1 response write"| KA
408
- KA -->|"no, close"| CLS["close socket"]
170
+ KA -->|"no"| CLS["close socket"]
409
171
  KA -->|"yes"| EAG
410
- EAG -.->|"bytes ready, parse+dispatch on same thread"| ATP
172
+ EAG -.->|"bytes ready, attempt inline parse"| ATP
173
+ ATP -->|"HTTP/1.1 incomplete request"| RCT
411
174
  EAG -->|"no bytes, reactor.persist"| RCT
175
+ RCT -.->|"keep-alive bytes ready, push proc"| ATP
412
176
  RCT -->|"deadline expired"| TO["PING or close"]
413
177
 
414
178
  STA -.->|"writes slot"| SHM[("mmap shared memory")]
@@ -431,197 +195,113 @@ flowchart TB
431
195
  class SHM storage
432
196
  ```
433
197
 
434
- The critical structural difference from Puma is that Raptor has a separate protocol pipeline for connections that need more I/O. The reactor reads, a Ractor parses, a collector routes the result, and an app thread runs Rack and writes the response. Raptor's eager paths collapse that machinery when a complete request is already available. This is a hybrid rather than a rule that every request must cross a Ractor.
435
-
436
- ## Part III: Head to head
437
-
438
- ### Parsing model
439
-
440
- **Puma.** Parsing happens on an app thread. The C parser callbacks build the env hash. A fresh client first enters the thread pool, where eager reads may complete it immediately; partial and idle keep-alive connections wait in the reactor before returning to the pool. Parsing shares the worker's GVL with the app.
441
-
442
- **Raptor.** Fresh and immediate keep-alive requests parse inline on the server or app thread. A connection that needs more bytes takes the longer path: reactor (I/O) → protocol Ractor pool (parse) → collector → app thread pool (Rack + write). Between keep-alive requests, the app thread checks for bytes without waiting before returning an idle connection to the reactor. Parsing in the Ractor pipeline has its own GVL; parsing on an eager path does not.
443
-
444
- In practice, Puma has one process-wide GVL per worker. Every Ruby thread inside that worker takes turns holding it. Raptor has the same main-Ractor GVL plus one GVL per protocol Ractor, so a pipeline Ractor can parse one connection while an app thread executes Rack for another. That parallelism is real, but so are the costs of making state shareable and crossing the Ractor and collector boundaries. Which side wins depends on how much work the request gives the protocol pipeline; the current CPU benchmark leaves Raptor and Puma close rather than proving a universal parsing advantage.
445
-
446
- ### Timeout management
447
-
448
- **Puma.** Ruby array re-sorted with `sort_by!` after each batch of inserts. Sort is O(n log n). Remove is a linear scan, O(n). Fine at small scale.
449
-
450
- **Raptor.** Red-black tree keyed by `timeout_at`. Insert O(log n), remove O(log n), in-order traversal breaks early on first non-expired node. Scales cleanly to thousands of connections.
198
+ ## Where the designs differ
451
199
 
452
- At moderate connection counts this is unlikely to dominate either server. The asymptotic difference becomes more relevant as a worker tracks more idle or partial connections and updates more individual deadlines.
200
+ ### Thread pools and admission control
453
201
 
454
- ### Work queue
202
+ Puma coordinates its work queue and pool bookkeeping with a mutex and condition variables. It can grow from a minimum to a maximum thread count as work arrives. Idle threads sleep.
455
203
 
456
- **Puma.** Ruby `Queue` plus a pool-level `Mutex` and `ConditionVariable`. Every enqueue takes the mutex and wakes a waiter; every dequeue happens while holding the mutex. The same critical section coordinates pool bookkeeping and autoscaling.
204
+ Raptor uses `AtomicThreadPool` from `atomic-ruby`. Enqueue and dequeue use a lock-free queue. Idle threads still park rather than spin. Its adaptive pool considers time spent executing Ruby, waiting for the GVL, and blocked outside it. Growth is intended to help blocking workloads without adding threads just because CPU-bound Ruby work is queued. The default has no upper limit, so deployments should choose a cap where resource limits require one.
457
205
 
458
- **Raptor.** A Michael-Scott FIFO with atomic head and tail pointers and a dummy sentinel node. Producers link at the tail and consumers advance the head using CAS. Queue size and active-thread counts live in separate atoms.
206
+ Raptor also checks queued and active work before accepting more connections. Under pressure, the accept loop yields and polls for capacity rather than parking until notified. This can consume CPU while the worker is saturated. It limits new accepts, not requests arriving on existing keep-alive connections. It is not a guarantee of progress or fairness for every client.
459
207
 
460
- The condvar is still there for parking idle threads (spinning would burn CPU), but the hot path when the queue has items is lock-free.
208
+ Puma's cluster accept delay backs off according to worker load. On supported Linux plain TCP bindings, Raptor can instead use BPF reuseport dispatch to choose the less loaded of two sampled workers. Its load estimate includes pending accepts. TLS and Unix sockets use shared listeners. Missing BPF prerequisites also leave Raptor on shared listeners, while a kernel rejection of the BPF program fails startup.
461
209
 
462
- Under moderate load, queue mechanics are unlikely to dominate either server. Under contention, Raptor avoids one queue-wide mutex and can read its backpressure counters without locking producers or consumers. That is a narrower claim than saying a lock-free queue is always faster: CAS retries and cache-line traffic still have costs.
210
+ ### Connection tracking
463
211
 
464
- ### Keep-alive fast path
212
+ Both reactors keep idle and partial connections off application threads. Puma tracks deadlines in a sorted array. Raptor uses a red-black tree for logarithmic insertion and removal of individual deadlines. Updating a connection's deadline does not require scanning the other connections.
465
213
 
466
- **Puma.** After a response, if the connection is keep-alive and there are already buffered bytes for the next request (`has_back_to_back_requests?`) and there is a spare app thread, loop inline. Otherwise, if `eagerly_finish` (non-blocking reads while data is already buffered) returns true, either loop inline (if spare threads) or hand back to the thread pool (`@thread_pool << client`). Otherwise, back to the reactor with `@persistent_timeout`.
214
+ ### Responses and streaming
467
215
 
468
- **Raptor.** After a response, the app thread checks `socket.wait_readable(0)`. If bytes are already available, it parses the next request inline. If other work is waiting, the parsed request goes to the back of the pool queue; otherwise the same thread dispatches it inline. If no bytes are ready, `reactor.persist` and return.
216
+ Ordinary HTTP/1.1 response writing runs on app threads in both servers. Raptor can gather headers and array-body buffers with `writev` and reuse per-thread response buffers. Puma uses its own buffered writes and Linux TCP corking. Both support file bodies and enumerable streaming bodies.
469
217
 
470
- Both servers therefore keep a back-to-back request on an app thread when its bytes are already waiting, and return an idle connection to the reactor without deliberately holding an app thread open. The details of their parsing and pool handoff still differ, but neither widens the inline window by waiting for the client.
218
+ A blocking Enumerator occupies an app thread while it waits and yields, in either threaded server. Raptor also provides `Raptor::DetachedBody`. It hands a long-lived response to the reactor and releases the app thread. Later writes use bounded buffers and must handle backpressure. This is a Raptor-specific API, not an automatic improvement for every existing streaming Rack app.
471
219
 
472
- Raptor closes an HTTP/1.1 connection after 1,000 requests by default. The finite limit bounds connection-lifetime state without forcing frequent TCP teardown and reconnection under sustained keep-alive traffic.
473
-
474
- ### Backpressure
475
-
476
- **Puma.** Cluster mode uses `accept_loop_delay` (sleep proportional to busy ratio) to prevent thundering herd across workers. Single-worker backpressure is implicit; if all threads are busy and the queue is growing, new accepts pile up in the kernel accept queue. Puma does have `queue_requests` (default true) which pushes partial requests into the reactor, freeing the accept loop, but there is no explicit "stop accepting" signal from the worker.
477
-
478
- **Raptor.** Explicit backpressure, read every iteration of the accept loop: hard skip when `backlog >= max(pool_size * 1.2, 8)`, and a softer `Thread.pass` yield when the queue alone exceeds the pool size. On supported Linux TCP bindings, the BPF reuseport program samples two workers, routes to the less loaded one, and reserves its map slot. TLS, Unix sockets, unsupported platforms, and installations without the BPF prerequisites use inherited shared listeners instead.
479
-
480
- ### Shared state (worker ↔ master)
481
-
482
- **Puma.** Pipes. Each worker writes ping messages to a pipe read by the master. Signals push the master to check status. Simple, works everywhere, but every stat update involves a syscall on both ends.
483
-
484
- **Raptor.** Anonymous mmap region shared across workers via `mmap-ruby`. Each worker writes a 49-byte slot for its own vitals every second. The master reads the region directly without a pipe exchange.
485
-
486
- The performance difference here is negligible because the update happens once per worker per second, outside request processing. The design mainly gives the master a fixed-size snapshot it can inspect without draining per-worker messages.
487
-
488
- For external monitoring, `control_url` can expose a read-only `GET /stats` endpoint over a Unix socket. Its per-worker status includes backlog, busy threads, free capacity, request count, and the thread count currently available. Adaptive pools report their current size as `max_threads`, rather than their configured limit, so `pool_capacity / max_threads` remains a useful measure while the pool grows and shrinks.
220
+ The SSE benchmark uses detached bodies for Raptor and blocking Enumerators for Puma and Falcon. It also runs Raptor with the same Enumerator body as a control. Falcon's fiber scheduler can yield during those sleeps without occupying an OS thread per stream.
489
221
 
490
222
  ### HTTP/2
491
223
 
492
- **Puma.** Not implemented. Puma's [position](https://github.com/puma/puma/issues/2697) is that HTTP/2 belongs at the edge (nginx, Caddy, ALB), which terminates it and speaks HTTP/1.1 to the app server. That's a reasonable call for the deployments Puma is aimed at, and it's where most Rails production actually sits.
493
-
494
- **Raptor.** Native C parser plus HPACK, per-stream flow control, reactor-owned response writes, detached long-lived responses, stream multiplexing over a single connection, configurable PING keepalive, and response trailers exposed through `env["raptor.response_trailers"]`. Once a request is complete it takes the same path as HTTP/1.1 and enters the same thread pool. Under HTTP/2, a single client connection can be issuing many concurrent requests, and Raptor services all of them in parallel on the same thread pool.
224
+ Raptor supports HTTP/2 over TLS with ALPN and cleartext prior-knowledge HTTP/2 on `h2c://` listeners. Completed streams enter the same Rack thread pool as HTTP/1.1 requests. Multiple streams can be in flight on one connection, but their application Ruby still shares the worker's main-Ractor GVL.
495
225
 
496
- Whether that matters depends on your setup. If you terminate TLS at an edge proxy that already speaks HTTP/2, both servers see HTTP/1.1 and it doesn't matter which of them you pick on this axis. If you're building an all-Ruby stack with no proxy in front, serving direct HTTP/2 clients, or measuring the app server itself, HTTP/2 support is where Raptor and Puma stop being comparable.
226
+ The reactor owns HTTP/2 socket writes after setup. App threads queue frames rather than competing to write directly to the shared connection. Ordinary bodies can still wait for stream or connection flow-control capacity. Detached bodies return the app thread and let the reactor schedule buffered output as capacity becomes available.
497
227
 
498
- At the throughput numbers the benchmark shows, a small set of concurrent connections multiplex many streams, so responses from several app threads share each socket. Those threads queue frames while the reactor owns non-blocking writes for the connection.
228
+ Puma does not implement HTTP/2. Its [position](https://github.com/puma/puma/issues/2697) is to leave it at the edge. If a proxy terminates HTTP/2 and forwards HTTP/1.1, native HTTP/2 in the app server is not part of that request path. It matters when clients or proxies actually use HTTP/2 to Raptor. Falcon is the direct benchmark comparison for that case.
499
229
 
500
- ### Response writing
230
+ ### Supervision and restarts
501
231
 
502
- Both servers support the same fundamental response shapes: file bodies through `IO.copy_stream`, non-blocking writes with `wait_writable(timeout)` on EAGAIN, and chunked transfer encoding for enumerable bodies without a known length. Puma uses `TCP_CORK` on Linux around HTTP/1.1 responses. Raptor only corks a response that will close the connection when its body cannot already be emitted with the headers in one `writev` call.
232
+ Both servers support phased worker replacement and hot restarts that preserve listening sockets. Raptor publishes routine worker stats through shared memory, while Puma uses pipe messages.
503
233
 
504
- On the HTTP/1.1 path, Raptor has a small `writev(2)` wrapper (`Raptor::VectorIO`) that can scatter-write the status line, headers, and body in one call for non-chunked responses. Puma sends the same content over multiple `write` calls batched by `TCP_CORK` at the kernel; Raptor groups the buffers in userspace and lets `writev` handle partial writes when necessary.
234
+ Raptor's Linux reforking promotes a warmed worker into a dedicated fork source. New workers can share its warmed state through copy-on-write. Puma also offers worker reforking. Reforking needs care around application threads, database connections and inherited resources. Raptor's benchmark disables it, so the reported memory results do not measure its benefit.
505
235
 
506
- HTTP/1.1 responses also reuse a per-thread String buffer for the status line and headers rather than allocating one per response. The buffer grows once to fit the largest response the thread has served and stays that size afterwards, so subsequent responses on that thread skip the allocation entirely.
236
+ ## What the current benchmarks show
507
237
 
508
- Chunked responses with an array or file body accumulate chunk-framed writes into the response buffer up to 512KB before flushing to the socket, collapsing what would be N `write` syscalls for an N-chunk body into a handful. Enumerable bodies (SSE, long-polling, per-line log tailing) still flush every chunk, so streaming semantics are preserved.
238
+ The README is the source of truth for versions, full tables and reproduction instructions. The current run compares Raptor 0.23.0, Puma 8.0.2 and Falcon 0.57.0 on Ruby 4.1 development head with YJIT on Linux. It uses ten workers, 120 HTTP/1.1 client connections, or 40 HTTP/2 connections with three streams each.
509
239
 
510
- Around the response boundary, HTTP/1.1 also amortises the common per-request allocations. A frozen Rack env template with the static keys (`rack.version`, `rack.hijack?`, `SCRIPT_NAME`, `QUERY_STRING`, `SERVER_SOFTWARE`) is duped per request, so the app-thread env build only writes the dynamic keys. Response header serialisation appends directly onto the response buffer instead of allocating intermediate `Array` wraps and `flat_map` products for each header value.
240
+ Each case has three fresh-process samples, interleaved in seeded shuffled rounds. Each sample has a three-second warmup and a fifteen-second measurement. Throughput, p95 and resource figures are independent medians. The reported spread is half the observed range divided by the median, not a confidence interval.
511
241
 
512
- ### Keep-alive request by request
242
+ The IO workload interleaves short sleeps with small JSON renders. The CPU workload interleaves larger JSON renders with much shorter sleeps. Both discard the rendered JSON and return a short response. They simulate application work, not a database or a complete Rails stack. A separate prebuilt 64 KiB body exercises response transfer.
513
243
 
514
- To make the keep-alive paths concrete, here is one possible timing for three requests on the same connection. The second request is already buffered when the first response completes; the third arrives after the non-blocking check. Drawn separately so the participant columns stay wide enough to read.
244
+ Raptor and Puma run with three fixed threads or scale from three to sixteen. The keep-alive IO case also tests sixteen prestarted threads. Falcon uses unbounded fibers. Raptor's request-local cleanup, CPU affinity and reforking are disabled, as is Puma's Fiber-per-request mode. These results do not describe every default configuration.
515
245
 
516
- **Puma, three keep-alive requests:**
246
+ CPU ms/request includes the master and descendants, including SSE event production, but excludes client CPU and warmup. It is divided by successful ordinary requests. Peak aggregate PSS is sampled every half second and apportions shared pages rather than counting them fully in each worker. This is a short-run memory measurement, not a long-term capacity estimate.
517
247
 
518
- ```mermaid
519
- sequenceDiagram
520
- autonumber
521
- participant Client
522
- participant PA as Server thread
523
- participant PR as Reactor thread
524
- participant PP as App thread
525
-
526
- Note over Client,PP: Request 1, initial
527
- Client->>PA: SYN, request bytes
528
- PA->>PP: push client to thread pool
529
- PP->>PP: eagerly_finish, complete, parse under GVL
530
- PP->>Client: response 1
531
-
532
- Note over Client,PP: Request 2, pipelined a few hundred us later
533
- Client->>PP: request bytes already buffered
534
- PP->>PP: has_back_to_back? spare threads? loop inline
535
- PP->>PP: parse under GVL contention
536
- PP->>Client: response 2
537
-
538
- Note over Client,PP: Request 3, arrives sub-millisecond later
539
- PP->>PP: eagerly_finish returns false, hand to reactor
540
- PP->>PR: reactor.add with persistent_timeout
541
- Client->>PR: request bytes
542
- PR->>PP: dispatch back to thread pool
543
- PP->>PP: acquire pool mutex, dequeue, parse
544
- PP->>Client: response 3
545
- ```
248
+ ### IO over HTTP/1.1
546
249
 
547
- **Raptor, three keep-alive requests:**
548
-
549
- ```mermaid
550
- sequenceDiagram
551
- autonumber
552
- participant Client
553
- participant RS as Server thread
554
- participant RR as Reactor thread
555
- participant RP as App thread
556
-
557
- Note over Client,RP: Request 1, initial
558
- Client->>RS: SYN, request bytes
559
- RS->>RS: eager_accept, read + parse inline
560
- RS->>RP: push proc to thread pool
561
- RP->>Client: response 1
562
-
563
- Note over Client,RP: Request 2, already buffered
564
- Client->>RP: request bytes already waiting
565
- RP->>RP: wait_readable(0) returns true
566
- RP->>RP: parse inline on app thread
567
- RP->>Client: response 2
568
-
569
- Note over Client,RP: Request 3, not yet available
570
- RP->>RP: wait_readable(0) returns false
571
- RP->>RR: reactor.persist
572
- Client->>RR: request bytes
573
- RR->>RP: dispatch back to thread pool
574
- RP->>Client: response 3
575
- ```
250
+ With three fixed threads, Raptor delivers about 2.2–2.3 times Puma's throughput, with and without keep-alive. That comes with roughly 2.4–2.5 times the server CPU per successful request.
576
251
 
577
- In this timing, both servers keep Request 2 inline and return Request 3 to the reactor. Their implementations differ, but the occupancy rule is the same: use the current app thread for bytes that are ready, not to wait for future bytes.
252
+ With scaling enabled, their throughput and p95 are close. The keep-alive fixed-sixteen control is also close to both scaling results. Pool capacity changes the comparison substantially.
578
253
 
579
- ## Part IV: What Raptor's design buys you
254
+ Falcon leads the fresh-connection IO run. With keep-alive, Falcon and the larger threaded pools are roughly tied in throughput.
580
255
 
581
- ### IO-bound work, where app concurrency matters
256
+ ### CPU over HTTP/1.1
582
257
 
583
- On the IO-bound benchmark profile, each request does 5 to 10 short sleeps interleaved with small CPU work, simulating a request that makes several DB or cache calls throughout its lifetime. The bottleneck is how many requests a worker can keep in flight while they wait on IO. Fixed Raptor and Puma cap application execution at three threads per worker. Falcon spawns a fiber per connection and cooperatively yields on every sleep, so many more client connections can make progress while others wait. That advantage gives Falcon the clear lead over both fixed-thread servers, especially without keep-alive.
258
+ The CPU results are mixed. On fresh connections, fixed Raptor reaches 3.61k req/s against Puma's 3.29k. Scaling Raptor reaches 3.54k against 2.89k. Raptor leads throughput by about 10% and 23% respectively, but its p95 is higher in both modes.
584
259
 
585
- Scaling Raptor starts with the same three threads, then adds temporary threads while work is queued and the active threads are mostly blocked outside the GVL. In the current results that raises throughput from 3.11k to 8.63k req/s without keep-alive, and from 3.21k to 9.43k with it. Falcon still leads the first by 29.3%; scaling Raptor leads the second by 51.3%. The benchmark reports fixed and scaling Raptor separately rather than hiding that difference in one result.
260
+ With keep-alive, fixed Puma leads throughput at 3.61k against Raptor's 3.12k, and also has lower p95. Scaling Raptor leads at 3.09k against Puma's 2.88k, with about 20% lower p95. Puma uses less CPU per successful request in both keep-alive cases. On fresh connections, CPU per request is close.
586
261
 
587
- Between the thread-based servers, Raptor holds a clear lead over Puma on both throughput and p95. Its eager paths, explicit admission control, response batching, and app pool are all designed to reduce coordination, but the benchmark does not isolate enough variables to assign the result to one of them.
262
+ Raptor's throughput spread is between ±1.2% and ±4.6% across these four cases.
588
263
 
589
- Real applications that spend most of their time waiting on a database or an upstream service look like this. If your app is IO-heavy and you're free to adopt the `async` ecosystem, Falcon and scaling Raptor are the interesting comparison: fibers are cheaper, while Raptor can run ordinary blocking Rack code without requiring a fiber-aware stack.
264
+ ### The 64 KiB response
590
265
 
591
- ### CPU-bound HTTP/1.1, where Puma and Raptor converge
266
+ This scaling keep-alive case is a clear measured strength for Raptor. It was rerun on Ruby 4.1-dev revision `33ac6afea7`; the other cases use `cda3527671`.
592
267
 
593
- On the CPU-bound benchmark profile, each POST request accepts a small JSON body and builds a JSON response in 3 to 5 chunks totalling 450 to 1500 items, with sub-100µs sleeps between chunks. It's roughly 95% CPU by wall time, so fibers can't multiplex their way to an advantage. The CPU work happens under a single Ruby VM regardless of concurrency model.
268
+ | Metric | Raptor | Puma |
269
+ | --- | --- | --- |
270
+ | Successful req/s | 102.11k | 88.55k |
271
+ | p95 | 3.37 ms | 3.63 ms |
272
+ | CPU per successful request | 0.047 ms | 0.054 ms |
273
+ | Peak aggregate PSS | 513.5 MiB | 1371.1 MiB |
594
274
 
595
- **Without keep-alive**, every request opens a fresh TCP connection, gets parsed, dispatched, served, and closes. Puma leads fixed Raptor by 5.0% on throughput and has the lower p95; Raptor still leads Falcon. Fixed and scaling Raptor both deliver 8.22k req/s. The Puma/Raptor gap is small enough that neither architecture has overwhelmed the CPU cost of the Rack workload.
275
+ Raptor serves about 15% more requests per second, with 7% lower p95, 13% less CPU per request and 63% less peak PSS. Throughput spread is ±0.4% for Raptor and ±0.6% for Puma. The prebuilt body avoids JSON-generation work. This result does not predict memory use for other response sizes or applications.
596
276
 
597
- **With keep-alive**, Puma leads fixed Raptor by 0.7% on throughput, while Raptor's p95 is 0.4ms lower; Raptor leads Falcon on both. Fixed and scaling Raptor both deliver 8.49k req/s, with scaling p95 matching Puma. The result is more useful as a guardrail than a victory claim: adaptive scaling provides the IO-bound gains without hurting this CPU-bound profile.
277
+ ### IO alongside SSE
598
278
 
599
- ### HTTP/2, when it matters
279
+ The harness opens twenty streams across the cluster, not two guaranteed streams on each worker. It keeps them connected while measuring ordinary IO requests. A stream without complete response headers after two seconds is counted as stalled and remains connected. Throughput counts successful ordinary requests only, not events or timed-out requests. Failure counts sum all three samples.
600
280
 
601
- Puma doesn't implement HTTP/2, and most Rails production terminates HTTP/2 at nginx or a similar edge proxy before it reaches the app server. If that describes your stack, Raptor's HTTP/2 support isn't going to help you. Both servers see HTTP/1.1 from the proxy and the throughput numbers above are what actually matter. Puma's [position](https://github.com/puma/puma/issues/2697) is that this is where h2 belongs, and it's a reasonable one.
281
+ With three fixed threads, native detached Raptor serves 3.46k req/s against Puma's 0.35k, nearly ten times the throughput. Puma records 54 timeouts and one stalled stream. Raptor's matched Enumerator control falls to 0.89k, with 21 timeouts. The control shows that the native result is not representative of running the same blocking streaming body on Raptor.
602
282
 
603
- Where Raptor's HTTP/2 support does matter is the all-Ruby stack: clients can speak h2 directly over TLS, or a trusted proxy can use cleartext HTTP/2 to an `h2c://` listener. In that setup, Puma uses HTTP/1.1 instead, so the app-server connection does not get HTTP/2 multiplexing or HPACK header compression.
283
+ With scaling enabled, native Raptor serves 7.48k req/s against Puma's 7.14k, about 5% ahead. Its p95 is about 5% lower, but it uses more CPU per successful request. Detached streaming frees application threads, but the benefit depends on how much thread capacity the rest of the workload needs.
604
284
 
605
- Falcon also speaks HTTP/2 natively, so it's the interesting comparison there rather than Puma. On the CPU profile, scaling narrows Raptor's throughput gap from 26.9% to 14.2% and improves its p95 advantage from 12.3% to 21.3%. On IO, scaling raises Raptor from 1.14k to 4.46k req/s, but remains 30.3% behind Falcon. The h2 samples vary substantially more than the h1 samples, so these results establish broad shape rather than a precise ranking.
285
+ ### HTTP/2 against Falcon
606
286
 
607
- The benchmark's h2 listener uses TLS, while Raptor's BPF reuseport path only wraps plain TCP listeners. BPF dispatch therefore cannot explain the h2 variance. With 40 physical connections spread across 10 workers, each carrying three streams, placement and per-connection scheduling have coarse granularity; more instrumentation is needed before assigning the variance to a specific mechanism.
287
+ On the CPU workload, Raptor reaches 3.37k–3.70k req/s against Falcon's 2.79k, a median throughput lead of about 21–33%. Its p95 is lower, but Falcon's throughput spread is ±25.3%. Falcon's samples range from 2.37k to 3.78k req/s, so the median lead is not consistent across all samples. Raptor also uses more CPU per successful request and has higher peak PSS.
608
288
 
609
- Raptor's HTTP/2 CPU-bound throughput remains in the same broad range as its HTTP/1.1 result while multiplexing streams onto shared sockets. The reactor-owned frame scheduler and flow-control atoms are part of how it coordinates that work, but this benchmark does not provide an alternative Raptor control case from which to quantify their individual effect.
289
+ For IO, scaling Raptor and Falcon are roughly tied at 7.17k and 7.13k req/s. Raptor has about 11% higher p95, 28% more CPU per successful request and 85% higher peak PSS at 916.8 MiB against 496.1 MiB. Its throughput spread is ±11.3%. With three fixed threads, Raptor falls behind at 1.59k req/s.
610
290
 
611
- ## Part V: What Raptor gives up
291
+ These tests include TLS, multiplexing and flow control. Raptor's plain-TCP BPF path is not used for this TLS listener.
612
292
 
613
- No design comes free. Two disclosures matter most.
293
+ ## Limits and choosing a server
614
294
 
615
- **Battle-tested.** Puma has been running production Rails apps since 2011. Raptor has been running my benchmarks since 2026. Those are not the same thing. Raptor is still young code. If your risk tolerance requires a decade of production hardening baked into the server itself, Puma is the answer today.
295
+ Puma is the safer default when production history, broad Ruby compatibility and operational familiarity matter most. Its simpler request path also has fewer Ractor boundaries to trace when debugging.
616
296
 
617
- **Ruby version.** Raptor requires Ruby 4.0 because it depends on `Ractor::Port` and on Ractor internals having stabilised. Puma works on 3.0 and up. If you need to support older Ruby, Puma wins by default.
297
+ Raptor requires Ruby 4 or newer and has more coordination machinery per worker. Pipeline requests pay for shareable state and cross-Ractor handoffs. Several core pieces depend on libraries developed alongside Raptor, including `atomic-ruby`, `ractor-pool`, `red-black-tree` and `mmap-ruby`. Their adoption and testing history are not equivalent to Puma's.
618
298
 
619
- A handful of smaller trade-offs are worth naming briefly. A request on Raptor's reactor pipeline pays for shareability and Ractor/collector handoffs; eager requests avoid them. Raptor's core dependencies (`ractor-pool`, `atomic-ruby`, `red-black-tree`, `mmap-ruby`, `libbpf-ruby`) are libraries I wrote specifically to make it work, which is either "purpose-built" or "narrower testing surface" depending on how you look at it. Debugging is harder because the slow path crosses Ractor and thread boundaries, so tracing it end-to-end means stitching several stack traces together. Raptor has no single-process mode, and on a single-CPU container that adds coordination without the possibility of parallel execution.
299
+ The published run uses development Ruby, not the minimum supported Ruby version. Ractor and runtime behavior can change between versions. Benchmark the version you intend to deploy rather than assuming these numbers apply to Ruby 4.0 or a later release.
620
300
 
621
- ## Which server to choose
301
+ Raptor's cluster-only model also carries process and coordination costs in a small container. Direct HTTP/2 and detached streaming are useful only when the deployment or application uses them. A synthetic fifteen-second run does not establish long-term stability, memory behavior, overload recovery or production readiness.
622
302
 
623
- Choose Puma when production history, broad Ruby compatibility, and a mature operational ecosystem matter most. That is still the default answer for most Rails deployments.
303
+ Request bodies are fully buffered before the app runs. Both protocols apply `connection.max_body_size`, which defaults to unlimited. HTTP/1 copies larger bodies to a tempfile after receipt, which does not bound peak receive memory. HTTP/2 keeps accepted bodies in memory. Public deployments should set a finite body-size limit and enforce concurrent-upload limits at a reverse proxy, with request buffering for slow uploads.
624
304
 
625
- Try Raptor when Ruby 4 is available and you want to evaluate explicit admission control, Ractor-parallel protocol paths, direct HTTP/2, and a lock-free app pool. Benchmark your own Rack application rather than extrapolating from mine. The current numbers justify the experiment; they do not replace production evidence.
305
+ Try Raptor with your thread-safe Rack application, especially if direct HTTP/2 or detached streaming would be useful. Measure throughput, latency, CPU, memory and failures alongside the server you use today.
626
306
 
627
- Choose Falcon when the application and its dependencies are fiber-aware and the workload benefits from keeping many I/O-bound requests or long-lived connections in flight. These are architectural choices, not a universal leaderboard.
307
+ Include Falcon when the application and its dependencies work well with fiber scheduling, especially for concurrent I/O or long-lived connections. Choose from measurements of your application and your operational constraints, not one benchmark leaderboard.