raptor 0.12.0 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 36b617c828f29b9196e8e5b2ea86e6fdeb5c51fece8d5f2678df486ef77c5bd6
4
- data.tar.gz: 433b1070b29e3b13d2b337a25ac7f6519c10c5ae8ab757b523ad93f426cf2f5f
3
+ metadata.gz: b7a531c4a4d9cfe93de0c4e8d255d9eac6a09d59f9af6d3884231cd0a383aca5
4
+ data.tar.gz: 3155fd5b27d53b5a65a9be9cc36d139da801ad3ce33a188ea9a46c77289ee97e
5
5
  SHA512:
6
- metadata.gz: 24e001cc6d45a3cc126f1bfdc1eb332bb0dc2353cb9ebbcaaa7bb0f111dc3c0f61715ac105700e5cf330c2fd9f4e514bea24610ec0242a15a446fd990489f368
7
- data.tar.gz: e852c7851d772da17e28650b6e9fae59a9fa34619ed4aeba38d506febb0aaa5ff6096e5ffea20841822f81fcc0588c44a2dad30a32407cb37126bfe118e8ebe4
6
+ metadata.gz: 53bfe180b9db4a6d9f7fdb3ea2f11bf2f19b5fa81c54beb09e74caa189381a52587756c6acae9b18ce16a5500a278a0b737934bc9b5e129fe3002605ee2ac5c1
7
+ data.tar.gz: a235a90619e1a8c1935208d1c516280bf911335e075bc6e91d5be78d134c5cc385381dba899328ebc2f8da213753d31618f5328e5eec8d609a06cfb1c30b5cb8
data/CHANGELOG.md CHANGED
@@ -1,5 +1,9 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [0.13.0] - 2026-07-21
4
+
5
+ - Tune BPF reuseport routing to spread bursts across near-idle workers
6
+
3
7
  ## [0.12.0] - 2026-07-19
4
8
 
5
9
  - Add `refork_after` and `before_refork` for reforking workers off a warmed source
data/README.md CHANGED
@@ -37,7 +37,7 @@ run proc { |_env| [200, { "content-type" => "text/plain" }, ["Hello, World!"]] }
37
37
  ```
38
38
  > bundle exec raptor -w 4 -t 3 hello_world.ru
39
39
  [Raptor 59013|Main|Main] Cluster initializing:
40
- [Raptor 59013|Main|Main] ├─ Version: 0.12.0
40
+ [Raptor 59013|Main|Main] ├─ Version: 0.13.0
41
41
  [Raptor 59013|Main|Main] ├─ Ruby Version: ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +YJIT +PRISM [arm64-darwin23]
42
42
  [Raptor 59013|Main|Main] ├─ Environment: development
43
43
  [Raptor 59013|Main|Main] ├─ Master PID: 59013
@@ -193,7 +193,7 @@ Worker 1 (phase 0): pid=91351, requests=1199, busy=1/3, backlog=0, booted, last_
193
193
 
194
194
  ## (Micro) Benchmarks
195
195
 
196
- Raptor 0.12.0 vs Puma 8.0.2 vs Falcon 0.55.5 across two workload profiles. **IO-bound** is a GET endpoint that
196
+ Raptor 0.13.0 vs Puma 8.0.2 vs Falcon 0.55.5 across two workload profiles. **IO-bound** is a GET endpoint that
197
197
  interleaves 5-10 short sleeps (total 2.5-15ms) with small CPU work, simulating a read path that makes several DB or
198
198
  cache calls. **CPU-bound** is a POST endpoint that accepts a small JSON body, interleaves 3-5 chunks of JSON item
199
199
  building (total 450-1500 items) with sub-100µs sleeps, and returns the built array, simulating a write path that does
@@ -202,17 +202,17 @@ most of its work in Ruby with a few near-zero-cost cache hits.
202
202
  Each cell reports the median throughput and median p95 latency independently across 5 runs, so the two numbers in a row
203
203
  may come from different runs. Every run starts a fresh server process so the samples are independent of each other;
204
204
  state accumulated in a previous run cannot bias the next. Across the whole table, the widest spread
205
- ((max - min) / 2 / median) between runs of a single cell was ±22.4% for throughput
206
- and ±31.1% for p95.
205
+ ((max - min) / 2 / median) between runs of a single cell was ±30.0% for throughput
206
+ and ±22.8% for p95.
207
207
 
208
208
  | Protocol | Workload | Raptor req/s | Raptor p95 | Puma req/s | Puma p95 | vs Puma req/s | vs Puma p95 | Falcon req/s | Falcon p95 | vs Falcon req/s | vs Falcon p95 |
209
209
  | --------------------- | -------- | ------------ | ---------- | ----------- | -------- | ------------- | ----------- | ------------ | ---------- | --------------- | ------------- |
210
- | HTTP/1.1 | IO | 1.36k req/s | 52.90 ms | 0.94k req/s | 66.00 ms | +44.6% | -19.8% | 4.65k req/s | 14.60 ms | -70.8% | +262.3% |
211
- | HTTP/1.1 | CPU | 3.46k req/s | 28.10 ms | 3.58k req/s | 16.70 ms | -3.3% | +68.3% | 3.57k req/s | 16.30 ms | -3.1% | +72.4% |
212
- | HTTP/1.1 (keep-alive) | IO | 1.33k req/s | 34.60 ms | 0.94k req/s | 65.40 ms | +41.6% | -47.1% | 4.17k req/s | 16.70 ms | -68.1% | +107.2% |
213
- | HTTP/1.1 (keep-alive) | CPU | 4.04k req/s | 12.10 ms | 3.73k req/s | 17.50 ms | +8.4% | -30.9% | 3.70k req/s | 17.90 ms | +9.1% | -32.4% |
214
- | HTTP/2 | IO | 0.69k req/s | 87.49 ms | N/A | N/A | - | - | 4.04k req/s | 17.11 ms | -83.0% | +411.3% |
215
- | HTTP/2 | CPU | 3.23k req/s | 22.63 ms | N/A | N/A | - | - | 3.85k req/s | 24.38 ms | -16.2% | -7.2% |
210
+ | HTTP/1.1 | IO | 1.39k req/s | 50.20 ms | 0.96k req/s | 66.20 ms | 44.1% higher | 24.2% lower | 4.54k req/s | 15.00 ms | 69.4% lower | 234.7% higher |
211
+ | HTTP/1.1 | CPU | 4.55k req/s | 18.10 ms | 3.64k req/s | 16.50 ms | 24.9% higher | 9.7% higher | 3.87k req/s | 15.00 ms | 17.5% higher | 20.7% higher |
212
+ | HTTP/1.1 (keep-alive) | IO | 1.42k req/s | 30.00 ms | 0.92k req/s | 65.80 ms | 54.8% higher | 54.4% lower | 4.08k req/s | 16.70 ms | 65.3% lower | 79.6% higher |
213
+ | HTTP/1.1 (keep-alive) | CPU | 4.51k req/s | 11.20 ms | 3.77k req/s | 17.50 ms | 19.8% higher | 36.0% lower | 4.00k req/s | 16.50 ms | 12.9% higher | 32.1% lower |
214
+ | HTTP/2 | IO | 0.71k req/s | 85.84 ms | N/A | N/A | - | - | 3.88k req/s | 17.43 ms | 81.7% lower | 392.3% higher |
215
+ | HTTP/2 | CPU | 4.07k req/s | 21.85 ms | N/A | N/A | - | - | 4.33k req/s | 30.84 ms | 6.0% lower | 29.2% lower |
216
216
 
217
217
  > ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +YJIT +PRISM [aarch64-linux]
218
218
  > 4 worker processes; Raptor and Puma run 3 threads per worker, Falcon runs unbounded fibers per worker;
@@ -1,10 +1,10 @@
1
1
  # Raptor vs Puma: A Design Comparison
2
2
 
3
- Raptor is a Ruby web server designed around Ruby 4's Ractors. On CPU-bound HTTP/1.1 with keep-alive it holds a real edge over the latest Puma release on the same hardware; on IO-heavy work fibers still win, and Falcon is the server to compare to there rather than Raptor or Puma. Raptor also speaks HTTP/2 natively, which Puma does not. This document is a systems-design walkthrough of how it manages both.
3
+ Raptor is a Ruby web server designed around Ruby 4's Ractors. On CPU-bound HTTP/1.1 it holds a real edge over the latest Puma release on the same hardware regardless of keep-alive; on IO-heavy work fibers still win, and Falcon is the server to compare to there rather than Raptor or Puma. Raptor also speaks HTTP/2 natively, which Puma does not. This document is a systems-design walkthrough of how it manages both.
4
4
 
5
5
  ## Why this document exists
6
6
 
7
- Raptor began as a curiosity project. Ruby 4.0 was landing with a more polished Ractor implementation, and I wanted to see what a web server would look like if you actually leaned into parallel Ruby instead of pretending the GVL was not there. The initial goal was modest: build something that could parse HTTP in parallel using Ractors, hook it up to Rack, and see if the numbers moved.
7
+ Raptor began as a curiosity project. Ruby 4.0 was landing with a more polished Ractor implementation, and I wanted to see what a web server would look like if you actually leaned into parallel Ruby instead of pretending the GVL was not there. The initial goal was modest. Build something that could parse HTTP in parallel using Ractors, hook it up to Rack, and see if the numbers moved.
8
8
 
9
9
  They did, but the more interesting result was structural. Once you commit to a Ractor-based parser, a whole set of other design choices become almost forced. You need a lock-free work queue to feed the Ractors without recreating a global bottleneck. You need a reactor that can register and deregister thousands of connections cheaply. You need a way to hand a response back to the socket without serialising through a mutex. You need to think about backpressure as a first-class concern instead of a last-minute hack. Each of those decisions individually gives a small edge; stacked on top of each other they produce the gap you see on the benchmark.
10
10
 
@@ -12,7 +12,7 @@ This document walks through both servers at systems-design depth. By the end you
12
12
 
13
13
  ## A note on Falcon
14
14
 
15
- Falcon is the other next-generation Ruby web server worth naming. It takes a different bet than either Puma or Raptor: fiber-based concurrency via the `async` gem instead of threads, native HTTP/2, per-request lightweight tasks instead of a fixed thread pool. Its strengths show up on workloads with lots of concurrent long-lived connections (WebSockets, SSE, streaming), applications built end-to-end on the `async` ecosystem, and HTTP/2-heavy traffic. On a traditional Rails workload where most of the request budget is a synchronous DB round-trip through ActiveRecord, its advantages over Puma are less pronounced because the fiber scheduler only helps when the underlying I/O is fiber-aware. If your app fits that sweet spot, Falcon belongs in your evaluation.
15
+ Falcon is the other next-generation Ruby web server worth naming. It takes a different bet than either Puma or Raptor. Concurrency comes from fibers via the `async` gem instead of threads, HTTP/2 is native, and each request is a lightweight task rather than a slot in a fixed thread pool. Its strengths show up on workloads with lots of concurrent long-lived connections (WebSockets, SSE, streaming), applications built end-to-end on the `async` ecosystem, and HTTP/2-heavy traffic. On a traditional Rails workload where most of the request budget is a synchronous DB round-trip through ActiveRecord, its advantages over Puma are less pronounced because the fiber scheduler only helps when the underlying I/O is fiber-aware. If your app fits that sweet spot, Falcon belongs in your evaluation.
16
16
 
17
17
  This document focuses on Puma because Puma is the incumbent that any new Ruby web server has to justify itself against; that is the comparison most readers actually need. Falcon appears in the benchmark table in the README as a third data point, but a full design comparison against Falcon would be its own document.
18
18
 
@@ -27,9 +27,9 @@ Two workload profiles are measured. **IO-bound** is a GET endpoint that does 5 t
27
27
  Each cell in the table reports the median throughput and median p95 latency independently across 5 runs, and every run boots a fresh server process so state cannot accumulate across measurements. Rather than pin specific numbers into this document (they drift with Ruby versions and hardware), the shape of the result is what matters.
28
28
 
29
29
  - On IO-bound work, Falcon is roughly 3-4x ahead of both thread-based servers; the fiber-per-connection model can keep every one of the 48 client connections in flight at once, while Raptor and Puma cap out at their 12 threads. Between the two thread-based servers, Raptor holds a clear lead over Puma on both throughput and p95.
30
- - On CPU-bound HTTP/1.1 without keep-alive, the three servers land close together on throughput. Raptor's p95 is meaningfully worse than Puma's and Falcon's here; the extra hop through the pipeline shows up as tail latency when connection setup and teardown are also on the critical path.
30
+ - On CPU-bound HTTP/1.1 without keep-alive, Raptor pulls ahead of Puma on throughput and lands close on p95. Connection setup and teardown are on every request's critical path, but the load-aware BPF-directed reuseport routing spreads accept bursts across the least-loaded workers at kernel level, so no single worker ends up herding the burst.
31
31
  - On CPU-bound HTTP/1.1 with keep-alive, Raptor pulls ahead of both on throughput and p95. Connection setup is amortised, the eager keep-alive loop keeps requests on the same app thread, and the Ractor-parallel parser earns its cost when many small requests are in flight.
32
- - On HTTP/2, Raptor and Falcon both implement it; Puma doesn't. On CPU-bound h2 Raptor holds a small edge on throughput and a meaningful one on p95; on IO-bound h2 Falcon dominates for the same reason it dominates h1 IO. This matters if you're terminating h2 at the app server, less so if nginx or another proxy in front is already handling it.
32
+ - On HTTP/2, Raptor and Falcon both implement it; Puma doesn't. On CPU-bound h2 Raptor lands close to Falcon on throughput and holds a meaningful edge on p95; on IO-bound h2 Falcon dominates for the same reason it dominates h1 IO. This matters if you're terminating h2 at the app server, less so if nginx or another proxy in front is already handling it.
33
33
 
34
34
  The rest of this doc explains why the shape looks like that.
35
35
 
@@ -46,7 +46,7 @@ The rest of this doc explains why the shape looks like that.
46
46
  | Work queue | `Mutex` + `ConditionVariable` + waiter list | Lock-free CAS on an `Atom` (Banker's queue) |
47
47
  | HTTP/2 | Not implemented | Native C parser + HPACK, lock-free per-connection frame writer |
48
48
  | Keep-alive fast path | Same-thread inline dispatch when spare threads exist | Same-thread inline plus a `wait_readable(1ms)` micro-poll on the app thread |
49
- | Native extensions | 1 (Ragel HTTP/1 parser + MiniSSL) | 2, both Ractor-safe (Ragel HTTP/1 parser; HTTP/2 parser + HPACK) |
49
+ | Native extensions | 1 (Ragel HTTP/1 parser + MiniSSL) | 3, all Ractor-safe (Ragel HTTP/1 parser; HTTP/2 parser + HPACK; `writev`, `sched_setaffinity`, `prctl` wrappers) |
50
50
  | Shared state (worker↔master) | Pipes and signals | Anonymous shared-memory `mmap` region |
51
51
  | Restart primitives | Phased (USR1), hot (USR2 re-exec, inherits FDs via env), refork (SIGURG) | Phased (USR1), hot (USR2 re-exec, inherits FDs via env) |
52
52
  | systemd integration | `sd_notify` via plugin, `LISTEN_FDS` via binder | Native `sd_notify` + `LISTEN_FDS` socket activation |
@@ -57,9 +57,9 @@ The rest of the document expands on each row.
57
57
 
58
58
  ### Process model
59
59
 
60
- Puma is bimodal. In single mode the CLI runs a single `Puma::Server` in the current process; in cluster mode the CLI runs a master process that forks N workers, each running its own `Puma::Server`. Cluster mode is the interesting one because it is what almost everyone uses in production, and since Puma 5 it pre-forks by default: with two or more workers, `preload_app` is enabled unless explicitly turned off. The app is loaded once in the master, then N workers are forked from it, which preserves copy-on-write memory. If your app boots threads during load, copy-on-write breaks; every mutation to a page in the child dirties it and forces a private copy. Puma tries to detect this and warns.
60
+ Puma is bimodal. In single mode the CLI runs a single `Puma::Server` in the current process; in cluster mode the CLI runs a master process that forks N workers, each running its own `Puma::Server`. Cluster mode is the interesting one because it is what almost everyone uses in production, and since Puma 5 it pre-forks by default; with two or more workers, `preload_app` is enabled unless explicitly turned off. The app is loaded once in the master, then N workers are forked from it, which preserves copy-on-write memory. If your app boots threads during load, copy-on-write breaks; every mutation to a page in the child dirties it and forces a private copy. Puma tries to detect this and warns.
61
61
 
62
- Puma also has a `fork_worker` option, which is Puma's answer to CoW decay over time. Worker 0 becomes a "fork server": new workers are spawned from worker 0 rather than from the master, so state built up in worker 0 (JIT caches, initialised gems, warm-up allocations) is shared with siblings via CoW. This is triggered manually with SIGURG or automatically after a request threshold. It is a clever workaround for a real problem, but it also mixes the "master supervises workers" and "worker services requests" concerns.
62
+ Puma also has a `fork_worker` option, which is Puma's answer to CoW decay over time. Worker 0 becomes a "fork server", spawning new workers from itself rather than from the master, so state built up in worker 0 (JIT caches, initialised gems, warm-up allocations) is shared with siblings via CoW. This is triggered manually with SIGURG or automatically after a request threshold. It is a clever workaround for a real problem, but it also mixes the "master supervises workers" and "worker services requests" concerns.
63
63
 
64
64
  Workers write their vitals (pid, boot state, timestamp) into a per-worker pipe read by the master. If a worker misses its `worker_check_interval` for longer than `worker_timeout` the master kills it and forks a replacement. Booting workers get a separate `worker_boot_timeout`.
65
65
 
@@ -67,7 +67,7 @@ Signals: INT and TERM start a graceful shutdown, USR1 does a phased restart (inc
67
67
 
68
68
  ### Threading model
69
69
 
70
- Inside a worker, request work is handled by `Puma::ThreadPool`. The pool has a minimum and maximum thread count, and it autoscales: when work arrives and there are more items queued than there are waiting threads, it spawns a new thread up to the max. The queue is a plain array guarded by a `Mutex`, with a `ConditionVariable` used to park idle threads. Adding work signals the condvar; a waiting thread wakes, dequeues an item, and processes it.
70
+ Inside a worker, request work is handled by `Puma::ThreadPool`. The pool has a minimum and maximum thread count, and it autoscales; when work arrives and there are more items queued than there are waiting threads, it spawns a new thread up to the max. The queue is a plain array guarded by a `Mutex`, with a `ConditionVariable` used to park idle threads. Adding work signals the condvar; a waiting thread wakes, dequeues an item, and processes it.
71
71
 
72
72
  This is a textbook thread pool. It works, and it has for a decade. But it has three characteristics worth noting for the comparison:
73
73
 
@@ -81,20 +81,20 @@ The pool exposes `busy_threads`, `waiting`, and `pool_capacity` metrics that the
81
81
 
82
82
  Puma has two threads doing I/O per worker: a **server thread** running the accept loop, and a **reactor thread** running `NIO::Selector`.
83
83
 
84
- The server thread is straightforward. It calls `IO.select` on all listener sockets with an `@idle_timeout` (default 30s), then `accept_nonblock` on any that are readable, wraps the resulting socket in a `Puma::Client` object, and pushes the client into the thread pool. That is all it does. Every accepted client is handed off to the thread pool, even if the request would have been readable immediately.
84
+ The server thread is straightforward. It calls `IO.select` on all listener sockets (no timeout by default, so it blocks until one is readable), then `accept_nonblock` on any that are, wraps the resulting socket in a `Puma::Client` object, and pushes the client into the thread pool. That is all it does. Every accepted client is handed off to the thread pool, even if the request would have been readable immediately.
85
85
 
86
- What happens next depends on `queue_requests`. `queue_requests` defaults to true and is the mode almost everyone runs in. When it is true, a thread-pool worker picks up the client and calls `client.eagerly_finish`, which does one non-blocking read attempt and one attempt at parsing. If the request is complete after that single read, the worker proceeds to invoke the Rack app inline. If it is not complete, the worker hands the client to the reactor and returns to the pool. When `queue_requests` is false, the worker skips the eagerly_finish attempt and does a blocking `client.finish(first_data_timeout)`. The `queue_requests: false` path is fine for fast trusted clients but does not scale; a slow client will hold a thread indefinitely.
86
+ What happens next depends on `queue_requests`. `queue_requests` defaults to true and is the mode almost everyone runs in. When it is true, a thread-pool worker picks up the client and calls `client.eagerly_finish`, which loops non-blocking reads while bytes are already buffered, parsing after each. If the request is complete by the time the socket runs dry, the worker proceeds to invoke the Rack app inline. If it is not complete, the worker hands the client to the reactor and returns to the pool. When `queue_requests` is false, the worker skips the eagerly_finish attempt and does a blocking `client.finish(first_data_timeout)`. The `queue_requests: false` path is fine for fast trusted clients but does not scale; a slow client will hold a thread indefinitely.
87
87
 
88
88
  The reactor is a separate thread running `Puma::Reactor`, which wraps an `NIO::Selector` from `nio4r`. Its job is to babysit two kinds of sockets:
89
89
 
90
90
  1. **Sockets in the middle of a request.** A client sent some headers, was not yet complete, and needs more data before the app can be called.
91
91
  2. **Keep-alive sockets between requests.** The previous request finished, the connection is still alive, and we are waiting for the next request to start.
92
92
 
93
- The reactor stores clients in a data structure keyed by timeout, and every time through the loop it computes the deadline of the earliest-expiring client, calls `selector.select(timeout)`, and then dispatches: any client whose socket is readable calls `try_to_finish`, which does a `read_nonblock` and attempts to parse whatever it got. If the request is now complete, the client is handed to the thread pool. If the socket is not ready or the request is still incomplete, the client goes back to sleep in the reactor.
93
+ The reactor stores clients in a data structure keyed by timeout, and every time through the loop it computes the deadline of the earliest-expiring client, calls `selector.select(timeout)`, and then dispatches. Any client whose socket is readable calls `try_to_finish`, which does a `read_nonblock` and attempts to parse whatever it got. If the request is now complete, the client is handed to the thread pool. If the socket is not ready or the request is still incomplete, the client goes back to sleep in the reactor.
94
94
 
95
95
  Puma stores its reactor timeouts in a Ruby array of `Client` objects. New clients are pushed onto the end and the array is re-sorted with `sort_by!(&:timeout_at)` after each batch of inserts. Deletion is a linear scan (`@timeouts.delete client`). That is fine for small numbers of clients but scales linearly in the number of connections. It is not a hot spot at moderate load, but the cost is there.
96
96
 
97
- Cross-thread waking is done through a pipe. When work needs to enter the reactor from another thread (a Client being registered), a byte is written to a pipe that the selector is also watching; the selector wakes, drains the input queue, and gets on with it.
97
+ Cross-thread waking uses `NIO::Selector#wakeup`. When work needs to enter the reactor from another thread (a Client being registered), the client is pushed onto a Ruby `Queue` and the selector is woken; the selector drains the queue and gets on with it.
98
98
 
99
99
  ### HTTP/1.1 request lifecycle
100
100
 
@@ -102,7 +102,7 @@ Putting the pieces together, a request through Puma looks like this:
102
102
 
103
103
  1. Server thread `IO.select` returns a readable listener.
104
104
  2. Server thread `accept_nonblock` produces a new socket. Puma wraps it in a `Client` and pushes it to the thread pool.
105
- 3. A thread-pool worker picks up the client and calls `client.eagerly_finish`. This does one `read_nonblock` and one attempt at parsing.
105
+ 3. A thread-pool worker picks up the client and calls `client.eagerly_finish`, which loops `read_nonblock` while bytes are already buffered and tries to parse after each.
106
106
  4. If the request is complete after that read, the worker calls `handle_request(client)` inline, which invokes the Rack app, formats the response, writes it back.
107
107
  5. If the request is not complete, the worker hands the client to the reactor and returns to the pool. The reactor waits for more bytes, retries the parse, and pushes the client back to the thread pool once complete.
108
108
  6. After the response is written, the client is either closed or, if keep-alive, kept inline on the same thread (if `has_back_to_back_requests?` is true, or `eagerly_finish` on the next request returns true and there is a spare thread), pushed back to the thread pool, or re-added to the reactor with `@persistent_timeout` as the new deadline.
@@ -180,13 +180,13 @@ Raptor takes a different position on nearly every axis. It is opinionated in a w
180
180
 
181
181
  ### Process model
182
182
 
183
- There is no single mode. Raptor is always a cluster: a master forks N workers, monitors them, and restarts crashed workers. The Rack app is always loaded in the master before forking, so copy-on-write is preserved by default (no user-visible `preload_app` knob).
183
+ There is no single mode. Raptor is always a cluster. A master forks N workers, monitors them, and restarts crashed workers. The Rack app is always loaded in the master before forking, so copy-on-write is preserved by default (no user-visible `preload_app` knob).
184
184
 
185
185
  The master is a supervisor. It never handles requests. It forks workers, watches them via a shared-memory region (more on that in a moment), traps signals, restarts crashed workers, and orchestrates restarts.
186
186
 
187
187
  Two kinds of restart are supported:
188
188
 
189
- 1. **Phased restart on SIGUSR1.** Same idea as Puma: kill each worker in sequence, wait for its replacement to boot, move on. Existing workers drain their connections while their replacements come up. This is cheap and safe when the change does not require a fresh master.
189
+ 1. **Phased restart on SIGUSR1.** Same idea as Puma. Kill each worker in sequence, wait for its replacement to boot, move on. Existing workers drain their connections while their replacements come up. This is cheap and safe when the change does not require a fresh master.
190
190
 
191
191
  2. **Hot restart on SIGUSR2.** Also known as "zero-downtime restart with FD inheritance". The master clears the close-on-exec flag on every listener, JSON-encodes the map from bind URI to file descriptor into an environment variable (`RAPTOR_INHERITED_FDS`), and re-execs itself with the original command line. The new master reads the environment variable and rebuilds its `Binder` from the inherited FDs instead of binding fresh sockets. Not a single connection is dropped, and the new master runs its initialisation from scratch (loading a newer Rack app, applying new config, whatever).
192
192
 
@@ -206,7 +206,7 @@ Successive thresholds promote fresh seeds. When the next threshold trips, the ma
206
206
 
207
207
  Master and seed communicate over two pipes opened before the initial fork. The fork pipe (`@fork_r`, `@fork_w`) carries an 8-byte packed `[slot_index, phase]` from master to seed for each replacement to fork. The response pipe (`@resp_r`, `@resp_w`) carries a 4-byte packed pid back the other way, plus a one-shot 4-byte zero the seed writes when it enters its loop so the master knows drain is complete. The master polls the ready marker non-blockingly on the supervision loop; the phased restart that fills the vacated slot and cycles the remaining workers only starts once the marker arrives (or the drain deadline expires, in which case the master clears the seed reference and falls back to direct forks).
208
208
 
209
- Workers forked by the seed are not the master's direct children. Before spawning anything the master calls `PR_SET_CHILD_SUBREAPER` (via a small `Raptor::Subreaper` C extension wrapping `prctl`), so seed-forked children reparent to the master immediately and `Process.wait2` sees them like any other child. When a seed is retired on the next promotion its still-running descendants stay attached to the master through the same subreaper bit. Refork depends on this: on platforms without `PR_SET_CHILD_SUBREAPER` everything except Linux `refork_after` defaults to `nil` and is ignored with a warning if the user sets it anyway.
209
+ Workers forked by the seed are not the master's direct children. Before spawning anything the master calls `PR_SET_CHILD_SUBREAPER` (via a small `prctl` wrapper, `Raptor::Subreaper`), so seed-forked children reparent to the master immediately and `Process.wait2` sees them like any other child. When a seed is retired on the next promotion its still-running descendants stay attached to the master through the same subreaper bit. Refork depends on this. On platforms without `PR_SET_CHILD_SUBREAPER` (everything except Linux), `refork_after` defaults to `nil` and is ignored with a warning if the user sets it anyway.
210
210
 
211
211
  The design is Pitchfork's, adapted for Raptor's process model. [Pitchfork](https://github.com/Shopify/pitchfork) was the first Ruby server built around `PR_SET_CHILD_SUBREAPER` and multi-generation mould promotion; Puma's shipping `fork_worker` keeps worker 0 serving traffic alongside its fork-server duties, and Instacart's [`mold_worker` PR #3643](https://github.com/puma/puma/pull/3643) is Puma catching up to the same shape Pitchfork established. Raptor's seed matches the Pitchfork shape: pure fork source, promoted from a warm worker, retired when the next generation takes over.
212
212
 
@@ -225,7 +225,7 @@ That is a lot of moving parts. Let us go through why.
225
225
 
226
226
  **Why Ractors for parsing.** Ractors are Ruby's answer to true parallelism. Multiple Ractors can execute Ruby code simultaneously on different OS threads, each with its own GVL. But Ractors are heavily restricted. They can only share frozen data, they cannot access most global mutable state, and code inside a Ractor cannot use most existing gems (which universally assume shared-state semantics).
227
227
 
228
- For a web server, this restriction turns out to be almost exactly right for HTTP parsing. Parsing a request is CPU-bound (tokenising bytes, uppercasing header names, decoding chunked bodies), it does not need to touch any global state, and it produces a result (a hash) that can be safely frozen and handed off. The native HTTP/1 parser (`raptor_http.c`) is declared `rb_ext_ractor_safe(true)`: it holds no per-parser Ruby state in the extension itself, and it writes only into the caller-supplied env hash. Same for the HTTP/2 parser plus HPACK.
228
+ For a web server, this restriction turns out to be almost exactly right for HTTP parsing. Parsing a request is CPU-bound (tokenising bytes, uppercasing header names, decoding chunked bodies), it does not need to touch any global state, and it produces a result (a hash) that can be safely frozen and handed off. The native HTTP/1 parser (`raptor_http.c`) is declared `rb_ext_ractor_safe(true)`; it holds no per-parser Ruby state in the extension itself, and it writes only into the caller-supplied env hash. Same for the HTTP/2 parser plus HPACK.
229
229
 
230
230
  The HTTP/1 parser also pre-interns the ~40 most common header keys (`HTTP_HOST`, `HTTP_USER_AGENT`, the `HTTP_ACCEPT_*` family, `CONTENT_LENGTH`, `HTTP_X_FORWARDED_*`, the `HTTP_SEC_FETCH_*` client hints, and so on) once at load time. During parsing, a `memcmp` lookup against that table returns the shared frozen `VALUE` for known keys and falls back to `rb_enc_interned_str` for the rest. Every request's env hash therefore reuses the same String object for its header names, which both skips per-request allocation and lets Ruby's hash lookup use the interned key's cached hash code.
231
231
 
@@ -235,7 +235,7 @@ The upshot is that while your Rack app runs on regular threads under the GVL (so
235
235
 
236
236
  Note that Ractors also have their own copy of the code, so booting them means loading dependencies inside each Ractor context. Raptor pre-loads only what the parser needs.
237
237
 
238
- **Why a custom thread pool.** The `AtomicThreadPool` in `atomic-ruby` (another one of my libraries) is a fixed-size pool where the work queue is stored as a frozen `{in:, out:, count:, shutdown:}` hash inside an `Atom` (a CAS-protected reference cell). Enqueuing is one CAS: the new work item is prepended to the `in` stack. Dequeuing is one CAS: pop from `out`, or if `out` is empty, atomically flip `in` and `out` (this is the "Banker's queue" pattern). Backpressure metrics (`queue_length`, `active_count`) are single reads of the atomic state, no lock required.
238
+ **Why a custom thread pool.** The `AtomicThreadPool` in `atomic-ruby` (another one of my libraries) is a fixed-size pool where the work queue is stored as a frozen `{in:, out:, count:, shutdown:}` hash inside an `Atom` (a CAS-protected reference cell). Enqueuing is one CAS that prepends the new work item to the `in` stack. Dequeuing is one CAS that pops from `out`, or if `out` is empty, atomically flips `in` and `out` (this is the "Banker's queue" pattern). Backpressure metrics (`queue_length`, `active_count`) are single reads of the atomic state, no lock required.
239
239
 
240
240
  The pool still uses an `AtomicConditionVariable` under the hood to park idle threads (idle threads call `Thread.stop` and get woken with `Thread#wakeup`; there is no spinning), because idle spinning would waste CPU. The difference from Puma's pool is not "no locks anywhere" but rather "the hot path (enqueue and dequeue when the queue has items) is lock-free". Once every worker is busy the mechanics look similar; where things diverge is under contention when you have many threads all trying to push and pop.
241
241
 
@@ -253,7 +253,7 @@ next if @reactor.backlog >= backpressure_threshold
253
253
 
254
254
  where `@reactor.backlog` is `thread_pool.queue_size + thread_pool.active_count`. If the total load on the thread pool is at 120% of the pool size, this worker stops accepting until it drains. `MIN_BACKPRESSURE_THRESHOLD` is 8, so small pools (say, 3 threads) trip backpressure at 8 concurrent items rather than at 120% of a very small number; the floor keeps saturated workers signaling early so the load-aware dispatcher (below) has time to route around them.
255
255
 
256
- Because Raptor is always in cluster mode and every worker listens in the same `SO_REUSEPORT` group, load balancing across workers happens at the kernel level. On Linux, Raptor attaches a small BPF program to the reuseport group. Each worker binds its own listener registered in a sockmap, and a dedicated reporter thread publishes the worker's current reactor backlog into a loads map at 100 Hz. The BPF program consults the map on every incoming connection and routes it to the least-loaded worker; when all workers report equal load (typical of a fresh accept burst), the program falls back to the connection's 4-tuple hash, which spreads accepts more evenly than a random draw across a small worker count. If the `libbpf-ruby` gem is not installed or the BPF object has not been compiled, Raptor falls back silently to the default four-tuple-hash routing; if the kernel refuses the program, startup raises. Either way, if a worker is saturated and stops calling `accept`, other workers pick up the slack.
256
+ Because Raptor is always in cluster mode and every worker listens in the same `SO_REUSEPORT` group, load balancing across workers happens at the kernel level. On Linux, Raptor attaches a small BPF program to the reuseport group. Each worker binds its own listener registered in a sockmap, and a dedicated reporter thread publishes the worker's current reactor backlog into a loads map every millisecond. The BPF program consults the map on every incoming connection. When the spread between the busiest and idlest worker is within one, it treats all workers as tied and hash-distributes the connection by its 4-tuple; otherwise it routes to the least-loaded worker. The tie band matters. Without it, a worker that briefly drained one request would attract every subsequent connection until the next load update, herding bursts onto whichever worker most recently reported the lowest load. If the `libbpf-ruby` gem is not installed or the BPF object has not been compiled, Raptor falls back silently to the default four-tuple-hash routing; if the kernel refuses the program, startup raises. Either way, if a worker is saturated and stops calling `accept`, other workers pick up the slack.
257
257
 
258
258
  The BPF-based approach was inspired by [a comment](https://github.com/puma/puma/issues/3934#issuecomment-4356462590) by John Hawthorn ([@jhawthorn](https://github.com/jhawthorn)) on a Puma issue about `EPOLLEXCLUSIVE`, where he floated `SO_ATTACH_REUSEPORT_EBPF` as a way to route each connection to the least-busy worker.
259
259
 
@@ -321,7 +321,7 @@ From there the shape is similar to HTTP/1.1:
321
321
  3. Completed requests (once `HEADERS` and `DATA` are complete for a stream) go to the thread pool as separate work items. **A single connection can be servicing many streams in parallel across the thread pool.**
322
322
  4. Each stream's response is written back through the connection's `Writer`, which serialises frame writes across threads without a mutex.
323
323
 
324
- The `Writer` is worth a paragraph. Naive per-connection writing would use a mutex: acquire, write, release. Contention grows with concurrent streams. Raptor's `Writer` stores the "pending frames" queue in an `Atom` whose value is either `:idle` (nobody is writing) or an array of frames waiting to go out. A thread that wants to write does a CAS:
324
+ The `Writer` is worth a paragraph. Naive per-connection writing would need a mutex around every socket write. Contention grows with concurrent streams. Raptor's `Writer` stores the "pending frames" queue in an `Atom` whose value is either `:idle` (nobody is writing) or an array of frames waiting to go out. A thread that wants to write does a CAS:
325
325
 
326
326
  - If current value is `:idle`, the thread claims the writer by CAS-ing to its own array of frames, then loops draining any additional frames other threads have appended.
327
327
  - If current value is an array (someone is already writing), the thread CAS-appends its frames and returns immediately; the current writer will pick them up and flush them.
@@ -413,7 +413,7 @@ The critical structural difference from Puma is that parsing is not on the app t
413
413
 
414
414
  **Raptor.** Parsing happens on Ractors. Between requests, the app thread does a 1ms micro-poll before handing back. Every request goes through: reactor (I/O only) → Ractor pool (parse only) → collector → thread pool (app + write) → 1ms wait → maybe repeat. Parsing does not share the GVL with the app because Ractors have their own GVLs.
415
415
 
416
- Concrete effect: Puma has one process-wide GVL. Every thread inside a worker (server, reactor, and every app thread) has to take turns holding it. Raptor has that same main-process GVL plus one additional GVL per Ractor. With 3 app threads and 1 pipeline Ractor, Raptor can genuinely run two things at the same time on two CPU cores: the Ractor parsing an incoming request, and an app thread executing the Rack app. Puma cannot. Under CPU-heavy parsing (many small requests, high header count), Raptor's parsing throughput is straightforwardly higher because the parse never contends with the app for the same GVL.
416
+ The concrete effect is that Puma has one process-wide GVL. Every thread inside a worker (server, reactor, and every app thread) has to take turns holding it. Raptor has that same main-process GVL plus one additional GVL per Ractor. With 3 app threads and 1 pipeline Ractor, Raptor can genuinely run two things at once on two CPU cores. The Ractor parses an incoming request while an app thread executes the Rack app. Puma cannot. Under CPU-heavy parsing (many small requests, high header count), Raptor's parsing throughput is straightforwardly higher because the parse never contends with the app for the same GVL.
417
417
 
418
418
  ### Timeout management
419
419
 
@@ -435,7 +435,7 @@ Under moderate load these look equivalent. Under high concurrency (many threads
435
435
 
436
436
  ### Keep-alive fast path
437
437
 
438
- **Puma.** After a response, if the connection is keep-alive and there are already buffered bytes for the next request (`has_back_to_back_requests?`) and there is a spare app thread, loop inline. Otherwise, if `eagerly_finish` (a single non-blocking read attempt) returns true, either loop inline (if spare threads) or hand back to the thread pool (`@thread_pool << client`). Otherwise, back to the reactor with `@persistent_timeout`.
438
+ **Puma.** After a response, if the connection is keep-alive and there are already buffered bytes for the next request (`has_back_to_back_requests?`) and there is a spare app thread, loop inline. Otherwise, if `eagerly_finish` (non-blocking reads while data is already buffered) returns true, either loop inline (if spare threads) or hand back to the thread pool (`@thread_pool << client`). Otherwise, back to the reactor with `@persistent_timeout`.
439
439
 
440
440
  **Raptor.** After a response, the app thread does `socket.wait_readable(0.001)`, waiting up to 1ms for bytes. If bytes arrive, it parses the next request inline. If the thread pool queue is at least as deep as the pool, the parsed request is handed back to the pool so other threads share the load; otherwise the same thread dispatches it inline. If no bytes arrive, `reactor.persist` and return.
441
441
 
@@ -445,7 +445,7 @@ This is where most of Raptor's keep-alive edge comes from. Subsequent requests a
445
445
 
446
446
  ### Backpressure
447
447
 
448
- **Puma.** Cluster mode uses `accept_loop_delay` (sleep proportional to busy ratio) to prevent thundering herd across workers. Single-worker backpressure is implicit: if all threads are busy and the queue is growing, new accepts pile up in the kernel accept queue. Puma does have `queue_requests` (default true) which pushes partial requests into the reactor, freeing the accept loop, but there is no explicit "stop accepting" signal from the worker.
448
+ **Puma.** Cluster mode uses `accept_loop_delay` (sleep proportional to busy ratio) to prevent thundering herd across workers. Single-worker backpressure is implicit; if all threads are busy and the queue is growing, new accepts pile up in the kernel accept queue. Puma does have `queue_requests` (default true) which pushes partial requests into the reactor, freeing the accept loop, but there is no explicit "stop accepting" signal from the worker.
449
449
 
450
450
  **Raptor.** Explicit backpressure formula, read every iteration of the accept loop: `if backlog >= max(pool_size * 1.2, 8), skip accept`. When a worker is saturated, other workers pick up the traffic. On Linux, an eBPF program attached to the reuseport group actively routes new connections to the least-loaded worker (see the I/O model section); elsewhere Raptor relies on the kernel's default four-tuple-hash routing.
451
451
 
@@ -461,17 +461,17 @@ The performance difference here is negligible. Stats writing is one syscall per
461
461
 
462
462
  **Puma.** Not implemented. Puma's [position](https://github.com/puma/puma/issues/2697) is that HTTP/2 belongs at the edge (nginx, Caddy, ALB), which terminates it and speaks HTTP/1.1 to the app server. That's a reasonable call for the deployments Puma is aimed at, and it's where most Rails production actually sits.
463
463
 
464
- **Raptor.** Native C parser plus HPACK, per-stream flow control, lock-free frame writer, stream multiplexing over a single connection. Same request path as HTTP/1.1 once a request is complete: it enters the same thread pool. Under HTTP/2, a single client connection can be issuing many concurrent requests, and Raptor services all of them in parallel on the same thread pool.
464
+ **Raptor.** Native C parser plus HPACK, per-stream flow control, lock-free frame writer, stream multiplexing over a single connection. Once a request is complete it takes the same path as HTTP/1.1 and enters the same thread pool. Under HTTP/2, a single client connection can be issuing many concurrent requests, and Raptor services all of them in parallel on the same thread pool.
465
465
 
466
466
  Whether that matters depends on your setup. If you terminate TLS at an edge proxy that already speaks HTTP/2, both servers see HTTP/1.1 and it doesn't matter which of them you pick on this axis. If you're building an all-Ruby stack with no proxy in front, running gRPC or similar all the way through, or measuring the app server itself, HTTP/2 support is where Raptor and Puma stop being comparable.
467
467
 
468
- At the throughput numbers the benchmark shows, with only a handful of concurrent connections each multiplexing many streams, Raptor is exercising the HTTP/2 multiplexing hard: many in-flight streams sharing one socket, all writing responses through the same connection. The writer's CAS-based coordination avoids the mutex contention that would otherwise cap throughput.
468
+ At the throughput numbers the benchmark shows, with only a handful of concurrent connections each multiplexing many streams, Raptor is exercising the HTTP/2 multiplexing hard, with many in-flight streams sharing one socket and all writing responses through the same connection. The writer's CAS-based coordination avoids the mutex contention that would otherwise cap throughput.
469
469
 
470
470
  ### Response writing
471
471
 
472
472
  Both servers do the same fundamental things: `TCP_CORK` on Linux to batch small responses, `IO.copy_stream` for File bodies (which invokes the sendfile syscall on Linux), non-blocking writes with `wait_writable(timeout)` on EAGAIN, chunked transfer encoding for enumerable bodies without a known length.
473
473
 
474
- On the HTTP/1.1 path, Raptor has a small `writev(2)` C extension (`Raptor::VectorIO`) that scatter-writes the status line, headers, and body in a single syscall for non-chunked responses. Puma sends the same content over multiple `write` calls batched by `TCP_CORK` at the kernel; Raptor collapses that to one syscall in userspace. The saving is a few microseconds per response, but it stacks on top of every h1 response.
474
+ On the HTTP/1.1 path, Raptor has a small `writev(2)` wrapper (`Raptor::VectorIO`) that scatter-writes the status line, headers, and body in a single syscall for non-chunked responses. Puma sends the same content over multiple `write` calls batched by `TCP_CORK` at the kernel; Raptor collapses that to one syscall in userspace. The saving is a few microseconds per response, but it stacks on top of every h1 response.
475
475
 
476
476
  HTTP/1.1 responses also reuse a per-thread String buffer for the status line and headers rather than allocating one per response. The buffer grows once to fit the largest response the thread has served and stays that size afterwards, so subsequent responses on that thread skip the allocation entirely.
477
477
 
@@ -559,7 +559,7 @@ Real applications that spend most of their time waiting on a database or an upst
559
559
 
560
560
  On the CPU-bound benchmark profile, each POST request accepts a small JSON body and builds a JSON response in 3 to 5 chunks totalling 450 to 1500 items, with sub-100µs sleeps between chunks. It's roughly 95% CPU by wall time, so fibers can't multiplex their way to an advantage. The CPU work happens under a single Ruby VM regardless of concurrency model.
561
561
 
562
- **Without keep-alive**, every request opens a fresh TCP connection, gets parsed, dispatched, served, and closes. All three servers land within a few percent of each other on throughput. Raptor's p95 is meaningfully worse than Puma's and Falcon's here; the extra hop through the Ractor pipeline shows up as tail latency when connection setup and teardown are also on the critical path.
562
+ **Without keep-alive**, every request opens a fresh TCP connection, gets parsed, dispatched, served, and closes. Raptor pulls ahead of both Puma and Falcon on throughput, but its p95 lands close to Puma's and moderately behind Falcon's. The extra hop through the Ractor pipeline shows up as tail latency when connection setup and teardown are also on the critical path.
563
563
 
564
564
  **With keep-alive**, Raptor pulls ahead of both Puma and Falcon on throughput and p95. Connection setup is amortised, the eager keep-alive loop keeps subsequent requests on the same app thread, and the Ractor-parallel parser earns its cost when many small requests are in flight. This is the shape of workload Raptor's design was optimised for.
565
565
 
@@ -569,7 +569,7 @@ Puma doesn't implement HTTP/2, and most Rails production terminates HTTP/2 at ng
569
569
 
570
570
  Where Raptor's HTTP/2 support does matter is the all-Ruby stack. No proxy in front, TLS terminated at the app, clients speaking h2 all the way through (`curl --http2`, gRPC-Ruby transports, HTTP/2-only SDKs, direct browser connections). In that setup, Puma silently falls back to HTTP/1.1 and you lose multiplexing, header compression, and prioritisation.
571
571
 
572
- Falcon also speaks HTTP/2 natively, so it's the interesting comparison there rather than Puma. The shape of the h2 result follows the h1 shape. On CPU-bound h2 Raptor holds a small edge on throughput and a meaningful one on p95, for the same reasons it wins CPU-bound h1 keep-alive. Parsing runs in parallel with the app, the lock-free writer serialises frames without a mutex, and per-stream flow-control atoms don't contend. On IO-bound h2 Falcon wins by a wide margin, again because fibers can keep every stream in flight simultaneously while Raptor's thread pool caps concurrency.
572
+ Falcon also speaks HTTP/2 natively, so it's the interesting comparison there rather than Puma. The shape of the h2 result follows the h1 shape. On CPU-bound h2 Raptor lands close to Falcon on throughput and holds a meaningful edge on p95, for the same reasons it wins CPU-bound h1 keep-alive. Parsing runs in parallel with the app, the lock-free writer serialises frames without a mutex, and per-stream flow-control atoms don't contend. On IO-bound h2 Falcon wins by a wide margin, again because fibers can keep every stream in flight simultaneously while Raptor's thread pool caps concurrency.
573
573
 
574
574
  Raptor's h2 IO numbers still show some run-to-run variance where h1 doesn't. With only 16 physical connections spread across four workers, even the BPF program's hash-based tie-break can leave one worker over-subscribed, and the concentrated worker becomes the bottleneck for that whole run. h2 CPU stays tight because once workers report distinguishable load the load-aware routing has enough signal to place new connections cleanly regardless of tie-break behaviour.
575
575
 
@@ -2,6 +2,7 @@
2
2
  #include <bpf/bpf_helpers.h>
3
3
 
4
4
  #define MAX_WORKERS 64
5
+ #define LOAD_TIE_TOLERANCE 1
5
6
 
6
7
  // Per-worker listening sockets, keyed by worker index.
7
8
  struct {
@@ -19,9 +20,12 @@ struct {
19
20
  __uint(max_entries, MAX_WORKERS + 1);
20
21
  } loads SEC(".maps");
21
22
 
22
- // Routes each incoming connection to the worker with the lowest backlog,
23
- // falling back to the connection's 4-tuple hash when loads are equal so
24
- // bursts of accepts spread across workers instead of clustering by chance.
23
+ // Routes each incoming connection to the least-loaded worker. When the
24
+ // spread between the busiest and idlest worker is within
25
+ // `LOAD_TIE_TOLERANCE`, all workers are treated as tied and the connection
26
+ // is placed by 4-tuple hash so bursts of accepts spread across workers
27
+ // instead of clustering on whichever worker most recently reported the
28
+ // lowest load.
25
29
  SEC("sk_reuseport")
26
30
  int select_least_loaded(struct sk_reuseport_md *ctx) {
27
31
  __u32 count_key = 0;
@@ -56,7 +60,7 @@ int select_least_loaded(struct sk_reuseport_md *ctx) {
56
60
  }
57
61
  }
58
62
 
59
- __u32 chosen_idx = (min_load == max_load) ? (ctx->hash % num_workers) : min_idx;
63
+ __u32 chosen_idx = (max_load - min_load <= LOAD_TIE_TOLERANCE) ? (ctx->hash % num_workers) : min_idx;
60
64
  bpf_sk_select_reuseport(ctx, &socks, &chosen_idx, 0);
61
65
  return SK_PASS;
62
66
  }
data/lib/raptor/binder.rb CHANGED
@@ -5,30 +5,9 @@ require "socket"
5
5
  require "uri"
6
6
 
7
7
  module Raptor
8
- # Manages binding to network addresses and creating listening sockets.
9
- #
10
- # Binder handles parsing URI bind specifications, creating TCP, Unix, and SSL
11
- # server sockets, and managing socket options for optimal performance. It
12
- # supports binding to multiple addresses simultaneously.
13
- #
14
- # @example TCP binding
15
- # binder = Binder.new(["tcp://0.0.0.0:3000", "tcp://[::1]:3000"])
16
- # puts binder.addresses #=> ["0.0.0.0:3000", "[::1]:3000"]
17
- # binder.close
18
- #
19
- # @example Unix socket binding
20
- # binder = Binder.new(["unix:///tmp/raptor.sock"])
21
- # puts binder.addresses #=> ["/tmp/raptor.sock"]
22
- # binder.close
23
- #
24
- # @example SSL binding
25
- # binder = Binder.new(["ssl://0.0.0.0:443?cert=/path/to.crt&key=/path/to.key"])
26
- # puts binder.addresses #=> ["ssl://0.0.0.0:443"]
27
- # binder.close
28
- #
29
- # @example Localhost binding
30
- # binder = Binder.new(["tcp://localhost:8080"])
31
- # # Binds to both IPv4 and IPv6 loopback addresses
8
+ # Binds `tcp://`, `unix://`, and `ssl://` URIs to listening sockets and
9
+ # holds them for the server. Reconstructs listeners from inherited file
10
+ # descriptors when provided (systemd socket activation, hot restart).
32
11
  #
33
12
  class Binder
34
13
  SOCKET_BACKLOG = 1024
@@ -38,11 +17,8 @@ module Raptor
38
17
  def initialize(scheme) = super("unknown scheme: #{scheme.inspect}")
39
18
  end
40
19
 
41
- # Wraps a TCPServer with an SSL context for accepting SSL connections.
42
- #
43
- # Holds both the underlying TCP server and the SSL context together so
44
- # the server thread can accept a TCP connection and then perform the SSL
45
- # handshake in a single coordinated step.
20
+ # Pairs a TCPServer with the SSL context to use for accepted
21
+ # connections.
46
22
  #
47
23
  SslListener = Data.define(:tcp_server, :ssl_context) do
48
24
  # @rbs () -> TCPServer
@@ -61,28 +37,24 @@ module Raptor
61
37
  # @rbs @listeners: Array[TCPServer | UNIXServer | SslListener]
62
38
  # @rbs @uri_listeners: Hash[String, Array[TCPServer | UNIXServer | SslListener]]
63
39
 
64
- # Array of bind URIs.
40
+ # Returns the array of bind URIs.
65
41
  #
66
- # @return [Array<String>] the bind URIs
42
+ # @return [Array<String>]
67
43
  attr_reader :bind_uris
68
44
 
69
- # Kernel listen() queue depth for TCP/SSL listeners.
45
+ # Returns the kernel `listen()` queue depth for TCP/SSL listeners.
70
46
  #
71
- # @return [Integer] the socket backlog
47
+ # @return [Integer]
72
48
  attr_reader :socket_backlog
73
49
 
74
- # Array of listening sockets.
50
+ # Returns the array of listening sockets.
75
51
  #
76
- # @return [Array<TCPServer, UNIXServer, SslListener>] the server sockets
52
+ # @return [Array<TCPServer, UNIXServer, SslListener>]
77
53
  attr_reader :listeners
78
54
 
79
- # Creates a new Binder with the specified bind URIs.
80
- #
81
- # Parses the provided bind URIs and creates listening sockets for each one.
82
- # Supports tcp://, unix://, and ssl:// schemes. Localhost is expanded to
83
- # all available loopback addresses (both IPv4 and IPv6). When `inherited_fds`
84
- # supplies file descriptors for a URI, the listener is reconstructed from
85
- # those FDs instead of binding fresh.
55
+ # Creates a new Binder and binds each URI. `localhost` expands to both
56
+ # IPv4 and IPv6 loopback; when `inherited_fds` supplies file descriptors
57
+ # for a URI the listener is rebuilt from those instead of binding fresh.
86
58
  #
87
59
  # @param bind_uris [Array<String>] array of URI strings to bind to
88
60
  # @param socket_backlog [Integer] kernel listen() queue depth for TCP/SSL listeners
@@ -90,9 +62,6 @@ module Raptor
90
62
  # @return [void]
91
63
  # @raise [UnknownBindSchemeError] if a URI has an unsupported scheme
92
64
  #
93
- # @example
94
- # binder = Binder.new(["tcp://0.0.0.0:3000", "unix:///tmp/raptor.sock"])
95
- #
96
65
  # @rbs (Array[String] bind_uris, ?socket_backlog: Integer, ?inherited_fds: Hash[String, Array[Integer]]) -> void
97
66
  def initialize(bind_uris, socket_backlog: SOCKET_BACKLOG, inherited_fds: {})
98
67
  @bind_uris = bind_uris
@@ -103,15 +72,10 @@ module Raptor
103
72
  parse
104
73
  end
105
74
 
106
- # Returns the bound addresses as strings.
75
+ # Returns the bound addresses as strings: TCP as `host:port`, Unix as
76
+ # the socket path, SSL as `ssl://host:port`.
107
77
  #
108
- # TCP listeners are returned as "host:port", Unix listeners as the socket
109
- # path, and SSL listeners as "ssl://host:port".
110
- #
111
- # @return [Array<String>] address strings for each bound listener
112
- #
113
- # @example
114
- # binder.addresses #=> ["127.0.0.1:3000", "/tmp/raptor.sock", "ssl://0.0.0.0:443"]
78
+ # @return [Array<String>]
115
79
  #
116
80
  # @rbs () -> Array[String]
117
81
  def addresses
@@ -129,12 +93,10 @@ module Raptor
129
93
  end
130
94
  end
131
95
 
132
- # Returns the port number of the first TCP or SSL listener.
96
+ # Returns the port of the first TCP or SSL listener, or 0 when none
97
+ # is configured.
133
98
  #
134
- # Used to populate SERVER_PORT in the Rack environment. Returns 0
135
- # if no TCP or SSL listener is configured (e.g., Unix socket only).
136
- #
137
- # @return [Integer] the port number, or 0 if no TCP listener exists
99
+ # @return [Integer]
138
100
  #
139
101
  # @rbs () -> Integer
140
102
  def server_port
@@ -153,9 +115,7 @@ module Raptor
153
115
  @listeners.each(&:close)
154
116
  end
155
117
 
156
- # Returns the file descriptors of every listener, grouped by the bind URI
157
- # they were created from. The result is the payload to hand to a successor
158
- # process via the `inherited_fds:` constructor argument.
118
+ # Returns the file descriptors of every listener, grouped by bind URI.
159
119
  #
160
120
  # @return [Hash{String => Array<Integer>}]
161
121
  #
@@ -185,24 +145,21 @@ module Raptor
185
145
  # @rbs () -> void
186
146
  def parse
187
147
  @uri_listeners = @bind_uris.to_h do |bind_uri|
188
- if filenos = @inherited_fds[bind_uri]
189
- [bind_uri, restore_listeners(bind_uri, filenos)]
190
- else
191
- [bind_uri, create_listeners(bind_uri)]
192
- end
148
+ uri = URI.parse(bind_uri)
149
+ filenos = @inherited_fds[bind_uri]
150
+ [bind_uri, filenos ? restore_listeners(uri, filenos) : create_listeners(uri)]
193
151
  end
194
152
  @listeners = @uri_listeners.values.flatten
195
153
  end
196
154
 
197
- # Creates fresh listeners for the given bind URI.
155
+ # Creates fresh listeners for the given URI.
198
156
  #
199
- # @param bind_uri [String] the URI to bind
157
+ # @param uri [URI] the parsed bind URI
200
158
  # @return [Array<TCPServer, UNIXServer, SslListener>]
201
159
  # @raise [UnknownBindSchemeError] if the URI scheme is not supported
202
160
  #
203
- # @rbs (String bind_uri) -> Array[TCPServer | UNIXServer | SslListener]
204
- def create_listeners(bind_uri)
205
- uri = URI.parse(bind_uri)
161
+ # @rbs (URI::Generic uri) -> Array[TCPServer | UNIXServer | SslListener]
162
+ def create_listeners(uri)
206
163
  case uri.scheme
207
164
  when "tcp"
208
165
  create_tcp_listeners(uri.host, uri.port)
@@ -215,17 +172,16 @@ module Raptor
215
172
  end
216
173
  end
217
174
 
218
- # Reconstructs listeners for the given bind URI from inherited file
175
+ # Reconstructs listeners for the given URI from inherited file
219
176
  # descriptors.
220
177
  #
221
- # @param bind_uri [String] the URI the FDs were bound to
178
+ # @param uri [URI] the parsed bind URI the FDs were bound to
222
179
  # @param filenos [Array<Integer>] file descriptors to wrap
223
180
  # @return [Array<TCPServer, UNIXServer, SslListener>]
224
181
  # @raise [UnknownBindSchemeError] if the URI scheme is not supported
225
182
  #
226
- # @rbs (String bind_uri, Array[Integer] filenos) -> Array[TCPServer | UNIXServer | SslListener]
227
- def restore_listeners(bind_uri, filenos)
228
- uri = URI.parse(bind_uri)
183
+ # @rbs (URI::Generic uri, Array[Integer] filenos) -> Array[TCPServer | UNIXServer | SslListener]
184
+ def restore_listeners(uri, filenos)
229
185
  case uri.scheme
230
186
  when "tcp"
231
187
  filenos.map { |fileno| TCPServer.for_fd(fileno) }
@@ -267,11 +223,9 @@ module Raptor
267
223
  [tcp_server]
268
224
  end
269
225
 
270
- # Creates a Unix domain server socket at the given path.
271
- #
272
- # Removes stale socket files left by crashed processes (when the socket
273
- # is not currently in use). Registers an at_exit hook to clean up the
274
- # socket file on normal process exit.
226
+ # Creates a Unix domain server socket at the given path. Removes stale
227
+ # socket files left by crashed processes, and cleans the file up on
228
+ # the master's clean exit.
275
229
  #
276
230
  # @param path [String] filesystem path for the Unix socket
277
231
  # @return [Array<UNIXServer>] array containing the created Unix server socket
@@ -306,15 +260,11 @@ module Raptor
306
260
  at_exit { File.delete(path) rescue nil if Process.pid == master_pid }
307
261
  end
308
262
 
309
- # Creates SSL server sockets for the given host, port, and SSL parameters.
310
- #
311
- # Wraps each TCP listener with an SSL context to produce SslListener objects.
312
- # The ssl_params hash must include "cert" and "key" entries pointing to the
313
- # certificate and private key files respectively.
263
+ # Creates SSL server sockets for the given host and port.
314
264
  #
315
265
  # @param host [String, nil] hostname or IP address to bind to
316
266
  # @param port [Integer, nil] port number to bind to
317
- # @param ssl_params [Hash<String, String>] SSL options ("cert" and "key" paths)
267
+ # @param ssl_params [Hash<String, String>] SSL options (`cert` and `key` file paths)
318
268
  # @return [Array<SslListener>] array containing the created SSL listener(s)
319
269
  #
320
270
  # @rbs (String? host, Integer? port, Hash[String, String] ssl_params) -> Array[SslListener]