raptor 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.buildkite/pipeline.yml +100 -0
- data/CHANGELOG.md +21 -0
- data/README.md +47 -31
- data/docs/raptor-vs-puma.md +104 -75
- data/ext/raptor_bpf/reuseport_select.bpf.c +3 -2
- data/ext/raptor_http/raptor_http.c +28 -3
- data/ext/raptor_native/raptor_native.c +16 -2
- data/lib/raptor/cli.rb +11 -2
- data/lib/raptor/cluster.rb +368 -29
- data/lib/raptor/http.rb +1 -0
- data/lib/raptor/http1.rb +71 -24
- data/lib/raptor/http2.rb +46 -3
- data/lib/raptor/reuseport_bpf.rb +13 -0
- data/lib/raptor/server.rb +14 -6
- data/lib/raptor/version.rb +1 -1
- data/sig/generated/raptor/cluster.rbs +173 -34
- data/sig/generated/raptor/http.rbs +2 -0
- data/sig/generated/raptor/http1.rbs +18 -5
- data/sig/generated/raptor/http2.rbs +16 -0
- data/sig/generated/raptor/reuseport_bpf.rbs +9 -0
- data/sig/generated/raptor/server.rbs +10 -4
- metadata +1 -1
data/docs/raptor-vs-puma.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Raptor vs Puma: A Design Comparison
|
|
2
2
|
|
|
3
|
-
Raptor is a Ruby web server designed around Ruby 4's Ractors. On CPU-bound HTTP/1.1
|
|
3
|
+
Raptor is a Ruby web server designed around Ruby 4's Ractors. On CPU-bound HTTP/1.1 with keep-alive it holds a real edge over the latest Puma release on the same hardware; on IO-heavy work fibers still win, and Falcon is the server to compare to there rather than Raptor or Puma. Raptor also speaks HTTP/2 natively, which Puma does not. This document is a systems-design walkthrough of how it manages both.
|
|
4
4
|
|
|
5
5
|
## Why this document exists
|
|
6
6
|
|
|
@@ -10,14 +10,26 @@ They did, but the more interesting result was structural. Once you commit to a R
|
|
|
10
10
|
|
|
11
11
|
This document walks through both servers at systems-design depth. By the end you should be able to describe how Puma and Raptor each move a request from `accept()` to `app.call(env)` and back, why the two architectures make the design decisions they do, and where the performance delta actually comes from. No source code required.
|
|
12
12
|
|
|
13
|
+
## A note on Falcon
|
|
14
|
+
|
|
15
|
+
Falcon is the other next-generation Ruby web server worth naming. It takes a different bet than either Puma or Raptor: fiber-based concurrency via the `async` gem instead of threads, native HTTP/2, per-request lightweight tasks instead of a fixed thread pool. Its strengths show up on workloads with lots of concurrent long-lived connections (WebSockets, SSE, streaming), applications built end-to-end on the `async` ecosystem, and HTTP/2-heavy traffic. On a traditional Rails workload where most of the request budget is a synchronous DB round-trip through ActiveRecord, its advantages over Puma are less pronounced because the fiber scheduler only helps when the underlying I/O is fiber-aware. If your app fits that sweet spot, Falcon belongs in your evaluation.
|
|
16
|
+
|
|
17
|
+
This document focuses on Puma because Puma is the incumbent that any new Ruby web server has to justify itself against; that is the comparison most readers actually need. Falcon appears in the benchmark table in the README as a third data point, but a full design comparison against Falcon would be its own document.
|
|
18
|
+
|
|
13
19
|
## The shape of the benchmark
|
|
14
20
|
|
|
15
|
-
|
|
21
|
+
Raptor is a research project. It hasn't run production traffic. The numbers in the README come from a repeatable microbenchmark, not from a real deployment. The benchmark measures only the **server** work: accepting connections, parsing, dispatching, and writing responses. It does not measure your application. In a typical Rails app where most of a request's time goes to ActiveRecord and downstream services, the server accounts for maybe 5 to 15 percent of the total, so a +N% number in this table will show up as a much smaller improvement in production. The gap is real, but it isn't what a Rails app against a real database will report.
|
|
22
|
+
|
|
23
|
+
The Raptor README carries the [current head-to-head numbers](../README.md#micro-benchmarks) against the latest Puma and Falcon releases, run on the same hardware with the same Rack app on a recent Ruby with YJIT enabled. All three servers run four worker processes; Raptor and Puma use three threads per worker, and Falcon uses unbounded fibers per worker. Load generators use 48 concurrent HTTP/1.1 client connections, and 16 h2 connections × 3 streams each for the h2 rows.
|
|
24
|
+
|
|
25
|
+
Two workload profiles are measured. **IO-bound** is a GET endpoint that does 5 to 10 short sleeps interleaved with small CPU work per request, simulating a read path that makes several DB or cache calls throughout its lifetime. **CPU-bound** is a POST endpoint with a small JSON body that builds a JSON response in 3 to 5 chunks interleaved with sub-100µs sleeps, simulating a write path that does most of its work in Ruby with a few near-zero-cost cache hits. The workloads are interleaved rather than a single bulk sleep or single bulk serialise so a fiber-per-connection server like Falcon doesn't look artificially good from one-shot IO, and the CPU-bound workload is heavily CPU-dominated by design (roughly 95% CPU / 5% IO by wall time) so it actually measures CPU work rather than smuggling in enough IO for fibers to multiplex.
|
|
16
26
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
- On
|
|
20
|
-
- On HTTP/
|
|
27
|
+
Each cell in the table reports the median throughput and median p95 latency independently across 5 runs, and every run boots a fresh server process so state cannot accumulate across measurements. Rather than pin specific numbers into this document (they drift with Ruby versions and hardware), the shape of the result is what matters.
|
|
28
|
+
|
|
29
|
+
- On IO-bound work, Falcon is roughly 3-4x ahead of both thread-based servers; the fiber-per-connection model can keep every one of the 48 client connections in flight at once, while Raptor and Puma cap out at their 12 threads. Between the two thread-based servers, Raptor holds a clear lead over Puma on both throughput and p95.
|
|
30
|
+
- On CPU-bound HTTP/1.1 without keep-alive, the three servers land close together on throughput. Raptor's p95 is meaningfully worse than Puma's and Falcon's here; the extra hop through the pipeline shows up as tail latency when connection setup and teardown are also on the critical path.
|
|
31
|
+
- On CPU-bound HTTP/1.1 with keep-alive, Raptor pulls ahead of both on throughput and p95. Connection setup is amortised, the eager keep-alive loop keeps requests on the same app thread, and the Ractor-parallel parser earns its cost when many small requests are in flight.
|
|
32
|
+
- On HTTP/2, Raptor and Falcon both implement it; Puma doesn't. On CPU-bound h2 Raptor holds a small edge on throughput and a meaningful one on p95; on IO-bound h2 Falcon dominates for the same reason it dominates h1 IO. This matters if you're terminating h2 at the app server, less so if nginx or another proxy in front is already handling it.
|
|
21
33
|
|
|
22
34
|
The rest of this doc explains why the shape looks like that.
|
|
23
35
|
|
|
@@ -73,7 +85,7 @@ The server thread is straightforward. It calls `IO.select` on all listener socke
|
|
|
73
85
|
|
|
74
86
|
What happens next depends on `queue_requests`. `queue_requests` defaults to true and is the mode almost everyone runs in. When it is true, a thread-pool worker picks up the client and calls `client.eagerly_finish`, which does one non-blocking read attempt and one attempt at parsing. If the request is complete after that single read, the worker proceeds to invoke the Rack app inline. If it is not complete, the worker hands the client to the reactor and returns to the pool. When `queue_requests` is false, the worker skips the eagerly_finish attempt and does a blocking `client.finish(first_data_timeout)`. The `queue_requests: false` path is fine for fast trusted clients but does not scale; a slow client will hold a thread indefinitely.
|
|
75
87
|
|
|
76
|
-
The reactor is a separate thread running `Puma::Reactor`, which wraps an `NIO::Selector` from nio4r
|
|
88
|
+
The reactor is a separate thread running `Puma::Reactor`, which wraps an `NIO::Selector` from `nio4r`. Its job is to babysit two kinds of sockets:
|
|
77
89
|
|
|
78
90
|
1. **Sockets in the middle of a request.** A client sent some headers, was not yet complete, and needs more data before the app can be called.
|
|
79
91
|
2. **Keep-alive sockets between requests.** The previous request finished, the connection is still alive, and we are waiting for the next request to start.
|
|
@@ -84,7 +96,7 @@ Puma stores its reactor timeouts in a Ruby array of `Client` objects. New client
|
|
|
84
96
|
|
|
85
97
|
Cross-thread waking is done through a pipe. When work needs to enter the reactor from another thread (a Client being registered), a byte is written to a pipe that the selector is also watching; the selector wakes, drains the input queue, and gets on with it.
|
|
86
98
|
|
|
87
|
-
###
|
|
99
|
+
### HTTP/1.1 request lifecycle
|
|
88
100
|
|
|
89
101
|
Putting the pieces together, a request through Puma looks like this:
|
|
90
102
|
|
|
@@ -104,16 +116,17 @@ Response writing is done from the app thread. Puma sets `TCP_CORK` on Linux, wri
|
|
|
104
116
|
```mermaid
|
|
105
117
|
flowchart TB
|
|
106
118
|
subgraph Master["Master process, cluster mode"]
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
119
|
+
MM["Master loop<br/>Reads worker pipes for<br/>per-worker vitals<br/>Handles signals"]
|
|
120
|
+
MM -->|"fork"| W1["Worker 0"]
|
|
121
|
+
MM -->|"fork"| W2["Worker 1"]
|
|
122
|
+
MM -->|"fork"| W3["Worker N"]
|
|
123
|
+
MM -.->|"SIGUSR2<br/>exec with argv"| MM
|
|
111
124
|
end
|
|
112
125
|
|
|
113
|
-
subgraph Worker["Puma worker process"]
|
|
114
|
-
|
|
126
|
+
subgraph Worker["Puma worker process, one of N"]
|
|
127
|
+
SRV["Server thread<br/>IO.select on listeners<br/>accept_nonblock"]
|
|
115
128
|
|
|
116
|
-
|
|
129
|
+
RCT["Reactor thread<br/>NIO::Selector<br/>sorted timeout array"]
|
|
117
130
|
|
|
118
131
|
subgraph TP["Thread pool, Mutex + CondVar"]
|
|
119
132
|
T1["Thread 1"]
|
|
@@ -121,37 +134,45 @@ flowchart TB
|
|
|
121
134
|
T3["Thread N"]
|
|
122
135
|
end
|
|
123
136
|
|
|
137
|
+
STA["Stat thread<br/>writes to worker pipe<br/>on worker_check_interval"]
|
|
138
|
+
|
|
124
139
|
C1{"Complete<br/>request<br/>on accept?"}
|
|
125
140
|
C2{"Complete<br/>request<br/>after read?"}
|
|
126
141
|
KA{"Keep-alive?"}
|
|
127
142
|
BB{"Back-to-back<br/>data in buffer?"}
|
|
128
143
|
|
|
129
|
-
|
|
144
|
+
SRV -->|"push new socket"| TP
|
|
130
145
|
TP -->|"eagerly_finish"| C1
|
|
131
146
|
C1 -->|"complete, app.call"| APP["Rack app"]
|
|
132
|
-
C1 -->|"incomplete, register"|
|
|
133
|
-
|
|
147
|
+
C1 -->|"incomplete, register"| RCT
|
|
148
|
+
RCT -->|"socket readable"| C2
|
|
134
149
|
C2 -->|"complete, push"| TP
|
|
135
|
-
C2 -->|"incomplete, keep waiting"|
|
|
136
|
-
|
|
150
|
+
C2 -->|"incomplete, keep waiting"| RCT
|
|
151
|
+
RCT -->|"timeout"| TO["write 408, close"]
|
|
137
152
|
APP -->|"write response"| KA
|
|
138
|
-
KA -->|"no"|
|
|
153
|
+
KA -->|"no"| CLS["close socket"]
|
|
139
154
|
KA -->|"yes"| BB
|
|
140
155
|
BB -->|"yes, reprocess"| TP
|
|
141
|
-
BB -->|"no, back to reactor"|
|
|
156
|
+
BB -->|"no, back to reactor"| RCT
|
|
157
|
+
|
|
158
|
+
STA -.->|"writes ping"| PIPE[("worker→master pipe")]
|
|
142
159
|
end
|
|
143
160
|
|
|
161
|
+
MM -.->|"reads ping"| PIPE
|
|
162
|
+
|
|
144
163
|
classDef accept fill:#dbeafe,stroke:#2563eb,color:#1e3a8a
|
|
145
164
|
classDef reactor fill:#fecdd3,stroke:#e11d48,color:#881337
|
|
146
165
|
classDef pool fill:#fef3c7,stroke:#d97706,color:#78350f
|
|
147
166
|
classDef app fill:#d1fae5,stroke:#059669,color:#064e3b
|
|
148
|
-
|
|
149
|
-
class
|
|
167
|
+
classDef storage fill:#e5e7eb,stroke:#6b7280,color:#374151
|
|
168
|
+
class SRV accept
|
|
169
|
+
class RCT reactor
|
|
150
170
|
class TP pool
|
|
151
171
|
class APP app
|
|
172
|
+
class PIPE storage
|
|
152
173
|
```
|
|
153
174
|
|
|
154
|
-
|
|
175
|
+
Notably, the parser runs inside an app thread. Puma's concurrency model is "many app threads, each doing one full request (parse plus app plus write) at a time". Under MRI's GVL this is genuine concurrency but not parallelism; at any given moment at most one of those threads is executing Ruby, and the others are either blocked on I/O (which releases the GVL and lets another thread proceed) or waiting their turn. Puma leans on the fact that most Rack apps spend most of their time blocked on the database or an external API, where threads do effectively overlap.
|
|
155
176
|
|
|
156
177
|
## Part II: Raptor
|
|
157
178
|
|
|
@@ -175,7 +196,21 @@ Master-to-worker communication does not use pipes. Every worker writes its stats
|
|
|
175
196
|
|
|
176
197
|
On Linux, each worker pins itself to a distinct CPU via `sched_setaffinity` when the worker count fits within the process's allowed CPU set, so it stays on one core and its L1/L2 caches stay warm. When workers outnumber available CPUs the pin is skipped and the kernel scheduler manages placement.
|
|
177
198
|
|
|
178
|
-
###
|
|
199
|
+
### Refork
|
|
200
|
+
|
|
201
|
+
Long-lived workers accumulate mutations. Every allocation, every YJIT-compiled block, every gem that lazy-loads on first use, every cache that populates over the first few thousand requests, all of it dirties pages that were shared copy-on-write with the master. After a few hours of traffic the CoW savings are mostly gone and each worker's RSS climbs toward its steady-state size. Reforking off a warmed source is the standard answer.
|
|
202
|
+
|
|
203
|
+
Raptor's version is driven by `refork_after`, an `Integer`, `Array<Integer>`, or `nil`. When any worker's request count crosses the next threshold, the master picks the most-experienced booted worker in the current phase and sends it `SIGURG`. That worker breaks out of its main loop, runs any `before_refork` hooks (typically to close database connections and any other file descriptors the fresh workers should not inherit), drains in-flight requests, and then transitions into a seed loop instead of exiting. From that point on the seed's only job is to fork replacement workers on the master's request; it never accepts another connection. The master then rolls the remaining workers through a phased restart, forking each replacement from the seed.
|
|
204
|
+
|
|
205
|
+
Successive thresholds promote fresh seeds. When the next threshold trips, the master picks a new candidate from the currently-serving generation, terminates the previous seed, and starts the same drain-and-promote cycle on the new candidate. Each generation's fork source is therefore a worker that has served enough traffic to warm its VM, its YJIT caches, and its lazy-loaded gems, and the previous generation's seed is retired the moment it is no longer needed. Passing an array to `refork_after` (e.g. `[1000, 5000, 20_000]`) sets successive thresholds; the last one repeats.
|
|
206
|
+
|
|
207
|
+
Master and seed communicate over two pipes opened before the initial fork. The fork pipe (`@fork_r`, `@fork_w`) carries an 8-byte packed `[slot_index, phase]` from master to seed for each replacement to fork. The response pipe (`@resp_r`, `@resp_w`) carries a 4-byte packed pid back the other way, plus a one-shot 4-byte zero the seed writes when it enters its loop so the master knows drain is complete. The master polls the ready marker non-blockingly on the supervision loop; the phased restart that fills the vacated slot and cycles the remaining workers only starts once the marker arrives (or the drain deadline expires, in which case the master clears the seed reference and falls back to direct forks).
|
|
208
|
+
|
|
209
|
+
Workers forked by the seed are not the master's direct children. Before spawning anything the master calls `PR_SET_CHILD_SUBREAPER` (via a small `Raptor::Subreaper` C extension wrapping `prctl`), so seed-forked children reparent to the master immediately and `Process.wait2` sees them like any other child. When a seed is retired on the next promotion its still-running descendants stay attached to the master through the same subreaper bit. Refork depends on this: on platforms without `PR_SET_CHILD_SUBREAPER` — everything except Linux — `refork_after` defaults to `nil` and is ignored with a warning if the user sets it anyway.
|
|
210
|
+
|
|
211
|
+
The design is Pitchfork's, adapted for Raptor's process model. [Pitchfork](https://github.com/Shopify/pitchfork) was the first Ruby server built around `PR_SET_CHILD_SUBREAPER` and multi-generation mould promotion; Puma's shipping `fork_worker` keeps worker 0 serving traffic alongside its fork-server duties, and Instacart's [`mold_worker` PR #3643](https://github.com/puma/puma/pull/3643) is Puma catching up to the same shape Pitchfork established. Raptor's seed matches the Pitchfork shape: pure fork source, promoted from a warm worker, retired when the next generation takes over.
|
|
212
|
+
|
|
213
|
+
### Threading model
|
|
179
214
|
|
|
180
215
|
This is where Raptor diverges dramatically. Inside a worker there are five kinds of concurrent activity:
|
|
181
216
|
|
|
@@ -188,11 +223,13 @@ This is where Raptor diverges dramatically. Inside a worker there are five kinds
|
|
|
188
223
|
|
|
189
224
|
That is a lot of moving parts. Let us go through why.
|
|
190
225
|
|
|
191
|
-
**Why Ractors for parsing.** Ractors are Ruby's answer to true parallelism. Multiple Ractors can execute Ruby code simultaneously on different OS threads, each with its own GVL. But Ractors are heavily restricted
|
|
226
|
+
**Why Ractors for parsing.** Ractors are Ruby's answer to true parallelism. Multiple Ractors can execute Ruby code simultaneously on different OS threads, each with its own GVL. But Ractors are heavily restricted. They can only share frozen data, they cannot access most global mutable state, and code inside a Ractor cannot use most existing gems (which universally assume shared-state semantics).
|
|
192
227
|
|
|
193
228
|
For a web server, this restriction turns out to be almost exactly right for HTTP parsing. Parsing a request is CPU-bound (tokenising bytes, uppercasing header names, decoding chunked bodies), it does not need to touch any global state, and it produces a result (a hash) that can be safely frozen and handed off. The native HTTP/1 parser (`raptor_http.c`) is declared `rb_ext_ractor_safe(true)`: it holds no per-parser Ruby state in the extension itself, and it writes only into the caller-supplied env hash. Same for the HTTP/2 parser plus HPACK.
|
|
194
229
|
|
|
195
|
-
The
|
|
230
|
+
The HTTP/1 parser also pre-interns the ~40 most common header keys (`HTTP_HOST`, `HTTP_USER_AGENT`, the `HTTP_ACCEPT_*` family, `CONTENT_LENGTH`, `HTTP_X_FORWARDED_*`, the `HTTP_SEC_FETCH_*` client hints, and so on) once at load time. During parsing, a `memcmp` lookup against that table returns the shared frozen `VALUE` for known keys and falls back to `rb_enc_interned_str` for the rest. Every request's env hash therefore reuses the same String object for its header names, which both skips per-request allocation and lets Ruby's hash lookup use the interned key's cached hash code.
|
|
231
|
+
|
|
232
|
+
The upshot is that while your Rack app runs on regular threads under the GVL (so your app does not need to be Ractor-safe), the protocol-level work runs in parallel across Ractors. Under heavy load with lots of small requests, the GVL contention that would otherwise dominate parsing simply is not there.
|
|
196
233
|
|
|
197
234
|
**How the Ractor pool actually works.** Raptor uses the `ractor-pool` gem, which is another one of my libraries. The pool has one coordinator Ractor and M pipeline Ractors. When a pipeline Ractor is idle, it sends itself back to the coordinator via `coordinator.send(Ractor.current, move: true)`. When work arrives at the coordinator, it either forwards it to a waiting Ractor (if any) or queues it. This coordinator-dispatch pattern guarantees that no Ractor sits idle while there is work. Results flow back through a shared `Ractor::Port` (a many-to-one channel added in recent Ruby versions and stable in 4.0) to a Ruby-side collector thread. If `M == 1` the coordinator is skipped and work goes straight to the single pipeline Ractor; this is the default because a single Ractor already parses in parallel with the app threads and adds enough headroom for typical workloads.
|
|
198
235
|
|
|
@@ -216,9 +253,11 @@ next if @reactor.backlog >= backpressure_threshold
|
|
|
216
253
|
|
|
217
254
|
where `@reactor.backlog` is `thread_pool.queue_size + thread_pool.active_count`. If the total load on the thread pool is at 120% of the pool size, this worker stops accepting until it drains. `MIN_BACKPRESSURE_THRESHOLD` is 8, so small pools (say, 3 threads) trip backpressure at 8 concurrent items rather than at 120% of a very small number; the floor keeps saturated workers signaling early so the load-aware dispatcher (below) has time to route around them.
|
|
218
255
|
|
|
219
|
-
Because Raptor is always in cluster mode and every worker listens in the same `SO_REUSEPORT` group, load balancing across workers happens at the kernel level. On Linux, Raptor attaches a small BPF program to the reuseport group. Each worker binds its own listener registered in a sockmap, and a dedicated reporter thread publishes the worker's current reactor backlog into a loads map at 100 Hz. The BPF program consults the map on every incoming connection and routes it to the least-loaded worker,
|
|
256
|
+
Because Raptor is always in cluster mode and every worker listens in the same `SO_REUSEPORT` group, load balancing across workers happens at the kernel level. On Linux, Raptor attaches a small BPF program to the reuseport group. Each worker binds its own listener registered in a sockmap, and a dedicated reporter thread publishes the worker's current reactor backlog into a loads map at 100 Hz. The BPF program consults the map on every incoming connection and routes it to the least-loaded worker; when all workers report equal load (typical of a fresh accept burst), the program falls back to the connection's 4-tuple hash, which spreads accepts more evenly than a random draw across a small worker count. If the `libbpf-ruby` gem is not installed or the BPF object has not been compiled, Raptor falls back silently to the default four-tuple-hash routing; if the kernel refuses the program, startup raises. Either way, if a worker is saturated and stops calling `accept`, other workers pick up the slack.
|
|
220
257
|
|
|
221
|
-
The
|
|
258
|
+
The BPF-based approach was inspired by [a comment](https://github.com/puma/puma/issues/3934#issuecomment-4356462590) by John Hawthorn ([@jhawthorn](https://github.com/jhawthorn)) on a Puma issue about `EPOLLEXCLUSIVE`, where he floated `SO_ATTACH_REUSEPORT_EBPF` as a way to route each connection to the least-busy worker.
|
|
259
|
+
|
|
260
|
+
The reactor is again an `NIO::Selector` loop. Two things make it different from Puma's:
|
|
222
261
|
|
|
223
262
|
1. **Read strategy.** When a socket is readable, the reactor does one `read_nonblock(64KB)` right there in the reactor thread, updates the buffered state for that connection, and only then decides what to do. If the request is not yet complete, the state stays in the reactor and awaits more data. If it is complete (headers parsed, body received), the state is pushed to the Ractor pool for parsing (or, for HTTP/2, straight to the parser). The reactor does not try to parse; it does the I/O and hands off the raw buffer.
|
|
224
263
|
|
|
@@ -279,10 +318,10 @@ From there the shape is similar to HTTP/1.1:
|
|
|
279
318
|
|
|
280
319
|
1. Reactor reads frames.
|
|
281
320
|
2. The HTTP/2 parser (native C, with an HPACK decoder using a static Huffman table) parses the frames in the Ractor pool.
|
|
282
|
-
3. Completed requests (once HEADERS
|
|
321
|
+
3. Completed requests (once `HEADERS` and `DATA` are complete for a stream) go to the thread pool as separate work items. **A single connection can be servicing many streams in parallel across the thread pool.**
|
|
283
322
|
4. Each stream's response is written back through the connection's `Writer`, which serialises frame writes across threads without a mutex.
|
|
284
323
|
|
|
285
|
-
The Writer is worth a paragraph. Naive per-connection writing would use a mutex: acquire, write, release. Contention grows with concurrent streams. Raptor's Writer stores the "pending frames" queue in an `Atom` whose value is either `:idle` (nobody is writing) or an array of frames waiting to go out. A thread that wants to write does a CAS:
|
|
324
|
+
The `Writer` is worth a paragraph. Naive per-connection writing would use a mutex: acquire, write, release. Contention grows with concurrent streams. Raptor's `Writer` stores the "pending frames" queue in an `Atom` whose value is either `:idle` (nobody is writing) or an array of frames waiting to go out. A thread that wants to write does a CAS:
|
|
286
325
|
|
|
287
326
|
- If current value is `:idle`, the thread claims the writer by CAS-ing to its own array of frames, then loops draining any additional frames other threads have appended.
|
|
288
327
|
- If current value is an array (someone is already writing), the thread CAS-appends its frames and returns immediately; the current writer will pick them up and flush them.
|
|
@@ -341,7 +380,7 @@ flowchart TB
|
|
|
341
380
|
ATP -->|"app.call + write"| KA
|
|
342
381
|
KA -->|"no, close"| CLS["close socket"]
|
|
343
382
|
KA -->|"yes"| EAG
|
|
344
|
-
EAG
|
|
383
|
+
EAG -.->|"bytes within 1ms, parse+dispatch on same thread"| ATP
|
|
345
384
|
EAG -->|"no bytes, reactor.persist"| RCT
|
|
346
385
|
RCT -->|"timeout expired"| TO["write 408, close"]
|
|
347
386
|
|
|
@@ -364,7 +403,7 @@ flowchart TB
|
|
|
364
403
|
class SHM storage
|
|
365
404
|
```
|
|
366
405
|
|
|
367
|
-
The critical structural difference from Puma
|
|
406
|
+
The critical structural difference from Puma is that parsing is not on the app thread. It is on the Ractor pool. The app thread only does the Rack call and the response write. This decouples the two costs and lets Ruby actually use more than one CPU for the protocol work.
|
|
368
407
|
|
|
369
408
|
## Part III: Head to head
|
|
370
409
|
|
|
@@ -388,7 +427,7 @@ At a moderate 100 concurrent connections this is a rounding error. At 1000+ (whi
|
|
|
388
427
|
|
|
389
428
|
**Puma.** Standard `Mutex` + `ConditionVariable` + Array-as-queue. Every enqueue takes the mutex, wakes a waiter. Every dequeue takes the mutex to check the queue. Under contention, the mutex is a serialisation point.
|
|
390
429
|
|
|
391
|
-
**Raptor.** CAS on an Atom holding a frozen `{in:, out:, count:}` hash (Banker's queue). Enqueue is a single CAS to prepend to the in-stack. Dequeue is a single CAS to pop from the out-stack (or, if empty, atomically flip in/out). Metrics (queue length, active count) are lock-free reads.
|
|
430
|
+
**Raptor.** CAS on an `Atom` holding a frozen `{in:, out:, count:}` hash (Banker's queue). Enqueue is a single CAS to prepend to the in-stack. Dequeue is a single CAS to pop from the out-stack (or, if empty, atomically flip in/out). Metrics (queue length, active count) are lock-free reads.
|
|
392
431
|
|
|
393
432
|
The condvar is still there for parking idle threads (spinning would burn CPU), but the hot path when the queue has items is lock-free.
|
|
394
433
|
|
|
@@ -398,11 +437,11 @@ Under moderate load these look equivalent. Under high concurrency (many threads
|
|
|
398
437
|
|
|
399
438
|
**Puma.** After a response, if the connection is keep-alive and there are already buffered bytes for the next request (`has_back_to_back_requests?`) and there is a spare app thread, loop inline. Otherwise, if `eagerly_finish` (a single non-blocking read attempt) returns true, either loop inline (if spare threads) or hand back to the thread pool (`@thread_pool << client`). Otherwise, back to the reactor with `@persistent_timeout`.
|
|
400
439
|
|
|
401
|
-
**Raptor.** After a response, the app thread does `socket.wait_readable(0.001)`, waiting up to 1ms for bytes. If bytes arrive,
|
|
440
|
+
**Raptor.** After a response, the app thread does `socket.wait_readable(0.001)`, waiting up to 1ms for bytes. If bytes arrive, it parses the next request inline. If the thread pool queue is at least as deep as the pool, the parsed request is handed back to the pool so other threads share the load; otherwise the same thread dispatches it inline. If no bytes arrive, `reactor.persist` and return.
|
|
402
441
|
|
|
403
|
-
The difference is subtle but the numbers show it up. Real-world clients pipelining requests often send the next request 100-500 microseconds after the previous response completes. Puma's `eagerly_finish` catches the case where bytes are already in the socket buffer; Raptor's `wait_readable(1ms)` catches both the "already buffered" case and the "arriving very shortly" case. And Raptor
|
|
442
|
+
The difference is subtle but the numbers show it up. Real-world clients pipelining requests often send the next request 100-500 microseconds after the previous response completes. Puma's `eagerly_finish` catches the case where bytes are already in the socket buffer; Raptor's `wait_readable(1ms)` catches both the "already buffered" case and the "arriving very shortly" case. And Raptor always parses the next request on the response-writing thread, so the parse itself never crosses a thread boundary.
|
|
404
443
|
|
|
405
|
-
This is where most of Raptor's keep-alive edge comes from.
|
|
444
|
+
This is where most of Raptor's keep-alive edge comes from. Subsequent requests are always parsed and dispatched without a reactor round-trip: the response-writing thread either continues serving that connection inline (when the pool queue is shallower than the pool) or hands the parsed request back to the pool for another thread to pick up (when it is not). Puma's app threads do the same when they can, but the coordination overhead between thread pool and reactor for the transitions costs meaningfully more.
|
|
406
445
|
|
|
407
446
|
### Backpressure
|
|
408
447
|
|
|
@@ -420,10 +459,12 @@ The performance difference here is negligible. Stats writing is one syscall per
|
|
|
420
459
|
|
|
421
460
|
### HTTP/2
|
|
422
461
|
|
|
423
|
-
**Puma.** Not implemented.
|
|
462
|
+
**Puma.** Not implemented. Puma's [position](https://github.com/puma/puma/issues/2697) is that HTTP/2 belongs at the edge (nginx, Caddy, ALB), which terminates it and speaks HTTP/1.1 to the app server. That's a reasonable call for the deployments Puma is aimed at, and it's where most Rails production actually sits.
|
|
424
463
|
|
|
425
464
|
**Raptor.** Native C parser plus HPACK, per-stream flow control, lock-free frame writer, stream multiplexing over a single connection. Same request path as HTTP/1.1 once a request is complete: it enters the same thread pool. Under HTTP/2, a single client connection can be issuing many concurrent requests, and Raptor services all of them in parallel on the same thread pool.
|
|
426
465
|
|
|
466
|
+
Whether that matters depends on your setup. If you terminate TLS at an edge proxy that already speaks HTTP/2, both servers see HTTP/1.1 and it doesn't matter which of them you pick on this axis. If you're building an all-Ruby stack with no proxy in front, running gRPC or similar all the way through, or measuring the app server itself, HTTP/2 support is where Raptor and Puma stop being comparable.
|
|
467
|
+
|
|
427
468
|
At the throughput numbers the benchmark shows, with only a handful of concurrent connections each multiplexing many streams, Raptor is exercising the HTTP/2 multiplexing hard: many in-flight streams sharing one socket, all writing responses through the same connection. The writer's CAS-based coordination avoids the mutex contention that would otherwise cap throughput.
|
|
428
469
|
|
|
429
470
|
### Response writing
|
|
@@ -434,6 +475,10 @@ On the HTTP/1.1 path, Raptor has a small `writev(2)` C extension (`Raptor::Vecto
|
|
|
434
475
|
|
|
435
476
|
HTTP/1.1 responses also reuse a per-thread String buffer for the status line and headers rather than allocating one per response. The buffer grows once to fit the largest response the thread has served and stays that size afterwards, so subsequent responses on that thread skip the allocation entirely.
|
|
436
477
|
|
|
478
|
+
Chunked responses with an array or file body accumulate chunk-framed writes into the response buffer up to 512KB before flushing to the socket, collapsing what would be N `write` syscalls for an N-chunk body into a handful. Enumerable bodies (SSE, long-polling, per-line log tailing) still flush every chunk, so streaming semantics are preserved.
|
|
479
|
+
|
|
480
|
+
Around the response boundary, HTTP/1.1 also amortises the common per-request allocations. A frozen Rack env template with the static keys (`rack.version`, `rack.hijack?`, `SCRIPT_NAME`, `QUERY_STRING`, `SERVER_SOFTWARE`) is duped per request, so the app-thread env build only writes the dynamic keys. Response header serialisation appends directly onto the response buffer instead of allocating intermediate `Array` wraps and `flat_map` products for each header value.
|
|
481
|
+
|
|
437
482
|
### Keep-alive request by request
|
|
438
483
|
|
|
439
484
|
Because the keep-alive fast path is where Raptor's throughput edge shows up, here is each server processing three back-to-back requests on the same connection. Drawn separately so the participant columns stay wide enough to read. Compare the number of boundaries each request has to cross.
|
|
@@ -476,16 +521,12 @@ sequenceDiagram
|
|
|
476
521
|
autonumber
|
|
477
522
|
participant Client
|
|
478
523
|
participant RS as Server thread
|
|
479
|
-
participant RR as Reactor thread
|
|
480
|
-
participant RC as Pipeline Ractor
|
|
481
524
|
participant RP as App thread
|
|
482
525
|
|
|
483
526
|
Note over Client,RP: Request 1, initial
|
|
484
527
|
Client->>RS: SYN, request bytes
|
|
485
|
-
RS->>
|
|
486
|
-
|
|
487
|
-
RC->>RC: parse on own Ractor GVL
|
|
488
|
-
RC->>RP: dispatch complete request
|
|
528
|
+
RS->>RS: eager_accept, read + parse inline
|
|
529
|
+
RS->>RP: push proc to thread pool
|
|
489
530
|
RP->>Client: response 1
|
|
490
531
|
|
|
491
532
|
Note over Client,RP: Request 2, pipelined a few hundred us later
|
|
@@ -506,27 +547,33 @@ Puma has to bounce Request 3 through the reactor and back through the mutex-prot
|
|
|
506
547
|
|
|
507
548
|
## Part IV: What Raptor's design buys you
|
|
508
549
|
|
|
509
|
-
### IO-bound work,
|
|
550
|
+
### IO-bound work, where Falcon wins and Raptor clearly beats Puma
|
|
551
|
+
|
|
552
|
+
On the IO-bound benchmark profile, each request does 5 to 10 short sleeps interleaved with small CPU work, simulating a request that makes several DB or cache calls throughout its lifetime. The bottleneck is how many requests a worker can keep in flight while they wait on IO. Raptor and Puma cap out at their 12 total threads (4 workers × 3), so with 48 client connections the extra 36 sit in the pool queue at any given moment. Falcon spawns a fiber per connection and cooperatively yields on every sleep, so all 48 requests can be in flight simultaneously. That gap shows up as roughly 3-4x throughput and much lower p95 for Falcon.
|
|
553
|
+
|
|
554
|
+
Between the thread-based servers, Raptor holds a clear lead over Puma on both throughput and p95. The eager accept path, writev-batched responses, and lock-free work queue all shave a bit of per-request time, and those savings stack.
|
|
555
|
+
|
|
556
|
+
Real applications that spend most of their time waiting on a database or an upstream service look like this. If your app is IO-heavy and you're free to adopt the `async` ecosystem, Falcon is the interesting comparison there, not Raptor or Puma.
|
|
510
557
|
|
|
511
|
-
|
|
558
|
+
### CPU-bound HTTP/1.1, where the pool model earns its keep
|
|
512
559
|
|
|
513
|
-
|
|
560
|
+
On the CPU-bound benchmark profile, each POST request accepts a small JSON body and builds a JSON response in 3 to 5 chunks totalling 450 to 1500 items, with sub-100µs sleeps between chunks. It's roughly 95% CPU by wall time, so fibers can't multiplex their way to an advantage. The CPU work happens under a single Ruby VM regardless of concurrency model.
|
|
514
561
|
|
|
515
|
-
|
|
562
|
+
**Without keep-alive**, every request opens a fresh TCP connection, gets parsed, dispatched, served, and closes. All three servers land within a few percent of each other on throughput. Raptor's p95 is meaningfully worse than Puma's and Falcon's here; the extra hop through the Ractor pipeline shows up as tail latency when connection setup and teardown are also on the critical path.
|
|
516
563
|
|
|
517
|
-
|
|
564
|
+
**With keep-alive**, Raptor pulls ahead of both Puma and Falcon on throughput and p95. Connection setup is amortised, the eager keep-alive loop keeps subsequent requests on the same app thread, and the Ractor-parallel parser earns its cost when many small requests are in flight. This is the shape of workload Raptor's design was optimised for.
|
|
518
565
|
|
|
519
|
-
|
|
566
|
+
### HTTP/2, when it matters
|
|
520
567
|
|
|
521
|
-
|
|
568
|
+
Puma doesn't implement HTTP/2, and most Rails production terminates HTTP/2 at nginx or a similar edge proxy before it reaches the app server. If that describes your stack, Raptor's HTTP/2 support isn't going to help you. Both servers see HTTP/1.1 from the proxy and the throughput numbers above are what actually matter. Puma's [position](https://github.com/puma/puma/issues/2697) is that this is where h2 belongs, and it's a reasonable one.
|
|
522
569
|
|
|
523
|
-
Raptor's
|
|
570
|
+
Where Raptor's HTTP/2 support does matter is the all-Ruby stack. No proxy in front, TLS terminated at the app, clients speaking h2 all the way through (`curl --http2`, gRPC-Ruby transports, HTTP/2-only SDKs, direct browser connections). In that setup, Puma silently falls back to HTTP/1.1 and you lose multiplexing, header compression, and prioritisation.
|
|
524
571
|
|
|
525
|
-
|
|
572
|
+
Falcon also speaks HTTP/2 natively, so it's the interesting comparison there rather than Puma. The shape of the h2 result follows the h1 shape. On CPU-bound h2 Raptor holds a small edge on throughput and a meaningful one on p95, for the same reasons it wins CPU-bound h1 keep-alive. Parsing runs in parallel with the app, the lock-free writer serialises frames without a mutex, and per-stream flow-control atoms don't contend. On IO-bound h2 Falcon wins by a wide margin, again because fibers can keep every stream in flight simultaneously while Raptor's thread pool caps concurrency.
|
|
526
573
|
|
|
527
|
-
|
|
574
|
+
Raptor's h2 IO numbers still show some run-to-run variance where h1 doesn't. With only 16 physical connections spread across four workers, even the BPF program's hash-based tie-break can leave one worker over-subscribed, and the concentrated worker becomes the bottleneck for that whole run. h2 CPU stays tight because once workers report distinguishable load the load-aware routing has enough signal to place new connections cleanly regardless of tie-break behaviour.
|
|
528
575
|
|
|
529
|
-
Beyond
|
|
576
|
+
Beyond that, Raptor's HTTP/2 CPU-bound throughput in the benchmark is on the same order as its HTTP/1.1 keep-alive throughput on the same profile, despite each connection multiplexing dozens of concurrent streams into a single socket. That only happens if the per-stream coordination is essentially free. The lock-free `Writer` and flow-control atoms are doing real work here. If they used mutexes, throughput would be capped by lock contention rather than by CPU.
|
|
530
577
|
|
|
531
578
|
## Part V: What Raptor gives up
|
|
532
579
|
|
|
@@ -536,22 +583,4 @@ No design comes free. Two disclosures matter most.
|
|
|
536
583
|
|
|
537
584
|
**Ruby version.** Raptor requires Ruby 4.0 because it depends on `Ractor::Port` and on Ractor internals having stabilised. Puma works on 3.0 and up. If you need to support older Ruby, Puma wins by default.
|
|
538
585
|
|
|
539
|
-
A handful of smaller trade-offs are worth naming briefly. Raptor pays some minimum-latency overhead per request because of the Ractor crossing, so at extremely low concurrency Puma's simpler flow can edge ahead; the eager-parse path on the server thread hides most of this in practice. Raptor's core dependencies (`ractor-pool`, `atomic-ruby`, `red-black-tree`, `mmap-ruby`, `libbpf-ruby`) are libraries I wrote specifically to make it work, which is either "purpose-built" or "narrower testing surface" depending on how you look at it. Debugging is harder because control flow crosses Ractor and thread boundaries, so tracing a request end-to-end means stitching several stack traces together. Raptor has no single-process mode; on a single-CPU container it wastes a small amount of coordination overhead.
|
|
540
|
-
|
|
541
|
-
## Part VI: The design in one line each
|
|
542
|
-
|
|
543
|
-
If you had to compress this whole document into a handful of one-line framings, these are the ones that carry the most:
|
|
544
|
-
|
|
545
|
-
- **"Puma is a thread pool with a reactor bolted on. Raptor is a pipeline with each stage on its own scheduling primitive."** Puma's core abstraction is the thread pool; the reactor exists to feed it. Raptor's core abstraction is the pipeline; the thread pool is one stage in it.
|
|
546
|
-
|
|
547
|
-
- **"The GVL is not the enemy. Serialising parsing behind the GVL is the enemy."** Ruby threads are fine for I/O-bound work; the app thread spends most of its time waiting on the database. What you cannot afford is spending CPU cycles under the GVL doing work that does not need the GVL, like tokenising HTTP headers.
|
|
548
|
-
|
|
549
|
-
- **"Ractors are impractical for user code but perfect for protocol code."** The very things that make Ractors annoying for application code (frozen data only, no shared mutable state) are exactly what you want in a parser: a pure function from bytes to headers with no side effects.
|
|
550
|
-
|
|
551
|
-
- **"Lock-free is not always faster, but it is always less variable."** A CAS-based queue does not out-perform a mutex in every scenario, but its worst-case latency is much better. Under contention, mutex-based code can suffer thundering herd wakeups and cache-line bouncing that a CAS design avoids.
|
|
552
|
-
|
|
553
|
-
- **"The keep-alive edge is not one big win, it is many small wins that stack."** Eager keep-alive, RBT timeouts, CAS work queue, Ractor parsing, lock-free backpressure. Each adds a few percent and stacked they produce a measurable lead.
|
|
554
|
-
|
|
555
|
-
- **"HTTP/2 is not a nice-to-have; it changes the arithmetic."** With HTTP/2, one client connection can be doing dozens of requests concurrently. Suddenly every serialisation point in the server becomes a bottleneck. The lock-free primitives Raptor uses for HTTP/1.1 are not optional for HTTP/2; they are structural.
|
|
556
|
-
|
|
557
|
-
- **"This is what Ruby 4 lets you do."** `Ractor::Port`, `Ractor.shareable_proc`, `rb_ext_ractor_safe`, native-extension Ractor safety. These are all recent additions. Raptor is essentially a demo of what becomes practical when you have them.
|
|
586
|
+
A handful of smaller trade-offs are worth naming briefly. Raptor pays some minimum-latency overhead per request because of the Ractor crossing, so at extremely low concurrency Puma's simpler flow can edge ahead; the eager-parse path on the server thread hides most of this in practice. Raptor's core dependencies (`ractor-pool`, `atomic-ruby`, `red-black-tree`, `mmap-ruby`, `libbpf-ruby`) are libraries I wrote specifically to make it work, which is either "purpose-built" or "narrower testing surface" depending on how you look at it. Debugging is harder because control flow crosses Ractor and thread boundaries, so tracing a request end-to-end means stitching several stack traces together. Raptor has no single-process mode; on a single-CPU container it wastes a small amount of coordination overhead.
|
|
@@ -20,7 +20,8 @@ struct {
|
|
|
20
20
|
} loads SEC(".maps");
|
|
21
21
|
|
|
22
22
|
// Routes each incoming connection to the worker with the lowest backlog,
|
|
23
|
-
//
|
|
23
|
+
// falling back to the connection's 4-tuple hash when loads are equal so
|
|
24
|
+
// bursts of accepts spread across workers instead of clustering by chance.
|
|
24
25
|
SEC("sk_reuseport")
|
|
25
26
|
int select_least_loaded(struct sk_reuseport_md *ctx) {
|
|
26
27
|
__u32 count_key = 0;
|
|
@@ -55,7 +56,7 @@ int select_least_loaded(struct sk_reuseport_md *ctx) {
|
|
|
55
56
|
}
|
|
56
57
|
}
|
|
57
58
|
|
|
58
|
-
__u32 chosen_idx = (min_load == max_load) ? (
|
|
59
|
+
__u32 chosen_idx = (min_load == max_load) ? (ctx->hash % num_workers) : min_idx;
|
|
59
60
|
bpf_sk_select_reuseport(ctx, &socks, &chosen_idx, 0);
|
|
60
61
|
return SK_PASS;
|
|
61
62
|
}
|
|
@@ -121,6 +121,21 @@ static inline void upcase_header_char(char *c) {
|
|
|
121
121
|
*c = '_';
|
|
122
122
|
}
|
|
123
123
|
|
|
124
|
+
static int contains_chunked(const char *value, long len) {
|
|
125
|
+
static const char chunked[] = "chunked";
|
|
126
|
+
static const long chunked_len = sizeof(chunked) - 1;
|
|
127
|
+
|
|
128
|
+
if (len < chunked_len) return 0;
|
|
129
|
+
for (long start = 0; start + chunked_len <= len; start++) {
|
|
130
|
+
long i;
|
|
131
|
+
for (i = 0; i < chunked_len; i++) {
|
|
132
|
+
if ((char)tolower((unsigned char)value[start + i]) != chunked[i]) break;
|
|
133
|
+
}
|
|
134
|
+
if (i == chunked_len) return 1;
|
|
135
|
+
}
|
|
136
|
+
return 0;
|
|
137
|
+
}
|
|
138
|
+
|
|
124
139
|
static const int raptor_parser_start = 1;
|
|
125
140
|
static const int raptor_parser_first_final = 46;
|
|
126
141
|
static const int raptor_parser_error = 0;
|
|
@@ -392,7 +407,7 @@ tr26:
|
|
|
392
407
|
parser->flags |= FLAG_HAS_BODY;
|
|
393
408
|
} else if (needs_http_prefix && parser->field_len == 17 &&
|
|
394
409
|
memcmp(field_ptr, "TRANSFER_ENCODING", 17) == 0) {
|
|
395
|
-
if (
|
|
410
|
+
if (contains_chunked(value_ptr, value_real_len)) {
|
|
396
411
|
parser->flags |= FLAG_CHUNKED | FLAG_HAS_BODY;
|
|
397
412
|
parser->content_len = 0;
|
|
398
413
|
}
|
|
@@ -452,7 +467,7 @@ tr29:
|
|
|
452
467
|
parser->flags |= FLAG_HAS_BODY;
|
|
453
468
|
} else if (needs_http_prefix && parser->field_len == 17 &&
|
|
454
469
|
memcmp(field_ptr, "TRANSFER_ENCODING", 17) == 0) {
|
|
455
|
-
if (
|
|
470
|
+
if (contains_chunked(value_ptr, value_real_len)) {
|
|
456
471
|
parser->flags |= FLAG_CHUNKED | FLAG_HAS_BODY;
|
|
457
472
|
parser->content_len = 0;
|
|
458
473
|
}
|
|
@@ -1160,7 +1175,10 @@ case 45:
|
|
|
1160
1175
|
|
|
1161
1176
|
done:
|
|
1162
1177
|
parser->cs = cs;
|
|
1163
|
-
parser->nread = p - buffer;
|
|
1178
|
+
parser->nread = (parser->flags & FLAG_FINISHED) ? parser->body_start : (size_t)(p - buffer);
|
|
1179
|
+
|
|
1180
|
+
if (parser->nread > MAX_HEADER_LENGTH)
|
|
1181
|
+
rb_raise(eHttpParserError, "request header too long");
|
|
1164
1182
|
|
|
1165
1183
|
assert(p <= pe && "buffer overflow after parsing execute");
|
|
1166
1184
|
assert(parser->nread <= len && "nread longer than length");
|
|
@@ -1267,6 +1285,12 @@ static VALUE parser_has_body_p(VALUE self) {
|
|
|
1267
1285
|
return raptor_parser_has_body(parser) ? Qtrue : Qfalse;
|
|
1268
1286
|
}
|
|
1269
1287
|
|
|
1288
|
+
static VALUE parser_chunked_p(VALUE self) {
|
|
1289
|
+
raptor_parser *parser;
|
|
1290
|
+
TypedData_Get_Struct(self, raptor_parser, &parser_type, parser);
|
|
1291
|
+
return raptor_parser_is_chunked(parser) ? Qtrue : Qfalse;
|
|
1292
|
+
}
|
|
1293
|
+
|
|
1270
1294
|
static VALUE parser_content_length(VALUE self) {
|
|
1271
1295
|
raptor_parser *parser;
|
|
1272
1296
|
TypedData_Get_Struct(self, raptor_parser, &parser_type, parser);
|
|
@@ -1322,6 +1346,7 @@ RUBY_FUNC_EXPORTED void Init_raptor_http(void) {
|
|
|
1322
1346
|
rb_define_method(cHttpParser, "execute", parser_execute, 3);
|
|
1323
1347
|
rb_define_method(cHttpParser, "finished?", parser_finished_p, 0);
|
|
1324
1348
|
rb_define_method(cHttpParser, "has_body?", parser_has_body_p, 0);
|
|
1349
|
+
rb_define_method(cHttpParser, "chunked?", parser_chunked_p, 0);
|
|
1325
1350
|
rb_define_method(cHttpParser, "content_length", parser_content_length, 0);
|
|
1326
1351
|
rb_define_method(cHttpParser, "nread", parser_nread, 0);
|
|
1327
1352
|
rb_define_method(cHttpParser, "reset", parser_reset, 0);
|
|
@@ -10,6 +10,7 @@
|
|
|
10
10
|
|
|
11
11
|
#ifdef __linux__
|
|
12
12
|
#include <sched.h>
|
|
13
|
+
#include <sys/prctl.h>
|
|
13
14
|
#include <unistd.h>
|
|
14
15
|
#endif
|
|
15
16
|
|
|
@@ -82,18 +83,31 @@ static VALUE raptor_native_cpu_count(VALUE self) {
|
|
|
82
83
|
#endif
|
|
83
84
|
}
|
|
84
85
|
|
|
86
|
+
static VALUE raptor_native_enable_subreaper(VALUE self) {
|
|
87
|
+
(void)self;
|
|
88
|
+
#if defined(__linux__) && defined(PR_SET_CHILD_SUBREAPER)
|
|
89
|
+
if (prctl(PR_SET_CHILD_SUBREAPER, 1, 0, 0, 0) < 0) rb_sys_fail("prctl");
|
|
90
|
+
return Qtrue;
|
|
91
|
+
#else
|
|
92
|
+
return Qfalse;
|
|
93
|
+
#endif
|
|
94
|
+
}
|
|
95
|
+
|
|
85
96
|
RUBY_FUNC_EXPORTED void Init_raptor_native(void) {
|
|
86
97
|
rb_ext_ractor_safe(true);
|
|
87
98
|
|
|
88
99
|
VALUE mRaptor = rb_define_module("Raptor");
|
|
89
|
-
VALUE mVectorIO = rb_define_module_under(mRaptor, "VectorIO");
|
|
90
|
-
VALUE mCPU = rb_define_module_under(mRaptor, "CPU");
|
|
91
100
|
|
|
101
|
+
VALUE mVectorIO = rb_define_module_under(mRaptor, "VectorIO");
|
|
92
102
|
rb_define_singleton_method(mVectorIO, "writev_nonblock", raptor_native_writev_nonblock, 2);
|
|
93
103
|
|
|
104
|
+
VALUE mCPU = rb_define_module_under(mRaptor, "CPU");
|
|
94
105
|
rb_define_singleton_method(mCPU, "pin", raptor_native_pin_to_cpu, 1);
|
|
95
106
|
rb_define_singleton_method(mCPU, "count", raptor_native_cpu_count, 0);
|
|
96
107
|
|
|
108
|
+
VALUE mSubreaper = rb_define_module_under(mRaptor, "Subreaper");
|
|
109
|
+
rb_define_singleton_method(mSubreaper, "enable", raptor_native_enable_subreaper, 0);
|
|
110
|
+
|
|
97
111
|
eEAGAINWaitWritable = rb_const_get(rb_cIO, rb_intern("EAGAINWaitWritable"));
|
|
98
112
|
rb_global_variable(&eEAGAINWaitWritable);
|
|
99
113
|
}
|
data/lib/raptor/cli.rb
CHANGED
|
@@ -56,6 +56,11 @@ module Raptor
|
|
|
56
56
|
worker_timeout: 60,
|
|
57
57
|
worker_drain_timeout: 25,
|
|
58
58
|
worker_shutdown_timeout: 30,
|
|
59
|
+
refork_after: (RUBY_PLATFORM.include?("linux") ? 1000 : nil),
|
|
60
|
+
before_fork: [].freeze,
|
|
61
|
+
before_worker_boot: [].freeze,
|
|
62
|
+
before_worker_shutdown: [].freeze,
|
|
63
|
+
before_refork: [].freeze,
|
|
59
64
|
stats_file: "tmp/raptor.json",
|
|
60
65
|
pid_file: nil,
|
|
61
66
|
stdout_file: nil,
|
|
@@ -187,9 +192,9 @@ module Raptor
|
|
|
187
192
|
#
|
|
188
193
|
# @rbs (Array[String] argv) -> String?
|
|
189
194
|
def extract_config_path(argv)
|
|
190
|
-
argv.each_with_index do |arg,
|
|
195
|
+
argv.each_with_index do |arg, index|
|
|
191
196
|
case arg
|
|
192
|
-
when "-c", "--config" then return argv[
|
|
197
|
+
when "-c", "--config" then return argv[index + 1]
|
|
193
198
|
when /\A--config=(.*)\z/, /\A-c(.+)\z/ then return Regexp.last_match(1)
|
|
194
199
|
end
|
|
195
200
|
end
|
|
@@ -312,6 +317,10 @@ module Raptor
|
|
|
312
317
|
@options[:worker_shutdown_timeout] = timeout
|
|
313
318
|
end
|
|
314
319
|
|
|
320
|
+
opts.on("--refork-after NUM", Integer, "Refork workers from a warmed source after any worker crosses NUM requests; 0 disables (default: 1000)") do |num|
|
|
321
|
+
@options[:refork_after] = num
|
|
322
|
+
end
|
|
323
|
+
|
|
315
324
|
opts.on("--stats-file PATH", String, "Stats file path (default: tmp/raptor.json)") do |path|
|
|
316
325
|
@options[:stats_file] = path
|
|
317
326
|
end
|