raptor 0.21.0 → 0.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/README.md +37 -71
- data/docs/raptor-vs-puma.md +12 -16
- data/lib/raptor/cluster.rb +1 -0
- data/lib/raptor/detached_body.rb +293 -0
- data/lib/raptor/http1.rb +86 -16
- data/lib/raptor/http2.rb +98 -36
- data/lib/raptor/reactor.rb +856 -33
- data/lib/raptor/version.rb +1 -1
- data/lib/raptor.rb +1 -0
- data/sig/generated/raptor/detached_body.rbs +127 -0
- data/sig/generated/raptor/http1.rbs +22 -4
- data/sig/generated/raptor/http2.rbs +41 -13
- data/sig/generated/raptor/reactor.rbs +320 -13
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: fc05eb1a4a916e5094c4496f18a3d3b179869a42efc1a7fd42ccda546b46e0bc
|
|
4
|
+
data.tar.gz: 1b975f682fcbaea183226775277cf441d4996899f4348237b96bf073586b1082
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c39ee7c2d2c4f68b8015a841c3fdadfb26b42f587041d6e85e69fc930abcb9fe65ce4841e3562c31b2ff6a0ac904787ba97ca1a26dafa25ba7a0f71b0069b2a1
|
|
7
|
+
data.tar.gz: 71604474925af588347c5e82f294eca27b5d51c3dc161ece9bb63508b9ea389196173cf890be8f8ebbd030dea284fc1e0d9f4aac26650f02c05303eb09707c7c
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
|
@@ -37,7 +37,7 @@ run proc { |_env| [200, { "content-type" => "text/plain" }, ["Hello, World!"]] }
|
|
|
37
37
|
```
|
|
38
38
|
> bundle exec raptor -w 10 -t 3 hello_world.ru
|
|
39
39
|
[Raptor 72876|Main|Main] Cluster initializing:
|
|
40
|
-
[Raptor 72876|Main|Main] ├─ Version: 0.
|
|
40
|
+
[Raptor 72876|Main|Main] ├─ Version: 0.22.0
|
|
41
41
|
[Raptor 72876|Main|Main] ├─ Ruby Version: ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +YJIT +PRISM [arm64-darwin23]
|
|
42
42
|
[Raptor 72876|Main|Main] ├─ Environment: development
|
|
43
43
|
[Raptor 72876|Main|Main] ├─ Master PID: 72876
|
|
@@ -78,21 +78,11 @@ a separately supervised process or container instead.
|
|
|
78
78
|
|
|
79
79
|
## Configuration
|
|
80
80
|
|
|
81
|
-
Raptor
|
|
82
|
-
|
|
81
|
+
Raptor reads options from a Ruby config file, environment variables, and command line flags, in increasing order of
|
|
82
|
+
precedence. Run `bundle exec raptor --help` for the full flag list.
|
|
83
83
|
|
|
84
|
-
The config file
|
|
85
|
-
|
|
86
|
-
`connection:` (shared across protocols), `http1:` (HTTP/1.1-specific), and `http2:` (HTTP/2-specific).
|
|
87
|
-
|
|
88
|
-
Use an `ssl://` bind for HTTP/2 negotiated with ALPN, or an `h2c://` bind for cleartext HTTP/2 clients using prior
|
|
89
|
-
knowledge. Each `h2c://` listener accepts HTTP/2 only.
|
|
90
|
-
|
|
91
|
-
HTTP/2 applications can populate `env["raptor.response_trailers"]` with trailing response headers. Values may be
|
|
92
|
-
strings or arrays of strings.
|
|
93
|
-
|
|
94
|
-
Idle HTTP/2 connections receive a PING after `keepalive_interval` seconds and close when its acknowledgement does not
|
|
95
|
-
arrive within `keepalive_timeout`. Set `keepalive_interval` to `0` to disable these probes.
|
|
84
|
+
The config file evaluates to a hash of options. By default Raptor loads `raptor.rb` then `config/raptor.rb` from the
|
|
85
|
+
working directory. Pass `-c PATH` to point at a specific file.
|
|
96
86
|
|
|
97
87
|
```ruby
|
|
98
88
|
# raptor.rb
|
|
@@ -146,35 +136,20 @@ arrive within `keepalive_timeout`. Set `keepalive_interval` to `0` to disable th
|
|
|
146
136
|
}
|
|
147
137
|
```
|
|
148
138
|
|
|
149
|
-
`
|
|
150
|
-
without a fixed limit when queued work is held up by blocking operations. It does not add threads when waiting for the
|
|
151
|
-
GVL is the bottleneck, and temporary threads leave after the queue drains. Set `max_threads` to cap growth, or set it
|
|
152
|
-
to the same value as `threads` for a fixed pool.
|
|
153
|
-
|
|
154
|
-
Set `cpu_affinity` to `true` to pin each worker to a distinct CPU when the worker count fits within the process's
|
|
155
|
-
allowed CPU set. It is off by default because container runtimes commonly expose CPUs that are shared with other
|
|
156
|
-
containers.
|
|
139
|
+
`RAPTOR_WORKERS`, `RAPTOR_THREADS`, and `RAPTOR_MAX_THREADS` set the corresponding options without a config file.
|
|
157
140
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
`RAPTOR_WORKERS`, `RAPTOR_THREADS`, and `RAPTOR_MAX_THREADS` can set the corresponding options without a config file.
|
|
162
|
-
Config files override defaults, environment variables override config files, and command-line options override both.
|
|
163
|
-
`RAPTOR_MAX_THREADS=unlimited` leaves adaptive growth uncapped.
|
|
164
|
-
|
|
165
|
-
`before_worker_boot` and `before_worker_shutdown` hooks receive the worker index.
|
|
141
|
+
By default each worker adds threads beyond `threads` only while queued requests are waiting on blocking work such as
|
|
142
|
+
database or network calls. It stops adding them once Ruby execution becomes the bottleneck, and the extra threads
|
|
143
|
+
retire when the queue is idle. Set `max_threads` to cap this growth, or set it to `threads` for a fixed pool.
|
|
166
144
|
|
|
167
145
|
## Bindings
|
|
168
146
|
|
|
169
|
-
|
|
147
|
+
`binds` accepts any combination of these URIs.
|
|
170
148
|
|
|
171
|
-
- `tcp://host:port` for TCP.
|
|
172
|
-
|
|
173
|
-
- `
|
|
174
|
-
|
|
175
|
-
- `ssl://host:port?cert=/path/to.crt&key=/path/to.key` for TLS. HTTP/1.1 and HTTP/2 are negotiated via ALPN.
|
|
176
|
-
|
|
177
|
-
Multiple binds can be combined freely.
|
|
149
|
+
- `tcp://host:port` for TCP. `localhost` binds both IPv4 and IPv6 loopback addresses.
|
|
150
|
+
- `unix:///path/to/socket` for a Unix domain socket.
|
|
151
|
+
- `ssl://host:port?cert=/path/to.crt&key=/path/to.key` for TLS, negotiating HTTP/1.1 or HTTP/2 via ALPN.
|
|
152
|
+
- `h2c://host:port` for cleartext HTTP/2.
|
|
178
153
|
|
|
179
154
|
## Signals
|
|
180
155
|
|
|
@@ -186,22 +161,16 @@ Send to the master process.
|
|
|
186
161
|
| `TERM` | Graceful shutdown |
|
|
187
162
|
| `HUP` | Reopen `stdout_file`, `stderr_file`, and `access_log_file` |
|
|
188
163
|
| `USR1` | Phased restart (rolling worker replacement) |
|
|
189
|
-
| `USR2` | Hot restart (
|
|
190
|
-
|
|
191
|
-
## Restarts
|
|
164
|
+
| `USR2` | Hot restart (restart master, keeping listening sockets) |
|
|
192
165
|
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
- **Hot restart** (`USR2`) re-execs the master process with its original command line, inheriting the listening sockets
|
|
197
|
-
so accepted connections continue to be served across the swap. The successor master re-runs initialization from
|
|
198
|
-
scratch. Use to pick up changes that affect master-level state (config layout, dependency upgrades, Raptor itself).
|
|
166
|
+
A phased restart replaces workers one at a time while the master keeps running, picking up application code changes.
|
|
167
|
+
A hot restart starts a new master with the original command line and the same listening sockets, picking up changes
|
|
168
|
+
to configuration, dependencies, or Raptor itself.
|
|
199
169
|
|
|
200
170
|
## systemd
|
|
201
171
|
|
|
202
|
-
Raptor
|
|
203
|
-
|
|
204
|
-
place of `binds:`. `READY=1`, `STOPPING=1`, and `RELOADING=1` lifecycle messages are emitted automatically.
|
|
172
|
+
Raptor supports socket activation and `sd_notify`, so it works with `Type=notify` units. When the socket unit is
|
|
173
|
+
active, Raptor serves the listening sockets systemd passes in place of `binds`.
|
|
205
174
|
|
|
206
175
|
```ini
|
|
207
176
|
# /etc/systemd/system/myapp.socket
|
|
@@ -224,8 +193,7 @@ KillMode=mixed
|
|
|
224
193
|
|
|
225
194
|
## Stats
|
|
226
195
|
|
|
227
|
-
|
|
228
|
-
memory and to a JSON file (default `tmp/raptor.json`; set via `stats_file`).
|
|
196
|
+
Workers publish their stats to `stats_file`, which defaults to `tmp/raptor.json`.
|
|
229
197
|
|
|
230
198
|
```
|
|
231
199
|
> bundle exec raptor stats
|
|
@@ -235,14 +203,12 @@ Worker 1 (phase 0): pid=91351, requests=1199, busy=1/3, backlog=0, booted, last_
|
|
|
235
203
|
...
|
|
236
204
|
```
|
|
237
205
|
|
|
238
|
-
Set `control_url` to a Unix socket URL such as `unix:///tmp/raptor-control.sock` to
|
|
239
|
-
|
|
240
|
-
`pool_capacity / max_threads` measures the capacity available at that moment rather than comparing against an
|
|
241
|
-
unbounded configured limit. The control server is read-only and currently exposes only `/stats`.
|
|
206
|
+
Set `control_url` to a Unix socket URL such as `unix:///tmp/raptor-control.sock` to serve the same stats over
|
|
207
|
+
`GET /stats`.
|
|
242
208
|
|
|
243
209
|
## (Micro) Benchmarks
|
|
244
210
|
|
|
245
|
-
Raptor 0.
|
|
211
|
+
Raptor 0.22.0 vs Puma 8.0.2 vs Falcon 0.57.0 across two workload profiles. **IO-bound** is a GET endpoint that
|
|
246
212
|
interleaves 5-10 short sleeps (total 2.5-15ms) with small CPU work, simulating a read path that makes several DB or
|
|
247
213
|
cache calls. **CPU-bound** is a POST endpoint that accepts a small JSON body, interleaves 3-5 chunks of JSON item
|
|
248
214
|
building (total 450-1500 items) with sub-100µs sleeps, and returns the built array, simulating a write path that does
|
|
@@ -256,22 +222,22 @@ disabled, and both threaded servers allow 999 requests per HTTP/1.1 keep-alive c
|
|
|
256
222
|
Each cell reports the median throughput and median p95 latency independently across 3 runs, so the two numbers in a row
|
|
257
223
|
may come from different runs. Every run starts a fresh server process so the samples are independent of each other;
|
|
258
224
|
state accumulated in a previous run cannot bias the next. Across the whole table, the widest spread
|
|
259
|
-
((max - min) / 2 / median) between runs of a single cell was ±
|
|
225
|
+
((max - min) / 2 / median) between runs of a single cell was ±13.2% for throughput and ±25.6% for p95.
|
|
260
226
|
|
|
261
227
|
| Protocol | Workload | Raptor mode | Raptor req/s | Raptor p95 | Puma req/s | Puma p95 | vs Puma req/s | vs Puma p95 | Falcon req/s | Falcon p95 | vs Falcon req/s | vs Falcon p95 |
|
|
262
228
|
| --------------------- | -------- | ----------- | ------------ | ---------- | ----------- | --------- | ------------- | ------------ | ------------ | ---------- | --------------- | ------------- |
|
|
263
|
-
| HTTP/1.1 | IO | Fixed | 2.
|
|
264
|
-
| HTTP/1.1 | IO | Scaling | 7.
|
|
265
|
-
| HTTP/1.1 | CPU | Fixed | 7.
|
|
266
|
-
| HTTP/1.1 | CPU | Scaling | 6.
|
|
267
|
-
| HTTP/1.1 (keep-alive) | IO | Fixed | 2.
|
|
268
|
-
| HTTP/1.1 (keep-alive) | IO | Scaling |
|
|
269
|
-
| HTTP/1.1 (keep-alive) | CPU | Fixed | 7.
|
|
270
|
-
| HTTP/1.1 (keep-alive) | CPU | Scaling | 7.
|
|
271
|
-
| HTTP/2 | IO | Fixed | 1.
|
|
272
|
-
| HTTP/2 | IO | Scaling | 6.
|
|
273
|
-
| HTTP/2 | CPU | Fixed | 6.
|
|
274
|
-
| HTTP/2 | CPU | Scaling | 7.
|
|
229
|
+
| HTTP/1.1 | IO | Fixed | 2.91k req/s | 82.40 ms | 1.51k req/s | 125.20 ms | 92.2% higher | 34.2% lower | 12.08k req/s | 14.40 ms | 75.9% lower | 472.2% higher |
|
|
230
|
+
| HTTP/1.1 | IO | Scaling | 7.01k req/s | 31.70 ms | 1.51k req/s | 125.20 ms | 362.9% higher | 74.7% lower | 12.08k req/s | 14.40 ms | 42.0% lower | 120.1% higher |
|
|
231
|
+
| HTTP/1.1 | CPU | Fixed | 7.24k req/s | 37.90 ms | 8.50k req/s | 23.10 ms | 14.9% lower | 64.1% higher | 6.44k req/s | 28.90 ms | 12.4% higher | 31.1% higher |
|
|
232
|
+
| HTTP/1.1 | CPU | Scaling | 6.48k req/s | 39.50 ms | 8.50k req/s | 23.10 ms | 23.8% lower | 71.0% higher | 6.44k req/s | 28.90 ms | 0.6% higher | 36.7% higher |
|
|
233
|
+
| HTTP/1.1 (keep-alive) | IO | Fixed | 2.26k req/s | 67.20 ms | 1.47k req/s | 105.60 ms | 53.7% higher | 36.4% lower | 6.22k req/s | 28.20 ms | 63.7% lower | 138.3% higher |
|
|
234
|
+
| HTTP/1.1 (keep-alive) | IO | Scaling | 8.20k req/s | 21.80 ms | 1.47k req/s | 105.60 ms | 458.3% higher | 79.4% lower | 6.22k req/s | 28.20 ms | 32.0% higher | 22.7% lower |
|
|
235
|
+
| HTTP/1.1 (keep-alive) | CPU | Fixed | 7.21k req/s | 27.10 ms | 8.38k req/s | 23.60 ms | 14.0% lower | 14.8% higher | 6.87k req/s | 34.20 ms | 5.0% higher | 20.8% lower |
|
|
236
|
+
| HTTP/1.1 (keep-alive) | CPU | Scaling | 7.56k req/s | 28.10 ms | 8.38k req/s | 23.60 ms | 9.8% lower | 19.1% higher | 6.87k req/s | 34.20 ms | 10.1% higher | 17.8% lower |
|
|
237
|
+
| HTTP/2 | IO | Fixed | 1.64k req/s | 112.08 ms | N/A | N/A | - | - | 6.34k req/s | 28.16 ms | 74.2% lower | 298.1% higher |
|
|
238
|
+
| HTTP/2 | IO | Scaling | 6.90k req/s | 27.33 ms | N/A | N/A | - | - | 6.34k req/s | 28.16 ms | 8.8% higher | 2.9% lower |
|
|
239
|
+
| HTTP/2 | CPU | Fixed | 6.90k req/s | 29.30 ms | N/A | N/A | - | - | 6.69k req/s | 65.77 ms | 3.2% higher | 55.4% lower |
|
|
240
|
+
| HTTP/2 | CPU | Scaling | 7.51k req/s | 26.86 ms | N/A | N/A | - | - | 6.69k req/s | 65.77 ms | 12.2% higher | 59.2% lower |
|
|
275
241
|
|
|
276
242
|
> ruby 4.0.7 (2026-09-15 revision 229531a6cf) +YJIT +PRISM [aarch64-linux]
|
|
277
243
|
> 10 worker processes; fixed Raptor and Puma run 3 threads per worker; scaling Raptor starts at 3 with no fixed limit;
|
data/docs/raptor-vs-puma.md
CHANGED
|
@@ -43,7 +43,7 @@ The rest of this doc explains why the shape looks like that.
|
|
|
43
43
|
| I/O multiplexing | `nio4r` reactor for keep-alive idle and slow reads | `nio4r` reactor for the same, plus a red-black tree for O(log n) timeouts |
|
|
44
44
|
| Cluster dispatch | Workers race on inherited listeners with a load-proportional accept delay | Two-choice load-aware BPF dispatch for TCP on Linux; shared-listener fallback |
|
|
45
45
|
| Work queue | Ruby `Queue` coordinated under the pool mutex | Lock-free Michael-Scott FIFO queue |
|
|
46
|
-
| HTTP/2 | Not implemented | Native C parser + HPACK,
|
|
46
|
+
| HTTP/2 | Not implemented | Native C parser + HPACK, reactor-owned frame scheduler |
|
|
47
47
|
| Keep-alive fast path | Same-thread inline dispatch when spare threads exist | Same-thread inline dispatch for bytes that are already waiting |
|
|
48
48
|
| Native extensions | 1 (Ragel HTTP/1 parser + MiniSSL) | 3, all Ractor-safe (Ragel HTTP/1 parser; HTTP/2 parser + HPACK; `writev`, `sched_setaffinity`, `prctl` wrappers) |
|
|
49
49
|
| Shared state (worker↔master) | Pipes and signals | Anonymous shared-memory `mmap` region |
|
|
@@ -326,6 +326,8 @@ Puma has a similar shape. It checks buffered back-to-back requests, then eagerly
|
|
|
326
326
|
|
|
327
327
|
The `reactor.persist` call re-registers the socket with the reactor using `persistent_data_timeout` (65s) as the new deadline. When the next bytes arrive, the reactor treats the socket like any other partially-read connection.
|
|
328
328
|
|
|
329
|
+
A `Raptor::DetachedBody` returns the application thread while keeping the response open, chunked on HTTP/1.1 and ended by closing the connection on HTTP/1.0. The reactor owns the response until it closes, buffers later writes within fixed limits, and pauses the connection's next request without occupying an application thread.
|
|
330
|
+
|
|
329
331
|
### HTTP/2 request lifecycle
|
|
330
332
|
|
|
331
333
|
Raptor speaks HTTP/2 on TLS connections where the client negotiates it via ALPN and on `h2c://` listeners for cleartext clients using prior knowledge. The binder sets `alpn_protocols = ["h2", "http/1.1"]` on the SSL context and the ALPN callback picks h2 whenever the client offers it. Puma does not do this. Puma's SSL context does not advertise `h2` in ALPN, so clients transparently fall back to HTTP/1.1.
|
|
@@ -337,26 +339,19 @@ From there the shape is similar to HTTP/1.1:
|
|
|
337
339
|
1. Reactor reads frames.
|
|
338
340
|
2. The HTTP/2 parser (native C, with an HPACK decoder using a static Huffman table) parses the frames in the HTTP/2 Ractor pool.
|
|
339
341
|
3. Completed requests (once `HEADERS` and `DATA` are complete for a stream) go to the thread pool as separate work items. **A single connection can be servicing many streams in parallel across the thread pool.**
|
|
340
|
-
4. Each stream's response
|
|
342
|
+
4. Each stream's response frames are queued for the reactor, which owns every socket write after connection setup.
|
|
341
343
|
|
|
342
344
|
Responses to `HEAD` requests and statuses that prohibit a message body end with the response `HEADERS` frame.
|
|
343
345
|
Early hints and response-finished callbacks follow the same Rack lifecycle as HTTP/1.1.
|
|
344
346
|
Trailing request `HEADERS` complete an open request. Rack has no standard request-trailer key, so Raptor validates them without adding them to the environment.
|
|
345
347
|
|
|
346
|
-
The `Writer`
|
|
347
|
-
|
|
348
|
-
- If current value is `:idle`, the thread claims the writer by CAS-ing to its own array of frames, then loops draining any additional frames other threads have appended.
|
|
349
|
-
- If current value is an array (someone is already writing), the thread CAS-appends its frames and returns immediately; the current writer will pick them up and flush them.
|
|
350
|
-
|
|
351
|
-
So under contention, only one thread does socket I/O at a time (because a socket can only be written to serially anyway), but no thread ever blocks on a lock. The "loser" of the CAS hands its frames off to the "winner" and returns immediately to whatever it was doing next, whether that is starting another stream, waiting for the next work item, or servicing a different connection.
|
|
352
|
-
|
|
353
|
-
Once the writer thread has claimed a batch of pending frames, it concatenates them into a single buffer and issues one socket write for the whole batch. Frames handed off concurrently can share that write, while sequential body chunks reach the socket as the Rack body yields or writes them.
|
|
348
|
+
The `Writer` hands serialized frames to the reactor, which writes as the socket becomes ready and closes clients that stop reading. Application threads never wait for socket writability, and the connection has a single I/O owner without a per-connection mutex.
|
|
354
349
|
|
|
355
|
-
Flow control uses similar CAS-protected atoms.
|
|
350
|
+
Flow control uses similar CAS-protected atoms. Ordinary Rack bodies wait for connection and stream capacity as they yield. A `Raptor::DetachedBody` instead returns its application thread immediately; the reactor schedules its bounded buffer as capacity becomes available and closes it when the client cancels. This keeps long-lived streams from consuming one application thread each.
|
|
356
351
|
|
|
357
352
|
Frame processing also has an eager loop. After processing one batch of frames, the h2 handler tries to `read_nonblock` one more time to see if the next batch is already available. Up to eight rounds are consumed inline before handing back to the reactor, and the loop bails out early once the app thread pool has more queued work than worker slots so one busy connection cannot starve the collector. This is the same principle as the HTTP/1.1 eager keep-alive: amortise the reactor round-trip when the client is actively sending, but back off under saturation.
|
|
358
353
|
|
|
359
|
-
During worker shutdown, Raptor stops accepting connections and sends GOAWAY with the last stream handed to the Rack application. Later streams are refused while
|
|
354
|
+
During worker shutdown, Raptor stops accepting connections and sends GOAWAY with the last stream handed to the Rack application. Later streams are refused while application work and detached bodies drain. The reactor remains active so in-flight responses can receive flow-control updates; detached bodies still open when the drain period expires are cancelled before their connections close.
|
|
360
355
|
|
|
361
356
|
### Raptor request flow diagram
|
|
362
357
|
|
|
@@ -408,7 +403,8 @@ flowchart TB
|
|
|
408
403
|
COL --> CHK
|
|
409
404
|
CHK -->|"no, more bytes needed"| RCT
|
|
410
405
|
CHK -->|"yes, push proc"| ATP
|
|
411
|
-
ATP -->|"
|
|
406
|
+
ATP -->|"HTTP/2 response frames"| RCT
|
|
407
|
+
ATP -->|"HTTP/1.1 response write"| KA
|
|
412
408
|
KA -->|"no, close"| CLS["close socket"]
|
|
413
409
|
KA -->|"yes"| EAG
|
|
414
410
|
EAG -.->|"bytes ready, parse+dispatch on same thread"| ATP
|
|
@@ -495,11 +491,11 @@ For external monitoring, `control_url` can expose a read-only `GET /stats` endpo
|
|
|
495
491
|
|
|
496
492
|
**Puma.** Not implemented. Puma's [position](https://github.com/puma/puma/issues/2697) is that HTTP/2 belongs at the edge (nginx, Caddy, ALB), which terminates it and speaks HTTP/1.1 to the app server. That's a reasonable call for the deployments Puma is aimed at, and it's where most Rails production actually sits.
|
|
497
493
|
|
|
498
|
-
**Raptor.** Native C parser plus HPACK, per-stream flow control,
|
|
494
|
+
**Raptor.** Native C parser plus HPACK, per-stream flow control, reactor-owned response writes, detached long-lived responses, stream multiplexing over a single connection, configurable PING keepalive, and response trailers exposed through `env["raptor.response_trailers"]`. Once a request is complete it takes the same path as HTTP/1.1 and enters the same thread pool. Under HTTP/2, a single client connection can be issuing many concurrent requests, and Raptor services all of them in parallel on the same thread pool.
|
|
499
495
|
|
|
500
496
|
Whether that matters depends on your setup. If you terminate TLS at an edge proxy that already speaks HTTP/2, both servers see HTTP/1.1 and it doesn't matter which of them you pick on this axis. If you're building an all-Ruby stack with no proxy in front, serving direct HTTP/2 clients, or measuring the app server itself, HTTP/2 support is where Raptor and Puma stop being comparable.
|
|
501
497
|
|
|
502
|
-
At the throughput numbers the benchmark shows, a small set of concurrent connections multiplex many streams, so responses from several app threads share each socket.
|
|
498
|
+
At the throughput numbers the benchmark shows, a small set of concurrent connections multiplex many streams, so responses from several app threads share each socket. Those threads queue frames while the reactor owns non-blocking writes for the connection.
|
|
503
499
|
|
|
504
500
|
### Response writing
|
|
505
501
|
|
|
@@ -610,7 +606,7 @@ Falcon also speaks HTTP/2 natively, so it's the interesting comparison there rat
|
|
|
610
606
|
|
|
611
607
|
The benchmark's h2 listener uses TLS, while Raptor's BPF reuseport path only wraps plain TCP listeners. BPF dispatch therefore cannot explain the h2 variance. With 40 physical connections spread across 10 workers, each carrying three streams, placement and per-connection scheduling have coarse granularity; more instrumentation is needed before assigning the variance to a specific mechanism.
|
|
612
608
|
|
|
613
|
-
Raptor's HTTP/2 CPU-bound throughput remains in the same broad range as its HTTP/1.1 result while multiplexing streams onto shared sockets. The
|
|
609
|
+
Raptor's HTTP/2 CPU-bound throughput remains in the same broad range as its HTTP/1.1 result while multiplexing streams onto shared sockets. The reactor-owned frame scheduler and flow-control atoms are part of how it coordinates that work, but this benchmark does not provide an alternative Raptor control case from which to quantify their individual effect.
|
|
614
610
|
|
|
615
611
|
## Part V: What Raptor gives up
|
|
616
612
|
|
data/lib/raptor/cluster.rb
CHANGED
|
@@ -0,0 +1,293 @@
|
|
|
1
|
+
# rbs_inline: enabled
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
require "atomic-ruby/atom"
|
|
5
|
+
|
|
6
|
+
module Raptor
|
|
7
|
+
# A bounded response body that may be written after its application thread
|
|
8
|
+
# has returned.
|
|
9
|
+
#
|
|
10
|
+
class DetachedBody
|
|
11
|
+
DEFAULT_MAX_BUFFER_SIZE = 64 * 1024
|
|
12
|
+
|
|
13
|
+
# Tracks buffered bytes shared by a group of detached bodies.
|
|
14
|
+
#
|
|
15
|
+
class Budget
|
|
16
|
+
# @rbs @max_size: Integer
|
|
17
|
+
# @rbs @size: Atom
|
|
18
|
+
|
|
19
|
+
# @rbs (Integer max_size) -> void
|
|
20
|
+
def initialize(max_size)
|
|
21
|
+
@max_size = max_size
|
|
22
|
+
@size = Atom.new(0)
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
# @rbs (Integer bytes) -> bool
|
|
26
|
+
def reserve(bytes)
|
|
27
|
+
reserved = false
|
|
28
|
+
@size.swap do |size|
|
|
29
|
+
reserved = size + bytes <= @max_size
|
|
30
|
+
reserved ? size + bytes : size
|
|
31
|
+
end
|
|
32
|
+
reserved
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# @rbs (Integer bytes) -> void
|
|
36
|
+
def release(bytes)
|
|
37
|
+
@size.swap { |size| size - bytes }
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
# @rbs @max_buffer_size: Integer
|
|
42
|
+
# @rbs @state: Atom
|
|
43
|
+
# @rbs @budgets: Array[Budget]
|
|
44
|
+
# @rbs @wake: ^() -> void
|
|
45
|
+
# @rbs @dispatch: ^(Proc) -> void
|
|
46
|
+
# @rbs @finished: ^(Symbol) -> void
|
|
47
|
+
# @rbs @on_open: (^(DetachedBody) -> void)?
|
|
48
|
+
# @rbs @on_close: (^(Symbol) -> void)?
|
|
49
|
+
|
|
50
|
+
# Creates a detached body with a bounded byte buffer.
|
|
51
|
+
#
|
|
52
|
+
# @param max_buffer_size [Integer] maximum body bytes waiting to be written
|
|
53
|
+
# @return [void]
|
|
54
|
+
#
|
|
55
|
+
# @rbs (?max_buffer_size: Integer) -> void
|
|
56
|
+
def initialize(max_buffer_size: DEFAULT_MAX_BUFFER_SIZE)
|
|
57
|
+
raise ArgumentError, "max_buffer_size must be positive" unless max_buffer_size.positive?
|
|
58
|
+
|
|
59
|
+
@max_buffer_size = max_buffer_size
|
|
60
|
+
@state = Atom.new({chunks: [], size: 0, closing: false, closed: false, notified: false, trailers: {}})
|
|
61
|
+
@budgets = []
|
|
62
|
+
@wake = proc {}
|
|
63
|
+
@dispatch = proc { |callback| callback.call }
|
|
64
|
+
@finished = proc {}
|
|
65
|
+
@on_open = nil
|
|
66
|
+
@on_close = nil
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
# Registers a callback for when the server accepts the response stream.
|
|
70
|
+
#
|
|
71
|
+
# @yieldparam stream [DetachedBody] the opened response stream
|
|
72
|
+
# @return [DetachedBody]
|
|
73
|
+
#
|
|
74
|
+
# @rbs () { (DetachedBody stream) -> void } -> DetachedBody
|
|
75
|
+
def on_open(&block)
|
|
76
|
+
@on_open = block
|
|
77
|
+
self
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
# Registers a callback for when the response stream closes.
|
|
81
|
+
#
|
|
82
|
+
# @yieldparam reason [Symbol] why the stream closed
|
|
83
|
+
# @return [DetachedBody]
|
|
84
|
+
#
|
|
85
|
+
# @rbs () { (Symbol reason) -> void } -> DetachedBody
|
|
86
|
+
def on_close(&block)
|
|
87
|
+
@on_close = block
|
|
88
|
+
self
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
# Adds body bytes without waiting for socket or flow-control capacity.
|
|
92
|
+
#
|
|
93
|
+
# @param chunk [String] response body bytes
|
|
94
|
+
# @return [Symbol] `:accepted`, `:full`, or `:closed`
|
|
95
|
+
#
|
|
96
|
+
# @rbs (String chunk) -> Symbol
|
|
97
|
+
def try_write(chunk)
|
|
98
|
+
raise TypeError, "body must yield String values" unless chunk.is_a?(String)
|
|
99
|
+
|
|
100
|
+
chunk = chunk.dup.freeze
|
|
101
|
+
if chunk.empty?
|
|
102
|
+
state = @state.value
|
|
103
|
+
return state[:closed] || state[:closing] ? :closed : :accepted
|
|
104
|
+
end
|
|
105
|
+
return :full unless reserve(chunk.bytesize)
|
|
106
|
+
|
|
107
|
+
result = :closed
|
|
108
|
+
wake = false
|
|
109
|
+
@state.swap do |state|
|
|
110
|
+
if state[:closed] || state[:closing]
|
|
111
|
+
result = :closed
|
|
112
|
+
wake = false
|
|
113
|
+
state
|
|
114
|
+
elsif state[:size] + chunk.bytesize > @max_buffer_size
|
|
115
|
+
result = :full
|
|
116
|
+
wake = false
|
|
117
|
+
state
|
|
118
|
+
else
|
|
119
|
+
result = :accepted
|
|
120
|
+
wake = !state[:notified]
|
|
121
|
+
state.merge(
|
|
122
|
+
chunks: state[:chunks] + [chunk],
|
|
123
|
+
size: state[:size] + chunk.bytesize,
|
|
124
|
+
notified: true
|
|
125
|
+
)
|
|
126
|
+
end
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
if result == :accepted
|
|
130
|
+
@wake.call if wake
|
|
131
|
+
else
|
|
132
|
+
release(chunk.bytesize)
|
|
133
|
+
end
|
|
134
|
+
result
|
|
135
|
+
end
|
|
136
|
+
|
|
137
|
+
# Finishes the body after all accepted bytes have been written.
|
|
138
|
+
#
|
|
139
|
+
# @param trailers [Hash] trailing response headers
|
|
140
|
+
# @return [void]
|
|
141
|
+
#
|
|
142
|
+
# @rbs (?trailers: Hash[String, String | Array[String]]) -> void
|
|
143
|
+
def close(trailers: {})
|
|
144
|
+
trailers = trailers.to_h do |name, value|
|
|
145
|
+
value = value.is_a?(Array) ? value.map { _1.dup.freeze }.freeze : value.dup.freeze
|
|
146
|
+
[name.dup.freeze, value]
|
|
147
|
+
end.freeze
|
|
148
|
+
wake = false
|
|
149
|
+
@state.swap do |state|
|
|
150
|
+
if state[:closed] || state[:closing]
|
|
151
|
+
wake = false
|
|
152
|
+
next state
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
wake = !state[:notified]
|
|
156
|
+
state.merge(closing: true, notified: true, trailers: trailers)
|
|
157
|
+
end
|
|
158
|
+
@wake.call if wake
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
# Returns whether the stream has closed.
|
|
162
|
+
#
|
|
163
|
+
# @return [Boolean]
|
|
164
|
+
#
|
|
165
|
+
# @rbs () -> bool
|
|
166
|
+
def closed?
|
|
167
|
+
@state.value[:closed]
|
|
168
|
+
end
|
|
169
|
+
|
|
170
|
+
# Connects the body to its reactor-owned response stream.
|
|
171
|
+
#
|
|
172
|
+
# @rbs (Array[Budget] budgets, ^() -> void wake, ^(Proc) -> void dispatch, ^(Symbol) -> void finished) -> bool
|
|
173
|
+
def attach(budgets, wake, dispatch, finished)
|
|
174
|
+
@wake = wake
|
|
175
|
+
@dispatch = dispatch
|
|
176
|
+
@finished = finished
|
|
177
|
+
size = @state.value[:size]
|
|
178
|
+
reserved = []
|
|
179
|
+
budgets.each do |budget|
|
|
180
|
+
unless budget.reserve(size)
|
|
181
|
+
reserved.each { |held| held.release(size) }
|
|
182
|
+
finish(:full)
|
|
183
|
+
return false
|
|
184
|
+
end
|
|
185
|
+
reserved << budget
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
@budgets = budgets
|
|
189
|
+
true
|
|
190
|
+
end
|
|
191
|
+
|
|
192
|
+
# Notifies the application that its response stream is ready.
|
|
193
|
+
#
|
|
194
|
+
# @rbs () -> void
|
|
195
|
+
def open
|
|
196
|
+
callback = @on_open
|
|
197
|
+
@dispatch.call(proc { callback.call(self) }) if callback
|
|
198
|
+
@wake.call if @state.value[:notified]
|
|
199
|
+
end
|
|
200
|
+
|
|
201
|
+
# Returns the size of the next buffered chunk, 0 when closing, or nil
|
|
202
|
+
# while waiting for more data.
|
|
203
|
+
#
|
|
204
|
+
# @rbs () -> Integer?
|
|
205
|
+
def next_size
|
|
206
|
+
state = @state.value
|
|
207
|
+
state[:chunks].first&.bytesize || (0 if state[:closing])
|
|
208
|
+
end
|
|
209
|
+
|
|
210
|
+
# Removes up to `max_bytes` from the next buffered chunk.
|
|
211
|
+
#
|
|
212
|
+
# @rbs (Integer max_bytes) -> String?
|
|
213
|
+
def shift(max_bytes)
|
|
214
|
+
chunk = nil
|
|
215
|
+
@state.swap do |state|
|
|
216
|
+
chunk = nil
|
|
217
|
+
first = state[:chunks].first
|
|
218
|
+
next state unless first
|
|
219
|
+
|
|
220
|
+
size = max_bytes < first.bytesize ? max_bytes : first.bytesize
|
|
221
|
+
chunk = first.byteslice(0, size)
|
|
222
|
+
chunks = if size == first.bytesize
|
|
223
|
+
state[:chunks].drop(1)
|
|
224
|
+
else
|
|
225
|
+
[first.byteslice(size..-1).freeze] + state[:chunks].drop(1)
|
|
226
|
+
end
|
|
227
|
+
state.merge(
|
|
228
|
+
chunks: chunks,
|
|
229
|
+
size: state[:size] - size,
|
|
230
|
+
notified: !chunks.empty? || state[:closing]
|
|
231
|
+
)
|
|
232
|
+
end
|
|
233
|
+
release(chunk.bytesize) if chunk
|
|
234
|
+
chunk
|
|
235
|
+
end
|
|
236
|
+
|
|
237
|
+
# Returns the response trailers supplied when the body closed.
|
|
238
|
+
#
|
|
239
|
+
# @rbs () -> Hash[String, String | Array[String]]
|
|
240
|
+
def trailers
|
|
241
|
+
@state.value[:trailers]
|
|
242
|
+
end
|
|
243
|
+
|
|
244
|
+
# Closes the stream and invokes its callback exactly once.
|
|
245
|
+
#
|
|
246
|
+
# @rbs (Symbol reason) -> void
|
|
247
|
+
def finish(reason)
|
|
248
|
+
callback = false
|
|
249
|
+
remaining = 0
|
|
250
|
+
@state.swap do |state|
|
|
251
|
+
if state[:closed]
|
|
252
|
+
callback = false
|
|
253
|
+
remaining = 0
|
|
254
|
+
state
|
|
255
|
+
else
|
|
256
|
+
callback = true
|
|
257
|
+
remaining = state[:size]
|
|
258
|
+
state.merge(chunks: [], size: 0, closed: true, notified: false)
|
|
259
|
+
end
|
|
260
|
+
end
|
|
261
|
+
return unless callback
|
|
262
|
+
|
|
263
|
+
release(remaining)
|
|
264
|
+
@dispatch.call(proc do
|
|
265
|
+
begin
|
|
266
|
+
@on_close&.call(reason)
|
|
267
|
+
ensure
|
|
268
|
+
@finished.call(reason)
|
|
269
|
+
end
|
|
270
|
+
end)
|
|
271
|
+
end
|
|
272
|
+
|
|
273
|
+
private
|
|
274
|
+
|
|
275
|
+
# @rbs (Integer bytes) -> bool
|
|
276
|
+
def reserve(bytes)
|
|
277
|
+
reserved = []
|
|
278
|
+
@budgets.each do |budget|
|
|
279
|
+
unless budget.reserve(bytes)
|
|
280
|
+
reserved.each { |held| held.release(bytes) }
|
|
281
|
+
return false
|
|
282
|
+
end
|
|
283
|
+
reserved << budget
|
|
284
|
+
end
|
|
285
|
+
true
|
|
286
|
+
end
|
|
287
|
+
|
|
288
|
+
# @rbs (Integer bytes) -> void
|
|
289
|
+
def release(bytes)
|
|
290
|
+
@budgets.each { |budget| budget.release(bytes) }
|
|
291
|
+
end
|
|
292
|
+
end
|
|
293
|
+
end
|