raptor 0.23.0 → 0.25.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/docs/talk.md CHANGED
@@ -2,16 +2,26 @@
2
2
 
3
3
  ## A Ruby 4 Web Server
4
4
 
5
- <img src="https://github.com/joshuay03.png" width="200" alt="Joshua Young" />
5
+ <img src="./assets/startup_output.svg" width="60%" align="right" alt="The README's complete Raptor startup output, including log prefixes, the process tree, and all ten worker boot messages." />
6
6
 
7
- _Joshua Young ([`@joshuay03`](https://github.com/joshuay03)) · Senior Software Engineer at [Buildkite](https://buildkite.com/)_
7
+ <img src="./assets/profile_picture.png" width="200" alt="Joshua Young" />
8
8
 
9
- _On the [Rails](https://github.com/rails/rails), [Puma](https://github.com/puma/puma), and [Concurrent Ruby](https://github.com/ruby-concurrency/concurrent-ruby) maintainer teams_
9
+ _Joshua Young ([`@joshuay03`](https://github.com/joshuay03))_
10
+
11
+ _Senior Software Engineer at [Buildkite](https://buildkite.com/)_
12
+
13
+ _[Rails](https://github.com/rails/rails) Issues team member_
14
+
15
+ _[Puma](https://github.com/puma/puma), [Concurrent Ruby](https://github.com/ruby-concurrency/concurrent-ruby), and [Vernier](https://github.com/jhawthorn/vernier) maintainer_
16
+
17
+ <hr width="35%" align="left">
10
18
 
11
19
  _[BrisRails](https://luma.com/t3l24s8h) · August 2026_
12
20
 
13
21
  _[Ruby Melbourne](https://www.meetup.com/ruby-melbourne/events/313408465/) · October 2026_
14
22
 
23
+ <br clear="all" />
24
+
15
25
  <br>
16
26
  <br>
17
27
  <br>
@@ -32,21 +42,19 @@ _[Ruby Melbourne](https://www.meetup.com/ruby-melbourne/events/313408465/) · Oc
32
42
  <br>
33
43
  <br>
34
44
 
35
- ## Hi
45
+ ## How Raptor started
36
46
 
37
- <big>I'm Josh. By day I work on Buildkite's Rails monolith, and I also spend a lot of time in the internals of Puma and Rails.</big>
47
+ <big>Working on Puma made me curious about what a web server built around Ractors could look like.</big>
38
48
 
39
- <big>Over the past year that turned into my own Ruby web server, Raptor. It runs any Rack app, Rails included, and parses HTTP in parallel using Ractors.</big>
49
+ <big>That became Raptor. It runs Rack apps, Rails included, and parses HTTP in parallel using Ractors.</big>
40
50
 
41
- <big>It's still young though, so I wouldn't put your production app behind it just yet.</big>
51
+ <big>**Your app doesn't need to be Ractor-safe.** It runs on ordinary threads,
52
+ with the same thread-safety requirements as multithreaded Puma.</big>
42
53
 
43
- Today is a tour of what's inside it, and the parts of Ruby and Linux I had to learn along the way:
54
+ <big>It's still young, and I'd love more people to try it on their apps and tell me what works and what doesn't.</big>
44
55
 
45
- - <big>Ractors, and what changed in Ruby 4</big>
46
- - <big>Lock-free data structures built on compare-and-swap</big>
47
- - <big>Red-black trees</big>
48
- - <big>Shared memory between processes</big>
49
- - <big>Tiny programs that run inside the Linux kernel</big>
56
+ Today we'll look inside Raptor at its Ractors, lock-free queues, red-black trees,
57
+ shared memory, and a tiny program running inside Linux.
50
58
 
51
59
  <br>
52
60
  <br>
@@ -70,18 +78,14 @@ Today is a tour of what's inside it, and the parts of Ruby and Linux I had to le
70
78
 
71
79
  ## What a web server actually does
72
80
 
73
- Rails isn't a web server. [Rack](https://github.com/rack/rack) isn't one either. Rack is the contract between a web server and your app:
81
+ [Rack](https://github.com/rack/rack) gives us a small boundary between the server and your app.
74
82
 
75
83
  ```ruby
76
84
  # A complete Rack app. Nothing more is needed.
77
85
  run ->(env) { [200, { "content-type" => "text/plain" }, ["hello\n"]] }
78
86
  ```
79
87
 
80
- - <big>Your app is anything that responds to `#call`</big>
81
- - <big>It takes a `Hash` describing the request (the "Rack env")</big>
82
- - <big>It returns `[status, headers, body]`</big>
83
-
84
- Everything else is the web server's job:
88
+ <big>Your app responds to `call`. It takes a request hash, the "Rack env", and returns `[status, headers, body]`.</big>
85
89
 
86
90
  ```mermaid
87
91
  sequenceDiagram
@@ -100,9 +104,7 @@ sequenceDiagram
100
104
  Note over C,S: Close, or keep the connection open for the next request
101
105
  ```
102
106
 
103
- - <big>The kernel does the TCP handshake. The server just calls `accept` and gets a ready socket.</big>
104
- - <big>The server reads bytes, parses them, calls your app, and writes the response back</big>
105
- - <big>With keep-alive, the same connection carries the next request</big>
107
+ <big>The kernel handles TCP. The server handles HTTP.</big>
106
108
 
107
109
  <br>
108
110
  <br>
@@ -132,9 +134,10 @@ sequenceDiagram
132
134
 
133
135
  **[Falcon](https://github.com/socketry/falcon)**
134
136
 
135
- <big>Built by Samuel Williams on his [`async`](https://github.com/socketry/async) gem. It uses fibers instead of a thread pool, speaks HTTP/2, and is great at streaming.</big>
137
+ <big>Built on [`async`](https://github.com/socketry/async). It uses fibers instead of a thread pool, speaks HTTP/2, and is great at streaming.</big>
136
138
 
137
- Both are great, but both were designed before Ractors were usable.
139
+ Fibers are another concurrency model. Raptor keeps your thread-safe Puma app on ordinary threads,
140
+ without requiring app changes.
138
141
 
139
142
  <br>
140
143
  <br>
@@ -160,13 +163,10 @@ Both are great, but both were designed before Ractors were usable.
160
163
 
161
164
  <big>Ractors have been around since Ruby 3.0, but the API kept changing, so most of us never touched them.</big>
162
165
 
163
- <big>Ruby 4 reworked them around `Ractor::Port`, which many Ractors can send to and one can receive from. That's exactly the shape of "many workers, one collector".</big>
166
+ <big>[Ruby 4.0](https://www.ruby-lang.org/en/news/2025/12/25/ruby-4-0-0-released/#ractor-improvements) introduced `Ractor::Port`.
167
+ Many Ractors can send to one receiver. A good fit for "many workers, one collector".</big>
164
168
 
165
- <big>They're still marked experimental, but Ruby 4.1 moves them closer to dropping that label. For one, each Ractor will collect its own garbage without stopping the others.</big>
166
-
167
- That got me wondering:
168
-
169
- > What would a Ruby web server look like if it was built around Ractors from the start?
169
+ <big>They're still marked experimental, but [Ruby 4.1 adds Ractor-local garbage collection](https://www.ruby-lang.org/en/news/2026/10/12/ruby-4-1-0-preview1-released/#ractor-improvements), so most collections no longer stop all the Ractors.</big>
170
170
 
171
171
  <br>
172
172
  <br>
@@ -190,20 +190,20 @@ That got me wondering:
190
190
 
191
191
  ## Ractors in one slide
192
192
 
193
- Ruby has a Global VM Lock (GVL). Only one thread at a time can run Ruby code.
193
+ Ruby threads normally share a Global VM Lock (GVL), so only one of them can run Ruby code at a time.
194
194
 
195
195
  A Ractor is an isolated area of Ruby with **its own GVL**, so two Ractors can run Ruby on two cores at the same time.
196
196
 
197
- The catch is isolation. A Ractor can't reach into its surroundings:
197
+ The catch is isolation. A Ractor can't reach into its surroundings.
198
198
 
199
199
  ```ruby
200
200
  data = { name: "hello" }
201
201
 
202
202
  Ractor.new { puts data }
203
- # ArgumentError: can not isolate a Proc because it accesses outer variables (data).
203
+ # Error: the block accesses an outer variable (data).
204
204
  ```
205
205
 
206
- Instead, Ractors pass messages:
206
+ Instead, Ractors pass messages.
207
207
 
208
208
  ```ruby
209
209
  port = Ractor::Port.new
@@ -216,9 +216,9 @@ worker.send("hello")
216
216
  port.receive # => "processed: hello"
217
217
  ```
218
218
 
219
- - <big>Frozen values are shared as is</big>
220
- - <big>Mutable values are deep-copied, or moved with `move: true` so the sender can't touch them again</big>
221
- - <big>No shared mutable state means no data races</big>
219
+ <big>Shareable values, like deeply frozen arrays and hashes, are shared as is. Mutable values are copied, or moved with `move: true` so the sender can no longer use them.</big>
220
+
221
+ <big>Isolation keeps ordinary mutable objects from being changed by two Ractors at once.</big>
222
222
 
223
223
  <br>
224
224
  <br>
@@ -242,12 +242,9 @@ port.receive # => "processed: hello"
242
242
 
243
243
  ## What Ractors are good for
244
244
 
245
- They won't make your app parallel on their own, because almost no gem you use is Ractor-safe. Where they do help is work that is:
245
+ They won't make your Rails app parallel on their own. Rails and many of its gems aren't Ractor-safe.
246
246
 
247
- - <big>Small and CPU-heavy</big>
248
- - <big>Free of global state</big>
249
- - <big>Free of gem dependencies</big>
250
- - <big>Easy to hand off: bytes in, result out</big>
247
+ <big>They suit small, CPU-heavy jobs that are easy to hand off, without relying on global state or Ractor-unsafe gems.</big>
251
248
 
252
249
  That describes HTTP parsing exactly.
253
250
 
@@ -273,13 +270,18 @@ That describes HTTP parsing exactly.
273
270
 
274
271
  ## The idea
275
272
 
276
- Puma parses each request on the same thread that's about to run your app. Parsing and your app take turns on the same GVL.
273
+ In Puma, parsing and your app share the worker's GVL, even when they run on different threads.
277
274
 
278
- If parsing runs in a Ractor instead, it gets its own GVL:
275
+ If parsing runs in a Ractor instead, it gets its own GVL.
279
276
 
280
277
  ```mermaid
278
+ ---
279
+ config:
280
+ gantt:
281
+ leftPadding: 220
282
+ ---
281
283
  gantt
282
- title One worker process, over time
284
+ title One worker process, over time (illustrative)
283
285
  dateFormat x
284
286
  axisFormat %L ms
285
287
 
@@ -326,21 +328,21 @@ If a whole request arrives in one read, Raptor skips the Ractor and parses it st
326
328
 
327
329
  ```mermaid
328
330
  flowchart TB
329
- M["Master process<br/>starts and watches workers"]
331
+ M["Master<br/>starts and watches workers"]
330
332
 
331
333
  subgraph W["Worker process (one of many)"]
332
- SRV["Server thread<br/>accepts connections"]
333
- RCT["Reactor thread<br/>waits on idle and partial connections"]
334
- RP["Ractor pools<br/>parse HTTP/1.1 and HTTP/2<br/>on their own GVLs"]
335
- COL["Collector threads<br/>gather parsed requests"]
336
- APP["App thread pool<br/>runs your Rack app<br/>and writes responses"]
337
-
338
- SRV -->|"whole request already here"| APP
339
- SRV -->|"needs more bytes"| RCT
340
- RCT -->|"raw bytes"| RP
334
+ SRV["Server thread<br/>accepts sockets"]
335
+ RCT["Reactor thread<br/>waits for bytes"]
336
+ RP["Parser Ractors<br/>HTTP/1.1 and HTTP/2"]
337
+ COL["Collector threads"]
338
+ APP["App threads<br/>Rack + response"]
339
+
340
+ SRV -->|"complete request"| APP
341
+ SRV -->|"incomplete"| RCT
342
+ RCT -->|"bytes"| RP
341
343
  RP --> COL
342
- COL -->|"complete request"| APP
343
- COL -->|"still incomplete"| RCT
344
+ COL -->|"ready"| APP
345
+ COL -->|"incomplete"| RCT
344
346
  end
345
347
 
346
348
  M -->|"fork"| W
@@ -370,7 +372,8 @@ Let's walk through each piece.
370
372
 
371
373
  ## The Ractor pools
372
374
 
373
- Each protocol gets its own pool of parser Ractors. A coordinator hands work to whichever parser is free.
375
+ Each protocol gets its own pool of parser Ractors.
376
+ With multiple parsers, a coordinator hands work to whichever is free.
374
377
 
375
378
  ```mermaid
376
379
  sequenceDiagram
@@ -381,7 +384,7 @@ sequenceDiagram
381
384
  box Coordinator Ractor
382
385
  participant C as Coordinator thread
383
386
  end
384
- box Parser Ractor
387
+ box One parser Ractor
385
388
  participant P as Parser thread
386
389
  end
387
390
 
@@ -424,9 +427,9 @@ I pulled this pattern out into its own gem, **[`ractor-pool`](https://github.com
424
427
 
425
428
  This is where your Rack app runs.
426
429
 
427
- Puma's pool is a queue guarded by a `Mutex`, so when things get busy, threads line up behind the lock.
430
+ Puma's work queue is guarded by a `Mutex`. Adding and taking work both acquire it.
428
431
 
429
- Raptor's pool has **no locks** at all. To explain how, we need one tiny CPU instruction.
432
+ Raptor's work queue has **no mutex**. To explain how, we need one tiny CPU instruction.
430
433
 
431
434
  <br>
432
435
  <br>
@@ -455,7 +458,7 @@ Raptor's pool has **no locks** at all. To explain how, we need one tiny CPU inst
455
458
  ```mermaid
456
459
  sequenceDiagram
457
460
  participant A as Thread A
458
- participant M as Memory: count
461
+ participant M as Memory holding count
459
462
  participant B as Thread B
460
463
 
461
464
  Note over M: count = 42
@@ -463,7 +466,7 @@ sequenceDiagram
463
466
  M-->>A: ✅ swapped, count is now 43
464
467
  B->>M: CAS(expected 42, new 43)
465
468
  M-->>B: ❌ failed, count is 43 not 42
466
- B->>M: retry: CAS(expected 43, new 44)
469
+ B->>M: retry CAS(expected 43, new 44)
467
470
  M-->>B: ✅ swapped, count is now 44
468
471
  ```
469
472
 
@@ -505,8 +508,8 @@ Raptor's work queue is [`atomic-ruby`](https://github.com/joshuay03/atomic-ruby)
505
508
  flowchart LR
506
509
  H(["head"]) --> S["sentinel"] --> A["work A"] --> B["work B"]
507
510
  T(["tail"]) --> B
508
- B -. "① CAS: link C after B" .-> C["work C"]
509
- T -. "② CAS: move tail to C" .-> C
511
+ B -. "① CAS links C after B" .-> C["work C"]
512
+ T -. "② CAS moves tail" .-> C
510
513
  ```
511
514
 
512
515
  **Taking work**
@@ -514,7 +517,7 @@ flowchart LR
514
517
  ```mermaid
515
518
  flowchart LR
516
519
  H(["head"]) --> S["sentinel"] --> A["work A"] --> B["work B"]
517
- H -. "CAS: move head to A, return A's work<br/>(A becomes the new sentinel)" .-> A
520
+ H -. "CAS moves head to A, returns A's work<br/>(A becomes the new sentinel)" .-> A
518
521
  ```
519
522
 
520
523
  <big>Adding and taking never block each other. If two threads race, one CAS wins and the other retries.</big>
@@ -539,11 +542,12 @@ flowchart LR
539
542
  <br>
540
543
  <br>
541
544
 
542
- ## No mutexes at all
545
+ ## No mutexes in Raptor's own code
543
546
 
544
547
  There isn't a single `Mutex` in Raptor's own code. Shared state is either owned by one thread, passed along as a message, or updated with CAS.
545
548
 
546
- That's not only about speed. Under the GVL, a thread can be switched out at almost any point, including while it's holding a lock:
549
+ That's not only about speed. Under the GVL, a thread can be switched out at almost any point,
550
+ including while it's holding a lock.
547
551
 
548
552
  ```mermaid
549
553
  sequenceDiagram
@@ -564,7 +568,7 @@ sequenceDiagram
564
568
 
565
569
  <big>CAS has nothing to hold. A thread switched out mid-update can't block anyone, and if it lost the race, it just retries.</big>
566
570
 
567
- It also means there's no lock ordering to get wrong, which is where deadlocks come from.
571
+ It also means there's no lock ordering to get wrong, removing one source of deadlocks.
568
572
 
569
573
  <br>
570
574
  <br>
@@ -588,22 +592,29 @@ It also means there's no lock ordering to get wrong, which is where deadlocks co
588
592
 
589
593
  ## How many threads?
590
594
 
591
- Most requests spend their time waiting on the database or the network. More threads help with waiting, but not with running Ruby.
595
+ Thread counts are hard to tune. Too few leave requests queued. Too many can compete for the GVL.
596
+ The right count changes with the workload.
592
597
 
593
- Raptor uses [`atomic-ruby`](https://github.com/joshuay03/atomic-ruby)'s `AtomicThreadPool`, which uses a CRuby hook to see what each thread is actually doing:
598
+ [`atomic-ruby`](https://github.com/joshuay03/atomic-ruby)'s pool uses CRuby's thread-event API
599
+ to measure time holding the GVL, waiting for it, and blocked outside it, such as during I/O.
600
+
601
+ It combines those timings with CPU time to decide whether more threads would help.
594
602
 
595
603
  ```mermaid
596
- flowchart TB
597
- Q{"Are requests waiting<br/>in the queue?"}
598
- Q -->|"yes"| W{"What are the threads<br/>mostly doing?"}
599
- Q -->|"no"| I{"Has the queue been<br/>empty for 5 seconds?"}
604
+ flowchart LR
605
+ Q{"Work queued?"}
606
+ Q -->|"yes"| W{"Where is time<br/>being spent?"}
607
+ Q -->|"no"| I{"Queue empty<br/>for 5 seconds?"}
600
608
  W -->|"waiting on I/O<br/>(database, network)"| Add["Add more threads"]
601
- W -->|"running Ruby<br/>(CPU is the limit)"| Trim["Trim back towards<br/>the base size"]
609
+ W -->|"Ruby CPU saturated"| Trim["Trim back towards<br/>the base size"]
602
610
  I -->|"no"| Stay["Keep the current size"]
603
- I -->|"yes"| Retire["Retire the extra threads"]
611
+ I -->|"yes"| Retire["Retire extra threads<br/>after their current work"]
604
612
  ```
605
613
 
606
- <big>It starts at `threads` (3 by default). Set `max_threads` to cap the growth, or match `threads` for a fixed pool.</big>
614
+ <big>It starts at `threads` (3 per worker by default) and adapts to the workload,
615
+ with no hard maximum. You don't have to choose one fixed thread count.</big>
616
+
617
+ <big>Set `max_threads` to fit your database pool or other resource limits, or match `threads` for a fixed pool.</big>
607
618
 
608
619
  <br>
609
620
  <br>
@@ -627,15 +638,15 @@ flowchart TB
627
638
 
628
639
  ## The reactor
629
640
 
630
- Some connections aren't ready to be read yet: slow clients, half-sent requests, idle keep-alive connections. The reactor watches all of them at once.
641
+ Slow clients, half-sent requests, and idle keep-alive connections aren't ready to be read yet. The reactor watches all of them at once.
642
+
643
+ <big>It uses [`io-event`](https://github.com/socketry/io-event), backed by `epoll` on Linux
644
+ and `kqueue` on macOS.</big>
631
645
 
632
- - <big>Built on [`nio4r`](https://github.com/socketry/nio4r) (`epoll` on Linux, `kqueue` on macOS), same as Puma</big>
633
- - <big>Every connection also has a deadline:</big>
634
- - 30s to send the first bytes
635
- - 10s between chunks of a request
636
- - 65s idle between keep-alive requests
646
+ <big>The HTTP/1.1 defaults are 30 seconds for the first bytes,
647
+ 10 seconds between request chunks, and 65 seconds idle on keep-alive.</big>
637
648
 
638
- Every time a connection sends bytes, its deadline moves. So the reactor needs a cheap way to keep **thousands of deadlines in order**.
649
+ As bytes arrive, deadlines move. The reactor needs to keep **thousands of them in order**.
639
650
 
640
651
  <br>
641
652
  <br>
@@ -661,7 +672,7 @@ Every time a connection sends bytes, its deadline moves. So the reactor needs a
661
672
 
662
673
  **Puma** keeps an `Array` and sorts it after changes. Sorting is O(n log n), and removing a specific connection means scanning the whole list.
663
674
 
664
- **Raptor** keeps a **red-black tree**: a binary search tree that rebalances itself as it changes.
675
+ **Raptor** keeps a **red-black tree**, a binary search tree that rebalances itself as it changes.
665
676
 
666
677
  ```mermaid
667
678
  flowchart TB
@@ -676,9 +687,9 @@ flowchart TB
676
687
  class LL,LR red
677
688
  ```
678
689
 
679
- <big>Inserting, removing, and finding the soonest deadline are all O(log n). Expiring connections means walking from the leftmost node until one isn't due.</big>
690
+ <big>Inserting and removing are O(log n). Finding the soonest deadline is O(1), because it's cached.</big>
680
691
 
681
- With a hundred connections you wouldn't notice the difference, but it adds up once you have thousands.
692
+ The reactor removes expired deadlines from the front. Keeping this cheap matters with thousands of connections.
682
693
 
683
694
  That one became another gem, **[`red-black-tree`](https://github.com/joshuay03/red-black-tree)**.
684
695
 
@@ -705,6 +716,7 @@ That one became another gem, **[`red-black-tree`](https://github.com/joshuay03/r
705
716
  ## Writing the response
706
717
 
707
718
  Once your app returns, the response still has to become bytes on the socket.
719
+ Here's the common HTTP/1.1 path with an array body.
708
720
 
709
721
  ```mermaid
710
722
  flowchart LR
@@ -717,17 +729,17 @@ flowchart LR
717
729
 
718
730
  subgraph R["Raptor"]
719
731
  direction TB
720
- RH["status line + headers<br/>(built in C)"] --> RV["writev"]
732
+ RH["cached status line<br/>+ headers formatted in C"] --> RV["writev"]
721
733
  RBody["body"] --> RV
722
- RV -->|"one syscall, no copy"| RS(["socket"])
734
+ RV -->|"headers + body together"| RS(["socket"])
723
735
  end
724
736
 
725
737
  P ~~~ R
726
738
  ```
727
739
 
728
- <big>Puma builds the response in Ruby, then either copies small bodies into its header buffer or corks the socket for larger ones.</big>
740
+ <big>Puma formats headers in Ruby and copies small bodies into the same buffer. It asks the OS to batch separate writes.</big>
729
741
 
730
- <big>Raptor builds it in C and sends the headers and body together with one `writev` call, without copying them into one string.</big>
742
+ <big>Raptor formats headers in C. `writev` sends headers and body without joining them into one string, in one syscall when possible.</big>
731
743
  <br>
732
744
  <br>
733
745
  <br>
@@ -750,7 +762,8 @@ flowchart LR
750
762
 
751
763
  ## HTTP/2 in one slide
752
764
 
753
- HTTP/1.1 sends one request at a time on a connection. HTTP/2 sends many at once, split into small binary frames:
765
+ HTTP/1.1 sends one request at a time on a connection. HTTP/2 sends many at once,
766
+ split into small binary frames.
754
767
 
755
768
  ```mermaid
756
769
  sequenceDiagram
@@ -766,9 +779,7 @@ sequenceDiagram
766
779
  S->>C: HEADERS (stream 5): 201
767
780
  ```
768
781
 
769
- - <big>**Multiplexing**: many requests (streams) share one connection</big>
770
- - <big>**Header compression** (HPACK): headers you've already sent become tiny numbers</big>
771
- - <big>**Flow control**: each side says how much data it's ready to receive</big>
782
+ <big>**Multiplexing** lets many requests share one connection. **Header compression** reuses earlier headers. **Flow control** limits how much data each side can send.</big>
772
783
 
773
784
  Raptor speaks HTTP/2 natively, and so does Falcon. Puma [leaves it to a proxy](https://github.com/puma/puma/issues/2697) like nginx.
774
785
 
@@ -796,7 +807,7 @@ Raptor speaks HTTP/2 natively, and so does Falcon. Puma [leaves it to a proxy](h
796
807
 
797
808
  <big>HTTP/2 gets its own C parser and Ractor pool. Each stream is its own job, so one connection can use several app threads.</big>
798
809
 
799
- <big>Only the reactor ever writes to the socket. App threads hand it frames and carry on.</big>
810
+ <big>For established connections, the reactor owns socket writes. App threads queue frames rather than competing to write to the same socket.</big>
800
811
 
801
812
  ```mermaid
802
813
  flowchart LR
@@ -807,7 +818,7 @@ flowchart LR
807
818
  R --> S(["socket"])
808
819
  ```
809
820
 
810
- <big>No thread ever waits for a slow client to read, and there's no lock around the socket because only one thread owns it.</big>
821
+ <big>An ordinary response can still wait for HTTP/2 flow control. Long-lived streams need a way to release the app thread too.</big>
811
822
 
812
823
  <br>
813
824
  <br>
@@ -831,9 +842,7 @@ flowchart LR
831
842
 
832
843
  ## Streams that outlive the request
833
844
 
834
- Some responses stay open for minutes, like server-sent events or gRPC streams. Normally each one ties up an app thread the whole time.
835
-
836
- Raptor's answer is `Raptor::DetachedBody`:
845
+ SSE and gRPC responses can stay open for minutes. `Raptor::DetachedBody` lets them release the app thread.
837
846
 
838
847
  ```ruby
839
848
  run ->(env) {
@@ -842,7 +851,6 @@ run ->(env) {
842
851
  body.on_close { Subscribers.remove(body) }
843
852
  [200, { "content-type" => "text/event-stream" }, body]
844
853
  }
845
-
846
854
  # Later, from any thread
847
855
  Subscribers.each { |body| body.try_write("data: update\n\n") }
848
856
  ```
@@ -855,19 +863,15 @@ sequenceDiagram
855
863
  participant C as Client
856
864
 
857
865
  A->>R: hand over the headers and the body
858
- A->>P: on_open adds the body
859
- Note over A: Thread goes back to the pool
860
866
  P->>R: try_write(message)
861
867
  R->>C: write when the socket is ready
862
868
  C--xR: client disconnects
863
- R->>P: on_close removes the body
869
+ R->>A: schedule on_close on the app pool
864
870
  ```
865
871
 
866
- <big>The app thread returns straight away and the reactor owns the stream from then on. `try_write` never blocks.</big>
867
-
868
- <big>When the client disconnects or the server shuts down, `on_close` runs immediately. It works over both HTTP/1.1 and HTTP/2.</big>
872
+ <big>`try_write` never waits for the client. It returns `:accepted`, `:full` (no buffer space), or `:closed`. Your publisher decides whether to retry or drop an update.</big>
869
873
 
870
- The first time I tried this at work, every new stream still held a thread for about 1.5ms. The handoff was polling, so I changed it to wake the thread instead.
874
+ Cleanup runs on the app pool via `on_close`. This works over HTTP/1.1 and HTTP/2.
871
875
 
872
876
  <br>
873
877
  <br>
@@ -891,17 +895,18 @@ The first time I tried this at work, every new stream still held a thread for ab
891
895
 
892
896
  ## Enough HTTP/2 for gRPC
893
897
 
894
- gRPC runs over HTTP/2 and relies on parts of it most apps never touch. Raptor supports them, so a `DetachedBody` can serve gRPC:
898
+ Raptor provides the HTTP/2 pieces for unary and server-streaming gRPC.
899
+ `DetachedBody` streams the response after the request body arrives in full.
895
900
 
896
901
  ```ruby
897
- body.try_write(encoded_message)
898
- body.close(trailers: { "grpc-status" => "0" })
902
+ if body.try_write(encoded_message) == :accepted
903
+ body.close(trailers: { "grpc-status" => "0" })
904
+ end # On :full, retry later rather than losing the message.
899
905
  ```
900
906
 
901
- - <big>**Trailers**: headers sent after the body. gRPC puts its final status there, because it isn't known until the stream ends.</big>
902
- - <big>**Cancellation**: when a client gives up on a stream, Raptor passes that on to your body straight away</big>
903
- - <big>**Keepalive PINGs**: idle connections get pinged, so dead ones are noticed and closed</big>
904
- - <big>**Graceful shutdown**: `GOAWAY` tells clients to stop opening new streams while the current ones finish</big>
907
+ <big>**Trailers** carry gRPC's final status after the body. **Cancellation** closes abandoned streams, and **PINGs** help detect dead connections.</big>
908
+
909
+ <big>On shutdown, `GOAWAY` tells clients to stop opening streams while existing ones finish.</big>
905
910
 
906
911
  <br>
907
912
  <br>
@@ -923,11 +928,11 @@ body.close(trailers: { "grpc-status" => "0" })
923
928
  <br>
924
929
  <br>
925
930
 
926
- ## Zooming out: the processes
931
+ ## Zooming out to the processes
927
932
 
928
933
  So far we've been inside one worker. Let's look at how the workers live together.
929
934
 
930
- **Preforking**: the master loads your app once, then forks a copy for each worker.
935
+ With **preforking**, the master loads your app once, then forks a copy for each worker.
931
936
 
932
937
  ```mermaid
933
938
  flowchart TB
@@ -964,9 +969,9 @@ flowchart TB
964
969
 
965
970
  ## Refork
966
971
 
967
- Copy-on-write savings fade as workers warm caches, compile YJIT code, and load gems lazily. Eventually each worker has its own copy of most pages.
972
+ Copy-on-write savings fade as workers warm caches, compile YJIT code, and load gems lazily. Workers can lose much of that sharing as they warm up independently.
968
973
 
969
- The fix: periodically fork new workers from a **warm** worker instead of the master.
974
+ The fix is to fork replacement workers from a **warm** worker instead of the master.
970
975
 
971
976
  ```mermaid
972
977
  sequenceDiagram
@@ -974,7 +979,7 @@ sequenceDiagram
974
979
  participant W as Warm worker
975
980
  participant N as New workers
976
981
 
977
- Note over M: Every refork_after requests
982
+ Note over M: Warm worker reaches the configured request threshold
978
983
  M->>W: You're the seed now
979
984
  W->>W: Finish in-flight requests
980
985
  M->>W: Fork replacement workers
@@ -982,9 +987,11 @@ sequenceDiagram
982
987
  Note over N: Start with the warm caches and YJIT code,<br/>shared copy-on-write
983
988
  ```
984
989
 
985
- <big>The idea comes from Shopify's [Pitchfork](https://github.com/Shopify/pitchfork). Puma's version is the experimental `fork_worker`, which I've fixed a few bugs in.</big>
990
+ <big>The idea comes from Shopify's [Pitchfork](https://github.com/Shopify/pitchfork). Puma has experimental `fork_worker` support too.</big>
986
991
 
987
- <big>It relies on `PR_SET_CHILD_SUBREAPER`, so the master can adopt workers it didn't fork. That makes it Linux only.</big>
992
+ <big>I recently added [`mold_worker`](https://github.com/puma/puma/pull/4038) on Puma's main branch. Like Raptor's seed, the mold stops serving requests and only forks replacements.</big>
993
+
994
+ Raptor uses Linux's `PR_SET_CHILD_SUBREAPER` so the master can adopt workers it didn't fork.
988
995
 
989
996
  <br>
990
997
  <br>
@@ -1029,7 +1036,7 @@ flowchart LR
1029
1036
 
1030
1037
  <big>The master creates the region with `mmap` before forking, so every worker inherits it. Each worker owns one 49-byte slot, so writes never clash.</big>
1031
1038
 
1032
- <big>Updating stats is just a write to memory, with no pipe, no parsing, and no syscall.</big>
1039
+ <big>Updating stats is a write to shared memory, with no pipe or syscall to send the update.</big>
1033
1040
 
1034
1041
  That's where **[`mmap-ruby`](https://github.com/joshuay03/mmap-ruby)** came from.
1035
1042
 
@@ -1055,11 +1062,11 @@ That's where **[`mmap-ruby`](https://github.com/joshuay03/mmap-ruby)** came from
1055
1062
 
1056
1063
  ## Which worker gets the next connection?
1057
1064
 
1058
- All workers listen on the same port, and by default Linux spreads connections by hashing the client's address. That's even **on average**, but blind to which workers are busy.
1065
+ With Linux's `SO_REUSEPORT`, workers get separate listeners on the same port. The kernel hashes connection details to pick one, without knowing which workers are busy.
1059
1066
 
1060
- Puma nudges the odds. Busier workers pause briefly before accepting, so quieter ones tend to win.
1067
+ In Puma, busier workers pause briefly before accepting new connections. This gives less busy workers more chance to accept them.
1061
1068
 
1062
- Raptor lets the kernel decide using real load, with **BPF**.
1069
+ On Linux, Raptor can use **BPF** to route plain TCP connections using real load.
1063
1070
 
1064
1071
  <big>BPF lets you run tiny, kernel-verified programs inside Linux. It's the same technology behind `bpftrace` and Cilium.</big>
1065
1072
 
@@ -1086,22 +1093,25 @@ Raptor lets the kernel decide using real load, with **BPF**.
1086
1093
  ## Raptor's BPF program
1087
1094
 
1088
1095
  ```mermaid
1089
- flowchart TB
1096
+ flowchart LR
1090
1097
  New(["new connection"]) --> Hash["hash picks two workers"]
1091
- Hash --> A["Worker 0<br/>5 waiting requests"]
1092
- Hash --> B["Worker 3<br/>6 waiting requests"]
1098
+ Hash --> A["Worker 0<br/>load 5"]
1099
+ Hash --> B["Worker 3<br/>load 6"]
1093
1100
  A --> Pick{"pick the less busy one"}
1094
1101
  B --> Pick
1095
- Pick -->|"reserve: 5 → 6"| W0["Worker 0 accepts it"]
1102
+ Pick -->|"reserve 5 → 6"| W0["Worker 0 accepts it"]
1096
1103
  ```
1097
1104
 
1098
- <big>Each worker publishes its load, including connections still waiting to be accepted, to a map the kernel can read. The program compares two workers and picks the quieter one ("power of two choices").</big>
1105
+ <big>A kernel-readable map tracks each worker's load, including pending connections.
1106
+ The program compares two workers and picks the quieter one.</big>
1099
1107
 
1100
- <big>It bumps that worker's count straight away, so a burst doesn't all land on one worker.</big>
1108
+ <big>Checking every worker adds work to every connection. The "power of two choices"
1109
+ helps avoid busy workers without scanning the whole cluster.</big>
1101
1110
 
1102
- <big>The whole program is about 70 lines of C. If BPF isn't available, Raptor quietly falls back to the default behaviour.</big>
1111
+ It reserves the connection straight away, so the next arrival sees the updated load.
1103
1112
 
1104
- To load it from Ruby, I wrote **[`libbpf-ruby`](https://github.com/joshuay03/libbpf-ruby)**.
1113
+ The program is about 70 lines of C, loaded through **[`libbpf-ruby`](https://github.com/joshuay03/libbpf-ruby)**.
1114
+ Without the optional dependencies, Raptor uses normal listeners.
1105
1115
 
1106
1116
  <br>
1107
1117
  <br>
@@ -1125,11 +1135,82 @@ To load it from Ruby, I wrote **[`libbpf-ruby`](https://github.com/joshuay03/lib
1125
1135
 
1126
1136
  ## Pinning workers to cores
1127
1137
 
1128
- Linux moves processes between cores whenever it likes, and each move lands a worker on cold caches.
1138
+ Moving workers between cores can lose warm CPU caches.
1139
+
1140
+ <big>On Linux, `cpu_affinity` pins workers to cores when enough are available.
1141
+ Puma doesn't have a built-in setting for this.</big>
1142
+
1143
+ <big>It's opt-in, because a pinned worker's threads and Ractors share one core.</big>
1144
+
1145
+ <br>
1146
+ <br>
1147
+ <br>
1148
+ <br>
1149
+ <br>
1150
+ <br>
1151
+ <br>
1152
+ <br>
1153
+
1154
+ ---
1155
+
1156
+ <br>
1157
+ <br>
1158
+ <br>
1159
+ <br>
1160
+ <br>
1161
+ <br>
1162
+ <br>
1163
+ <br>
1164
+
1165
+ ## The numbers over HTTP/1.1 without keep-alive
1166
+
1167
+ Scaling uses 3–16 threads per worker. Fixed uses 3. Falcon uses unbounded fibers.
1168
+
1169
+ **I/O-bound requests**
1170
+
1171
+ <big>With scaling threads, Raptor matches Puma on throughput and p95.
1172
+ Falcon has 38% higher throughput than Raptor, with lower p95.
1173
+ With 3 fixed threads, Raptor has over twice Puma's throughput.</big>
1174
+
1175
+ **CPU-bound requests**
1176
+
1177
+ <big>With scaling threads, Raptor gets about 23% more throughput than Puma, with 10% higher p95.
1178
+ With 3 fixed threads, the throughput lead is 10%. Falcon leads on both metrics.</big>
1179
+
1180
+ <br>
1181
+ <br>
1182
+ <br>
1183
+ <br>
1184
+ <br>
1185
+ <br>
1186
+ <br>
1187
+ <br>
1188
+
1189
+ ---
1190
+
1191
+ <br>
1192
+ <br>
1193
+ <br>
1194
+ <br>
1195
+ <br>
1196
+ <br>
1197
+ <br>
1198
+ <br>
1199
+
1200
+ ## The numbers over HTTP/1.1 keep-alive
1129
1201
 
1130
- <big>With `cpu_affinity` on, each worker pins itself to its own core and keeps its caches warm. Puma doesn't offer this.</big>
1202
+ Scaling uses 3–16 threads per worker. Fixed uses 3. Falcon uses unbounded fibers.
1131
1203
 
1132
- <big>It's opt-in, because a pinned worker's threads and Ractors share one core. I've seen it help, but I don't have numbers yet.</big>
1204
+ **I/O-bound requests**
1205
+
1206
+ <big>With scaling threads, Raptor matches Puma and Falcon on throughput and p95.
1207
+ With 3 fixed threads, Raptor has over twice Puma's throughput.</big>
1208
+
1209
+ **CPU-bound requests**
1210
+
1211
+ <big>With scaling threads, Raptor gets 7% more throughput and 20% lower p95 than scaling Puma.
1212
+ With 3 fixed threads, Puma leads on both metrics.
1213
+ Falcon has higher throughput, but Raptor's p95 is lower.</big>
1133
1214
 
1134
1215
  <br>
1135
1216
  <br>
@@ -1151,21 +1232,21 @@ Linux moves processes between cores whenever it likes, and each move lands a wor
1151
1232
  <br>
1152
1233
  <br>
1153
1234
 
1154
- ## The numbers
1235
+ ## The numbers over HTTP/2
1236
+
1237
+ Raptor vs Falcon. Puma has no native HTTP/2.
1238
+
1239
+ **I/O-bound requests**
1155
1240
 
1156
- Full results and the setup are in the [README](../README.md#micro-benchmarks). **I/O** requests mostly wait on fake database calls, and **CPU** requests mostly run Ruby.
1241
+ <big>With scaling threads, Raptor matches Falcon's throughput, with p95 around 30 ms versus 27 ms.
1242
+ With 3 fixed threads, throughput is 78% lower than Falcon.</big>
1157
1243
 
1158
- - <big>**I/O-heavy HTTP/1.1 with 3 threads per worker**: roughly 2.3 times Puma's throughput</big>
1159
- - Raptor polls for accept capacity under saturation. That trades CPU time for throughput and latency; the same thread count doesn't mean the same CPU use.
1160
- - Existing keep-alive traffic can also delay new connections at saturation; throughput and p95 don't capture every client's wait.
1161
- - <big>**I/O-heavy HTTP/1.1 with scaling threads**: approximately tied with Puma</big>
1162
- - Puma adds a thread whenever work is waiting. Raptor only adds one when the busy threads are blocked, to avoid adding threads CPU-heavy apps can't use.
1163
- - <big>**HTTP/2**: direct multiplexing, compared with Falcon</big>
1164
- - Puma doesn't implement HTTP/2. Raptor and Falcon do; their numbers move around a lot between runs, so check the spread as well as the medians.
1165
- - <big>**I/O with server-sent events and 3 threads per worker**: 9.5 times Puma's throughput, with Puma also reporting timeouts</big>
1166
- - A detached body hands the stream to the reactor, so the app thread is free again. On Puma, every open stream holds a thread until it closes.
1244
+ **CPU-bound requests**
1167
1245
 
1168
- Raptor wins on throughput in three of the four CPU-heavy HTTP/1.1 comparisons; Puma leads slightly with fixed threads and keep-alive. Puma still has lower p95 on fresh connections, while Raptor has lower p95 with keep-alive. These are micro benchmarks though, so benchmark your own application before choosing a server.
1246
+ <big>With scaling threads, Raptor gets 21% more throughput than Falcon at about half the p95.
1247
+ With 3 fixed threads, the throughput lead is 33%.</big>
1248
+
1249
+ Full results and the setup are in the [README](../README.md#micro-benchmarks).
1169
1250
 
1170
1251
  <br>
1171
1252
  <br>
@@ -1189,10 +1270,12 @@ Raptor wins on throughput in three of the four CPU-heavy HTTP/1.1 comparisons; P
1189
1270
 
1190
1271
  ## What you give up
1191
1272
 
1192
- - <big>**Battle-testing**: Puma has run production Rails since 2011. Raptor is less than a year old.</big>
1193
- - <big>**Older Rubies**: Raptor needs Ruby 4.0.</big>
1194
- - <big>**Ractor overhead**: on Ruby 4.0, once a process starts a Ractor, the rest of its Ruby runs slower too. On the upcoming Ruby 4.1, it's gone in my measurements.</big>
1195
- - <big>**Easy debugging**: a request can cross threads and Ractors, so tracing it means stitching stack traces together.</big>
1273
+ <big>Raptor has less **battle-testing**. Puma has years of production Rails use. Raptor is less than a year old.</big>
1274
+
1275
+ <big>It needs **Ruby 4.0 or newer**. Ractors add overhead on 4.0 that I don't see in my 4.1 measurements.</big>
1276
+
1277
+ <big>**Debugging server internals** can involve tracing work across threads and Ractors.
1278
+ Most app developers won't need to do that. Your app stays on ordinary threads.</big>
1196
1279
 
1197
1280
  <br>
1198
1281
  <br>
@@ -1216,14 +1299,18 @@ Raptor wins on throughput in three of the four CPU-heavy HTTP/1.1 comparisons; P
1216
1299
 
1217
1300
  ## Why it's worth being curious
1218
1301
 
1219
- I'm not asking you to swap your production server. What I hope you take away is how much more is possible from Ruby now:
1302
+ I'd love you to try Raptor, but I also hope you leave curious about what's possible from Ruby now.
1303
+
1304
+ <big>Raptor combines Ractors, lock-free queues, shared memory, and BPF in one Rack server.</big>
1305
+
1306
+ <big>Let the pool adapt instead of choosing one fixed thread count.
1307
+ Your app still runs on ordinary Rack threads.</big>
1308
+
1309
+ In these tests, scaling threads match Falcon's I/O throughput over keep-alive HTTP/1.1 and HTTP/2.
1220
1310
 
1221
- - <big>Ractors are finally practical</big>
1222
- - <big>We can build lock-free data structures in Ruby</big>
1223
- - <big>We can share memory between processes with `mmap`</big>
1224
- - <big>We can load BPF programs into the kernel from Ruby</big>
1311
+ <big>Native HTTP/2 and detached streaming are there when you need them.</big>
1225
1312
 
1226
- None of these ideas are new on their own. What's new is that we can combine them in one Ruby server, and see how far that gets us.
1313
+ Try your own workload to see how these pieces work together.
1227
1314
 
1228
1315
  <br>
1229
1316
  <br>
@@ -1248,15 +1335,17 @@ None of these ideas are new on their own. What's new is that we can combine them
1248
1335
  ## Try it
1249
1336
 
1250
1337
  ```
1251
- gem install raptor # Ruby 4.0+
1338
+ bundle add raptor # Ruby 4.0+
1252
1339
  bundle exec raptor # boots config.ru
1253
1340
  bundle exec rails server -u raptor
1254
1341
  ```
1255
1342
 
1256
- - <big>The CLI mirrors Puma's on purpose</big>
1257
- - <big>Feedback and bug reports very welcome</big>
1343
+ <big>The CLI mirrors Puma's on purpose. Try it on your app.
1344
+ Feedback and bug reports are very welcome.</big>
1258
1345
 
1259
- The long version of this talk, with much more detail, is [`docs/raptor-vs-puma.md`](raptor-vs-puma.md).
1346
+ Test your real traffic, then try a small production canary with an easy way back to Puma.
1347
+
1348
+ For a detailed comparison of Raptor and Puma's designs, see [`docs/raptor-vs-puma.md`](raptor-vs-puma.md).
1260
1349
 
1261
1350
  <br>
1262
1351
  <br>
@@ -1286,10 +1375,14 @@ The long version of this talk, with much more detail, is [`docs/raptor-vs-puma.m
1286
1375
 
1287
1376
  **The gems that came out of it**
1288
1377
 
1289
- - <big>[`ractor-pool`](https://github.com/joshuay03/ractor-pool): a pool of Ractor workers</big>
1290
- - <big>[`atomic-ruby`](https://github.com/joshuay03/atomic-ruby): lock-free primitives and the thread pool</big>
1291
- - <big>[`red-black-tree`](https://github.com/joshuay03/red-black-tree): a self-balancing tree</big>
1292
- - <big>[`mmap-ruby`](https://github.com/joshuay03/mmap-ruby): shared memory</big>
1293
- - <big>[`libbpf-ruby`](https://github.com/joshuay03/libbpf-ruby): BPF from Ruby</big>
1378
+ [`ractor-pool`](https://github.com/joshuay03/ractor-pool) provides Ractor workers.
1379
+
1380
+ [`atomic-ruby`](https://github.com/joshuay03/atomic-ruby) provides lock-free primitives and the thread pool.
1381
+
1382
+ [`red-black-tree`](https://github.com/joshuay03/red-black-tree) keeps deadlines in order.
1383
+
1384
+ [`mmap-ruby`](https://github.com/joshuay03/mmap-ruby) provides shared memory.
1385
+
1386
+ [`libbpf-ruby`](https://github.com/joshuay03/libbpf-ruby) loads BPF programs from Ruby.
1294
1387
 
1295
1388
  _Questions?_