raptor 0.10.0 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1b7d4b6790a256864f0c2b4e8d061231c372a110493ef2846f547f4566232d5d
-  data.tar.gz: dc6454ad796395d9eb7ab81327410ddd2004f0e1dd0bb03857636c508e84f7bc
+  metadata.gz: be4ff0cf1729c188fab8261b749c2a279616bd565dbad81d18f5a3e75311b734
+  data.tar.gz: 0acc2ece57f69207a34696df792e45bf52bfce22bb7617c5ceb3113223c175f5
 SHA512:
-  metadata.gz: 6c0198c8697a569b7caf4492cf80c64edfbe7e6d0c1b4d7be7880175655bfa36b89040d3ab5b9597db995ac469eed26a1c74b7bb7c0c3206c9c47193b3930a48
-  data.tar.gz: 6a73d6a927a118dea756b8012108dbdaf1f2462ea9ea5a6fbd4dabd56cfffa51a23f059f2c51c39cd5a4f41a3fc0e752022cd1963115edacecb3cd53e56b83fe
+  metadata.gz: 787fabc9128f3c4e16a394dc5634ff8435916b60b68ff3f6d569b0db561f07af0c61ab5d17e7440301c850d87fbbe1f0dc3691274b1c060d1cbbd6f7bc84bfbf
+  data.tar.gz: 26d154b6c0abbc63986262e1212e526bd9e4e492d06f9a86e723e73fd85afd142550a15324b55ad91b575633db2d0231f262aebb818f198e2de4c96c9afa8666

data/CHANGELOG.md

@@ -1,5 +1,19 @@
 ## [Unreleased]
 
+## [0.11.0] - 2026-07-12
+
+- Yield subsequent HTTP/1.1 keep-alive requests to the thread pool only under saturation
+- Reuse a per-thread HTTP/1.1 parser across requests
+- Skip a per-response allocation when formatting response headers
+- Reject HTTP/2 streams with malformed pseudo-headers with `RST_STREAM`
+- Populate `PATH_INFO` from absolute-form request targets
+- Consume the trailer section after the chunked-body terminator
+- Reject request headers larger than 112KB with 400
+- Reject chunk sizes containing non-hex characters with 400
+- Advance the parser's `nread` past the request header terminator
+- Detect chunked `Transfer-Encoding` case-insensitively
+- Reject HTTP/1.1 requests without a valid `Host` header with 400
+
 ## [0.10.0] - 2026-07-07
 
 - Memoize the server port string for the Rack env

data/README.md

@@ -37,7 +37,7 @@ run proc { |_env| [200, { "content-type" => "text/plain" }, ["Hello, World!"]] }
 ```
 > bundle exec raptor -w 4 -t 3 hello_world.ru
 [Raptor 76577|Main|Main] Cluster initializing:
-[Raptor 76577|Main|Main] ├─ Version: 0.9.0
+[Raptor 76577|Main|Main] ├─ Version: 0.11.0
 [Raptor 76577|Main|Main] ├─ Ruby Version: ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +YJIT +PRISM [arm64-darwin23]
 [Raptor 76577|Main|Main] ├─ Environment: development
 [Raptor 76577|Main|Main] ├─ Master PID: 76577
@@ -187,20 +187,20 @@ Worker 1 (phase 0): pid=91351, requests=1199, busy=1/3, backlog=0, booted, last_
 
 ## (Micro) Benchmarks
 
-Raptor 0.10.0 vs Puma 8.0.2, median of 3 runs across two workload profiles: IO-bound (sleep for a random 5-50ms then
+Raptor 0.11.0 vs Puma 8.0.2, median of 3 runs across two workload profiles: IO-bound (sleep for a random 1-10ms then
 return small JSON) and CPU-bound (serialise a JSON array of 20-200 items).
 
 | Protocol              | Workload | Raptor       | Puma         | +/- vs Puma |
 | --------------------- | -------- | ------------ | ------------ | ----------- |
-| HTTP/1.1              | IO       | 0.43k req/s  | 0.41k req/s  | +4.0%       |
-| HTTP/1.1              | CPU      | 10.99k req/s | 9.22k req/s  | +19.1%      |
-| HTTP/1.1 (keep-alive) | IO       | 0.41k req/s  | 0.40k req/s  | +2.6%       |
-| HTTP/1.1 (keep-alive) | CPU      | 27.42k req/s | 25.75k req/s | +6.5%       |
-| HTTP/2                | IO       | 0.31k req/s  | N/A          | -           |
-| HTTP/2                | CPU      | 26.28k req/s | N/A          | -           |
+| HTTP/1.1              | IO       | 2.05k req/s  | 1.94k req/s  | +5.7%       |
+| HTTP/1.1              | CPU      | 13.66k req/s | 9.21k req/s  | +48.3%      |
+| HTTP/1.1 (keep-alive) | IO       | 2.08k req/s  | 1.95k req/s  | +6.6%       |
+| HTTP/1.1 (keep-alive) | CPU      | 27.85k req/s | 27.47k req/s | +1.4%       |
+| HTTP/2                | IO       | 0.96k req/s  | N/A          | -           |
+| HTTP/2                | CPU      | 28.92k req/s | N/A          | -           |
 
 > ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +YJIT +PRISM [aarch64-linux]
-> 4 workers, 3 threads, 24 concurrent connections
+> 4 workers, 3 threads, 48 concurrent connections
 
 See [bin/benchmark](bin/benchmark) for more details.
 

data/docs/raptor-vs-puma.md

@@ -12,7 +12,7 @@ This document walks through both servers at systems-design depth. By the end you
 
 ## The shape of the benchmark
 
-The Raptor README carries the [current head-to-head numbers](../README.md#micro-benchmarks) against the latest Puma release, run on the same hardware with the same Rack app, 4 workers, 3 threads, 24 concurrent connections, on a recent Ruby with YJIT enabled. Two workload profiles are measured: **IO-bound** (each request sleeps for a random 5-50ms then returns a small JSON response) and **CPU-bound** (each request serialises a JSON array of 20-200 items). Rather than pin specific numbers into this document (they drift with Ruby versions and hardware), the shape of the result is what matters.
+The Raptor README carries the [current head-to-head numbers](../README.md#micro-benchmarks) against the latest Puma release, run on the same hardware with the same Rack app, 4 workers, 3 threads, 48 concurrent connections, on a recent Ruby with YJIT enabled. Two workload profiles are measured: **IO-bound** (each request sleeps for a random 1-10ms then returns a small JSON response) and **CPU-bound** (each request serialises a JSON array of 20-200 items). Rather than pin specific numbers into this document (they drift with Ruby versions and hardware), the shape of the result is what matters.
 
 - On IO-bound work, both servers land near parity across all protocols; throughput is bounded by the per-request sleep and the benchmark client's concurrency, not by anything the server does.
 - On CPU-bound HTTP/1.1 without keep-alive, Raptor holds a meaningful lead; per-request parsing and response-writing overhead is a real fraction of the work.
@@ -84,7 +84,7 @@ Puma stores its reactor timeouts in a Ruby array of `Client` objects. New client
 
 Cross-thread waking is done through a pipe. When work needs to enter the reactor from another thread (a Client being registered), a byte is written to a pipe that the selector is also watching; the selector wakes, drains the input queue, and gets on with it.
 
-### Request lifecycle (HTTP/1.1)
+### HTTP/1.1 request lifecycle
 
 Putting the pieces together, a request through Puma looks like this:
 
@@ -104,16 +104,17 @@ Response writing is done from the app thread. Puma sets `TCP_CORK` on Linux, wri
 ```mermaid
 flowchart TB
     subgraph Master["Master process, cluster mode"]
-        M["Master loop"]
-        M -->|fork| W1["Worker 0"]
-        M -->|fork| W2["Worker 1"]
-        M -->|fork| W3["Worker N"]
+        MM["Master loop<br/>Reads worker pipes for<br/>per-worker vitals<br/>Handles signals"]
+        MM -->|"fork"| W1["Worker 0"]
+        MM -->|"fork"| W2["Worker 1"]
+        MM -->|"fork"| W3["Worker N"]
+        MM -.->|"SIGUSR2<br/>exec with argv"| MM
     end
 
-    subgraph Worker["Puma worker process"]
-        A["Server thread<br/>IO.select on listeners<br/>accept_nonblock"]
+    subgraph Worker["Puma worker process, one of N"]
+        SRV["Server thread<br/>IO.select on listeners<br/>accept_nonblock"]
 
-        R["Reactor thread<br/>NIO::Selector<br/>sorted timeout array"]
+        RCT["Reactor thread<br/>NIO::Selector<br/>sorted timeout array"]
 
         subgraph TP["Thread pool, Mutex + CondVar"]
             T1["Thread 1"]
@@ -121,37 +122,45 @@ flowchart TB
             T3["Thread N"]
         end
 
+        STA["Stat thread<br/>writes to worker pipe<br/>on worker_check_interval"]
+
         C1{"Complete<br/>request<br/>on accept?"}
         C2{"Complete<br/>request<br/>after read?"}
         KA{"Keep-alive?"}
         BB{"Back-to-back<br/>data in buffer?"}
 
-        A -->|"push new socket"| TP
+        SRV -->|"push new socket"| TP
         TP -->|"eagerly_finish"| C1
         C1 -->|"complete, app.call"| APP["Rack app"]
-        C1 -->|"incomplete, register"| R
-        R -->|"socket readable"| C2
+        C1 -->|"incomplete, register"| RCT
+        RCT -->|"socket readable"| C2
         C2 -->|"complete, push"| TP
-        C2 -->|"incomplete, keep waiting"| R
-        R -->|"timeout"| CLOSE1["write 408, close"]
+        C2 -->|"incomplete, keep waiting"| RCT
+        RCT -->|"timeout"| TO["write 408, close"]
         APP -->|"write response"| KA
-        KA -->|"no"| CLOSE2["close socket"]
+        KA -->|"no"| CLS["close socket"]
         KA -->|"yes"| BB
         BB -->|"yes, reprocess"| TP
-        BB -->|"no, back to reactor"| R
+        BB -->|"no, back to reactor"| RCT
+
+        STA -.->|"writes ping"| PIPE[("worker→master pipe")]
     end
 
+    MM -.->|"reads ping"| PIPE
+
     classDef accept fill:#dbeafe,stroke:#2563eb,color:#1e3a8a
     classDef reactor fill:#fecdd3,stroke:#e11d48,color:#881337
     classDef pool fill:#fef3c7,stroke:#d97706,color:#78350f
     classDef app fill:#d1fae5,stroke:#059669,color:#064e3b
-    class A accept
-    class R reactor
+    classDef storage fill:#e5e7eb,stroke:#6b7280,color:#374151
+    class SRV accept
+    class RCT reactor
     class TP pool
     class APP app
+    class PIPE storage
 ```
 
-The key thing to notice: the parser runs inside an app thread. Puma's concurrency model is "many app threads, each doing one full request (parse plus app plus write) at a time". Under MRI's GVL this is genuine concurrency but not parallelism; at any given moment at most one of those threads is executing Ruby, and the others are either blocked on I/O (which releases the GVL and lets another thread proceed) or waiting their turn. Puma leans on the fact that most Rack apps spend most of their time blocked on the database or an external API, where threads do effectively overlap.
+Notably, the parser runs inside an app thread. Puma's concurrency model is "many app threads, each doing one full request (parse plus app plus write) at a time". Under MRI's GVL this is genuine concurrency but not parallelism; at any given moment at most one of those threads is executing Ruby, and the others are either blocked on I/O (which releases the GVL and lets another thread proceed) or waiting their turn. Puma leans on the fact that most Rack apps spend most of their time blocked on the database or an external API, where threads do effectively overlap.
 
 ## Part II: Raptor
 
@@ -175,7 +184,7 @@ Master-to-worker communication does not use pipes. Every worker writes its stats
 
 On Linux, each worker pins itself to a distinct CPU via `sched_setaffinity` when the worker count fits within the process's allowed CPU set, so it stays on one core and its L1/L2 caches stay warm. When workers outnumber available CPUs the pin is skipped and the kernel scheduler manages placement.
 
-### Per-worker threading model
+### Threading model
 
 This is where Raptor diverges dramatically. Inside a worker there are five kinds of concurrent activity:
 
@@ -188,11 +197,13 @@ This is where Raptor diverges dramatically. Inside a worker there are five kinds
 
 That is a lot of moving parts. Let us go through why.
 
-**Why Ractors for parsing.** Ractors are Ruby's answer to true parallelism. Multiple Ractors can execute Ruby code simultaneously on different OS threads, each with its own GVL. But Ractors are heavily restricted: they can only share frozen data, they cannot access most global mutable state, and code inside a Ractor cannot use most existing gems (which universally assume shared-state semantics).
+**Why Ractors for parsing.** Ractors are Ruby's answer to true parallelism. Multiple Ractors can execute Ruby code simultaneously on different OS threads, each with its own GVL. But Ractors are heavily restricted. They can only share frozen data, they cannot access most global mutable state, and code inside a Ractor cannot use most existing gems (which universally assume shared-state semantics).
 
 For a web server, this restriction turns out to be almost exactly right for HTTP parsing. Parsing a request is CPU-bound (tokenising bytes, uppercasing header names, decoding chunked bodies), it does not need to touch any global state, and it produces a result (a hash) that can be safely frozen and handed off. The native HTTP/1 parser (`raptor_http.c`) is declared `rb_ext_ractor_safe(true)`: it holds no per-parser Ruby state in the extension itself, and it writes only into the caller-supplied env hash. Same for the HTTP/2 parser plus HPACK.
 
-The upshot: while your Rack app runs on regular threads under the GVL (so your app does not need to be Ractor-safe), the protocol-level work runs in parallel across Ractors. Under heavy load with lots of small requests, the GVL contention that would otherwise dominate parsing simply is not there.
+The HTTP/1 parser also pre-interns the ~40 most common header keys (`HTTP_HOST`, `HTTP_USER_AGENT`, the `HTTP_ACCEPT_*` family, `CONTENT_LENGTH`, `HTTP_X_FORWARDED_*`, the `HTTP_SEC_FETCH_*` client hints, and so on) once at load time. During parsing, a `memcmp` lookup against that table returns the shared frozen `VALUE` for known keys and falls back to `rb_enc_interned_str` for the rest. Every request's env hash therefore reuses the same String object for its header names, which both skips per-request allocation and lets Ruby's hash lookup use the interned key's cached hash code.
+
+The upshot is that while your Rack app runs on regular threads under the GVL (so your app does not need to be Ractor-safe), the protocol-level work runs in parallel across Ractors. Under heavy load with lots of small requests, the GVL contention that would otherwise dominate parsing simply is not there.
 
 **How the Ractor pool actually works.** Raptor uses the `ractor-pool` gem, which is another one of my libraries. The pool has one coordinator Ractor and M pipeline Ractors. When a pipeline Ractor is idle, it sends itself back to the coordinator via `coordinator.send(Ractor.current, move: true)`. When work arrives at the coordinator, it either forwards it to a waiting Ractor (if any) or queues it. This coordinator-dispatch pattern guarantees that no Ractor sits idle while there is work. Results flow back through a shared `Ractor::Port` (a many-to-one channel added in recent Ruby versions and stable in 4.0) to a Ruby-side collector thread. If `M == 1` the coordinator is skipped and work goes straight to the single pipeline Ractor; this is the default because a single Ractor already parses in parallel with the app threads and adds enough headroom for typical workloads.
 
@@ -341,7 +352,7 @@ flowchart TB
         ATP -->|"app.call + write"| KA
         KA -->|"no, close"| CLS["close socket"]
         KA -->|"yes"| EAG
-        EAG -->|"bytes within 1ms"| ATP
+        EAG -.->|"bytes within 1ms, parse+dispatch on same thread"| ATP
         EAG -->|"no bytes, reactor.persist"| RCT
         RCT -->|"timeout expired"| TO["write 408, close"]
 
@@ -364,7 +375,7 @@ flowchart TB
     class SHM storage
 ```
 
-The critical structural difference from Puma: parsing is not on the app thread. It is on the Ractor pool. The app thread only does the Rack call and the response write. This decouples the two costs and lets Ruby actually use more than one CPU for the protocol work.
+The critical structural difference from Puma is that parsing is not on the app thread. It is on the Ractor pool. The app thread only does the Rack call and the response write. This decouples the two costs and lets Ruby actually use more than one CPU for the protocol work.
 
 ## Part III: Head to head
 
@@ -398,11 +409,11 @@ Under moderate load these look equivalent. Under high concurrency (many threads
 
 **Puma.** After a response, if the connection is keep-alive and there are already buffered bytes for the next request (`has_back_to_back_requests?`) and there is a spare app thread, loop inline. Otherwise, if `eagerly_finish` (a single non-blocking read attempt) returns true, either loop inline (if spare threads) or hand back to the thread pool (`@thread_pool << client`). Otherwise, back to the reactor with `@persistent_timeout`.
 
-**Raptor.** After a response, the app thread does `socket.wait_readable(0.001)`, waiting up to 1ms for bytes. If bytes arrive, parse and dispatch the next request inline on the same thread. Otherwise `reactor.persist` and return.
+**Raptor.** After a response, the app thread does `socket.wait_readable(0.001)`, waiting up to 1ms for bytes. If bytes arrive, it parses the next request inline. If the thread pool queue is at least as deep as the pool, the parsed request is handed back to the pool so other threads share the load; otherwise the same thread dispatches it inline. If no bytes arrive, `reactor.persist` and return.
 
-The difference is subtle but the numbers show it up. Real-world clients pipelining requests often send the next request 100-500 microseconds after the previous response completes. Puma's `eagerly_finish` catches the case where bytes are already in the socket buffer; Raptor's `wait_readable(1ms)` catches both the "already buffered" case and the "arriving very shortly" case. And Raptor never has to hand the socket back to a different thread; the response-writing thread parses the next request itself.
+The difference is subtle but the numbers show it up. Real-world clients pipelining requests often send the next request 100-500 microseconds after the previous response completes. Puma's `eagerly_finish` catches the case where bytes are already in the socket buffer; Raptor's `wait_readable(1ms)` catches both the "already buffered" case and the "arriving very shortly" case. And Raptor always parses the next request on the response-writing thread, so the parse itself never crosses a thread boundary.
 
-This is where most of Raptor's keep-alive edge comes from. Under the benchmark load, with a small number of concurrent connections all doing keep-alive, every connection is issuing back-to-back requests, and Raptor's app threads are essentially processing a single connection each in a tight loop until the connection stops. Puma's app threads do the same when they can, but the coordination overhead between thread pool and reactor for the transitions costs meaningfully more.
+This is where most of Raptor's keep-alive edge comes from. Subsequent requests are always parsed and dispatched without a reactor round-trip: the response-writing thread either continues serving that connection inline (when the pool queue is shallower than the pool) or hands the parsed request back to the pool for another thread to pick up (when it is not). Puma's app threads do the same when they can, but the coordination overhead between thread pool and reactor for the transitions costs meaningfully more.
 
 ### Backpressure
 
@@ -434,6 +445,10 @@ On the HTTP/1.1 path, Raptor has a small `writev(2)` C extension (`Raptor::Vecto
 
 HTTP/1.1 responses also reuse a per-thread String buffer for the status line and headers rather than allocating one per response. The buffer grows once to fit the largest response the thread has served and stays that size afterwards, so subsequent responses on that thread skip the allocation entirely.
 
+Chunked responses with an array or file body accumulate chunk-framed writes into the response buffer up to 512KB before flushing to the socket, collapsing what would be N `write` syscalls for an N-chunk body into a handful. Enumerable bodies (SSE, long-polling, per-line log tailing) still flush every chunk, so streaming semantics are preserved.
+
+Around the response boundary, HTTP/1.1 also amortises the common per-request allocations. A frozen Rack env template with the static keys (`rack.version`, `rack.hijack?`, `SCRIPT_NAME`, `QUERY_STRING`, `SERVER_SOFTWARE`) is duped per request, so the app-thread env build only writes the dynamic keys. Response header serialisation appends directly onto the response buffer instead of allocating intermediate `Array` wraps and `flat_map` products for each header value.
+
 ### Keep-alive request by request
 
 Because the keep-alive fast path is where Raptor's throughput edge shows up, here is each server processing three back-to-back requests on the same connection. Drawn separately so the participant columns stay wide enough to read. Compare the number of boundaries each request has to cross.
@@ -476,16 +491,12 @@ sequenceDiagram
     autonumber
     participant Client
     participant RS as Server thread
-    participant RR as Reactor thread
-    participant RC as Pipeline Ractor
     participant RP as App thread
 
     Note over Client,RP: Request 1, initial
     Client->>RS: SYN, request bytes
-    RS->>RR: reactor.add
-    RR->>RC: shareable state
-    RC->>RC: parse on own Ractor GVL
-    RC->>RP: dispatch complete request
+    RS->>RS: eager_accept, read + parse inline
+    RS->>RP: push proc to thread pool
     RP->>Client: response 1
 
     Note over Client,RP: Request 2, pipelined a few hundred us later
@@ -508,7 +519,7 @@ Puma has to bounce Request 3 through the reactor and back through the mutex-prot
 
 ### IO-bound work, close to parity
 
-On the IO-bound benchmark profile, each request sleeps for a random 5-50ms then returns a small JSON payload. Throughput is bounded by the per-request sleep and the benchmark client's 24-connection concurrency; both servers push those requests through their pipelines faster than the sleep completes, so what shows up in the numbers is the workload cost itself, not the server cost. Raptor and Puma land within a few percent of each other in both keep-alive modes because there is nothing to differentiate.
+On the IO-bound benchmark profile, each request sleeps for a random 1-10ms then returns a small JSON payload. Throughput is bounded by the per-request sleep and the benchmark client's 48-connection concurrency; both servers push those requests through their pipelines faster than the sleep completes, so what shows up in the numbers is the workload cost itself, not the server cost. Raptor and Puma land within a few percent of each other in both keep-alive modes because there is nothing to differentiate.
 
 Real applications that spend most of their time waiting on a database or an upstream service look like this. The choice between Raptor and Puma there is not throughput-driven.
 

data/ext/raptor_http/raptor_http.c

@@ -121,6 +121,21 @@ static inline void upcase_header_char(char *c) {
     *c = '_';
 }
 
+static int contains_chunked(const char *value, long len) {
+  static const char chunked[] = "chunked";
+  static const long chunked_len = sizeof(chunked) - 1;
+
+  if (len < chunked_len) return 0;
+  for (long start = 0; start + chunked_len <= len; start++) {
+    long i;
+    for (i = 0; i < chunked_len; i++) {
+      if ((char)tolower((unsigned char)value[start + i]) != chunked[i]) break;
+    }
+    if (i == chunked_len) return 1;
+  }
+  return 0;
+}
+
 static const int raptor_parser_start = 1;
 static const int raptor_parser_first_final = 46;
 static const int raptor_parser_error = 0;
@@ -392,7 +407,7 @@ tr26:
         parser->flags |= FLAG_HAS_BODY;
     } else if (needs_http_prefix && parser->field_len == 17 &&
                memcmp(field_ptr, "TRANSFER_ENCODING", 17) == 0) {
-      if (strstr(RSTRING_PTR(value), "chunked")) {
+      if (contains_chunked(value_ptr, value_real_len)) {
         parser->flags |= FLAG_CHUNKED | FLAG_HAS_BODY;
         parser->content_len = 0;
       }
@@ -452,7 +467,7 @@ tr29:
         parser->flags |= FLAG_HAS_BODY;
     } else if (needs_http_prefix && parser->field_len == 17 &&
                memcmp(field_ptr, "TRANSFER_ENCODING", 17) == 0) {
-      if (strstr(RSTRING_PTR(value), "chunked")) {
+      if (contains_chunked(value_ptr, value_real_len)) {
         parser->flags |= FLAG_CHUNKED | FLAG_HAS_BODY;
         parser->content_len = 0;
       }
@@ -1160,7 +1175,10 @@ case 45:
 
 done:
   parser->cs = cs;
-  parser->nread = p - buffer;
+  parser->nread = (parser->flags & FLAG_FINISHED) ? parser->body_start : (size_t)(p - buffer);
+
+  if (parser->nread > MAX_HEADER_LENGTH)
+    rb_raise(eHttpParserError, "request header too long");
 
   assert(p <= pe && "buffer overflow after parsing execute");
   assert(parser->nread <= len && "nread longer than length");
@@ -1267,6 +1285,12 @@ static VALUE parser_has_body_p(VALUE self) {
   return raptor_parser_has_body(parser) ? Qtrue : Qfalse;
 }
 
+static VALUE parser_chunked_p(VALUE self) {
+  raptor_parser *parser;
+  TypedData_Get_Struct(self, raptor_parser, &parser_type, parser);
+  return raptor_parser_is_chunked(parser) ? Qtrue : Qfalse;
+}
+
 static VALUE parser_content_length(VALUE self) {
   raptor_parser *parser;
   TypedData_Get_Struct(self, raptor_parser, &parser_type, parser);
@@ -1322,6 +1346,7 @@ RUBY_FUNC_EXPORTED void Init_raptor_http(void) {
   rb_define_method(cHttpParser, "execute", parser_execute, 3);
   rb_define_method(cHttpParser, "finished?", parser_finished_p, 0);
   rb_define_method(cHttpParser, "has_body?", parser_has_body_p, 0);
+  rb_define_method(cHttpParser, "chunked?", parser_chunked_p, 0);
   rb_define_method(cHttpParser, "content_length", parser_content_length, 0);
   rb_define_method(cHttpParser, "nread", parser_nread, 0);
   rb_define_method(cHttpParser, "reset", parser_reset, 0);

data/lib/raptor/cli.rb

@@ -187,9 +187,9 @@ module Raptor
     #
     # @rbs (Array[String] argv) -> String?
     def extract_config_path(argv)
-      argv.each_with_index do |arg, i|
+      argv.each_with_index do |arg, index|
         case arg
-        when "-c", "--config" then return argv[i + 1]
+        when "-c", "--config" then return argv[index + 1]
         when /\A--config=(.*)\z/, /\A-c(.+)\z/ then return Regexp.last_match(1)
         end
       end

data/lib/raptor/http.rb

@@ -18,6 +18,7 @@ module Raptor
     CONTENT_TYPE = "CONTENT_TYPE"
     HTTP_VERSION = "HTTP_VERSION"
     REMOTE_ADDR = "REMOTE_ADDR"
+    REQUEST_URI = "REQUEST_URI"
     SERVER_SOFTWARE = "SERVER_SOFTWARE"
     SERVER_SOFTWARE_VALUE = "Raptor/#{Raptor::VERSION}".freeze
 

data/lib/raptor/http1.rb

@@ -58,6 +58,22 @@ module Raptor
 
     ILLEGAL_HEADER_KEY_REGEX = /[\x00-\x20\(\)<>@,;:\\"\/\[\]\?=\{\}\x7F]/
     ILLEGAL_HEADER_VALUE_REGEX = /[\x00-\x08\x0A-\x1F]/
+    CHUNK_SIZE_REGEX = /\A[0-9A-Fa-f]+\z/
+
+    # Returns true when an HTTP/1.1 request lacks a valid `Host` header per
+    # RFC 9112 section 3.2, where a valid value is a non-empty single-value
+    # line.
+    #
+    # @param env [Hash] the Rack environment after header parsing
+    # @return [Boolean]
+    #
+    # @rbs (Hash[String, untyped] env) -> bool
+    def self.invalid_host?(env)
+      return false unless env[Rack::SERVER_PROTOCOL] == HTTP_11
+
+      http_host = env[Rack::HTTP_HOST]
+      !http_host || http_host.empty? || http_host.include?(",")
+    end
 
     # Returns true when the message framing shows a request-smuggling vector
     # per RFC 9112 section 6.3: a `Transfer-Encoding` where `chunked` is
@@ -89,9 +105,10 @@ module Raptor
     # Decodes a chunked transfer-encoded body buffer.
     #
     # Returns the decoded bytes and a state symbol: `:complete` when the
-    # terminating zero-length chunk was found, `:too_large` when the decoded
-    # size would exceed `max_size`, `:malformed` when chunk framing overhead
-    # exceeds `MAX_CHUNK_OVERHEAD`, or `:incomplete` otherwise.
+    # terminating zero-length chunk and trailer section were fully consumed,
+    # `:too_large` when the decoded size would exceed `max_size`, `:malformed`
+    # when a chunk-size line is not valid hex or chunk framing overhead exceeds
+    # `MAX_CHUNK_OVERHEAD`, or `:incomplete` otherwise.
     #
     # @param buffer [String] the raw body buffer to decode
     # @param max_size [Integer, nil] maximum decoded body size, or nil for unlimited
@@ -107,8 +124,24 @@ module Raptor
         crlf = buffer.index("\r\n", offset)
         return [decoded, :incomplete] unless crlf
 
-        chunk_size = buffer.byteslice(offset, crlf - offset).to_i(16)
-        return [decoded, :complete] if chunk_size == 0
+        size_line = buffer.byteslice(offset, crlf - offset)
+        semicolon = size_line.index(";")
+        size_part = semicolon ? size_line.byteslice(0, semicolon) : size_line
+        return [decoded, :malformed] unless size_part.match?(CHUNK_SIZE_REGEX)
+
+        chunk_size = size_part.to_i(16)
+
+        if chunk_size == 0
+          trailer_offset = crlf + 2
+          loop do
+            trailer_crlf = buffer.index("\r\n", trailer_offset)
+            return [decoded, :incomplete] unless trailer_crlf
+            return [decoded, :complete] if trailer_crlf == trailer_offset
+
+            trailer_offset = trailer_crlf + 2
+          end
+        end
+
         return [decoded, :too_large] if max_size && (decoded.bytesize + chunk_size) > max_size
 
         overhead += (crlf - offset) + 4
@@ -240,7 +273,8 @@ module Raptor
         buffer << socket.read_nonblock(socket.pending)
       end
 
-      parser = HttpParser.new
+      parser = (Thread.current[:raptor_http_parser] ||= HttpParser.new)
+      parser.reset
       env = @env_template.dup
       nread = begin
         parser.execute(env, buffer, 0)
@@ -254,7 +288,7 @@ module Raptor
       if !parser.finished?
         fallback_to_reactor(socket, id, buffer, env, parse_data, reactor, 0, remote_addr, url_scheme, persisted: false)
         return
-      elsif Http1.request_smuggling?(env)
+      elsif Http1.invalid_host?(env) || Http1.request_smuggling?(env)
         reject_malformed(socket)
         return
       elsif parser.has_body?
@@ -265,7 +299,7 @@ module Raptor
 
         body = buffer.byteslice(nread..-1) || ""
 
-        if env[HTTP_TRANSFER_ENCODING]&.include?(TRANSFER_ENCODING_CHUNKED)
+        if parser.chunked?
           body, chunked_state = Http1.decode_chunked(body, @max_body_size)
           case chunked_state
           when :complete
@@ -322,14 +356,14 @@ module Raptor
         parse_data[:parse_count] += 1
 
         message = if parser.finished?
-          if Raptor::Http1.request_smuggling?(env)
+          if Raptor::Http1.invalid_host?(env) || Raptor::Http1.request_smuggling?(env)
             data.merge(env: env, body: nil, parse_data: parse_data, complete: true, malformed: true)
           elsif parser.has_body?
             body_buffer = data[:buffer].byteslice(nread..-1) || ""
 
             if max_body_size && parser.content_length > max_body_size
               data.merge(env: env, body: nil, parse_data: parse_data, complete: true, too_large: true)
-            elsif env[HTTP_TRANSFER_ENCODING]&.include?(TRANSFER_ENCODING_CHUNKED)
+            elsif parser.chunked?
               decoded_body, chunked_state = Raptor::Http1.decode_chunked(body_buffer, max_body_size)
 
               case chunked_state
@@ -571,7 +605,8 @@ module Raptor
           buffer << socket.read_nonblock(socket.pending)
         end
 
-        parser = HttpParser.new
+        parser = (Thread.current[:raptor_http_parser] ||= HttpParser.new)
+        parser.reset
         env = @env_template.dup
         nread = begin
           parser.execute(env, buffer, 0)
@@ -588,8 +623,7 @@ module Raptor
         elsif parser.has_body?
           body = buffer.byteslice(nread..-1) || ""
 
-          chunked = env[HTTP_TRANSFER_ENCODING]&.include?(TRANSFER_ENCODING_CHUNKED)
-          if chunked || parser.content_length > body.bytesize
+          if parser.chunked? || parser.content_length > body.bytesize
             fallback_to_reactor(socket, id, buffer, env, parse_data, reactor, request_count, remote_addr, url_scheme)
             return
           end
@@ -597,7 +631,7 @@ module Raptor
 
         request_count += 1
 
-        if thread_pool.queue_size > 0
+        if thread_pool.queue_size >= thread_pool.size
           thread_pool << proc do
             process_client(
               socket,
@@ -718,8 +752,23 @@ module Raptor
         send_early_hints(socket, hints) rescue nil
       end
 
-      env[Rack::PATH_INFO] = env.delete(Rack::REQUEST_PATH) if env.key?(Rack::REQUEST_PATH)
-      env[Rack::PATH_INFO] = "" unless env.key?(Rack::PATH_INFO)
+      unless env.key?(Rack::PATH_INFO)
+        request_uri = env[Http::REQUEST_URI]
+        scheme_end = request_uri&.index("://")
+        if scheme_end
+          authority_end = request_uri.index("/", scheme_end + 3) || request_uri.bytesize
+          path_and_query = request_uri.byteslice(authority_end..-1) || ""
+          if query_delim = path_and_query.index("?")
+            env[Rack::PATH_INFO] = query_delim.zero? ? "/" : path_and_query.byteslice(0, query_delim)
+            env[Rack::QUERY_STRING] = path_and_query.byteslice(query_delim + 1..-1)
+          else
+            env[Rack::PATH_INFO] = path_and_query.empty? ? "/" : path_and_query
+          end
+        else
+          env[Rack::PATH_INFO] = ""
+        end
+      end
+
       if (content_length = parse_data[:content_length]).positive?
         env[Http::CONTENT_LENGTH] = content_length.to_s
       end
@@ -976,7 +1025,7 @@ module Raptor
     #
     # @rbs (TCPSocket socket, String response, Hash[String, String | Array[String]] headers, ^(TCPSocket) -> void response_hijack) -> void
     def write_hijacked_response(socket, response, headers, response_hijack)
-      response << format_headers(headers)
+      format_headers(response, headers)
       response << "\r\n"
       socket_write(socket, response)
       uncork_socket(socket)
@@ -1001,7 +1050,7 @@ module Raptor
         headers[Rack::CONTENT_LENGTH] = "0" unless headers.key?(Rack::CONTENT_LENGTH) || headers.key?(Rack::TRANSFER_ENCODING)
       end
 
-      response << format_headers(headers)
+      format_headers(response, headers)
       response << "\r\n"
       socket_write(socket, response)
     end
@@ -1023,7 +1072,7 @@ module Raptor
     # @rbs (TCPSocket socket, String response, Hash[String, String | Array[String]] headers, untyped body, String http_version) -> void
     def write_full_response(socket, response, headers, body, http_version)
       if body.respond_to?(:call)
-        response << format_headers(headers)
+        format_headers(response, headers)
         response << "\r\n"
         socket_write(socket, response)
         uncork_socket(socket)
@@ -1049,7 +1098,7 @@ module Raptor
         headers[Rack::TRANSFER_ENCODING] = TRANSFER_ENCODING_CHUNKED
       end
 
-      response << format_headers(headers)
+      format_headers(response, headers)
       response << "\r\n"
 
       if body.respond_to?(:to_path) && (path = body.to_path) && File.readable?(path)
@@ -1253,9 +1302,8 @@ module Raptor
     # @param headers [Hash] normalized response headers
     # @return [String] formatted header lines, each ending with CRLF
     #
-    # @rbs (Hash[String, String | Array[String]] headers) -> String
-    def format_headers(headers)
-      result = +""
+    # @rbs (String result, Hash[String, String | Array[String]] headers) -> void
+    def format_headers(result, headers)
       headers.each do |name, value|
         next if illegal_header_key?(name)
 
@@ -1265,7 +1313,6 @@ module Raptor
           append_header_value(result, name, value)
         end
       end
-      result
     end
 
     # Appends one or more `name: value` header lines to `result`. Newline-

data/lib/raptor/http2.rb

@@ -247,6 +247,39 @@ module Raptor
     RACK_HEADER_PREFIX = "rack."
     HOP_BY_HOP_HEADERS = ["connection", "transfer-encoding", "keep-alive", "upgrade", "proxy-connection"].freeze
 
+    REQUEST_PSEUDO_HEADERS = [":method", ":scheme", ":path", ":authority"].freeze
+    REQUIRED_REQUEST_PSEUDO_HEADERS = [":method", ":scheme", ":path"].freeze
+
+    # Returns true when a decoded header block violates the HTTP/2 pseudo-header
+    # rules from RFC 9113 section 8.3: an unknown pseudo-header, a duplicate
+    # pseudo-header, a pseudo-header appearing after any regular header, or a
+    # required pseudo-header (`:method`, `:scheme`, `:path`) missing on
+    # non-`CONNECT` requests.
+    #
+    # @param headers [Array<Array(String, String)>] decoded header pairs
+    # @return [Boolean]
+    #
+    # @rbs (Array[[String, String]] headers) -> bool
+    def self.invalid_pseudo_headers?(headers)
+      seen_pseudo = {}
+      seen_regular = false
+
+      headers.each do |name, _value|
+        if name.start_with?(":")
+          return true if seen_regular
+          return true unless REQUEST_PSEUDO_HEADERS.include?(name)
+          return true if seen_pseudo[name]
+          seen_pseudo[name] = true
+        else
+          seen_regular = true
+        end
+      end
+
+      return false if seen_pseudo[":method"] && headers.assoc(":method")&.last == "CONNECT"
+
+      REQUIRED_REQUEST_PSEUDO_HEADERS.any? { |name| !seen_pseudo[name] }
+    end
+
     # @rbs @app: ^(Hash[String, untyped]) -> [Integer, Hash[String, String | Array[String]], untyped]
     # @rbs @server_port: Integer
     # @rbs @write_timeout: Integer
@@ -370,7 +403,12 @@ module Raptor
 
           if (frame[:flags] & FLAG_END_HEADERS) != 0
             decoded_headers, hpack_table = parser.parse_headers(header_payload, hpack_table)
-            streams, completed_requests = finalize_headers(streams, completed_requests, stream_id, decoded_headers, end_stream)
+            if invalid_pseudo_headers?(decoded_headers)
+              streams.delete(stream_id)
+              outgoing_frames << parser.build_frame(:rst_stream, 0, stream_id, [ERROR_PROTOCOL_ERROR].pack("N"))
+            else
+              streams, completed_requests = finalize_headers(streams, completed_requests, stream_id, decoded_headers, end_stream)
+            end
           else
             pending_headers = { stream_id: stream_id, buffer: header_payload, end_stream: end_stream }
           end
@@ -386,7 +424,12 @@ module Raptor
           if (frame[:flags] & FLAG_END_HEADERS) != 0
             stream_id = pending_headers[:stream_id]
             decoded_headers, hpack_table = parser.parse_headers(pending_headers[:buffer], hpack_table)
-            streams, completed_requests = finalize_headers(streams, completed_requests, stream_id, decoded_headers, pending_headers[:end_stream])
+            if invalid_pseudo_headers?(decoded_headers)
+              streams.delete(stream_id)
+              outgoing_frames << parser.build_frame(:rst_stream, 0, stream_id, [ERROR_PROTOCOL_ERROR].pack("N"))
+            else
+              streams, completed_requests = finalize_headers(streams, completed_requests, stream_id, decoded_headers, pending_headers[:end_stream])
+            end
             pending_headers = nil
           end
 
@@ -688,7 +731,7 @@ module Raptor
         next if HOP_BY_HOP_HEADERS.include?(lowered)
 
         if value.is_a?(Array)
-          value.each { |val| header_pairs << [lowered, val.to_s] }
+          value.each { |entry| header_pairs << [lowered, entry.to_s] }
         else
           header_pairs << [lowered, value.to_s]
         end

data/lib/raptor/version.rb

@@ -2,5 +2,5 @@
 # frozen_string_literal: true
 
 module Raptor
-  VERSION = "0.10.0"
+  VERSION = "0.11.0"
 end

data/sig/generated/raptor/http.rbs

@@ -15,6 +15,8 @@ module Raptor
 
     REMOTE_ADDR: ::String
 
+    REQUEST_URI: ::String
+
     SERVER_SOFTWARE: ::String
 
     SERVER_SOFTWARE_VALUE: untyped

data/sig/generated/raptor/http1.rbs

@@ -64,6 +64,18 @@ module Raptor
 
     ILLEGAL_HEADER_VALUE_REGEX: ::Regexp
 
+    CHUNK_SIZE_REGEX: ::Regexp
+
+    # Returns true when an HTTP/1.1 request lacks a valid `Host` header per
+    # RFC 9112 section 3.2, where a valid value is a non-empty single-value
+    # line.
+    #
+    # @param env [Hash] the Rack environment after header parsing
+    # @return [Boolean]
+    #
+    # @rbs (Hash[String, untyped] env) -> bool
+    def self.invalid_host?: (Hash[String, untyped] env) -> bool
+
     # Returns true when the message framing shows a request-smuggling vector
     # per RFC 9112 section 6.3: a `Transfer-Encoding` where `chunked` is
     # missing, not the final encoding, or duplicated; a `Transfer-Encoding`
@@ -79,9 +91,10 @@ module Raptor
     # Decodes a chunked transfer-encoded body buffer.
     #
     # Returns the decoded bytes and a state symbol: `:complete` when the
-    # terminating zero-length chunk was found, `:too_large` when the decoded
-    # size would exceed `max_size`, `:malformed` when chunk framing overhead
-    # exceeds `MAX_CHUNK_OVERHEAD`, or `:incomplete` otherwise.
+    # terminating zero-length chunk and trailer section were fully consumed,
+    # `:too_large` when the decoded size would exceed `max_size`, `:malformed`
+    # when a chunk-size line is not valid hex or chunk framing overhead exceeds
+    # `MAX_CHUNK_OVERHEAD`, or `:incomplete` otherwise.
     #
     # @param buffer [String] the raw body buffer to decode
     # @param max_size [Integer, nil] maximum decoded body size, or nil for unlimited
@@ -579,8 +592,8 @@ module Raptor
     # @param headers [Hash] normalized response headers
     # @return [String] formatted header lines, each ending with CRLF
     #
-    # @rbs (Hash[String, String | Array[String]] headers) -> String
-    def format_headers: (Hash[String, String | Array[String]] headers) -> String
+    # @rbs (String result, Hash[String, String | Array[String]] headers) -> void
+    def format_headers: (String result, Hash[String, String | Array[String]] headers) -> void
 
     # Appends one or more `name: value` header lines to `result`. Newline-
     # separated values are emitted as separate lines; empty values and values

data/sig/generated/raptor/http2.rbs

@@ -145,6 +145,22 @@ module Raptor
 
     HOP_BY_HOP_HEADERS: untyped
 
+    REQUEST_PSEUDO_HEADERS: untyped
+
+    REQUIRED_REQUEST_PSEUDO_HEADERS: untyped
+
+    # Returns true when a decoded header block violates the HTTP/2 pseudo-header
+    # rules from RFC 9113 section 8.3: an unknown pseudo-header, a duplicate
+    # pseudo-header, a pseudo-header appearing after any regular header, or a
+    # required pseudo-header (`:method`, `:scheme`, `:path`) missing on
+    # non-`CONNECT` requests.
+    #
+    # @param headers [Array<Array(String, String)>] decoded header pairs
+    # @return [Boolean]
+    #
+    # @rbs (Array[[String, String]] headers) -> bool
+    def self.invalid_pseudo_headers?: (Array[[ String, String ]] headers) -> bool
+
     @initial_settings_frame: String
 
     @on_error: ^(Hash[String, untyped]?, Exception) -> void | nil

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: raptor
 version: !ruby/object:Gem::Version
-  version: 0.10.0
+  version: 0.11.0
 platform: ruby
 authors:
 - Joshua Young