h1p 0.2 → 0.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 58f7bb5105298c97edf87c91b8038ccc50ac6732417b981331a39cf56711da9b
-  data.tar.gz: d5745229262d866ed3564eede8b52706b14cb791652886be9f0a3ec2bc44eaf4
+  metadata.gz: 53aadd6f6c4ae112ff844b68e8325fe334d51dd0a09bb11cc54d017190e97677
+  data.tar.gz: e9f6d504813d74c050a46b4e973e9fbc227dd790c02395a0073b62f43a7393da
 SHA512:
-  metadata.gz: d64c4623aa5c1cffd019222f6b75e3ec07da72997a5118490e07437ee0b0449371da96d89f4755182703fc71bc0974f415e6c9eecffa1f173c3bcb59f894d376
-  data.tar.gz: 9df79da840b37ecacb1df7818a661a33560970b5c07e4b82250e29466b868c6087848b527a8123a1373ead55ec1a37a4626e1a702c04037a4515e66ac6220476
+  metadata.gz: 3ac98c2d7e702f8cf5f9052e78e9abe486ee3ba0814c4c16fe222acdccfb5338ac9d5c9b4a5d1da2178fca65472693c94a4997cc86de05db70f13c90f1bc767e
+  data.tar.gz: 73ce78e3435eb40f3e6d46ee8eef803284e8da99f32d610dbd723c3661548a74ed454ae70e1ec02dcfbdb70aa346059fb7713eab19de3e6e6e65a4ca859926c7

data/CHANGELOG.md

@@ -1,4 +1,15 @@
-## 0.42 2021-08-16
+## 0.5 2022-03-19
+
+- Implement `Parser#splice_body_to` (#3)
+
+## 0.4 2022-02-28
+
+- Rename `__parser_read_method__` to `__read_method__`
+
+## 0.3 2022-02-03
+
+- Add support for parsing HTTP responses (#1)
+- Put state directly in parser struct
 
 ## 0.2 2021-08-20
 

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    h1p (0.2)
+    h1p (0.5)
 
 GEM
   remote: https://rubygems.org/

data/README.md

@@ -1,7 +1,7 @@
 # H1P - a blocking HTTP/1 parser for Ruby
 
 [![Gem Version](https://badge.fury.io/rb/h1p.svg)](http://rubygems.org/gems/h1p)
-[![Modulation Test](https://github.com/digital-fabric/h1p/workflows/Tests/badge.svg)](https://github.com/digital-fabric/h1p/actions?query=workflow%3ATests)
+[![H1P Test](https://github.com/digital-fabric/h1p/workflows/Tests/badge.svg)](https://github.com/digital-fabric/h1p/actions?query=workflow%3ATests)
 [![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/digital-fabric/h1p/blob/master/LICENSE)
 
 H1P is a blocking/synchronous HTTP/1 parser for Ruby with a simple and intuitive
@@ -23,8 +23,11 @@ The H1P was originally written as part of
 - Simple, blocking/synchronous API
 - Zero dependencies
 - Transport-agnostic
+- Parses both HTTP request and HTTP response
 - Support for chunked encoding
 - Support for both `LF` and `CRLF` line breaks
+- Support for **splicing** request/response bodies (when used with
+  [Polyphony](https://github.com/digital-fabric/polyphony))
 - Track total incoming traffic
 
 ## Installing
@@ -41,15 +44,21 @@ You can then run `bundle install` to install it. Otherwise, just run `gem instal
 
 ## Usage
 
-Start by creating an instance of H1P::Parser, passing a connection instance:
+Start by creating an instance of `H1P::Parser`, passing a connection instance and the parsing mode:
 
 ```ruby
 require 'h1p'
 
-parser = H1P::Parser.new(conn)
+parser = H1P::Parser.new(conn, :server)
 ```
 
-To read the next request from the connection, call `#parse_headers`:
+In order to parse HTTP responses, change the mode to `:client`:
+
+```ruby
+parser = H1P::Parser.new(conn, :client)
+```
+
+To read the next message from the connection, call `#parse_headers`:
 
 ```ruby
 loop do
@@ -65,13 +74,21 @@ headers. In case the client has closed the connection, `#parse_headers` will
 return `nil` (see the guard clause above).
 
 In addition to the header keys and values, the resulting hash also contains the
-following "pseudo-headers":
+following "pseudo-headers" (in server mode):
 
 - `:method`: the HTTP method (in upper case)
 - `:path`: the request target
 - `:protocol`: the protocol used (either `'http/1.0'` or `'http/1.1'`)
 - `:rx`: the total bytes read by the parser
 
+In client mode, the following pseudo-headers will be present:
+
+- `:protocol`: the protocol used (either `'http/1.0'` or `'http/1.1'`)
+- `:status': the HTTP status as an integer
+- `:status_message`: the HTTP status message
+- `:rx`: the total bytes read by the parser
+
+
 The header keys are always lower-cased. Consider the following HTTP request:
 
 ```
@@ -101,24 +118,24 @@ where the value is an array containing the corresponding values. For example,
 multiple `Cookie` headers will appear in the hash as a single `"cookie"` entry,
 e.g. `{ "cookie" => ['a=1', 'b=2'] }`
 
-### Handling of invalid requests 
+### Handling of invalid message
 
-When an invalid request is encountered, the parser will raise a `H1P::Error`
-exception. An incoming request may be considered invalid if an invalid character
-has been encountered at any point in parsing the request, or if any of the
+When an invalid message is encountered, the parser will raise a `H1P::Error`
+exception. An incoming message may be considered invalid if an invalid character
+has been encountered at any point in parsing the message, or if any of the
 tokens have an invalid length. You can consult the limits used by the parser
 [here](https://github.com/digital-fabric/h1p/blob/main/ext/h1p/limits.rb).
 
-### Reading the request body
+### Reading the message body
 
-To read the request body use `#read_body`:
+To read the message body use `#read_body`:
 
 ```ruby
 # read entire body
 body = parser.read_body
 ```
 
-The H1P parser knows how to read both request bodies with a specified
+The H1P parser knows how to read both message bodies with a specified
 `Content-Length` and request bodies in chunked encoding. The method call will
 return when the entire body has been read. If the body is incomplete or has
 invalid formatting, the parser will raise a `H1P::Error` exception.
@@ -146,12 +163,42 @@ end
 The `#read_body` and `#read_body_chunk` methods will return `nil` if no body is
 expected (based on the received headers).
 
+## Splicing request/response bodies
+
+> Splicing of request/response bodies is available only on Linux, and works only
+> with [Polyphony](https://github.com/digital-fabric/polyphony).
+
+H1P also lets you [splice](https://man7.org/linux/man-pages/man2/splice.2.html)
+request or response bodies directly to a pipe. This is particularly useful for
+uploading or downloading large files, as the data does not need to be loaded
+into Ruby strings. In fact, the data will stay almost entirely in kernel
+buffers, which means any data copying is reduced to the absolute minimum.
+
+The following example sends a request, then splices the response body to a file:
+
+```ruby
+require 'polyphony'
+require 'h1p'
+
+socket = TCPSocket.new('example.com', 80)
+socket << "GET /bigfile HTTP/1.1\r\nHost: example.com\r\n\r\n"
+
+parser = H1P::Parser.new(socket, :client)
+headers = parser.parse_headers
+
+pipe = Polyphony.pipe
+File.open('bigfile', 'w+') do |f|
+  spin { parser.splice_body_to(pipe) }
+  f.splice_from(pipe)
+end
+```
+
 ## Parsing from arbitrary transports
 
 The H1P parser was built to read from any arbitrary transport or source, as long
 as they conform to one of two alternative interfaces:
 
-- An object implementing a `__parser_read_method__` method, which returns any of
+- An object implementing a `__read_method__` method, which returns any of
   the following values:
 
   - `:stock_readpartial` - to be used for instances of `IO`, `Socket`,
@@ -211,8 +258,7 @@ performance.
 Here are some of the features and enhancements planned for H1P:
 
 - Add conformance and security tests
-- Add ability to parse HTTP/1 responses (for implementing HTTP clients)
-- Add ability to splice the request body into an arbitrary fd
+- Add ability to splice the message body into an arbitrary fd
   (Polyphony-specific)
 - Improve performance
 

data/Rakefile

@@ -12,5 +12,5 @@ task :recompile => [:clean, :compile]
 task :default => [:compile, :test]
 
 task :test do
-  exec 'ruby test/test_h1p.rb'
+  exec 'ruby test/run.rb'
 end

data/benchmarks/bm_http1_parser.rb

@@ -36,7 +36,7 @@ end
 def benchmark_other_http1_parser(iterations)
   STDOUT << "http_parser.rb: "
   require 'http_parser.rb'
-  
+
   i, o = IO.pipe
   parser = Http::Parser.new
   done = false

data/benchmarks/pipelined.rb

@@ -0,0 +1,101 @@
+# frozen_string_literal: true
+
+HTTP_REQUEST = "GET /foo HTTP/1.1\r\nHost: example.com\r\nAccept: */*\r\nUser-Agent: foobar\r\n\r\n" +
+               "GET /bar HTTP/1.1\r\nHost: example.com\r\nAccept: */*\r\nUser-Agent: foobar\r\n\r\n"
+
+def measure_time_and_allocs
+  4.times { GC.start }
+  GC.disable
+
+  t0 = Time.now
+  a0 = object_count
+  yield
+  t1 = Time.now
+  a1 = object_count
+  [t1 - t0, a1 - a0]
+ensure
+  GC.enable
+end
+
+def object_count
+  count = ObjectSpace.count_objects
+  count[:TOTAL] - count[:FREE]
+end
+
+def benchmark_other_http1_parser(iterations)
+  STDOUT << "http_parser.rb: "
+  require 'http_parser.rb'
+
+  i, o = IO.pipe
+  parser = Http::Parser.new
+  done = false
+  queue = nil
+  rx = 0
+  req_count = 0
+  parser.on_headers_complete = proc do |h|
+    h[':method'] = parser.http_method
+    h[':path'] = parser.request_url
+    h[':rx'] = rx
+    queue << h
+  end
+  parser.on_message_complete = proc { done = true }
+
+  writer = Thread.new do
+    iterations.times { o << HTTP_REQUEST }
+    o.close
+  end
+
+  elapsed, allocated = measure_time_and_allocs do
+    queue = []
+    done = false
+    rx = 0
+    loop do
+      data = i.readpartial(4096) rescue nil
+      break unless data
+
+      rx += data.bytesize
+      parser << data
+      while (req = queue.shift)
+        req_count += 1
+      end
+    end
+  end
+  puts(format('count: %d, elapsed: %f, allocated: %d (%f/req), rate: %f ips', req_count, elapsed, allocated, allocated.to_f / iterations, iterations / elapsed))
+end
+
+def benchmark_h1p_parser(iterations)
+  STDOUT << "H1P parser: "
+  require_relative '../lib/h1p'
+  i, o = IO.pipe
+  parser = H1P::Parser.new(i)
+  req_count = 0
+
+  writer = Thread.new do
+    iterations.times { o << HTTP_REQUEST }
+    o.close
+  end
+
+  elapsed, allocated = measure_time_and_allocs do
+    while (headers = parser.parse_headers)
+      req_count += 1
+    end
+  end
+  puts(format('count: %d, elapsed: %f, allocated: %d (%f/req), rate: %f ips', req_count, elapsed, allocated, allocated.to_f / iterations, iterations / elapsed))
+end
+
+def fork_benchmark(method, iterations)
+  pid = fork do
+    send(method, iterations)
+  rescue Exception => e
+    p e
+    p e.backtrace
+    exit!
+  end
+  Process.wait(pid)
+end
+
+x = 100000
+fork_benchmark(:benchmark_other_http1_parser, x)
+fork_benchmark(:benchmark_h1p_parser, x)
+
+# benchmark_h1p_parser(x)

data/examples/callable.rb

@@ -4,7 +4,7 @@ require 'bundler/setup'
 require 'h1p'
 
 data = ['GET ', '/foo', " HTTP/1.1\r\n", "\r\n"]
-parser = H1P::Parser.new(proc { data.shift })
+parser = H1P::Parser.new(proc { data.shift }, :server)
 
 headers = parser.parse_headers
 p headers

data/examples/http_server.rb

@@ -10,13 +10,13 @@ trap('SIGINT') { exit! }
 
 def handle_client(conn)
   Thread.new do
-    parser = H1P::Parser.new(conn)
+    parser = H1P::Parser.new(conn, :server)
     loop do
       headers = parser.parse_headers
       break unless headers
 
       req_body = parser.read_body
-      
+
       p headers: headers
       p body: req_body