puma-http1 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +12 -0
- data/LICENSE.md +32 -0
- data/README.md +103 -0
- data/Rakefile +15 -0
- data/lib/puma/http1/parser.rb +565 -0
- data/lib/puma/http1/version.rb +7 -0
- data/lib/puma/http1.rb +18 -0
- data/sig/puma/http1.rbs +6 -0
- metadata +68 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 833fbce5f9dc90feed1f860e3fa34424fdcd2ba444e5663c3f575f096164c278
|
|
4
|
+
data.tar.gz: 241a9909f7898a5ddbde2706833d85544ce65de1a0c4af92940720507f7d3b60
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: d167920f38c3c0c156e5cd82450f19ef2c040e87582cb34e6561e548eb8577e9313a32805cd52629c0d1c6be10212b0958a8f267e130d70fc97928e2278478ce
|
|
7
|
+
data.tar.gz: 53a5d316e0b4e49e68f2907bcdf088c84d1d9c21bd76b988c8f8b3833217901a60ccc20caed03681b05bfe2b0af76bcfdc82877f82b57e9f89c5c2a9f752496d
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
## [Unreleased]
|
|
2
|
+
|
|
3
|
+
## [0.0.1] - 2026-09-26
|
|
4
|
+
|
|
5
|
+
The first release, an early version of a reference implementation for Puma's proposed `http_parser` option.
|
|
6
|
+
|
|
7
|
+
- Add `Puma::HTTP1::Parser`, a Ruby port of the HTTP parser in Puma's `puma_http11` C extension, for Puma's proposed `http_parser` option
|
|
8
|
+
- Add an example app that runs Puma with `Puma::HTTP1::Parser`, with and without `puma_http11`
|
|
9
|
+
- Add `script/benchmark`, comparing parsing time with `Puma::HttpParser`
|
|
10
|
+
- Add `script/benchmark-server`, comparing requests per second with ApacheBench, and the results of a run on Ruby 4.0.7
|
|
11
|
+
- Add `script/test-puma-suite`, running Puma's own tests with `Puma::HTTP1::Parser`, and run it in CI
|
|
12
|
+
- Write the tests in minitest, like Puma's own test suite
|
data/LICENSE.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# BSD 3-Clause License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026, Shane Becker.
|
|
4
|
+
|
|
5
|
+
Ported from Puma, Copyright (c) 2019, Evan Phoenix. Some code by Zed Shaw, (c) 2005.
|
|
6
|
+
|
|
7
|
+
All rights reserved.
|
|
8
|
+
|
|
9
|
+
Redistribution and use in source and binary forms, with or without
|
|
10
|
+
modification, are permitted provided that the following conditions are met:
|
|
11
|
+
|
|
12
|
+
1. Redistributions of source code must retain the above copyright notice, this
|
|
13
|
+
list of conditions and the following disclaimer.
|
|
14
|
+
|
|
15
|
+
2. Redistributions in binary form must reproduce the above copyright notice,
|
|
16
|
+
this list of conditions and the following disclaimer in the documentation
|
|
17
|
+
and/or other materials provided with the distribution.
|
|
18
|
+
|
|
19
|
+
3. Neither the name of the copyright holder nor the names of its
|
|
20
|
+
contributors may be used to endorse or promote products derived from
|
|
21
|
+
this software without specific prior written permission.
|
|
22
|
+
|
|
23
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
|
24
|
+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
25
|
+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
26
|
+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
|
27
|
+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
28
|
+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
|
29
|
+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
|
30
|
+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
|
31
|
+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|
32
|
+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
data/README.md
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# puma-http1
|
|
2
|
+
|
|
3
|
+
An HTTP/1.x parser for [Puma](https://github.com/puma/puma), written in Ruby.
|
|
4
|
+
|
|
5
|
+
`Puma::HTTP1::Parser` is a drop-in for `Puma::HttpParser`, the parser in Puma’s `puma_http11` C extension. It follows the same Ragel grammar state for state, fills the same env keys, enforces the same length limits, and raises the same errors with the same messages. The tests run every request through both parsers and expect the same result.
|
|
6
|
+
|
|
7
|
+
[`script/test-puma-suite`](script/test-puma-suite) also runs Puma’s own tests with `Puma::HTTP1::Parser` in place of `Puma::HttpParser`. The only failures are two tests that check the header names in Puma’s “Bad headers” error message, explained [below](#differences-from-pumahttpparser). It skips Puma’s integration tests, which start Puma as a separate process and so always use the C parser. CI runs it on every push.
|
|
8
|
+
|
|
9
|
+
With it, Puma can run where its C extension can't be built or loaded.
|
|
10
|
+
|
|
11
|
+
## Status
|
|
12
|
+
|
|
13
|
+
This is an early reference implementation for a proposed `http_parser` option in Puma. That option isn't in a Puma release yet, so this gem needs Puma from the `pluggable-http-parser` branch of [veganstraightedge/puma](https://github.com/veganstraightedge/puma/tree/pluggable-http-parser).
|
|
14
|
+
|
|
15
|
+
## Installation
|
|
16
|
+
|
|
17
|
+
```ruby
|
|
18
|
+
# Gemfile
|
|
19
|
+
gem 'puma', github: 'veganstraightedge/puma', branch: 'pluggable-http-parser'
|
|
20
|
+
gem 'puma-http1'
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Usage
|
|
24
|
+
|
|
25
|
+
Require the gem and pass the parser class to Puma’s `http_parser` option.
|
|
26
|
+
|
|
27
|
+
```ruby
|
|
28
|
+
# config/puma.rb
|
|
29
|
+
require 'puma/http1'
|
|
30
|
+
|
|
31
|
+
http_parser Puma::HTTP1::Parser
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Puma’s [HTTP parser documentation](https://github.com/veganstraightedge/puma/blob/pluggable-http-parser/docs/http_parser.md) describes the interface a parser has to follow.
|
|
35
|
+
|
|
36
|
+
The gem is named `puma-http1`, and the module `Puma::HTTP1`, rather than `Puma::HTTP`, because `Puma::Const::HTTP` already exists. Puma’s code refers to it without the `Const::` prefix, so a `Puma::HTTP` module would be found first and break Puma.
|
|
37
|
+
|
|
38
|
+
### Without the C extension
|
|
39
|
+
|
|
40
|
+
When Puma’s `puma_http11` extension can't be loaded, Puma still starts, as long as `http_parser` is set. SSL isn't available then, because Puma’s SSL support is in the same extension.
|
|
41
|
+
|
|
42
|
+
The [example app](example) runs both ways:
|
|
43
|
+
|
|
44
|
+
```sh
|
|
45
|
+
$ script/example
|
|
46
|
+
=== With puma_http11 installed ===
|
|
47
|
+
HTTP parser: Puma::HTTP1::Parser
|
|
48
|
+
puma_http11: loaded
|
|
49
|
+
...
|
|
50
|
+
|
|
51
|
+
=== With puma_http11 hidden ===
|
|
52
|
+
HTTP parser: Puma::HTTP1::Parser
|
|
53
|
+
puma_http11: not loaded
|
|
54
|
+
...
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Differences from Puma::HttpParser
|
|
58
|
+
|
|
59
|
+
The C parser upcases header names inside the request buffer as it parses. This parser leaves the buffer alone. So when Puma reports a request with bad headers, the error message shows the header names as the client sent them. Puma’s Java parser, used on JRuby, also leaves the buffer alone, so this parser matches JRuby here.
|
|
60
|
+
|
|
61
|
+
## Performance
|
|
62
|
+
|
|
63
|
+
It’s slower than the C parser. On an Apple M1 with Ruby 4.0.7, from [`script/benchmark`](script/benchmark):
|
|
64
|
+
|
|
65
|
+
| request | Puma::HttpParser | Puma::HTTP1::Parser | Puma::HTTP1::Parser with YJIT |
|
|
66
|
+
| :---------------------- | ---------------: | ------------------: | ----------------------------: |
|
|
67
|
+
| minimal GET | 0.46 | 3.19 | 2.79 |
|
|
68
|
+
| browser GET, 13 headers | 2.81 | 17.18 | 16.20 |
|
|
69
|
+
| API POST, 7 headers | 1.56 | 10.53 | 9.92 |
|
|
70
|
+
|
|
71
|
+
All values are microseconds per parse. Smaller is better.
|
|
72
|
+
|
|
73
|
+
That’s roughly 2 to 14 µs more per request. A hello world app, with keep-alive and 4 threads, serves this many requests per second, from [`script/benchmark-server`](script/benchmark-server) using [ApacheBench](https://httpd.apache.org/docs/current/programs/ab.html):
|
|
74
|
+
|
|
75
|
+
| request | JIT | Puma::HttpParser | Puma::HTTP1::Parser | gap |
|
|
76
|
+
| :-------- | :--- | ---------------: | ------------------: | ---: |
|
|
77
|
+
| minimal | none | 26,815 | 23,009 | -14% |
|
|
78
|
+
| 9 headers | none | 25,145 | 18,500 | -26% |
|
|
79
|
+
| minimal | YJIT | 30,234 | 26,998 | -11% |
|
|
80
|
+
| 9 headers | YJIT | 28,270 | 22,323 | -21% |
|
|
81
|
+
|
|
82
|
+
All values are requests per second, except the gap, which is how much slower `Puma::HTTP1::Parser` is. Larger is better for requests per second. Smaller is better for the gap.
|
|
83
|
+
|
|
84
|
+
In an app doing real work per request, the difference should be a small fraction of the total.
|
|
85
|
+
|
|
86
|
+
The complete output of these runs, with every sample, the commits measured, and the machine's load, is in [benchmarks/2026-09-26-ruby-4.0.7.md](benchmarks/2026-09-26-ruby-4.0.7.md).
|
|
87
|
+
|
|
88
|
+
## Development
|
|
89
|
+
|
|
90
|
+
```sh
|
|
91
|
+
script/setup # install dependencies, for the gem and the example app
|
|
92
|
+
script/test # run the tests and RuboCop
|
|
93
|
+
script/test-puma-suite # run Puma's own tests with Puma::HTTP1::Parser
|
|
94
|
+
script/example # run the example app with and without puma_http11
|
|
95
|
+
script/server # run the example app on port 9292
|
|
96
|
+
script/benchmark # compare parsing time with Puma::HttpParser
|
|
97
|
+
script/benchmark-server # compare requests per second, with ApacheBench
|
|
98
|
+
script/console # start IRB with the gem loaded
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## License
|
|
102
|
+
|
|
103
|
+
BSD 3-Clause, the same as Puma. The parser is ported from Puma’s C extension, so the license keeps Puma’s copyright notice. See [LICENSE.md](LICENSE.md).
|
data/Rakefile
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "bundler/gem_tasks"
|
|
4
|
+
require "rake/testtask"
|
|
5
|
+
|
|
6
|
+
Rake::TestTask.new(:test) do |task|
|
|
7
|
+
task.libs << "test"
|
|
8
|
+
task.test_files = FileList["test/test_*.rb"]
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
require "rubocop/rake_task"
|
|
12
|
+
|
|
13
|
+
RuboCop::RakeTask.new
|
|
14
|
+
|
|
15
|
+
task default: %i[test rubocop]
|
|
@@ -0,0 +1,565 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Puma
|
|
4
|
+
# Puma defines this too. Defining it here, with the same superclass, lets
|
|
5
|
+
# the parser run without Puma loaded.
|
|
6
|
+
class HttpParserError < StandardError; end
|
|
7
|
+
|
|
8
|
+
module HTTP1
|
|
9
|
+
# Ruby port of the HTTP parser in Puma's `puma_http11` C extension
|
|
10
|
+
# (`ext/puma_http11` in Puma).
|
|
11
|
+
#
|
|
12
|
+
# It is a streaming state machine that mirrors the Ragel grammar in
|
|
13
|
+
# `ext/puma_http11/http11_parser_common.rl`, one state per grammar element.
|
|
14
|
+
# `execute` may be called repeatedly with a growing buffer and the byte
|
|
15
|
+
# offset returned by the previous call; the parser resumes where it left off.
|
|
16
|
+
#
|
|
17
|
+
# Behavior matches the C extension byte for byte, with one deliberate
|
|
18
|
+
# exception: the C extension upcases header names inside the caller's
|
|
19
|
+
# buffer as a side effect of parsing. This implementation never modifies
|
|
20
|
+
# the buffer.
|
|
21
|
+
class Parser
|
|
22
|
+
TAB = 9
|
|
23
|
+
LF = 10
|
|
24
|
+
CR = 13
|
|
25
|
+
SPACE = 32
|
|
26
|
+
HASH = 35
|
|
27
|
+
STAR = 42
|
|
28
|
+
DOT = 46
|
|
29
|
+
SLASH = 47
|
|
30
|
+
COLON = 58
|
|
31
|
+
QUESTION = 63
|
|
32
|
+
|
|
33
|
+
# Byte lookup tables (index 0..255) for the character classes in the grammar.
|
|
34
|
+
CONTROL_BYTES = (0..31).to_a << 127
|
|
35
|
+
URI_EXCLUDED_BYTES = CONTROL_BYTES + " \"#<>".bytes
|
|
36
|
+
|
|
37
|
+
# uchar | reserved
|
|
38
|
+
URI_BYTE = Array.new(256) { !URI_EXCLUDED_BYTES.include?(it) }.freeze
|
|
39
|
+
# pchar | "/"
|
|
40
|
+
PATH_BYTE = URI_BYTE.each_with_index.map { |allowed, byte| allowed && byte != QUESTION }.freeze
|
|
41
|
+
# alpha | digit | "+" | "-" | "."
|
|
42
|
+
SCHEME_BYTE = Array.new(256) do
|
|
43
|
+
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789+-.".bytes.include?(it)
|
|
44
|
+
end.freeze
|
|
45
|
+
# upper | digit | safe
|
|
46
|
+
METHOD_BYTE = Array.new(256) { "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789$-_.".bytes.include?(it) }.freeze
|
|
47
|
+
DIGIT_BYTE = Array.new(256) { "0123456789".bytes.include?(it) }.freeze
|
|
48
|
+
TSPECIAL_BYTES = "()<>@,;:\\\"/[]?={} \t".bytes
|
|
49
|
+
# token = ascii -- (CTL | tspecials)
|
|
50
|
+
FIELD_NAME_BYTE = Array.new(256) do
|
|
51
|
+
it < 128 && !CONTROL_BYTES.include?(it) && !TSPECIAL_BYTES.include?(it)
|
|
52
|
+
end.freeze
|
|
53
|
+
# (any -- CTL) | "\t"
|
|
54
|
+
FIELD_VALUE_BYTE = Array.new(256) { it == TAB || !CONTROL_BYTES.include?(it) }.freeze
|
|
55
|
+
|
|
56
|
+
# The first byte that ends a run of each class above, so that a run can be
|
|
57
|
+
# skipped with one String#index call instead of a Ruby loop over each byte.
|
|
58
|
+
# Each must agree with its table; test/test_parser.rb checks that.
|
|
59
|
+
METHOD_RUN_END = /[^A-Z0-9$\-_.]/n
|
|
60
|
+
SCHEME_RUN_END = /[^A-Za-z0-9+\-.]/n
|
|
61
|
+
URI_RUN_END = /[\x00-\x20"#<>\x7f]/n
|
|
62
|
+
PATH_RUN_END = /[\x00-\x20"#<>?\x7f]/n
|
|
63
|
+
FIELD_NAME_RUN_END = /[^!#$%&'*+\-.0-9A-Z^_`a-z|~]/n
|
|
64
|
+
FIELD_VALUE_RUN_END = /[\x00-\x08\x0a-\x1f\x7f]/n
|
|
65
|
+
|
|
66
|
+
# A complete header line, using the same byte classes as FIELD_NAME_BYTE
|
|
67
|
+
# and FIELD_VALUE_BYTE, so that a whole line can be handled in one match.
|
|
68
|
+
# Lines that don't match, because they are incomplete or invalid, take the
|
|
69
|
+
# byte by byte path through the states below.
|
|
70
|
+
HEADER_LINE = /\G([!#$%&'*+\-.0-9A-Z^_`a-z|~]+): *([^\x00-\x08\x0a-\x1f\x7f]*)\r\n/n
|
|
71
|
+
|
|
72
|
+
METHOD_MAX_LENGTH = 20
|
|
73
|
+
PROTOCOL_PREFIX = "HTTP/"
|
|
74
|
+
HTTP_PREFIX = "HTTP_"
|
|
75
|
+
|
|
76
|
+
# Env keys for the headers we expect to receive, so that parsing them
|
|
77
|
+
# allocates no key strings. CONTENT_LENGTH and CONTENT_TYPE have no HTTP_
|
|
78
|
+
# prefix, following the CGI convention.
|
|
79
|
+
COMMON_FIELDS = {
|
|
80
|
+
"ACCEPT" => "HTTP_ACCEPT",
|
|
81
|
+
"ACCEPT_CHARSET" => "HTTP_ACCEPT_CHARSET",
|
|
82
|
+
"ACCEPT_ENCODING" => "HTTP_ACCEPT_ENCODING",
|
|
83
|
+
"ACCEPT_LANGUAGE" => "HTTP_ACCEPT_LANGUAGE",
|
|
84
|
+
"ALLOW" => "HTTP_ALLOW",
|
|
85
|
+
"AUTHORIZATION" => "HTTP_AUTHORIZATION",
|
|
86
|
+
"CACHE_CONTROL" => "HTTP_CACHE_CONTROL",
|
|
87
|
+
"CONNECTION" => "HTTP_CONNECTION",
|
|
88
|
+
"CONTENT_ENCODING" => "HTTP_CONTENT_ENCODING",
|
|
89
|
+
"CONTENT_LENGTH" => "CONTENT_LENGTH",
|
|
90
|
+
"CONTENT_TYPE" => "CONTENT_TYPE",
|
|
91
|
+
"COOKIE" => "HTTP_COOKIE",
|
|
92
|
+
"DATE" => "HTTP_DATE",
|
|
93
|
+
"EXPECT" => "HTTP_EXPECT",
|
|
94
|
+
"FROM" => "HTTP_FROM",
|
|
95
|
+
"HOST" => "HTTP_HOST",
|
|
96
|
+
"IF_MATCH" => "HTTP_IF_MATCH",
|
|
97
|
+
"IF_MODIFIED_SINCE" => "HTTP_IF_MODIFIED_SINCE",
|
|
98
|
+
"IF_NONE_MATCH" => "HTTP_IF_NONE_MATCH",
|
|
99
|
+
"IF_RANGE" => "HTTP_IF_RANGE",
|
|
100
|
+
"IF_UNMODIFIED_SINCE" => "HTTP_IF_UNMODIFIED_SINCE",
|
|
101
|
+
"KEEP_ALIVE" => "HTTP_KEEP_ALIVE",
|
|
102
|
+
"MAX_FORWARDS" => "HTTP_MAX_FORWARDS",
|
|
103
|
+
"PRAGMA" => "HTTP_PRAGMA",
|
|
104
|
+
"PROXY_AUTHORIZATION" => "HTTP_PROXY_AUTHORIZATION",
|
|
105
|
+
"RANGE" => "HTTP_RANGE",
|
|
106
|
+
"REFERER" => "HTTP_REFERER",
|
|
107
|
+
"TE" => "HTTP_TE",
|
|
108
|
+
"TRAILER" => "HTTP_TRAILER",
|
|
109
|
+
"TRANSFER_ENCODING" => "HTTP_TRANSFER_ENCODING",
|
|
110
|
+
"UPGRADE" => "HTTP_UPGRADE",
|
|
111
|
+
"USER_AGENT" => "HTTP_USER_AGENT",
|
|
112
|
+
"VIA" => "HTTP_VIA",
|
|
113
|
+
"WARNING" => "HTTP_WARNING",
|
|
114
|
+
"X_FORWARDED_FOR" => "HTTP_X_FORWARDED_FOR",
|
|
115
|
+
"X_REAL_IP" => "HTTP_X_REAL_IP"
|
|
116
|
+
}.freeze
|
|
117
|
+
|
|
118
|
+
REQUEST_METHOD = "REQUEST_METHOD"
|
|
119
|
+
REQUEST_URI = "REQUEST_URI"
|
|
120
|
+
FRAGMENT = "FRAGMENT"
|
|
121
|
+
QUERY_STRING = "QUERY_STRING"
|
|
122
|
+
SERVER_PROTOCOL = "SERVER_PROTOCOL"
|
|
123
|
+
REQUEST_PATH = "REQUEST_PATH"
|
|
124
|
+
|
|
125
|
+
INVALID_FORMAT_MESSAGE = "Invalid HTTP format, parsing fails. " \
|
|
126
|
+
"Are you trying to open an SSL connection to a non-SSL Puma?"
|
|
127
|
+
|
|
128
|
+
# Maximum allowed lengths of the request elements, and their error messages.
|
|
129
|
+
# The messages spell the limits exactly as the C extension does, where they
|
|
130
|
+
# come from the macro text, e.g. "(1024 * 12)". Tests assert on them.
|
|
131
|
+
MAX_FIELD_NAME_LENGTH = 256
|
|
132
|
+
MAX_FIELD_NAME_LENGTH_ERR = "HTTP element FIELD_NAME is longer than the 256 allowed length (was %d)"
|
|
133
|
+
MAX_FIELD_VALUE_LENGTH = 80 * 1024
|
|
134
|
+
MAX_FIELD_VALUE_LENGTH_ERR = "HTTP element FIELD_VALUE is longer than the 80 * 1024 allowed length (was %d)"
|
|
135
|
+
MAX_REQUEST_URI_LENGTH = 1024 * 12
|
|
136
|
+
MAX_REQUEST_URI_LENGTH_ERR = "HTTP element REQUEST_URI is longer than the (1024 * 12) allowed length (was %d)"
|
|
137
|
+
MAX_FRAGMENT_LENGTH = 1024
|
|
138
|
+
MAX_FRAGMENT_LENGTH_ERR = "HTTP element FRAGMENT is longer than the 1024 allowed length (was %d)"
|
|
139
|
+
MAX_REQUEST_PATH_LENGTH = 8192
|
|
140
|
+
MAX_REQUEST_PATH_LENGTH_ERR = "HTTP element REQUEST_PATH is longer than the (8192) allowed length (was %d)"
|
|
141
|
+
MAX_QUERY_STRING_LENGTH = 1024 * 10
|
|
142
|
+
MAX_QUERY_STRING_LENGTH_ERR = "HTTP element QUERY_STRING is longer than the (1024 * 10) allowed length (was %d)"
|
|
143
|
+
MAX_HEADER_LENGTH = 1024 * (80 + 32)
|
|
144
|
+
MAX_HEADER_LENGTH_ERR = "HTTP element HEADER is longer than the (1024 * (80 + 32)) allowed length (was %d)"
|
|
145
|
+
|
|
146
|
+
def initialize
|
|
147
|
+
reset
|
|
148
|
+
end
|
|
149
|
+
|
|
150
|
+
# Resets the parser to its initial state so that it can be reused
|
|
151
|
+
# rather than making new ones.
|
|
152
|
+
def reset
|
|
153
|
+
@state = :method
|
|
154
|
+
@nread = 0
|
|
155
|
+
@mark = 0
|
|
156
|
+
@query_start = 0
|
|
157
|
+
@field_start = 0
|
|
158
|
+
@field_len = 0
|
|
159
|
+
@body_start = 0
|
|
160
|
+
@body = nil
|
|
161
|
+
@env = nil
|
|
162
|
+
nil
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
# Finishes a parser early. You should call reset after finish.
|
|
166
|
+
# The name comes from Puma::HttpParser.
|
|
167
|
+
def finish # rubocop:disable Naming/PredicateMethod
|
|
168
|
+
finished?
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
def error?
|
|
172
|
+
@state == :error
|
|
173
|
+
end
|
|
174
|
+
|
|
175
|
+
def finished?
|
|
176
|
+
@state == :done
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
# The amount of data processed so far during this processing cycle.
|
|
180
|
+
# It is 0 after initialize or reset and is incremented by each execute.
|
|
181
|
+
attr_reader :nread
|
|
182
|
+
|
|
183
|
+
# If the request included a body, returns it.
|
|
184
|
+
attr_reader :body
|
|
185
|
+
|
|
186
|
+
# Takes a Hash and a String of data, parses the String of data filling in
|
|
187
|
+
# the Hash, returning an Integer to indicate how much of the data has been
|
|
188
|
+
# read. Raises HttpParserError when the data is not valid HTTP.
|
|
189
|
+
#
|
|
190
|
+
# The third argument allows for parsing a partial request and then
|
|
191
|
+
# continuing the parsing from that position. It needs all of the original
|
|
192
|
+
# data as well, so you have to append to the data buffer as you read.
|
|
193
|
+
def execute(env, data, start)
|
|
194
|
+
raise HttpParserError, "Requested start is after data buffer end." if start >= data.bytesize
|
|
195
|
+
|
|
196
|
+
@env = env
|
|
197
|
+
# Operate on bytes, like the C extension does.
|
|
198
|
+
@data = data.encoding == Encoding::BINARY ? data : data.b
|
|
199
|
+
stopped_at = run(start, @data.bytesize)
|
|
200
|
+
@nread += stopped_at - start
|
|
201
|
+
|
|
202
|
+
validate_max_length(@nread, MAX_HEADER_LENGTH, MAX_HEADER_LENGTH_ERR)
|
|
203
|
+
raise HttpParserError, INVALID_FORMAT_MESSAGE if error?
|
|
204
|
+
|
|
205
|
+
@nread
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
private
|
|
209
|
+
|
|
210
|
+
# Runs the state machine over @data from position `from` up to `to`.
|
|
211
|
+
# Returns the position it stopped at: `to` when more data is needed,
|
|
212
|
+
# one past the final LF when done, or the offending byte on error.
|
|
213
|
+
def run(from, to)
|
|
214
|
+
state = @state
|
|
215
|
+
position = from
|
|
216
|
+
|
|
217
|
+
while position < to
|
|
218
|
+
byte = @data.getbyte(position)
|
|
219
|
+
|
|
220
|
+
case state
|
|
221
|
+
when :method
|
|
222
|
+
if METHOD_BYTE[byte]
|
|
223
|
+
position = [end_of_run(METHOD_RUN_END, position), @mark + METHOD_MAX_LENGTH].min
|
|
224
|
+
break if position == to
|
|
225
|
+
|
|
226
|
+
byte = @data.getbyte(position)
|
|
227
|
+
end
|
|
228
|
+
if byte == SPACE && position > @mark
|
|
229
|
+
request_method(position)
|
|
230
|
+
state = :uri_start
|
|
231
|
+
else
|
|
232
|
+
state = :error
|
|
233
|
+
break
|
|
234
|
+
end
|
|
235
|
+
when :uri_start
|
|
236
|
+
@mark = position
|
|
237
|
+
if byte == SLASH then state = :path
|
|
238
|
+
elsif byte == STAR then state = :uri_star
|
|
239
|
+
elsif byte == COLON then state = :absolute_uri
|
|
240
|
+
elsif SCHEME_BYTE[byte] then state = :scheme
|
|
241
|
+
else
|
|
242
|
+
state = :error
|
|
243
|
+
break
|
|
244
|
+
end
|
|
245
|
+
when :uri_star
|
|
246
|
+
if byte == SPACE
|
|
247
|
+
request_uri(position)
|
|
248
|
+
state = :protocol_start
|
|
249
|
+
elsif byte == HASH
|
|
250
|
+
request_uri(position)
|
|
251
|
+
state = :fragment_start
|
|
252
|
+
else
|
|
253
|
+
state = :error
|
|
254
|
+
break
|
|
255
|
+
end
|
|
256
|
+
when :scheme
|
|
257
|
+
if SCHEME_BYTE[byte]
|
|
258
|
+
position = end_of_run(SCHEME_RUN_END, position)
|
|
259
|
+
break if position == to
|
|
260
|
+
|
|
261
|
+
byte = @data.getbyte(position)
|
|
262
|
+
end
|
|
263
|
+
if byte == COLON
|
|
264
|
+
state = :absolute_uri
|
|
265
|
+
else
|
|
266
|
+
state = :error
|
|
267
|
+
break
|
|
268
|
+
end
|
|
269
|
+
when :absolute_uri
|
|
270
|
+
if URI_BYTE[byte]
|
|
271
|
+
position = end_of_run(URI_RUN_END, position)
|
|
272
|
+
break if position == to
|
|
273
|
+
|
|
274
|
+
byte = @data.getbyte(position)
|
|
275
|
+
end
|
|
276
|
+
if byte == SPACE
|
|
277
|
+
request_uri(position)
|
|
278
|
+
state = :protocol_start
|
|
279
|
+
elsif byte == HASH
|
|
280
|
+
request_uri(position)
|
|
281
|
+
state = :fragment_start
|
|
282
|
+
else
|
|
283
|
+
state = :error
|
|
284
|
+
break
|
|
285
|
+
end
|
|
286
|
+
when :path
|
|
287
|
+
if PATH_BYTE[byte]
|
|
288
|
+
position = end_of_run(PATH_RUN_END, position)
|
|
289
|
+
break if position == to
|
|
290
|
+
|
|
291
|
+
byte = @data.getbyte(position)
|
|
292
|
+
end
|
|
293
|
+
case byte
|
|
294
|
+
when QUESTION
|
|
295
|
+
request_path(position)
|
|
296
|
+
@query_start = position + 1
|
|
297
|
+
state = :query
|
|
298
|
+
when SPACE
|
|
299
|
+
request_path(position)
|
|
300
|
+
request_uri(position)
|
|
301
|
+
state = :protocol_start
|
|
302
|
+
when HASH
|
|
303
|
+
request_path(position)
|
|
304
|
+
request_uri(position)
|
|
305
|
+
state = :fragment_start
|
|
306
|
+
else
|
|
307
|
+
state = :error
|
|
308
|
+
break
|
|
309
|
+
end
|
|
310
|
+
when :query
|
|
311
|
+
if URI_BYTE[byte]
|
|
312
|
+
position = end_of_run(URI_RUN_END, position)
|
|
313
|
+
break if position == to
|
|
314
|
+
|
|
315
|
+
byte = @data.getbyte(position)
|
|
316
|
+
end
|
|
317
|
+
if byte == SPACE
|
|
318
|
+
query_string(position)
|
|
319
|
+
request_uri(position)
|
|
320
|
+
state = :protocol_start
|
|
321
|
+
elsif byte == HASH
|
|
322
|
+
query_string(position)
|
|
323
|
+
request_uri(position)
|
|
324
|
+
state = :fragment_start
|
|
325
|
+
else
|
|
326
|
+
state = :error
|
|
327
|
+
break
|
|
328
|
+
end
|
|
329
|
+
when :fragment_start
|
|
330
|
+
@mark = position
|
|
331
|
+
if URI_BYTE[byte]
|
|
332
|
+
state = :fragment
|
|
333
|
+
elsif byte == SPACE
|
|
334
|
+
fragment(position)
|
|
335
|
+
state = :protocol_start
|
|
336
|
+
else
|
|
337
|
+
state = :error
|
|
338
|
+
break
|
|
339
|
+
end
|
|
340
|
+
when :fragment
|
|
341
|
+
if URI_BYTE[byte]
|
|
342
|
+
position = end_of_run(URI_RUN_END, position)
|
|
343
|
+
break if position == to
|
|
344
|
+
|
|
345
|
+
byte = @data.getbyte(position)
|
|
346
|
+
end
|
|
347
|
+
if byte == SPACE
|
|
348
|
+
fragment(position)
|
|
349
|
+
state = :protocol_start
|
|
350
|
+
else
|
|
351
|
+
state = :error
|
|
352
|
+
break
|
|
353
|
+
end
|
|
354
|
+
when :protocol_start
|
|
355
|
+
@mark = position
|
|
356
|
+
if byte == PROTOCOL_PREFIX.getbyte(0)
|
|
357
|
+
state = :protocol_prefix
|
|
358
|
+
else
|
|
359
|
+
state = :error
|
|
360
|
+
break
|
|
361
|
+
end
|
|
362
|
+
when :protocol_prefix
|
|
363
|
+
if byte == PROTOCOL_PREFIX.getbyte(position - @mark)
|
|
364
|
+
state = :protocol_major if position - @mark == PROTOCOL_PREFIX.bytesize - 1
|
|
365
|
+
else
|
|
366
|
+
state = :error
|
|
367
|
+
break
|
|
368
|
+
end
|
|
369
|
+
when :protocol_major
|
|
370
|
+
if DIGIT_BYTE[byte]
|
|
371
|
+
state = :protocol_major_digits
|
|
372
|
+
else
|
|
373
|
+
state = :error
|
|
374
|
+
break
|
|
375
|
+
end
|
|
376
|
+
when :protocol_major_digits
|
|
377
|
+
if DIGIT_BYTE[byte]
|
|
378
|
+
# continue
|
|
379
|
+
elsif byte == DOT
|
|
380
|
+
state = :protocol_minor
|
|
381
|
+
else
|
|
382
|
+
state = :error
|
|
383
|
+
break
|
|
384
|
+
end
|
|
385
|
+
when :protocol_minor
|
|
386
|
+
if DIGIT_BYTE[byte]
|
|
387
|
+
state = :protocol_minor_digits
|
|
388
|
+
else
|
|
389
|
+
state = :error
|
|
390
|
+
break
|
|
391
|
+
end
|
|
392
|
+
when :protocol_minor_digits
|
|
393
|
+
if DIGIT_BYTE[byte]
|
|
394
|
+
# continue
|
|
395
|
+
elsif byte == CR
|
|
396
|
+
server_protocol(position)
|
|
397
|
+
state = :request_line_lf
|
|
398
|
+
else
|
|
399
|
+
state = :error
|
|
400
|
+
break
|
|
401
|
+
end
|
|
402
|
+
when :header_lf, :request_line_lf
|
|
403
|
+
if byte == LF
|
|
404
|
+
state = :header_line
|
|
405
|
+
else
|
|
406
|
+
state = :error
|
|
407
|
+
break
|
|
408
|
+
end
|
|
409
|
+
when :header_line
|
|
410
|
+
if byte == CR
|
|
411
|
+
state = :final_lf
|
|
412
|
+
elsif FIELD_NAME_BYTE[byte]
|
|
413
|
+
if (line = HEADER_LINE.match(@data, position))
|
|
414
|
+
@field_start = position
|
|
415
|
+
@field_len = line.end(1) - position
|
|
416
|
+
@mark = line.begin(2)
|
|
417
|
+
http_field(line.end(2))
|
|
418
|
+
position = line.end(0)
|
|
419
|
+
next
|
|
420
|
+
end
|
|
421
|
+
@field_start = position
|
|
422
|
+
state = :field_name
|
|
423
|
+
else
|
|
424
|
+
state = :error
|
|
425
|
+
break
|
|
426
|
+
end
|
|
427
|
+
when :field_name
|
|
428
|
+
if FIELD_NAME_BYTE[byte]
|
|
429
|
+
position = end_of_run(FIELD_NAME_RUN_END, position)
|
|
430
|
+
break if position == to
|
|
431
|
+
|
|
432
|
+
byte = @data.getbyte(position)
|
|
433
|
+
end
|
|
434
|
+
if byte == COLON
|
|
435
|
+
@field_len = position - @field_start
|
|
436
|
+
state = :field_value_start
|
|
437
|
+
else
|
|
438
|
+
state = :error
|
|
439
|
+
break
|
|
440
|
+
end
|
|
441
|
+
when :field_value_start
|
|
442
|
+
if byte == SPACE
|
|
443
|
+
# leading spaces are not part of the value
|
|
444
|
+
elsif FIELD_VALUE_BYTE[byte]
|
|
445
|
+
@mark = position
|
|
446
|
+
state = :field_value
|
|
447
|
+
elsif byte == CR
|
|
448
|
+
@mark = position
|
|
449
|
+
http_field(position)
|
|
450
|
+
state = :header_lf
|
|
451
|
+
else
|
|
452
|
+
state = :error
|
|
453
|
+
break
|
|
454
|
+
end
|
|
455
|
+
when :field_value
|
|
456
|
+
if FIELD_VALUE_BYTE[byte]
|
|
457
|
+
position = end_of_run(FIELD_VALUE_RUN_END, position)
|
|
458
|
+
break if position == to
|
|
459
|
+
|
|
460
|
+
byte = @data.getbyte(position)
|
|
461
|
+
end
|
|
462
|
+
if byte == CR
|
|
463
|
+
http_field(position)
|
|
464
|
+
state = :header_lf
|
|
465
|
+
else
|
|
466
|
+
state = :error
|
|
467
|
+
break
|
|
468
|
+
end
|
|
469
|
+
when :final_lf
|
|
470
|
+
if byte == LF
|
|
471
|
+
header_done(position)
|
|
472
|
+
state = :done
|
|
473
|
+
position += 1
|
|
474
|
+
else
|
|
475
|
+
state = :error
|
|
476
|
+
end
|
|
477
|
+
break
|
|
478
|
+
when :done
|
|
479
|
+
state = :error
|
|
480
|
+
break
|
|
481
|
+
else
|
|
482
|
+
break
|
|
483
|
+
end
|
|
484
|
+
|
|
485
|
+
position += 1
|
|
486
|
+
end
|
|
487
|
+
|
|
488
|
+
@state = state
|
|
489
|
+
position
|
|
490
|
+
end
|
|
491
|
+
|
|
492
|
+
# The byte at `position` is known to be in the run. Returns the position of
|
|
493
|
+
# the first byte after the run, or the end of the data.
|
|
494
|
+
def end_of_run(run_end, position)
|
|
495
|
+
@data.index(run_end, position + 1) || @data.bytesize
|
|
496
|
+
end
|
|
497
|
+
|
|
498
|
+
def validate_max_length(length, max_length, message)
|
|
499
|
+
raise HttpParserError, format(message, length) if length > max_length
|
|
500
|
+
end
|
|
501
|
+
|
|
502
|
+
def http_field(position)
|
|
503
|
+
value_length = position - @mark
|
|
504
|
+
validate_max_length(@field_len, MAX_FIELD_NAME_LENGTH, MAX_FIELD_NAME_LENGTH_ERR)
|
|
505
|
+
validate_max_length(value_length, MAX_FIELD_VALUE_LENGTH, MAX_FIELD_VALUE_LENGTH_ERR)
|
|
506
|
+
|
|
507
|
+
# Upcase, "-" becomes "_", and "_" becomes "," so that a header with
|
|
508
|
+
# underscores cannot impersonate one with dashes.
|
|
509
|
+
name = @data.byteslice(@field_start, @field_len)
|
|
510
|
+
name.upcase!
|
|
511
|
+
name.tr!("-_", "_,")
|
|
512
|
+
key = COMMON_FIELDS[name] || -"#{HTTP_PREFIX}#{name}"
|
|
513
|
+
|
|
514
|
+
value = @data.byteslice(@mark, value_length)
|
|
515
|
+
# Only spaces and tabs can be present; the grammar rejects other control bytes.
|
|
516
|
+
value.strip!
|
|
517
|
+
|
|
518
|
+
existing = @env[key]
|
|
519
|
+
if existing.nil?
|
|
520
|
+
@env[key] = value
|
|
521
|
+
else
|
|
522
|
+
# duplicate headers are normalized to comma-separated values
|
|
523
|
+
existing << ", " << value
|
|
524
|
+
end
|
|
525
|
+
end
|
|
526
|
+
|
|
527
|
+
def request_method(position)
|
|
528
|
+
@env[REQUEST_METHOD] = @data.byteslice(@mark, position - @mark)
|
|
529
|
+
end
|
|
530
|
+
|
|
531
|
+
def request_uri(position)
|
|
532
|
+
length = position - @mark
|
|
533
|
+
validate_max_length(length, MAX_REQUEST_URI_LENGTH, MAX_REQUEST_URI_LENGTH_ERR)
|
|
534
|
+
@env[REQUEST_URI] = @data.byteslice(@mark, length)
|
|
535
|
+
end
|
|
536
|
+
|
|
537
|
+
def fragment(position)
|
|
538
|
+
length = position - @mark
|
|
539
|
+
validate_max_length(length, MAX_FRAGMENT_LENGTH, MAX_FRAGMENT_LENGTH_ERR)
|
|
540
|
+
@env[FRAGMENT] = @data.byteslice(@mark, length)
|
|
541
|
+
end
|
|
542
|
+
|
|
543
|
+
def request_path(position)
|
|
544
|
+
length = position - @mark
|
|
545
|
+
validate_max_length(length, MAX_REQUEST_PATH_LENGTH, MAX_REQUEST_PATH_LENGTH_ERR)
|
|
546
|
+
@env[REQUEST_PATH] = @data.byteslice(@mark, length)
|
|
547
|
+
end
|
|
548
|
+
|
|
549
|
+
def query_string(position)
|
|
550
|
+
length = position - @query_start
|
|
551
|
+
validate_max_length(length, MAX_QUERY_STRING_LENGTH, MAX_QUERY_STRING_LENGTH_ERR)
|
|
552
|
+
@env[QUERY_STRING] = @data.byteslice(@query_start, length)
|
|
553
|
+
end
|
|
554
|
+
|
|
555
|
+
def server_protocol(position)
|
|
556
|
+
@env[SERVER_PROTOCOL] = @data.byteslice(@mark, position - @mark)
|
|
557
|
+
end
|
|
558
|
+
|
|
559
|
+
def header_done(position)
|
|
560
|
+
@body_start = position + 1
|
|
561
|
+
@body = @data.byteslice(@body_start, @data.bytesize - @body_start)
|
|
562
|
+
end
|
|
563
|
+
end
|
|
564
|
+
end
|
|
565
|
+
end
|
data/lib/puma/http1.rb
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "http1/version"
|
|
4
|
+
require_relative "http1/parser"
|
|
5
|
+
|
|
6
|
+
module Puma
|
|
7
|
+
# An HTTP/1.x parser for Puma, written in Ruby. Use it with Puma's
|
|
8
|
+
# `http_parser` option:
|
|
9
|
+
#
|
|
10
|
+
# require "puma/http1"
|
|
11
|
+
# http_parser Puma::HTTP1::Parser
|
|
12
|
+
#
|
|
13
|
+
# It's named HTTP1, not HTTP, because Puma::Const::HTTP exists, and Puma's
|
|
14
|
+
# code refers to it without the Const:: prefix. A Puma::HTTP module would be
|
|
15
|
+
# found first and break Puma.
|
|
16
|
+
module HTTP1
|
|
17
|
+
end
|
|
18
|
+
end
|
data/sig/puma/http1.rbs
ADDED
metadata
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: puma-http1
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.0.1
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Shane Becker
|
|
8
|
+
bindir: exe
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: puma
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '0'
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - ">="
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: '0'
|
|
26
|
+
description: A drop-in for the HTTP parser in Puma's puma_http11 C extension, for
|
|
27
|
+
use with Puma's http_parser option.
|
|
28
|
+
email:
|
|
29
|
+
- veganstraightedge@gmail.com
|
|
30
|
+
executables: []
|
|
31
|
+
extensions: []
|
|
32
|
+
extra_rdoc_files: []
|
|
33
|
+
files:
|
|
34
|
+
- CHANGELOG.md
|
|
35
|
+
- LICENSE.md
|
|
36
|
+
- README.md
|
|
37
|
+
- Rakefile
|
|
38
|
+
- lib/puma/http1.rb
|
|
39
|
+
- lib/puma/http1/parser.rb
|
|
40
|
+
- lib/puma/http1/version.rb
|
|
41
|
+
- sig/puma/http1.rbs
|
|
42
|
+
homepage: https://github.com/veganstraightedge/puma-http1
|
|
43
|
+
licenses:
|
|
44
|
+
- BSD-3-Clause
|
|
45
|
+
metadata:
|
|
46
|
+
allowed_push_host: https://rubygems.org
|
|
47
|
+
homepage_uri: https://github.com/veganstraightedge/puma-http1
|
|
48
|
+
source_code_uri: https://github.com/veganstraightedge/puma-http1
|
|
49
|
+
changelog_uri: https://github.com/veganstraightedge/puma-http1/blob/main/CHANGELOG.md
|
|
50
|
+
rubygems_mfa_required: 'true'
|
|
51
|
+
rdoc_options: []
|
|
52
|
+
require_paths:
|
|
53
|
+
- lib
|
|
54
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
55
|
+
requirements:
|
|
56
|
+
- - ">="
|
|
57
|
+
- !ruby/object:Gem::Version
|
|
58
|
+
version: 3.4.0
|
|
59
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
60
|
+
requirements:
|
|
61
|
+
- - ">="
|
|
62
|
+
- !ruby/object:Gem::Version
|
|
63
|
+
version: '0'
|
|
64
|
+
requirements: []
|
|
65
|
+
rubygems_version: 4.0.21
|
|
66
|
+
specification_version: 4
|
|
67
|
+
summary: An HTTP parser for Puma, written in Ruby.
|
|
68
|
+
test_files: []
|