protocol-http 0.53.0 → 0.54.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/context/headers.md +94 -0
- data/context/index.yaml +3 -0
- data/lib/protocol/http/header/accept.rb +6 -0
- data/lib/protocol/http/header/authorization.rb +7 -1
- data/lib/protocol/http/header/connection.rb +7 -0
- data/lib/protocol/http/header/cookie.rb +7 -0
- data/lib/protocol/http/header/date.rb +8 -1
- data/lib/protocol/http/header/digest.rb +70 -0
- data/lib/protocol/http/header/etag.rb +7 -0
- data/lib/protocol/http/header/multiple.rb +7 -0
- data/lib/protocol/http/header/server_timing.rb +92 -0
- data/lib/protocol/http/header/split.rb +7 -0
- data/lib/protocol/http/header/te.rb +131 -0
- data/lib/protocol/http/header/trailer.rb +23 -0
- data/lib/protocol/http/header/transfer_encoding.rb +78 -0
- data/lib/protocol/http/headers.rb +55 -7
- data/lib/protocol/http/version.rb +1 -1
- data/readme.md +7 -0
- data/releases.md +49 -0
- data.tar.gz.sig +0 -0
- metadata +7 -1
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c76db95baeca082a769340ab2a29ee5948bb837894e55039c082653453aa4f4f
|
4
|
+
data.tar.gz: 868cdcc4a21786c640c8c06e7892bfd6ef19f81670a90274692cfadef10a0c39
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: acc4afb3f7caba7ec2d2611ab72dce9c954e88e8dfe72645645e442d8116909c3f228b22c2d460633d429510ffff5f6f4713bfff751f18d4873db167e773fcbe
|
7
|
+
data.tar.gz: ef0e92bac69dba33417d074c24ce6d2f2009e19e1d81d317eecaf945817f3c7fb0775052048ca24df32b3f39c15f8de83ca45054669e2d89b5bf3b5b445c7cf1
|
checksums.yaml.gz.sig
CHANGED
Binary file
|
data/context/headers.md
ADDED
@@ -0,0 +1,94 @@
|
|
1
|
+
# Headers
|
2
|
+
|
3
|
+
This guide explains how to work with HTTP headers using `protocol-http`.
|
4
|
+
|
5
|
+
## Core Concepts
|
6
|
+
|
7
|
+
`protocol-http` provides several core concepts for working with HTTP headers:
|
8
|
+
|
9
|
+
- A {ruby Protocol::HTTP::Headers} class which represents a collection of HTTP headers with built-in security and policy features.
|
10
|
+
- Header-specific classes like {ruby Protocol::HTTP::Header::Accept} and {ruby Protocol::HTTP::Header::Authorization} which provide specialized parsing and formatting.
|
11
|
+
- Trailer security validation to prevent HTTP request smuggling attacks.
|
12
|
+
|
13
|
+
## Usage
|
14
|
+
|
15
|
+
The {Protocol::HTTP::Headers} class provides a comprehensive interface for creating and manipulating HTTP headers:
|
16
|
+
|
17
|
+
```ruby
|
18
|
+
require 'protocol/http'
|
19
|
+
|
20
|
+
headers = Protocol::HTTP::Headers.new
|
21
|
+
headers.add('content-type', 'text/html')
|
22
|
+
headers.add('set-cookie', 'session=abc123')
|
23
|
+
|
24
|
+
# Access headers
|
25
|
+
content_type = headers['content-type'] # => "text/html"
|
26
|
+
|
27
|
+
# Check if header exists
|
28
|
+
headers.include?('content-type') # => true
|
29
|
+
```
|
30
|
+
|
31
|
+
### Header Policies
|
32
|
+
|
33
|
+
Different header types have different behaviors for merging, validation, and trailer handling:
|
34
|
+
|
35
|
+
```ruby
|
36
|
+
# Some headers can be specified multiple times
|
37
|
+
headers.add('set-cookie', 'first=value1')
|
38
|
+
headers.add('set-cookie', 'second=value2')
|
39
|
+
|
40
|
+
# Others are singletons and will raise errors if duplicated
|
41
|
+
headers.add('content-length', '100')
|
42
|
+
# headers.add('content-length', '200') # Would raise DuplicateHeaderError
|
43
|
+
```
|
44
|
+
|
45
|
+
### Structured Headers
|
46
|
+
|
47
|
+
Some headers have specialized classes for parsing and formatting:
|
48
|
+
|
49
|
+
```ruby
|
50
|
+
# Accept header with media ranges
|
51
|
+
accept = Protocol::HTTP::Header::Accept.new('text/html,application/json;q=0.9')
|
52
|
+
media_ranges = accept.media_ranges
|
53
|
+
|
54
|
+
# Authorization header
|
55
|
+
auth = Protocol::HTTP::Header::Authorization.basic('username', 'password')
|
56
|
+
# => "Basic dXNlcm5hbWU6cGFzc3dvcmQ="
|
57
|
+
```
|
58
|
+
|
59
|
+
### Trailer Security
|
60
|
+
|
61
|
+
HTTP trailers are headers that appear after the message body. For security reasons, only certain headers are allowed in trailers:
|
62
|
+
|
63
|
+
```ruby
|
64
|
+
# Working with trailers
|
65
|
+
headers = Protocol::HTTP::Headers.new([
|
66
|
+
['content-type', 'text/html'],
|
67
|
+
['content-length', '1000']
|
68
|
+
])
|
69
|
+
|
70
|
+
# Start trailer section
|
71
|
+
headers.trailer!
|
72
|
+
|
73
|
+
# These will be allowed (safe metadata)
|
74
|
+
headers.add('etag', '"12345"')
|
75
|
+
headers.add('date', Time.now.httpdate)
|
76
|
+
|
77
|
+
# These will be silently ignored for security
|
78
|
+
headers.add('authorization', 'Bearer token') # Ignored - credential leakage risk
|
79
|
+
headers.add('connection', 'close') # Ignored - hop-by-hop header
|
80
|
+
```
|
81
|
+
|
82
|
+
The trailer security system prevents HTTP request smuggling by restricting which headers can appear in trailers:
|
83
|
+
|
84
|
+
**Allowed headers** (return `true` for `trailer?`):
|
85
|
+
- `date` - Response generation timestamps.
|
86
|
+
- `digest` - Content integrity verification.
|
87
|
+
- `etag` - Cache validation tags.
|
88
|
+
- `server-timing` - Performance metrics.
|
89
|
+
|
90
|
+
**Forbidden headers** (return `false` for `trailer?`):
|
91
|
+
- `authorization` - Prevents credential leakage.
|
92
|
+
- `connection`, `te`, `transfer-encoding` - Hop-by-hop headers that control connection behavior.
|
93
|
+
- `cookie`, `set-cookie` - State information needed during initial processing.
|
94
|
+
- `accept` - Content negotiation must occur before response generation.
|
data/context/index.yaml
CHANGED
@@ -14,6 +14,9 @@ files:
|
|
14
14
|
title: Message Body
|
15
15
|
description: This guide explains how to work with HTTP request and response message
|
16
16
|
bodies using `Protocol::HTTP::Body` classes.
|
17
|
+
- path: headers.md
|
18
|
+
title: Headers
|
19
|
+
description: This guide explains how to work with HTTP headers using `protocol-http`.
|
17
20
|
- path: middleware.md
|
18
21
|
title: Middleware
|
19
22
|
description: This guide explains how to build and use HTTP middleware with `Protocol::HTTP::Middleware`.
|
@@ -92,6 +92,12 @@ module Protocol
|
|
92
92
|
join(",")
|
93
93
|
end
|
94
94
|
|
95
|
+
# Whether this header is acceptable in HTTP trailers.
|
96
|
+
# @returns [Boolean] `false`, as Accept headers are used for response content negotiation.
|
97
|
+
def self.trailer?
|
98
|
+
false
|
99
|
+
end
|
100
|
+
|
95
101
|
# Parse the `accept` header.
|
96
102
|
#
|
97
103
|
# @returns [Array(Charset)] the list of content types and their associated parameters.
|
@@ -1,7 +1,7 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
3
|
# Released under the MIT License.
|
4
|
-
# Copyright, 2019-
|
4
|
+
# Copyright, 2019-2025, by Samuel Williams.
|
5
5
|
# Copyright, 2024, by Earlopain.
|
6
6
|
|
7
7
|
module Protocol
|
@@ -34,6 +34,12 @@ module Protocol
|
|
34
34
|
"Basic #{strict_base64_encoded}"
|
35
35
|
)
|
36
36
|
end
|
37
|
+
|
38
|
+
# Whether this header is acceptable in HTTP trailers.
|
39
|
+
# @returns [Boolean] `false`, as authorization headers are used for request authentication.
|
40
|
+
def self.trailer?
|
41
|
+
false
|
42
|
+
end
|
37
43
|
end
|
38
44
|
end
|
39
45
|
end
|
@@ -50,6 +50,13 @@ module Protocol
|
|
50
50
|
def upgrade?
|
51
51
|
self.include?(UPGRADE)
|
52
52
|
end
|
53
|
+
|
54
|
+
# Whether this header is acceptable in HTTP trailers.
|
55
|
+
# Connection headers control the current connection and must not appear in trailers.
|
56
|
+
# @returns [Boolean] `false`, as connection headers are hop-by-hop and forbidden in trailers.
|
57
|
+
def self.trailer?
|
58
|
+
false
|
59
|
+
end
|
53
60
|
end
|
54
61
|
end
|
55
62
|
end
|
@@ -23,6 +23,13 @@ module Protocol
|
|
23
23
|
|
24
24
|
cookies.map{|cookie| [cookie.name, cookie]}.to_h
|
25
25
|
end
|
26
|
+
|
27
|
+
# Whether this header is acceptable in HTTP trailers.
|
28
|
+
# Cookie headers should not appear in trailers as they contain state information needed early in processing.
|
29
|
+
# @returns [Boolean] `false`, as cookie headers are needed during initial request processing.
|
30
|
+
def self.trailer?
|
31
|
+
false
|
32
|
+
end
|
26
33
|
end
|
27
34
|
|
28
35
|
# The `set-cookie` header sends cookies from the server to the user agent.
|
@@ -1,7 +1,7 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
3
|
# Released under the MIT License.
|
4
|
-
# Copyright, 2023-
|
4
|
+
# Copyright, 2023-2025, by Samuel Williams.
|
5
5
|
|
6
6
|
require "time"
|
7
7
|
|
@@ -25,6 +25,13 @@ module Protocol
|
|
25
25
|
def to_time
|
26
26
|
::Time.parse(self)
|
27
27
|
end
|
28
|
+
|
29
|
+
# Whether this header is acceptable in HTTP trailers.
|
30
|
+
# Date headers can safely appear in trailers as they provide metadata about response generation.
|
31
|
+
# @returns [Boolean] `true`, as date headers are metadata that can be computed after response generation.
|
32
|
+
def self.trailer?
|
33
|
+
true
|
34
|
+
end
|
28
35
|
end
|
29
36
|
end
|
30
37
|
end
|
@@ -0,0 +1,70 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Released under the MIT License.
|
4
|
+
# Copyright, 2025, by Samuel Williams.
|
5
|
+
|
6
|
+
require_relative "split"
|
7
|
+
require_relative "quoted_string"
|
8
|
+
require_relative "../error"
|
9
|
+
|
10
|
+
module Protocol
|
11
|
+
module HTTP
|
12
|
+
module Header
|
13
|
+
# The `digest` header provides a digest of the message body for integrity verification.
|
14
|
+
#
|
15
|
+
# This header allows servers to send cryptographic hashes of the response body, enabling clients to verify data integrity. Multiple digest algorithms can be specified, and the header is particularly useful as a trailer since the digest can only be computed after the entire message body is available.
|
16
|
+
#
|
17
|
+
# ## Examples
|
18
|
+
#
|
19
|
+
# ```ruby
|
20
|
+
# digest = Digest.new("sha-256=X48E9qOokqqrvdts8nOJRJN3OWDUoyWxBf7kbu9DBPE=")
|
21
|
+
# digest << "md5=9bb58f26192e4ba00f01e2e7b136bbd8"
|
22
|
+
# puts digest.to_s
|
23
|
+
# # => "sha-256=X48E9qOokqqrvdts8nOJRJN3OWDUoyWxBf7kbu9DBPE=, md5=9bb58f26192e4ba00f01e2e7b136bbd8"
|
24
|
+
# ```
|
25
|
+
class Digest < Split
|
26
|
+
ParseError = Class.new(Error)
|
27
|
+
|
28
|
+
# https://tools.ietf.org/html/rfc3230#section-4.3.2
|
29
|
+
ENTRY = /\A(?<algorithm>[a-zA-Z0-9][a-zA-Z0-9\-]*)\s*=\s*(?<value>.*)\z/
|
30
|
+
|
31
|
+
# A single digest entry in the Digest header.
|
32
|
+
Entry = Struct.new(:algorithm, :value) do
|
33
|
+
# Create a new digest entry.
|
34
|
+
#
|
35
|
+
# @parameter algorithm [String] the digest algorithm (e.g., "sha-256", "md5").
|
36
|
+
# @parameter value [String] the base64-encoded or hex-encoded digest value.
|
37
|
+
def initialize(algorithm, value)
|
38
|
+
super(algorithm.downcase, value)
|
39
|
+
end
|
40
|
+
|
41
|
+
# Convert the entry to its string representation.
|
42
|
+
#
|
43
|
+
# @returns [String] the formatted digest string.
|
44
|
+
def to_s
|
45
|
+
"#{algorithm}=#{value}"
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
# Parse the `digest` header value into a list of digest entries.
|
50
|
+
#
|
51
|
+
# @returns [Array(Entry)] the list of digest entries with their algorithms and values.
|
52
|
+
def entries
|
53
|
+
self.map do |value|
|
54
|
+
if match = value.match(ENTRY)
|
55
|
+
Entry.new(match[:algorithm], match[:value])
|
56
|
+
else
|
57
|
+
raise ParseError.new("Could not parse digest value: #{value.inspect}")
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
# Whether this header is acceptable in HTTP trailers.
|
63
|
+
# @returns [Boolean] `true`, as digest headers contain integrity hashes that can only be calculated after the entire message body is available.
|
64
|
+
def self.trailer?
|
65
|
+
true
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
69
|
+
end
|
70
|
+
end
|
@@ -25,6 +25,13 @@ module Protocol
|
|
25
25
|
def weak?
|
26
26
|
self.start_with?("W/")
|
27
27
|
end
|
28
|
+
|
29
|
+
# Whether this header is acceptable in HTTP trailers.
|
30
|
+
# ETag headers can safely appear in trailers as they provide cache validation metadata.
|
31
|
+
# @returns [Boolean] `true`, as ETag headers are metadata that can be computed after response generation.
|
32
|
+
def self.trailer?
|
33
|
+
true
|
34
|
+
end
|
28
35
|
end
|
29
36
|
end
|
30
37
|
end
|
@@ -25,6 +25,13 @@ module Protocol
|
|
25
25
|
def to_s
|
26
26
|
join("\n")
|
27
27
|
end
|
28
|
+
|
29
|
+
# Whether this header is acceptable in HTTP trailers.
|
30
|
+
# This is a base class for headers with multiple values, default is to disallow in trailers.
|
31
|
+
# @returns [Boolean] `false`, as most multiple-value headers should not appear in trailers by default.
|
32
|
+
def self.trailer?
|
33
|
+
false
|
34
|
+
end
|
28
35
|
end
|
29
36
|
end
|
30
37
|
end
|
@@ -0,0 +1,92 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Released under the MIT License.
|
4
|
+
# Copyright, 2025, by Samuel Williams.
|
5
|
+
|
6
|
+
require_relative "split"
|
7
|
+
require_relative "quoted_string"
|
8
|
+
require_relative "../error"
|
9
|
+
|
10
|
+
module Protocol
|
11
|
+
module HTTP
|
12
|
+
module Header
|
13
|
+
# The `server-timing` header communicates performance metrics about the request-response cycle to the client.
|
14
|
+
#
|
15
|
+
# This header allows servers to send timing information about various server-side operations, which can be useful for performance monitoring and debugging. Each metric can include a name, optional duration, and optional description.
|
16
|
+
#
|
17
|
+
# ## Examples
|
18
|
+
#
|
19
|
+
# ```ruby
|
20
|
+
# server_timing = ServerTiming.new("db;dur=53.2")
|
21
|
+
# server_timing << "cache;dur=12.1;desc=\"Redis lookup\""
|
22
|
+
# puts server_timing.to_s
|
23
|
+
# # => "db;dur=53.2, cache;dur=12.1;desc=\"Redis lookup\""
|
24
|
+
# ```
|
25
|
+
class ServerTiming < Split
|
26
|
+
ParseError = Class.new(Error)
|
27
|
+
|
28
|
+
# https://www.w3.org/TR/server-timing/
|
29
|
+
METRIC = /\A(?<name>[a-zA-Z0-9][a-zA-Z0-9_\-]*)(;(?<parameters>.*))?\z/
|
30
|
+
PARAMETER = /(?<key>dur|desc)=((?<value>#{TOKEN})|(?<quoted_value>#{QUOTED_STRING}))/
|
31
|
+
|
32
|
+
# A single metric in the Server-Timing header.
|
33
|
+
Metric = Struct.new(:name, :duration, :description) do
|
34
|
+
# Create a new server timing metric.
|
35
|
+
#
|
36
|
+
# @parameter name [String] the name of the metric.
|
37
|
+
# @parameter duration [Float | Nil] the duration in milliseconds.
|
38
|
+
# @parameter description [String | Nil] the description of the metric.
|
39
|
+
def initialize(name, duration = nil, description = nil)
|
40
|
+
super(name, duration, description)
|
41
|
+
end
|
42
|
+
|
43
|
+
# Convert the metric to its string representation.
|
44
|
+
#
|
45
|
+
# @returns [String] the formatted metric string.
|
46
|
+
def to_s
|
47
|
+
result = name.dup
|
48
|
+
result << ";dur=#{duration}" if duration
|
49
|
+
result << ";desc=\"#{description}\"" if description
|
50
|
+
result
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
# Parse the `server-timing` header value into a list of metrics.
|
55
|
+
#
|
56
|
+
# @returns [Array(Metric)] the list of metrics with their names, durations, and descriptions.
|
57
|
+
def metrics
|
58
|
+
self.map do |value|
|
59
|
+
if match = value.match(METRIC)
|
60
|
+
name = match[:name]
|
61
|
+
parameters = match[:parameters] || ""
|
62
|
+
|
63
|
+
duration = nil
|
64
|
+
description = nil
|
65
|
+
|
66
|
+
parameters.scan(PARAMETER) do |key, value, quoted_value|
|
67
|
+
value = QuotedString.unquote(quoted_value) if quoted_value
|
68
|
+
|
69
|
+
case key
|
70
|
+
when "dur"
|
71
|
+
duration = value.to_f
|
72
|
+
when "desc"
|
73
|
+
description = value
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
Metric.new(name, duration, description)
|
78
|
+
else
|
79
|
+
raise ParseError.new("Could not parse server timing metric: #{value.inspect}")
|
80
|
+
end
|
81
|
+
end
|
82
|
+
end
|
83
|
+
|
84
|
+
# Whether this header is acceptable in HTTP trailers.
|
85
|
+
# @returns [Boolean] `true`, as server-timing headers contain performance metrics that are typically calculated during response generation.
|
86
|
+
def self.trailer?
|
87
|
+
true
|
88
|
+
end
|
89
|
+
end
|
90
|
+
end
|
91
|
+
end
|
92
|
+
end
|
@@ -40,6 +40,13 @@ module Protocol
|
|
40
40
|
join(",")
|
41
41
|
end
|
42
42
|
|
43
|
+
# Whether this header is acceptable in HTTP trailers.
|
44
|
+
# This is a base class for comma-separated headers, default is to disallow in trailers.
|
45
|
+
# @returns [Boolean] `false`, as most comma-separated headers should not appear in trailers by default.
|
46
|
+
def self.trailer?
|
47
|
+
false
|
48
|
+
end
|
49
|
+
|
43
50
|
protected
|
44
51
|
|
45
52
|
def reverse_find(&block)
|
@@ -0,0 +1,131 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Released under the MIT License.
|
4
|
+
# Copyright, 2025, by Samuel Williams.
|
5
|
+
|
6
|
+
require_relative "split"
|
7
|
+
require_relative "quoted_string"
|
8
|
+
require_relative "../error"
|
9
|
+
|
10
|
+
module Protocol
|
11
|
+
module HTTP
|
12
|
+
module Header
|
13
|
+
# The `te` header indicates the transfer encodings the client is willing to accept. AKA `accept-transfer-encoding`. How we ended up with `te` instead of `accept-transfer-encoding` is a mystery lost to time.
|
14
|
+
#
|
15
|
+
# The `te` header allows a client to indicate which transfer encodings it can handle, and in what order of preference using quality factors.
|
16
|
+
class TE < Split
|
17
|
+
ParseError = Class.new(Error)
|
18
|
+
|
19
|
+
# Transfer encoding token pattern
|
20
|
+
TOKEN = /[!#$%&'*+\-.0-9A-Z^_`a-z|~]+/
|
21
|
+
|
22
|
+
# Quality value pattern (0.0 to 1.0)
|
23
|
+
QVALUE = /0(\.[0-9]{0,3})?|1(\.[0]{0,3})?/
|
24
|
+
|
25
|
+
# Pattern for parsing transfer encoding with optional quality factor
|
26
|
+
TRANSFER_CODING = /\A(?<name>#{TOKEN})(\s*;\s*q=(?<q>#{QVALUE}))?\z/
|
27
|
+
|
28
|
+
# The `chunked` transfer encoding
|
29
|
+
CHUNKED = "chunked"
|
30
|
+
|
31
|
+
# The `gzip` transfer encoding
|
32
|
+
GZIP = "gzip"
|
33
|
+
|
34
|
+
# The `deflate` transfer encoding
|
35
|
+
DEFLATE = "deflate"
|
36
|
+
|
37
|
+
# The `compress` transfer encoding
|
38
|
+
COMPRESS = "compress"
|
39
|
+
|
40
|
+
# The `identity` transfer encoding
|
41
|
+
IDENTITY = "identity"
|
42
|
+
|
43
|
+
# The `trailers` pseudo-encoding indicates willingness to accept trailer fields
|
44
|
+
TRAILERS = "trailers"
|
45
|
+
|
46
|
+
# A single transfer coding entry with optional quality factor
|
47
|
+
TransferCoding = Struct.new(:name, :q) do
|
48
|
+
def quality_factor
|
49
|
+
(q || 1.0).to_f
|
50
|
+
end
|
51
|
+
|
52
|
+
def <=> other
|
53
|
+
other.quality_factor <=> self.quality_factor
|
54
|
+
end
|
55
|
+
|
56
|
+
def to_s
|
57
|
+
if q && q != 1.0
|
58
|
+
"#{name};q=#{q}"
|
59
|
+
else
|
60
|
+
name.to_s
|
61
|
+
end
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
65
|
+
# Initializes the TE header with the given value. The value is split into distinct entries and converted to lowercase for normalization.
|
66
|
+
#
|
67
|
+
# @parameter value [String | Nil] the raw header value containing transfer encodings separated by commas.
|
68
|
+
def initialize(value = nil)
|
69
|
+
super(value&.downcase)
|
70
|
+
end
|
71
|
+
|
72
|
+
# Adds one or more comma-separated values to the TE header. The values are converted to lowercase for normalization.
|
73
|
+
#
|
74
|
+
# @parameter value [String] the value or values to add, separated by commas.
|
75
|
+
def << value
|
76
|
+
super(value.downcase)
|
77
|
+
end
|
78
|
+
|
79
|
+
# Parse the `te` header value into a list of transfer codings with quality factors.
|
80
|
+
#
|
81
|
+
# @returns [Array(TransferCoding)] the list of transfer codings and their associated quality factors.
|
82
|
+
def transfer_codings
|
83
|
+
self.map do |value|
|
84
|
+
if match = value.match(TRANSFER_CODING)
|
85
|
+
TransferCoding.new(match[:name], match[:q])
|
86
|
+
else
|
87
|
+
raise ParseError.new("Could not parse transfer coding: #{value.inspect}")
|
88
|
+
end
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
# @returns [Boolean] whether the `chunked` encoding is accepted.
|
93
|
+
def chunked?
|
94
|
+
self.any? {|value| value.start_with?(CHUNKED)}
|
95
|
+
end
|
96
|
+
|
97
|
+
# @returns [Boolean] whether the `gzip` encoding is accepted.
|
98
|
+
def gzip?
|
99
|
+
self.any? {|value| value.start_with?(GZIP)}
|
100
|
+
end
|
101
|
+
|
102
|
+
# @returns [Boolean] whether the `deflate` encoding is accepted.
|
103
|
+
def deflate?
|
104
|
+
self.any? {|value| value.start_with?(DEFLATE)}
|
105
|
+
end
|
106
|
+
|
107
|
+
# @returns [Boolean] whether the `compress` encoding is accepted.
|
108
|
+
def compress?
|
109
|
+
self.any? {|value| value.start_with?(COMPRESS)}
|
110
|
+
end
|
111
|
+
|
112
|
+
# @returns [Boolean] whether the `identity` encoding is accepted.
|
113
|
+
def identity?
|
114
|
+
self.any? {|value| value.start_with?(IDENTITY)}
|
115
|
+
end
|
116
|
+
|
117
|
+
# @returns [Boolean] whether trailers are accepted.
|
118
|
+
def trailers?
|
119
|
+
self.any? {|value| value.start_with?(TRAILERS)}
|
120
|
+
end
|
121
|
+
|
122
|
+
# Whether this header is acceptable in HTTP trailers.
|
123
|
+
# TE headers negotiate transfer encodings and must not appear in trailers.
|
124
|
+
# @returns [Boolean] `false`, as TE headers are hop-by-hop and control message framing.
|
125
|
+
def self.trailer?
|
126
|
+
false
|
127
|
+
end
|
128
|
+
end
|
129
|
+
end
|
130
|
+
end
|
131
|
+
end
|
@@ -0,0 +1,23 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Released under the MIT License.
|
4
|
+
# Copyright, 2025, by Samuel Williams.
|
5
|
+
|
6
|
+
require_relative "split"
|
7
|
+
|
8
|
+
module Protocol
|
9
|
+
module HTTP
|
10
|
+
module Header
|
11
|
+
# Represents headers that can contain multiple distinct values separated by commas.
|
12
|
+
#
|
13
|
+
# This isn't a specific header class is a utility for handling headers with comma-separated values, such as `accept`, `cache-control`, and other similar headers. The values are split and stored as an array internally, and serialized back to a comma-separated string when needed.
|
14
|
+
class Trailer < Split
|
15
|
+
# Whether this header is acceptable in HTTP trailers.
|
16
|
+
# @returns [Boolean] `false`, as Trailer headers control trailer processing and must appear before the message body.
|
17
|
+
def self.trailer?
|
18
|
+
false
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
@@ -0,0 +1,78 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Released under the MIT License.
|
4
|
+
# Copyright, 2025, by Samuel Williams.
|
5
|
+
|
6
|
+
require_relative "split"
|
7
|
+
|
8
|
+
module Protocol
|
9
|
+
module HTTP
|
10
|
+
module Header
|
11
|
+
# The `transfer-encoding` header indicates the encoding transformations that have been applied to the message body.
|
12
|
+
#
|
13
|
+
# The `transfer-encoding` header is used to specify the form of encoding used to safely transfer the message body between the sender and receiver.
|
14
|
+
class TransferEncoding < Split
|
15
|
+
# The `chunked` transfer encoding allows a server to send data of unknown length by breaking it into chunks.
|
16
|
+
CHUNKED = "chunked"
|
17
|
+
|
18
|
+
# The `gzip` transfer encoding compresses the message body using the gzip algorithm.
|
19
|
+
GZIP = "gzip"
|
20
|
+
|
21
|
+
# The `deflate` transfer encoding compresses the message body using the deflate algorithm.
|
22
|
+
DEFLATE = "deflate"
|
23
|
+
|
24
|
+
# The `compress` transfer encoding compresses the message body using the compress algorithm.
|
25
|
+
COMPRESS = "compress"
|
26
|
+
|
27
|
+
# The `identity` transfer encoding indicates no transformation has been applied.
|
28
|
+
IDENTITY = "identity"
|
29
|
+
|
30
|
+
# Initializes the transfer encoding header with the given value. The value is split into distinct entries and converted to lowercase for normalization.
|
31
|
+
#
|
32
|
+
# @parameter value [String | Nil] the raw header value containing transfer encodings separated by commas.
|
33
|
+
def initialize(value = nil)
|
34
|
+
super(value&.downcase)
|
35
|
+
end
|
36
|
+
|
37
|
+
# Adds one or more comma-separated values to the transfer encoding header. The values are converted to lowercase for normalization.
|
38
|
+
#
|
39
|
+
# @parameter value [String] the value or values to add, separated by commas.
|
40
|
+
def << value
|
41
|
+
super(value.downcase)
|
42
|
+
end
|
43
|
+
|
44
|
+
# @returns [Boolean] whether the `chunked` encoding is present.
|
45
|
+
def chunked?
|
46
|
+
self.include?(CHUNKED)
|
47
|
+
end
|
48
|
+
|
49
|
+
# @returns [Boolean] whether the `gzip` encoding is present.
|
50
|
+
def gzip?
|
51
|
+
self.include?(GZIP)
|
52
|
+
end
|
53
|
+
|
54
|
+
# @returns [Boolean] whether the `deflate` encoding is present.
|
55
|
+
def deflate?
|
56
|
+
self.include?(DEFLATE)
|
57
|
+
end
|
58
|
+
|
59
|
+
# @returns [Boolean] whether the `compress` encoding is present.
|
60
|
+
def compress?
|
61
|
+
self.include?(COMPRESS)
|
62
|
+
end
|
63
|
+
|
64
|
+
# @returns [Boolean] whether the `identity` encoding is present.
|
65
|
+
def identity?
|
66
|
+
self.include?(IDENTITY)
|
67
|
+
end
|
68
|
+
|
69
|
+
# Whether this header is acceptable in HTTP trailers.
|
70
|
+
# Transfer-Encoding headers control message framing and must not appear in trailers.
|
71
|
+
# @returns [Boolean] `false`, as Transfer-Encoding headers are hop-by-hop and must precede the message body.
|
72
|
+
def self.trailer?
|
73
|
+
false
|
74
|
+
end
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|
78
|
+
end
|
@@ -17,11 +17,16 @@ require_relative "header/vary"
|
|
17
17
|
require_relative "header/authorization"
|
18
18
|
require_relative "header/date"
|
19
19
|
require_relative "header/priority"
|
20
|
+
require_relative "header/trailer"
|
21
|
+
require_relative "header/server_timing"
|
22
|
+
require_relative "header/digest"
|
20
23
|
|
21
24
|
require_relative "header/accept"
|
22
25
|
require_relative "header/accept_charset"
|
23
26
|
require_relative "header/accept_encoding"
|
24
27
|
require_relative "header/accept_language"
|
28
|
+
require_relative "header/transfer_encoding"
|
29
|
+
require_relative "header/te"
|
25
30
|
|
26
31
|
module Protocol
|
27
32
|
module HTTP
|
@@ -65,7 +70,7 @@ module Protocol
|
|
65
70
|
#
|
66
71
|
# @parameter fields [Array] An array of `[key, value]` pairs.
|
67
72
|
# @parameter tail [Integer | Nil] The index of the trailer start in the @fields array.
|
68
|
-
def initialize(fields = [], tail = nil, indexed: nil)
|
73
|
+
def initialize(fields = [], tail = nil, indexed: nil, policy: POLICY)
|
69
74
|
@fields = fields
|
70
75
|
|
71
76
|
# Marks where trailer start in the @fields array:
|
@@ -73,6 +78,21 @@ module Protocol
|
|
73
78
|
|
74
79
|
# The cached index of headers:
|
75
80
|
@indexed = nil
|
81
|
+
|
82
|
+
@policy = policy
|
83
|
+
end
|
84
|
+
|
85
|
+
# @attribute [Hash] The policy for the headers.
|
86
|
+
attr :policy
|
87
|
+
|
88
|
+
# Set the policy for the headers.
|
89
|
+
#
|
90
|
+
# The policy is used to determine how headers are merged and normalized. For example, if a header is specified multiple times, the policy will determine how the values are merged.
|
91
|
+
#
|
92
|
+
# @parameter policy [Hash] The policy for the headers.
|
93
|
+
def policy=(policy)
|
94
|
+
@policy = policy
|
95
|
+
@indexed = nil
|
76
96
|
end
|
77
97
|
|
78
98
|
# Initialize a copy of the headers.
|
@@ -250,17 +270,23 @@ module Protocol
|
|
250
270
|
"content-disposition" => false,
|
251
271
|
"content-length" => false,
|
252
272
|
"content-type" => false,
|
273
|
+
"expect" => false,
|
253
274
|
"from" => false,
|
254
275
|
"host" => false,
|
255
276
|
"location" => false,
|
256
277
|
"max-forwards" => false,
|
278
|
+
"range" => false,
|
257
279
|
"referer" => false,
|
258
280
|
"retry-after" => false,
|
281
|
+
"server" => false,
|
282
|
+
"transfer-encoding" => Header::TransferEncoding,
|
259
283
|
"user-agent" => false,
|
284
|
+
"trailer" => Header::Trailer,
|
260
285
|
|
261
286
|
# Custom headers:
|
262
287
|
"connection" => Header::Connection,
|
263
288
|
"cache-control" => Header::CacheControl,
|
289
|
+
"te" => Header::TE,
|
264
290
|
"vary" => Header::Vary,
|
265
291
|
"priority" => Header::Priority,
|
266
292
|
|
@@ -299,6 +325,12 @@ module Protocol
|
|
299
325
|
"accept-charset" => Header::AcceptCharset,
|
300
326
|
"accept-encoding" => Header::AcceptEncoding,
|
301
327
|
"accept-language" => Header::AcceptLanguage,
|
328
|
+
|
329
|
+
# Performance headers:
|
330
|
+
"server-timing" => Header::ServerTiming,
|
331
|
+
|
332
|
+
# Content integrity headers:
|
333
|
+
"digest" => Header::Digest,
|
302
334
|
}.tap{|hash| hash.default = Split}
|
303
335
|
|
304
336
|
# Delete all header values for the given key, and return the merged value.
|
@@ -316,7 +348,7 @@ module Protocol
|
|
316
348
|
|
317
349
|
if @indexed
|
318
350
|
return @indexed.delete(key)
|
319
|
-
elsif policy =
|
351
|
+
elsif policy = @policy[key]
|
320
352
|
(key, value), *tail = deleted
|
321
353
|
merged = policy.new(value)
|
322
354
|
|
@@ -334,14 +366,24 @@ module Protocol
|
|
334
366
|
# @parameter hash [Hash] The hash to merge into.
|
335
367
|
# @parameter key [String] The header key.
|
336
368
|
# @parameter value [String] The raw header value.
|
337
|
-
protected def merge_into(hash, key, value)
|
338
|
-
if policy =
|
369
|
+
protected def merge_into(hash, key, value, trailer = @tail)
|
370
|
+
if policy = @policy[key]
|
371
|
+
# Check if we're adding to trailers and this header is allowed:
|
372
|
+
if trailer && !policy.trailer?
|
373
|
+
return false
|
374
|
+
end
|
375
|
+
|
339
376
|
if current_value = hash[key]
|
340
377
|
current_value << value
|
341
378
|
else
|
342
379
|
hash[key] = policy.new(value)
|
343
380
|
end
|
344
381
|
else
|
382
|
+
# By default, headers are not allowed in trailers:
|
383
|
+
if trailer
|
384
|
+
return false
|
385
|
+
end
|
386
|
+
|
345
387
|
if hash.key?(key)
|
346
388
|
raise DuplicateHeaderError, key
|
347
389
|
end
|
@@ -362,11 +404,17 @@ module Protocol
|
|
362
404
|
#
|
363
405
|
# @returns [Hash] A hash table of `{key, value}` pairs.
|
364
406
|
def to_h
|
365
|
-
@indexed
|
366
|
-
|
407
|
+
unless @indexed
|
408
|
+
@indexed = {}
|
367
409
|
|
368
|
-
|
410
|
+
@fields.each_with_index do |(key, value), index|
|
411
|
+
trailer = (@tail && index >= @tail)
|
412
|
+
|
413
|
+
merge_into(@indexed, key.downcase, value, trailer)
|
414
|
+
end
|
369
415
|
end
|
416
|
+
|
417
|
+
return @indexed
|
370
418
|
end
|
371
419
|
|
372
420
|
alias as_json to_h
|
data/readme.md
CHANGED
@@ -18,6 +18,8 @@ Please see the [project documentation](https://socketry.github.io/protocol-http/
|
|
18
18
|
|
19
19
|
- [Message Body](https://socketry.github.io/protocol-http/guides/message-body/index) - This guide explains how to work with HTTP request and response message bodies using `Protocol::HTTP::Body` classes.
|
20
20
|
|
21
|
+
- [Headers](https://socketry.github.io/protocol-http/guides/headers/index) - This guide explains how to work with HTTP headers using `protocol-http`.
|
22
|
+
|
21
23
|
- [Middleware](https://socketry.github.io/protocol-http/guides/middleware/index) - This guide explains how to build and use HTTP middleware with `Protocol::HTTP::Middleware`.
|
22
24
|
|
23
25
|
- [Hypertext References](https://socketry.github.io/protocol-http/guides/hypertext-references/index) - This guide explains how to use `Protocol::HTTP::Reference` for constructing and manipulating hypertext references (URLs with parameters).
|
@@ -32,6 +34,11 @@ Please see the [project documentation](https://socketry.github.io/protocol-http/
|
|
32
34
|
|
33
35
|
Please see the [project releases](https://socketry.github.io/protocol-http/releases/index) for all releases.
|
34
36
|
|
37
|
+
### v0.54.0
|
38
|
+
|
39
|
+
- Introduce rich support for `Header::Digest`, `Header::ServerTiming`, `Header::TE`, `Header::Trailer` and `Header::TransferEncoding`.
|
40
|
+
- [Improved HTTP Trailer Security](https://socketry.github.io/protocol-http/releases/index#improved-http-trailer-security)
|
41
|
+
|
35
42
|
### v0.53.0
|
36
43
|
|
37
44
|
- Improve consistency of Body `#inspect`.
|
data/releases.md
CHANGED
@@ -1,5 +1,54 @@
|
|
1
1
|
# Releases
|
2
2
|
|
3
|
+
## v0.54.0
|
4
|
+
|
5
|
+
- Introduce rich support for `Header::Digest`, `Header::ServerTiming`, `Header::TE`, `Header::Trailer` and `Header::TransferEncoding`.
|
6
|
+
|
7
|
+
### Improved HTTP Trailer Security
|
8
|
+
|
9
|
+
This release introduces significant security improvements for HTTP trailer handling, addressing potential HTTP request smuggling vulnerabilities by implementing a restrictive-by-default policy for trailer headers.
|
10
|
+
|
11
|
+
- **Security-by-default**: HTTP trailers are now validated and restricted by default to prevent HTTP request smuggling attacks.
|
12
|
+
- Only safe headers are permitted in trailers:
|
13
|
+
- `date` - Response generation timestamps (safe metadata)
|
14
|
+
- `digest` - Content integrity verification (safe metadata)
|
15
|
+
- `etag` - Cache validation tags (safe metadata)
|
16
|
+
- `server-timing` - Performance metrics (safe metadata)
|
17
|
+
- All other trailers are ignored by default.
|
18
|
+
|
19
|
+
If you are using this library for gRPC, you will need to use a custom policy to allow the `grpc-status` and `grpc-message` trailers:
|
20
|
+
|
21
|
+
``` ruby
|
22
|
+
module GRPCStatus
|
23
|
+
def self.new(value)
|
24
|
+
Integer(value)
|
25
|
+
end
|
26
|
+
|
27
|
+
def self.trailer?
|
28
|
+
true
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
module GRPCMessage
|
33
|
+
def self.new(value)
|
34
|
+
value
|
35
|
+
end
|
36
|
+
|
37
|
+
def self.trailer?
|
38
|
+
true
|
39
|
+
end
|
40
|
+
end
|
41
|
+
|
42
|
+
GRPC_POLICY = Protocol::HTTP::Headers::POLICY.dup
|
43
|
+
GRPC_POLICY['grpc-status'] = GRPCStatus
|
44
|
+
GRPC_POLICY['grpc-message'] = GRPCMessage
|
45
|
+
|
46
|
+
# Reinterpret the headers using the new policy:
|
47
|
+
response.headers.policy = GRPC_POLICY
|
48
|
+
response.headers['grpc-status'] # => 0
|
49
|
+
response.headers['grpc-message'] # => "OK"
|
50
|
+
```
|
51
|
+
|
3
52
|
## v0.53.0
|
4
53
|
|
5
54
|
- Improve consistency of Body `#inspect`.
|
data.tar.gz.sig
CHANGED
Binary file
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: protocol-http
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.54.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Samuel Williams
|
@@ -55,6 +55,7 @@ extra_rdoc_files: []
|
|
55
55
|
files:
|
56
56
|
- context/design-overview.md
|
57
57
|
- context/getting-started.md
|
58
|
+
- context/headers.md
|
58
59
|
- context/hypertext-references.md
|
59
60
|
- context/index.yaml
|
60
61
|
- context/message-body.md
|
@@ -90,12 +91,17 @@ files:
|
|
90
91
|
- lib/protocol/http/header/connection.rb
|
91
92
|
- lib/protocol/http/header/cookie.rb
|
92
93
|
- lib/protocol/http/header/date.rb
|
94
|
+
- lib/protocol/http/header/digest.rb
|
93
95
|
- lib/protocol/http/header/etag.rb
|
94
96
|
- lib/protocol/http/header/etags.rb
|
95
97
|
- lib/protocol/http/header/multiple.rb
|
96
98
|
- lib/protocol/http/header/priority.rb
|
97
99
|
- lib/protocol/http/header/quoted_string.rb
|
100
|
+
- lib/protocol/http/header/server_timing.rb
|
98
101
|
- lib/protocol/http/header/split.rb
|
102
|
+
- lib/protocol/http/header/te.rb
|
103
|
+
- lib/protocol/http/header/trailer.rb
|
104
|
+
- lib/protocol/http/header/transfer_encoding.rb
|
99
105
|
- lib/protocol/http/header/vary.rb
|
100
106
|
- lib/protocol/http/headers.rb
|
101
107
|
- lib/protocol/http/methods.rb
|
metadata.gz.sig
CHANGED
Binary file
|