protocol-http 0.51.1 → 0.53.0

data/context/url-parsing.md

@@ -0,0 +1,130 @@
+# URL Parsing
+
+This guide explains how to use `Protocol::HTTP::URL` for parsing and manipulating URL components, particularly query strings and parameters.
+
+## Overview
+
+{ruby Protocol::HTTP::URL} provides utilities for parsing and manipulating URL components, particularly query strings and parameters. It offers robust encoding/decoding capabilities for complex parameter structures.
+
+While basic query parameter encoding follows the `application/x-www-form-urlencoded` standard, there is no universal standard for serializing complex nested structures (arrays, nested objects) in URLs. Different frameworks use varying conventions for these cases, and this implementation follows common patterns where possible.
+
+## Basic Query Parameter Parsing
+
+``` ruby
+require 'protocol/http/url'
+
+# Parse query parameters from a URL:
+reference = Protocol::HTTP::Reference.parse("/search?q=ruby&category=programming&page=2")
+parameters = Protocol::HTTP::URL.decode(reference.query)
+# => {"q" => "ruby", "category" => "programming", "page" => "2"}
+
+# Symbolize keys for easier access:
+parameters = Protocol::HTTP::URL.decode(reference.query, symbolize_keys: true)
+# => {:q => "ruby", :category => "programming", :page => "2"}
+```
+
+## Complex Parameter Structures
+
+The URL module handles nested parameters, arrays, and complex data structures:
+
+``` ruby
+# Array parameters:
+query = "tags[]=ruby&tags[]=programming&tags[]=web"
+parameters = Protocol::HTTP::URL.decode(query)
+# => {"tags" => ["ruby", "programming", "web"]}
+
+# Nested hash parameters:
+query = "user[name]=John&user[email]=john@example.com&user[preferences][theme]=dark"
+parameters = Protocol::HTTP::URL.decode(query)
+# => {"user" => {"name" => "John", "email" => "john@example.com", "preferences" => {"theme" => "dark"}}}
+
+# Mixed structures:
+query = "filters[categories][]=books&filters[categories][]=movies&filters[price][min]=10&filters[price][max]=100"
+parameters = Protocol::HTTP::URL.decode(query)
+# => {"filters" => {"categories" => ["books", "movies"], "price" => {"min" => "10", "max" => "100"}}}
+```
+
+## Encoding Parameters to Query Strings
+
+``` ruby
+# Simple parameters:
+parameters = {"search" => "protocol-http", "limit" => "20"}
+query = Protocol::HTTP::URL.encode(parameters)
+# => "search=protocol-http&limit=20"
+
+# Array parameters:
+parameters = {"tags" => ["ruby", "http", "protocol"]}
+query = Protocol::HTTP::URL.encode(parameters)
+# => "tags[]=ruby&tags[]=http&tags[]=protocol"
+
+# Nested parameters:
+parameters = {
+	user: {
+		profile: {
+			name: "Alice",
+			settings: {
+				notifications: true,
+				theme: "light"
+			}
+		}
+	}
+}
+query = Protocol::HTTP::URL.encode(parameters)
+# => "user[profile][name]=Alice&user[profile][settings][notifications]=true&user[profile][settings][theme]=light"
+```
+
+## URL Escaping and Unescaping
+
+``` ruby
+# Escape special characters:
+Protocol::HTTP::URL.escape("hello world!")
+# => "hello%20world%21"
+
+# Escape path components (preserves path separators):
+Protocol::HTTP::URL.escape_path("/path/with spaces/file.html")
+# => "/path/with%20spaces/file.html"
+
+# Unescape percent-encoded strings:
+Protocol::HTTP::URL.unescape("hello%20world%21")
+# => "hello world!"
+
+# Handle Unicode characters:
+Protocol::HTTP::URL.escape("café")
+# => "caf%C3%A9"
+
+Protocol::HTTP::URL.unescape("caf%C3%A9")
+# => "café"
+```
+
+## Scanning and Processing Query Strings
+
+For custom processing, you can scan query strings directly:
+
+``` ruby
+query = "name=John&age=30&active=true"
+
+Protocol::HTTP::URL.scan(query) do |key, value|
+	puts "#{key}: #{value}"
+end
+# Output:
+# name: John
+# age: 30
+# active: true
+```
+
+## Security and Limits
+
+The URL module includes built-in protection against deeply nested parameter attacks:
+
+``` ruby
+# This will raise an error to prevent excessive nesting:
+begin
+	Protocol::HTTP::URL.decode("a[b][c][d][e][f][g][h][i]=value")
+rescue ArgumentError => error
+	puts error.message
+	# => "Key length exceeded limit!"
+end
+
+# You can adjust the maximum nesting level:
+Protocol::HTTP::URL.decode("a[b][c]=value", 5)  # Allow up to 5 levels of nesting
+```

data/lib/protocol/http/accept_encoding.rb

@@ -21,7 +21,7 @@ module Protocol
 			# The default wrappers to use for decoding content.
 			DEFAULT_WRAPPERS = {
 				"gzip" => Body::Inflate.method(:for),
-				"identity" => ->(body) { body }, # Identity means no encoding
+				"identity" => ->(body) {body}, # Identity means no encoding
 				
 				# There is no point including this:
 				# 'identity' => ->(body){body},

data/lib/protocol/http/body/buffered.rb

@@ -153,8 +153,10 @@ module Protocol
 				#
 				# @returns [String] a string representation of the buffered body.
 				def inspect
-					if @chunks
-						"\#<#{self.class} #{@chunks.size} chunks, #{self.length} bytes>"
+					if @chunks and @chunks.size > 0
+						"#<#{self.class} #{@index}/#{@chunks.size} chunks, #{self.length} bytes>"
+					else
+						"#<#{self.class} empty>"
 					end
 				end
 			end

data/lib/protocol/http/body/completable.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2019-2024, by Samuel Williams.
+# Copyright, 2019-2025, by Samuel Williams.
 
 require_relative "wrapper"
 
@@ -53,6 +53,23 @@ module Protocol
 					
 					super
 				end
+				
+				# Convert the body to a hash suitable for serialization.
+				#
+				# @returns [Hash] The body as a hash.
+				def as_json(...)
+					super.merge(
+						callback: @callback&.to_s
+					)
+				end
+				
+				# Inspect the completable body.
+				#
+				# @returns [String] a string representation of the completable body.
+				def inspect
+					callback_status = @callback ? "callback pending" : "callback completed"
+					return "#{super} | #<#{self.class} #{callback_status}>"
+				end
 			end
 		end
 	end

data/lib/protocol/http/body/deflate.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2019-2024, by Samuel Williams.
+# Copyright, 2019-2025, by Samuel Williams.
 
 require_relative "wrapper"
 
@@ -75,11 +75,22 @@ module Protocol
 					end
 				end
 				
+				# Convert the body to a hash suitable for serialization.
+				#
+				# @returns [Hash] The body as a hash.
+				def as_json(...)
+					super.merge(
+						input_length: @input_length,
+						output_length: @output_length,
+						compression_ratio: (ratio * 100).round(2)
+					)
+				end
+				
 				# Inspect the body, including the compression ratio.
 				#
 				# @returns [String] a string representation of the body.
 				def inspect
-					"#{super} | \#<#{self.class} #{(ratio*100).round(2)}%>"
+					"#{super} | #<#{self.class} #{(ratio*100).round(2)}%>"
 				end
 			end
 			

data/lib/protocol/http/body/digestable.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2020-2024, by Samuel Williams.
+# Copyright, 2020-2025, by Samuel Williams.
 
 require_relative "wrapper"
 
@@ -34,7 +34,7 @@ module Protocol
 					@digest = digest
 					@callback = callback
 				end
-
+				
 				# @attribute [Digest] digest the digest object.
 				attr :digest
 				
@@ -64,6 +64,16 @@ module Protocol
 						return nil
 					end
 				end
+				
+				# Convert the body to a hash suitable for serialization.
+				#
+				# @returns [Hash] The body as a hash.
+				def as_json(...)
+					super.merge(
+						digest_class: @digest.class.name,
+						callback: @callback&.to_s
+					)
+				end
 			end
 		end
 	end

data/lib/protocol/http/body/file.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2019-2024, by Samuel Williams.
+# Copyright, 2019-2025, by Samuel Williams.
 
 require_relative "readable"
 
@@ -135,7 +135,11 @@ module Protocol
 				#
 				# @returns [String] a string representation of the file body.
 				def inspect
-					"\#<#{self.class} file=#{@file.inspect} offset=#{@offset} remaining=#{@remaining}>"
+					if @offset > 0
+						"#<#{self.class} #{@file.inspect} +#{@offset}, #{@remaining} bytes remaining>"
+					else
+						"#<#{self.class} #{@file.inspect}, #{@remaining} bytes remaining>"
+					end
 				end
 			end
 		end

data/lib/protocol/http/body/head.rb

@@ -53,6 +53,13 @@ module Protocol
 				def length
 					@length
 				end
+				
+				# Inspect the head body.
+				#
+				# @returns [String] a string representation of the head body.
+				def inspect
+					"#<#{self.class} #{@length} bytes (empty)>"
+				end
 			end
 		end
 	end

data/lib/protocol/http/body/inflate.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2019-2024, by Samuel Williams.
+# Copyright, 2019-2025, by Samuel Williams.
 
 require "zlib"
 
@@ -34,7 +34,7 @@ module Protocol
 							
 							break unless chunk&.empty?
 						end
-					
+						
 						if chunk
 							@output_length += chunk.bytesize
 						elsif !stream.closed?

data/lib/protocol/http/body/rewindable.rb

@@ -82,11 +82,21 @@ module Protocol
 					true
 				end
 				
+				# Convert the body to a hash suitable for serialization.
+				#
+				# @returns [Hash] The body as a hash.
+				def as_json(...)
+					super.merge(
+						index: @index,
+						chunks: @chunks.size
+					)
+				end
+				
 				# Inspect the rewindable body.
 				#
 				# @returns [String] a string representation of the body.
 				def inspect
-					"\#<#{self.class} #{@index}/#{@chunks.size} chunks read>"
+					"#{super} | #<#{self.class} #{@index}/#{@chunks.size} chunks read>"
 				end
 			end
 		end

data/lib/protocol/http/body/stream.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2019-2024, by Samuel Williams.
+# Copyright, 2019-2025, by Samuel Williams.
 # Copyright, 2023, by Genki Takiuchi.
 
 require_relative "buffered"
@@ -123,7 +123,7 @@ module Protocol
 							if buffer.bytesize > length
 								# This ensures the subsequent `slice!` works correctly.
 								buffer.force_encoding(Encoding::BINARY)
-
+								
 								@buffer = buffer.byteslice(length, buffer.bytesize)
 								buffer.slice!(length, buffer.bytesize)
 							end
@@ -386,6 +386,21 @@ module Protocol
 					@closed
 				end
 				
+				# Inspect the stream.
+				#
+				# @returns [String] a string representation of the stream.
+				def inspect
+					buffer_info = @buffer ? "#{@buffer.bytesize} bytes buffered" : "no buffer"
+					
+					status = []
+					status << "closed" if @closed
+					status << "read-closed" if @closed_read
+					
+					status_info = status.empty? ? "open" : status.join(", ")
+					
+					return "#<#{self.class} #{buffer_info}, #{status_info}>"
+				end
+				
 				# @returns [Boolean] Whether there are any output chunks remaining.
 				def empty?
 					@output.empty?

data/lib/protocol/http/body/streamable.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2019-2024, by Samuel Williams.
+# Copyright, 2019-2025, by Samuel Williams.
 
 require_relative "readable"
 require_relative "writable"
@@ -147,7 +147,23 @@ module Protocol
 					#
 					# @parameter error [Exception | Nil] The error that caused this stream to be closed, if any.
 					def close_output(error = nil)
-						@output&.close(error)
+						if output = @output
+							@output = nil
+							output.close(error)
+						end
+					end
+					
+					# Inspect the streaming body.
+					#
+					# @returns [String] a string representation of the streaming body.
+					def inspect
+						if @block
+							"#<#{self.class} block available, not consumed>"
+						elsif @output
+							"#<#{self.class} block consumed, output active>"
+						else
+							"#<#{self.class} block consumed, output closed>"
+						end
 					end
 				end
 				

data/lib/protocol/http/body/writable.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2024, by Samuel Williams.
+# Copyright, 2024-2025, by Samuel Williams.
 
 require_relative "readable"
 
@@ -166,9 +166,9 @@ module Protocol
 				# @returns [String] A string representation of the body.
 				def inspect
 					if @error
-						"\#<#{self.class} #{@count} chunks written, #{status}, error=#{@error}>"
+						"#<#{self.class} #{@count} chunks written, #{status}, error=#{@error}>"
 					else
-						"\#<#{self.class} #{@count} chunks written, #{status}>"
+						"#<#{self.class} #{@count} chunks written, #{status}>"
 					end
 				end
 				

data/lib/protocol/http/error.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2018-2023, by Samuel Williams.
+# Copyright, 2018-2025, by Samuel Williams.
 
 module Protocol
 	module HTTP

data/lib/protocol/http/header/cache_control.rb

@@ -120,7 +120,7 @@ module Protocol
 				# @parameter value_name [String] the directive name to search for (e.g., "max-age").
 				# @returns [Integer | Nil] the parsed integer value, or `nil` if not found or invalid.
 				def find_integer_value(value_name)
-					if value = self.find { |value| value.start_with?(value_name) }
+					if value = self.find{|value| value.start_with?(value_name)}
 						_, age = value.split("=", 2)
 						
 						if age =~ /\A[0-9]+\z/

data/lib/protocol/http/headers.rb

@@ -64,13 +64,15 @@ module Protocol
 			# Initialize the headers with the specified fields.
 			#
 			# @parameter fields [Array] An array of `[key, value]` pairs.
-			# @parameter indexed [Hash] A hash table of normalized headers, if available.
-			def initialize(fields = [], indexed = nil)
+			# @parameter tail [Integer | Nil] The index of the trailer start in the @fields array.
+			def initialize(fields = [], tail = nil, indexed: nil)
 				@fields = fields
-				@indexed = indexed
 				
-				# Marks where trailer start in the @fields array.
-				@tail = nil
+				# Marks where trailer start in the @fields array:
+				@tail = tail
+				
+				# The cached index of headers:
+				@indexed = nil
 			end
 			
 			# Initialize a copy of the headers.
@@ -86,8 +88,8 @@ module Protocol
 			# Clear all headers.
 			def clear
 				@fields.clear
-				@indexed = nil
 				@tail = nil
+				@indexed = nil
 			end
 			
 			# Flatten trailer into the headers, in-place.
@@ -108,6 +110,14 @@ module Protocol
 			# @attribute [Array] An array of `[key, value]` pairs.
 			attr :fields
 			
+			# @attribute [Integer | Nil] The index where trailers begin.
+			attr :tail
+			
+			# @returns [Array] The fields of the headers.
+			def to_a
+				@fields
+			end
+			
 			# @returns [Boolean] Whether there are any trailers.
 			def trailer?
 				@tail != nil

data/lib/protocol/http/reference.rb

@@ -142,24 +142,38 @@ module Protocol
 			end
 			
 			# Update the reference with the given path, parameters and fragment.
-			# @argument path [String] Append the string to this reference similar to `File.join`.
-			# @argument parameters [Hash] Append the parameters to this reference.
-			# @argument fragment [String] Set the fragment to this value.
-			# @argument pop [Boolean] If the path contains a trailing filename, pop the last component of the path before appending the new path.
-			# @argument merge [Boolean] If the parameters are specified, merge them with the existing parameters.
-			def with(path: nil, parameters: nil, fragment: @fragment, pop: false, merge: true)
-				if @parameters
-					if parameters and merge
-						parameters = @parameters.merge(parameters)
-					else
+			#
+			# @parameter path [String] Append the string to this reference similar to `File.join`.
+			# @parameter parameters [Hash] Append the parameters to this reference.
+			# @parameter fragment [String] Set the fragment to this value.
+			# @parameter pop [Boolean] If the path contains a trailing filename, pop the last component of the path before appending the new path.
+			# @parameter merge [Boolean] If the parameters are specified, merge them with the existing parameters, otherwise replace them (including query string).
+			def with(path: nil, parameters: false, fragment: @fragment, pop: false, merge: true)
+				if merge
+					# Merge mode: combine new parameters with existing, keep query:
+					# parameters = (@parameters || {}).merge(parameters || {})
+					if @parameters
+						if parameters
+							parameters = @parameters.merge(parameters)
+						else
+							parameters = @parameters
+						end
+					elsif !parameters
 						parameters = @parameters
 					end
-				end
-				
-				if @query and !merge
-					query = nil
-				else
+					
 					query = @query
+				else
+					# Replace mode: use new parameters if provided, clear query when replacing:
+					if parameters == false
+						# No new parameters provided, keep existing:
+						parameters = @parameters
+						query = @query
+					else
+						# New parameters provided, replace and clear query:
+						# parameters = parameters
+						query = nil
+					end
 				end
 				
 				if path

data/lib/protocol/http/request.rb

@@ -68,7 +68,7 @@ module Protocol
 			
 			# @attribute [Body::Readable] the request body. It should only be read once (it may not be idempotent).
 			attr_accessor :body
-
+			
 			# @attribute [String | Array(String) | Nil] the request protocol, usually empty, but occasionally `"websocket"` or `"webtransport"`. In HTTP/1, it is used to request a connection upgrade, and in HTTP/2 it is used to indicate a specfic protocol for the stream.
 			attr_accessor :protocol
 			

data/lib/protocol/http/response.rb

@@ -1,10 +1,11 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2019-2024, by Samuel Williams.
+# Copyright, 2019-2025, by Samuel Williams.
 
 require_relative "body/buffered"
 require_relative "body/reader"
+require_relative "headers"
 
 module Protocol
 	module HTTP

data/lib/protocol/http/version.rb

@@ -5,6 +5,6 @@
 
 module Protocol
 	module HTTP
-		VERSION = "0.51.1"
+		VERSION = "0.53.0"
 	end
 end

data/readme.md

@@ -14,16 +14,35 @@ Provides abstractions for working with the HTTP protocol.
 
 Please see the [project documentation](https://socketry.github.io/protocol-http/) for more details.
 
-  - [Streaming](https://socketry.github.io/protocol-http/guides/streaming/index) - This guide gives an overview of how to implement streaming requests and responses.
-
   - [Getting Started](https://socketry.github.io/protocol-http/guides/getting-started/index) - This guide explains how to use `protocol-http` for building abstract HTTP interfaces.
 
+  - [Message Body](https://socketry.github.io/protocol-http/guides/message-body/index) - This guide explains how to work with HTTP request and response message bodies using `Protocol::HTTP::Body` classes.
+
+  - [Middleware](https://socketry.github.io/protocol-http/guides/middleware/index) - This guide explains how to build and use HTTP middleware with `Protocol::HTTP::Middleware`.
+
+  - [Hypertext References](https://socketry.github.io/protocol-http/guides/hypertext-references/index) - This guide explains how to use `Protocol::HTTP::Reference` for constructing and manipulating hypertext references (URLs with parameters).
+
+  - [URL Parsing](https://socketry.github.io/protocol-http/guides/url-parsing/index) - This guide explains how to use `Protocol::HTTP::URL` for parsing and manipulating URL components, particularly query strings and parameters.
+
+  - [Streaming](https://socketry.github.io/protocol-http/guides/streaming/index) - This guide gives an overview of how to implement streaming requests and responses.
+
   - [Design Overview](https://socketry.github.io/protocol-http/guides/design-overview/index) - This guide explains the high level design of `protocol-http` in the context of wider design patterns that can be used to implement HTTP clients and servers.
 
 ## Releases
 
 Please see the [project releases](https://socketry.github.io/protocol-http/releases/index) for all releases.
 
+### v0.53.0
+
+  - Improve consistency of Body `#inspect`.
+  - Improve `as_json` support for Body wrappers.
+
+### v0.52.0
+
+  - Add `Protocol::HTTP::Headers#to_a` method that returns the fields array, providing compatibility with standard Ruby array conversion pattern.
+  - Expose `tail` in `Headers.new` so that trailers can be accurately reproduced.
+  - Add agent context.
+
 ### v0.51.0
 
   - `Protocol::HTTP::Headers` now raise a `DuplicateHeaderError` when a duplicate singleton header (e.g. `content-length`) is added.

data/releases.md

@@ -1,5 +1,16 @@
 # Releases
 
+## v0.53.0
+
+  - Improve consistency of Body `#inspect`.
+  - Improve `as_json` support for Body wrappers.
+
+## v0.52.0
+
+  - Add `Protocol::HTTP::Headers#to_a` method that returns the fields array, providing compatibility with standard Ruby array conversion pattern.
+  - Expose `tail` in `Headers.new` so that trailers can be accurately reproduced.
+  - Add agent context.
+
 ## v0.51.0
 
   - `Protocol::HTTP::Headers` now raise a `DuplicateHeaderError` when a duplicate singleton header (e.g. `content-length`) is added.

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: protocol-http
 version: !ruby/object:Gem::Version
-  version: 0.51.1
+  version: 0.53.0
 platform: ruby
 authors:
 - Samuel Williams
@@ -53,6 +53,14 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- context/design-overview.md
+- context/getting-started.md
+- context/hypertext-references.md
+- context/index.yaml
+- context/message-body.md
+- context/middleware.md
+- context/streaming.md
+- context/url-parsing.md
 - lib/protocol/http.rb
 - lib/protocol/http/accept_encoding.rb
 - lib/protocol/http/body.rb