patron 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8f1026343c941740b978f220122296a7accd4de8770b18b317cdb6310e86dd5e
-  data.tar.gz: 0b90aed0ce0f16d4ae6b66ad0f44131b69e65b06d9e0e5e516d18c209b464160
+  metadata.gz: 9fe1ac5162e7ee75918ceb165caf4c9472c55a8f7261e6dacb54855f1c037799
+  data.tar.gz: 339a2034c51711350be141b6a6812afa220e5209dd2449b84e7a84a9e4051b5a
 SHA512:
-  metadata.gz: cce780fab9bb4444992709dc18e2222bcb6e25fa03987b903270347e0a651080fe9024fa848da519b70a1511f429a4199f7e62425f554e4fb12b3bce198e31e7
-  data.tar.gz: d49a557d56dc10e57bc362f1af616eb779333da820c2899c4bb958347317d3bcc6eb171f8090ca03c2aae9ed94c5a53039e5c455206bb1372d743f6d029b97c3
+  metadata.gz: 3f758953f5bfd94011887a9b824d30e996c99ecb263fc96df2f2955ccda81b88a5fb588b1533acc8bea386ab9f5f30a50fb922dc658ba65d69abbdeb57fbef08
+  data.tar.gz: 9d1ee100c09241ace6d0879ece310d26a74214d929541858f8877abf31ab61c11a770af93d09bd4e36e315eb16283284772cd37a991cd90b819b52989b35eecf

data/CHANGELOG.md

@@ -1,5 +1,48 @@
 ### Unreleased
 
+### 1.1.0
+
+* `Session#stream(action, url, headers, **options) { |stream| }` and the `get_stream`, `put_stream`,
+  `post_stream` (a Hash is form-encoded like `post`) and `patch_stream` helpers yield the response as soon
+  as its head has arrived, and return a `Patron::TransferStats` once the block is done. The body is received
+  by reading from the stream, which drives the transfer in the calling thread or fiber, so a Fiber
+  scheduler, `Session#interrupt`, `Timeout` and `Thread#raise` all work mid-body
+* `Patron::ResponseStream` has the head (`status`, `headers`, `content_length`, `content_type`, ...) and the
+  IO-like readers `read`, `readpartial`, `each`, `each_line`, `drain!`, `eof?` and `close`. `read` and
+  `readpartial` fill a String you pass in place, so `IO.copy_stream(stream, destination)` does not allocate
+  per chunk. Leaving the block early closes the connection; `drain!` keeps it
+* `Patron::TransferStats`, the receipt: the same head readers plus `downloaded_bytes`, `uploaded_bytes`,
+  `timings`, `trailers` and `complete?`. Frozen and Ractor-shareable
+* `Session#read_timeout` / `read_timeout:` bounds the wait for the next piece of a response, for streamed and
+  buffered requests alike, raising `Patron::TimeoutError`. Unlike `timeout` it does not bound the whole transfer
+* `Response` gains `content_length` and `content_type` (the media type without parameters)
+* `get_file` (and `request(:get, ..., file: path)`) is built on `stream` and `IO.copy_stream`, and returns the
+  same `Response` with a nil body as before. A `Request` with a `file_name` handed to `handle_request` as a GET
+  raises `ArgumentError` now
+* `Patron::SSEClient`, a server-sent events client built on `stream`: WHATWG event stream parsing, reconnects
+  after the server's `retry:` interval with `Last-Event-ID`, an idle timeout, `close` from another thread or
+  fiber, `UnexpectedResponse` for anything that is not an event stream and `InvalidEncoding` for an event that
+  is not UTF-8. `Patron::SSEClient.parse(readable) { |event| }` runs the parser over any stream, such as the
+  body of a POST inside a `post_stream` block. Closes #158
+* Under a Fiber scheduler, `Session#interrupt` and `SSEClient#close` called from another fiber on the same
+  thread take effect at once: `Patron::Aborted` is raised into the waiting fiber through the scheduler, the way
+  `Async::Task#stop` does it. They used to land within a second
+* Under a Fiber scheduler on Linux, a request right after an aborted one on the same Session could time out
+  connecting: the new socket got the old descriptor number and the same IO wrapper, which io-event's epoll
+  selector took for still registered. The wrappers are now made afresh for every request
+* webmock's adapter overrides `handle_request` and does not intercept streams, nor `get_file` now
+* `Response#charset` and `#decoded_body` find the charset in a lowercase `content-type` header too, as sent by HTTP/2 servers and Puma 8
+* Test suite: Puma 8, Rack 3 lowercase header names in the test app
+* A request body that responds to `read` (a StringIO, an `IO.pipe` reader, a `Zlib::GzipReader`) is streamed
+  to the server through libcurl's read callback, sized from its `size`, from a `Content-Length` header, or sent
+  chunked. It is rewound with `seek` when libcurl has to send it again on a 307/308 redirect, and a body that
+  cannot be rewound raises `Patron::Error` with `curl_code` 65 instead of being silently sent empty.
+  It is read with `readpartial` when it has one, so a pipe or a socket goes out as the writer writes to it
+  rather than in 64KB pieces
+* `put_file`, `post_file` and File bodies are sent again when a 307/308 redirect or an auth round-trip needs
+  the body once more; such a request used to fail with "necessary data rewind wasn't possible"
+* A StringIO request body is sent as its contents; it used to be sent as its `inspect` string
+
 ### 1.0.0
 
 Patron 1.0 is a rewrite of the C extension for modern Ruby and modern libcurl. It follows Ivo Anjo's

data/README.md

@@ -76,6 +76,21 @@ You can ship custom headers with a single request:
 sess.post("/foo/stuff", "some data", {"Content-Type" => "text/plain"})
 ```
 
+A request body can be a String, a File (uploaded straight from disk by libcurl) or anything that
+responds to `readpartial` or `read`, which is streamed. With `readpartial` a pipe or a socket goes out
+as soon as something is written to it. A body without a `size` goes out with chunked transfer-encoding,
+unless you pass a `Content-Length` header:
+
+```ruby
+reader, writer = IO.pipe
+Thread.new { 3.times { |i| writer.write("line #{i}\n") }; writer.close }
+sess.put("/foo/log", reader)
+sess.post("/foo/stuff", StringIO.new("some data"))
+```
+
+If the body has to be sent again (a 307 redirect, say) Patron rewinds it with `seek`,
+and a body that can't be rewound, like a pipe, fails the request with a `Patron::Error`.
+
 libcurl's transfer timings are available on the response, in seconds since the request started:
 
 ```ruby
@@ -105,6 +120,88 @@ This depends on the TLS library libcurl is built against: with GnuTLS these surf
 Client certificates are tested with the OpenSSL and LibreSSL backends - libcurl's Secure Transport backend
 can't load a PEM certificate from a file at all.
 
+## Streaming responses
+
+`Session#stream` (and `get_stream`, `post_stream`, `put_stream`, `patch_stream`) yields the response as
+soon as its head has arrived, and the block receives the body by reading from it. Nothing is buffered
+meanwhile: every read drives libcurl until some bytes have arrived. `timeout` still bounds the whole
+transfer, so set it to 0 and use `read_timeout` to bound the wait for each next piece:
+
+```ruby
+stats = sess.get_stream("/events", { "Accept" => "text/event-stream" }, timeout: 0, read_timeout: 30) do |stream|
+  raise "Unexpected #{stream.status}" unless stream.ok? && stream.content_type == "text/event-stream"
+  stream.each_line do |line|
+    puts line.force_encoding(Encoding::UTF_8)
+  end
+end
+stats.complete?  # => true when the body was received to its end
+stats.timings    # the same Hash as Response#timings
+stats.trailers   # chunked or HTTP/2 trailers, {} when none
+```
+
+`read(length, buffer)` and `readpartial(length, buffer)` fill a String you own, so copying a body
+somewhere allocates nothing per chunk. `IO.copy_stream` uses exactly that, into a File or anything
+with `write`:
+
+```ruby
+sess.get_stream("/large.bin") do |stream|
+  raise "Too big" if stream.content_length.to_i > 1024 * 1024 * 1024
+  File.open("large.bin", "wb") { |file| IO.copy_stream(stream, file) }
+end
+```
+
+Leaving the block early (`break`, `return`, an exception) aborts the transfer and closes its connection;
+call `stream.drain!` first to receive and discard the rest and keep the connection. Using the stream after
+its block raises `IOError`. The block is mandatory, and the `Session` is busy while it runs. Under a Fiber
+scheduler use `each` and `each_line` with a block rather than an external `Enumerator`.
+
+### Server-sent events
+
+`Patron::SSEClient` reads a `text/event-stream` on top of `stream`, and reconnects the way a browser's
+EventSource does: after the retry interval (the server's `retry:` field wins), sending `Last-Event-ID`.
+
+```ruby
+client = Patron::SSEClient.new(
+  "https://example.com/stream",
+  headers: { "Authorization" => "Bearer ..." },
+  idle_timeout: 45,     # seconds of silence before reconnecting
+  retry: 3,             # seconds between reconnects until the server sends a retry field
+  max_reconnects: nil,  # nil: forever
+  last_event_id: nil,   # resume from a stored id
+  session: nil          # or a configured Patron::Session: TLS, proxy, auth, cookies, base_url
+)
+
+client.each_event do |event|
+  event.type  # "message" unless the server sent an event: field
+  event.data  # data: lines joined with "\n", UTF-8
+  event.id
+  break if event.type == "done"
+end
+client.last_event_id  # to persist between process runs
+```
+
+A 204 ends `each_event`. Any other status than 200, or another Content-Type, raises
+`Patron::SSEClient::UnexpectedResponse` with the `status`, `headers` and first 4 KB of `body`, since retrying
+a 503 is the application's call. An event that is not valid UTF-8 raises `Patron::SSEClient::InvalidEncoding`
+and ends the stream, where a browser would substitute U+FFFD. `client.close` from another thread or fiber makes
+`each_event` return.
+
+The client only issues GETs, like an `EventSource`. For an event stream that answers a POST, which is how the
+LLM completion APIs stream, run the same parser over the body yourself:
+
+```ruby
+session.post_stream("/v1/chat/completions", body.to_json, headers) do |stream|
+  raise "status #{stream.status}" unless stream.status == 200
+  Patron::SSEClient.parse(stream) do |event|
+    break if event.data == "[DONE]"
+    print JSON.parse(event.data).dig("choices", 0, "delta", "content")
+  end
+end
+```
+
+`Patron::SSEClient.parse` takes anything with `readpartial`, so a file or a socket works as well as a
+`ResponseStream`, and returns the last event ID it saw.
+
 ## Threading
 
 A `Patron::Session` performs one request at a time and must not be shared between threads: each `Session`
@@ -150,7 +247,8 @@ end
 ```
 
 Each fiber needs its own `Session`, just like each thread does. `Async::Task#stop` and
-`Task#with_timeout` interrupt a request in flight the same way `Thread#kill` and `Timeout` do.
+`Task#with_timeout` interrupt a request in flight the same way `Thread#kill` and `Timeout` do, and
+`Session#interrupt` and `SSEClient#close` from another fiber reach the waiting one at once.
 
 ## Ractors