fast_curl 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 00f4a847ee360180fa617212693206970a8e7af85ef02e6370d37cb62d2b384e
-  data.tar.gz: f5633dc59f41b4b31efa6e07eb32bc6a85b146e85126fa68b24b742294df5e8b
+  metadata.gz: 32aca3a9aec734c69bc40aa9b84b519aeddc15defc3ad3a9e8f278f63d9f0ffb
+  data.tar.gz: 8037377b3d3e5ab02f4171c3f4ae56e6f42552a6cb5c6d7e2f5fd54921e2cc71
 SHA512:
-  metadata.gz: b7f2e299c89da40d0a546e8545310430a46f0cca8abafff53e46953871c4c205d8fc53c16e33e65b9ff3a22c951d212f41f562476977200de84136ce0342cacc
-  data.tar.gz: a5fa06d7e44c0846de3cccd3c955a136fd7ee05e01611f05b940bf174cbd356315d23418ebc40ea77d94667c928ae3f2c7087c227df0ea7a39c0b3a27f52da77
+  metadata.gz: 670eefe8b9fa597e947ae0858bda833d35d15914fbd62e3093483cdf8c2c68560c121586c4b01eec75f81d3a4f6d49d04829d5711180178247f051cb510c8cd6
+  data.tar.gz: 709792e880465cc09de983246936ce66a843f5b7899e69ac6f5a390cef35b4cdc8baaf1dbda36abe39627cc2fe457e994076329ab746887d000d1a089e649af7

data/README.md

@@ -12,9 +12,9 @@ Ultra-fast parallel HTTP client for Ruby. C extension built on libcurl `curl_mul
 
 ## Installation
 
-**Requirements**: Ruby >= 3.1, libcurl
+**Requirements**: Ruby >= 2.7, libcurl
 
-> **Why Ruby 3.1?** The C extension uses `rb_fiber_scheduler_current`, `rb_fiber_scheduler_block` and `rb_fiber_scheduler_unblock` to properly yield control to the Fiber Scheduler during I/O. These APIs are stable starting from Ruby 3.1. Without them, there is no correct way for a C extension to cooperate with the scheduler — earlier approaches (`rb_thread_schedule`) hold the GVL and block other fibers.
+> **Fiber Scheduler support requires Ruby >= 3.1.** The C extension uses `rb_fiber_scheduler_current`, `rb_fiber_scheduler_block` and `rb_fiber_scheduler_unblock` to yield control to the Fiber Scheduler during I/O; these APIs are stable from Ruby 3.1. On 2.7 and 3.0 the extension builds and runs correctly, but that code is compiled out — so a request made inside a scheduler blocks the whole thread and **no sibling fiber runs until it finishes**. Other OS threads are unaffected, since the GVL is still released. If you use `async`, use Ruby >= 3.1.
 
 ```ruby
 gem 'fast_curl'
@@ -50,16 +50,37 @@ end
 
 ### POST with body and headers
 
+Be explicit about the encoding — `json:` and `form:` set the matching
+`Content-Type` for you:
+
 ```ruby
 FastCurl.post([
   {
     url: "https://api.example.com/users",
     headers: { "Authorization" => "Bearer token" },
-    body: { name: "John" }
+    json: { name: "John" }               # application/json
+  },
+  {
+    url: "https://api.example.com/login",
+    form: { user: "john", pass: "x" }    # application/x-www-form-urlencoded
+  },
+  {
+    url: "https://api.example.com/blob",
+    headers: { "Content-Type" => "application/xml" },
+    body: "<user/>"                      # sent as-is
   }
 ])
 ```
 
+A raw String `body:` without an explicit `Content-Type` is sent as
+`application/octet-stream`. A Hash `body:` is still encoded as JSON.
+
+Query parameters can be passed separately:
+
+```ruby
+FastCurl.get([{ url: "https://api.example.com/search", params: { q: "ruby", page: 2 } }])
+```
+
 ### First N responses (cancel the rest)
 
 ```ruby
@@ -78,13 +99,29 @@ FastCurl.stream_get(urls, connections: 50) do |index, response|
 end
 ```
 
-### Retry functionality (v0.2.0+)
+### Retries
+
+**Only idempotent methods (GET, HEAD, PUT, DELETE, OPTIONS) are retried.**
+Several retryable curl errors — `GOT_NOTHING`, `SEND_ERROR`, `RECV_ERROR`,
+`PARTIAL_FILE` — can occur *after* the server has already accepted and processed
+the request, so replaying a `POST` or `PATCH` may duplicate its side effects.
+If you know the endpoint is safe to replay (e.g. it takes an idempotency key),
+opt in with `retry_non_idempotent: true`.
+
+`timeout` applies to a single attempt. Use `total_timeout` to bound the whole
+call, including retries and backoff:
+
+```ruby
+FastCurl.get(urls, timeout: 5, retries: 3, total_timeout: 10_000)
+```
+
+Delays use exponential backoff with full jitter, starting from `retry_delay`.
 
 ```ruby
 # Automatic retry on network errors (timeout, connection issues)
 results = FastCurl.get([
   { url: "https://unreliable-api.com/data" }
-], retries: 3, retry_delay: 1000)  # 3 retries with 1s delay
+], retries: 3, retry_delay: 1000)  # base delay 1s, doubling with jitter
 
 # Retry on specific HTTP status codes
 results = FastCurl.get([
@@ -109,14 +146,35 @@ end
 
 ## Response format
 
+Every response — successful or not — has the same keys:
+
 ```ruby
 [index, {
-  status: 200,             # HTTP status code (0 on error)
-  headers: { "Key" => "Value" },
-  body: "response body"
+  status: 200,                    # HTTP status code, 0 on error
+  headers: { "content-type" => "application/json" },
+  body: "response body",
+  error: nil,                     # nil, or :curl_error / :invalid_request /
+                                  # :not_completed / :deadline_exceeded
+  error_code: nil,                # CURLcode when error == :curl_error
+  effective_url: "https://...",   # final URL after redirects
+  attempts: 1                     # attempts made, including the first
 }]
 ```
 
+Check `response[:error]` rather than `response[:status] == 200`; a status of `0`
+always means the request never produced an HTTP response.
+
+Header names are normalised to lower case (HTTP/2 sends them that way and
+HTTP/1.1 may not), and lookups are case-insensitive:
+
+```ruby
+response[:headers]["Content-Type"]   # => "application/json"
+response[:headers]["content-type"]   # => "application/json"
+```
+
+Repeated fields fold into one comma-separated String. `set-cookie` cannot be
+folded and is **always** an Array, even for a single cookie.
+
 ## Available methods
 
 | Method | Description |
@@ -137,10 +195,32 @@ end
 | Option | Default | Description |
 |---|---|---|
 | `connections` | 20 | Max parallel connections |
-| `timeout` | 30 | Per-request timeout in seconds |
-| `retries` | 1 | Number of retry attempts (0-10) |
-| `retry_delay` | 0 | Delay between retries in milliseconds |
+| `timeout` | 30 | Timeout for a single attempt, in seconds (1-300) |
+| `connect_timeout` | 10000 | Connection phase timeout, in milliseconds |
+| `total_timeout` | none | Wall-clock budget for the whole call, in milliseconds |
+| `retries` | 1 | Retry attempts for idempotent methods (0-10) |
+| `retry_delay` | 100 | Base backoff in milliseconds; doubles with jitter |
 | `retry_codes` | [] | HTTP status codes to retry on |
+| `retry_non_idempotent` | false | Also retry POST and PATCH |
+| `follow_redirects` | true | Follow `Location` headers |
+| `max_redirects` | 5 | Redirect limit (0-100) |
+
+DNS results and TLS sessions are cached process-wide, so repeated calls to the
+same host skip resolution and can resume TLS. TCP connections are pooled only
+within a single call — see Known limitations.
+
+## Known limitations
+
+- TCP connections are not reused across separate calls; each call builds its own
+  `curl_multi` handle. Sharing libcurl's connection cache across concurrent
+  multi handles deadlocks or crashes, so only the DNS and TLS session caches are
+  shared.
+- The whole response body is buffered in memory (100 MB cap per response);
+  `stream_execute` streams *responses*, not bodies.
+- HTTP/2 multiplexing is enabled, but `connections` caps in-flight requests and
+  TCP connections with the same number, so multiplexing cannot be exploited
+  beyond that limit.
+- No multipart, cookie jar, proxy or auth helpers yet.
 
 ## Performance
 

data/ext/fast_curl/extconf.rb

@@ -8,6 +8,9 @@ have_header("ruby/fiber/scheduler.h")
 
 have_func("curl_multi_wakeup", "curl/curl.h")
 have_func("rb_fiber_scheduler_current", "ruby.h")
+have_func("rb_fiber_scheduler_block", "ruby.h")
+have_func("rb_fiber_scheduler_unblock", "ruby.h")
+have_func("rb_fiber_current", "ruby.h")
 have_func("rb_io_wait", "ruby.h")
 
 $CFLAGS << " -std=c99 -O2 -Wall -Wextra -Wno-unused-parameter"