curb 1.4.1 → 1.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b3e51fdd9738a5fa5c1a7a393d09518ce25078374d4dde2929de0825817b4d84
-  data.tar.gz: a6ad92343bdfd8fee02f23be8ce808722793fa55cb1f4b1be03cdd1c1d65eeaa
+  metadata.gz: 0d59fbe309c871b35fe03022151783ee2f88a2a348847758034c4ef83034e868
+  data.tar.gz: 3c143d3940a09679b69bd9d975342bbdbe1a200d1cf24fa6756714570d977d1b
 SHA512:
-  metadata.gz: 7a88d188c5511dc5d28e918bff804d73a4b855880eefe9fb45eb840803cb177a7767572a852599a47c697cd47a218206da5d838366e3f5f1f57b20096d861ce6
-  data.tar.gz: 2c7094094a6227ec09ccab48810269b528b876e1f4dbeca459124e44b963d6d71f3100486637f67ec2282199192f5327e809634ad4ed92add7d409f73ca23a13
+  metadata.gz: 41ecf3ceed954e7a79b19db4b4969e2186407cf9d071b570471e4867d1a20521adf1a0091fa2c2c98902817f3c6c2ed435d11d0f9c3b9137ffd9923a11792468
+  data.tar.gz: 04cdc459106867e381df0c89db22448636cca5dbe6f7073ba0f7b70d9124030bd5a909b3c90de6242d3e2fe865ea6a296fcbd1fd35960baba5d6ca51f241c6c0

data/ChangeLog.md

@@ -1,5 +1,20 @@
 # ChangeLog
-## 1.4.1 (unreleased)
+## 1.5.0
+* Behavior changes (may affect code that does not use the safety features): redirects are now capped at 30 when `max_redirects` is unset, where curb previously passed unlimited (set `max_redirects = -1` to restore unlimited); CR, LF, and NUL in headers, proxy headers, `useragent`, `cookies`, custom request methods, and `CURLOPT_REQUEST_TARGET` now raise `ArgumentError` instead of being sent; `easy.cookies=` is no longer sent to other hosts after a cross-host redirect on libcurl 7.83+; and replacing `post_body`/`put_data` or calling `http_post`/`http_put`/`http_patch`/`http_delete`/`http(verb)` while a transfer is active now raises `RuntimeError`. Under `network_policy = :public` (including `Curl.safe!`), `file://` URLs without a protocol allowlist, non-IP-literal DoH URLs, and Unix sockets without `allow_unix_socket` are refused, and safe mode now defaults to a 60s timeout and 20 redirects. Details for each are below.
+* Read `Curl::Multi.default_timeout` once per `Curl::Multi#perform` instead of on every wait-loop pass. The setting lives in Ractor-local storage, so each read cost two Ruby method calls, and the polling loop read it up to twice per pass (50 MB download: 132–166 `Ractor.current` calls per perform → 14, independent of transfer length). Applies to both the polling and Fiber-scheduler socket-action loops. Changing `default_timeout` from a callback or perform block now takes effect on the next `perform`.
+* Fix a use-after-free when request state is replaced while a transfer is active: `post_body=`, `put_data=`, `autoreferer=`, `http_post`, `http_put`, `http_patch`, `http_delete`, and `http(verb)` called from a callback on the same handle, or on a sibling handle in the same performing `Curl::Multi`, freed the body buffer libcurl was still sending and could crash or send freed memory. These now raise `RuntimeError` like `setopt` does, and re-running native setup on an active handle is rejected the same way. Reconfiguring a handle from completion callbacks or once its transfer finishes is unchanged. Add regression coverage for own-handle and sibling-handle mutation.
+* Test suite: fix intermittent "Could not connect" failures in the per-test WEBrick fixture (seen in the macOS Fiber scheduler tests). It reserved a free port, released it, and re-bound it without a bind address; if the IPv4 bind then lost a race for that ephemeral port, WEBrick silently kept only its IPv6 listener and `127.0.0.1` refused connections. The fixture now binds `127.0.0.1` explicitly and lets the kernel choose the port at bind time.
+* Make `CURLOPT_STDERR` safer against programmer mistakes such as closing the IO too early: previously libcurl wrote verbose output through the IO's own `FILE*`, so closing the IO (including the block form of `File.open`, or closing it after `Curl::Multi#add`) and performing again wrote through freed memory and could corrupt the heap. curb now hands libcurl a stream it owns on a duplicate descriptor, so verbose output keeps reaching the file after the caller closes their IO. The descriptor is released on replacement, `reset`, `close`, and GC; clones get their own. Non-writable IOs now raise `IOError` instead of silently dropping output. Add regression coverage for early close, multi transfers, clones, and descriptor release.
+* Fix a `network_policy = :public` bypass through non-IP sockets: a Unix socket path set with `setopt(CURLOPT_UNIX_SOCKET_PATH, ...)` after `Curl::Multi#add` was used without any destination check. The socket-open policy check now refuses every non-IP socket unless Unix sockets were explicitly allowed (`allow_unix_socket`), regardless of when the option was set. Add regression coverage for late Unix socket configuration.
+* Require public IP-literal DoH endpoints under `network_policy = :public`: libcurl's DoH lookups do not go through the socket policy, so a DoH URL such as `https://localhost/dns-query` (or `127.1`, `[::1]`, a private address) reached local services even with `allow_doh` enabled, and the per-handle policy did not check DoH URLs at all. DoH hostnames are now rejected because they resolve outside the policy; use an address such as `https://1.1.1.1/dns-query`. The host is taken from libcurl's own URL parser, so numeric shorthand forms are normalized before the check. Add regression coverage for loopback, private, shorthand, IPv6, and hostname endpoints.
+* Reject `file://` URLs under `network_policy = :public` when no protocol allowlist is configured, since reading a local file opens no socket for the policy to check. An explicit allowlist that includes `:file` is still honored, and `Curl.safe!`/`safe_*` keep raising `UnsupportedProtocolError` through their HTTP/HTTPS allowlist. Add regression coverage.
+* Stop sending `easy.cookies=` to other hosts on redirect: cookies were passed as `CURLOPT_COOKIE`, which libcurl sends to every host a redirect reaches. They are now sent as a `Cookie` header, which libcurl 7.83+ drops on cross-host redirects. An explicit `Cookie` header still takes precedence, and clearing `cookies` now stops sending them on a reused handle. With the cookie engine enabled (`enable_cookies` or `cookielist`), `cookies=` is still passed as `CURLOPT_COOKIE` so libcurl merges both into one `Cookie` line, and is still forwarded on redirect; use the cookie engine's domain-scoped cookies there. Add regression coverage for cross-host and same-host redirects.
+* Cap redirects by default: curb passed `-1` (unlimited) as `CURLOPT_MAXREDIRS` whenever `max_redirects` was unset, overriding libcurl's own limit. Unset now means at most 30 redirects (libcurl 8.3+'s default, applied on every libcurl version); `max_redirects = -1` still allows unlimited redirects. `Curl.safe!`, `safe_*` helpers, and `Easy#safe_http!` additionally default to a 60-second timeout and at most 20 redirects, configurable through `SafetyConfig#timeout`/`#max_redirects` and the `:timeout`/`:max_redirects` helper options, and applied only when a handle has not set its own (unlimited redirects are capped). Add regression coverage for redirect loops and safe-mode limits.
+* Reject CR, LF, and NUL characters in request headers, proxy headers, `useragent`, `cookies`, custom request methods (including `http(verb)`), and `CURLOPT_REQUEST_TARGET`, raising `ArgumentError`. libcurl sends these values verbatim, so a value built from user input could inject extra headers or split the request. Add regression coverage.
+* Stop reserving 32 KB/16 KB body and header buffers on every perform: that capacity stayed attached to each returned `body_str`/`header_str`, so a 2 KB response held 32 KB. The buffers now start small, grow as data arrives, and are trimmed to fit once the transfer completes (5,000 retained 2 KB bodies: 164 MB → 10.4 MB; 20,000 reused-handle GETs: 29 minor GCs → 2). Request time and large downloads are unchanged. Add regression coverage for retained capacity and large bodies.
+* Cut fiber-scheduler overhead for `Curl::Easy#perform`: the scheduler driver yielded on every `Curl::Multi#perform` pass, about five yields per request, although the socket-action loop already waits through the scheduler on each pass. It now yields only on the fallback wait loop (Windows and builds without socket-action support), which blocks the thread. Under Async, a sequential request drops from ~42 µs/53 allocations to ~33 µs/41, and 100 concurrent fibers from ~30 µs to ~19 µs per request. Add regression coverage that sibling fibers keep running while a large body streams.
+
+## 1.4.1
 * Fix CI fixtures for Windows file-URL normalization and libcurl/libpsl localhost cookie loading. Keep parallel multipart Ractor coverage on Windows without triggering Ruby VM bug #21537 through explicit worker GC; retain multipart compaction coverage in the PostField tests. Include this changelog and the transfer-guard regression tests in the gem package.
 * Make `Curl.safe!` default to `network_policy = :public`, including calls without a configuration block. Global safe mode, `safe_*` helpers, and `Easy#safe_http!` now default to a configurable 10 MiB response-body cap, enforced for buffered responses and body callbacks. Explicit helper limits cannot weaken a tighter global cap. Trusted private destinations require an explicit global policy opt-out; raw requests outside safe mode retain their existing defaults. Add regression coverage for default SSRF blocking, multi policy refresh, body limits, and explicit overrides.
 * Complete idle easy-handle detachment during repeated `initialize`, `close`, and `reset`: clear native attachment counts and Ruby request bookkeeping so the handle can be re-added immediately. Check failed libcurl escape/unescape allocations before accessing their results, raising `Curl::Err::CurlError`, and free returned buffers even if Ruby string allocation raises.

data/README.md

@@ -208,12 +208,17 @@ helpers without application-level validation. For user-supplied URLs, enable the
 safety policy for the current Ractor before making requests. `Curl.safe!` defaults
 to HTTP/HTTPS and `network_policy = :public`, blocking local/private destinations
 even when called without a block. Safe mode also defaults to a 10 MiB
-(10,485,760-byte) response-body limit:
+(10,485,760-byte) response-body limit, a 60-second transfer timeout, and at most
+20 redirects. The timeout and redirect cap only fill in limits a handle has not
+set itself (an unlimited `max_redirects = -1` is capped); set either to `nil` to
+disable it:
 
 ```ruby
 Curl.safe! do |config|
   config.network_policy = :public      # default; block local/private destination IPs
   config.max_body_bytes = 1_000_000     # optional tighter cap; default is 10 MiB
+  config.timeout = 30                   # seconds; default is 60
+  config.max_redirects = 5              # default is 20
 end
 
 curl = Curl.get(user_url)               # allows only http/https, including redirects
@@ -239,21 +244,27 @@ exceptions from `Curl.safe!` honored. Unsupported libcurl builds fail closed.
 Safe helpers and `easy.safe_http!` default to the global safe-mode body limit,
 or 10 MiB when global safe mode is not enabled. Override it explicitly with
 `Curl.safe_get(user_url, :max_body_bytes => 1_000_000)` or
-`easy.safe_http!(:max_body_bytes => 20 * 1024 * 1024)`. A tighter global limit
+`easy.safe_http!(:max_body_bytes => 20 * 1024 * 1024)`; `:timeout` and
+`:max_redirects` are accepted the same way. A tighter global limit
 always takes precedence. Set `config.max_body_bytes = nil` (or `0`) to disable
 the global cap; a helper's explicit `:max_body_bytes => nil` or `0` disables its
 own cap, but cannot bypass a global cap or an existing tighter handle limit.
-Raw requests outside safe mode remain unlimited unless configured otherwise.
+Raw requests outside safe mode have no body limit or timeout unless configured
+otherwise, and follow at most 30 redirects when `max_redirects` is unset.
 These limits cover body bytes, not response headers.
 For trusted local services, use the raw helpers with explicit protocol limits.
 If global safe mode is enabled, explicitly set `config.network_policy = :none`
 to opt out of peer-address filtering; `safe_*` helpers still enforce `:public`.
 
 With `network_policy = :public`, curb checks peer addresses when libcurl opens
-the socket and blocks local/private destinations. Proxies, `resolve`,
+the socket and blocks local/private destinations, `file://` URLs (unless an
+explicit protocol allowlist includes `:file`), and any non-IP socket such as a
+Unix socket path set after the policy was applied. Proxies, `resolve`,
 `connect_to`, DoH URL overrides, and Unix socket paths are disabled by default
-under this policy unless explicitly allowed in the safety config. Custom DNS
-server overrides are rejected. To use a trusted explicit proxy without
+under this policy unless explicitly allowed in the safety config. When DoH is
+allowed, the DoH URL must use a public IP address (for example
+`https://1.1.1.1/dns-query`): libcurl resolves a DoH hostname outside the
+policy, so hostnames are rejected. Custom DNS server overrides are rejected. To use a trusted explicit proxy without
 re-enabling environment proxies, set `allowed_proxy_hosts` and configure
 `easy.proxy_url` on the request.
 

data/ext/curb.h

@@ -28,11 +28,11 @@
 #include "curb_macros.h"
 
 // These should be managed from the Rake 'release' task.
-#define CURB_VERSION   "1.4.1"
-#define CURB_VER_NUM   1041
+#define CURB_VERSION   "1.5.0"
+#define CURB_VER_NUM   1050
 #define CURB_VER_MAJ   1
-#define CURB_VER_MIN   4
-#define CURB_VER_MIC   1
+#define CURB_VER_MIN   5
+#define CURB_VER_MIC   0
 #define CURB_VER_PATCH 0