email-validation-sdk 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cef3267cdbc0f471acc37793832fa635ad5fae74f6a12e885dd852ee9f2e6673
-  data.tar.gz: 8bf364214be99dc1b0faef104424c0585ac41808b3034b8b26dbde4596e41858
+  metadata.gz: 42a80685440b79d131e81f340ddb16dd70462a855e80163f84226c48ba38af54
+  data.tar.gz: 3afbbd8c4875d74653bd85a3d74a6be2fedc5a03b98de5143c2f348a393da188
 SHA512:
-  metadata.gz: 986d270b3fb54ceef83d8cd1ea279ddd57d4cc5b20136978a58c87d1c41a6de1321442cb69a175d368ddb1037c6b594e15dd1268c08b097af30d4170cd276178
-  data.tar.gz: 0751cdba9c258ca15890e9d94a862e061292b95c50b73d8b43ac0deb1f420ba6618a9c03726e060dd4e5f81a453210b5885a6c68e9cb2032059537a1f1f59f2f
+  metadata.gz: 6f98f9cda933bb3c97d28499f7182a80612a85af93b495c3cb5e96a8295933b5a5dcfdc395d5829fd653ac3e6c959d42355cf3e02ea45c25dd3f75bcfc02a3d4
+  data.tar.gz: 29d75a98ca4d6fbd475e54240c55ceaff627a47944033f837df48382f368877d82cc8b0ae2b9d0304578b0ff72924f6de4176fd0827ccff92bbd297422494c22

data/CHANGELOG.md

@@ -4,6 +4,45 @@ All notable changes to the Ruby SDK (`email-validation-sdk`). The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the SDK uses [Semantic Versioning](https://semver.org/):
 a new vocabulary value, option or operation is a minor release; a removed or renamed public symbol is a major one.
 
+## [1.1.0] - 2026-11-06
+
+Hardening: one behaviour for retries, timeouts, errors, bulk transfers and webhooks across the official SDKs.
+
+### Added
+
+- `request_timeout:` on every method of `Client` and `Bulk` (each poll of `bulk.wait`): replaces `timeout` for that
+  one call; a value that is not a finite number of seconds greater than 0 raises `ArgumentError`.
+- `client.bulk.download(job_id, path, resume: true, filter:, columns:, escape_formulas:)`: streams the results into a
+  file and resumes a partial one with `Range` (an already complete file, answered `416 bytes */N`, is left as is).
+- `Error#headers` (the answer's response headers) and `HttpRequest#transfer?`.
+- `StatusCounts#extra`: a status this version does not know is kept and re-encoded by `to_h` / `to_json`. It is
+  not a `Data` member: `members`, positional `new` and array patterns stay the five statuses of 1.0.0.
+
+### Changed
+
+- Transfers (a `file:` upload, `bulk.results`, `bulk.download`) use `timeout` as an idle timeout instead of a cap on
+  the whole attempt, so a slow but progressing upload or download is no longer cut (and a whole-body download is no
+  longer retried from byte 0). The timer never runs while a results block is busy. JSON calls keep the hard bound.
+- A transport you pass in is bounded by the client itself (a worker thread per attempt), so one that ignores
+  `request.timeout` cannot hang a call. On a timeout the worker gets an error raised (as on a network failure), so a
+  Net::HTTP-based transport closes the connection whose answer it did not read; a late answer never reaches the next
+  call.
+- A whole file answered to a `range:` request is refused when its head arrives and the body is dropped unread (also
+  without a block).
+- `filter: []` and `"all"` inside an Array are refused with `ArgumentError` before any request.
+- A binary `file:` String is sent as is (no copy); an IO is read into one binary String.
+- `timeout` refuses `Float::INFINITY` like `NaN`.
+- Webhooks: signatures joined with `, ` by a proxy (and runs of spaces) verify.
+
+### Fixed
+
+- An answer cut short (the connection closed before its `Content-Length`) raises `ConnectionError`
+  `connection_error` instead of passing as complete: a truncated `bulk.results` body, `bulk.download` file or gzip
+  body no longer looks whole, and a cut JSON answer to a `GET` is retried instead of failing as `unexpected_response`
+  (the built-in transport sets Net::HTTP's `ignore_eof` to false).
+- `unexpected_response` (an answer that is not JSON) no longer keeps the JSON parser's error as its `cause`, whose
+  message quoted the start of the body.
+
 ## [1.0.0] - 2026-11-04
 
 First version, with the same behaviour as the JavaScript and Python SDKs (API v1).

data/README.md

@@ -3,7 +3,7 @@
 Official Ruby client for the [email-validation.org](https://email-validation.org) API: validate single addresses, read
 credits, usage and service status, run bulk jobs and verify webhook signatures. Immutable result models, typed errors,
 automatic retries for answers that are never charged, and no runtime dependencies (only the standard library:
-`net/http`, `json`, `openssl`). Ruby 3.2 or later.
+`net/http`, `json`, `openssl`). Needs Ruby 3.2 or later; tested on Ruby 4.0.
 
 Documentation: https://email-validation.org/docs/sdks/ruby/
 
@@ -35,8 +35,9 @@ puts result.reason  # => accepted
 puts result.action  # => send
 ```
 
-Addresses at `sandbox.email-validation.org` are free and answer the reason in their local part
-(`catch_all@…`, `no_mailbox@…`, `timeout@…`); `EmailValidation.sandbox_address("catch_all")` builds one.
+Every result has one of five statuses: `valid`, `invalid`, `catch_all`, `unknown` or `disposable`; `reason` says why
+and `action` what to do. Addresses at `sandbox.email-validation.org` are free and answer the reason in their local
+part (`catch_all@…`, `no_mailbox@…`, `timeout@…`); `EmailValidation.sandbox_address("catch_all")` builds one.
 
 Build the client once and share it: it is safe to use from several threads and keeps connections alive.
 `client.close` releases them.
@@ -46,22 +47,36 @@ Build the client once and share it: it is safe to use from several threads and k
 ```ruby
 client = EmailValidation::Client.new(
   api_key: ENV.fetch("EV_API_KEY"), # required; never logged or shown by #inspect
-  base_url: "https://api.email-validation.org", # default
-  timeout: 30,          # seconds per attempt (connect + whole response), default 30
+  base_url: EmailValidation::EU_BASE_URL, # default EmailValidation::DEFAULT_BASE_URL
+  timeout: 30,          # seconds per attempt, default 30 (see below)
   max_retries: 2,       # retries after the first attempt, default 2
   max_retry_wait: 10    # longest single wait (seconds) the client accepts before giving up, default 10
 )
+
+client.credits(request_timeout: 5) # every method takes request_timeout: for that one call
 ```
 
-EU data residency (validated and stored in the EU): pass `base_url: EmailValidation::EU_BASE_URL` to call `https://api-eu.email-validation.org`
-instead of the default `https://api.email-validation.org`. See https://email-validation.org/docs/eu-data-residency/.
+- `api_key`: your key (`ev_live_…`). Keep it on the server.
+- `base_url`: `https://api.email-validation.org` by default. EU data residency (validated and stored in the EU):
+  `EmailValidation::EU_BASE_URL` (`https://api-eu.email-validation.org`), see
+  https://email-validation.org/docs/eu-data-residency/.
+- `timeout`: seconds per attempt. For JSON calls (validate, credits, usage, status, a job's state, start, cancel, an
+  upload of `emails:`) it bounds the whole attempt: connect, send and the whole answer. For transfers (a `file:`
+  upload, `bulk.results`, `bulk.download`) it is an idle timeout: the attempt fails only when no byte was sent or
+  received for that long, so a slow but progressing 100 MB file is never cut. Keep it a few seconds above the
+  `timeout:` you pass to `validate` (the time the service may spend, up to 15 s).
+- `request_timeout:` (every method of `client` and `client.bulk`, also each poll of `bulk.wait`): replaces `timeout`
+  for that one call, with the same JSON and transfer rules.
+- `max_retries`, `max_retry_wait`: see [Retries, timeouts and cancellation](#retries-timeouts-and-cancellation);
+  negative values count as 0.
+- `transport:`: replaces the HTTP stack (see [Bring your own HTTP client](#bring-your-own-http-client)).
 
-A missing key, a key containing whitespace, a `base_url` that is not `http://` or `https://`, or a non-positive
-`timeout` raises `ArgumentError` at construction.
+A missing key, a key containing whitespace, a `base_url` that is not `http://` or `https://`, or a `timeout` (or
+`request_timeout`) that is not a finite number of seconds greater than 0 raises `ArgumentError` before any request.
 
-## Methods
+## Validate an address
 
-### `client.validate(email, ip_address: nil, timeout: nil, fresh: nil)` → `ValidationResult`
+### `client.validate(email, ip_address: nil, timeout: nil, fresh: nil, request_timeout: nil)` → `ValidationResult`
 
 Checks one address. Costs 1 credit, except `unknown` results and local answers (malformed and sandbox addresses),
 which are free (`result.credits_used` is `0` or `1`).
@@ -87,36 +102,32 @@ puts result.did_you_mean if result.did_you_mean
 `role_account`, `disposable`, `catch_all_domain`, `mx_found`, `mx_record`, `smtp_provider`, `domain_age_days`,
 `processed_at`, `credits_used`, `request_id`.
 
-### `client.credits` → `Credits`
+Results are immutable `Data` objects with the API's field names. `to_h` returns the API's fields (nested models
+converted, `nil` kept) and `to_json` re-encodes the API's JSON. Times are kept as the API's ISO 8601 UTC strings
+(`Time.iso8601(result.processed_at)` parses one). Unknown fields are ignored and unknown values (a reason or provider
+added after this SDK version) are kept as the raw string, so compare with the constants rather than assuming a closed
+list; a status this version does not know in a job's `counts` is kept in `counts.extra` and re-encoded.
+
+## Credits, usage and status
 
 ```ruby
 credits = client.credits
 puts credits.balance, credits.debt, credits.pending
 credits.lots.each { |lot| puts "#{lot.remaining} expire at #{lot.expires_at}" } # soonest-expiring first
-```
 
-### `client.usage(from: nil, to: nil, group_by: nil)` → `UsageResponse`
-
-Dates are `"YYYY-MM-DD"` strings (UTC) or `Date` objects. `group_by` is `"day"` (default) or `"key"`.
-
-```ruby
-usage = client.usage(from: "2026-10-01", to: "2026-10-20")
+usage = client.usage(from: "2026-10-01", to: "2026-10-20") # "YYYY-MM-DD" strings (UTC) or Date objects
 puts usage.totals.credits_used
-usage.items.each { |day| puts "#{day.date}: #{day.credits_used}" }            # UsageDay
+usage.items.each { |day| puts "#{day.date}: #{day.credits_used}" } # UsageDay
 
 by_key = client.usage(group_by: "key")
 by_key.items.each { |key| puts "#{key.name || "(deleted key)"}: #{key.credits_used}" } # UsageByKey
-```
-
-### `client.status` → `ServiceStatus`
 
-```ruby
 status = client.status
 puts status.status            # operational, degraded or outage
 puts status.components.api    # also website, bulk, sign_in, payments, support
 ```
 
-### Bulk jobs: `client.bulk`
+## Bulk jobs
 
 A key with the `bulk` scope can validate lists of up to 100,000 addresses (JSON) or files up to 100 MB.
 
@@ -124,19 +135,25 @@ A key with the `bulk` scope can validate lists of up to 100,000 addresses (JSON)
 # From a list (sent as JSON) …
 job = client.bulk.create(emails: ["accepted@sandbox.email-validation.org", "no_mailbox@sandbox.email-validation.org"])
 
-# … or from a CSV / plain-text file: a String, an IO or a Pathname.
+# … or from a CSV / plain-text file: a String (the content), an IO (read once) or a Pathname.
 job = client.bulk.create(file: Pathname.new("newsletter.csv"), column: "email", has_header: true)
 job = client.bulk.create(file: File.open("list.txt", "rb"), content_type: "text/plain")
 
+# A job created with start: false waits in "ready": call start before wait.
+job = client.bulk.create(emails: ["accepted@sandbox.email-validation.org"], start: false)
+client.bulk.start(job.id)
+
 job = client.bulk.wait(job.id, interval: 5, timeout: 3600) do |progress|
   puts "#{progress.status}: #{progress.processed}/#{progress.unique_emails || "?"}"
 end
 
 if job.status == EmailValidation::BulkJobStatus::COMPLETED
-  csv = client.bulk.results(job.id, filter: %w[valid catch_all])          # whole body as a String
-  File.open("results.csv", "wb") do |file|                             # or streamed in chunks
-    client.bulk.results(job.id) { |chunk| file.write(chunk) }
+  csv = client.bulk.results(job.id, filter: %w[valid catch_all])           # whole body as a String
+  File.open("valid.csv", "wb") do |file|                                 # or streamed in chunks
+    client.bulk.results(job.id, filter: "valid") { |chunk| file.write(chunk) }
   end
+  part = client.bulk.results(job.id, range: "bytes=0-1048575")             # one part (answered 206)
+  bytes = client.bulk.download(job.id, "results.csv")                      # to a file; a second call resumes
 end
 ```
 
@@ -147,20 +164,30 @@ end
 | `bulk.start(job_id)` | `POST /v1/bulk/{id}/start` (credits are held now) | `BulkJob` |
 | `bulk.cancel(job_id)` | `DELETE /v1/bulk/{id}` (unused credits return) | `BulkJob` |
 | `bulk.results(job_id, filter: nil, columns: nil, escape_formulas: nil, range: nil) { \|chunk\| … }` | `GET /v1/bulk/{id}/results` | CSV `String`, or `nil` with a block |
+| `bulk.download(job_id, path, resume: true, filter: nil, columns: nil, escape_formulas: nil)` | `GET /v1/bulk/{id}/results` into a file | bytes in the file |
 | `bulk.wait(job_id, interval: 5, timeout: nil, on_progress: nil) { \|job\| … }` | polls `bulk.get` | `BulkJob` |
 
-- `create`: pass exactly one of `emails:` (an Array) or `file:`; `start: false` keeps the job in `ready` until you call
-  `start`. A file given as an IO is read whole, so a retry can resend the same bytes.
-- `results`: `filter` is `"all"`, one status, or an Array of statuses; `columns` is `"original"` (default) or
-  `"email_only"`; `escape_formulas` defaults to `true` on the server. The CSV is returned unchanged (never parsed).
-  `range: "bytes=0-1048575"` fetches one part of the file to resume a download; only `filter: "all"` (or one status
+Every method also takes `request_timeout:`.
+
+- `create`: pass exactly one of `emails:` (an Array) or `file:`. A file is sent with its length (never chunked) and
+  held in memory once: a binary String is sent as is, an IO or a Pathname is read into one String, so a retry
+  resends the same bytes. `start: false` keeps the job in `ready` until you call `start`.
+- `results`: `filter` is `"all"`, one status, or a non-empty Array of statuses (`"all"` cannot be combined; the
+  former names `spamtrap`, `abuse`, `do_not_mail` and `risky` are refused: use `%w[catch_all disposable]` for
+  `risky`); `columns` is `"original"` (default) or `"email_only"`; `escape_formulas` defaults to `true` on the
+  server. The CSV is returned unchanged (never parsed). With a block, chunks reach the block as they arrive, so a
+  large file never sits in memory. `range: "bytes=0-1048575"` fetches one part; only `filter: "all"` (or one status
   with `escape_formulas: false`) with `columns: "original"` can be served in parts, and any other shape raises
-  `InvalidRequestError` instead of returning the whole file.
+  `InvalidRequestError` (status = the answer's) instead of returning the whole file, which is dropped unread.
+- `download`: streams the CSV into `path` (a String or Pathname) and returns the file's length. With `resume: true`
+  (default) and a non-empty file there, only the missing end is fetched and appended; an already complete file is
+  left as it is. Resuming needs a shape served in parts (as for `range`), otherwise `ArgumentError`; a resume
+  answered with the whole file raises `InvalidRequestError` and leaves the file untouched. A failure while writing
+  leaves the partial file in place: call `download` again to resume. `resume: false` downloads it again.
 - `wait` returns once the job is `completed`, `failed`, `cancelled` or `expired`, or once it is `ready` with an
   `error_code` (its start was refused, e.g. `insufficient_credits`: buy credits, then call `start`). Check
-  `job.status` before downloading. A job created with `start: false` waits in `ready` until you call `start`, so call
-  `start` before `wait`. `interval` is at least 1 second; when `timeout` (seconds) passes, `wait` raises
-  `ConnectionError` with code `timeout`.
+  `job.status` before downloading. `interval` is at least 1 second (default 5, also for `nil`); when `timeout`
+  (seconds) would pass before the next poll, `wait` raises `ConnectionError` with code `timeout`.
 - A job id is checked locally before any request (`job_` followed by 22 letters and digits); a malformed one raises
   `InvalidRequestError` with status `0`.
 
@@ -168,15 +195,7 @@ end
 `local_answered`, `processed`, `credits_reserved`, `credits_charged`, `counts`, `error_code`, `webhook_endpoint_id`,
 `created_at`, `started_at`, `finished_at`, `expires_at`; helpers `terminal?` and `stuck?`.
 
-### Models
-
-Results are immutable `Data` objects with the API's field names. `to_h` returns the API's fields (nested models
-converted, `nil` kept) and `to_json` re-encodes the API's JSON. Times are kept as the API's ISO 8601 UTC strings
-(`Time.iso8601(result.processed_at)` parses one). Unknown fields are ignored and unknown values (a reason or provider
-added after this SDK version) are kept as the raw string, so compare with the constants rather than assuming a closed
-list.
-
-## Errors
+## Handle errors
 
 Every failure raises a subclass of `EmailValidation::Error` (a `StandardError`). Branch on the class or on `code`,
 never on `message` (it is for people and may change).
@@ -196,7 +215,9 @@ end
 ```
 
 Every error has `status` (HTTP status, `0` when no answer arrived or the SDK refused the call before sending),
-`code`, `message`, `request_id` (quote it to support), `retry_after` (seconds or `nil`) and `cause`.
+`code`, `message`, `request_id` (quote it to support), `retry_after` (seconds or `nil`), `headers` (the answer's
+response headers, `{}` without an answer) and `cause`. An uncaught error shows its kind, status, code and request id,
+never the key, the request body or the checked address.
 
 | Class | When | Codes |
 |---|---|---|
@@ -213,20 +234,27 @@ Every error has `status` (HTTP status, `0` when no answer arrived or the SDK ref
 | `Error` (base) | a 2xx answer that is not JSON | `unexpected_response` |
 
 An answer without our JSON error envelope (or with an unknown code) has code `unexpected_response`. Argument mistakes
-(e.g. `emails:` and `file:` together, an unknown `filter`) raise `ArgumentError` before any request. Ruby has no
-cancellation mechanism for blocking calls, so the `aborted` code is not used by this SDK.
+(e.g. `emails:` and `file:` together, an unknown `filter`, a bad `request_timeout`) raise `ArgumentError` before any
+request.
 
-## Retries and timeouts
+## Retries, timeouts and cancellation
 
-- `429` and `503` answers are never charged, so they are retried for every method, honouring `Retry-After`, up to
-  `max_retries` times, unless the wait would be longer than `max_retry_wait` seconds. `key_daily_cap` is never
-  retried (it resets the next day).
+- `429` and `503` answers are never charged, so they are retried for every method, honouring `Retry-After` (seconds
+  or an HTTP date), up to `max_retries` times, unless the wait would be longer than `max_retry_wait` seconds.
+  `key_daily_cap` is never retried (it resets the next day). Without `Retry-After` the wait is exponential with
+  jitter (0.25–0.5 s, 0.5–1 s, 1–2 s …, at most 8 s).
 - Network failures and timeouts are retried only for `GET` requests. A validation, bulk upload, start or cancel that
-  reached the service may already have been charged or applied, so it is not sent again.
-- Nothing else is retried. Without `Retry-After` the wait is exponential with jitter (0.25–0.5 s, 0.5–1 s, 1–2 s …,
-  at most 8 s).
-- `timeout` bounds each attempt (connect, TLS and the whole response), not the whole call. For very large result
-  downloads, raise it or fetch the file in parts with `range:`.
+  reached the service may already have been charged or applied, so it is never sent again, neither by the SDK nor by
+  Net::HTTP (its own retries are off). A results download is not retried once a chunk has reached your block (call
+  `bulk.download` again to resume a file). An answer cut short (the connection closes before the announced
+  `Content-Length`) is a `ConnectionError` `connection_error`, never a complete result.
+- Nothing else is retried (`500`, `502`, `504` and other errors are raised at once).
+- `timeout` bounds each attempt, not the whole call: the whole attempt for JSON calls, an idle timeout for transfers
+  (uploads of a file, results and downloads), with no cap on a transfer that keeps moving. A results block may take
+  as long as it needs: the timer runs only while the SDK waits for data. `request_timeout:` changes the bound for
+  one call.
+- Cancellation: Ruby has no standard cancellation for blocking calls, so this SDK has none (the `aborted` code is
+  not used). Bound a call with `request_timeout:` or `bulk.wait(timeout:)` instead.
 - Redirects are never followed, so the `Authorization` header never travels to another host.
 
 ## Verify webhooks
@@ -247,7 +275,8 @@ when "webhook.test" then puts "Test from endpoint #{event.data.endpoint_id}"
 end
 ```
 
-- Headers: any Hash-like object with header names in any case, or a Rack env (`HTTP_WEBHOOK_ID` keys).
+- Headers: any Hash-like object with header names in any case (String or Array values; several values are joined
+  with a space, and signatures a proxy joined with `, ` still verify), or a Rack env (`HTTP_WEBHOOK_ID` keys).
 - Failures raise `EmailValidation::WebhookVerificationError` (missing headers, a timestamp outside `tolerance`
   seconds (default 300), an invalid secret, no matching signature or a body that is not JSON). Answer `400` then.
 - Deduplicate on the `webhook-id` header (retries reuse it) and answer `2xx` quickly; do slow work afterwards.
@@ -289,7 +318,8 @@ EmailValidation::WebhookEventType::CREDITS_LOW # "credits.low"
 
 The client talks to a transport: any object with `call(request)` that returns an `EmailValidation::HttpResponse`, or
 raises `EmailValidation::TransportError` (with `timeout: true` for a timeout) when no answer arrived. `request` has
-`method`, `url`, `headers` (lower-case names), `body` (a binary String or `nil`) and `timeout` (seconds per attempt).
+`method`, `url`, `headers` (lower-case names), `body` (a binary String or `nil`; do not change it), `timeout` (seconds
+per attempt) and `transfer?` (true for a file upload or a results download, where `timeout` is an idle timeout).
 The default is `EmailValidation::NetHttpTransport` (Net::HTTP with a keep-alive pool; it honours the standard proxy
 environment variables like Net::HTTP does).
 
@@ -320,8 +350,30 @@ with `(status, headers)` once the response head arrives; if it returns a callabl
 return an `HttpResponse` with an empty body. A transport that ignores the block still works (the body is handed over
 in one piece). `request.inspect` never shows the headers or the body.
 
+The client bounds a transport you pass itself, even if it ignores `request.timeout`: it runs each attempt on a
+short-lived worker thread (thread-local state of the calling thread is not visible there) and gives up with
+`ConnectionError` `timeout` when the answer, or for a transfer the next chunk, does not arrive in time. It then
+raises an error in the worker, as a network failure would (like `Timeout.timeout` does), so the transport's `rescue`
+and `ensure` clauses run, and ignores whatever the transport does after that. The built-in transport needs no
+worker.
+
+Two rules keep a transport from returning wrong data or charging twice:
+
+- Never reuse a connection whose answer was not read to the end (after an error or an interrupted read): close it.
+  Net::HTTP does this when an error interrupts a request; a pool or a raw socket you manage yourself must do the
+  same, or the next call reads the previous call's late answer, the result of another address.
+- Never retry a request yourself, and turn off your HTTP library's own retries: the client's retry rules decide what
+  may be sent again. A validation, bulk upload, start or cancel that reached the service may already have been
+  charged or applied. `Net::HTTP` resends an idempotent request (a `DELETE`, which cancels a job) once by default:
+  set `max_retries = 0` on it, as the built-in transport does; `net-http-persistent` does the same unless its
+  `max_retries` is `0`.
+
 ## Privacy
 
 The API key is sent only in the `Authorization` header. It never appears in a URL, in an error message, or in the
-`inspect`/`to_s`/`pp` output of the client, its config or a request. The SDK writes nothing to stdout or stderr and
-has no telemetry.
+`inspect`/`to_s`/`pp` output of the client, its config or a request; request bodies (the checked addresses) are not
+shown either. The SDK writes nothing to stdout or stderr and has no telemetry.
+
+## Licence
+
+MIT, see [LICENSE](LICENSE).

data/lib/email_validation_sdk/bulk.rb

@@ -11,7 +11,9 @@ module EmailValidation
     BAD_JOB_ID = "Not a bulk job id (expected job_ followed by 22 letters and digits)."
     NOT_RANGEABLE = 'This download cannot be fetched in parts: use filter: "all" (or one status with ' \
                     'escape_formulas: false) and columns: "original" with a range.'
-    private_constant :JOB_ID, :CONTENT_TYPES, :FILTERS, :BAD_JOB_ID, :NOT_RANGEABLE
+    NOT_RESUMABLE = 'This download cannot be resumed: use filter: "all" (or one status with escape_formulas: false) ' \
+                    'and columns: "original", or pass resume: false.'
+    private_constant :JOB_ID, :CONTENT_TYPES, :FILTERS, :BAD_JOB_ID, :NOT_RANGEABLE, :NOT_RESUMABLE
 
     # Wraps an exception raised by the caller's chunk block, so a transport never mistakes it for a network error.
     class SinkError < StandardError
@@ -24,6 +26,20 @@ module EmailValidation
     end
     private_constant :SinkError
 
+    # Raised when the head of a whole-file answer to a range request arrives, so the transport drops the connection
+    # without reading the body (it can be the whole 100 MB file). Outside StandardError, so neither a transport's
+    # +rescue => e+ nor a network-error rescue mistakes it for a failure to retry.
+    class RangeRefused < Exception # rubocop:disable Lint/InheritException
+      attr_reader :status, :request_id
+
+      def initialize(status, request_id)
+        super("range refused")
+        @status = status
+        @request_id = request_id
+      end
+    end
+    private_constant :RangeRefused
+
     def initialize(client)
       @client = client
     end
@@ -34,9 +50,11 @@ module EmailValidation
     #   (up to 100 MB), with +content_type+ "text/csv" (default) or "text/plain".
     #
     # Options: +column+ (1–512 or a header name), +has_header+, +start+ (server default true; with +false+ the job
-    # waits in "ready" until #start), +webhook_endpoint_id+.
+    # waits in "ready" until #start), +webhook_endpoint_id+. Every Bulk method also takes +request_timeout:+ (seconds
+    # per attempt for that call, instead of the client's +timeout+). A file upload is a transfer: the timeout is an
+    # idle timeout, so a slow but progressing upload is never cut.
     def create(emails: nil, file: nil, content_type: "text/csv", column: nil, has_header: nil, start: nil,
-               webhook_endpoint_id: nil)
+               webhook_endpoint_id: nil, request_timeout: nil)
       raise ArgumentError, "pass exactly one of emails: or file:" if emails.nil? == file.nil?
       raise ArgumentError, 'content_type must be "text/csv" or "text/plain"' unless
         CONTENT_TYPES.include?(content_type)
@@ -47,25 +65,27 @@ module EmailValidation
         raise ArgumentError, "emails must be an Array of addresses, not a String" if emails.is_a?(String)
         raise ArgumentError, "emails must be an Array of addresses" unless emails.respond_to?(:to_a)
 
-        return @client.request_model(BulkJob, "POST", "/v1/bulk", query: query, json: { "emails" => emails.to_a })
+        return @client.request_model(BulkJob, "POST", "/v1/bulk", query: query, json: { "emails" => emails.to_a },
+                                                                  request_timeout: request_timeout)
       end
       @client.request_model(BulkJob, "POST", "/v1/bulk", query: query, body: read_file(file),
-                                                         content_type: content_type)
+                                                         content_type: content_type, transfer: true,
+                                                         request_timeout: request_timeout)
     end
 
     # The job's current state.
-    def get(job_id)
-      @client.request_model(BulkJob, "GET", job_path(job_id))
+    def get(job_id, request_timeout: nil)
+      @client.request_model(BulkJob, "GET", job_path(job_id), request_timeout: request_timeout)
     end
 
     # Starts a job waiting in "ready"; credits are held now.
-    def start(job_id)
-      @client.request_model(BulkJob, "POST", job_path(job_id, "/start"))
+    def start(job_id, request_timeout: nil)
+      @client.request_model(BulkJob, "POST", job_path(job_id, "/start"), request_timeout: request_timeout)
     end
 
     # Cancels a job; unused credits are returned.
-    def cancel(job_id)
-      @client.request_model(BulkJob, "DELETE", job_path(job_id))
+    def cancel(job_id, request_timeout: nil)
+      @client.request_model(BulkJob, "DELETE", job_path(job_id), request_timeout: request_timeout)
     end
 
     # Downloads the result CSV (UTF-8, returned unchanged; never parsed).
@@ -75,19 +95,42 @@ module EmailValidation
     # - +escape_formulas+: server default true.
     # - +range+: e.g. "bytes=0-1048575", to fetch one part of the file (resume a download). Only
     #   +filter: "all"+ (or one status with +escape_formulas: false+) and +columns: "original"+ can be served in parts;
-    #   any other shape raises InvalidRequestError instead of returning the whole file.
+    #   any other shape raises InvalidRequestError (status = the answer's) instead of returning the whole file, and
+    #   that body is dropped unread.
     #
     # Without a block it returns the whole body as a String. With a block it yields the body in chunks (binary
-    # Strings) as they arrive and returns nil, so a large file never sits in memory:
+    # Strings) as they arrive and returns nil, so a large file never sits in memory (#download writes it to a file):
     #
     #   File.open("results.csv", "wb") { |f| client.bulk.results(job_id) { |chunk| f.write(chunk) } }
-    def results(job_id, filter: nil, columns: nil, escape_formulas: nil, range: nil, &block)
+    #
+    # A download is a transfer: the timeout is an idle timeout (each read may wait up to it, never while your block
+    # runs). It is retried after a network failure only while no chunk has reached you.
+    def results(job_id, filter: nil, columns: nil, escape_formulas: nil, range: nil, request_timeout: nil, &block)
       query = { "filter" => results_filter(filter), "columns" => columns, "escape_formulas" => escape_formulas }
       headers = range ? { "range" => range } : nil
       path = job_path(job_id, "/results")
-      return download(path, query, headers, range) unless block
+      return stream_download(path, query, headers, range, block, request_timeout) if block
 
-      stream_download(path, query, headers, range, block)
+      buffer = +"".b
+      body = stream_download(path, query, headers, range, ->(chunk) { buffer << chunk }, request_timeout)
+      (body ? body.dup : buffer).force_encoding(Encoding::UTF_8)
+    end
+
+    # Streams the result CSV into the file at +path+ (a String or Pathname) and returns the file's length in bytes;
+    # the body never sits in memory. With +resume: true+ (default) and a non-empty file there, only the missing end
+    # is fetched (+Range: bytes=<size>-+) and appended; a file that is already complete is left as it is. Resuming
+    # needs a shape the API serves in parts (+filter: "all"+, or one status with +escape_formulas: false+, and
+    # +columns: "original"+), otherwise ArgumentError; a whole file answered to the range raises InvalidRequestError
+    # and leaves the file untouched. A failure while writing leaves the partial file in place: call again to resume.
+    def download(job_id, path, resume: true, filter: nil, columns: nil, escape_formulas: nil, request_timeout: nil)
+      job_path(job_id)
+      size = resume && File.file?(path) ? File.size(path) : 0
+      options = { filter: filter, columns: columns, escape_formulas: escape_formulas, request_timeout: request_timeout }
+      return write_fresh(job_id, path, options) if size.zero?
+
+      raise ArgumentError, NOT_RESUMABLE unless resumable?(filter, columns, escape_formulas)
+
+      append_rest(job_id, path, size, options)
     end
 
     # Polls a job until it is completed, failed, cancelled or expired, or until it is "ready" with an +error_code+
@@ -99,13 +142,15 @@ module EmailValidation
     # - +on_progress+ (or a block): called with every polled BulkJob.
     #
     # A job created with +start: false+ waits in "ready" until you call #start, so call #start before #wait.
-    def wait(job_id, interval: 5, timeout: nil, on_progress: nil, &block)
+    # - +request_timeout+: seconds per attempt of each poll, instead of the client's +timeout+.
+    def wait(job_id, interval: 5, timeout: nil, on_progress: nil, request_timeout: nil, &block)
       job_path(job_id)
+      Config.check_timeout(request_timeout, "request_timeout") unless request_timeout.nil?
       progress = on_progress || block
       interval = [(interval || 5).to_f, 1.0].max
       deadline = timeout.nil? ? nil : monotonic + timeout
       loop do
-        job = get(job_id)
+        job = get(job_id, request_timeout: request_timeout)
         progress&.call(job)
         return job if job.terminal? || job.stuck?
         if deadline && monotonic + interval > deadline
@@ -130,16 +175,27 @@ module EmailValidation
       "/v1/bulk/#{job_id}#{suffix}"
     end
 
+    # "all", one status, or a non-empty Array of statuses (no "all" in it). Retired names (spamtrap, abuse,
+    # do_not_mail, risky) are refused like any other unknown value.
     def results_filter(value)
       return nil if value.nil?
 
-      parts = value.is_a?(Array) ? value.map(&:to_s) : [value.to_s]
+      if value.is_a?(Array)
+        raise ArgumentError, "filter must not be an empty Array (omit it, or use \"all\")" if value.empty?
+
+        parts = value.map(&:to_s)
+        raise ArgumentError, 'filter: "all" cannot be combined with statuses' if parts.include?("all")
+      else
+        parts = [value.to_s]
+      end
       parts.each do |part|
         raise ArgumentError, "unknown filter: #{part.inspect} (use \"all\" or a status)" unless FILTERS.include?(part)
       end
       parts.join(",")
     end
 
+    # Returns the upload as a String held once: a binary String goes out as is (never copied), a Pathname or an IO
+    # is read into one binary String.
     def read_file(file)
       case file
       when String then file
@@ -150,27 +206,61 @@ module EmailValidation
         data = file.read
         raise ArgumentError, "file returned no data" if data.nil?
 
-        data
+        # A real IO hands back a fresh String: relabel it binary instead of copying it.
+        file.is_a?(IO) && !data.frozen? ? data.force_encoding(Encoding::BINARY) : data
       end
     end
 
-    def download(path, query, headers, range)
-      response = @client.perform("GET", path, query: query, accept: "text/csv", headers: headers)
-      refuse_whole_file(response.status, response.header("x-request-id")) if range && response.status != 206
-      response.body.dup.force_encoding(Encoding::UTF_8)
-    end
-
-    def stream_download(path, query, headers, range, block)
+    # Streams the body into +block+ and returns nil, or the whole body when the transport could not stream it.
+    def stream_download(path, query, headers, range, block, request_timeout)
       stream = Stream.new(range, block)
-      response = @client.perform("GET", path, query: query, accept: "text/csv", headers: headers, stream: stream)
+      response = @client.perform("GET", path, query: query, accept: "text/csv", headers: headers, stream: stream,
+                                              transfer: true, request_timeout: request_timeout)
       refuse_whole_file(response.status, response.header("x-request-id")) if range && response.status != 206
+      return nil if stream.head_seen? || response.body.empty?
+
       # A transport without streaming support buffered the body: hand it over in one piece.
-      block.call(response.body) unless stream.head_seen? || response.body.empty?
+      block.call(response.body)
       nil
+    rescue RangeRefused => e
+      refuse_whole_file(e.status, e.request_id)
     rescue SinkError => e
       raise e.original, cause: e.original.cause
     end
 
+    def resumable?(filter, columns, escape_formulas)
+      return false unless columns.nil? || columns.to_s == "original"
+
+      parts = filter.is_a?(Array) ? filter.map(&:to_s) : [filter&.to_s].compact
+      return true if parts.empty? || parts == ["all"]
+
+      parts.size == 1 && STATUSES.include?(parts.first) && escape_formulas == false
+    end
+
+    # The file is created (or truncated) only when the first chunk arrives, so a refused call leaves it as it was.
+    def write_fresh(job_id, path, options)
+      file = nil
+      begin
+        results(job_id, **options) { |chunk| (file ||= File.open(path, "wb")).write(chunk) }
+      ensure
+        file&.close
+      end
+      File.open(path, "wb").close if file.nil? # an empty result file
+      File.size(path)
+    end
+
+    def append_rest(job_id, path, size, options)
+      appended = 0
+      File.open(path, "ab") do |file|
+        results(job_id, **options, range: "bytes=#{size}-") { |chunk| appended += file.write(chunk) }
+      end
+      size + appended
+    rescue InvalidRequestError => e
+      raise unless e.status == 416 && e.headers["content-range"] == "bytes */#{size}"
+
+      size # already complete
+    end
+
     def refuse_whole_file(status, request_id)
       raise InvalidRequestError.new(NOT_RANGEABLE, status: status, code: "invalid_request", request_id: request_id)
     end
@@ -189,11 +279,11 @@ module EmailValidation
         @started = false
       end
 
-      def on_head(status, _headers)
+      def on_head(status, headers)
         return nil unless (200..299).cover?(status)
 
         @head_seen = true
-        return ->(_chunk) {} if @range && status != 206
+        raise RangeRefused.new(status, request_id(headers)) if @range && status != 206
 
         lambda do |chunk|
           @started = true
@@ -209,6 +299,11 @@ module EmailValidation
         @head_seen
       end
 
+      def request_id(headers)
+        headers.each { |name, value| return value.to_s if name.to_s.casecmp?("x-request-id") }
+        nil
+      end
+
       def started?
         @started
       end

data/lib/email_validation_sdk/client.rb

@@ -18,8 +18,8 @@ module EmailValidation
 
       url = (base_url || DEFAULT_BASE_URL).to_s
       raise ArgumentError, "base_url must start with http:// or https://" unless url.start_with?("https://", "http://")
-      raise ArgumentError, "timeout must be a number of seconds greater than 0" unless
-        timeout.is_a?(Numeric) && timeout.positive?
+
+      Config.check_timeout(timeout, "timeout")
 
       @api_key = api_key.dup.freeze
       @base_url = url.sub(%r{/+\z}, "").freeze
@@ -29,6 +29,14 @@ module EmailValidation
       freeze
     end
 
+    # @api private
+    # Raises ArgumentError unless +value+ is a finite number of seconds greater than 0.
+    def self.check_timeout(value, name)
+      return value if value.is_a?(Numeric) && value.real? && value.finite? && value.positive?
+
+      raise ArgumentError, "#{name} must be a finite number of seconds greater than 0"
+    end
+
     # @api private
     def authorization
       "Bearer #{@api_key}"
@@ -84,11 +92,16 @@ module EmailValidation
     # Options:
     # - +api_key+ (required): your key (+ev_live_…+). Keep it on the server.
     # - +base_url+: default +https://api.email-validation.org+.
-    # - +timeout+: seconds per attempt (connect and the whole response), default 30. A check may take up to 15 s on
-    #   our side, so keep it a few seconds above the +timeout+ you pass to #validate.
+    # - +timeout+: seconds per attempt, default 30. For JSON calls it bounds the whole attempt (connect and the whole
+    #   response); for transfers (a file upload, a results download) it is an idle timeout: the longest time without
+    #   a byte sent or received, with no cap on the whole transfer. A check may take up to 15 s on our side, so keep
+    #   it a few seconds above the +timeout+ you pass to #validate. Every method also takes +request_timeout:+, which
+    #   replaces it for that one call.
     # - +max_retries+: retries after the first attempt, default 2 (negative values count as 0).
     # - +max_retry_wait+: the longest single wait in seconds the client accepts before giving up, default 10.
     # - +transport+: replaces the HTTP stack (see NetHttpTransport); a transport you pass is never closed by #close.
+    #   It runs on a short-lived worker thread per attempt, so the client bounds it even if it ignores
+    #   +request.timeout+ (thread-local state of the calling thread is not visible there).
     # - +sleep+, +clock+: injectable for tests (+sleep.call(seconds)+; +clock.call+ returns the current Time).
     def initialize(api_key:, base_url: nil, timeout: 30, max_retries: 2, max_retry_wait: 10, transport: nil,
                    sleep: nil, clock: nil)
@@ -96,6 +109,8 @@ module EmailValidation
                            max_retry_wait: max_retry_wait)
       @owns_transport = transport.nil?
       @transport = transport || NetHttpTransport.new
+      # The built-in transport bounds its own attempts; any other one runs under the SDK's guard (TransportGuard).
+      @guarded = !@transport.instance_of?(NetHttpTransport)
       @sleep = sleep || ->(seconds) { Kernel.sleep(seconds) }
       @clock = clock || -> { Time.now }
       @bulk = Bulk.new(self)
@@ -110,29 +125,32 @@ module EmailValidation
     # - +fresh+: +true+ checks the address again even when it was checked in the last 12 months (by default the
     #   earlier definite result may be returned; +processed_at+ is when the answer was produced).
     #
+    # - +request_timeout+: seconds per attempt for this call only, instead of the client's +timeout+ (every method
+    #   takes it).
+    #
     # A validation is never retried after a network failure or timeout (it may already have been charged).
-    def validate(email, ip_address: nil, timeout: nil, fresh: nil)
+    def validate(email, ip_address: nil, timeout: nil, fresh: nil, request_timeout: nil)
       raise ArgumentError, "email must be a String" unless email.is_a?(String)
 
       body = { "email" => email, "ip_address" => ip_address, "timeout" => timeout, "fresh" => fresh }.compact
-      request_model(ValidationResult, "POST", "/v1/validate", json: body)
+      request_model(ValidationResult, "POST", "/v1/validate", json: body, request_timeout: request_timeout)
     end
 
     # Balance, debt, pending credits and credit lots (soonest-expiring first).
-    def credits
-      request_model(Credits, "GET", "/v1/credits")
+    def credits(request_timeout: nil)
+      request_model(Credits, "GET", "/v1/credits", request_timeout: request_timeout)
     end
 
     # Credit usage by day (default) or by API key (+group_by: "key"+). Dates are +"YYYY-MM-DD"+ strings (UTC) or
     # Date objects.
-    def usage(from: nil, to: nil, group_by: nil)
+    def usage(from: nil, to: nil, group_by: nil, request_timeout: nil)
       query = { "from" => from, "to" => to, "group_by" => group_by }
-      request_model(UsageResponse, "GET", "/v1/usage", query: query)
+      request_model(UsageResponse, "GET", "/v1/usage", query: query, request_timeout: request_timeout)
     end
 
     # The public service status (overall and per component).
-    def status
-      request_model(ServiceStatus, "GET", "/v1/status")
+    def status(request_timeout: nil)
+      request_model(ServiceStatus, "GET", "/v1/status", request_timeout: request_timeout)
     end
 
     # Releases the HTTP connections of the built-in transport. A transport you passed in is left open.
@@ -164,14 +182,15 @@ module EmailValidation
     # @api private
     # Sends one call with the retry policy and returns the 2xx HttpResponse. +stream+, when given, is an object with
     # +on_head(status, headers)+ (passed to the transport as its block) and +started?+ (true once a chunk reached the
-    # caller, after which nothing is retried).
+    # caller, after which nothing is retried). +transfer+ marks a file upload or a results download (idle timeout).
     def perform(method, path, query: nil, json: nil, body: nil, content_type: nil, accept: JSON_TYPE, headers: nil,
-                stream: nil)
-      request = build_request(method, path, query, json, body, content_type, accept, headers)
+                stream: nil, transfer: false, request_timeout: nil)
+      timeout = request_timeout.nil? ? config.timeout : Config.check_timeout(request_timeout, "request_timeout")
+      request = build_request(method, path, query, json, body, content_type, accept, headers, timeout, transfer)
       attempt = 0
       loop do
         begin
-          response = stream ? @transport.call(request) { |s, h| stream.on_head(s, h) } : @transport.call(request)
+          response = send_attempt(request, stream)
         rescue TransportError => e
           if method == "GET" && attempt < config.max_retries && !stream&.started?
             sleep_for(backoff(attempt))
@@ -208,20 +227,27 @@ module EmailValidation
 
     private
 
-    def build_request(method, path, query, json, body, content_type, accept, extra_headers)
+    def send_attempt(request, stream)
+      return TransportGuard.call(@transport, request, stream) if @guarded
+      return @transport.call(request) { |status, headers| stream.on_head(status, headers) } if stream
+
+      @transport.call(request)
+    end
+
+    def build_request(method, path, query, json, body, content_type, accept, extra_headers, timeout, transfer)
       url = config.base_url + path
       encoded = encode_query(query)
       url = "#{url}?#{encoded}" unless encoded.empty?
       headers = { "authorization" => config.authorization, "accept" => accept, "user-agent" => USER_AGENT }
       if json
-        body = JSON.generate(json).b
+        body = JSON.generate(json).b.freeze
         headers["content-type"] = JSON_TYPE
       elsif body
-        body = body.b
+        body = body.b.freeze unless body.encoding == Encoding::BINARY # a binary String is sent as is, never copied
         headers["content-type"] = content_type if content_type
       end
       extra_headers&.each { |name, value| headers[name.to_s.downcase] = value.to_s }
-      HttpRequest.new(method: method, url: url, headers: headers, body: body, timeout: config.timeout)
+      HttpRequest.new(method: method, url: url, headers: headers, body: body, timeout: timeout, transfer: transfer)
     end
 
     def encode_query(query)
@@ -241,9 +267,11 @@ module EmailValidation
 
       parsed
     rescue JSON::ParserError, EncodingError
+      # no cause: the parser's message quotes the body, which can hold a checked address or result rows
       raise Error.new("The API answered with something other than JSON.", status: response.status,
                                                                            code: "unexpected_response",
-                                                                           request_id: response.header("x-request-id"))
+                                                                           request_id: response.header("x-request-id")),
+            cause: nil
     end
 
     def error_from(response)
@@ -264,7 +292,8 @@ module EmailValidation
         code = "unexpected_response"
         message = "Unexpected HTTP #{response.status} from the API."
       end
-      klass.new(message, status: response.status, code: code, request_id: request_id, retry_after: retry_after)
+      klass.new(message, status: response.status, code: code, request_id: request_id, retry_after: retry_after,
+                         headers: response.headers)
     end
 
     def retry_wait(error, attempt)

data/lib/email_validation_sdk/errors.rb

@@ -11,16 +11,19 @@ module EmailValidation
   #   +timeout+, +unexpected_response+, +aborted+).
   # - +request_id+: our id for the request (quote it to support), when the API answered.
   # - +retry_after+: whole seconds from the +Retry-After+ header, or nil.
+  # - +headers+: the answer's response headers (lower-case names), +{}+ when no answer arrived.
   # - +cause+: the underlying exception, when there is one (Ruby sets it automatically).
   class Error < StandardError
-    attr_reader :status, :code, :request_id, :retry_after
+    attr_reader :status, :code, :request_id, :retry_after, :headers
 
-    def initialize(message = nil, status: 0, code: "unexpected_response", request_id: nil, retry_after: nil)
+    def initialize(message = nil, status: 0, code: "unexpected_response", request_id: nil, retry_after: nil,
+                   headers: {})
       super(message || code)
       @status = status
       @code = code
       @request_id = request_id
       @retry_after = retry_after
+      @headers = headers.frozen? ? headers : headers.dup.freeze
     end
 
     # The canonical kind name, e.g. "RateLimitError" (the class name without the module).

data/lib/email_validation_sdk/models.rb

@@ -53,13 +53,55 @@ module EmailValidation
   end
 
   # Counts by result status. In bulk jobs and webhook events the object is partial (only the statuses seen so far),
-  # so absent statuses are nil and are left out again by #to_h.
+  # so absent statuses are nil and are left out again by #to_h. A status this SDK version does not know yet is kept
+  # in #extra (a Hash of the API's key to its count) and re-encoded by #to_h and #to_json. +extra+ is not a Data
+  # member, so +members+, +to_a+ and array patterns (+in [valid, invalid, catch_all, unknown, disposable]+) are the
+  # five statuses, as in 1.0.0.
   StatusCounts = Data.define(:valid, :invalid, :catch_all, :unknown, :disposable) do
     include Model
 
+    def self.from_h(hash)
+      raise ArgumentError, "#{name} expects a Hash, got #{hash.class}" unless hash.is_a?(Hash)
+
+      values = members.to_h { |m| [m, hash.key?(m.to_s) ? hash[m.to_s] : hash[m]] }
+      extra = hash.each_with_object({}) { |(k, v), out| out[k.to_s] = v unless members.include?(k.to_sym) }
+      new(**values, extra: extra)
+    end
+
+    # Statuses this SDK version does not know: the API's key (a String) to its count. Empty when there are none.
+    attr_reader :extra
+
+    def initialize(valid: nil, invalid: nil, catch_all: nil, unknown: nil, disposable: nil, extra: {})
+      @extra = extra.to_h { |k, v| [k.to_s.freeze, v] }.freeze
+      super(valid: valid, invalid: invalid, catch_all: catch_all, unknown: unknown, disposable: disposable)
+    end
+
     def to_h
-      super.compact
+      super.compact.merge(extra.transform_keys(&:to_sym))
+    end
+
+    def with(**changes)
+      return self if changes.empty?
+
+      self.class.new(**members.to_h { |m| [m, public_send(m)] }, extra: extra, **changes)
+    end
+
+    def ==(other)
+      super && extra == other.extra
+    end
+
+    def eql?(other)
+      super && extra.eql?(other.extra)
+    end
+
+    def hash
+      [super, extra].hash
+    end
+
+    def inspect
+      extra.empty? ? super : super.sub(/>\z/, ", extra=#{extra.inspect}>")
     end
+    alias_method :to_s, :inspect
   end
 
   # The result of one check (+POST /v1/validate+). Fields in the API's order.

data/lib/email_validation_sdk/transport.rb

@@ -12,16 +12,25 @@ module EmailValidation
   class HttpRequest
     attr_reader :method, :url, :headers, :body, :timeout
 
-    # +headers+ is a Hash with lower-case names; +body+ is a binary String or nil; +timeout+ is seconds per attempt.
-    def initialize(method:, url:, headers:, body:, timeout:)
+    # +headers+ is a Hash with lower-case names; +body+ is a binary String or nil (a caller's binary String is passed
+    # as is, never copied: do not change it); +timeout+ is seconds per attempt; +transfer+ marks a file upload or a
+    # results download, for which +timeout+ is an idle timeout instead of a bound on the whole attempt.
+    def initialize(method:, url:, headers:, body:, timeout:, transfer: false)
       @method = method
       @url = url
       @headers = headers.freeze
-      @body = body&.freeze
+      @body = body
       @timeout = timeout
+      @transfer = transfer ? true : false
       freeze
     end
 
+    # True for a file upload or a results download: +timeout+ is then the longest time without a byte sent or
+    # received, with no cap on the whole transfer.
+    def transfer?
+      @transfer
+    end
+
     def inspect
       "#<#{self.class.name} #{method} #{url} headers=[hidden] body=[hidden]>"
     end
@@ -67,8 +76,11 @@ module EmailValidation
   end
 
   # The default transport, on the standard library's Net::HTTP. Redirects are never followed (Net::HTTP does not
-  # follow them), connections are kept alive and reused across threads through a small pool, and the client's
-  # +timeout+ bounds every attempt (connect, TLS, request and the whole response).
+  # follow them), connections are kept alive and reused across threads through a small pool, and Net::HTTP never
+  # resends a request by itself (+max_retries = 0+). The client's +timeout+ bounds every attempt of a JSON call
+  # (connect, TLS, request and the whole response); for a transfer (HttpRequest#transfer?) it is an idle timeout:
+  # connecting, each write of the upload and each read of the answer may take up to +timeout+, with no cap on the
+  # whole transfer, so a slow but progressing 100 MB upload or download is never cut.
   #
   # A transport is any object with +call(request)+ returning an HttpResponse (raise TransportError when no answer
   # arrives). Optional streaming: when +call+ is given a block, the transport calls it once with
@@ -80,7 +92,13 @@ module EmailValidation
       Net::HTTPBadResponse, Zlib::Error
     ].freeze
     TIMEOUT_ERRORS = [Net::OpenTimeout, Net::ReadTimeout, Net::WriteTimeout, Timeout::Error].freeze
-    private_constant :NETWORK_ERRORS, :TIMEOUT_ERRORS
+    # The kernel buffers megabytes of an upload: a write can wait long for room while those bytes still move, and
+    # after the last write they drain while Net::HTTP waits for the response head. Neither wait is seen as
+    # progress, so each gets +timeout + min(size, 4 MiB) / 128 KiB/s+; the body reads after the head
+    # are idle-bounded by +timeout+ again.
+    UPLOAD_RATE = 128 * 1024
+    DRAIN_CAP = 4 * 1024 * 1024
+    private_constant :NETWORK_ERRORS, :TIMEOUT_ERRORS, :UPLOAD_RATE, :DRAIN_CAP
 
     # Raised by the attempt deadline. A private class outside StandardError, so neither a timeout raised by other
     # code nor a +rescue => e+ in a results block can be confused with it or swallow it.
@@ -101,10 +119,16 @@ module EmailValidation
       conn = nil
       reusable = false
       begin
-        response = Timeout.timeout(request.timeout, AttemptTimeout) do
-          conn = checkout(uri, request.timeout)
-          exchange(conn, uri, request, on_head)
-        end
+        response = if request.transfer?
+                     conn = checkout(uri, request.timeout)
+                     allow_drain(conn, request) if request.body
+                     exchange(conn, uri, request, on_head)
+                   else
+                     Timeout.timeout(request.timeout, AttemptTimeout) do
+                       conn = checkout(uri, request.timeout)
+                       exchange(conn, uri, request, on_head)
+                     end
+                   end
         reusable = true
         response
       rescue AttemptTimeout, *TIMEOUT_ERRORS
@@ -140,6 +164,7 @@ module EmailValidation
       headers = nil
       body = nil
       conn.request(req) do |res|
+        conn.read_timeout = request.timeout # the head is in: each body read is idle-bounded by timeout again
         status = res.code.to_i
         headers = {}
         res.each_header { |name, value| headers[name] = value }
@@ -154,6 +179,12 @@ module EmailValidation
       HttpResponse.new(status: status, headers: headers, body: body)
     end
 
+    def allow_drain(conn, request)
+      wait = request.timeout + [request.body.bytesize, DRAIN_CAP].min.fdiv(UPLOAD_RATE)
+      conn.write_timeout = wait
+      conn.read_timeout = wait
+    end
+
     def build_request(uri, request)
       has_body = !request.body.nil?
       # Net::HTTPGenericRequest lets a body-less POST go out without a body or a default Content-Type.
@@ -170,6 +201,9 @@ module EmailValidation
       conn = Net::HTTP.new(uri.host, uri.port)
       conn.use_ssl = uri.scheme == "https"
       conn.max_retries = 0 # the client owns the retry policy (a POST that reached us may have been charged)
+      # Net::HTTP's default (true) ends a body cut short by a closed connection as if it were complete: a truncated
+      # result file or JSON answer would pass as whole. Off, a short body raises EOFError (a connection_error).
+      conn.ignore_eof = false
       conn.keep_alive_timeout = @keep_alive_timeout
       configure(conn, timeout)
       conn.start
@@ -206,3 +240,92 @@ module EmailValidation
     end
   end
 end
+
+module EmailValidation
+  # @api private
+  # Runs a caller-supplied transport on a short-lived worker thread, so the SDK itself bounds every attempt even when
+  # the transport ignores +request.timeout+. A JSON call gets a hard deadline of +timeout+. A
+  # transfer gets +timeout+ plus the time to upload its body at 128 KiB/s until the response head (the SDK cannot see
+  # a caller's upload progress), then +timeout+ for each body chunk. The bound is pull-based: it runs only while the
+  # SDK waits for the transport, never while the caller's block handles a chunk, and the worker can run at most one
+  # chunk ahead. On expiry TransportError(timeout) is raised, and the worker is not killed: a killed thread skips the
+  # transport's +rescue+ clauses, which left a kept-alive connection with an unread answer (the next call then read
+  # the previous call's answer, another address's result). It gets an Abandoned error raised instead, as a network
+  # error would, so a stack such as Net::HTTP closes the connection whose answer it did not read; whatever the
+  # transport does after that is discarded, and no SDK code runs on its thread any more.
+  module TransportGuard
+    UPLOAD_RATE = 128 * 1024 # bytes per second
+    private_constant :UPLOAD_RATE
+
+    # Raised in an abandoned worker. A StandardError, so the transport's own error handling runs (Net::HTTP closes
+    # its socket on one).
+    class Abandoned < StandardError
+      def initialize(message = "The SDK stopped waiting for this request (timeout).")
+        super
+      end
+    end
+
+    module_function
+
+    def call(transport, request, stream)
+      events = SizedQueue.new(1)
+      worker = Thread.new { run(transport, request, stream, events) }
+      worker.report_on_exception = false
+      deadline = now + first_bound(request)
+      sink = nil
+      finished = false
+      loop do
+        event = events.pop(timeout: [deadline - now, 0].max)
+        raise TransportError.new("The request timed out.", timeout: true) if event.nil?
+
+        kind, value = event
+        case kind
+        when :head then sink = value
+        when :chunk then sink.call(value)
+        when :done
+          finished = true
+          return value
+        else
+          finished = true
+          raise value
+        end
+        deadline = now + request.timeout if request.transfer? # idle: counted from the end of the caller's block
+      end
+    ensure
+      events&.close
+      worker.raise(Abandoned) if worker&.alive? && !finished
+    end
+
+    def run(transport, request, stream, events)
+      response = if stream
+                   transport.call(request) do |status, headers|
+                     raise Abandoned if events.closed? # the caller gave up: no SDK code on this thread any more
+
+                     sink = stream.on_head(status, headers)
+                     events.push([:head, sink])
+                     sink && ->(chunk) { events.push([:chunk, chunk]) }
+                   end
+                 else
+                   transport.call(request)
+                 end
+      events.push([:done, response])
+    rescue Exception => e # rubocop:disable Lint/RescueException -- handed to the caller's thread unchanged
+      begin
+        events.push([:error, e])
+      rescue ClosedQueueError
+        nil # the caller gave up (timeout) and no longer listens
+      end
+    end
+
+    def first_bound(request)
+      return request.timeout unless request.transfer?
+
+      request.timeout + (request.body ? request.body.bytesize.fdiv(UPLOAD_RATE) : 0)
+    end
+
+    def now
+      Process.clock_gettime(Process::CLOCK_MONOTONIC)
+    end
+  end
+  private_constant :TransportGuard
+end

data/lib/email_validation_sdk/version.rb

@@ -2,5 +2,5 @@
 
 module EmailValidation
   # SDK version (Semantic Versioning). Sent in the User-Agent header.
-  VERSION = "1.0.0"
+  VERSION = "1.1.0"
 end

data/lib/email_validation_sdk/webhooks.rb

@@ -44,8 +44,9 @@ module EmailValidation
 
       body = bytes(payload)
       expected = OpenSSL::HMAC.digest("SHA256", secret_key(secret), "#{msg_id}.#{timestamp}.".b + body)
-      signatures.split(/ /).each do |entry|
-        version, signature = entry.split(",", 2)
+      # Runs of spaces separate entries; one trailing "," is stripped, so values a stack joined with ", " still verify.
+      signatures.split(/ +/).each do |entry|
+        version, signature = entry.delete_suffix(",").split(",", 2)
         next unless version == "v1" && signature && !signature.empty?
 
         given = decode64(signature)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: email-validation-sdk
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - email-validation.org