nohead 0.8.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: aaf4c7fc6c9f1c9783210f08d5d42cd01aa7ae30e3b30f3398c5bc5eb252c960
-  data.tar.gz: afd201145f4734efbb695c717b8073c5f100fac11fdec822467dc4345ea7a1de
+  metadata.gz: 68e5883cc4d13b186ac691a79aeb84b5a10462c2e350ddbffe17cd832ffc6d30
+  data.tar.gz: 702a971c87efbda7fdc438410984f93fd7f315ca655958e6510b6027ca970089
 SHA512:
-  metadata.gz: ff59ead7b5f84026e63d42fd7751c272b7eff489fad447f0bd397f00d21ab57cb475fd989c0bca218efdcc5abcfdf44e681a91a2a0c0c61252b508d08a6e09c4
-  data.tar.gz: 21afb854940c2fdda4181b0c8a03e9accd46127e16fe2f7455f17f0fb0e6ab1b8f7033ac414cee25917c599d59d832ae6f7e185759f8571b15588ebc5d9a300c
+  metadata.gz: f8f38c6c8844668251282d375d1f20373621460aefb0ab340beffa29c8bb000fc38025b17746fae7b7dde10f7007dee92806240e3f0de996774e7ebfeef6e234
+  data.tar.gz: 387779fad0fd93d5eb68e4956e434ee2eedbb70ad0f00f1fbc80f23b8d34891eaf8338d27c6b97927432951846d85aa7ea921065b741043a185bf2903a1d03d4

data/CHANGELOG.md

@@ -5,6 +5,96 @@ Changes to the `nohead` gem that you can notice. Versions follow
 releases; a change that could break your code is a major one. Each release's
 section is its GitHub release's notes.
 
+## 0.9.0
+
+- **Breaking:** the client no longer reads `NOHEAD_API_URL`: a stray value
+  sent requests, with the API key, to another host. The base URL is
+  `base_url:`, else `https://api.nohead.io`; pass `base_url:` to use another
+  API, as in `Nohead::Client.new(base_url: ENV.fetch("NOHEAD_API_URL"))`.
+- A request is sent once per attempt. Net::HTTP retried a GET, PUT or DELETE
+  that timed out or lost its connection once more on its own, so a stalled
+  request waited twice the `timeout:`, `max_retries:` sent more attempts than
+  asked (unseen by `on_retry:`), and a stalled upload was sent again with its
+  stream already read.
+- `Images.url` writes the same URL as the other SDKs for the same image and
+  options: a whole-number Float is written as an integer (`dpr: 2.0` is
+  `dpr:2`, not `dpr:2.0`), and `dpr: 0` or `quality: 0` is left out rather
+  than written as `dpr:0` or `q:0`.
+- Deprecation and plan-usage warnings are given once per process, as in the
+  other SDKs, rather than once per client: an app that builds a client per
+  request no longer repeats them on every request.
+- `assets.upload` of a stream that cannot seek, such as a pipe or `$stdin`,
+  with `byte_size:` no longer raises `Errno::ESPIPE`. Its bytes are sent
+  once: a storage error is not retried, since they can't be read again.
+- `Webhooks.unwrap` reads headers, body and secret as bytes. A non-ASCII
+  `webhook-id` with a binary body (as Rack passes it) no longer raises
+  `Encoding::CompatibilityError`, a secret in another encoding raises
+  `WebhookVerificationError`, and a header that is not valid UTF-8 is refused
+  for its signature or timestamp, not as "The webhook secret is not base64".
+- `Webhooks.unwrap` refuses an unset or empty secret with
+  `WebhookVerificationError` ("The webhook secret is empty"), instead of
+  `NoMethodError` or a signature mismatch, and verifies a repeated
+  `webhook-signature` header given as an Array.
+- A path value of `"."` or `".."` (an ID such as `records.get(".")`) raises
+  `Nohead::Error` (`Invalid record_id: "."`) before anything is sent, as
+  `nil` or `""` does, instead of reaching a URL that a proxy could resolve to
+  another path.
+- An empty string is the same as leaving an option out, as in the other
+  SDKs: `project_id: ""` looks the project up, `idempotency_key: ""`
+  generates a key (it was sent empty) and `change_note: ""` sends no
+  `Nohead-Change-Note` header.
+- `base_url:` loses every trailing slash (it lost one), and a base URL that
+  isn't `http` or `https` raises `Nohead::Error` ("base_url must be an http or
+  https URL") when the client is built, instead of `NoMethodError` at the
+  first request.
+- Headers are sent once per name, whatever its case: the client's
+  `headers:` replace the SDK's own, and the ones request options set
+  (`Content-Type`, `Idempotency-Key`, `If-Match`, `Nohead-Change-Note`)
+  replace both. A custom `transport:` used to get both spellings.
+- `if_match:` takes a Hash with a revision (`{ revision: 3 }`) as well as a
+  revision number or a record or draft; anything else, such as `"3"`, raises
+  `ArgumentError` (a Hash or String was sent as it was written).
+- The project lookup (`GET /v1/me`, without `project_id:`) runs once when
+  several threads need it at the same time.
+- `search` with both `collections:` and a `collection` filter searches
+  `collections:`, as the other SDKs do (the filter won).
+- `APIError#api_message` is the API's own message, without the status, type
+  and request ID that `message` adds.
+- A redirect is an `APIError` whose message says to check the base URL
+  (`301 redirect: The API answered with a redirect to <Location>; check the
+  base URL`), instead of "Request failed with status 301".
+- A malformed response (a broken status line or header,
+  `Net::HTTPBadResponse`, `Net::HTTPHeaderSyntaxError`, `Net::ProtocolError`,
+  or a broken compressed body) is a `ConnectionError`, retried like a
+  dropped connection, instead of escaping as Net::HTTP's own error.
+- A list call whose response isn't a page raises `Nohead::Error` ("The API's
+  response could not be read (HTTP 200)") instead of `KeyError` or
+  `NoMethodError`.
+- `Retry-After` is read with spaces around it (`" 5"`); anything but whole or
+  decimal seconds or an HTTP date is ignored, as before.
+- Error details that aren't objects are kept, with the code `"unknown"` and
+  the value as their message, instead of left out.
+- An empty `Deprecation` header no longer warns.
+- `on_retry:`, a client option, is called before each retry's wait (of
+  uploads' bytes too) with a `Nohead::RetryEvent`: the attempt, the delay in
+  seconds, the URL, and the status or error that failed, as the other SDKs'
+  retry callbacks are.
+- `with_options(timeout:, max_retries:, headers:, on_retry:)` returns a copy
+  of the client with other settings, sharing its project lookup.
+- `timeout:` is documented as what it is: the longest wait without progress
+  (connecting, waiting for the response, between reads and writes of a
+  body), not a limit on the whole attempt.
+- `assets.upload` raises `UploadError` ("Could not upload a.png: <reason>",
+  with the `ConnectionError` as its `cause`) when storage can't be reached
+  or times out, as it does when storage refuses the bytes, instead of
+  `ConnectionError` or `TimeoutError`.
+- `assets.upload` with `filename: ""` or `content_type: ""` uses the
+  defaults (the file's name, else `upload`; the extension's type, else
+  `application/octet-stream`) instead of sending them empty.
+- `Images.url` leaves out options given as `""` (`gravity: ""`,
+  `format: ""`...) instead of writing `g:` or a trailing `@`, and `fit: nil`
+  (or `""`) means the default `contain` instead of raising `ArgumentError`.
+
 ## 0.8.0
 
 - Assets have `file_url`, the file as uploaded, which downloads under its

data/README.md

@@ -41,24 +41,36 @@ gem "nohead"
 
 It needs Ruby 3.3 or newer. It uses only Ruby's standard library (Net::HTTP, JSON, OpenSSL). Use it on servers: API keys are secrets.
 
+Until 1.0, any release may break your code: pin a version, and read the changelog's **Breaking** entries before upgrading.
+
 ## Configuration
 
 ```ruby
 nohead = Nohead::Client.new(
-  api_key: ENV.fetch("NOHEAD_API_KEY"), # default: NOHEAD_API_KEY
-  base_url: "https://api.nohead.io" # default: NOHEAD_API_URL, else production
+  api_key: ENV.fetch("NOHEAD_API_KEY"), # the default
+  timeout: 10,
+  max_retries: 3
 )
 ```
 
 | Option | Default | |
 |---|---|---|
 | `api_key:` | `NOHEAD_API_KEY` | A project API key (`nohead_sk_...`, or `nohead_pk_...` when published-only). Required. |
-| `base_url:` | `NOHEAD_API_URL`, else `https://api.nohead.io` | |
+| `base_url:` | `https://api.nohead.io` | For a proxy, or a mock server in tests. |
 | `project_id:` | the key's project | Looked up once with `GET /v1/me` when omitted. |
 | `max_retries:` | `2` | See [retries](#retries-and-idempotency). |
-| `timeout:` | `60` | Seconds per attempt. |
+| `timeout:` | `60` | The longest wait in seconds without progress, per attempt: connecting, waiting for the response, and between reads and writes of a body. A response that keeps arriving doesn't time out. |
 | `headers:` | `{}` | Added to every request. |
 | `warnings:` | `true` | Warns about deprecated operations and plan usage, once each. |
+| `on_retry:` | | Called before each retry's wait. See [retries](#retries-and-idempotency). |
+
+An empty string is the same as leaving an option out (`api_key: ""` reads `NOHEAD_API_KEY`, `base_url: ""` uses `https://api.nohead.io`). The base URL comes only from `base_url:`, never from the environment. A `base_url:` that isn't `http` or `https` raises `Nohead::Error`.
+
+`with_options` gives a copy of the client with other settings (`timeout:`, `max_retries:`, `headers:`, added to the client's, and `on_retry:`), sharing its project lookup:
+
+```ruby
+nohead.with_options(timeout: 5, max_retries: 0).records.get(id)
+```
 
 API keys belong to a project, so methods like `collections.list` need no project ID. Collections can be named by ID or slug everywhere.
 
@@ -132,7 +144,7 @@ nohead.records.list("products", filter: { price: { lt: 50 }, category: { in: %w[
 
 ## Errors
 
-Every error is a `Nohead::Error`. API errors are `Nohead::APIError`s with `status`, `type`, `message`, `request_id`, `details` and `headers`, in a class per type:
+Every error is a `Nohead::Error`. API errors are `Nohead::APIError`s with `status`, `type`, `message`, `api_message` (the API's own message, without the status, type and request ID), `request_id`, `details` and `headers`, in a class per type:
 
 | Class | Status |
 |---|---|
@@ -148,6 +160,8 @@ Every error is a `Nohead::Error`. API errors are `Nohead::APIError`s with `statu
 | `InternalServerError` | 500 and other 5xx |
 | `ServiceUnavailableError` | 503 |
 
+A redirect is an `APIError` too ("301 redirect: The API answered with a redirect to …; check the base URL"): the SDK doesn't follow redirects.
+
 Other errors:
 
 - `Nohead::ConnectionError`, and `Nohead::TimeoutError`, which is a kind of `ConnectionError`
@@ -167,7 +181,13 @@ end
 Failed requests are retried twice by default (`max_retries:`), with exponential backoff:
 
 - what's retried: connection errors, timeouts, 429, 500, 502, 503, 504, and a 409 for a request that is still running
-- `Retry-After` is honored up to 60 seconds; a longer one raises `RateLimitError` straight away
+- `Retry-After` (seconds, or an HTTP date) is honored up to 60 seconds; a longer one raises `RateLimitError` straight away
+
+`on_retry:` hears of each retry before its wait, uploads' too, as a `Nohead::RetryEvent` (`attempt`, from 1; `delay`, in seconds; `url`; `status`, nil when the connection failed; and `error`):
+
+```ruby
+nohead = Nohead::Client.new(on_retry: ->(event) { logger.warn("Retry #{event.attempt} of #{event.url} in #{event.delay} s") })
+```
 
 Every write gets an `Idempotency-Key` that stays the same across its retries, so a retry after a lost response never writes twice. To make a write safe across your own retries (a job that may run twice), pass a key:
 
@@ -190,14 +210,13 @@ rescue Nohead::PreconditionFailedError => e
 end
 ```
 
-`if_match:` takes a record (or draft) or a revision number, on `records.update`, `delete`, `restore`, `publish`, `unpublish` and `revisions.revert`, and on `drafts.update`, `discard` and `merge`.
+`if_match:` takes a record (or draft), a revision number or `{ revision: 3 }`, on `records.update`, `delete`, `restore`, `publish`, `unpublish` and `revisions.revert`, and on `drafts.update`, `discard` and `merge`.
 
 ## Assets
 
 ```ruby
 asset = nohead.assets.upload("cover.jpg")
 nohead.records.update(id, data: { cover: asset.id })
-
 ```
 
 **Images.** Image URLs are open: an image's `image_url` serves the whole image, and any of [imgproxy's processing options](https://docs.imgproxy.net/usage/processing) written into it transform it. `Nohead::Images.url` builds them without a request:
@@ -217,18 +236,18 @@ Thumbnails: an image's `preview_url` fits in 640×640, as WebP.
 2. Streams the bytes straight to storage.
 3. Completes the upload, which checks the file, and returns the `ready` asset.
 
-**Errors:** `UploadError` if storage refuses the bytes; `ValidationError` if the file fails the checks.
+**Errors:** `UploadError` if the bytes don't reach storage: refused (`Storage refused the upload of cover.jpg (403)`, with its `status`), or unreachable or timed out (`Could not upload cover.jpg: …`, with the `ConnectionError` as its `cause`). `ValidationError` if the file fails the checks.
 
 **Resuming:** with an `idempotency_key:`, running the same upload again resumes it: the key starts the upload, and an asset that is already `ready` comes back without sending the bytes again. Once the upload's URL has expired (its `upload.expires_at`), it raises `UploadError`: start over with a new key.
 
-**Uploading from a browser:** create the upload on your server with `create_upload`, `PUT` the file from the browser, then `complete` it.
-
 **Renaming and alt text:** `update` changes an asset's `filename:` and `alt:` text, the default wherever the asset is used (`expand:` returns it with the asset). The file itself never changes.
 
 ```ruby
 nohead.assets.update(asset.id, alt: "The spring menu, on a chalkboard")
 ```
 
+**Deleting:** `usage` shows which records use an asset. `delete` keeps the file until its `purge_after`, during which `restore` undoes it; `purge` removes a deleted asset for good, now.
+
 ## Search
 
 ```ruby
@@ -320,20 +339,32 @@ bundle exec rake generate   # after updating openapi.json
 bundle exec rake samples    # after changing test/calls.rb (the docs' code samples)
 ```
 
+Before pushing, run what CI runs: the tests, RuboCop, and a check that the generated files are up to date:
+
+```bash
+bundle exec rake test
+bundle exec rubocop
+bundle exec rake generate samples && git diff --exit-code   # with your changes committed: a diff is a stale file
+gem build nohead.gemspec
+```
+
 **How the code is organized:**
 
 - `openapi.json` is the API's published contract. `rake generate` derives the operation table, `lib/nohead/operations.rb`, from it.
 - The methods are written by hand.
 - `test/contract_test.rb` runs every call in `test/calls.rb`. It fails when an API-key operation in the contract has no method, when a query parameter is never sent (no method takes it, or no call passes it), or when a request doesn't match its operation (method, path, query parameters and JSON body, against their schemas). Each call gets an example of its operation's response, built from the contract's schema, and must return it.
+- The unit tests answer with `test/fake_transport.rb`, which checks every request to the API and every reply against `openapi.json`, so a fixture can't drift from the API. Wrap a reply that breaks the contract on purpose (a gateway's HTML page, an unknown error type) in `raw(...)`.
+- `test/network_test.rb` runs the real transport (Net::HTTP) against an HTTP server on 127.0.0.1, `test/local_server.rb`, for what goes over the wire: lengths, timeouts, redirects and broken replies.
+- A test that makes no assertions fails.
 
-The smoke test (`smoke/smoke.rb`) runs the core flow against a real API, with the gem as installed from its built package. Nohead's own CI runs it on every API contract change.
+The smoke test (`smoke/smoke.rb`) runs the core flow against a real API, with the gem as installed from its built package. The Nohead API repository runs it in its end-to-end tests and before every release.
 
 ## Releasing
 
-1. Bump `lib/nohead/version.rb`.
+1. Bump `lib/nohead/version.rb`, then `bundle install` (`Gemfile.lock` records the version, and CI installs frozen).
 2. Add a section for the version to `CHANGELOG.md` (`## 1.2.3`), which becomes the release's notes.
 3. Merge to `main`. Its ruleset requires the **CI passed** check, so the commit goes through a pull request or a branch whose CI passed, and force pushes are refused.
-4. Run the **SDK release** workflow in the Nohead API repository. It runs this commit's smoke test against the API and pushes the tag `v1.2.3`. Nobody else can push `v*` tags: a tag ruleset lets only that workflow's deploy key through.
+4. Run the **SDK release** workflow in the Nohead API repository. It runs this commit's smoke test against the API and pushes the tag `v1.2.3`. A tag ruleset lets only deploy keys push `v*` tags.
 5. The tag starts `.github/workflows/release.yml`. A build job checks the tag is on `main` and matches the version and its notes, and packs the gem, with no publishing credential. Its publishing job runs in the `release` environment, which only `v*` tags can use, and the registry's trusted publisher accepts only that environment. It pushes that gem to RubyGems through trusted publishing (no API key), then creates the GitHub release with the gem.
 
 ## License

data/lib/nohead/client.rb

@@ -6,66 +6,100 @@ require "time"
 require "uri"
 
 module Nohead
+  # A retry about to happen, as `on_retry:` gets it: `attempt` is 1 for a
+  # request's first retry, `delay` the seconds the client waits before it,
+  # `url` the URL retried, `status` the status that failed (nil when the
+  # connection did), and `error` the APIError or ConnectionError (nil for
+  # a status from storage).
+  RetryEvent = Data.define(:attempt, :delay, :url, :status, :error)
+
   # A client for the Nohead API, authenticated with a project API key.
   #
   #   nohead = Nohead::Client.new # NOHEAD_API_KEY
   #   nohead.records.list("posts").each { |post| puts post.data["title"] }
   class Client
     DEFAULT_BASE_URL = "https://api.nohead.io"
+
+    # Each warning is given once per process, as the other SDKs do: an app
+    # that builds a client per request would otherwise warn on every one.
+    WARNED = Set.new
+    WARNED_LOCK = Mutex.new
+    private_constant :WARNED_LOCK
+
+    def self.first_warning?(key) = WARNED_LOCK.synchronize { !WARNED.add?(key).nil? }
     RETRYABLE_STATUSES = [429, 500, 502, 503, 504].freeze
     MAX_RETRY_AFTER = 60
 
     attr_reader :records, :revisions, :drafts, :collections, :fields, :migrations, :assets, :webhooks,
                 :audit_events, :feature_flags, :me, :health
 
+    # The key's project ID, looked up once, and once at a time, for a client
+    # and its copies (with_options).
+    class ProjectLookup
+      def initialize(id)
+        @id = id
+        @lock = Mutex.new
+      end
+
+      def get
+        @id || @lock.synchronize { @id ||= yield }
+      end
+    end
+    private_constant :ProjectLookup
+
     # api_key::     A project API key ("nohead_sk_...", or "nohead_pk_..." when
     #               published-only). Defaults to NOHEAD_API_KEY.
-    # base_url::    Defaults to NOHEAD_API_URL, else https://api.nohead.io.
+    # base_url::    The API's address, an http or https URL. Defaults to
+    #               https://api.nohead.io (the environment isn't read for it).
     # project_id::  The key's project. Looked up once with GET /v1/me when omitted.
     # max_retries:: Retries of failed requests (see the README, "Retries").
-    # timeout::     Seconds per attempt.
+    # timeout::     The longest wait in seconds without progress, per attempt:
+    #               connecting, waiting for the response, and between reads
+    #               and writes of a body. A response that keeps arriving
+    #               doesn't time out.
     # headers::     Headers added to every request.
     # warnings::    Warn about deprecated operations and plan usage (once each).
+    # on_retry::    Called with a RetryEvent before each retry's wait, for
+    #               example to log it.
+    #
+    # An empty string is the same as leaving an option out.
     def initialize(api_key: nil, base_url: nil, project_id: nil, max_retries: 2, timeout: 60,
-                   headers: {}, warnings: true, transport: Transport.new)
+                   headers: {}, warnings: true, on_retry: nil, transport: Transport.new)
       @api_key = present(api_key) || present(ENV.fetch("NOHEAD_API_KEY", nil)) or
         raise Error, "Missing API key: pass api_key or set NOHEAD_API_KEY"
-      @base_url = (present(base_url) || present(ENV.fetch("NOHEAD_API_URL", nil)) ||
-                   DEFAULT_BASE_URL).chomp("/")
-      @project_id = project_id
+      @base_url = checked_base_url(present(base_url) || DEFAULT_BASE_URL)
+      @project = ProjectLookup.new(present(project_id))
       @max_retries = max_retries
       @timeout = timeout
-      @headers = headers.to_h { |key, value| [key.to_s, value.to_s] }
+      @headers = merge_headers({}, headers.to_h)
       @warnings = warnings
+      @on_retry = on_retry
       @transport = transport
-      @warned = {}
+      build_resources
+    end
 
-      @records = Resources::Records.new(self)
-      @revisions = Resources::RevisionsById.new(self)
-      @drafts = Resources::Drafts.new(self)
-      @collections = Resources::Collections.new(self)
-      @fields = Resources::Fields.new(self)
-      @migrations = Resources::Migrations.new(self)
-      @assets = Resources::Assets.new(self)
-      @webhooks = Resources::Webhooks.new(self)
-      @audit_events = Resources::AuditEvents.new(self)
-      @feature_flags = Resources::FeatureFlags.new(self)
-      @me = Resources::Me.new(self)
-      @health = Resources::Health.new(self)
+    # A copy of the client with other settings, which shares its transport
+    # and project lookup: `nohead.with_options(timeout: 5).records.get(id)`.
+    # `headers:` are added to the client's.
+    def with_options(timeout: @timeout, max_retries: @max_retries, headers: {}, on_retry: @on_retry)
+      dup.tap { |copy| copy.configure(timeout, max_retries, merge_headers(@headers, headers.to_h), on_retry) }
     end
 
     # Full-text search across the key's project (or some of its collections),
     # most relevant first, through the first 1,000 hits. `filter` takes
     # `published_at` ({ published_at: { exists: true } } for published
-    # records), as records.list does.
+    # records), as records.list does. `collections` wins over a `collection`
+    # filter.
     def search(query, collections: nil, filter: nil, limit: nil, cursor: nil)
-      filter = { collection: (collections.nil? ? nil : { in: collections }), **filter.to_h.transform_keys(&:to_sym) }
+      filter = filter.to_h.transform_keys(&:to_sym)
+      filter = filter.merge(collection: { in: collections }) unless collections.nil?
       paginate("projects_search", query: { q: query, filter: filter, limit: limit, cursor: cursor })
     end
 
-    # The API key's project, from GET /v1/me the first time.
+    # The API key's project, from GET /v1/me the first time. Threads that
+    # need it at the same time wait for one lookup.
     def project_id
-      @project_id ||= begin
+      @project.get do
         me = request("me_get")
         me.api_key or raise Error, "The credentials are not a project API key"
         me.api_key.project_id
@@ -77,9 +111,8 @@ module Nohead
     # does not wrap yet.
     def request(operation, path: {}, query: {}, body: nil, idempotency_key: nil,
                 change_note: nil, if_match: nil)
-      NoheadObject.wrap(send_request(operation, path: path, query: query, body: body,
-                                                idempotency_key: idempotency_key,
-                                                change_note: change_note, if_match: if_match))
+      NoheadObject.wrap(exchange(operation, path: path, query: query, body: body, idempotency_key: idempotency_key,
+                                            change_note: change_note, if_match: if_match).first)
     end
 
     # A list operation as a Page that fetches the following pages as needed.
@@ -87,9 +120,12 @@ module Nohead
       params = query.dup
       cursor = params.delete(:cursor)
       fetch = lambda do |next_cursor|
-        list = send_request(operation, path: path, query: params.merge(cursor: next_cursor))
-        Page.new(NoheadObject.wrap(list.fetch("data")), NoheadObject.wrap(list.fetch("meta")),
-                 &fetch)
+        list, status = exchange(operation, path: path, query: params.merge(cursor: next_cursor))
+        unless list.is_a?(Hash) && list["data"].is_a?(Array) && list["meta"].is_a?(Hash)
+          raise Error, "The API's response could not be read (HTTP #{status})"
+        end
+
+        Page.new(NoheadObject.wrap(list["data"]), NoheadObject.wrap(list["meta"]), &fetch)
       end
       fetch.call(cursor)
     end
@@ -104,10 +140,14 @@ module Nohead
                                      headers.merge("Content-Length" => source.byte_size.to_s),
                                      source.io, @timeout)
           return response unless RETRYABLE_STATUSES.include?(response.status) && attempt < retries
-        rescue ConnectionError
+
+          failed = { status: response.status }
+        rescue ConnectionError => e
           raise if attempt >= retries
+
+          failed = { error: e }
         end
-        pause(backoff(attempt))
+        wait_to_retry(url, attempt, backoff(attempt), **failed)
         attempt += 1
         source.rewind
       end
@@ -115,10 +155,41 @@ module Nohead
 
     def inspect = "#<Nohead::Client #{@base_url}>"
 
+    protected
+
+    # with_options' settings, on its copy.
+    def configure(timeout, max_retries, headers, on_retry)
+      @timeout = timeout
+      @max_retries = max_retries
+      @headers = headers
+      @on_retry = on_retry
+    end
+
     private
 
-    def send_request(operation, path: {}, query: {}, body: nil, idempotency_key: nil,
-                     change_note: nil, if_match: nil)
+    def initialize_copy(source)
+      super
+      build_resources # theirs send through `source`
+    end
+
+    def build_resources
+      @records = Resources::Records.new(self)
+      @revisions = Resources::RevisionsById.new(self)
+      @drafts = Resources::Drafts.new(self)
+      @collections = Resources::Collections.new(self)
+      @fields = Resources::Fields.new(self)
+      @migrations = Resources::Migrations.new(self)
+      @assets = Resources::Assets.new(self)
+      @webhooks = Resources::Webhooks.new(self)
+      @audit_events = Resources::AuditEvents.new(self)
+      @feature_flags = Resources::FeatureFlags.new(self)
+      @me = Resources::Me.new(self)
+      @health = Resources::Health.new(self)
+    end
+
+    # The parsed body of a 2xx response, and its status.
+    def exchange(operation, path: {}, query: {}, body: nil, idempotency_key: nil,
+                 change_note: nil, if_match: nil)
       method, template = OPERATIONS.fetch(operation)
       url = build_url(template, path) + query_string(query)
       headers = request_headers(method, body, idempotency_key, change_note, if_match)
@@ -127,23 +198,23 @@ module Nohead
       loop do
         begin
           response = @transport.call(method, url, headers, payload, @timeout)
-        rescue ConnectionError
+        rescue ConnectionError => e
           raise if attempt >= @max_retries
 
-          pause(backoff(attempt))
+          wait_to_retry(url, attempt, backoff(attempt), error: e)
           attempt += 1
           next
         end
 
         warn_about(operation, response.headers)
         parsed = parse(response)
-        return parsed if response.status.between?(200, 299)
+        return [parsed, response.status] if response.status.between?(200, 299)
 
         error = Nohead.api_error(response.status, parsed, response.headers)
         delay = retry_delay(error, attempt)
         raise error if delay.nil?
 
-        pause(delay)
+        wait_to_retry(url, attempt, delay, status: response.status, error: error)
         attempt += 1
       end
     end
@@ -156,10 +227,13 @@ module Nohead
 
       values["project_id"] ||= project_id if template.include?("{project_id}")
       path = template.gsub(/\{(\w+)\}/) do
-        value = values[Regexp.last_match(1)]
-        raise Error, "Missing #{Regexp.last_match(1)}" if value.nil? || value.to_s.empty?
+        name = Regexp.last_match(1)
+        value = values[name]
+        raise Error, "Missing #{name}" if value.nil? || value.to_s.empty?
+        # Encoded, "." and ".." would still be resolved as the path's own segments.
+        raise Error, %(Invalid #{name}: "#{value}") if %w[. ..].include?(value.to_s)
 
-        URI.encode_www_form_component(value.to_s).gsub("+", "%20")
+        URI.encode_uri_component(value.to_s)
       end
       @base_url + path
     end
@@ -196,33 +270,69 @@ module Nohead
       case value
       when Hash then value.to_h { |key, inner| [key.to_s, json_ready(inner)] }
       when Array then value.map { |item| json_ready(item) }
-      when Time, DateTime then value.to_time.utc.iso8601(3)
-      when Date then value.iso8601
+      when Time, DateTime, Date then scalar(value)
       when Symbol then value.to_s
       else value
       end
     end
 
+    # The SDK's headers, then the client's `headers:`, then the ones request
+    # options set, which win.
     def request_headers(method, body, idempotency_key, change_note, if_match)
-      headers = {
-        "Accept" => "application/json",
-        "Authorization" => "Bearer #{@api_key}",
-        "Nohead-Client" => "sdk-ruby/#{VERSION}",
-        "User-Agent" => "nohead-ruby/#{VERSION} ruby/#{RUBY_VERSION}"
-      }.merge(@headers)
-      headers["Content-Type"] = "application/json" unless body.nil?
-      headers["Idempotency-Key"] = idempotency_key || SecureRandom.uuid unless method == "GET"
-      unless if_match.nil?
-        revision = if_match.respond_to?(:revision) ? if_match.revision : if_match
-        headers["If-Match"] = %("#{revision}")
-      end
+      headers = merge_headers({
+                                "Accept" => "application/json",
+                                "Authorization" => "Bearer #{@api_key}",
+                                "Nohead-Client" => "sdk-ruby/#{VERSION}",
+                                "User-Agent" => "nohead-ruby/#{VERSION} ruby/#{RUBY_VERSION}"
+                              }, @headers)
+      options = {}
+      options["Content-Type"] = "application/json" unless body.nil?
+      options["Idempotency-Key"] = present(idempotency_key) || SecureRandom.uuid unless method == "GET"
+      options["If-Match"] = %("#{revision_of(if_match)}") unless if_match.nil?
       # Percent-encoded UTF-8, as encodeURIComponent: header values are bytes
       # (the API decodes it). Notes in other encodings are converted first.
-      if change_note
+      if present(change_note)
         note = change_note.to_s.encode(Encoding::UTF_8, invalid: :replace, undef: :replace)
-        headers["Nohead-Change-Note"] = URI.encode_uri_component(note)
+        options["Nohead-Change-Note"] = URI.encode_uri_component(note)
+      end
+      merge_headers(headers, options)
+    end
+
+    # `extra` over `headers`, matching names without regard to case, so each
+    # name is sent once.
+    def merge_headers(headers, extra)
+      extra.each_with_object(headers.dup) do |(name, value), merged|
+        name = name.to_s
+        merged.delete_if { |key, _| key.casecmp?(name) }
+        merged[name] = value.to_s
       end
-      headers
+    end
+
+    # A revision number, or a record or draft (anything with a revision, a
+    # Hash too).
+    def revision_of(if_match)
+      revision = if if_match.is_a?(Hash)
+                   if_match.key?(:revision) ? if_match[:revision] : if_match["revision"]
+                 else
+                   if_match.respond_to?(:revision) ? if_match.revision : if_match
+                 end
+      return revision if revision.is_a?(Integer)
+
+      raise ArgumentError, "if_match takes a revision number, or a record, draft or hash with a revision"
+    end
+
+    def checked_base_url(url)
+      # A loop rather than a regex, which backtracks on long runs of slashes.
+      url = url.to_s
+      stop = url.length
+      stop -= 1 while stop.positive? && url[stop - 1] == "/"
+      url = url[0, stop]
+      uri = URI.parse(url)
+      return url if %w[http https].include?(uri.scheme&.downcase) && !uri.host.to_s.empty?
+
+      raise Error, "base_url must be an http or https URL"
+    rescue URI::InvalidURIError
+      raise Error, "base_url must be an http or https URL"
     end
 
     def parse(response)
@@ -252,6 +362,11 @@ module Nohead
       [0.5 * (2**attempt), 8.0].min * (1 - (rand * 0.25))
     end
 
+    def wait_to_retry(url, attempt, delay, status: nil, error: nil)
+      @on_retry&.call(RetryEvent.new(attempt: attempt + 1, delay: delay, url: url, status: status, error: error))
+      pause(delay)
+    end
+
     def pause(seconds)
       sleep(seconds) if seconds.positive?
     end
@@ -259,15 +374,13 @@ module Nohead
     def warn_about(operation, headers)
       return unless @warnings
 
-      if headers["deprecation"] && !@warned[operation]
-        @warned[operation] = true
+      if !headers["deprecation"].to_s.strip.empty? && self.class.first_warning?(operation)
         sunset = headers["sunset"] ? " and will be removed after #{headers['sunset']}" : ""
         link = headers["link"].to_s[/<([^>]+)>/, 1]
         warn "[nohead] #{operation} is deprecated#{sunset}#{". See #{link}" if link}"
       end
-      return unless headers["nohead-usage-warning"] && !@warned[:usage]
+      return unless headers["nohead-usage-warning"] && self.class.first_warning?(:usage)
 
-      @warned[:usage] = true
       warn "[nohead] Over a plan limit: #{headers['nohead-usage-warning']}"
     end
 

data/lib/nohead/errors.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require "json"
+
 module Nohead
   # The base of every error the SDK raises.
   class Error < StandardError; end
@@ -18,22 +20,36 @@ module Nohead
     attr_reader :headers
     # The parsed response body, or its text when it was not JSON.
     attr_reader :body
+    # The API's own message, without the status, type and request ID that
+    # `message` adds.
+    attr_reader :api_message
 
     def initialize(status, body, headers)
-      error = Nohead.error_envelope(body)
+      # The SDK doesn't follow redirects: one means the base URL is off.
+      error = redirect?(status) ? {} : Nohead.error_envelope(body)
       @status = status
-      @type = error["type"]
+      @type = redirect?(status) ? "redirect" : error["type"]
       @request_id = error["request_id"] || headers["x-request-id"]
       @details = Array(error["details"]).map { |detail| NoheadObject.wrap(detail) }
       @headers = headers
       @body = body
-      super(error["message"] || "Request failed with status #{status}")
+      @api_message = if redirect?(status)
+                       to = headers["location"].to_s.empty? ? "" : " to #{headers['location']}"
+                       "The API answered with a redirect#{to}; check the base URL"
+                     else
+                       error["message"] || "Request failed with status #{status}"
+                     end
+      super(@api_message)
     end
 
     def to_s
       suffix = request_id ? " (request #{request_id})" : ""
       "#{status} #{type || 'error'}: #{super}#{suffix}"
     end
+
+    private
+
+    def redirect?(status) = status.between?(300, 399)
   end
 
   class InvalidRequestError < APIError; end
@@ -99,7 +115,8 @@ module Nohead
 
   # The error for a response, by its `type`, or by status when it has none.
   def self.api_error(status, body, headers)
-    klass = ERROR_CLASSES[error_envelope(body)["type"]] ||
+    klass = (APIError if status < 400) ||
+            ERROR_CLASSES[error_envelope(body)["type"]] ||
             if status == 503 then ServiceUnavailableError
             elsif status >= 500 then InternalServerError
             else APIError
@@ -114,13 +131,20 @@ module Nohead
     return {} unless error.is_a?(Hash)
 
     fields = error.slice("type", "message", "request_id").select { |_, value| value.is_a?(String) }
-    # Only hashes: current_revision reads each detail's fields.
-    fields["details"] = error["details"].grep(Hash) if error["details"].is_a?(Array)
+    fields["details"] = error["details"].map { |detail| error_detail(detail) } if error["details"].is_a?(Array)
     fields
   end
 
+  # A detail that isn't an object is kept, as its text, with the code "unknown".
+  def self.error_detail(detail)
+    return detail if detail.is_a?(Hash)
+
+    { "code" => "unknown", "message" => detail.is_a?(String) ? detail : JSON.generate(detail) }
+  end
+
+  # Seconds from `Retry-After`: a number of seconds or an HTTP date, else nil.
   def self.retry_after_seconds(headers)
-    value = headers["retry-after"] or return nil
+    value = headers["retry-after"]&.strip or return nil
     return [Float(value), 0].max if value.match?(/\A\d+(\.\d+)?\z/)
 
     [Time.httpdate(value) - Time.now, 0].max

data/lib/nohead/images.rb

@@ -13,29 +13,42 @@ module Nohead
     module_function
 
     # A rendition of an image (an asset, as an object or a Hash, or its `image_url`). fit: "contain"
-    # fits inside the box, "cover" fills it, cropping, "fill" stretches to it.
+    # (the default, also for nil) fits inside the box, "cover" fills it, cropping, "fill" stretches to it.
     # gravity: is what a crop keeps ("sm" for smart, "fp:0.5:0.3" for a focal
     # point); options: any other processing options, as written in a URL
-    # (["bl:2"]).
+    # (["bl:2"]). An empty string is the same as leaving an option out.
     def url(image, width: nil, height: nil, fit: "contain", format: nil, quality: nil, dpr: nil,
             gravity: nil, options: [])
+      gravity, format = [gravity, format].map { |value| value unless value.to_s.empty? }
+      options = [*options].reject { |option| option.to_s.empty? }
       base = base_of(image)
       check_options([gravity, *options])
       at = base.index("/plain/")
-      parts = [resize(fit, width, height), ("g:#{gravity}" if gravity), ("dpr:#{dpr}" if dpr),
+      dpr = number(dpr)
+      quality = number(quality)
+      parts = [resize(fit, number(width), number(height)), ("g:#{gravity}" if gravity), ("dpr:#{dpr}" if dpr),
                ("q:#{quality}" if quality), *options].compact
       base[0...at] + parts.map { |part| "/#{part}" }.join + source(base[at..], format)
     end
 
-    # rs:<fit>:<width>:<height>, or nothing without a size (0 is none).
+    # rs:<fit>:<width>:<height>, or nothing without a size.
     def resize(fit, width, height)
-      type = FITS.fetch(fit.to_s) { raise ArgumentError, "fit must be one of #{FITS.keys.join(', ')}" }
-      width = nil if width&.zero?
-      height = nil if height&.zero?
+      fit = fit.to_s.empty? ? "contain" : fit.to_s
+      type = FITS.fetch(fit) { raise ArgumentError, "fit must be one of #{FITS.keys.join(', ')}" }
       "rs:#{type}:#{width || 0}:#{height || 0}" if width || height
     end
     private_class_method :resize
 
+    # A number as the other SDKs write it (2.0 is 2), or nil for none (0 and "" are none).
+    def number(value)
+      return nil if value == ""
+      return value unless value.is_a?(Numeric)
+      return nil if value.zero?
+
+      value.is_a?(Float) && value.finite? && value == value.round ? value.round : value
+    end
+    private_class_method :number
+
     def base_of(image)
       base = case image
              when String then image

data/lib/nohead/resources/assets.rb

@@ -7,8 +7,9 @@ module Nohead
       # Uploads a file and returns the `ready` asset: creates the upload, sends
       # the bytes straight to storage, then completes it (which checks the
       # file). `file` is a path or an IO; for bytes, pass StringIO.new(bytes).
-      # Raises UploadError if storage refuses the bytes, and ValidationError if
-      # the file fails the checks.
+      # Raises UploadError if the bytes don't reach storage (it refuses them,
+      # can't be reached or times out, with the ConnectionError as its
+      # `cause`), and ValidationError if the file fails the checks.
       #
       # With an `idempotency_key`, running the same upload again resumes it:
       # the key starts the upload, and an asset that is already `ready` is
@@ -31,7 +32,11 @@ module Nohead
             end
           end
           upload = created.upload
-          response = @client.put_upload(upload.url, upload["method"], upload.headers.to_h, source)
+          response = begin
+            @client.put_upload(upload.url, upload["method"], upload.headers.to_h, source)
+          rescue ConnectionError => e
+            raise UploadError, "Could not upload #{source.filename}: #{e.message}"
+          end
         ensure
           source.close
         end
@@ -105,16 +110,6 @@ module Nohead
       def usage(asset)
         @client.request("assets_usage", path: { asset_id: asset })
       end
-
-      private
-
-      # One value as it is, several as `[in]`; an empty list (like an asset
-      # field's accepted_types) is no filter.
-      def one_or_in(value)
-        return value unless value.is_a?(Array)
-
-        value.empty? ? nil : { in: value }
-      end
     end
   end
 end

data/lib/nohead/resources/other.rb

@@ -14,8 +14,7 @@ module Nohead
       # `sort` is "-created_at" (the default) or "created_at".
       def list(filter: nil, sort: nil, limit: nil, cursor: nil)
         filter = filter&.transform_keys(&:to_sym)
-        types = filter&.[](:resource_type)
-        filter = filter.merge(resource_type: { in: types }) if types.is_a?(Array)
+        filter[:resource_type] = one_or_in(filter[:resource_type]) if filter&.key?(:resource_type)
         @client.paginate("audit_events_list",
                          query: { filter: filter, sort: sort, limit: limit, cursor: cursor })
       end

data/lib/nohead/resources/records.rb

@@ -25,6 +25,14 @@ module Nohead
 
         params
       end
+
+      # One value as it is, several as `{ in: [...] }`; an empty list (like an
+      # asset field's accepted_types) is no filter.
+      def one_or_in(value)
+        return value unless value.is_a?(Array)
+
+        value.empty? ? nil : { in: value }
+      end
     end
 
     # Records: the content of a collection. `collection` is a collection ID or

data/lib/nohead/transport.rb

@@ -3,6 +3,7 @@
 require "net/http"
 require "openssl"
 require "uri"
+require "zlib"
 
 module Nohead
   # A response: status, headers (lowercase names) and body text.
@@ -20,15 +21,23 @@ module Nohead
       elsif body
         request.body = body
       end
-      http = Net::HTTP.new(uri.host, uri.port)
-      http.use_ssl = uri.scheme == "https"
-      http.open_timeout = http.read_timeout = http.write_timeout = timeout
-      response = http.start { http.request(request) }
+      # The proxy comes from the environment (http_proxy, no_proxy), as Net::HTTP.new.
+      # No retries of its own (it retries a GET, PUT or DELETE that times out
+      # or loses its connection once, and a PUT with the stream already
+      # read): the client retries, and knows when it can.
+      response = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https", open_timeout: timeout,
+                                                     read_timeout: timeout, write_timeout: timeout,
+                                                     max_retries: 0) do |http|
+        http.request(request)
+      end
       Response.new(response.code.to_i, response.each_header.to_h, response.body.to_s)
     rescue Net::OpenTimeout, Net::ReadTimeout, Net::WriteTimeout
       raise TimeoutError, "The request to #{uri.host} timed out after #{timeout} s"
     rescue SocketError, SystemCallError, IOError, OpenSSL::SSL::SSLError => e
       raise ConnectionError, "Could not reach #{uri.host}: #{e.message}"
+    # A broken status line, header or compressed body: retried like a dropped connection.
+    rescue Net::HTTPBadResponse, Net::HTTPHeaderSyntaxError, Net::ProtocolError, Zlib::Error => e
+      raise ConnectionError, "The response from #{uri.host} could not be read: #{e.message}"
     end
   end
 end

data/lib/nohead/uploads.rb

@@ -19,16 +19,18 @@ module Nohead
 
     def initialize(file, filename: nil, content_type: nil, byte_size: nil)
       name, size = file.is_a?(String) || file.is_a?(Pathname) ? open_path(Pathname(file)) : open_io(file)
-      @filename = filename || name || "upload"
+      # An empty filename or content type is the same as none.
+      @filename = present(filename) || present(name) || "upload"
       @byte_size = byte_size || size or
         raise UploadError, "Uploading a stream that cannot seek needs byte_size:"
-      @content_type = content_type ||
+      @content_type = present(content_type) ||
                       CONTENT_TYPES.fetch(File.extname(@filename).downcase, "application/octet-stream")
-      @start = @io.respond_to?(:pos) ? @io.pos : 0
+      @start = position(@io)
     end
 
-    # Whether the bytes can be sent again (for retries).
-    def replayable? = @io.respond_to?(:rewind) && @io.respond_to?(:pos)
+    # Whether the bytes can be sent again (for retries): a pipe or socket
+    # can't seek back to where they start.
+    def replayable? = !@start.nil? && @io.respond_to?(:pos=)
 
     def rewind
       @io.pos = @start if replayable?
@@ -40,6 +42,10 @@ module Nohead
 
     private
 
+    def present(value)
+      value unless value.to_s.empty?
+    end
+
     def open_path(path)
       @io = path.open("rb")
       @owned = true
@@ -54,6 +60,12 @@ module Nohead
       [name, remaining(io)]
     end
 
+    def position(io)
+      io.pos if io.respond_to?(:pos)
+    rescue IOError, SystemCallError
+      nil
+    end
+
     def remaining(io)
       return io.size - io.pos if io.respond_to?(:size) && io.respond_to?(:pos)
 

data/lib/nohead/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Nohead
-  VERSION = "0.8.0"
+  VERSION = "0.9.0"
 end

data/lib/nohead/webhooks.rb

@@ -31,8 +31,8 @@ module Nohead
         raise WebhookVerificationError, "The webhook timestamp is too far from the current time"
       end
 
-      key = secret.delete_prefix("whsec_").unpack1("m0")
-      expected = [OpenSSL::HMAC.digest("SHA256", key, "#{id}.#{timestamp}.#{body}")].pack("m0")
+      # The body as bytes too, so a non-ASCII id and body in different encodings combine.
+      expected = [OpenSSL::HMAC.digest("SHA256", key(secret), "#{id}.#{timestamp}.#{body.to_s.b}")].pack("m0")
       matched = signatures.split.any? do |entry|
         version, signature = entry.split(",", 2)
         version == "v1" && signature && OpenSSL.secure_compare(signature, expected)
@@ -40,16 +40,32 @@ module Nohead
       raise WebhookVerificationError, "No webhook signature matches" unless matched
 
       NoheadObject.wrap(JSON.parse(body))
-    rescue ArgumentError
-      raise WebhookVerificationError, "The webhook secret is not base64"
     rescue JSON::ParserError
       raise WebhookVerificationError, "The webhook body is not JSON"
     end
 
-    def self.header(headers, name)
-      return headers[name] || headers[name.split("-").map(&:capitalize).join("-")] unless headers.is_a?(Hash)
+    def self.key(secret)
+      key = secret.to_s.b.delete_prefix("whsec_").unpack1("m0")
+      # Unset (nil) or empty, as from a missing environment variable.
+      raise WebhookVerificationError, "The webhook secret is empty" if key.empty?
+
+      key
+    rescue ArgumentError
+      raise WebhookVerificationError, "The webhook secret is not base64"
+    end
+    private_class_method :key
 
-      headers.find { |key, _| key.to_s.downcase == name }&.last
+    # As bytes: headers come in any encoding, and not always valid, which
+    # must not raise ArgumentError or Encoding::CompatibilityError.
+    def self.header(headers, name)
+      value = if headers.is_a?(Hash)
+                headers.find { |key, _| key.to_s.downcase == name }&.last
+              else
+                headers[name] || headers[name.split("-").map(&:capitalize).join("-")]
+              end
+      # A repeated header, as some frameworks give it: its values, space-separated.
+      value = value.join(" ") if value.is_a?(Array)
+      value&.b
     end
     private_class_method :header
   end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: nohead
 version: !ruby/object:Gem::Version
-  version: 0.8.0
+  version: 0.9.0
 platform: ruby
 authors:
 - Nohead