w3c_api 0.3.2 → 0.3.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f45cd3142ec5509995fee4a123992a0f74d1abc6fcc4c0477909855d017b5630
-  data.tar.gz: 03cf1921072c56fba3024094242f8225caed25a603471e5f8358272b01ad4b92
+  metadata.gz: 307e08b89228dff85498435c4ec1e7893a61ed4363d338f02a434dacc7880866
+  data.tar.gz: 4561cf4b6b4ae80c65d4a5a6edc3a7e2d8d98a8ca09df41ad6ca47bcee6940a9
 SHA512:
-  metadata.gz: 57a1d8396a7bba59cff32430641bf90a75a251f9b8ea227710ec6e27bd4fc1fef2d5d9c667bfb7e5a4c6782e099ad96ddd72060e1c9973fbe9c7068b223b4845
-  data.tar.gz: b0178bf92b0e73a16d226aeeb0c91f0cee3ca3b91857a7b6241ff2423a604ca62181c9edb76af63833b875cf314842ac572badb9dd158a8bcbb172612a21bf03
+  metadata.gz: b7343b0927f3b330a8f6fa8c19f388557206dbbdfb16bfa94cc3800eb6b7a3fdcdd01c1ce9296fee17e6bc98961c6248edb82ca4e12b8479c7014ef7cd3eb21d
+  data.tar.gz: fb1e3a885d85112a4778c8b763857df9840214885176b83a69e8488fdc76a3fc26d7fde8bfa481666186bb99afe78917b93c35687f1969a2aaf09397f48c0cef

data/CLAUDE.md

@@ -74,13 +74,58 @@ that embedded data instead of issuing a new HTTP request (see
 Two cooperating layers, both tuned to grow 1→2→4→8→16s:
 - lutaml-hal's `RateLimiter` (via `rate_limiting_options`) retries **429 and
   5xx**.
-- A Faraday `:retry` middleware in `connection` covers what lutaml-hal does not:
-  the W3C API signals rate-limiting with **HTTP 403**, plus connection/timeout
-  errors.
+- A Faraday `:retry` middleware in `connection` (`DEFAULT_RETRY_OPTIONS`) covers
+  what lutaml-hal does not: the W3C API signals rate-limiting with **HTTP 403**,
+  plus connection/timeout errors.
+
+Two footguns in the Faraday layer, both encoded in constants in `hal.rb` — do
+not "tidy" them away (see issue #23):
+- `exceptions:` **replaces** faraday-retry's `DEFAULT_EXCEPTIONS`, it does not
+  merge. `retry_statuses: [403]` is implemented by raising
+  `Faraday::RetriableResponse` internally and rescuing it via `exceptions:`, so
+  dropping that class disables 403 retries *and* leaks the raise to the caller
+  on the first 403. Hence `RETRY_EXCEPTIONS` is built on top of the defaults.
+- `max_interval` caps `Retry-After` as well as the computed backoff, and a
+  `Retry-After` above it makes faraday-retry stop retrying entirely rather than
+  wait longer. Hence 60s, well above the largest computed backoff (16s).
 
 Owning retries in the client means consumers get resilience without wrapping.
-Tune via `Hal.instance.configure_rate_limiting(...)`; changing options resets
-the memoized client.
+Tune via `Hal.instance.configure_rate_limiting(...)` and
+`Hal.instance.configure_retry(...)` — both go through the memoization cascade
+below. There is no `disable_retry`; `configure_retry(max: 0)` is the off switch.
+
+Once retries are exhausted a 403 surfaces as `Lutaml::Hal::Error` (its
+`handle_response` has no 403 branch, so 403 falls through to the generic
+`raise Error`), not as a `Faraday::RetriableResponse`.
+
+### User agent (`hal.rb`)
+
+Requests carry `w3c_api/<VERSION> (+<repo url>)` (`DEFAULT_USER_AGENT`) — a bare
+Faraday user agent is a prime trigger for the Cloudflare bot heuristics in front
+of `api.w3.org`, and W3C asks consumers to identify themselves. Override with
+`Hal.instance.configure_user_agent(...)` (wins), the `W3C_API_USER_AGENT`
+environment variable, or the CLI's `--user-agent` flag. The flag comes from
+`Commands::UserAgentOption`, mixed into every command class — Thor cannot parse
+options placed before a subcommand name, so it cannot live on the root `Cli`.
+
+Note that per-request `headers:` passed to `Client` methods never reach the wire:
+lutaml-hal only forwards headers declared as endpoint parameters with
+`location: :header`, and `SimpleParameter` only produces `:path`/`:query`. The
+connection-level user agent is the working mechanism.
+
+### Memoization cascade (`hal.rb`)
+
+Memoization runs `@register → @client → @connection`, and `ModelRegister` keeps
+the client it was built with — so any connection- or client-level change must
+rebuild all three. Every `configure_*` setter funnels through
+`reset_connection → reset_client → rebuild_register` accordingly.
+
+`rebuild_register` rebuilds **eagerly**, and that is load-bearing: `Link#realize`
+resolves the register via `GlobalRegister.instance.get(:w3c_api)`, which *raises*
+when the name is absent. A lazy `reset_register` would leave every
+already-fetched model unable to realize its links until something re-entered
+`Hal#register`. The trade-off is that each `configure_*` call starts a fresh
+object cache — they are start-up knobs, not mid-crawl ones.
 
 ### Caching (`hal.rb`)
 
@@ -98,10 +143,22 @@ unregisters from lutaml-hal's `GlobalRegister`, otherwise the rebuild raises
 Specs use **VCR** (`hook_into :faraday`) with cassettes in
 `spec/fixtures/vcr_cassettes/`. Default record mode is `:new_episodes` and
 requests match on `method, uri, body` — so a new test that hits an unrecorded
-request will perform a real HTTP call and record it. `spec_helper.rb` resets the
-`Hal` singleton's register and the lutaml-hal `GlobalRegister` around every
-example to prevent cross-test endpoint-registration bleed — which also gives
-each example a fresh object cache, so caching doesn't mask expected requests.
+request will perform a real HTTP call and record it (pass `record: :none` to a
+cassette to make a mismatch raise instead). `spec_helper.rb` calls
+`configure_user_agent(nil)` before every example, which cascades through the
+whole memoization chain — connection, client, register — and unregisters from
+the lutaml-hal `GlobalRegister` to prevent cross-test endpoint-registration
+bleed. That also gives each example a fresh object cache, so caching doesn't
+mask expected requests.
+
+`spec/w3c_api/hal_spec.rb` is the one spec that opts out of VCR: it drives a
+`Faraday::Adapter::Test` connection and wraps every example in
+`VCR.turned_off`, because the `:faraday` hook injects VCR into *that*
+connection too and would otherwise reject the request as unhandled. It also
+restores `@retry_options` in an `around` hook: the per-example
+`configure_user_agent(nil)` rebuilds connection, client and register, but
+nothing resets the retry options, so a `configure_retry` call would otherwise
+persist for the whole suite process.
 
 ## Conventions
 

data/README.adoc

@@ -17,7 +17,8 @@ https://github.com/lutaml/lutaml-model[lutaml-model]
 for all W3C API resources
 * HAL (Hypertext Application Language) implementation for recursive resource
 traversal
-* Built-in rate limiting with exponential backoff for reliable API access
+* Built-in rate limiting with exponential backoff, plus automatic retries of
+HTTP 403 and transient network errors, for reliable API access
 * A command-line interface using Thor following GitHub CLI patterns
 
 The endpoint supported is at https://api.w3.org.
@@ -90,7 +91,7 @@ client = W3cApi::Client.new
 W3cApi::Hal.instance.configure_rate_limiting(
   max_retries: 5,        # Maximum retry attempts
   base_delay: 1.0,       # Initial delay in seconds
-  max_delay: 60.0,       # Maximum delay cap
+  max_delay: 30.0,       # Maximum delay cap
   backoff_factor: 2.0    # Exponential backoff multiplier
 )
 
@@ -98,6 +99,46 @@ W3cApi::Hal.instance.configure_rate_limiting(
 specifications = client.specifications
 ----
 
+==== Retrying HTTP 403
+
+The W3C API signals rate limiting with `HTTP 403` rather than `429`, which the
+rate limiter above does not cover. A Faraday retry middleware in the shared
+connection handles it: a `403` response is retried up to 5 times with
+exponential backoff (1, 2, 4, 8, 16 seconds), as are connection and timeout
+errors. If the response carries a `Retry-After` (or `RateLimit-Reset`) header,
+that value is used in place of the computed backoff when it is the longer of the
+two -- unless it exceeds `max_interval`, in which case the middleware gives up
+rather than waiting. See the note below.
+
+[source,ruby]
+----
+W3cApi::Hal.instance.configure_retry(
+  max: 5,              # Maximum retry attempts
+  interval: 1.0,       # Initial delay in seconds
+  backoff_factor: 2,   # Exponential backoff multiplier
+  max_interval: 60.0,  # Delay cap -- also caps `Retry-After`, see below
+  retry_statuses: [403]
+)
+
+# Effectively turn retrying off
+W3cApi::Hal.instance.configure_retry(max: 0)
+----
+
+Options are merged into the current ones, and calling `configure_retry` rebuilds
+the connection, the client and the endpoint register -- so, like the other
+`configure_*` setters, call it once at start-up rather than mid-crawl (see the
+note under <<User agent>>).
+
+[NOTE]
+====
+`max_interval` bounds the `Retry-After` header as well as the computed backoff,
+but it does not clamp it. If a server asks for longer than `max_interval`, the
+middleware stops retrying rather than waiting longer, so keep `max_interval`
+comfortably above the largest `Retry-After` you expect.
+====
+
+If every retry is exhausted, the final `403` surfaces as a `Lutaml::Hal::Error`.
+
 ==== Demo and examples
 
 To see rate limiting in action, run the included demonstration scripts:
@@ -123,6 +164,43 @@ The stress test example demonstrates:
 * Dynamic configuration changes
 * Bulk operation patterns
 
+=== User agent
+
+Every request identifies the gem, its version and a contact URL, e.g. `w3c_api/0.3.2 (+https://github.com/relaton/w3c_api)`.
+
+`api.w3.org` is fronted by Cloudflare, and a generic HTTP library user agent (the Faraday default) is a common trigger for its bot and rate limiting heuristics. Applications that embed this gem should add their own identity so that W3C can tell traffic sources apart and reach the operator instead of throttling silently.
+
+==== Quick start
+
+[source,ruby]
+----
+require 'w3c_api'
+
+# Inspect the current value
+W3cApi::Hal.instance.user_agent
+# => "w3c_api/0.3.2 (+https://github.com/relaton/w3c_api)"
+
+# Identify your application, keeping the gem's own token
+W3cApi::Hal.instance.configure_user_agent(
+  "my-crawler/1.4 (+https://example.com/contact) #{W3cApi::Hal::DEFAULT_USER_AGENT}"
+)
+
+# Restore the default (or the W3C_API_USER_AGENT value)
+W3cApi::Hal.instance.configure_user_agent(nil)
+----
+
+Alternatively, set the `W3C_API_USER_AGENT` environment variable, or pass `--user-agent` to any CLI command:
+
+[source,shell]
+----
+$ export W3C_API_USER_AGENT="my-crawler/1.4 (+https://example.com/contact)"
+$ w3c_api specification fetch --user-agent "my-crawler/1.4 (+https://example.com/contact)"
+----
+
+An explicit `configure_user_agent` call (which is what `--user-agent` performs) takes precedence over the environment variable, which in turn takes precedence over the gem default.
+
+NOTE: Changing the user agent rebuilds the HTTP connection, the client and the endpoint register — as do `configure_rate_limiting` and the cache setters. The rebuilt register starts with an empty in-memory object cache, so configure these once at start-up, before issuing requests, rather than mid-crawl. Models fetched earlier keep working: the register is rebuilt eagerly, so their links still realize.
+
 === Caching
 
 Realized objects are cached keyed by their (canonical) URL, so a resource linked from many places — editors, working groups, versions — is fetched only once per process. Caching is enabled by default (in-memory) and works transparently with `fetch` and link `.realize`.
@@ -311,9 +389,10 @@ end
 specifications = client.specifications(page: 2, per_page: 50)
 groups = client.groups(page: 1, per_page: 10, limit: 25, offset: 100)
 
-# HTTP client options
-user = client.user('hash', timeout: 30, headers: { 'User-Agent' => 'MyApp/1.0' })
-spec = client.specification('html5', read_timeout: 45, open_timeout: 10)
+# NOTE: HTTP-level settings are not per-request. Options such as `headers:`,
+# `timeout:`, `read_timeout:` and `open_timeout:` are not endpoint parameters
+# and are silently ignored. Configure the user agent connection-wide with
+# W3cApi::Hal.instance.configure_user_agent (see "User agent" above).
 
 # Query parameters for filtering and sorting
 rec_specs = client.specifications_by_status('REC', sort: 'date', order: 'desc')
@@ -323,8 +402,6 @@ active_groups = client.groups(type: 'working-group', status: 'active')
 options = {
   page: 1,
   per_page: 25,
-  headers: { 'Accept-Language' => 'en-US' },
-  timeout: 60,
   sort: 'name'
 }
 specs = client.specifications(options)

data/lib/w3c_api/commands/affiliation.rb

@@ -2,6 +2,7 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 require_relative "../client"
 
 module W3cApi
@@ -9,6 +10,7 @@ module W3cApi
     # Thor CLI command for affiliation operations
     class Affiliation < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "fetch [OPTIONS]", "Fetch affiliations"
       option :id, type: :numeric, desc: "Affiliation ID"

data/lib/w3c_api/commands/ecosystem.rb

@@ -2,6 +2,7 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 require_relative "../client"
 
 module W3cApi
@@ -9,6 +10,7 @@ module W3cApi
     # Thor CLI command for ecosystem operations
     class Ecosystem < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "fetch [OPTIONS]", "Fetch ecosystems"
       option :shortname, type: :string, desc: "Ecosystem shortname"

data/lib/w3c_api/commands/group.rb

@@ -2,6 +2,7 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 require_relative "../client"
 
 module W3cApi
@@ -9,6 +10,7 @@ module W3cApi
     # Thor CLI command for group operations
     class Group < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "fetch [OPTIONS]", "Fetch groups"
       option :id, type: :string, desc: "Group ID"

data/lib/w3c_api/commands/participation.rb

@@ -2,6 +2,7 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 require_relative "../client"
 
 module W3cApi
@@ -9,6 +10,7 @@ module W3cApi
     # Thor CLI command for participation operations
     class Participation < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "fetch", "Fetch a participation by ID"
       option :id, type: :numeric, required: true, desc: "Participation ID"

data/lib/w3c_api/commands/series.rb

@@ -2,6 +2,7 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 require_relative "../client"
 
 module W3cApi
@@ -9,6 +10,7 @@ module W3cApi
     # Thor CLI command for series operations
     class Series < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "fetch [OPTIONS]", "Fetch specification series"
       option :shortname, type: :string, desc: "Series shortname"

data/lib/w3c_api/commands/specification.rb

@@ -2,6 +2,7 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 require_relative "../client"
 
 module W3cApi
@@ -9,6 +10,7 @@ module W3cApi
     # Thor CLI command for specification operations
     class Specification < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "fetch [OPTIONS]", "Fetch specifications"
       option :shortname, type: :string, desc: "Filter by shortname"

data/lib/w3c_api/commands/specification_version.rb

@@ -2,11 +2,13 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 
 module W3cApi
   module Commands
     class SpecificationVersion < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "editors", "Fetch editors of a specification version"
       option :shortname, type: :string, required: true,

data/lib/w3c_api/commands/translation.rb

@@ -2,6 +2,7 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 require_relative "../client"
 
 module W3cApi
@@ -9,6 +10,7 @@ module W3cApi
     # Thor CLI command for translation operations
     class Translation < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "fetch [OPTIONS]", "Fetch a translation by ID"
       option :id, type: :string, desc: "Translation ID"

data/lib/w3c_api/commands/user.rb

@@ -2,6 +2,7 @@
 
 require "thor"
 require_relative "output_formatter"
+require_relative "user_agent_option"
 require_relative "../client"
 
 module W3cApi
@@ -9,6 +10,7 @@ module W3cApi
     # Thor CLI command for user operations
     class User < Thor
       include OutputFormatter
+      include UserAgentOption
 
       desc "fetch [OPTIONS]", "Fetch a user by ID"
       option :id, type: :string, required: true, desc: "User ID (required)"

data/lib/w3c_api/commands/user_agent_option.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+require_relative "../hal"
+
+module W3cApi
+  module Commands
+    # Adds a --user-agent flag to a command class and applies it before the
+    # command runs. Thor cannot parse options that precede a subcommand name,
+    # so the flag lives on each command class rather than on the root Cli:
+    #
+    #   w3c_api specification fetch --user-agent "my-crawler/1.0 (+url)"
+    module UserAgentOption
+      def self.included(base)
+        base.class_option :user_agent,
+                          type: :string,
+                          desc: "User-Agent sent to api.w3.org " \
+                                "(default: #{Hal::DEFAULT_USER_AGENT})"
+      end
+
+      def initialize(*args)
+        super
+        # Configure before the command body builds a Client: changing the user
+        # agent rebuilds the connection, client and register.
+        Hal.instance.configure_user_agent(options[:user_agent]) if options[:user_agent]
+      end
+    end
+  end
+end

data/lib/w3c_api/hal.rb

@@ -2,7 +2,9 @@
 
 require "singleton"
 require "faraday/retry"
+require "faraday/follow_redirects"
 require "lutaml/hal"
+require_relative "version"
 require_relative "models"
 
 module W3cApi
@@ -44,6 +46,51 @@ module W3cApi
 
     API_URL = "https://api.w3.org/"
 
+    # Exceptions the Faraday retry middleware treats as retriable.
+    #
+    # faraday-retry implements `retry_statuses` by raising
+    # Faraday::RetriableResponse internally and rescuing it with a matcher built
+    # from `exceptions:`. Passing `exceptions:` REPLACES faraday-retry's
+    # DEFAULT_EXCEPTIONS, it does not merge with them -- so omitting
+    # Faraday::RetriableResponse silently disables `retry_statuses` *and* lets
+    # the internal raise escape to the caller on the very first matching
+    # response. Always build this list on top of the defaults.
+    RETRY_EXCEPTIONS = (
+      Faraday::Retry::Middleware::DEFAULT_EXCEPTIONS + [Faraday::ConnectionFailed]
+    ).freeze
+
+    # Retry policy for the W3C-specific transient failures: HTTP 403 (how the
+    # W3C API signals rate-limiting) plus connection/timeout errors. The
+    # computed backoff grows 1, 2, 4, 8, 16s, matching rate_limiting_options.
+    #
+    # max_interval sits far above the largest computed backoff (16s) because it
+    # also caps `Retry-After`: faraday-retry's `calculate_sleep_amount` returns
+    # nil -- aborting the retry loop entirely, with zero retries -- when a
+    # response's `Retry-After` exceeds max_interval. Too small a cap turns a
+    # polite server hint into an immediate give-up.
+    DEFAULT_RETRY_OPTIONS = {
+      max: 5,
+      interval: 1.0,
+      backoff_factor: 2,
+      max_interval: 60.0,
+      # Both nested values are frozen too: Hash#freeze is shallow, and
+      # retry_options hands out a shallow dup, so an unfrozen array here would
+      # let a caller corrupt this baseline for the whole process.
+      retry_statuses: [403].freeze,
+      exceptions: RETRY_EXCEPTIONS,
+    }.freeze
+
+    # api.w3.org is fronted by Cloudflare, and a bare HTTP-library User-Agent
+    # such as "Faraday v2.12.2" is a well-known trigger for its bot heuristics —
+    # which is how a polite crawl turns into an all-or-nothing 429 storm. W3C
+    # also asks API consumers to identify themselves, so name the gem, its
+    # version, and a contact URL.
+    DEFAULT_USER_AGENT = "w3c_api/#{VERSION} (+https://github.com/relaton/w3c_api)".freeze
+
+    # Lets an operator put their own contact details on the wire without a code
+    # change. An explicit configure_user_agent call still wins.
+    USER_AGENT_ENV_VAR = "W3C_API_USER_AGENT"
+
     def initialize
       # Don't call setup here - it will be called when register is first accessed
     end
@@ -56,35 +103,69 @@ module W3cApi
       )
     end
 
+    # User-Agent sent with every request. Precedence: an explicit
+    # configure_user_agent call, then W3C_API_USER_AGENT, then the gem default.
+    def user_agent
+      @user_agent ||=
+        normalize_user_agent(ENV.fetch(USER_AGENT_ENV_VAR, nil)) ||
+        DEFAULT_USER_AGENT
+    end
+
+    # Set the User-Agent sent with every request.
+    #
+    # Applications embedding this gem should identify themselves and keep the
+    # gem's own token, e.g.
+    #   configure_user_agent("my-crawler/1.4 (+https://example.com) #{DEFAULT_USER_AGENT}")
+    #
+    # Pass nil or a blank string to fall back to W3C_API_USER_AGENT / the gem
+    # default. Rebuilds the connection, so call it before issuing requests.
+    def configure_user_agent(user_agent)
+      @user_agent = normalize_user_agent(user_agent)
+      reset_connection
+      self.user_agent
+    end
+
     # Faraday connection mirroring lutaml-hal's default middleware stack, with a
     # retry layer for the failures lutaml-hal's RateLimiter does not cover: the
     # W3C API signals rate-limiting with HTTP 403, plus transient connection and
     # timeout errors. (lutaml-hal still retries 429 and 5xx.) Owning retries here
     # means every consumer of the client is resilient without its own wrapper.
     def connection
-      @connection ||= Faraday.new(url: API_URL.delete_suffix("/")) do |conn|
+      @connection ||= Faraday.new(
+        url: API_URL.delete_suffix("/"),
+        headers: { "User-Agent" => user_agent },
+      ) do |conn|
         conn.request :retry, retry_options
         conn.use Faraday::FollowRedirects::Middleware
-        conn.request :json
-        conn.response :json, content_type: /\bjson$/
+        # No `response :json`: Faraday's middleware parses with
+        # `::JSON.parse(body, @parser_options || {})` — a positional options
+        # hash json 3.0 rejects, which forced a json < 3 pin on dependent
+        # bundles. Bodies arrive raw; lutaml-hal parses them with yeptris.
         conn.adapter Faraday.default_adapter
       end
     end
 
-    # Retry policy for the W3C-specific transient failures (HTTP 403 and
-    # connection/timeout). Grows 1, 2, 4, 8, 16s, matching rate_limiting_options.
+    # Retry policy for the W3C-specific transient failures. See
+    # DEFAULT_RETRY_OPTIONS for why `exceptions` and `max_interval` are what
+    # they are -- both have non-obvious failure modes.
     def retry_options
-      {
-        max: 5,
-        interval: 1.0,
-        backoff_factor: 2,
-        max_interval: 30.0,
-        retry_statuses: [403],
-        exceptions: [
-          Errno::ETIMEDOUT, Timeout::Error,
-          Faraday::TimeoutError, Faraday::ConnectionFailed
-        ],
-      }
+      # dup: DEFAULT_RETRY_OPTIONS is frozen and must stay the pristine baseline.
+      @retry_options ||= DEFAULT_RETRY_OPTIONS.dup
+    end
+
+    # Set retry options (merged into the current ones)
+    #
+    # The retry middleware is baked into the Faraday connection when the
+    # connection is built, so this goes through the usual
+    # reset_connection -> reset_client -> rebuild_register cascade. Like the
+    # other configure_* setters it starts a fresh object cache, so it is a
+    # start-up knob rather than a mid-crawl one.
+    #
+    # There is no separate disable switch -- `configure_retry(max: 0)` turns
+    # retrying off, returning the response as-is on the first attempt.
+    def configure_retry(options = {})
+      @retry_options = retry_options.merge(options)
+      reset_connection
     end
 
     # Configure rate limiting options
@@ -106,8 +187,10 @@ module W3cApi
     # Set rate limiting options
     def configure_rate_limiting(options = {})
       @rate_limiting_options = rate_limiting_options.merge(options)
-      # Reset client to pick up new options
-      @client = nil
+      # Reset the client *and* the register to pick up the new options: the
+      # register keeps the client it was built with, so dropping @client alone
+      # left an already-built register on the old rate-limiting settings.
+      reset_client
     end
 
     # Disable rate limiting
@@ -136,19 +219,19 @@ module W3cApi
     # Set cache options (merged into the current ones)
     def configure_cache(options = {})
       @cache_options = (cache_options || {}).merge(options)
-      reset_register
+      rebuild_register
     end
 
     # Disable caching of realized objects
     def disable_cache
       @cache_options = nil
-      reset_register
+      rebuild_register
     end
 
     # Enable caching of realized objects
     def enable_cache(options = nil)
       @cache_options = options || { adapter: :memory }
-      reset_register
+      rebuild_register
     end
 
     def register
@@ -167,6 +250,30 @@ module W3cApi
       @register
     end
 
+    # Drop the memoized Faraday connection. Memoization runs
+    # register -> client -> connection, so a connection-level change (headers,
+    # middleware) only takes effect once all three are rebuilt.
+    def reset_connection
+      @connection = nil
+      reset_client
+    end
+
+    # Drop the memoized lutaml-hal client. The register holds the client it was
+    # constructed with (and hands it to its cache manager), so it must go too.
+    def reset_client
+      @client = nil
+      rebuild_register
+    end
+
+    # Rebuild the register immediately rather than lazily. Link#realize looks
+    # the register up in lutaml-hal's GlobalRegister, which *raises* when the
+    # name is absent — so leaving it torn down would break `.realize` on every
+    # model fetched before a configure_* call.
+    def rebuild_register
+      reset_register
+      register
+    end
+
     def reset_register
       # Drop the global registration too, otherwise rebuilding the register
       # raises "replacing another one" when it re-registers the same name.
@@ -176,6 +283,17 @@ module W3cApi
 
     private
 
+    # Header values cannot contain CR/LF, and the value can come from an
+    # environment variable or a CLI flag — fold them to spaces rather than let
+    # the adapter raise deep inside a request. Blank becomes nil so the caller
+    # falls back to the environment or the default.
+    def normalize_user_agent(value)
+      return nil if value.nil?
+
+      normalized = value.to_s.gsub(/[\r\n]+/, " ").strip
+      normalized.empty? ? nil : normalized
+    end
+
     # Common pagination parameters (simplified without EndpointParameter)
     def pagination_parameters
       [

data/lib/w3c_api/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module W3cApi
-  VERSION = "0.3.2"
+  VERSION = "0.3.4"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: w3c_api
 version: !ruby/object:Gem::Version
-  version: 0.3.2
+  version: 0.3.4
 platform: ruby
 authors:
 - Ribose Inc.
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-06-03 00:00:00.000000000 Z
+date: 2026-10-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: faraday
@@ -154,6 +154,7 @@ files:
 - lib/w3c_api/commands/specification_version.rb
 - lib/w3c_api/commands/translation.rb
 - lib/w3c_api/commands/user.rb
+- lib/w3c_api/commands/user_agent_option.rb
 - lib/w3c_api/embed.rb
 - lib/w3c_api/hal.rb
 - lib/w3c_api/models.rb