w3c_api 0.2.0 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b1b6d2f332eeee4355bee1550931410695851be8e5349ee1ef0168fe370dab36
-  data.tar.gz: b9d8dfa4caae188d47acbd6ebe28e5c29a37035a9e43b60bae880467de5f22df
+  metadata.gz: ceb115051fa4d11d8bdfb690b26c93305556c3086a911a95e1605787e91ad437
+  data.tar.gz: 4c67a5092e6888768c13ff63c1b77b94b59db847e9df99c01501bdd401934199
 SHA512:
-  metadata.gz: 3b056eed82f4d6d6134f1e6f51b2b18a04d7e92683c359cdb5762495885afe5c142d8da2603cf05bf4c09eb75639343f7e0ec6c0433d38d9b8c602f417c4bae6
-  data.tar.gz: 43030538729e62863ddf3166c726c1f05a841d0b59a2c62f5590dce395a489601220be7ebd52b853ee5b55bb970209a60bc4ec74d29b091aa51af2f9de134580
+  metadata.gz: ed887189c60decd73e63fd1ee772baa656efdef09a667c921b0b438d3c6d53b92795b764526b461fb81223b92095ba552f18f0000063600f3d9a0f3b05e7b2d8
+  data.tar.gz: 33852b89ce9551bad83c5a1a524e06fe82393047daff471cffb540a4670c22f3dbd02986d91531fe4fb40a73c5ceb4a8628d14178b04f74c973e7b320fa396a2

data/CLAUDE.md

@@ -0,0 +1,113 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Overview
+
+`w3c_api` is a Ruby client + Thor CLI for the W3C API (https://api.w3.org).
+It is a thin layer over [lutaml-hal](https://github.com/lutaml/lutaml-hal):
+lutaml-hal owns the HTTP client, HAL link realization, and pagination; this gem
+contributes the endpoint registry, the resource models, a convenience `Client`
+facade, and the CLI.
+
+## Commands
+
+```sh
+bundle install              # install dependencies
+bundle exec rake            # default task: spec + rubocop
+bundle exec rake spec       # run all tests
+bundle exec rake rubocop    # lint only
+bundle exec rspec spec/w3c_api/client_spec.rb          # single file
+bundle exec rspec spec/w3c_api/client_spec.rb:42       # single example by line
+exe/w3c_api <command>       # run the CLI without installing
+```
+
+Note: the README's `bin/setup` and `bin/console` do not exist in this repo —
+use `bundle install` and `bundle exec`.
+
+## Architecture
+
+The request path is: **CLI command → `Client` → `Hal` register → lutaml-hal →
+`Models`**.
+
+- **`lib/w3c_api/hal.rb`** — the heart of the gem. `Hal` is a `Singleton` that
+  builds the Faraday connection (with retry + cache config) and registers
+  *every* API endpoint in `setup`
+  (one `add_endpoint` call per endpoint, mapping an endpoint id like
+  `:specification_resource` → URL template + model + parameters). To add or
+  change an API endpoint, edit `setup` here. `SimpleParameter` exists only to
+  satisfy lutaml-hal's parameter-validation interface without pulling in its
+  full `EndpointParameter` machinery.
+
+- **`lib/w3c_api/client.rb`** — `Client` is a flat convenience facade. Almost
+  every method is one line delegating to `Hal.instance.register.fetch(endpoint_id, **params)`
+  via the private `fetch_resource`. Many method families
+  (`group_*`, `user_*`, `affiliation_*`, `ecosystem_*`) are generated with
+  `define_method` loops — follow that pattern rather than writing them out.
+
+- **`lib/w3c_api/models/`** — one file per resource. Two kinds:
+  - **Resources** subclass `Lutaml::Hal::Resource` (e.g. `Specification`).
+  - **Indexes** subclass `Lutaml::Hal::Page` (e.g. `SpecificationIndex`) and
+    provide pagination.
+  Models declare attributes, `hal_link` relationships (each link names a
+  `realize_class` and may be a `collection`), and a `key_value` block mapping
+  the API's hyphenated JSON keys (`series-version`) to underscored Ruby
+  attributes (`series_version`). `models.rb` requires every model file.
+
+- **`lib/w3c_api/commands/`** — one Thor class per resource, all wired into the
+  root `Cli` in `cli.rb` as subcommands. Commands instantiate `Client`, call
+  it, and print via the shared `OutputFormatter` (`--format yaml|json`, YAML
+  default). `exe/w3c_api` is the executable.
+
+### HAL link realization
+
+The defining feature: links are lazy. Calling `.realize` on a link follows its
+`href` and returns the typed model — and chains
+(`spec.links.latest_version.realize.links.editors...`). With `embed: true` on
+supported index endpoints (`:specification_index`, `:group_index`,
+`:serie_index`), the index response embeds child resources and `.realize` uses
+that embedded data instead of issuing a new HTTP request (see
+`lib/w3c_api/embed.rb` and `Client.embed_supported_endpoints`).
+
+### Rate limiting & retries (`hal.rb`)
+
+Two cooperating layers, both tuned to grow 1→2→4→8→16s:
+- lutaml-hal's `RateLimiter` (via `rate_limiting_options`) retries **429 and
+  5xx**.
+- A Faraday `:retry` middleware in `connection` covers what lutaml-hal does not:
+  the W3C API signals rate-limiting with **HTTP 403**, plus connection/timeout
+  errors.
+
+Owning retries in the client means consumers get resilience without wrapping.
+Tune via `Hal.instance.configure_rate_limiting(...)`; changing options resets
+the memoized client.
+
+### Caching (`hal.rb`)
+
+lutaml-hal caches realized objects keyed by their canonical URL, so a document
+linked from many places (editors, working groups, versions) is fetched once per
+process. It is **enabled by default** (in-memory) — the register is built with
+`cache: cache_options`. Tune or persist via `Hal.instance.configure_cache(...)`
+(e.g. a `:filesystem` adapter) or turn it off with `disable_cache`; like the
+rate-limiting setters these rebuild the register. `reset_register` also
+unregisters from lutaml-hal's `GlobalRegister`, otherwise the rebuild raises
+"replacing another one".
+
+## Testing
+
+Specs use **VCR** (`hook_into :faraday`) with cassettes in
+`spec/fixtures/vcr_cassettes/`. Default record mode is `:new_episodes` and
+requests match on `method, uri, body` — so a new test that hits an unrecorded
+request will perform a real HTTP call and record it. `spec_helper.rb` resets the
+`Hal` singleton's register and the lutaml-hal `GlobalRegister` around every
+example to prevent cross-test endpoint-registration bleed — which also gives
+each example a fresh object cache, so caching doesn't mask expected requests.
+
+## Conventions
+
+- Ruby >= 3.1. RuboCop inherits Ribose's shared OSS config; `LineLength` max is
+  180. `.rubocop_todo.yml` holds grandfathered offences — prefer fixing over
+  growing it.
+- Endpoint ids follow `<resource>_resource` (single) / `<resource>_index`
+  (collection) naming; keep new ids consistent so the `define_method` loops and
+  `embed_supported?` discovery keep working.

data/README.adoc

@@ -123,6 +123,36 @@ The stress test example demonstrates:
 * Dynamic configuration changes
 * Bulk operation patterns
 
+=== Caching
+
+Realized objects are cached keyed by their (canonical) URL, so a resource linked from many places — editors, working groups, versions — is fetched only once per process. Caching is enabled by default (in-memory) and works transparently with `fetch` and link `.realize`.
+
+==== Quick start
+
+[source,ruby]
+----
+require 'w3c_api'
+
+# Object caching is enabled by default (in-memory)
+client = W3cApi::Client.new
+
+# The same linked resource is fetched once, then served from cache
+specs = client.specifications
+spec = specs.links.specifications.first.realize   # HTTP request
+spec.links.latest_version.realize                 # HTTP request
+spec.links.latest_version.realize                 # cache hit, no HTTP
+
+# Disable caching entirely
+W3cApi::Hal.instance.disable_cache
+
+# Or persist the cache across runs on disk
+W3cApi::Hal.instance.configure_cache(
+  adapter: { type: :filesystem, options: { path: "tmp/w3c_cache" } }
+)
+----
+
+The cache is backed by https://github.com/lutaml/lutaml-store[lutaml-store]: the default `:memory` adapter lives for the process, while the `:filesystem` (and `:sqlite`) adapters persist across runs.
+
 === Embed support with auto-realize
 
 ==== General

data/lib/w3c_api/hal.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require "singleton"
+require "faraday/retry"
 require "lutaml/hal"
 require_relative "models"
 
@@ -50,18 +51,55 @@ module W3cApi
     def client
       @client ||= Lutaml::Hal::Client.new(
         api_url: API_URL,
+        connection: connection,
         rate_limiting: rate_limiting_options,
       )
     end
 
+    # Faraday connection mirroring lutaml-hal's default middleware stack, with a
+    # retry layer for the failures lutaml-hal's RateLimiter does not cover: the
+    # W3C API signals rate-limiting with HTTP 403, plus transient connection and
+    # timeout errors. (lutaml-hal still retries 429 and 5xx.) Owning retries here
+    # means every consumer of the client is resilient without its own wrapper.
+    def connection
+      @connection ||= Faraday.new(url: API_URL.delete_suffix("/")) do |conn|
+        conn.request :retry, retry_options
+        conn.use Faraday::FollowRedirects::Middleware
+        conn.request :json
+        conn.response :json, content_type: /\bjson$/
+        conn.adapter Faraday.default_adapter
+      end
+    end
+
+    # Retry policy for the W3C-specific transient failures (HTTP 403 and
+    # connection/timeout). Grows 1, 2, 4, 8, 16s, matching rate_limiting_options.
+    def retry_options
+      {
+        max: 5,
+        interval: 1.0,
+        backoff_factor: 2,
+        max_interval: 30.0,
+        retry_statuses: [403],
+        exceptions: [
+          Errno::ETIMEDOUT, Timeout::Error,
+          Faraday::TimeoutError, Faraday::ConnectionFailed
+        ],
+      }
+    end
+
     # Configure rate limiting options
+    #
+    # lutaml-hal's RateLimiter retries 429 and 5xx responses with exponential
+    # backoff (base_delay * backoff_factor**(attempt - 1), capped at max_delay).
+    # These defaults grow 1, 2, 4, 8, 16s so a rate-limited or briefly
+    # overloaded W3C API is given real room to recover during a bulk crawl.
     def rate_limiting_options
       @rate_limiting_options ||= {
         enabled: true,
         max_retries: 5,
-        base_delay: 0.1,
-        max_delay: 10.0,
-        backoff_factor: 1.5,
+        base_delay: 1.0,
+        max_delay: 30.0,
+        backoff_factor: 2.0,
       }
     end
 
@@ -82,10 +120,45 @@ module W3cApi
       configure_rate_limiting(enabled: true)
     end
 
+    # Cache options for the model register
+    #
+    # lutaml-hal caches realized objects keyed by their (canonical) URL, so a
+    # document linked from many places is fetched once. In-memory by default;
+    # pass an adapter config such as
+    # `{ adapter: { type: :filesystem, options: { path: "..." } } }` for
+    # cross-run persistence. Returns nil when caching is disabled.
+    def cache_options
+      return @cache_options if defined?(@cache_options)
+
+      @cache_options = { adapter: :memory }
+    end
+
+    # Set cache options (merged into the current ones)
+    def configure_cache(options = {})
+      @cache_options = (cache_options || {}).merge(options)
+      reset_register
+    end
+
+    # Disable caching of realized objects
+    def disable_cache
+      @cache_options = nil
+      reset_register
+    end
+
+    # Enable caching of realized objects
+    def enable_cache(options = nil)
+      @cache_options = options || { adapter: :memory }
+      reset_register
+    end
+
     def register
       return @register if @register
 
-      @register = Lutaml::Hal::ModelRegister.new(name: :w3c_api, client: client)
+      @register = Lutaml::Hal::ModelRegister.new(
+        name: :w3c_api,
+        client: client,
+        cache: cache_options,
+      )
       Lutaml::Hal::GlobalRegister.instance.register(:w3c_api, @register)
 
       # Re-run setup to register all endpoints with the new register
@@ -95,6 +168,9 @@ module W3cApi
     end
 
     def reset_register
+      # Drop the global registration too, otherwise rebuilding the register
+      # raises "replacing another one" when it re-registers the same name.
+      Lutaml::Hal::GlobalRegister.instance.unregister(:w3c_api)
       @register = nil
     end
 

data/lib/w3c_api/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module W3cApi
-  VERSION = "0.2.0"
+  VERSION = "0.3.1"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: w3c_api
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.1
 platform: ruby
 authors:
 - Ribose Inc.
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-04-21 00:00:00.000000000 Z
+date: 2026-06-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: faraday
@@ -38,20 +38,34 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: faraday-retry
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
 - !ruby/object:Gem::Dependency
   name: lutaml-hal
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.1.10
+        version: 0.2.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.1.10
+        version: 0.2.0
 - !ruby/object:Gem::Dependency
   name: lutaml-model
   requirement: !ruby/object:Gem::Requirement
@@ -118,6 +132,7 @@ extra_rdoc_files: []
 files:
 - ".rubocop.yml"
 - ".rubocop_todo.yml"
+- CLAUDE.md
 - LICENSE.md
 - README.adoc
 - Rakefile