howlongtobeat 0.2.4 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5e8314a8423ed470f6b0f5f80097b903ece915d2a6eabc356cdb34104dacb929
-  data.tar.gz: 7e6c9c045ee96345593fa961007988a96b78ffd4dd1f44b0f4529f79c14f2ecd
+  metadata.gz: 460063aabeca602d857275839c6454dd8c0fb8d4eaa17f8e0a986c08d72cb3f3
+  data.tar.gz: a45bd306c0fb36cc2006b1bdde32d5b361cce1eb405e8f71ce31ef4aabd32dfa
 SHA512:
-  metadata.gz: e6cb621ce224ad454077852c33e4295f8bab8394c81865703c397a93f6d83d710142cd426b43f14e9d94bca24e2f9e9aafb1ab800db66709864f7a7531d29b27
-  data.tar.gz: e59d86c29009a8e2759874ef021c0ff49285d797778454ef60cb639d8d8899412ce445d5cf6180c3d6cfc756b6590b60324abffda6f305f6ca5648b61c6eb630
+  metadata.gz: cf147ef1c998ee37ce46a1d676614fa233617555a9095bc89f12f24f6ef2619a2f9340781fc866a500b45c1500c4efcc8a638cd26f3bc02b0a12a3be49c8fb57
+  data.tar.gz: 18ff0a91a30dfa9b5d0380935c5d2053d586dfdb253da52a72f40e571f793ee8d7cb9e8075a290ec6e03b24a27c58fce040c8cde0f27391b5bb21559ad3d1c7b

data/CHANGELOG.md

@@ -25,6 +25,44 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Security
 - None
 
+## [0.3.0] - 2026-10-05
+
+HowLongToBeat started answering 429 to bursts of requests in late September 2026, which is why this release paces them.
+
+### Added
+- `HowLongToBeat::Client`: a stateful client meant to be kept for the life of a process.
+  - `#game(id)` reads a game with one request to its HLTB page. It returns a `GameDetail`: times in hours, release date, aliases and Steam app id.
+  - `#search(title)` returns raw `SearchResult` rows; `#search_json(title)` returns the raw body.
+- `HowLongToBeat::Client.default`: a lazily built, process-wide client. The legacy `HowLongToBeat::HowLongToBeat` API uses it unless you pass `client:`, so instances no longer each pay for a full endpoint discovery.
+- Distinct errors: `RateLimitedError` (429), `RequestError` (network, timeout, 5xx, 403, a 404 that is not HLTB's own; `#status` carries the HTTP status) and `ParseError` (unexpected page or response shape, a `/init` response without a token, or a search endpoint that cannot be found).
+
+### Changed
+- Requests are paced: at least 2 seconds apart by default.
+- The search endpoint is cached for an hour and the `/init` token for 60 seconds, with a 1.2 second pause before the first search with a new token.
+- `HowLongToBeat#search` and `#search_from_id` now run on `Client`. `search_from_id` takes one request instead of about eight.
+- One current, fixed browser User-Agent.
+- Explicit 5-second connect and 10-second read timeouts.
+- `HowLongToBeat#search_from_id` now builds its entry from the game page: it sets id, name, alias, type, web link, release year, the six times and similarity (1.0); image URL, review score, developer, platforms, JSON content and complexity flags stay unset, and `auto_filter_times` no longer applies to it.
+
+### Deprecated
+- The public `HTMLRequests.send_web_request` and `HTMLRequests.get_game_title` are unpaced legacy helpers. Use `Client` instead.
+
+### Removed
+- After a 429 the client no longer cycles through retired endpoints.
+- The fallback that disabled SSL certificate verification is gone (applies to the new client).
+
+### Fixed
+- `HowLongToBeat#search` and `#search_from_id` return nil for malformed responses and non-numeric ids instead of raising.
+
+## [0.2.5] - 2026-09-04
+
+### Fixed
+- HLTB moved the search endpoint from `/api/bleed` to the nested path `/api/search/site` (around 2026-08-26). Runtime discovery matched the new `fetch("/api/search/site", { method: "POST" })` call but truncated it to the first path segment (`/api/search`), whose `/init` returns 404. Every search and `search_from_id` returned `nil`. The full path is now kept verbatim (mirrors Python package fix for ScrappyCocco/HowLongToBeat-PythonAPI#59).
+- Refresh the hardcoded fallbacks (`SEARCH_URL`, the `fetch_search_token` default, and the candidate list) to put `/api/search/site` first.
+
+### Changed
+- Discovery now prefers the POST fetch that sends `x-auth-token` (the search call's signature). The HLTB bundle also contains an unrelated `fetch("/api/error", { method: "POST" })` chunk and chunk order is not stable, so stopping on the first POST fetch could pick the wrong endpoint. `SearchInfo#authenticated?` exposes the distinction; a plain POST fetch is still used as a last resort.
+
 ## [0.2.4] - 2026-05-07
 
 ### Fixed

data/README.md

@@ -1,6 +1,7 @@
 # HowLongToBeat Ruby API
 
 ![CI](https://github.com/dpashutskii/howlongtobeat/actions/workflows/ci.yml/badge.svg)
+[![Buy Me a Coffee](https://img.shields.io/badge/Buy_Me_a_Coffee-FFDD00?logo=buymeacoffee&logoColor=black)](https://buymeacoffee.com/dpashutskii)
 
 A simple Ruby API to read data from howlongtobeat.com.
 
@@ -26,6 +27,30 @@ $ gem install howlongtobeat
 
 ## Usage
 
+### Client (recommended)
+
+Keep one `Client` per process: use `HowLongToBeat::Client.default`, or build a single instance of your own. It paces requests (2 seconds apart by default) and caches the search endpoint and auth token, so every extra instance pays for its own endpoint discovery. Errors are distinct, so "rate-limited" never looks like "not found". The legacy `HowLongToBeat::HowLongToBeat` API uses `Client.default` unless you pass `client:`.
+
+```ruby
+client = HowLongToBeat::Client.default
+
+detail = client.game(10270)          # one request; nil if HLTB's 404 page says there is no such game
+detail.main_story                    # => 51.68 (hours)
+detail.steam_app_id                  # => 292030
+
+begin
+  client.search("The Witcher 3")     # => [HowLongToBeat::SearchResult, ...]; [] when nothing matches
+rescue HowLongToBeat::RateLimitedError
+  # HLTB answered 429: back off before trying again
+rescue HowLongToBeat::RequestError
+  # network failure, timeout or an HTTP error: retry later
+rescue HowLongToBeat::ParseError
+  # HLTB changed its site, so the gem needs an update
+end
+```
+
+HowLongToBeat has no official API and rate-limits bursts. Please keep the default pacing.
+
 ### Basic Search
 
 ```ruby
@@ -35,7 +60,7 @@ hltb = HowLongToBeat::HowLongToBeat.new
 results = hltb.search("The Witcher 3")
 ```
 
-The `search` method returns an array of possible games, or `nil` if no results were found or there was an error in the request.
+The `search` method returns an array of possible games (`[]` when nothing matches), or `nil` if the game name is empty or the request failed.
 
 Each result is an [`HowLongToBeatEntry`](https://github.com/dpashutskii/howlongtobeat/blob/main/lib/howlongtobeat/how_long_to_beat_entry.rb) object containing:
 
@@ -79,6 +104,8 @@ result = hltb.search_from_id(10270)  # The Witcher 3: Wild Hunt
 
 This returns a single [`HowLongToBeatEntry`](https://github.com/dpashutskii/howlongtobeat/blob/main/lib/howlongtobeat/how_long_to_beat_entry.rb) object or `nil` if not found.
 
+Since 0.3.0 `search_from_id` reads the game page in one request, so it fills fewer fields than before (no image URL, review score, developer, platforms, JSON content or complexity flags). See the [CHANGELOG](CHANGELOG.md) for the exact list.
+
 ### Search Modifiers
 
 You can filter your search results using modifiers:
@@ -115,7 +142,7 @@ This gem was originally created for and is being used by [SearchToPlay](https://
 
 ## Development
 
-After checking out the repo, run `bundle install` to install dependencies. Then, run `rake test` to run the tests.
+After checking out the repo, run `bundle install` to install dependencies. Then run `bundle exec rspec --tag '~live'` to run the offline tests. `rake spec` runs everything, including the `live` specs that call howlongtobeat.com (paced, about 3 minutes); run those with `bundle exec rspec --tag live` before a release.
 
 ## Contributing
 

data/lib/howlongtobeat/client.rb

@@ -0,0 +1,178 @@
+require 'json'
+require 'nokogiri'
+
+module HowLongToBeat
+  # Polite, stateful HLTB client. Keep one instance per process (Client.default
+  # is one): it remembers the discovered search endpoint and the auth token,
+  # and its Http enforces a minimum gap between requests. Safe to share
+  # between threads.
+  class Client
+    BASE_URL = Http::BASE_URL
+
+    # Tried in order only when no script chunk reveals the search endpoint.
+    KNOWN_SEARCH_PATHS = %w[/api/search/site /api/bleed /api/finder /api/search /api/s].freeze
+
+    # HLTB's own Next.js 404 page embeds this in its __NEXT_DATA__.
+    NOT_FOUND_MARKER = '"page":"/404"'.freeze
+
+    Endpoint = Struct.new(:path, :search_info, :found_at)
+    Token = Struct.new(:auth, :issued_at)
+    EndpointGone = Class.new(StandardError)
+    DEFAULT_LOCK = Mutex.new
+    private_constant :Endpoint, :Token, :EndpointGone, :DEFAULT_LOCK
+
+    # The process-wide client, built on first use. Share it so every caller
+    # pays for one endpoint discovery and one paced stream of requests.
+    def self.default
+      DEFAULT_LOCK.synchronize { @default ||= new }
+    end
+
+    def initialize(http: Http.new, endpoint_ttl: 3600, token_ttl: 60, token_warmup: 1.2,
+                   clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) },
+                   sleeper: ->(seconds) { sleep(seconds) })
+      @http = http
+      @endpoint_ttl = endpoint_ttl
+      @token_ttl = token_ttl
+      @token_warmup = token_warmup
+      @clock = clock
+      @sleeper = sleeper
+      @endpoint = nil
+      @token = nil
+      @lock = Mutex.new # guards @endpoint and @token; never held during the search POST
+    end
+
+    # One GET of the game page. nil only when HLTB's own 404 page says there
+    # is no such game; any other 404 (a CDN or proxy page) is a RequestError.
+    def game(id)
+      response = @http.get("#{BASE_URL}/game/#{Integer(id.to_s, 10)}")
+      if response.code == 404
+        return nil if response.body.include?(NOT_FOUND_MARKER)
+
+        raise RequestError.new("HLTB returned an unrecognised 404 for /game/#{id}", status: 404)
+      end
+
+      GamePageParser.parse(response.body)
+    end
+
+    # Raw candidates for a title. [] is a genuine "not on HLTB".
+    def search(title, modifier: HTMLRequests::SearchModifiers::NONE)
+      SearchResultParser.parse(search_json(title, modifier: modifier))
+    end
+
+    # The raw search response body (the legacy API parses it itself).
+    def search_json(title, modifier: HTMLRequests::SearchModifiers::NONE)
+      post_search(title, modifier)
+    rescue EndpointGone
+      forget_endpoint!
+      begin
+        post_search(title, modifier)
+      rescue EndpointGone
+        raise ParseError, 'HLTB search endpoint returns 404 even after rediscovery'
+      end
+    end
+
+    private
+
+    def post_search(title, modifier)
+      endpoint, auth = @lock.synchronize do
+        current = current_endpoint
+        [current, current_token(current)]
+      end
+      headers = HTMLRequests.get_search_request_headers(auth)
+      body = HTMLRequests.get_search_request_data(title, modifier, 1, endpoint.search_info, auth)
+      response = @http.post("#{BASE_URL}#{endpoint.path}", headers, body)
+      raise EndpointGone if response.code == 404
+
+      response.body
+    end
+
+    # Callers hold @lock, so the ivars are read once and updated without a race.
+    def current_endpoint
+      endpoint = @endpoint
+      return endpoint if endpoint && @clock.call - endpoint.found_at < @endpoint_ttl
+
+      @token = nil
+      @endpoint = discover
+    end
+
+    def forget_endpoint!
+      @lock.synchronize do
+        @endpoint = nil
+        @token = nil
+      end
+    end
+
+    def current_token(endpoint)
+      cached = @token
+      return cached.auth if cached && @clock.call - cached.issued_at < @token_ttl
+
+      response = @http.get(init_url(endpoint.path))
+      raise EndpointGone if response.code == 404
+
+      store_token(response.body)
+    end
+
+    # Parses an /init body, caches the token, and gives HLTB a moment to
+    # accept it before the first search.
+    def store_token(body)
+      @token = fresh = Token.new(parse_auth(body), @clock.call)
+      @sleeper.call(@token_warmup)
+      fresh.auth
+    end
+
+    def init_url(path)
+      "#{BASE_URL}#{path}/init?t=#{Time.now.to_i}"
+    end
+
+    def parse_auth(body)
+      json = JSON.parse(body.to_s)
+      raise ParseError, 'HLTB /init response is not an object' unless json.is_a?(Hash)
+
+      token = json['token'] || json.dig('data', 'token') || json['auth_token'] || json['authToken']
+      raise ParseError, 'HLTB /init response has no token' if token.to_s.empty?
+
+      key = json.find { |name, _| name.downcase.include?('key') }&.last
+      value = json.find { |name, _| name.downcase.include?('val') }&.last
+      HTMLRequests::AuthStruct.new(token, key, value)
+    rescue JSON::ParserError => e
+      raise ParseError, "HLTB /init response is not JSON: #{e.message}"
+    end
+
+    # The search call is the POST fetch in HLTB's JS bundle that sends
+    # x-auth-token. Chunk names are opaque, so walk them in page order.
+    def discover
+      homepage = @http.get(BASE_URL)
+      raise RequestError, 'HLTB homepage returned 404' if homepage.code == 404
+
+      Nokogiri::HTML(homepage.body).css('script[src]').each do |script|
+        info = search_info_from(script['src'])
+        return Endpoint.new("/#{info.search_url}", info, @clock.call) if info
+      end
+
+      fallback_endpoint
+    end
+
+    def search_info_from(src)
+      url = src.start_with?('http') ? src : "#{BASE_URL}#{src}"
+      response = @http.get(url)
+      return nil unless response.code == 200
+
+      info = HTMLRequests::SearchInfo.new(response.body)
+      info if info.authenticated? && !info.search_url.to_s.empty?
+    rescue RequestError
+      nil # a redirected or broken chunk (e.g. the consent script) is not fatal
+    end
+
+    def fallback_endpoint
+      KNOWN_SEARCH_PATHS.each do |path|
+        response = @http.get(init_url(path))
+        next unless response.code == 200
+
+        store_token(response.body) # the probe already fetched /init; don't fetch it again
+        return Endpoint.new(path, nil, @clock.call)
+      end
+
+      raise ParseError, 'Could not discover the HLTB search endpoint'
+    end
+  end
+end

data/lib/howlongtobeat/errors.rb

@@ -0,0 +1,21 @@
+module HowLongToBeat
+  class Error < StandardError; end
+
+  # HLTB answered 429. Back off; don't retry right away.
+  class RateLimitedError < Error; end
+
+  # Network failure, timeout, or an HTTP status that means "try later"
+  # (5xx, 403, unexpected redirects). Never means "not found".
+  # `status` is the HTTP status when there was one, nil for network failures.
+  class RequestError < Error
+    attr_reader :status
+
+    def initialize(message = nil, status: nil)
+      super(message)
+      @status = status
+    end
+  end
+
+  # HLTB answered, but not in the shape we know. Usually a site change.
+  class ParseError < Error; end
+end

data/lib/howlongtobeat/fields.rb

@@ -0,0 +1,46 @@
+require 'date'
+
+module HowLongToBeat
+  # Conversions shared by the game-page and search parsers.
+  module Fields
+    module_function
+
+    # HLTB reports times in seconds; 0 means "no data".
+    def hours(seconds)
+      value = seconds.to_f
+      value.positive? ? (value / 3600.0).round(2) : nil
+    end
+
+    def aliases(value)
+      value.to_s.split(',').map(&:strip).reject(&:empty?)
+    end
+
+    def positive_int(value)
+      number = value.to_i
+      number.positive? ? number : nil
+    end
+
+    # Game pages send "2015-05-19"; search rows send 2015.
+    def release_date(value)
+      Date.iso8601(value.to_s) if value.to_s.match?(/\A\d{4}-\d{2}-\d{2}\z/)
+    rescue Date::Error
+      nil
+    end
+
+    def release_year(value)
+      year = value.to_s[/\A\d{4}/].to_i
+      year.positive? ? year : nil
+    end
+
+    def times(game)
+      {
+        main_story: hours(game['comp_main']),
+        main_extra: hours(game['comp_plus']),
+        completionist: hours(game['comp_100']),
+        all_styles: hours(game['comp_all']),
+        coop: hours(game['invested_co']),
+        multiplayer: hours(game['invested_mp'])
+      }
+    end
+  end
+end

data/lib/howlongtobeat/game_page_parser.rb

@@ -0,0 +1,37 @@
+require 'json'
+
+module HowLongToBeat
+  # Reads the record Next.js embeds in every /game/<id> page, so refreshing
+  # a known game takes one GET and no search or auth token.
+  module GamePageParser
+    NEXT_DATA = %r{<script[^>]*\bid="__NEXT_DATA__"[^>]*>(.*?)</script>}m
+
+    module_function
+
+    def parse(html)
+      encoded_html = html.to_s.dup.force_encoding(Encoding::UTF_8).scrub
+      json = encoded_html[NEXT_DATA, 1]
+      raise ParseError, 'HLTB game page has no __NEXT_DATA__ script' unless json
+
+      data = JSON.parse(json)
+      game = data.dig('props', 'pageProps', 'game', 'data', 'game')
+      game = game.first if game.is_a?(Array)
+      raise ParseError, 'HLTB game page has no game record' unless game.is_a?(Hash) && game['game_id']
+
+      GameDetail.new(
+        id: game['game_id'].to_i,
+        name: game['game_name'],
+        aliases: Fields.aliases(game['game_alias']),
+        release_date: Fields.release_date(game['release_world']),
+        release_year: Fields.release_year(game['release_world']),
+        steam_app_id: Fields.positive_int(game['profile_steam']),
+        game_type: game['game_type'],
+        **Fields.times(game)
+      )
+    rescue JSON::ParserError => e
+      raise ParseError, "HLTB game page JSON is invalid: #{e.message}"
+    rescue TypeError, NoMethodError => e
+      raise ParseError, "HLTB game page has unexpected structure: #{e.message}"
+    end
+  end
+end

data/lib/howlongtobeat/how_long_to_beat.rb

@@ -1,37 +1,66 @@
 module HowLongToBeat
+  # The original API, kept for existing callers. It now runs on Client, so
+  # it shares the pacing, caching and error handling; any failure still
+  # returns nil, as before.
+  #
+  # Note: search_from_id now builds entries from the game page only. It sets
+  # id, name, alias, type, web link, release year, and the six time fields
+  # (main_story, main_extra, completionist, all_styles, coop_time, mp_time)
+  # plus similarity (1.0). Image URL, review score, developer, platforms,
+  # JSON content, and complexity flags stay unset. auto_filter_times does
+  # not apply to search_from_id results.
   class HowLongToBeat
-    def initialize(input_minimum_similarity = 0.4, input_auto_filter_times = false)
+    def initialize(input_minimum_similarity = 0.4, input_auto_filter_times = false, client: Client.default)
       @minimum_similarity = input_minimum_similarity
       @auto_filter_times = input_auto_filter_times
+      @client = client
     end
 
     def search(game_name, search_modifiers = HTMLRequests::SearchModifiers::NONE,
-              similarity_case_sensitive = true)
+               similarity_case_sensitive = true)
       return nil if game_name.nil? || game_name.empty?
 
-      html_result = HTMLRequests.send_web_request(game_name, search_modifiers)
-      return nil unless html_result
-
-      parse_web_result(game_name, html_result, nil, similarity_case_sensitive)
+      json = @client.search_json(game_name, modifier: search_modifiers)
+      # Validate the shape first; SearchResultParser raises ParseError for any
+      # malformed response, which rescue Error below will turn into nil.
+      SearchResultParser.parse(json)
+      parse_web_result(game_name, json, nil, similarity_case_sensitive)
+    rescue Error
+      nil
     end
 
     def search_from_id(game_id)
       return nil if game_id.nil? || game_id == 0
+      # Validate the ID is numeric before calling the client; game_id is expected to be
+      # an integer, but Integer() will coerce strings like "123" and raise ArgumentError on "abc"
+      return nil unless game_id.is_a?(Integer) || (game_id.is_a?(String) && game_id.match?(/\A\d+\z/))
 
-      game_title = HTMLRequests.get_game_title(game_id)
-      return nil unless game_title
-
-      html_result = HTMLRequests.send_web_request(game_title)
-      return nil unless html_result
-
-      result_list = parse_web_result(game_title, html_result, game_id)
-      return nil unless result_list && result_list.size == 1
-
-      result_list.first
+      detail = @client.game(game_id)
+      detail && entry_from(detail)
+    rescue Error, ArgumentError
+      nil
     end
 
     private
 
+    def entry_from(detail)
+      HowLongToBeatEntry.new.tap do |entry|
+        entry.game_id = detail.id
+        entry.game_name = detail.name
+        entry.game_alias = detail.aliases.join(', ')
+        entry.game_type = detail.game_type
+        entry.game_web_link = "#{JSONResultParser::GAME_URL_PREFIX}#{detail.id}"
+        entry.release_world = detail.release_year
+        entry.main_story = detail.main_story
+        entry.main_extra = detail.main_extra
+        entry.completionist = detail.completionist
+        entry.all_styles = detail.all_styles
+        entry.coop_time = detail.coop
+        entry.mp_time = detail.multiplayer
+        entry.similarity = 1.0
+      end
+    end
+
     def parse_web_result(game_name, html_result, game_id = nil, similarity_case_sensitive = true)
       parser = JSONResultParser.new(
         game_name,

data/lib/howlongtobeat/html_requests.rb

@@ -8,10 +8,11 @@ module HowLongToBeat
     BASE_URL = 'https://howlongtobeat.com'
     REFERER_HEADER = BASE_URL
     GAME_URL = "#{BASE_URL}/game"
-    # HLTB renames this endpoint periodically (most recent: /api/find -> /api/finder -> /api/bleed).
+    # HLTB renames this endpoint periodically
+    # (most recent: /api/find -> /api/finder -> /api/bleed -> /api/search/site).
     # The runtime discovery in `send_website_request_getcode` is the source of truth;
     # this constant is the fallback when discovery fails.
-    SEARCH_URL = "#{BASE_URL}/api/bleed"
+    SEARCH_URL = "#{BASE_URL}/api/search/site"
 
     class SearchModifiers
       NONE = ""
@@ -24,12 +25,25 @@ module HowLongToBeat
     class SearchInfo
       attr_accessor :search_url, :api_key
 
+      # A POST fetch to /api/... whose request options mention x-auth-token.
+      # This is the signature of the search call and distinguishes it from
+      # other POST fetches in the bundle (e.g. /api/error).
+      AUTHENTICATED_FETCH_MARKER = /x-auth-token/i
+
       def initialize(script_content)
         @api_key = extract_api_from_script(script_content)
+        @authenticated = false
         @search_url = extract_search_url_script(script_content)
         @search_url = @search_url&.strip&.gsub(/^\/+|\/+$/, '')
       end
 
+      # True when the discovered search_url came from a fetch that sends
+      # x-auth-token. Callers use this to prefer the real search endpoint over
+      # any other POST fetch found in an earlier script.
+      def authenticated?
+        @authenticated
+      end
+
       private
 
       def extract_api_from_script(script_content)
@@ -51,13 +65,21 @@ module HowLongToBeat
 
       def extract_search_url_script(script_content)
         # Prefer the search endpoint used in POST fetch calls, which is more stable
-        # than hardcoding "/api/search" and works with variants like "/api/finder"
-        # and "/api/bleed" (current as of 2026-05).
-        post_fetch_pattern = /fetch\s*\(\s*["']\/api\/([a-zA-Z0-9_\/-]+)[^"']*["']\s*,\s*{[^}]*method:\s*["']POST["'][^}]*}/mi
-        if (match = script_content.match(post_fetch_pattern))
-          path_suffix = match[1]
-          base_path = path_suffix.include?('/') ? path_suffix.split('/').first : path_suffix
-          return "/api/#{base_path}"
+        # than hardcoding "/api/search" and works with variants like "/api/finder",
+        # "/api/bleed" and nested paths like "/api/search/site" (current as of
+        # 2026-09). The full path is kept verbatim: truncating to the first
+        # segment turned "/api/search/site" into "/api/search", which 404s.
+        post_fetch_pattern = /fetch\s*\(\s*["']\/api\/([a-zA-Z0-9_\/-]+)[^"']*["']\s*,\s*({[^}]*method:\s*["']POST["'][^}]*})/mi
+        matches = script_content.to_enum(:scan, post_fetch_pattern).map { Regexp.last_match }
+        unless matches.empty?
+          # A single chunk can contain several POST fetches; the search one is
+          # the one that sends x-auth-token. Fall back to the first POST fetch
+          # so a header rename doesn't leave us with nothing.
+          match = matches.find { |m| m[2].match?(AUTHENTICATED_FETCH_MARKER) }
+          @authenticated = !match.nil?
+          match ||= matches.first
+          path_suffix = match[1].sub(%r{/+\z}, '')
+          return "/api/#{path_suffix}"
         end
 
         # Legacy fallback for previous script shape using .concat(...)
@@ -182,7 +204,7 @@ module HowLongToBeat
 
       def fetch_search_token(parsed_search_url = nil)
         base_endpoint = parsed_search_url.to_s.strip
-        base_endpoint = '/api/bleed' if base_endpoint.empty?
+        base_endpoint = '/api/search/site' if base_endpoint.empty?
         base_endpoint = "/#{base_endpoint}" unless base_endpoint.start_with?('/')
         base_endpoint = base_endpoint.sub(%r{/+\z}, '')
         base_endpoint = base_endpoint.sub(%r{/init\z}, '')
@@ -218,7 +240,7 @@ module HowLongToBeat
         candidates = []
         candidates << preferred unless preferred.empty?
         # Known historical endpoints, newest first. HLTB rotates this name periodically.
-        candidates.concat(['/api/bleed', '/api/finder', '/api/search', '/api/s'])
+        candidates.concat(['/api/search/site', '/api/bleed', '/api/finder', '/api/search', '/api/s'])
 
         normalized = candidates.map do |endpoint|
           next nil if endpoint.nil? || endpoint.strip.empty?
@@ -234,8 +256,12 @@ module HowLongToBeat
       # contains a `fetch("/api/<name>", { method: "POST" })` call. HLTB used to
       # bundle the relevant code under `_app-*.js`, but the modern (Turbopack)
       # build emits opaque chunk names like `0-~-0up.q3_p0.js`, so a name-based
-      # filter is no longer reliable — we just iterate and stop on the first
-      # script that yields a `search_url`.
+      # filter is no longer reliable — we iterate every script.
+      #
+      # The bundle also contains unrelated POST fetches (e.g. `/api/error`), and
+      # chunk order is not stable, so we stop on the first chunk whose POST fetch
+      # sends `x-auth-token` (the search call's signature) and only fall back to
+      # the first plain POST fetch if no chunk is authenticated.
       def send_website_request_getcode
         headers = get_title_request_headers
         response = make_get_request(BASE_URL, headers)
@@ -244,19 +270,22 @@ module HowLongToBeat
         doc = Nokogiri::HTML(response)
         script_urls = doc.css('script[src]').map { |script| script['src'] }
 
+        fallback = nil
         script_urls.each do |script_url|
           url = script_url.start_with?('http') ? script_url : "#{BASE_URL}#{script_url}"
           script_content = make_get_request(url, headers)
           next unless script_content
 
           search_info = SearchInfo.new(script_content)
-          # Only return on a search_url match — an api_key without a search_url
-          # leaves us with no idea where to POST, and the loop should keep
-          # looking for a chunk that gives us the endpoint.
-          return search_info if search_info.search_url && !search_info.search_url.empty?
+          # Only consider a search_url match — an api_key without a search_url
+          # leaves us with no idea where to POST.
+          next unless search_info.search_url && !search_info.search_url.empty?
+
+          return search_info if search_info.authenticated?
+          fallback ||= search_info
         end
 
-        nil
+        fallback
       end
 
       def get_title_request_headers
@@ -310,7 +339,7 @@ module HowLongToBeat
       end
 
       def random_user_agent
-        'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
+        Http::USER_AGENT
       end
     end
   end

data/lib/howlongtobeat/http.rb

@@ -0,0 +1,87 @@
+require 'net/http'
+require 'openssl'
+require 'uri'
+require 'zlib'
+
+module HowLongToBeat
+  # Paced HTTP transport for HLTB. Every request waits until at least
+  # `min_interval` seconds have passed since the previous one, however the
+  # caller loops. 2xx and 404 come back as a Response; 429 raises
+  # RateLimitedError; anything else raises RequestError, so a failure can
+  # never look like "not found".
+  class Http
+    Response = Struct.new(:code, :body)
+
+    USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 ' \
+                 '(KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36'
+    BASE_URL = 'https://howlongtobeat.com'
+
+    NETWORK_ERRORS = [
+      Timeout::Error, IOError, EOFError, SocketError, SystemCallError,
+      OpenSSL::SSL::SSLError, Net::HTTPBadResponse, Net::ProtocolError, Zlib::Error
+    ].freeze
+
+    def initialize(min_interval: 2.0, open_timeout: 5, read_timeout: 10,
+                   clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) },
+                   sleeper: ->(seconds) { sleep(seconds) })
+      @min_interval = min_interval
+      @open_timeout = open_timeout
+      @read_timeout = read_timeout
+      @clock = clock
+      @sleeper = sleeper
+      @mutex = Mutex.new
+      @last_request_at = nil
+    end
+
+    def get(url, headers = {})
+      uri = URI(url)
+      perform(uri, Net::HTTP::Get.new(uri.request_uri, default_headers.merge(headers)))
+    end
+
+    def post(url, headers, body)
+      uri = URI(url)
+      request = Net::HTTP::Post.new(uri.request_uri, default_headers.merge(headers))
+      request.body = body
+      perform(uri, request)
+    end
+
+    private
+
+    def perform(uri, request)
+      pace!
+      # Disable Net::HTTP's silent retry (max_retries: 1 by default) so every request,
+      # including automatic retries after transient errors, goes through pace!.
+      response = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https',
+                                                     max_retries: 0,
+                                                     open_timeout: @open_timeout,
+                                                     read_timeout: @read_timeout) do |connection|
+        connection.request(request)
+      end
+      to_response(uri, response)
+    rescue *NETWORK_ERRORS => e
+      raise RequestError, "#{request.method} #{uri.path} failed: #{e.class}: #{e.message}"
+    end
+
+    def to_response(uri, response)
+      code = response.code.to_i
+      raise RateLimitedError, "HLTB rate limited #{uri.path} (429)" if code == 429
+      return Response.new(code, response.body.to_s) if (200..299).cover?(code) || code == 404
+
+      raise RequestError.new("HLTB returned #{code} for #{uri.path}", status: code)
+    end
+
+    def pace!
+      @mutex.synchronize do
+        if @last_request_at
+          wait = @min_interval - (@clock.call - @last_request_at)
+          @sleeper.call(wait) if wait.positive?
+        end
+        @last_request_at = @clock.call
+      end
+    end
+
+    def default_headers
+      { 'User-Agent' => USER_AGENT, 'Referer' => BASE_URL, 'Origin' => BASE_URL, 'Accept' => '*/*' }
+    end
+  end
+end

data/lib/howlongtobeat/records.rb

@@ -0,0 +1,17 @@
+module HowLongToBeat
+  # A game as its /game/<id> page describes it. Times are hours, nil when
+  # HLTB has no data.
+  GameDetail = Struct.new(
+    :id, :name, :aliases, :release_date, :release_year, :steam_app_id, :game_type,
+    :main_story, :main_extra, :completionist, :all_styles, :coop, :multiplayer,
+    keyword_init: true
+  )
+
+  # One row of a title search. Search rows carry a release year, not a date,
+  # and no Steam id; fetch the game page for those.
+  SearchResult = Struct.new(
+    :id, :name, :aliases, :release_year, :game_type,
+    :main_story, :main_extra, :completionist, :all_styles, :coop, :multiplayer,
+    keyword_init: true
+  )
+end

data/lib/howlongtobeat/search_result_parser.rb

@@ -0,0 +1,33 @@
+require 'json'
+
+module HowLongToBeat
+  # Turns the search API response into SearchResult rows. An empty `data`
+  # array is a genuine "not on HLTB"; any other shape is a ParseError.
+  module SearchResultParser
+    module_function
+
+    def parse(json)
+      payload = JSON.parse(json.to_s.dup.force_encoding(Encoding::UTF_8).scrub)
+      games = payload['data'] if payload.is_a?(Hash)
+      raise ParseError, 'HLTB search response has no data array' unless games.is_a?(Array)
+
+      games.map do |game|
+        raise ParseError, 'HLTB search response game row is not a dict' unless game.is_a?(Hash)
+        raise ParseError, 'HLTB search response game row has no game_id' unless game['game_id']
+
+        SearchResult.new(
+          id: game['game_id'].to_i,
+          name: game['game_name'],
+          aliases: Fields.aliases(game['game_alias']),
+          release_year: Fields.release_year(game['release_world']),
+          game_type: game['game_type'],
+          **Fields.times(game)
+        )
+      end
+    rescue JSON::ParserError => e
+      raise ParseError, "HLTB search response is not JSON: #{e.message}"
+    rescue NoMethodError, TypeError => e
+      raise ParseError, "HLTB search response has unexpected structure: #{e.message}"
+    end
+  end
+end

data/lib/howlongtobeat/version.rb

@@ -1,3 +1,3 @@
 module HowLongToBeat
-  VERSION = "0.2.4"
+  VERSION = "0.3.0"
 end

data/lib/howlongtobeat.rb

@@ -1,9 +1,12 @@
 require_relative 'howlongtobeat/version'
+require_relative 'howlongtobeat/errors'
+require_relative 'howlongtobeat/http'
+require_relative 'howlongtobeat/records'
+require_relative 'howlongtobeat/fields'
+require_relative 'howlongtobeat/game_page_parser'
+require_relative 'howlongtobeat/search_result_parser'
 require_relative 'howlongtobeat/how_long_to_beat'
 require_relative 'howlongtobeat/how_long_to_beat_entry'
 require_relative 'howlongtobeat/html_requests'
 require_relative 'howlongtobeat/json_result_parser'
-
-module HowLongToBeat
-  class Error < StandardError; end
-end
+require_relative 'howlongtobeat/client'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: howlongtobeat
 version: !ruby/object:Gem::Version
-  version: 0.2.4
+  version: 0.3.0
 platform: ruby
 authors:
 - Dmitrii Pashutskii
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-05-07 00:00:00.000000000 Z
+date: 2026-10-06 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: nokogiri
@@ -80,6 +80,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.23'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.23'
 description: A simple Ruby gem to fetch game completion times from HowLongToBeat.com
 email:
 - dpashutskii@gmail.com
@@ -91,10 +105,17 @@ files:
 - CODE_OF_CONDUCT.md
 - README.md
 - lib/howlongtobeat.rb
+- lib/howlongtobeat/client.rb
+- lib/howlongtobeat/errors.rb
+- lib/howlongtobeat/fields.rb
+- lib/howlongtobeat/game_page_parser.rb
 - lib/howlongtobeat/how_long_to_beat.rb
 - lib/howlongtobeat/how_long_to_beat_entry.rb
 - lib/howlongtobeat/html_requests.rb
+- lib/howlongtobeat/http.rb
 - lib/howlongtobeat/json_result_parser.rb
+- lib/howlongtobeat/records.rb
+- lib/howlongtobeat/search_result_parser.rb
 - lib/howlongtobeat/version.rb
 homepage: https://github.com/dpashutskii/howlongtobeat
 licenses: