agentry 3.1.0 → 3.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ca414103cb27f143a4872d36801bd26e012ff0950552ac96646497be9a2bb8f3
-  data.tar.gz: 38d3bb2c5529a324160476b7c5874d645689bb81409234e14ba2ed7f01da4d61
+  metadata.gz: 712d74b9f09a7f5755ade4414301f82b2ee08445c8af1eca147fa8039ead5b92
+  data.tar.gz: bca61032c6cceef348582436594532a5144d71f2483300b151fa4dd053bd95d6
 SHA512:
-  metadata.gz: 755ee15fab59790728e25954113018ca3d25fb1255933a6a029f6dabcdc2767d5781649592c88f0ba6f554f2772319d318df86b9a6009a31eba9926764f01bdf
-  data.tar.gz: ce39ec99a84d2326c2575c3a9ae8c76c3e2205283f4cf51d8b7f7ea5e6b1429e603078e2aac99947970323c89d0ba86902f32aebf014d76936c7d08edadfe215
+  metadata.gz: c48381ca8074cef8ebe58d2a27aab5e5a6441b590525d3ee147bd38f950db5da93c0e0a251b70d842f56cb61cabb85cf183bc7117e498024e8dfe481abc0d204
+  data.tar.gz: 4db69cd9aab4db9689ead804daff544bd1caac63c56e0eb663421392aa7054960e8cbd201f8e4266e9da64a5ee6f913db16d1456cd2537ba31d052de4e0243bf

data/CHANGELOG.md

@@ -2,6 +2,19 @@
 
 ## [Unreleased]
 
+## [3.1.1] — 2026-11-07
+
+Crafted request headers can no longer grow a process for good or make it raise.
+
+**Fixed**
+- Version checks no longer fill RubyGems' process-wide `Gem::Version` cache: 20,000 user agents with unique macOS
+  versions grew a process by 200 MB.
+- An Accept-Language header listing more than 32 languages is parsed on each request instead of cached: 1,000 such
+  headers held 107 MB.
+- Bytes that are not valid UTF-8 in the user agent or the Accept-Language header become U+FFFD instead of raising
+  `ArgumentError`, and such client hints and Cloudflare headers are ignored.
+- `Browser.new` checks the size of the user agent before reading it, so an oversized one is rejected in microseconds.
+
 ## [3.1.0] — 2026-11-06
 
 A Google Cloud VM no longer passes as a verified Googlebot, and a forged Googlebot can no longer stall requests on reverse DNS.
@@ -100,7 +113,8 @@ First release, forked from browser 6.2.0 (`dd88933`). Same API: `gem "agentry"`
 - A corpus of 20,328 user agents, with snapshot and diff scripts proving answers match upstream, and a
   request benchmark.
 
-[Unreleased]: https://github.com/levups/agentry/compare/v3.1.0...HEAD
+[Unreleased]: https://github.com/levups/agentry/compare/v3.1.1...HEAD
+[3.1.1]: https://github.com/levups/agentry/compare/v3.1.0...v3.1.1
 [3.1.0]: https://github.com/levups/agentry/compare/v3.0.0...v3.1.0
 [3.0.0]: https://github.com/levups/agentry/compare/v2.1.0...v3.0.0
 [2.1.0]: https://github.com/levups/agentry/compare/v2.0.0...v2.1.0

data/README.md

@@ -158,6 +158,11 @@ Browser::Bot::IPRanges.dns_timeout = 0.5       # seconds per query and nameserve
 Browser::Bot::IPRanges.refresh                 # fetches the ranges again, in a long-lived process
 ```
 
+The reverse lookup reads answers from a name server the requester chooses, and versions of the `resolv` gem before
+0.7.2 (0.3.2 in the 0.3 series) can be made to use up memory or CPU by such answers (CVE-2026-80212,
+CVE-2025-24294). Before turning `reverse_dns:` on, check `bundle info resolv`, or add `gem "resolv", ">= 0.7.2"`
+to your Gemfile.
+
 ## Cloudflare and Client Hints
 
 When the Rack env is passed as `headers:` (the Rails helper and the middleware do it for you), Agentry reads two
@@ -229,7 +234,14 @@ Browser.cache_size = 5_000 # 0 turns it off
 ```
 
 A user agent or an `Accept-Language` header longer than 2048 bytes raises `Browser::Error`. Both limits can be
-raised with `Browser.user_agent_size_limit = 4096` and `Browser.accept_language_size_limit = 4096`.
+raised with `Browser.user_agent_size_limit = 4096` and `Browser.accept_language_size_limit = 4096`. An
+`Accept-Language` header listing more than 32 languages is parsed on each request instead of cached, so that
+crafted headers cannot fill the cache with thousands of languages each.
+
+Headers whose bytes are not valid UTF-8, forged or read back from a log file, do not raise: Agentry reads the user
+agent and the `Accept-Language` header with each invalid byte replaced by U+FFFD, and ignores such a Client Hint or
+Cloudflare header. Version checks such as `browser.chrome?(">= 65")` leave the versions they read from a request
+out of RubyGems' version cache, where those an attacker sends would pile up for the life of the process.
 
 The lists and matchers are built on first use: the first request of each process takes about 13 ms more, the
 following ones 0.2 ms. `Browser.eager_load!` builds them at boot, so that a server that loads the app before

data/lib/browser/accept_language.rb

@@ -9,6 +9,7 @@ module Browser
     def self.parse(accept_language)
       return [] unless accept_language
 
+      accept_language = accept_language.scrub unless accept_language.valid_encoding?
       languages = accept_language.split(",")
       languages.map! { |string| new(string.squeeze(" ").strip) }
       languages.reject! { |al| al.quality.zero? }

data/lib/browser/base.rb

@@ -9,7 +9,7 @@ module Browser
     # headers: the Rack env (request.env), for the client hints and the
     # Cloudflare headers.
     def initialize(ua, accept_language: nil, headers: nil)
-      validate_size(:user_agent, ua.to_s)
+      Base.validate_size(:user_agent, ua.to_s)
 
       @ua = ua
       @accept_language_raw = accept_language.to_s
@@ -24,7 +24,7 @@ module Browser
     # Return an array with all preferred languages that this browser accepts.
     def accept_language
       @accept_language ||= begin
-        validate_size(:accept_language, @accept_language_raw)
+        Base.validate_size(:accept_language, @accept_language_raw)
         Cache.accept_language(@accept_language_raw)
       end
     end
@@ -288,7 +288,7 @@ module Browser
       @detection ||= Cache.detection(ua)
     end
 
-    private def validate_size(subject, input)
+    def self.validate_size(subject, input)
       actual_bytesize = input.bytesize
       size_limit =
         if subject == :user_agent

data/lib/browser/bot.rb

@@ -117,6 +117,7 @@ module Browser
     private_class_method :trie_source, :trie_node_source
 
     def self.why?(ua)
+      ua = ua.scrub unless ua.valid_encoding?
       ua = ua.downcase.strip
       browser = Browser.new(ua)
       matchers.find { |matcher| matcher.call(ua, browser) }

data/lib/browser/browser.rb

@@ -179,6 +179,10 @@ module Browser
 
   def self.new(user_agent, **)
     user_agent ||= EMPTY_STRING
+    # Before the first regexp: an oversized user agent then costs nothing, and
+    # a byte that is not UTF-8 (forged, or read from a log) would make it raise.
+    Base.validate_size(:user_agent, user_agent.to_s)
+    user_agent = user_agent.scrub unless user_agent.valid_encoding?
     browser_class = Cache.cached_detection(user_agent)&.browser_class
     return browser_class.new(user_agent, **) if browser_class
 

data/lib/browser/cache.rb

@@ -6,6 +6,10 @@ module Browser
   # GVL a Hash read is atomic, and a thread that misses detects again. Writes
   # do, for the Rubies without a GVL.
   module Cache
+    # Far more than a browser sends. A longer list is parsed again each time: 2 KB of "a,a,a" makes a thousand
+    # objects, and a thousand such headers would hold 100 MB.
+    MAX_LANGUAGES_PER_CACHED_HEADER = 32
+
     @mutex = Mutex.new
     @detection_by_user_agent = {}
     @languages_by_header = {}
@@ -28,11 +32,13 @@ module Browser
     def self.accept_language(header)
       return AcceptLanguage.parse(header) if Browser.cache_size.zero?
 
-      languages = @languages_by_header[header] || @mutex.synchronize do
-        store(@languages_by_header, -header,
-          AcceptLanguage.parse(header).freeze)
-      end
-      languages.dup
+      languages = @languages_by_header[header]
+      return languages.dup if languages
+
+      languages = AcceptLanguage.parse(header)
+      return languages if languages.size > MAX_LANGUAGES_PER_CACHED_HEADER
+
+      @mutex.synchronize { store(@languages_by_header, -header, languages.freeze) }.dup
     end
 
     # Bumped when a matcher list, the bot list or its exceptions change: the

data/lib/browser/client_hints.rb

@@ -4,7 +4,7 @@ module Browser
   # The User-Agent Client Hints of Chromium browsers, read from a Rack env.
   # Safari and Firefox send none: every answer is then nil.
   class ClientHints
-    # A browser's hints are a few dozen bytes: longer values are ignored.
+    # A browser's hints are a few dozen bytes: longer values, and invalid UTF-8, are ignored.
     HEADER_SIZE_LIMIT_BYTES = 512
     PLATFORM_CLASS_BY_NAME = {
       "Android" => Platform::Android,
@@ -54,7 +54,7 @@ module Browser
 
     private def header(headers, name)
       value = headers[name]
-      value if value.is_a?(String) && value.bytesize <= HEADER_SIZE_LIMIT_BYTES
+      value if value.is_a?(String) && value.bytesize <= HEADER_SIZE_LIMIT_BYTES && value.valid_encoding?
     end
 
     private def parse_brands(value)

data/lib/browser/cloudflare.rb

@@ -5,7 +5,7 @@ module Browser
   # sets (docs/guide.html#cloudflare) and its own CF-IPCountry. Each site names
   # its headers in its rule: Browser::Cloudflare.header_names= changes them.
   class Cloudflare
-    # Cloudflare's values are a few bytes: a longer one is forged, and ignored.
+    # Cloudflare's values are a few bytes: a longer one, or invalid UTF-8, is forged, and ignored.
     HEADER_SIZE_LIMIT_BYTES = 128
     DEFAULT_HEADER_NAMES = {
       verified_bot: "X-Bot-Verified",
@@ -61,7 +61,7 @@ module Browser
 
     private def header(signal)
       value = @headers&.[](self.class.env_key_by_signal[signal])
-      value if value.is_a?(String) && value.bytesize <= HEADER_SIZE_LIMIT_BYTES
+      value if value.is_a?(String) && value.bytesize <= HEADER_SIZE_LIMIT_BYTES && value.valid_encoding?
     end
   end
 end

data/lib/browser/detect_version.rb

@@ -2,6 +2,21 @@
 
 module Browser
   module DetectVersion
+    # Gem::Version.new and Gem::Version#release keep every version they build in class variables, for the life of
+    # the process: a version read from a header must not reach them, or each new one an attacker sends stays.
+    # A subclass skips the first; this #release is RubyGems' own without the second.
+    class UncachedVersion < Gem::Version
+      def release
+        return self unless prerelease?
+
+        self.class.new(segments.take_while { |segment| segment.is_a?(Integer) }.join("."))
+      end
+    end
+
+    # A browser's version is a few bytes: a longer one is not kept, so that a cached platform stays small.
+    MAX_KEPT_VERSION_BYTES = 32
+    private_constant :UncachedVersion, :MAX_KEPT_VERSION_BYTES
+
     private def detect_version?(actual_version, expected_version)
       return true unless expected_version
       return false unless actual_version
@@ -10,11 +25,21 @@ module Browser
       actual_version = parse_version(actual_version)
 
       Gem::Requirement.create(expected_version)
-        .satisfied_by?(Gem::Version.create(actual_version))
+        .satisfied_by?(gem_version(actual_version))
     rescue ArgumentError
       false
     end
 
+    # The last one built: a cached platform asks for the same version on each request.
+    private def gem_version(version)
+      string, gem_version = @gem_version
+      return gem_version if string == version
+
+      gem_version = UncachedVersion.create(version)
+      @gem_version = [version, gem_version] if version.bytesize <= MAX_KEPT_VERSION_BYTES
+      gem_version
+    end
+
     private def parse_version(version)
       version.is_a?(Numeric) ? version.to_s : version
     end

data/lib/browser/middleware.rb

@@ -60,7 +60,7 @@ module Browser
     end
 
     def html?(request)
-      request.env["HTTP_ACCEPT"].to_s.match(ACCEPT_REGEX)
+      request.env["HTTP_ACCEPT"].to_s.scrub.match(ACCEPT_REGEX)
     end
 
     def assets?(request)

data/lib/browser/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Browser
-  VERSION = "3.1.0"
+  VERSION = "3.1.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: agentry
 version: !ruby/object:Gem::Version
-  version: 3.1.0
+  version: 3.1.1
 platform: ruby
 authors:
 - Nando Vieira
@@ -122,8 +122,10 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '1.0'
-description: 'A drop-in replacement for the browser gem: the same Browser API and
-  require paths, answering nearly twice as fast with half the allocations.'
+description: Agentry replaces the browser gem without changing a call site, with the
+  same Browser class, methods and require paths. It is faster, updates its bot list
+  every week, names the category of each bot and checks the IP addresses of known
+  crawlers.
 executables: []
 extensions: []
 extra_rdoc_files: []
@@ -270,5 +272,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 4.0.20
 specification_version: 4
-summary: 'Browser detection for Ruby: the browser gem''s API, faster'
+summary: Detects the browser, platform, device and bot behind a User-Agent header
 test_files: []