whois-parser-next 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 64090073c595bd3cbd6e1a52628cf6d3481de3933b791d9338bf1793922dc6a4
-  data.tar.gz: b15050ea47d7662aedb25a2f0e6e0e8997c7c825b2de70e149dce0f85fd7dd8c
+  metadata.gz: 5c90a3f9e48b3ede88480bcaec9eabccfe31a7f60cf867df226192f54e0dbe29
+  data.tar.gz: 683529a7de7ea08c12065878dbc59d364f43163264d2c8338f1b90011aec5821
 SHA512:
-  metadata.gz: 8946d38a18bdb012fc79661fc31fcf67f4e63b7cdd728e14dacf48c77a72317322f145544d89b97725f292d4e1860a2aa9ba4842824378745541aea0e9408eee
-  data.tar.gz: 8d10ae4db2cfdf8aee03bc17d4867afd56e03dfa19ddcf4ca31610acbd4957bad49a34a3c6fbbbc2988c77ae42c7f86e1c4ae8e1f2a0bd1342ffe5febd68bbbe
+  metadata.gz: d9608362d4824d5c3a9044b3dd59e2c82935e915c88b938af75c9f8904852fab632b2e76132f25fd7a4faf7652ed63cdd54f86327c73e9efa1326df5d0225c14
+  data.tar.gz: 8c93323b08ea3c950f809bd201026a4a2a5086f9d5fddfcb31dad8a2888a5857ad4810f3932b3995b1abf4adee44fa4bf6354f96191527b5799952f4508a40e2

data/CHANGELOG.md

@@ -3,7 +3,65 @@
 This project uses [Semantic Versioning 2.0.0](http://semver.org/).
 
 
-#### Unreleased 0.1.0
+#### Release 0.3.0 (2026-09-25)
+
+- ADDED: Added evidence-backed `.top` registration and exact authoritative
+  absence parsing. `.us` now recognizes registration evidence while keeping
+  the registry's non-authoritative `No Data Found` response unknown.
+- ADDED: Added conservative `.pk` parsing for exact PKNIC registered and
+  explicitly available responses. The standard `whois` 6.0.3 mapping selects
+  a legacy web adapter, and `Whois::Client.new(host: ...)` cannot override
+  that adapter. DomScan needs a fixed-host port-43 route through its managed
+  relay before `.pk` counts as effective coverage.
+- ADDED: Tightened `.uz` and `.sa` parsing against current registered and
+  exact authoritative-absence responses. `.uz` reserved names remain
+  `:reserved`, not registered or available. Added a conservative `.ge` parser
+  for exact registered and absence responses from IANA's `whois.nic.ge` host;
+  `whois` 6.0.3 defaults `.ge` to `whois.registration.ge`, so a fixed-host
+  route is still required before counting it as effective coverage.
+- SAFETY: Kept blocked, empty, and ambiguous `.ch` responses unknown, and
+  added unavailable-only handling for explicit unsupported responses from
+  `.digital`, `.email`, `.live`, `.media`, `.network`, and `.services`.
+- DOCS: Added a scorecard for 1,000 observed TLD strings from the ICANN DNS
+  Magnitude snapshot dated 2026-09-12. Its strict count of 175 paired rows is
+  pinned to parser baseline `f2c822c` and excludes the Top 100, whose summary
+  labels do not expose row-level evidence pairs. The scorecard does not claim
+  1,000 effective WHOIS integrations or a single effective-coverage rate.
+  Later `.uz`/`.sa` evidence and `.pk`/`.ge` parsers are follow-ups, not
+  additions to that pinned count.
+- SOURCES: The audit notes link to the [ICANN DNS Magnitude
+  snapshot](https://magnitude.research.icann.org/historic/20260912.full.html),
+  [IANA `.top`](https://www.iana.org/domains/root/db/top.html),
+  [IANA `.us`](https://www.iana.org/domains/root/db/us.html),
+  [IANA `.ch`](https://www.iana.org/domains/root/db/ch.html),
+  [IANA `.pk`](https://www.iana.org/domains/root/db/pk.html), and the
+  [official PKNIC site](https://www.pknic.net.pk/). The
+  [rank 104-105 report](docs/audits/top-101-125-tlds-2026-09-19.md) cites
+  [IANA `.uz`](https://www.iana.org/domains/root/db/uz.html),
+  [UZINFOCOM](https://www.cctld.uz/),
+  [IANA `.sa`](https://www.iana.org/domains/root/db/sa.html), and
+  [IANA `.ge`](https://www.iana.org/domains/root/db/ge.html).
+- EVIDENCE: [Top-1,000 scorecard](docs/audits/top-1000-effective-coverage-scorecard-2026-09-25.md),
+  [current `.top`/`.us` observations](docs/audits/top-us-current-2026-09-25.md),
+  [`.pk` routing and registry evidence](docs/audits/top1000-es-pk-shop-dev-app-2026-09-25.md),
+  and [rank 101-125 follow-up evidence](docs/audits/top-101-125-tlds-2026-09-19.md).
+
+
+#### Release 0.2.0 (2026-09-25)
+
+- ADDED: Extended the usage-ranked audit and parser work through 1,000 ICANN
+  DNS Magnitude TLD strings from the 2026-09-12 snapshot.
+- CLARIFIED: The 1,000 rows are observed TLD strings, not 1,000 guaranteed
+  working registries or domain-availability results. The audit keeps verified
+  parser evidence, explicit unavailable or unsupported responses,
+  unknown or unresolved endpoints, and classification-only rows distinct.
+- ADDED: Preserved the source manifest, verification evidence, planning
+  inventory, and five implementation-package reports for ranks 301-1000.
+- TESTED: The combined parser and audit suite passes 6,768 examples with 0
+  failures on Ruby 3.2.2 and 3.4.5.
+
+
+#### Release 0.1.0 (2026-09-19)
 
 - CHANGED: Continued the project as `whois-parser-next`, with the original MIT
   license, attribution, and Git history preserved.

data/README.md

@@ -38,8 +38,9 @@ of current WHOIS responses, explicit unknown states, registry-specific
 fixtures, and safe handling of malformed, rate-limited, retired, or changed
 server responses.
 
-The current development gem name is `whois-parser-next`. It has not yet been
-published to RubyGems.
+The current source version is `0.3.0`. Check the [RubyGems project
+page](https://rubygems.org/gems/whois-parser-next) for its publication status
+and the latest published version.
 
 ## Compatibility
 
@@ -53,12 +54,11 @@ Whois::Parser
 ```
 
 Existing applications can retain `require "whois-parser"` and their
-`Whois::Parser` integrations while changing the dependency source:
+`Whois::Parser` integrations. To install version 0.3.0 after it appears on
+RubyGems, pin it as follows:
 
 ```ruby
-gem "whois-parser-next",
-  github: "estevecastells/whois-parser-next",
-  require: "whois-parser"
+gem "whois-parser-next", "= 0.3.0", require: "whois-parser"
 ```
 
 ## Requirements
@@ -83,12 +83,31 @@ bundle exec rspec
 
 ## Coverage evidence
 
-The 2026-09-19 live audit covers the ICANN DNS Magnitude ordering through rank
-300. Start with the [Top 20 review](docs/audits/top-20-tlds-2026-09-19.md) and
-[Top 100 audit](docs/audits/top-100-tlds-2026-09-19.md); the same directory
-contains the contiguous 25-TLD reports for ranks 101 through 300. Each report
-separates verified parsing from unavailable, retired, restricted, web-only,
-and unresolved WHOIS paths.
+The [2026-09-25 scorecard](docs/audits/top-1000-effective-coverage-scorecard-2026-09-25.md)
+summarizes evidence for 1,000 observed TLD strings from the
+[ICANN DNS Magnitude snapshot dated 2026-09-12](https://magnitude.research.icann.org/historic/20260912.full.html).
+These strings are not 1,000 guaranteed working WHOIS registries or
+domain-availability results. Its strict paired count is pinned to baseline
+`f2c822c`, excludes the Top 100 whose summary labels do not expose row-level
+evidence pairs, and does not claim an effective-coverage percentage. The
+scorecard keeps verified parser evidence, explicit unavailable or unsupported
+responses, unknown or unresolved endpoints, and classification-only rows
+distinct. Later `.uz`/`.sa` paired evidence and `.pk`/`.ge` parsers are
+follow-ups, not changes to the pinned scorecard counts. The `.pk` parser does
+not make the default `whois` 6.0.3 Web adapter use port 43; `Whois::Client`
+host configuration does not override that adapter. The `.ge` parser targets
+IANA's `whois.nic.ge`, while the default `whois` route is
+`whois.registration.ge`. Both require explicit fixed-host routing before an
+application can count them as effective coverage.
+
+Start with the [source verification report](docs/audits/top-1000-source-verification-2026-09-19.md),
+the [Top 20 review](docs/audits/top-20-tlds-2026-09-19.md), and the [Top 100
+audit](docs/audits/top-100-tlds-2026-09-19.md). The same directory contains
+the contiguous reports for ranks 101 through 300, the [301-1000 work-package
+plan](docs/audits/whois-parser-top1000-work-packages-2026-09-19.md), and the
+five package audit reports. Each report preserves the evidence boundary and
+does not turn unavailable, unsupported, unresolved, or classification-only
+rows into availability claims.
 
 ## Join the work
 

data/RELEASING.md

@@ -21,8 +21,8 @@ RubyGems API key to the repository or its Actions secrets.
 Create and push the annotated release tag only after the checks above pass:
 
 ```shell
-git tag -a v0.1.0 -m "whois-parser-next 0.1.0"
-git push origin v0.1.0
+git tag -a v0.3.0 -m "whois-parser-next 0.3.0"
+git push origin v0.3.0
 ```
 
 The release workflow repeats the full suite and package smoke test before it

data/ROADMAP.md

@@ -6,24 +6,29 @@ feature calendar.
 
 ## Current release
 
-- Establish a safe, maintained successor with Ruby 3.2 through Ruby 4 support.
-- Audit the first 300 rows of a dated, reproducible TLD-usage snapshot.
-- Replace unsafe fallback classifications with explicit unknown or unavailable
-  outcomes when a response lacks authoritative evidence.
-- Keep the gem compatible with `require "whois-parser"` and `Whois::Parser`.
-- Publish through short-lived RubyGems trusted-publishing credentials.
+- Published `whois-parser-next` 0.2.0 to RubyGems, retaining the established
+  `require "whois-parser"` and `Whois::Parser` compatibility.
+- Completed the audit of 1,000 observed TLD strings from the ICANN DNS Magnitude
+  snapshot dated 2026-09-12. These rows are ranked TLD strings, not 1,000
+  confirmed working WHOIS registries or domain-availability results.
+- Kept verified parser evidence, unavailable or unsupported responses,
+  unresolved endpoints, and classification-only rows distinct.
 
 ## Next priorities
 
-1. Continue evidence-backed TLD coverage in reproducible, non-overlapping
-   batches.
-2. Add regression fixtures when registries change format, server, policy, or
-   availability wording.
-3. Reduce avoidable runtime dependencies without changing public behavior.
-4. Improve contributor tooling for fixture privacy, server-mapping drift, and
-   ambiguous-response safety.
-5. Document parser confidence and provenance without pretending that a parsed
-   WHOIS response is an availability guarantee.
+1. Refresh the existing 1,000-row cohort only against a new, dated source
+   snapshot, keeping each snapshot's ranks and labels separate.
+2. Close parser evidence gaps for mapped official WHOIS hosts with bounded
+   probes and sanitized fixtures for exact response behavior. DNS failures,
+   timeouts, unsupported replies, and ambiguous responses remain unresolved or
+   unavailable.
+3. Add or change parser behavior only when current response evidence is backed
+   by a regression spec; do not infer one host's behavior for unmapped or
+   unverified TLDs.
+4. Scope any expansion beyond rank 1,000 from a separately documented dataset
+   before treating it as parser-coverage work.
+5. Reduce avoidable runtime dependencies and improve contributor tooling for
+   fixture privacy, server-mapping drift, and response-safety checks.
 
 ## Performance and Rust
 

data/SECURITY.md

@@ -2,12 +2,12 @@
 
 ## Supported versions
 
-Until the first release, only the current main branch is supported.
+The current main branch and the latest published release are supported.
 
 | Version | Supported |
 |---|---|
 | main | Yes |
-| Released versions | No released versions yet |
+| 0.2.0 (latest release) | Yes |
 
 ## Reporting a vulnerability
 

data/lib/whois/parser/version.rb

@@ -1,6 +1,6 @@
 module Whois
   class Parser
     # The current library version.
-    VERSION = "0.1.0".freeze
+    VERSION = "0.3.0".freeze
   end
 end

data/lib/whois/parsers/base_centralnic_icann.rb

@@ -0,0 +1,12 @@
+require_relative 'base_icann_compliant'
+
+module Whois
+  class Parsers
+    # CentralNic registries that expose the ICANN key/value record shape.
+    class BaseCentralnicIcann < BaseIcannCompliant
+      self.scanner = Scanners::BaseIcannCompliant, {
+        pattern_available: /^The queried object does not exist:\s+DOMAIN NOT FOUND\s*$/i,
+      }
+    end
+  end
+end

data/lib/whois/parsers/base_cira_registry.rb

@@ -0,0 +1,19 @@
+require_relative 'whois.cira.ca'
+require_relative 'registry_response_safety'
+
+module Whois
+  class Parsers
+    # CIRA-shaped registries use the same record scanner, but an unexpected
+    # non-record response must remain unknown rather than becoming a parser
+    # error that callers might accidentally treat as registration.
+    class BaseCiraRegistry < WhoisCiraCa
+      include RegistryResponseSafety
+
+      property_supported :status do
+        super()
+      rescue ParserError
+        :unknown
+      end
+    end
+  end
+end

data/lib/whois/parsers/base_icann_compliant.rb

@@ -54,7 +54,7 @@ module Whois
       end
 
       property_supported :registered? do
-        [:registered, :reserved].include?(status)
+        status == :registered
       end
 
 

data/lib/whois/parsers/base_identity_digital.rb

@@ -0,0 +1,12 @@
+require_relative 'base_icann_compliant'
+
+module Whois
+  class Parsers
+    # Shared ICANN key/value parser for Identity Digital registries.
+    #
+    # Individual registries differ only in the authoritative absence marker;
+    # host-specific subclasses keep those markers explicit.
+    class BaseIdentityDigital < BaseIcannCompliant
+    end
+  end
+end

data/lib/whois/parsers/base_jwhois.rb

@@ -0,0 +1,48 @@
+require_relative 'base'
+require_relative 'registry_response_safety'
+
+module Whois
+  class Parsers
+    # Minimal parser contract for JWhoisServer responses used by the French
+    # overseas registries.  Registration is reported only from a domain line;
+    # the registry's explicit NO OBJECT FOUND marker is the sole absence proof.
+    class BaseJwhois < Base
+      include RegistryResponseSafety
+
+      property_supported :domain do
+        content_for_scanner[/^domain:\s*(\S+)\s*$/i, 1]&.downcase
+      end
+
+      property_supported :status do
+        if available?
+          :available
+        elsif registered_evidence?
+          :registered
+        else
+          :unknown
+        end
+      end
+
+      property_supported :available? do
+        content_for_scanner.match?(/^NO OBJECT FOUND!\s*$/i)
+      end
+
+      property_supported :registered? do
+        status == :registered
+      end
+
+      property_supported :nameservers do
+        content_for_scanner.scan(/^nameserver:\s*(\S+)\s*$/i).flatten.map do |name|
+          Parser::Nameserver.new(name: name.downcase)
+        end
+      end
+
+      private
+
+      def registered_evidence?
+        content_for_scanner.match?(/^domain:\s*\S+/i) &&
+          content_for_scanner.match?(/^(?:changed|nameserver):\s*\S+/i)
+      end
+    end
+  end
+end

data/lib/whois/parsers/base_nic_reserved_available.rb

@@ -0,0 +1,17 @@
+require_relative 'base_icann_compliant'
+
+module Whois
+  class Parsers
+    # Shared parser for registries that use the ICANN response layout and
+    # report an exact `Available` line for an absent object.
+    class BaseNicReservedAvailable < BaseIcannCompliant
+      self.scanner = Scanners::BaseIcannCompliant, {
+        pattern_available: /^Available\s*$/i,
+      }
+
+      def reserved?
+        content_for_scanner.match?(/^Reserved:\s*/i)
+      end
+    end
+  end
+end

data/lib/whois/parsers/base_reserved_registry.rb

@@ -0,0 +1,27 @@
+require_relative 'base'
+
+module Whois
+  class Parsers
+    # A registry response that explicitly reserves or prohibits the queried
+    # name. It is not evidence of either registration or availability.
+    class BaseReservedRegistry < Base
+      property_supported :status do
+        :unknown
+      end
+
+      property_supported :available? do
+        false
+      end
+
+      property_supported :registered? do
+        false
+      end
+
+      def response_unavailable?
+        content_for_scanner.match?(
+          /\A(?:Reserved Domain Name|This name is reserved by the Registry\b|The queried object does not exist:.*reserved list)/i
+        )
+      end
+    end
+  end
+end

data/lib/whois/parsers/base_top1000_icann.rb

@@ -0,0 +1,39 @@
+require_relative 'base_icann_compliant'
+
+module Whois
+  class Parsers
+    # Conservative ICANN-style parser shared by the package-3 registries.
+    # The absence markers are complete lines observed in current port-43
+    # responses. Empty, denied, and throttled responses remain unavailable via
+    # RegistryResponseSafety inherited from BaseIcannCompliant.
+    class BaseTop1000Icann < BaseIcannCompliant
+      AVAILABLE_MARKERS = [
+        /^No Data Found\s*$/i,
+        /^The queried object does not exist:\s*(?:DOMAIN NOT FOUND|no matching objects found)\s*$/i,
+        /^>>> Domain \S+ is available for registration\s*$/i,
+        /^Domain not found\s*$/i,
+        /^%% NOT FOUND\s*$/i,
+      ].freeze
+
+      RESERVED_MARKERS = [
+        /^Reserved Domain Name\s*$/i,
+        /^This name is reserved by the Registry\.\s*$/i,
+      ].freeze
+
+      property_supported :available? do
+        AVAILABLE_MARKERS.any? { |marker| content_for_scanner.match?(marker) }
+      end
+
+      property_supported :reserved? do
+        RESERVED_MARKERS.any? { |marker| content_for_scanner.match?(marker) }
+      end
+
+      # `reserved?` is intentionally a parser predicate rather than a public
+      # record property. BaseIcannCompliant consults it while determining the
+      # status of a reserved registry response.
+      def reserved?
+        RESERVED_MARKERS.any? { |marker| content_for_scanner.match?(marker) }
+      end
+    end
+  end
+end

data/lib/whois/parsers/base_top1000_retired.rb

@@ -0,0 +1,26 @@
+require_relative 'base'
+
+module Whois
+  class Parsers
+    # WHOIS endpoints that have retired port-43 service in favour of RDAP.
+    class BaseTop1000Retired < Base
+      property_supported :status do
+        :unknown
+      end
+
+      property_supported :available? do
+        false
+      end
+
+      property_supported :registered? do
+        false
+      end
+
+      def response_unavailable?
+        content_for_scanner.match?(
+          /^Notice: Effective May 1, 2026, the WHOIS service has been retired\b/i
+        )
+      end
+    end
+  end
+end

data/lib/whois/parsers/base_unsupported_registry.rb

@@ -21,7 +21,7 @@ module Whois
 
       def response_unavailable?
         content_for_scanner.match?(
-          /^(?:TLD is not supported\.|No Data Found)\s*$/i
+          /\A(?:TLD is not supported\.|No Data Found)[ \t]*(?:\n|\z)/i
         )
       end
     end

data/lib/whois/parsers/cwhois.cnnic.cn.rb

@@ -0,0 +1,10 @@
+require_relative 'whois.cnnic.cn'
+
+module Whois
+  class Parsers
+    # The registry publishes both `cwhois.cnnic.cn` and the historical
+    # `whois.cnnic.cn` service name.  Keep both names on the same parser.
+    class CwhoisCnnicCn < WhoisCnnicCn
+    end
+  end
+end

data/lib/whois/parsers/registry_response_safety.rb

@@ -22,6 +22,8 @@ module Whois
       def response_unavailable?
         response_incomplete? || content_for_scanner.match?(
           /^(?:\s*(?:%|#)\s*)?(?:requests of this client are not permitted|access to the whois service is denied|whois service is unavailable)\b/i
+        ) || content_for_scanner.match?(
+          /\ATLD is not supported\.[ \t]*(?:\n|\z)/i
         )
       end
     end

data/lib/whois/parsers/whois.ax.rb

@@ -8,6 +8,7 @@
 
 
 require_relative 'base'
+require_relative 'registry_response_safety'
 
 
 module Whois
@@ -23,21 +24,25 @@ module Whois
     #   The Example parser for the list of all available methods.
     #
     class WhoisAx < Base
+      include RegistryResponseSafety
 
       property_supported :status do
         if available?
           :available
-        else
+        elsif registered_evidence?
           :registered
+        else
+          :unknown
         end
       end
 
       property_supported :available? do
-        !!(content_for_scanner =~ /^No records matching .+ found/)
+        !!(content_for_scanner =~ /^No records matching \S+ found\.\s*$/i ||
+           content_for_scanner =~ /^Domain not found\s*$/i)
       end
 
       property_supported :registered? do
-        !available?
+        status == :registered
       end
 
 
@@ -58,6 +63,13 @@ module Whois
         end
       end
 
+      private
+
+      def registered_evidence?
+        content_for_scanner.match?(/^Domain Name:\s+\S+/i) &&
+          content_for_scanner.match?(/^(?:Created|Name Server \d):\s+\S+/i)
+      end
+
     end
 
   end

data/lib/whois/parsers/whois.cctld.uz.rb

@@ -27,26 +27,15 @@ module Whois
       include RegistryResponseSafety
 
       property_supported :status do
-        if available?
-          :available
-        elsif content_for_scanner =~ /^Status: (.+?)\n/
-          case ::Regexp.last_match(1).downcase
-          when "active" then :registered
-          when "reserved" then :reserved
-          else
-            Whois::Parser.bug!(ParserError, "Unknown status `#{::Regexp.last_match(1)}'.")
-          end
-        else
-          :unknown
-        end
+        classify_status
       end
 
       property_supported :available? do
-        !!(content_for_scanner =~ /not found in database/)
+        classify_status == :available
       end
 
       property_supported :registered? do
-        [:registered, :reserved].include?(status)
+        classify_status == :registered
       end
 
 
@@ -77,6 +66,53 @@ module Whois
         end
       end
 
+      private
+
+      def classify_status
+        return :unknown if absence_conflicts_with_record?
+
+        evidence = []
+        evidence << :registered if registered_evidence?
+        evidence << :reserved if reserved_evidence?
+        evidence << :available if available_evidence?
+        evidence.one? ? evidence.first : :unknown
+      end
+
+      def absence_conflicts_with_record?
+        return false unless absence_domain_names.any?
+
+        domain_names.any? || status_values.any? || content_for_scanner.match?(/^[ \t]*Domain Name:/i)
+      end
+
+      def registered_evidence?
+        domain_names.one? && status_values == ["active"]
+      end
+
+      def reserved_evidence?
+        domain_names.one? && status_values == ["reserved"]
+      end
+
+      def available_evidence?
+        absence_domain_names.one? && status_values.empty? &&
+          !content_for_scanner.match?(/^[ \t]*Domain Name:/i)
+      end
+
+      def domain_names
+        content_for_scanner.scan(
+          /^[ \t]*Domain Name:[ \t]*((?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+uz)[ \t]*$/i
+        ).flatten.map(&:downcase).uniq
+      end
+
+      def status_values
+        content_for_scanner.scan(/^[ \t]*Status:[ \t]*([a-z]+)[ \t]*$/i).flatten.map(&:downcase).uniq
+      end
+
+      def absence_domain_names
+        content_for_scanner.scan(
+          /^[ \t]*Sorry, but domain:[ \t]*"((?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+uz)", not found in database[ \t]*$/i
+        ).flatten.map(&:downcase).uniq
+      end
+
     end
 
   end