nondisposable 0.2.1 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6d6da844eae687a1d72201694a880a5ec5f16dcbd5e498de792a19d421027e6b
-  data.tar.gz: ea77c8f6ea5ddf4726f95b3438fadb7c7f0dedaed83bf7b674fdba78f95cbbed
+  metadata.gz: 9d2c949362fe7f9506247f3ae809ced6dca5a842086fa540346c8e89ea5dc5fd
+  data.tar.gz: a13fa7bec1531089fe68895529ecab5d9f05a2653292e2e39a1f4dfdeb818cf7
 SHA512:
-  metadata.gz: af761f9347d7e630239973256600cb52996f166f8479908c2ee8d379cae8b46d2035455c4bc2dc3c305941d7fe393dcac634e8d6b287e95eeee4b8bf9271db64
-  data.tar.gz: f58d7598f288515f684aaffc55a9399e01398fc9644b5744fd8e136510e4e51d358db12b4b33f99a58c7d06b5a05bd06a5313ff8e5f9f35e2d8ce95166791d38
+  metadata.gz: 696ce476987cdd1a90008b8065a9f8af9d6c19035e42cbd3cb0b03cf9b36ec0bf3fd9158d474552a8d2729e4d454ec19ac7ee31e24af844f17a42e27346532bc
+  data.tar.gz: 1a220333975bbd3ba6229a29172e42b4e9c91ddc6cc4ee97aca417ab743c3756c5ff024f70d57963f0c7643378d4d1a4153135632d7f65cd4a96068bb3bb3dc0

data/CHANGELOG.md

@@ -1,3 +1,45 @@
+## [0.4.0] - 2026-08-25
+
+Catching the other kind of bad address. A disposable address is someone hiding from you; a typo is someone who wanted to reach you and can't. `user@gmail.con` isn't a throwaway — it's a real person whose account nobody will ever be able to reach, because `.con` has never existed. Both new checks are **opt-in**: upgrading this gem will not start rejecting addresses that were fine yesterday.
+
+### Added
+
+- **TLD validation** (`config.check_tld`, default `false`): rejects addresses whose TLD isn't in the IANA root zone. The gem now bundles a snapshot of [data.iana.org/TLD/tlds-alpha-by-domain.txt](https://data.iana.org/TLD/tlds-alpha-by-domain.txt) (~1,438 entries, ~9 KB) loaded once into a frozen `Set` — no migration, no table, no query per signup, and it can't be empty on a fresh install. A domain with no dot at all (`example@gmailmcom`) has no TLD and is rejected too; IP literals and unicode TLDs are deliberately left alone.
+- **`config.additional_tlds`**: accept a TLD delegated after your installed version shipped, without waiting for a release. This is the escape hatch that keeps a stale snapshot from ever locking anyone out.
+- **`Nondisposable::Tld::SPECIAL_USE`**: the RFC 6761 / RFC 2606 reserved names (`test`, `example`, `invalid`, `localhost`, `local`, `onion`). ⚠️ **Heads up when you switch `check_tld` on**: these are reserved so they can never be delegated, so they aren't in the root zone and are rejected — which is correct for a signup form, and will also turn your fixtures at `user@example.test` red. One line, scoped to where it belongs: `config.additional_tlds = Nondisposable::Tld::SPECIAL_USE if Rails.env.local?`
+- **`config.blocked_tlds`** and **`config.allowed_tlds`**: refuse real TLDs you don't want (the free Freenom set `tk ml ga cf gq` is the usual reason), or invert it into an allowlist.
+- **Lookalike detection**: `Nondisposable.suggestion_for("someone@gmial.com") # => "someone@gmail.com"`, matching against a bundled list of ~240 well-known providers using optimal string alignment distance, so an adjacent swap counts as the one mistake a human actually made. This catches what a TLD check structurally cannot — `.co`, `.cm` and `.om` are Colombia, Cameroon and Oman, all real. Always available, never blocks anything.
+- **`config.reject_lookalike_domains`** (default `false`): turns that suggestion into a validation error naming the correction. Off by default on purpose — a suggestion is a guess about intent, and a wrong guess stops a real person signing up with their real address. `config.lookalike_distance` (default `1`) and `config.additional_email_providers` tune it.
+- **`Nondisposable.valid_tld?(email)`** and **`Nondisposable.suggestion_for(email)`** as direct checks, alongside the existing `disposable?`.
+- **`rake nondisposable:tlds:update`**: refreshes the bundled snapshot from IANA. A maintenance task for a checkout of this gem, not something host apps run — it refuses to overwrite a good list with an implausibly short one, writes atomically, and preserves IANA's version header (readable via `Nondisposable::Tld.version`).
+- New error messages: `invalid_tld_error_message`, `blocked_tld_error_message`, `lookalike_error_message` (which interpolates `%{suggestion}`).
+
+### Changed
+
+- The validator now adds **at most one error** however many checks fire, and prefers the most useful message available: whenever the gem can name a correction, that phrasing wins over the generic "doesn't look like a real email address". The disposable check still takes precedence over both — a correct spelling wouldn't help a throwaway provider.
+
+## [0.3.0] - 2026-08-09
+
+### Added
+
+- **Bundled seed list**: the gem now ships a snapshot of the upstream blocklist (8,201 domains) and `DomainListUpdater.seed` populates the table from it when (and only when) the table is empty. The install generator's migration seeds automatically, and `DomainListUpdater.update` falls back to seeding when the remote fetch fails against an empty table. This closes the fresh-install fail-open window where every signup passed until the first successful remote update.
+- **Parent-domain matching** (`config.check_parent_domains`, default `true`): an email at `x.tempmail.com` is now blocked when `tempmail.com` is on the list. The check walks up to 3 parent labels and never matches bare TLDs. Set to `false` to restore exact-only matching.
+- **Configurable failure mode** (`config.on_check_failure`, default `:allow`): controls what happens when the disposable check itself raises (e.g. database hiccup). `:allow` lets the record through with a loud error log (availability-first); `:reject` preserves the previous behavior and error message. Invalid values raise `ArgumentError`.
+
+### Changed
+
+- **Behavior change**: a broken check no longer rejects signups by default (previously any `StandardError` during validation added an error to the record). Set `config.on_check_failure = :reject` to keep the old fail-closed behavior.
+- **Behavior change**: subdomains of listed domains are now blocked by default (see parent-domain matching above). Set `config.check_parent_domains = false` for the old exact-only behavior.
+- `DomainListUpdater` now uses 10s open/read timeouts on the fetch (previously Net::HTTP's 60s defaults) so a hung connection can't block the update job.
+- Domain list updates use a single bulk `insert_all` instead of per-row `create`, and report any rows skipped by the unique index instead of silently swallowing them.
+- Downloaded domains are normalized before insert: whitespace/CR stripped, blank lines dropped, case-insensitive dedupe. `additional_domains` and `excluded_domains` now match case-insensitively everywhere.
+- An exact entry in `excluded_domains` always wins, so a specific subdomain can be allowlisted under a listed parent domain.
+- `Nondisposable.configuration` is lazily initialized: apps without an initializer no longer fail every validation with "cannot check if it's disposable".
+
+### Removed
+
+- Dead `require 'open-uri'` in the domain list updater.
+
 ## [0.2.1] - 2026-01-17
 
 - Add `[nondisposable]` prefix to all logger calls for better log identification

data/README.md

@@ -17,6 +17,17 @@ That's it! You're done.
 
 The gem also provides a job you can run daily to keep your disposable domain list up to date.
 
+It can also catch the other kind of bad address — the typo. `user@gmail.con` isn't a throwaway, it's a real person whose account nobody will ever be able to reach, because `.con` doesn't exist. Two opt-in checks:
+
+```ruby
+Nondisposable.configure do |config|
+  config.check_tld = true                # reject TLDs that aren't in the IANA root zone
+  config.reject_lookalike_domains = true # and addresses one keystroke from a real provider
+end
+```
+
+See [Catching typos](#catching-typos).
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -43,7 +54,7 @@ This will create the necessary migration file, initializer, and a job for schedu
 rails db:migrate
 ```
 
-Finally, populate the initial list of disposable domains:
+The migration also seeds the table from a snapshot of the disposable-domain list bundled with the gem, so your app is protected immediately. To fetch the very latest list, run:
 
 ```ruby
 Nondisposable::DomainListUpdater.update
@@ -94,9 +105,120 @@ Nondisposable.configure do |config|
   config.error_message = "provider is not allowed. Please use a non-disposable email address."
   config.additional_domains = ['custom-disposable-domain.com']
   config.excluded_domains = ['false-positive-domain.com']
+
+  # What to do when the disposable check itself fails (e.g. database hiccup):
+  #   :allow  - let the signup through and log an error (availability-first, default)
+  #   :reject - block the signup with a validation error (fail closed)
+  config.on_check_failure = :allow
+
+  # Also match parent domains: an email at x.tempmail.com is blocked when
+  # tempmail.com is on the list. Set to false for exact matches only.
+  config.check_parent_domains = true
+
+  # --- Catching typos (both OFF by default) ---
+
+  # Reject addresses whose TLD isn't in the IANA root zone: gmail.con, outlook.ed
+  config.check_tld = true
+  config.additional_tlds = []                # accept a TLD newer than this gem
+  config.blocked_tlds = %w[tk ml ga cf gq]   # refuse real TLDs you don't want
+  config.allowed_tlds = nil                  # or allowlist: %w[es com] rejects everything else
+
+  # Reject addresses one keystroke from a well-known provider: gmail.co, gmial.com
+  config.reject_lookalike_domains = false
+  config.lookalike_distance = 1              # edits that still count as a typo
+  config.additional_email_providers = []     # your own domains / regional providers
+
+  config.invalid_tld_error_message = "doesn't look like a real email address"
+  config.blocked_tld_error_message = "domain ending is not allowed"
+  config.lookalike_error_message = "looks like a typo. Did you mean %{suggestion}?"
 end
 ```
 
+#### Parent domain matching
+
+With `check_parent_domains` enabled (the default since 0.3.0), `user@x.tempmail.com` is blocked when `tempmail.com` is on the list. The check walks up to 3 parent labels (`a.b.c.tempmail.com` → `b.c.tempmail.com` → `c.tempmail.com` → `tempmail.com`) and never matches against bare TLDs like `com`. All candidates are checked in a single indexed query.
+
+This is a deliberately minimal, dependency-free approximation of "registrable domain" matching: the gem doesn't ship a full [public suffix list](https://publicsuffix.org), so it can't tell that `co.uk` is a public suffix — which is harmless in practice, because public suffixes don't appear on the blocklist. If you need to allowlist a specific subdomain under a listed parent, add the exact subdomain to `excluded_domains`; an exact exclusion always wins. Excluding a parent domain also unblocks its subdomains.
+
+#### Failure mode
+
+By default (`on_check_failure = :allow`), if the disposable check raises — say, the database is briefly unavailable — the signup goes through and an error is logged. This is availability-first: a broken check should not lock everyone out of signup. If you'd rather fail closed (reject signups whenever the check cannot run, as versions before 0.3.0 did), set `config.on_check_failure = :reject`.
+
+### Catching typos
+
+A disposable address is someone hiding from you. A typo is someone who wanted to reach you and won't be able to. Both leave you with a useless row in the users table, so `nondisposable` can catch both — but the typo checks are **opt-in**, because upgrading a gem should never start rejecting addresses that were fine yesterday.
+
+#### `config.check_tld` — is that a real domain ending?
+
+Every TLD that exists is in the [IANA root zone database](https://data.iana.org/TLD/tlds-alpha-by-domain.txt), and the gem ships a snapshot of it (~1,438 entries, about 9 KB, loaded once into a frozen `Set`). `.con` has never been in it, and never will be.
+
+```ruby
+config.check_tld = true
+
+Nondisposable.valid_tld?("user@gmail.com")   # => true
+Nondisposable.valid_tld?("user@gmail.con")   # => false
+Nondisposable.valid_tld?("example@gmailmcom") # => false  (no TLD at all)
+```
+
+Details worth knowing:
+
+- **No dot, no TLD.** `example@gmailmcom` models a typo observed in production: the dot was missed entirely. A domain with no TLD can't end in a real one, so it's rejected. If your app accepts single-label intranet addresses like `you@localhost`, leave this off for that model.
+- **IP literals are left alone.** `user@192.168.0.1` and `user@[10.0.0.1]` are a format question, and a TLD check has no business answering it. Pair with `format:` if you care.
+- **Unicode TLDs are never rejected.** IANA lists internationalised TLDs in punycode (`XN--FIQS8S`), and converting `例え.テスト` to that form needs an IDN library this gem deliberately doesn't depend on. Rather than reject every unicode address, it declines to judge them. Punycode-form addresses — what mail clients actually send — are checked normally.
+- **A stale snapshot can't trap you.** If ICANN delegates a TLD after your gem version shipped, `config.additional_tlds = %w[newtld]` accepts it immediately, with no release to wait for.
+
+> [!IMPORTANT]
+> **`.test` and `.example` are rejected, and that will turn your test suite red.**
+>
+> RFC 6761 and RFC 2606 reserve `test`, `example`, `invalid` and `localhost` precisely so they can never be delegated — which is why they aren't in the root zone, and why your fixtures live at `user@example.test` in the first place. Rejecting them is right for a signup form (no human types `me@home.test`, and mail could never be delivered there), but it's a configuration question, not a bug:
+>
+> ```ruby
+> # config/initializers/nondisposable.rb
+> config.additional_tlds = Nondisposable::Tld::SPECIAL_USE if Rails.env.local?
+> ```
+>
+> Scope it to your non-production environments, so production still refuses an address nobody could ever answer.
+
+`blocked_tlds` refuses TLDs that are perfectly real but that you'd rather not see — the historically free Freenom set (`tk`, `ml`, `ga`, `cf`, `gq`) is the usual suspect. `allowed_tlds` inverts it into an allowlist; be careful, `%w[es]` turns away every `.com` customer you have.
+
+#### `config.reject_lookalike_domains` — did they mean gmail.com?
+
+A TLD check structurally **cannot** catch the most common typos, because `.co` (Colombia), `.cm` (Cameroon) and `.om` (Oman) are all real, delegated TLDs. `user@gmail.co` is a well-formed address at a real TLD and still almost always a finger that slipped off the `m`.
+
+So the second check asks a different question: is this domain one edit away from a well-known provider, while not being one itself?
+
+```ruby
+Nondisposable.suggestion_for("someone@gmial.com") # => "someone@gmail.com"
+Nondisposable.suggestion_for("someone@gmail.co")  # => "someone@gmail.com"
+Nondisposable.suggestion_for("someone@gmail.com") # => nil
+Nondisposable.suggestion_for("someone@mail.com")  # => nil  (a real provider)
+```
+
+`suggestion_for` is always available and **never blocks anything** — show it as a hint and let the human decide. That's the gentler way to use this, and the one to reach for first:
+
+```erb
+<% if (did_you_mean = Nondisposable.suggestion_for(@user.email)) %>
+  <p>Did you mean <%= did_you_mean %>?</p>
+<% end %>
+```
+
+Setting `reject_lookalike_domains = true` turns the same guess into a validation error naming the correction. Do that deliberately: a suggestion is a guess about intent, and a wrong guess stops a real person signing up with their real address. Two things keep that rare — matching stops at **one** edit by default (`lookalike_distance`), and an exact match against the bundled provider list always wins, which is why that list includes awkward pairs like `mail.com` (one insertion from `gmail.com`). Add anything we've missed with `config.additional_email_providers`; doing so both makes it a suggestion candidate and stops its users being told they mistyped.
+
+The matching uses optimal string alignment distance rather than plain Levenshtein, so an adjacent swap counts as the one mistake a human actually made: `gmial.com` is distance 1, not 2.
+
+#### How the two combine
+
+At most one error is added, however many checks fire — and whenever the gem can name a correction, that phrasing wins:
+
+| Address | `check_tld` only | with `reject_lookalike_domains` |
+|---|---|---|
+| `user@gmail.con` | "looks like a typo. Did you mean user@gmail.com?" | same |
+| `user@zzz.con` | "doesn't look like a real email address" | same |
+| `user@gmail.co` | accepted | "looks like a typo. Did you mean user@gmail.com?" |
+| `user@tempmail.com` | "provider is not allowed" | same |
+
+"Doesn't look like a real email address" is true but useless when we know what they meant, so a nameable correction always outranks the generic message.
+
 ### Direct Check
 
 You can also check if an email is disposable directly:
@@ -104,6 +226,8 @@ You can also check if an email is disposable directly:
 ```ruby
 Nondisposable.disposable?('user@example.com') # => false
 Nondisposable.disposable?('user@disposable-email.com') # => true
+Nondisposable.valid_tld?('user@example.con')  # => false
+Nondisposable.suggestion_for('user@gmial.com') # => "user@gmail.com"
 ```
 
 ## Updating disposable domains
@@ -114,6 +238,8 @@ To manually update the list of disposable domains, run:
 Nondisposable::DomainListUpdater.update
 ```
 
+The fetch uses 10-second open/read timeouts and replaces the whole table atomically. If the remote fetch fails and your table is empty (e.g. a fresh install without network access), the updater automatically seeds from the blocklist snapshot bundled with the gem so you're never left unprotected; you can also trigger that explicitly with `Nondisposable::DomainListUpdater.seed` (it only seeds an empty table, and never overwrites existing data).
+
 It's important you keep your disposable domain list up to date. `nondisposable` will read from the latest version of the [`disposable-email-domains`](https://github.com/disposable-email-domains/disposable-email-domains) list, which is typically updated every few days.
 
 For this, `nondisposable` provides you with an Active Job (`DisposableEmailDomainListUpdateJob`) that you can use to schedule daily updates. How you do that, exactly, depends on the queueing system you're using.
@@ -127,6 +253,24 @@ production:
     schedule: every day at 3am US/Pacific
 ```
 
+## Updating the TLD list
+
+The TLD snapshot is the deliberate opposite of the disposable list: it ships **inside the gem**, not in your database, and there is nothing for your app to schedule. Disposable domains number in the thousands and change every few days; TLDs are ~1,438 strings that change a handful of times a year, so a frozen `Set` costs one file read at boot, answers in O(1) with no query per signup, needs no migration, and can't be empty on a fresh install.
+
+If ICANN delegates a TLD your installed version doesn't know about, don't wait for a release — that's what `config.additional_tlds` is for.
+
+Maintainers refresh the snapshot from a checkout of this gem:
+
+```bash
+rake nondisposable:tlds:update   # rewrites data/iana_tlds.txt from data.iana.org
+```
+
+It refuses to overwrite a good list with an implausibly short one (a captive portal or a truncated body served with a `200`), writes to a temp file and moves it into place so an interrupted run can't leave half a root zone behind, and keeps IANA's own version header so you can always see which root zone you're shipping:
+
+```ruby
+Nondisposable::Tld.version # => "2026082301, Last Updated Mon Aug 24 07:07:01 2026 UTC"
+```
+
 ## Troubleshooting
 
 ### SSL certificate verify failed (unable to get certificate CRL)

data/Rakefile

@@ -3,6 +3,11 @@
 require "bundler/gem_tasks"
 require "rake/testtask"
 
+# nondisposable:tlds:update — refreshes data/iana_tlds.txt from the IANA root
+# zone. A maintenance task for this checkout, not something host apps run; see
+# the header of lib/nondisposable/tld_list_updater.rb.
+Dir.glob("lib/tasks/*.rake").each { |task| load task }
+
 Rake::TestTask.new(:test) do |t|
   t.libs << "lib"
   t.libs << "test"

data/app/models/nondisposable/disposable_domain.rb

@@ -4,12 +4,52 @@ module Nondisposable
   class DisposableDomain < ApplicationRecord
     validates :name, presence: true, uniqueness: { case_sensitive: false }
 
+    # How many leading labels the parent-domain walk strips when
+    # check_parent_domains is enabled: user@a.b.c.tempmail.com is checked as
+    # a.b.c.tempmail.com, b.c.tempmail.com, c.tempmail.com and tempmail.com.
+    #
+    # Trade-off (documented instead of pulling in a full public-suffix-list
+    # dependency): the walk never queries single-label suffixes (bare TLDs like
+    # "com"), but without a PSL it cannot tell that multi-label suffixes like
+    # "co.uk" are public — those candidates are harmless unless a public suffix
+    # is deliberately added to the list. Conversely, a throwaway address nested
+    # more than 3 labels below a listed domain escapes the walk; blocklisted
+    # domains are registrable domains, so 3 levels covers practical abuse.
+    PARENT_MATCH_DEPTH = 3
+
     class << self
       def disposable?(domain)
         return false if domain.blank?
+
         domain = domain.to_s.downcase
-        Nondisposable.configuration.additional_domains.include?(domain) ||
-          (where(name: domain).exists? && !Nondisposable.configuration.excluded_domains.include?(domain))
+        config = Nondisposable.configuration
+        candidates = match_candidates(domain)
+
+        additional = config.additional_domains.map { |d| d.to_s.downcase }
+        return true if (candidates & additional).any? # rubocop compat: Array#intersect? needs Ruby >= 3.1, gem supports 3.0
+
+        excluded = config.excluded_domains.map { |d| d.to_s.downcase }
+        # An exact exclusion always wins, so a specific subdomain can be
+        # allowlisted even when a parent domain is on the list.
+        return false if excluded.include?(domain)
+
+        where(name: candidates - excluded).exists?
+      end
+
+      # The exact domain plus up to PARENT_MATCH_DEPTH parent domains,
+      # never descending below two labels (bare TLDs are never candidates).
+      def match_candidates(domain)
+        return [domain] unless Nondisposable.configuration.check_parent_domains
+
+        labels = domain.split('.')
+        candidates = [domain]
+        1.upto(PARENT_MATCH_DEPTH) do |depth|
+          parent = labels.drop(depth)
+          break if parent.size < 2
+
+          candidates << parent.join('.')
+        end
+        candidates
       end
     end
 

data/context7.json

@@ -0,0 +1,4 @@
+{
+  "url": "https://context7.com/rameerez/nondisposable",
+  "public_key": "pk_HibNJE5rTFvy1txHHXUot"
+}