nondisposable 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 691fa38ec02a652cd53aa77bd8fac7513553bfd5b0dac8ba0e224be41ff58fdd
-  data.tar.gz: f145924d4cb664331fc215ae0e562f2e4110ff04bf501ea52fddf5f7089de0f9
+  metadata.gz: 9d2c949362fe7f9506247f3ae809ced6dca5a842086fa540346c8e89ea5dc5fd
+  data.tar.gz: a13fa7bec1531089fe68895529ecab5d9f05a2653292e2e39a1f4dfdeb818cf7
 SHA512:
-  metadata.gz: 97ce0386481ec1e70fa828ce3a18b0b2dce4a19855b34660c2abc782042d14512da0c77a7f09b6687a4f602a52d5c5d045ea8f9d9fa8f0e68a5123dbe067b36a
-  data.tar.gz: df2aaf590f850fe10fe9ee588861630721679f4bd2ae3bfb43a8cd6aab904bba2069cced44ce8fb60644e0f6ca21d679f3cf9593d1e90592a6f675029fcf8e0e
+  metadata.gz: 696ce476987cdd1a90008b8065a9f8af9d6c19035e42cbd3cb0b03cf9b36ec0bf3fd9158d474552a8d2729e4d454ec19ac7ee31e24af844f17a42e27346532bc
+  data.tar.gz: 1a220333975bbd3ba6229a29172e42b4e9c91ddc6cc4ee97aca417ab743c3756c5ff024f70d57963f0c7643378d4d1a4153135632d7f65cd4a96068bb3bb3dc0

data/CHANGELOG.md

@@ -1,3 +1,23 @@
+## [0.4.0] - 2026-08-25
+
+Catching the other kind of bad address. A disposable address is someone hiding from you; a typo is someone who wanted to reach you and can't. `user@gmail.con` isn't a throwaway — it's a real person whose account nobody will ever be able to reach, because `.con` has never existed. Both new checks are **opt-in**: upgrading this gem will not start rejecting addresses that were fine yesterday.
+
+### Added
+
+- **TLD validation** (`config.check_tld`, default `false`): rejects addresses whose TLD isn't in the IANA root zone. The gem now bundles a snapshot of [data.iana.org/TLD/tlds-alpha-by-domain.txt](https://data.iana.org/TLD/tlds-alpha-by-domain.txt) (~1,438 entries, ~9 KB) loaded once into a frozen `Set` — no migration, no table, no query per signup, and it can't be empty on a fresh install. A domain with no dot at all (`example@gmailmcom`) has no TLD and is rejected too; IP literals and unicode TLDs are deliberately left alone.
+- **`config.additional_tlds`**: accept a TLD delegated after your installed version shipped, without waiting for a release. This is the escape hatch that keeps a stale snapshot from ever locking anyone out.
+- **`Nondisposable::Tld::SPECIAL_USE`**: the RFC 6761 / RFC 2606 reserved names (`test`, `example`, `invalid`, `localhost`, `local`, `onion`). ⚠️ **Heads up when you switch `check_tld` on**: these are reserved so they can never be delegated, so they aren't in the root zone and are rejected — which is correct for a signup form, and will also turn your fixtures at `user@example.test` red. One line, scoped to where it belongs: `config.additional_tlds = Nondisposable::Tld::SPECIAL_USE if Rails.env.local?`
+- **`config.blocked_tlds`** and **`config.allowed_tlds`**: refuse real TLDs you don't want (the free Freenom set `tk ml ga cf gq` is the usual reason), or invert it into an allowlist.
+- **Lookalike detection**: `Nondisposable.suggestion_for("someone@gmial.com") # => "someone@gmail.com"`, matching against a bundled list of ~240 well-known providers using optimal string alignment distance, so an adjacent swap counts as the one mistake a human actually made. This catches what a TLD check structurally cannot — `.co`, `.cm` and `.om` are Colombia, Cameroon and Oman, all real. Always available, never blocks anything.
+- **`config.reject_lookalike_domains`** (default `false`): turns that suggestion into a validation error naming the correction. Off by default on purpose — a suggestion is a guess about intent, and a wrong guess stops a real person signing up with their real address. `config.lookalike_distance` (default `1`) and `config.additional_email_providers` tune it.
+- **`Nondisposable.valid_tld?(email)`** and **`Nondisposable.suggestion_for(email)`** as direct checks, alongside the existing `disposable?`.
+- **`rake nondisposable:tlds:update`**: refreshes the bundled snapshot from IANA. A maintenance task for a checkout of this gem, not something host apps run — it refuses to overwrite a good list with an implausibly short one, writes atomically, and preserves IANA's version header (readable via `Nondisposable::Tld.version`).
+- New error messages: `invalid_tld_error_message`, `blocked_tld_error_message`, `lookalike_error_message` (which interpolates `%{suggestion}`).
+
+### Changed
+
+- The validator now adds **at most one error** however many checks fire, and prefers the most useful message available: whenever the gem can name a correction, that phrasing wins over the generic "doesn't look like a real email address". The disposable check still takes precedence over both — a correct spelling wouldn't help a throwaway provider.
+
 ## [0.3.0] - 2026-08-09
 
 ### Added

data/README.md

@@ -17,6 +17,17 @@ That's it! You're done.
 
 The gem also provides a job you can run daily to keep your disposable domain list up to date.
 
+It can also catch the other kind of bad address — the typo. `user@gmail.con` isn't a throwaway, it's a real person whose account nobody will ever be able to reach, because `.con` doesn't exist. Two opt-in checks:
+
+```ruby
+Nondisposable.configure do |config|
+  config.check_tld = true                # reject TLDs that aren't in the IANA root zone
+  config.reject_lookalike_domains = true # and addresses one keystroke from a real provider
+end
+```
+
+See [Catching typos](#catching-typos).
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -103,6 +114,23 @@ Nondisposable.configure do |config|
   # Also match parent domains: an email at x.tempmail.com is blocked when
   # tempmail.com is on the list. Set to false for exact matches only.
   config.check_parent_domains = true
+
+  # --- Catching typos (both OFF by default) ---
+
+  # Reject addresses whose TLD isn't in the IANA root zone: gmail.con, outlook.ed
+  config.check_tld = true
+  config.additional_tlds = []                # accept a TLD newer than this gem
+  config.blocked_tlds = %w[tk ml ga cf gq]   # refuse real TLDs you don't want
+  config.allowed_tlds = nil                  # or allowlist: %w[es com] rejects everything else
+
+  # Reject addresses one keystroke from a well-known provider: gmail.co, gmial.com
+  config.reject_lookalike_domains = false
+  config.lookalike_distance = 1              # edits that still count as a typo
+  config.additional_email_providers = []     # your own domains / regional providers
+
+  config.invalid_tld_error_message = "doesn't look like a real email address"
+  config.blocked_tld_error_message = "domain ending is not allowed"
+  config.lookalike_error_message = "looks like a typo. Did you mean %{suggestion}?"
 end
 ```
 
@@ -116,6 +144,81 @@ This is a deliberately minimal, dependency-free approximation of "registrable do
 
 By default (`on_check_failure = :allow`), if the disposable check raises — say, the database is briefly unavailable — the signup goes through and an error is logged. This is availability-first: a broken check should not lock everyone out of signup. If you'd rather fail closed (reject signups whenever the check cannot run, as versions before 0.3.0 did), set `config.on_check_failure = :reject`.
 
+### Catching typos
+
+A disposable address is someone hiding from you. A typo is someone who wanted to reach you and won't be able to. Both leave you with a useless row in the users table, so `nondisposable` can catch both — but the typo checks are **opt-in**, because upgrading a gem should never start rejecting addresses that were fine yesterday.
+
+#### `config.check_tld` — is that a real domain ending?
+
+Every TLD that exists is in the [IANA root zone database](https://data.iana.org/TLD/tlds-alpha-by-domain.txt), and the gem ships a snapshot of it (~1,438 entries, about 9 KB, loaded once into a frozen `Set`). `.con` has never been in it, and never will be.
+
+```ruby
+config.check_tld = true
+
+Nondisposable.valid_tld?("user@gmail.com")   # => true
+Nondisposable.valid_tld?("user@gmail.con")   # => false
+Nondisposable.valid_tld?("example@gmailmcom") # => false  (no TLD at all)
+```
+
+Details worth knowing:
+
+- **No dot, no TLD.** `example@gmailmcom` models a typo observed in production: the dot was missed entirely. A domain with no TLD can't end in a real one, so it's rejected. If your app accepts single-label intranet addresses like `you@localhost`, leave this off for that model.
+- **IP literals are left alone.** `user@192.168.0.1` and `user@[10.0.0.1]` are a format question, and a TLD check has no business answering it. Pair with `format:` if you care.
+- **Unicode TLDs are never rejected.** IANA lists internationalised TLDs in punycode (`XN--FIQS8S`), and converting `例え.テスト` to that form needs an IDN library this gem deliberately doesn't depend on. Rather than reject every unicode address, it declines to judge them. Punycode-form addresses — what mail clients actually send — are checked normally.
+- **A stale snapshot can't trap you.** If ICANN delegates a TLD after your gem version shipped, `config.additional_tlds = %w[newtld]` accepts it immediately, with no release to wait for.
+
+> [!IMPORTANT]
+> **`.test` and `.example` are rejected, and that will turn your test suite red.**
+>
+> RFC 6761 and RFC 2606 reserve `test`, `example`, `invalid` and `localhost` precisely so they can never be delegated — which is why they aren't in the root zone, and why your fixtures live at `user@example.test` in the first place. Rejecting them is right for a signup form (no human types `me@home.test`, and mail could never be delivered there), but it's a configuration question, not a bug:
+>
+> ```ruby
+> # config/initializers/nondisposable.rb
+> config.additional_tlds = Nondisposable::Tld::SPECIAL_USE if Rails.env.local?
+> ```
+>
+> Scope it to your non-production environments, so production still refuses an address nobody could ever answer.
+
+`blocked_tlds` refuses TLDs that are perfectly real but that you'd rather not see — the historically free Freenom set (`tk`, `ml`, `ga`, `cf`, `gq`) is the usual suspect. `allowed_tlds` inverts it into an allowlist; be careful, `%w[es]` turns away every `.com` customer you have.
+
+#### `config.reject_lookalike_domains` — did they mean gmail.com?
+
+A TLD check structurally **cannot** catch the most common typos, because `.co` (Colombia), `.cm` (Cameroon) and `.om` (Oman) are all real, delegated TLDs. `user@gmail.co` is a well-formed address at a real TLD and still almost always a finger that slipped off the `m`.
+
+So the second check asks a different question: is this domain one edit away from a well-known provider, while not being one itself?
+
+```ruby
+Nondisposable.suggestion_for("someone@gmial.com") # => "someone@gmail.com"
+Nondisposable.suggestion_for("someone@gmail.co")  # => "someone@gmail.com"
+Nondisposable.suggestion_for("someone@gmail.com") # => nil
+Nondisposable.suggestion_for("someone@mail.com")  # => nil  (a real provider)
+```
+
+`suggestion_for` is always available and **never blocks anything** — show it as a hint and let the human decide. That's the gentler way to use this, and the one to reach for first:
+
+```erb
+<% if (did_you_mean = Nondisposable.suggestion_for(@user.email)) %>
+  <p>Did you mean <%= did_you_mean %>?</p>
+<% end %>
+```
+
+Setting `reject_lookalike_domains = true` turns the same guess into a validation error naming the correction. Do that deliberately: a suggestion is a guess about intent, and a wrong guess stops a real person signing up with their real address. Two things keep that rare — matching stops at **one** edit by default (`lookalike_distance`), and an exact match against the bundled provider list always wins, which is why that list includes awkward pairs like `mail.com` (one insertion from `gmail.com`). Add anything we've missed with `config.additional_email_providers`; doing so both makes it a suggestion candidate and stops its users being told they mistyped.
+
+The matching uses optimal string alignment distance rather than plain Levenshtein, so an adjacent swap counts as the one mistake a human actually made: `gmial.com` is distance 1, not 2.
+
+#### How the two combine
+
+At most one error is added, however many checks fire — and whenever the gem can name a correction, that phrasing wins:
+
+| Address | `check_tld` only | with `reject_lookalike_domains` |
+|---|---|---|
+| `user@gmail.con` | "looks like a typo. Did you mean user@gmail.com?" | same |
+| `user@zzz.con` | "doesn't look like a real email address" | same |
+| `user@gmail.co` | accepted | "looks like a typo. Did you mean user@gmail.com?" |
+| `user@tempmail.com` | "provider is not allowed" | same |
+
+"Doesn't look like a real email address" is true but useless when we know what they meant, so a nameable correction always outranks the generic message.
+
 ### Direct Check
 
 You can also check if an email is disposable directly:
@@ -123,6 +226,8 @@ You can also check if an email is disposable directly:
 ```ruby
 Nondisposable.disposable?('user@example.com') # => false
 Nondisposable.disposable?('user@disposable-email.com') # => true
+Nondisposable.valid_tld?('user@example.con')  # => false
+Nondisposable.suggestion_for('user@gmial.com') # => "user@gmail.com"
 ```
 
 ## Updating disposable domains
@@ -148,6 +253,24 @@ production:
     schedule: every day at 3am US/Pacific
 ```
 
+## Updating the TLD list
+
+The TLD snapshot is the deliberate opposite of the disposable list: it ships **inside the gem**, not in your database, and there is nothing for your app to schedule. Disposable domains number in the thousands and change every few days; TLDs are ~1,438 strings that change a handful of times a year, so a frozen `Set` costs one file read at boot, answers in O(1) with no query per signup, needs no migration, and can't be empty on a fresh install.
+
+If ICANN delegates a TLD your installed version doesn't know about, don't wait for a release — that's what `config.additional_tlds` is for.
+
+Maintainers refresh the snapshot from a checkout of this gem:
+
+```bash
+rake nondisposable:tlds:update   # rewrites data/iana_tlds.txt from data.iana.org
+```
+
+It refuses to overwrite a good list with an implausibly short one (a captive portal or a truncated body served with a `200`), writes to a temp file and moves it into place so an interrupted run can't leave half a root zone behind, and keeps IANA's own version header so you can always see which root zone you're shipping:
+
+```ruby
+Nondisposable::Tld.version # => "2026082301, Last Updated Mon Aug 24 07:07:01 2026 UTC"
+```
+
 ## Troubleshooting
 
 ### SSL certificate verify failed (unable to get certificate CRL)

data/Rakefile

@@ -3,6 +3,11 @@
 require "bundler/gem_tasks"
 require "rake/testtask"
 
+# nondisposable:tlds:update — refreshes data/iana_tlds.txt from the IANA root
+# zone. A maintenance task for this checkout, not something host apps run; see
+# the header of lib/nondisposable/tld_list_updater.rb.
+Dir.glob("lib/tasks/*.rake").each { |task| load task }
+
 Rake::TestTask.new(:test) do |t|
   t.libs << "lib"
   t.libs << "test"

data/data/email_providers.txt

@@ -0,0 +1,276 @@
+# Well-known consumer email domains, used by Nondisposable::Suggestion.
+#
+# This list does TWO jobs, and the second one is the reason it needs to be
+# generous rather than short:
+#
+#   1. It supplies the candidates a typo can be corrected TO. A domain one edit
+#      away from an entry here is very probably a slip of the finger.
+#   2. It is the false-positive guard. An EXACT match here is never flagged, so
+#      every real provider that happens to sit one edit from another real one
+#      must be present. `mail.com` is the canonical example: it is one insertion
+#      away from `gmail.com`, and leaving it out would tell every one of its
+#      users they had made a typo.
+#
+# Hosts add their own with `config.additional_email_providers`. Order is
+# irrelevant; comments and blank lines are ignored.
+
+# --- global ---
+gmail.com
+googlemail.com
+outlook.com
+hotmail.com
+live.com
+msn.com
+passport.com
+yahoo.com
+ymail.com
+rocketmail.com
+icloud.com
+me.com
+mac.com
+aol.com
+mail.com
+email.com
+usa.com
+gmx.com
+gmx.net
+gmx.de
+gmx.at
+gmx.ch
+zoho.com
+zohomail.com
+fastmail.com
+fastmail.fm
+hushmail.com
+protonmail.com
+protonmail.ch
+proton.me
+pm.me
+tutanota.com
+tutanota.de
+tuta.com
+tuta.io
+hey.com
+mailfence.com
+posteo.de
+disroot.org
+riseup.net
+runbox.com
+migadu.com
+purelymail.com
+skiff.com
+duck.com
+
+# --- Spain ---
+hotmail.es
+outlook.es
+yahoo.es
+live.com.es
+telefonica.net
+movistar.es
+terra.es
+terra.com
+ya.com
+wanadoo.es
+orange.es
+jazzfree.com
+jazztel.es
+euskaltel.net
+euskalnet.net
+ono.com
+vodafone.es
+telecable.es
+mundo-r.com
+airtel.net
+navegalia.com
+eresmas.com
+inicia.es
+teleline.es
+
+# --- Portugal / Latin America ---
+sapo.pt
+netcabo.pt
+clix.pt
+uol.com.br
+bol.com.br
+ig.com.br
+terra.com.br
+globo.com
+globomail.com
+prodigy.net.mx
+hotmail.com.ar
+hotmail.com.mx
+yahoo.com.ar
+yahoo.com.mx
+yahoo.com.br
+
+# --- France ---
+orange.fr
+wanadoo.fr
+free.fr
+sfr.fr
+laposte.net
+bbox.fr
+neuf.fr
+aliceadsl.fr
+club-internet.fr
+numericable.fr
+hotmail.fr
+outlook.fr
+yahoo.fr
+live.fr
+
+# --- Germany / Austria / Switzerland ---
+web.de
+t-online.de
+freenet.de
+arcor.de
+1und1.de
+online.de
+gmx.info
+hotmail.de
+outlook.de
+yahoo.de
+live.de
+bluewin.ch
+sunrise.ch
+gmail.ch
+
+# --- United Kingdom / Ireland ---
+btinternet.com
+sky.com
+virginmedia.com
+talktalk.net
+ntlworld.com
+blueyonder.co.uk
+hotmail.co.uk
+outlook.co.uk
+yahoo.co.uk
+live.co.uk
+googlemail.co.uk
+eircom.net
+
+# --- Italy ---
+libero.it
+virgilio.it
+alice.it
+tiscali.it
+tin.it
+fastwebnet.it
+hotmail.it
+outlook.it
+yahoo.it
+live.it
+pec.it
+
+# --- Benelux / Nordics / other Europe ---
+telenet.be
+skynet.be
+proximus.be
+ziggo.nl
+kpnmail.nl
+home.nl
+planet.nl
+xs4all.nl
+telia.com
+telia.se
+bredband.net
+spray.se
+hotmail.se
+online.no
+broadpark.no
+tele2.no
+elisanet.fi
+luukku.com
+suomi24.fi
+seznam.cz
+centrum.cz
+wp.pl
+o2.pl
+onet.pl
+interia.pl
+gazeta.pl
+abv.bg
+mail.bg
+freemail.hu
+citromail.hu
+t-online.hu
+
+# --- Russia / Eastern Europe ---
+mail.ru
+inbox.ru
+list.ru
+bk.ru
+internet.ru
+yandex.ru
+yandex.com
+ya.ru
+rambler.ru
+ukr.net
+i.ua
+meta.ua
+
+# --- Asia ---
+qq.com
+foxmail.com
+163.com
+126.com
+yeah.net
+sina.com
+sina.cn
+sohu.com
+aliyun.com
+naver.com
+daum.net
+hanmail.net
+nate.com
+kakao.com
+docomo.ne.jp
+ezweb.ne.jp
+softbank.ne.jp
+yahoo.co.jp
+ybb.ne.jp
+nifty.com
+biglobe.ne.jp
+so-net.ne.jp
+rediffmail.com
+sify.com
+
+# --- North America ISPs ---
+comcast.net
+xfinity.com
+verizon.net
+att.net
+sbcglobal.net
+bellsouth.net
+ameritech.net
+pacbell.net
+cox.net
+charter.net
+spectrum.net
+earthlink.net
+juno.com
+netzero.net
+optonline.net
+roadrunner.com
+rr.com
+windstream.net
+frontier.com
+centurylink.net
+shaw.ca
+rogers.com
+sympatico.ca
+telus.net
+videotron.ca
+bell.net
+
+# --- Oceania / Africa ---
+bigpond.com
+bigpond.net.au
+optusnet.com.au
+iinet.net.au
+tpg.com.au
+xtra.co.nz
+webmail.co.za
+telkomsa.net
+mweb.co.za