tldextract 4.0.0__tar.gz → 5.0.0__tar.gz

{tldextract-4.0.0 → tldextract-5.0.0}/.travis.yml

@@ -2,8 +2,6 @@ dist: focal
 language: python
 matrix:
   include:
-    - python: "3.7"
-      env: TOXENV=py37
     - python: "3.8"
       env: TOXENV=py38
     - python: "3.9"
@@ -12,7 +10,7 @@ matrix:
       env: TOXENV=py310
     - python: "3.11"
       env: TOXENV=py311
-    - python: pypy3.7-7.3.9
+    - python: pypy3.8-7.3.9
       dist: xenial
       env: TOXENV=pypy3
     - env: TOXENV=codestyle

{tldextract-4.0.0 → tldextract-5.0.0}/CHANGELOG.md

@@ -3,6 +3,30 @@
 After upgrading, update your cache file by deleting it or via `tldextract
 --update`.
 
+## 5.0.0 (2023-10-11)
+
+* Breaking Changes
+    * Migrate `ExtractResult` from `namedtuple` to `dataclass` ([#306](https://github.com/john-kurkowski/tldextract/issues/306))
+        * This means no more iterating/indexing/slicing/unpacking the result
+          object returned by this library. You must directly reference the
+          fields you're interested in. For example, instead of
+          ```python
+          tldextract.extract("example.com")[1:3]
+          ```
+          you must use
+          ```python
+          ext = tldextract.extract("example.com")
+          (ext.domain, ext.suffix)
+          ```
+* Bugfixes
+    * Drop support for EOL Python 3.7
+* Misc.
+    * Switch from pycodestyle and Pylint to Ruff ([#304](https://github.com/john-kurkowski/tldextract/issues/304))
+    * Consolidate config files
+    * Type tests
+    * Require docstrings in tests
+    * Remove obsolete tests
+
 ## 4.0.0 (2023-10-11)
 
 * **Breaking** bugfixes

{tldextract-4.0.0 → tldextract-5.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: tldextract
-Version: 4.0.0
+Version: 5.0.0
 Summary: Accurately separates a URL's subdomain, domain, and public suffix, using the Public Suffix List (PSL). By default, this includes the public ICANN TLDs and their exceptions. You can optionally support the Public Suffix List's private domains as well.
 Author-email: John Kurkowski <john.kurkowski@gmail.com>
 License: BSD-3-Clause
@@ -10,12 +10,11 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Topic :: Utilities
 Classifier: License :: OSI Approved :: BSD License
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.7
 Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Requires-Python: >=3.7
+Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: idna
@@ -56,20 +55,6 @@ ExtractResult(subdomain='forums', domain='bbc', suffix='co.uk', is_private=False
 ExtractResult(subdomain='www', domain='worldbank', suffix='org.kg', is_private=False)
 ```
 
-`ExtractResult` is a namedtuple, so it's simple to access the parts you want.
-
-```python
->>> ext = tldextract.extract('http://forums.bbc.co.uk')
->>> (ext.subdomain, ext.domain, ext.suffix)
-('forums', 'bbc', 'co.uk')
->>> # rejoin subdomain and domain
->>> '.'.join(ext[:2])
-'forums.bbc'
->>> # a common alias
->>> ext.registered_domain
-'bbc.co.uk'
-```
-
 Note subdomain and suffix are _optional_. Not all URL-like inputs have a
 subdomain or a valid suffix.
 
@@ -84,17 +69,14 @@ ExtractResult(subdomain='google', domain='notavalidsuffix', suffix='', is_privat
 ExtractResult(subdomain='', domain='127.0.0.1', suffix='', is_private=False)
 ```
 
-If you want to rejoin the whole namedtuple, regardless of whether a subdomain
-or suffix were found:
+To rejoin the original hostname, if it was indeed a valid, registered hostname:
 
 ```python
->>> ext = tldextract.extract('http://127.0.0.1:8080/deployed/')
->>> # this has unwanted dots
->>> '.'.join(ext[:3])
-'.127.0.0.1.'
->>> # join each part only if it's truthy
->>> '.'.join(part for part in ext[:3] if part)
-'127.0.0.1'
+>>> ext = tldextract.extract('http://forums.bbc.co.uk')
+>>> ext.registered_domain
+'bbc.co.uk'
+>>> ext.fqdn
+'forums.bbc.co.uk'
 ```
 
 By default, this package supports the public ICANN TLDs and their exceptions.
@@ -303,7 +285,7 @@ Run all tests against a specific Python environment configuration:
 
 ```zsh
 tox -l
-tox -e py37
+tox -e py311
 ```
 
 ### Code Style

{tldextract-4.0.0 → tldextract-5.0.0}/README.md

@@ -31,20 +31,6 @@ ExtractResult(subdomain='forums', domain='bbc', suffix='co.uk', is_private=False
 ExtractResult(subdomain='www', domain='worldbank', suffix='org.kg', is_private=False)
 ```
 
-`ExtractResult` is a namedtuple, so it's simple to access the parts you want.
-
-```python
->>> ext = tldextract.extract('http://forums.bbc.co.uk')
->>> (ext.subdomain, ext.domain, ext.suffix)
-('forums', 'bbc', 'co.uk')
->>> # rejoin subdomain and domain
->>> '.'.join(ext[:2])
-'forums.bbc'
->>> # a common alias
->>> ext.registered_domain
-'bbc.co.uk'
-```
-
 Note subdomain and suffix are _optional_. Not all URL-like inputs have a
 subdomain or a valid suffix.
 
@@ -59,17 +45,14 @@ ExtractResult(subdomain='google', domain='notavalidsuffix', suffix='', is_privat
 ExtractResult(subdomain='', domain='127.0.0.1', suffix='', is_private=False)
 ```
 
-If you want to rejoin the whole namedtuple, regardless of whether a subdomain
-or suffix were found:
+To rejoin the original hostname, if it was indeed a valid, registered hostname:
 
 ```python
->>> ext = tldextract.extract('http://127.0.0.1:8080/deployed/')
->>> # this has unwanted dots
->>> '.'.join(ext[:3])
-'.127.0.0.1.'
->>> # join each part only if it's truthy
->>> '.'.join(part for part in ext[:3] if part)
-'127.0.0.1'
+>>> ext = tldextract.extract('http://forums.bbc.co.uk')
+>>> ext.registered_domain
+'bbc.co.uk'
+>>> ext.fqdn
+'forums.bbc.co.uk'
 ```
 
 By default, this package supports the public ICANN TLDs and their exceptions.
@@ -278,7 +261,7 @@ Run all tests against a specific Python environment configuration:
 
 ```zsh
 tox -l
-tox -e py37
+tox -e py311
 ```
 
 ### Code Style

{tldextract-4.0.0 → tldextract-5.0.0}/pyproject.toml

@@ -23,13 +23,12 @@ classifiers = [
     "Topic :: Utilities",
     "License :: OSI Approved :: BSD License",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.7",
     "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
 ]
-requires-python = ">=3.7"
+requires-python = ">=3.8"
 dependencies = [
     "idna",
     "requests>=2.1.0",
@@ -67,11 +66,27 @@ version = {attr = "setuptools_scm.get_version"}
 check_untyped_defs = true
 disallow_incomplete_defs = true
 disallow_untyped_calls = true
-
-[[tool.mypy.overrides]]
-module = ["tldextract.*"]
 disallow_untyped_defs = true
 
-[tool.pylint.master]
-disable = "fixme"
-no-docstring-rgx = "(^_|test_.*)"
+[tool.pytest.ini_options]
+addopts = "--doctest-modules"
+
+[tool.ruff]
+select = [
+  "A",
+  "B",
+  "C",
+  "D",
+  "E",
+  "F",
+  "I",
+  "N",
+  "UP",
+  "W",
+]
+ignore = [
+  "E501", # line too long; if Black does its job, not worried about the rare long line
+]
+
+[tool.ruff.pydocstyle]
+convention = "pep257"

tldextract-5.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

{tldextract-4.0.0 → tldextract-5.0.0}/tests/cli_test.py

@@ -8,7 +8,8 @@ from tldextract.cli import main
 from tldextract.tldextract import PUBLIC_SUFFIX_LIST_URLS
 
 
-def test_cli_no_input(monkeypatch):
+def test_cli_no_input(monkeypatch: pytest.MonkeyPatch) -> None:
+    """Test CLI without args."""
     monkeypatch.setattr(sys, "argv", ["tldextract"])
     with pytest.raises(SystemExit) as ex:
         main()
@@ -16,7 +17,8 @@ def test_cli_no_input(monkeypatch):
     assert ex.value.code == 1
 
 
-def test_cli_parses_args(monkeypatch):
+def test_cli_parses_args(monkeypatch: pytest.MonkeyPatch) -> None:
+    """Test CLI with nonsense args."""
     monkeypatch.setattr(sys, "argv", ["tldextract", "--some", "nonsense"])
     with pytest.raises(SystemExit) as ex:
         main()
@@ -24,7 +26,10 @@ def test_cli_parses_args(monkeypatch):
     assert ex.value.code == 2
 
 
-def test_cli_posargs(capsys, monkeypatch):
+def test_cli_posargs(
+    capsys: pytest.CaptureFixture, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """Test CLI with basic, positional args."""
     monkeypatch.setattr(
         sys, "argv", ["tldextract", "example.com", "bbc.co.uk", "forums.bbc.co.uk"]
     )
@@ -36,7 +41,10 @@ def test_cli_posargs(capsys, monkeypatch):
     assert stdout == " example com\n bbc co.uk\nforums bbc co.uk\n"
 
 
-def test_cli_namedargs(capsys, monkeypatch):
+def test_cli_namedargs(
+    capsys: pytest.CaptureFixture, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """Test CLI with basic, positional args, and that it parses an optional argument (though it doesn't change output)."""
     monkeypatch.setattr(
         sys,
         "argv",

{tldextract-4.0.0 → tldextract-5.0.0}/tests/conftest.py

@@ -8,12 +8,10 @@ import tldextract.cache
 
 
 @pytest.fixture(autouse=True)
-def reset_log_level():
+def reset_log_level() -> None:
     """Automatically reset log level verbosity between tests.
 
     Generally want test output the Unix way: silence is golden.
     """
-    tldextract.cache._DID_LOG_UNABLE_TO_CACHE = (  # pylint: disable=protected-access
-        False
-    )
+    tldextract.cache._DID_LOG_UNABLE_TO_CACHE = False
     logging.getLogger().setLevel(logging.WARN)

{tldextract-4.0.0 → tldextract-5.0.0}/tests/custom_suffix_test.py

@@ -4,6 +4,7 @@ import os
 import tempfile
 
 import tldextract
+from tldextract.tldextract import ExtractResult
 
 FAKE_SUFFIX_LIST_URL = "file://" + os.path.join(
     os.path.dirname(os.path.abspath(__file__)), "fixtures/fake_suffix_list_fixture.dat"
@@ -23,11 +24,12 @@ extract_using_extra_suffixes = tldextract.TLDExtract(
 )
 
 
-def test_private_extraction():
+def test_private_extraction() -> None:
+    """Test this library's uncached, offline, private domain extraction."""
     tld = tldextract.TLDExtract(cache_dir=tempfile.mkdtemp(), suffix_list_urls=[])
 
-    assert tld("foo.blogspot.com") == ("foo", "blogspot", "com", False)
-    assert tld("foo.blogspot.com", include_psl_private_domains=True) == (
+    assert tld("foo.blogspot.com") == ExtractResult("foo", "blogspot", "com", False)
+    assert tld("foo.blogspot.com", include_psl_private_domains=True) == ExtractResult(
         "",
         "foo",
         "blogspot.com",
@@ -35,7 +37,8 @@ def test_private_extraction():
     )
 
 
-def test_suffix_which_is_not_in_custom_list():
+def test_suffix_which_is_not_in_custom_list() -> None:
+    """Test a custom suffix list without .com."""
     for fun in (
         extract_using_fake_suffix_list,
         extract_using_fake_suffix_list_no_cache,
@@ -44,7 +47,8 @@ def test_suffix_which_is_not_in_custom_list():
         assert result.suffix == ""
 
 
-def test_custom_suffixes():
+def test_custom_suffixes() -> None:
+    """Test a custom suffix list with common, metasyntactic suffixes."""
     for fun in (
         extract_using_fake_suffix_list,
         extract_using_fake_suffix_list_no_cache,
@@ -54,12 +58,14 @@ def test_custom_suffixes():
             assert result.suffix == custom_suffix
 
 
-def test_suffix_which_is_not_in_extra_list():
+def test_suffix_which_is_not_in_extra_list() -> None:
+    """Test a custom suffix list and extra suffixes without .com."""
     result = extract_using_extra_suffixes("www.google.com")
     assert result.suffix == ""
 
 
-def test_extra_suffixes():
+def test_extra_suffixes() -> None:
+    """Test extra suffixes."""
     for custom_suffix in EXTRA_SUFFIXES:
         netloc = "www.foo.bar.baz.quux" + "." + custom_suffix
         result = extract_using_extra_suffixes(netloc)

tldextract-5.0.0/tests/integration_test.py

@@ -0,0 +1,13 @@
+"""tldextract integration tests."""
+
+import pytest
+
+import tldextract
+
+
+def test_bad_kwargs_no_way_to_fetch() -> None:
+    """Test an impossible combination of kwargs that disable all ways to fetch data."""
+    with pytest.raises(ValueError, match="disable all ways"):
+        tldextract.TLDExtract(
+            cache_dir=None, suffix_list_urls=(), fallback_to_snapshot=False
+        )