tldextract 5.1.2__tar.gz → 5.1.3__tar.gz

{tldextract-5.1.2 → tldextract-5.1.3}/.github/workflows/ci.yml

@@ -14,22 +14,21 @@ jobs:
         os: [macos-latest, windows-latest, ubuntu-latest]
         language:
           [
-            {python-version: "3.8", toxenv: "py38"},
             {python-version: "3.9", toxenv: "py39"},
             {python-version: "3.10", toxenv: "py310"},
             {python-version: "3.11", toxenv: "py311"},
             {python-version: "3.12", toxenv: "py312"},
-            {python-version: "pypy3.8", toxenv: "pypy38"},
+            {python-version: "3.13", toxenv: "py313"},
             {python-version: "pypy3.9", toxenv: "pypy39"},
             {python-version: "pypy3.10", toxenv: "pypy310"},
           ]
         include:
           - os: ubuntu-latest
-            language: {python-version: "3.8", toxenv: "codestyle"}
+            language: {python-version: "3.9", toxenv: "codestyle"}
           - os: ubuntu-latest
-            language: {python-version: "3.8", toxenv: "lint"}
+            language: {python-version: "3.9", toxenv: "lint"}
           - os: ubuntu-latest
-            language: {python-version: "3.8", toxenv: "typecheck"}
+            language: {python-version: "3.9", toxenv: "typecheck"}
     runs-on: ${{ matrix.os }}
     steps:
       - name: Check out repository
@@ -41,7 +40,7 @@ jobs:
           check-latest: true
       - name: Install Python requirements
         run: |
-          pip install --upgrade tox
+          pip install --upgrade tox tox-uv
       - name: Test
         run: tox
         env:

{tldextract-5.1.2 → tldextract-5.1.3}/CHANGELOG.md

@@ -3,6 +3,20 @@
 After upgrading, update your cache file by deleting it or via `tldextract
 --update`.
 
+## 5.1.3 (2024-11-04)
+
+* Bugfixes
+  * Reduce logging errors ([`921a825`](https://github.com/john-kurkowski/tldextract/commit/921a82523c0e4403d21d50b2c3410d9af43520ac))
+  * Drop support for EOL Python 3.8 ([#340](https://github.com/john-kurkowski/tldextract/issues/340))
+  * Support Python 3.13 ([#341](https://github.com/john-kurkowski/tldextract/issues/341))
+  * Update bundled snapshot
+* Documentation
+  * Clarify how to use your own definitions
+  * Clarify first-successful definitions vs. merged definitions
+* Misc.
+  * Switch from Black to Ruff ([#333](https://github.com/john-kurkowski/tldextract/issues/333))
+  * Switch from pip to uv, during tox ([#324](https://github.com/john-kurkowski/tldextract/issues/324))
+
 ## 5.1.2 (2024-03-18)
 
 * Bugfixes

{tldextract-5.1.2 → tldextract-5.1.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: tldextract
-Version: 5.1.2
+Version: 5.1.3
 Summary: Accurately separates a URL's subdomain, domain, and public suffix, using the Public Suffix List (PSL). By default, this includes the public ICANN TLDs and their exceptions. You can optionally support the Public Suffix List's private domains as well.
 Author-email: John Kurkowski <john.kurkowski@gmail.com>
 License: BSD-3-Clause
@@ -10,12 +10,12 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Topic :: Utilities
 Classifier: License :: OSI Approved :: BSD License
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
-Requires-Python: >=3.8
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: idna
@@ -26,7 +26,6 @@ Provides-Extra: release
 Requires-Dist: build; extra == "release"
 Requires-Dist: twine; extra == "release"
 Provides-Extra: testing
-Requires-Dist: black; extra == "testing"
 Requires-Dist: mypy; extra == "testing"
 Requires-Dist: pytest; extra == "testing"
 Requires-Dist: pytest-gitignore; extra == "testing"
@@ -35,6 +34,7 @@ Requires-Dist: responses; extra == "testing"
 Requires-Dist: ruff; extra == "testing"
 Requires-Dist: syrupy; extra == "testing"
 Requires-Dist: tox; extra == "testing"
+Requires-Dist: tox-uv; extra == "testing"
 Requires-Dist: types-filelock; extra == "testing"
 Requires-Dist: types-requests; extra == "testing"
 
@@ -129,8 +129,8 @@ tldextract http://forums.bbc.co.uk
 
 Beware when first calling `tldextract`, it updates its TLD list with a live HTTP
 request. This updated TLD set is usually cached indefinitely in `$HOME/.cache/python-tldextract`.
-To control the cache's location, set TLDEXTRACT_CACHE environment variable or set the
-cache_dir path in TLDExtract initialization.
+To control the cache's location, set the `TLDEXTRACT_CACHE` environment variable or set the
+`cache_dir` path when constructing a `TLDExtract`.
 
 (Arguably runtime bootstrapping like that shouldn't be the default behavior,
 like for production systems. But I want you to have the latest TLDs, especially
@@ -219,10 +219,12 @@ extract = tldextract.TLDExtract(
     fallback_to_snapshot=False)
 ```
 
-The above snippet will fetch from the URL *you* specified, upon first need to download the
-suffix list (i.e. if the cached version doesn't exist).
+If the cached version of public suffix definitions doesn't exist, such as on
+the first run, the above snippet will request the URLs you specified in order,
+and use the first successful response.
 
-If you want to use input data from your local filesystem, just use the `file://` protocol:
+If you want to use input data from your local filesystem, use the `file://`
+protocol with an absolute path:
 
 ```python
 extract = tldextract.TLDExtract(
@@ -231,17 +233,24 @@ extract = tldextract.TLDExtract(
     fallback_to_snapshot=False)
 ```
 
-Use an absolute path when specifying the `suffix_list_urls` keyword argument.
-`os.path` is your friend.
-
-The command line update command can be used with a URL or local file you specify:
+This also works via command line update:
 
 ```zsh
 tldextract --update --suffix_list_url "http://foo.bar.baz"
 ```
 
-This could be useful in production when you don't want the delay associated with updating the suffix
-list on first use, or if you are behind a complex firewall that prevents a simple update from working.
+Using your own URLs could be useful in production when you don't want the delay
+with updating the suffix list on first use, or if you are behind a complex
+firewall.
+
+You can also specify additional suffixes in the `extra_suffixes` param. These
+will be merged into whatever public suffix definitions are already in use by
+`tldextract`.
+
+```python
+extract = tldextract.TLDExtract(
+    extra_suffixes=["foo", "bar", "baz"])
+```
 
 ## FAQ
 
@@ -250,9 +259,9 @@ list on first use, or if you are behind a complex firewall that prevents a simpl
 This project doesn't contain an actual list of public suffixes. That comes from
 [the Public Suffix List (PSL)](https://publicsuffix.org/). Submit amendments there.
 
-(In the meantime, you can tell tldextract about your exception by either
+In the meantime, you can tell tldextract about your exception by either
 forking the PSL and using your fork in the `suffix_list_urls` param, or adding
-your suffix piecemeal with the `extra_suffixes` param.)
+your suffix piecemeal with the `extra_suffixes` param.
 
 ### I see my suffix in [the Public Suffix List (PSL)](https://publicsuffix.org/), but this library doesn't extract it.
 
@@ -309,5 +318,5 @@ tox -e py311
 Automatically format all code:
 
 ```zsh
-black .
+ruff format .
 ```

{tldextract-5.1.2 → tldextract-5.1.3}/README.md

@@ -89,8 +89,8 @@ tldextract http://forums.bbc.co.uk
 
 Beware when first calling `tldextract`, it updates its TLD list with a live HTTP
 request. This updated TLD set is usually cached indefinitely in `$HOME/.cache/python-tldextract`.
-To control the cache's location, set TLDEXTRACT_CACHE environment variable or set the
-cache_dir path in TLDExtract initialization.
+To control the cache's location, set the `TLDEXTRACT_CACHE` environment variable or set the
+`cache_dir` path when constructing a `TLDExtract`.
 
 (Arguably runtime bootstrapping like that shouldn't be the default behavior,
 like for production systems. But I want you to have the latest TLDs, especially
@@ -179,10 +179,12 @@ extract = tldextract.TLDExtract(
     fallback_to_snapshot=False)
 ```
 
-The above snippet will fetch from the URL *you* specified, upon first need to download the
-suffix list (i.e. if the cached version doesn't exist).
+If the cached version of public suffix definitions doesn't exist, such as on
+the first run, the above snippet will request the URLs you specified in order,
+and use the first successful response.
 
-If you want to use input data from your local filesystem, just use the `file://` protocol:
+If you want to use input data from your local filesystem, use the `file://`
+protocol with an absolute path:
 
 ```python
 extract = tldextract.TLDExtract(
@@ -191,17 +193,24 @@ extract = tldextract.TLDExtract(
     fallback_to_snapshot=False)
 ```
 
-Use an absolute path when specifying the `suffix_list_urls` keyword argument.
-`os.path` is your friend.
-
-The command line update command can be used with a URL or local file you specify:
+This also works via command line update:
 
 ```zsh
 tldextract --update --suffix_list_url "http://foo.bar.baz"
 ```
 
-This could be useful in production when you don't want the delay associated with updating the suffix
-list on first use, or if you are behind a complex firewall that prevents a simple update from working.
+Using your own URLs could be useful in production when you don't want the delay
+with updating the suffix list on first use, or if you are behind a complex
+firewall.
+
+You can also specify additional suffixes in the `extra_suffixes` param. These
+will be merged into whatever public suffix definitions are already in use by
+`tldextract`.
+
+```python
+extract = tldextract.TLDExtract(
+    extra_suffixes=["foo", "bar", "baz"])
+```
 
 ## FAQ
 
@@ -210,9 +219,9 @@ list on first use, or if you are behind a complex firewall that prevents a simpl
 This project doesn't contain an actual list of public suffixes. That comes from
 [the Public Suffix List (PSL)](https://publicsuffix.org/). Submit amendments there.
 
-(In the meantime, you can tell tldextract about your exception by either
+In the meantime, you can tell tldextract about your exception by either
 forking the PSL and using your fork in the `suffix_list_urls` param, or adding
-your suffix piecemeal with the `extra_suffixes` param.)
+your suffix piecemeal with the `extra_suffixes` param.
 
 ### I see my suffix in [the Public Suffix List (PSL)](https://publicsuffix.org/), but this library doesn't extract it.
 
@@ -269,5 +278,5 @@ tox -e py311
 Automatically format all code:
 
 ```zsh
-black .
+ruff format .
 ```

{tldextract-5.1.2 → tldextract-5.1.3}/pyproject.toml

@@ -23,13 +23,13 @@ classifiers = [
     "Topic :: Utilities",
     "License :: OSI Approved :: BSD License",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
 ]
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 dynamic = ["version"]
 readme = "README.md"
 
@@ -46,7 +46,6 @@ release = [
     "twine",
 ]
 testing = [
-    "black",
     "mypy",
     "pytest",
     "pytest-gitignore",
@@ -55,6 +54,7 @@ testing = [
     "ruff",
     "syrupy",
     "tox",
+    "tox-uv",
     "types-filelock",
     "types-requests",
 ]
@@ -104,7 +104,7 @@ select = [
   "W",
 ]
 ignore = [
-  "E501", # line too long; if Black does its job, not worried about the rare long line
+  "E501", # line too long; if formatter does its job, not worried about the rare long line
 ]
 
 [tool.ruff.lint.pydocstyle]

{tldextract-5.1.2 → tldextract-5.1.3}/scripts/release.py

@@ -13,7 +13,7 @@ It will:
 Prerequisites:
     - This must be run from the root of the repository.
     - The repo must have a clean git working tree.
-    - The user must have the GITHUB_TOKEN environment variable set to a valid GitHub personal access token.
+    - The user must have the GITHUB_TOKEN environment variable set to a GitHub personal access token with repository "Contents" read and write permission.
     - The user will need credentials for the PyPI repository, which the user will be prompted for during the upload step. The user will need to paste the token manually from a password manager or similar.
     - The CHANGELOG.md file must already contain an entry for the version being released.
     - Install requirements with: pip install --upgrade --editable '.[release]'
@@ -22,19 +22,27 @@ Prerequisites:
 
 from __future__ import annotations
 
+import contextlib
 import os
 import re
 import subprocess
 import sys
+from collections.abc import Iterator
 from pathlib import Path
 
 import requests
 
 
-def add_git_tag_for_version(version: str) -> None:
+@contextlib.contextmanager
+def add_git_tag_for_version(version: str) -> Iterator[None]:
     """Add a git tag for the given version."""
     subprocess.run(["git", "tag", "-a", version, "-m", version], check=True)
     print(f"Version {version} tag added successfully.")
+    try:
+        yield
+    except:
+        subprocess.run(["git", "tag", "-d", version])
+        raise
 
 
 def remove_previous_dist() -> None:
@@ -68,14 +76,16 @@ def verify_build(is_test: str) -> None:
     confirmation = input("Does the build look correct? (y/n): ")
     if confirmation == "y":
         print("Build verified successfully.")
-        upload_build_to_pypi(is_test)
-        push_git_tags()
     else:
         raise Exception("Could not verify. Build was not uploaded.")
 
 
 def generate_github_release_notes_body(token: str, version: str) -> str:
-    """Generate and grab release notes URL from Github."""
+    """Generate and grab release notes URL from Github.
+
+    Delete their first paragraph, because we track its contents in a tighter
+    form in CHANGELOG.md. See `get_changelog_release_notes`.
+    """
     response = requests.post(
         "https://api.github.com/repos/john-kurkowski/tldextract/releases/generate-notes",
         headers={
@@ -94,24 +104,13 @@ def generate_github_release_notes_body(token: str, version: str) -> str:
             file=sys.stderr,
         )
         return ""
-    return str(response.json()["body"])
 
+    body = str(response.json()["body"])
+    paragraphs = body.split("\n\n")
+    return "\n\n".join(paragraphs[1:])
 
-def get_release_notes_url(body: str) -> str:
-    """Parse the release notes content to get the changelog URL."""
-    url_pattern = re.compile(r"\*\*Full Changelog\*\*: (.*)$")
-    match = url_pattern.search(body)
-    if match:
-        return match.group(1)
-    else:
-        print(
-            "WARNING: Failed to parse release notes URL from GitHub response.",
-            file=sys.stderr,
-        )
-        return ""
 
-
-def get_changelog_release_notes(release_notes_url: str, version: str) -> str:
+def get_changelog_release_notes(version: str) -> str:
     """Get the changelog release notes.
 
     Uses a regex starting on a heading beginning with the version number
@@ -125,25 +124,15 @@ def get_changelog_release_notes(release_notes_url: str, version: str) -> str:
     if match:
         return str(match.group(1)).strip()
     else:
-        print(
-            f"WARNING: Failed to parse changelog release notes. Manually copy this version's notes from the CHANGELOG.md file to {release_notes_url}.",
-            file=sys.stderr,
-        )
         return ""
 
 
-def create_release_notes_body(token: str, version: str) -> str:
-    """Compile the release notes."""
-    github_release_body = generate_github_release_notes_body(token, version)
-    release_notes_url = get_release_notes_url(github_release_body)
-    changelog_notes = get_changelog_release_notes(release_notes_url, version)
-    full_release_notes = f"{changelog_notes}\n\n**Full Changelog**: {release_notes_url}"
-    return full_release_notes
-
-
 def create_github_release_draft(token: str, version: str) -> None:
     """Create a release on GitHub."""
-    release_body = create_release_notes_body(token, version)
+    github_release_body = generate_github_release_notes_body(token, version)
+    changelog_notes = get_changelog_release_notes(version)
+    release_body = f"{changelog_notes}\n\n{github_release_body}"
+
     response = requests.post(
         "https://api.github.com/repos/john-kurkowski/tldextract/releases",
         headers={
@@ -168,8 +157,15 @@ def create_github_release_draft(token: str, version: str) -> None:
             file=sys.stderr,
         )
         return
+
     print(f'Release created successfully: {response.json()["html_url"]}')
 
+    if not changelog_notes:
+        print(
+            "WARNING: Failed to parse changelog release notes. Manually copy this version's notes from the CHANGELOG.md file to the above URL.",
+            file=sys.stderr,
+        )
+
 
 def upload_build_to_pypi(is_test: str) -> None:
     """Upload the build to PyPI."""
@@ -227,10 +223,12 @@ def main() -> None:
     is_test = get_is_test_response()
     version_number = input("Enter the version number: ")
 
-    add_git_tag_for_version(version_number)
-    remove_previous_dist()
-    create_build()
-    verify_build(is_test)
+    with add_git_tag_for_version(version_number):
+        remove_previous_dist()
+        create_build()
+        verify_build(is_test)
+        upload_build_to_pypi(is_test)
+    push_git_tags()
     create_github_release_draft(github_token, version_number)
 
 

{tldextract-5.1.2 → tldextract-5.1.3}/tests/__snapshots__/test_release.ambr

@@ -72,6 +72,9 @@
               * Misc.
                   * Increase typecheck aggression
               
+              ## New Contributors
+              * @jdoe contributed
+              
               **Full Changelog**: fake-body
             ''',
             'draft': True,

{tldextract-5.1.2 → tldextract-5.1.3}/tests/main_test.py

@@ -4,7 +4,6 @@ from __future__ import annotations
 
 import logging
 import os
-import sys
 import tempfile
 from collections.abc import Sequence
 from pathlib import Path
@@ -168,8 +167,7 @@ def test_looks_like_ipv6() -> None:
     assert looks_like_ipv6("aBcD:ef01:2345:6789:aBcD:ef01:aaaa:2288") is True
     assert looks_like_ipv6("aBcD:ef01:2345:6789:aBcD:ef01:127.0.0.1") is True
     assert looks_like_ipv6("ZBcD:ef01:2345:6789:aBcD:ef01:127.0.0.1") is False
-    if sys.version_info >= (3, 8, 12):  # noqa: UP036
-        assert looks_like_ipv6("aBcD:ef01:2345:6789:aBcD:ef01:127.0.0.01") is False
+    assert looks_like_ipv6("aBcD:ef01:2345:6789:aBcD:ef01:127.0.0.01") is False
     assert looks_like_ipv6("aBcD:ef01:2345:6789:aBcD:") is False
 
 

{tldextract-5.1.2 → tldextract-5.1.3}/tests/test_cache.py

@@ -32,7 +32,7 @@ def test_disk_cache(tmp_path: Path) -> None:
 
 def test_get_pkg_unique_identifier(monkeypatch: pytest.MonkeyPatch) -> None:
     """Test generating a unique identifier for the version of this package."""
-    monkeypatch.setattr(sys, "version_info", (3, 8, 1, "final", 0))
+    monkeypatch.setattr(sys, "version_info", (3, 9, 1, "final", 0))
     monkeypatch.setattr(sys, "prefix", "/home/john/.pyenv/versions/myvirtualenv")
 
     mock_version_module = types.ModuleType("tldextract._version", "mocked module")
@@ -41,13 +41,13 @@ def test_get_pkg_unique_identifier(monkeypatch: pytest.MonkeyPatch) -> None:
 
     assert (
         get_pkg_unique_identifier()
-        == "3.8.1.final__myvirtualenv__f01a7b__tldextract-1.2.3"
+        == "3.9.1.final__myvirtualenv__f01a7b__tldextract-1.2.3"
     )
 
 
 def test_get_cache_dir(monkeypatch: pytest.MonkeyPatch) -> None:
     """Test finding the cache directory."""
-    pkg_identifier = "3.8.1.final__myvirtualenv__f01a7b__tldextract-1.2.3"
+    pkg_identifier = "3.9.1.final__myvirtualenv__f01a7b__tldextract-1.2.3"
     monkeypatch.setattr(
         tldextract.cache, "get_pkg_unique_identifier", lambda: pkg_identifier
     )

{tldextract-5.1.2 → tldextract-5.1.3}/tests/test_parallel.py

@@ -44,22 +44,10 @@ def test_cache_cleared_by_other_process(
     extract("google.com")
     orig_unlink = os.unlink
 
-    def is_relative_to(path: Path, other_path: str | Path) -> bool:
-        """Return True if path is relative to other_path or False.
-
-        Taken from the Python 3.9 standard library.
-        Reference: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.is_relative_to
-        """
-        try:
-            path.relative_to(other_path)
-            return True
-        except ValueError:
-            return False
-
     def evil_unlink(filename: str | Path) -> None:
         """Simulate someone deletes the file right before we try to."""
         if (isinstance(filename, str) and filename.startswith(cache_dir)) or (
-            isinstance(filename, Path) and is_relative_to(filename, cache_dir)
+            isinstance(filename, Path) and filename.is_relative_to(cache_dir)
         ):
             orig_unlink(filename)
         orig_unlink(filename)

{tldextract-5.1.2 → tldextract-5.1.3}/tests/test_release.py

@@ -34,11 +34,12 @@ class Mocks:
 @pytest.fixture
 def mocks() -> Iterator[Mocks]:
     """Stub network and subprocesses."""
-    with mock.patch("builtins.input") as mock_input, mock.patch(
-        "os.listdir"
-    ) as mock_listdir, mock.patch("requests.post") as mock_requests, mock.patch(
-        "subprocess.run"
-    ) as mock_subprocess:
+    with (
+        mock.patch("builtins.input") as mock_input,
+        mock.patch("os.listdir") as mock_listdir,
+        mock.patch("requests.post") as mock_requests,
+        mock.patch("subprocess.run") as mock_subprocess,
+    ):
         yield Mocks(
             input=mock_input,
             listdir=mock_listdir,
@@ -76,7 +77,7 @@ def test_happy_path(
         return mock.Mock(
             json=mock.Mock(
                 return_value={
-                    "body": "Body start **Full Changelog**: fake-body",
+                    "body": "## What's Changed\nGitHub changelog here\n\n## New Contributors\n* @jdoe contributed\n\n**Full Changelog**: fake-body",
                     "html_url": "https://github.com/path/to/release",
                 }
             ),