tldextract 5.1.1__tar.gz → 5.1.3__tar.gz

{tldextract-5.1.1 → tldextract-5.1.3}/.github/workflows/ci.yml

@@ -1,5 +1,11 @@
 name: build
-on: [push, pull_request]
+on:
+  pull_request: {}
+  push:
+    branches:
+      - "master"
+    tags-ignore:
+      - "**"
 jobs:
   test:
     strategy:
@@ -8,32 +14,33 @@ jobs:
         os: [macos-latest, windows-latest, ubuntu-latest]
         language:
           [
-            {python-version: "3.8", toxenv: "py38"},
             {python-version: "3.9", toxenv: "py39"},
             {python-version: "3.10", toxenv: "py310"},
             {python-version: "3.11", toxenv: "py311"},
             {python-version: "3.12", toxenv: "py312"},
-            {python-version: "pypy3.8", toxenv: "pypy38"},
+            {python-version: "3.13", toxenv: "py313"},
+            {python-version: "pypy3.9", toxenv: "pypy39"},
+            {python-version: "pypy3.10", toxenv: "pypy310"},
           ]
         include:
           - os: ubuntu-latest
-            language: {python-version: "3.8", toxenv: "codestyle"}
+            language: {python-version: "3.9", toxenv: "codestyle"}
           - os: ubuntu-latest
-            language: {python-version: "3.8", toxenv: "lint"}
+            language: {python-version: "3.9", toxenv: "lint"}
           - os: ubuntu-latest
-            language: {python-version: "3.8", toxenv: "typecheck"}
+            language: {python-version: "3.9", toxenv: "typecheck"}
     runs-on: ${{ matrix.os }}
     steps:
       - name: Check out repository
         uses: actions/checkout@v4
       - name: Setup Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.language.python-version }}
+          check-latest: true
       - name: Install Python requirements
         run: |
-          pip install --upgrade pip
-          pip install --upgrade --editable '.[testing]'
+          pip install --upgrade tox tox-uv
       - name: Test
         run: tox
         env:

{tldextract-5.1.1 → tldextract-5.1.3}/CHANGELOG.md

@@ -3,6 +3,30 @@
 After upgrading, update your cache file by deleting it or via `tldextract
 --update`.
 
+## 5.1.3 (2024-11-04)
+
+* Bugfixes
+  * Reduce logging errors ([`921a825`](https://github.com/john-kurkowski/tldextract/commit/921a82523c0e4403d21d50b2c3410d9af43520ac))
+  * Drop support for EOL Python 3.8 ([#340](https://github.com/john-kurkowski/tldextract/issues/340))
+  * Support Python 3.13 ([#341](https://github.com/john-kurkowski/tldextract/issues/341))
+  * Update bundled snapshot
+* Documentation
+  * Clarify how to use your own definitions
+  * Clarify first-successful definitions vs. merged definitions
+* Misc.
+  * Switch from Black to Ruff ([#333](https://github.com/john-kurkowski/tldextract/issues/333))
+  * Switch from pip to uv, during tox ([#324](https://github.com/john-kurkowski/tldextract/issues/324))
+
+## 5.1.2 (2024-03-18)
+
+* Bugfixes
+  * Remove `socket.inet_pton`, to fix platform-dependent IP parsing ([#318](https://github.com/john-kurkowski/tldextract/issues/318))
+  * Use non-capturing groups for IPv4 address detection, for a slight speed boost ([#323](https://github.com/john-kurkowski/tldextract/issues/323))
+* Misc.
+  * Add CI for PyPy3.9 and PyPy3.10 ([#316](https://github.com/john-kurkowski/tldextract/issues/316))
+  * Add script to automate package release process ([#325](https://github.com/john-kurkowski/tldextract/issues/325))
+  * Update LICENSE copyright years
+
 ## 5.1.1 (2023-11-16)
 
 * Bugfixes

{tldextract-5.1.1 → tldextract-5.1.3}/LICENSE

@@ -1,6 +1,6 @@
 BSD 3-Clause License
 
-Copyright (c) 2020, John Kurkowski
+Copyright (c) 2013-2024, John Kurkowski
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

{tldextract-5.1.1 → tldextract-5.1.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: tldextract
-Version: 5.1.1
+Version: 5.1.3
 Summary: Accurately separates a URL's subdomain, domain, and public suffix, using the Public Suffix List (PSL). By default, this includes the public ICANN TLDs and their exceptions. You can optionally support the Public Suffix List's private domains as well.
 Author-email: John Kurkowski <john.kurkowski@gmail.com>
 License: BSD-3-Clause
@@ -10,27 +10,31 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Topic :: Utilities
 Classifier: License :: OSI Approved :: BSD License
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
-Requires-Python: >=3.8
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: idna
 Requires-Dist: requests>=2.1.0
 Requires-Dist: requests-file>=1.4
 Requires-Dist: filelock>=3.0.8
+Provides-Extra: release
+Requires-Dist: build; extra == "release"
+Requires-Dist: twine; extra == "release"
 Provides-Extra: testing
-Requires-Dist: black; extra == "testing"
 Requires-Dist: mypy; extra == "testing"
 Requires-Dist: pytest; extra == "testing"
 Requires-Dist: pytest-gitignore; extra == "testing"
 Requires-Dist: pytest-mock; extra == "testing"
 Requires-Dist: responses; extra == "testing"
 Requires-Dist: ruff; extra == "testing"
+Requires-Dist: syrupy; extra == "testing"
 Requires-Dist: tox; extra == "testing"
+Requires-Dist: tox-uv; extra == "testing"
 Requires-Dist: types-filelock; extra == "testing"
 Requires-Dist: types-requests; extra == "testing"
 
@@ -125,8 +129,8 @@ tldextract http://forums.bbc.co.uk
 
 Beware when first calling `tldextract`, it updates its TLD list with a live HTTP
 request. This updated TLD set is usually cached indefinitely in `$HOME/.cache/python-tldextract`.
-To control the cache's location, set TLDEXTRACT_CACHE environment variable or set the
-cache_dir path in TLDExtract initialization.
+To control the cache's location, set the `TLDEXTRACT_CACHE` environment variable or set the
+`cache_dir` path when constructing a `TLDExtract`.
 
 (Arguably runtime bootstrapping like that shouldn't be the default behavior,
 like for production systems. But I want you to have the latest TLDs, especially
@@ -215,10 +219,12 @@ extract = tldextract.TLDExtract(
     fallback_to_snapshot=False)
 ```
 
-The above snippet will fetch from the URL *you* specified, upon first need to download the
-suffix list (i.e. if the cached version doesn't exist).
+If the cached version of public suffix definitions doesn't exist, such as on
+the first run, the above snippet will request the URLs you specified in order,
+and use the first successful response.
 
-If you want to use input data from your local filesystem, just use the `file://` protocol:
+If you want to use input data from your local filesystem, use the `file://`
+protocol with an absolute path:
 
 ```python
 extract = tldextract.TLDExtract(
@@ -227,17 +233,24 @@ extract = tldextract.TLDExtract(
     fallback_to_snapshot=False)
 ```
 
-Use an absolute path when specifying the `suffix_list_urls` keyword argument.
-`os.path` is your friend.
-
-The command line update command can be used with a URL or local file you specify:
+This also works via command line update:
 
 ```zsh
 tldextract --update --suffix_list_url "http://foo.bar.baz"
 ```
 
-This could be useful in production when you don't want the delay associated with updating the suffix
-list on first use, or if you are behind a complex firewall that prevents a simple update from working.
+Using your own URLs could be useful in production when you don't want the delay
+with updating the suffix list on first use, or if you are behind a complex
+firewall.
+
+You can also specify additional suffixes in the `extra_suffixes` param. These
+will be merged into whatever public suffix definitions are already in use by
+`tldextract`.
+
+```python
+extract = tldextract.TLDExtract(
+    extra_suffixes=["foo", "bar", "baz"])
+```
 
 ## FAQ
 
@@ -246,9 +259,9 @@ list on first use, or if you are behind a complex firewall that prevents a simpl
 This project doesn't contain an actual list of public suffixes. That comes from
 [the Public Suffix List (PSL)](https://publicsuffix.org/). Submit amendments there.
 
-(In the meantime, you can tell tldextract about your exception by either
+In the meantime, you can tell tldextract about your exception by either
 forking the PSL and using your fork in the `suffix_list_urls` param, or adding
-your suffix piecemeal with the `extra_suffixes` param.)
+your suffix piecemeal with the `extra_suffixes` param.
 
 ### I see my suffix in [the Public Suffix List (PSL)](https://publicsuffix.org/), but this library doesn't extract it.
 
@@ -305,5 +318,5 @@ tox -e py311
 Automatically format all code:
 
 ```zsh
-black .
+ruff format .
 ```

{tldextract-5.1.1 → tldextract-5.1.3}/README.md

@@ -89,8 +89,8 @@ tldextract http://forums.bbc.co.uk
 
 Beware when first calling `tldextract`, it updates its TLD list with a live HTTP
 request. This updated TLD set is usually cached indefinitely in `$HOME/.cache/python-tldextract`.
-To control the cache's location, set TLDEXTRACT_CACHE environment variable or set the
-cache_dir path in TLDExtract initialization.
+To control the cache's location, set the `TLDEXTRACT_CACHE` environment variable or set the
+`cache_dir` path when constructing a `TLDExtract`.
 
 (Arguably runtime bootstrapping like that shouldn't be the default behavior,
 like for production systems. But I want you to have the latest TLDs, especially
@@ -179,10 +179,12 @@ extract = tldextract.TLDExtract(
     fallback_to_snapshot=False)
 ```
 
-The above snippet will fetch from the URL *you* specified, upon first need to download the
-suffix list (i.e. if the cached version doesn't exist).
+If the cached version of public suffix definitions doesn't exist, such as on
+the first run, the above snippet will request the URLs you specified in order,
+and use the first successful response.
 
-If you want to use input data from your local filesystem, just use the `file://` protocol:
+If you want to use input data from your local filesystem, use the `file://`
+protocol with an absolute path:
 
 ```python
 extract = tldextract.TLDExtract(
@@ -191,17 +193,24 @@ extract = tldextract.TLDExtract(
     fallback_to_snapshot=False)
 ```
 
-Use an absolute path when specifying the `suffix_list_urls` keyword argument.
-`os.path` is your friend.
-
-The command line update command can be used with a URL or local file you specify:
+This also works via command line update:
 
 ```zsh
 tldextract --update --suffix_list_url "http://foo.bar.baz"
 ```
 
-This could be useful in production when you don't want the delay associated with updating the suffix
-list on first use, or if you are behind a complex firewall that prevents a simple update from working.
+Using your own URLs could be useful in production when you don't want the delay
+with updating the suffix list on first use, or if you are behind a complex
+firewall.
+
+You can also specify additional suffixes in the `extra_suffixes` param. These
+will be merged into whatever public suffix definitions are already in use by
+`tldextract`.
+
+```python
+extract = tldextract.TLDExtract(
+    extra_suffixes=["foo", "bar", "baz"])
+```
 
 ## FAQ
 
@@ -210,9 +219,9 @@ list on first use, or if you are behind a complex firewall that prevents a simpl
 This project doesn't contain an actual list of public suffixes. That comes from
 [the Public Suffix List (PSL)](https://publicsuffix.org/). Submit amendments there.
 
-(In the meantime, you can tell tldextract about your exception by either
+In the meantime, you can tell tldextract about your exception by either
 forking the PSL and using your fork in the `suffix_list_urls` param, or adding
-your suffix piecemeal with the `extra_suffixes` param.)
+your suffix piecemeal with the `extra_suffixes` param.
 
 ### I see my suffix in [the Public Suffix List (PSL)](https://publicsuffix.org/), but this library doesn't extract it.
 
@@ -269,5 +278,5 @@ tox -e py311
 Automatically format all code:
 
 ```zsh
-black .
+ruff format .
 ```

{tldextract-5.1.1 → tldextract-5.1.3}/pyproject.toml

@@ -23,13 +23,13 @@ classifiers = [
     "Topic :: Utilities",
     "License :: OSI Approved :: BSD License",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
 ]
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 dynamic = ["version"]
 readme = "README.md"
 
@@ -41,15 +41,20 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
+release = [
+    "build",
+    "twine",
+]
 testing = [
-    "black",
     "mypy",
     "pytest",
     "pytest-gitignore",
     "pytest-mock",
     "responses",
     "ruff",
+    "syrupy",
     "tox",
+    "tox-uv",
     "types-filelock",
     "types-requests",
 ]
@@ -79,12 +84,13 @@ write_to = "tldextract/_version.py"
 version = {attr = "setuptools_scm.get_version"}
 
 [tool.mypy]
+explicit_package_bases = true
 strict = true
 
 [tool.pytest.ini_options]
 addopts = "--doctest-modules"
 
-[tool.ruff]
+[tool.ruff.lint]
 select = [
   "A",
   "B",
@@ -98,8 +104,8 @@ select = [
   "W",
 ]
 ignore = [
-  "E501", # line too long; if Black does its job, not worried about the rare long line
+  "E501", # line too long; if formatter does its job, not worried about the rare long line
 ]
 
-[tool.ruff.pydocstyle]
+[tool.ruff.lint.pydocstyle]
 convention = "pep257"

tldextract-5.1.3/scripts/release.py

@@ -0,0 +1,236 @@
+"""
+This script automates the release process for a Python package.
+
+It will:
+- Add a git tag for the given version.
+- Remove the previous dist folder.
+- Create a build.
+- Ask the user to verify the build.
+- Upload the build to PyPI.
+- Push all git tags to the remote.
+- Create a draft release on GitHub using the version notes in CHANGELOG.md.
+
+Prerequisites:
+    - This must be run from the root of the repository.
+    - The repo must have a clean git working tree.
+    - The user must have the GITHUB_TOKEN environment variable set to a GitHub personal access token with repository "Contents" read and write permission.
+    - The user will need credentials for the PyPI repository, which the user will be prompted for during the upload step. The user will need to paste the token manually from a password manager or similar.
+    - The CHANGELOG.md file must already contain an entry for the version being released.
+    - Install requirements with: pip install --upgrade --editable '.[release]'
+
+"""
+
+from __future__ import annotations
+
+import contextlib
+import os
+import re
+import subprocess
+import sys
+from collections.abc import Iterator
+from pathlib import Path
+
+import requests
+
+
+@contextlib.contextmanager
+def add_git_tag_for_version(version: str) -> Iterator[None]:
+    """Add a git tag for the given version."""
+    subprocess.run(["git", "tag", "-a", version, "-m", version], check=True)
+    print(f"Version {version} tag added successfully.")
+    try:
+        yield
+    except:
+        subprocess.run(["git", "tag", "-d", version])
+        raise
+
+
+def remove_previous_dist() -> None:
+    """Check for dist folder, and if it exists, remove it."""
+    subprocess.run(["rm", "-rf", Path("dist")], check=True)
+    print("Previous dist folder removed successfully.")
+
+
+def create_build() -> None:
+    """Create a build."""
+    subprocess.run(["python", "-m", "build"], check=True)
+    print("Build created successfully.")
+
+
+def verify_build(is_test: str) -> None:
+    """Verify the build.
+
+    Print the archives in dist/ and ask the user to manually inspect and
+    confirm they contain the expected files, e.g. source files and test files.
+    """
+    build_files = os.listdir("dist")
+    if len(build_files) != 2:
+        print(
+            "WARNING: dist folder contains incorrect number of files.", file=sys.stderr
+        )
+    print("Contents of dist folder:")
+    subprocess.run(["ls", "-l", Path("dist")], check=True)
+    print("Contents of tar files in dist folder:")
+    for build_file in build_files:
+        subprocess.run(["tar", "tvf", Path("dist") / build_file], check=True)
+    confirmation = input("Does the build look correct? (y/n): ")
+    if confirmation == "y":
+        print("Build verified successfully.")
+    else:
+        raise Exception("Could not verify. Build was not uploaded.")
+
+
+def generate_github_release_notes_body(token: str, version: str) -> str:
+    """Generate and grab release notes URL from Github.
+
+    Delete their first paragraph, because we track its contents in a tighter
+    form in CHANGELOG.md. See `get_changelog_release_notes`.
+    """
+    response = requests.post(
+        "https://api.github.com/repos/john-kurkowski/tldextract/releases/generate-notes",
+        headers={
+            "Accept": "application/vnd.github+json",
+            "Authorization": f"Bearer {token}",
+            "X-GitHub-Api-Version": "2022-11-28",
+        },
+        json={"tag_name": version},
+    )
+
+    try:
+        response.raise_for_status()
+    except requests.exceptions.HTTPError as err:
+        print(
+            f"WARNING: Failed to generate release notes from Github: {err}",
+            file=sys.stderr,
+        )
+        return ""
+
+    body = str(response.json()["body"])
+    paragraphs = body.split("\n\n")
+    return "\n\n".join(paragraphs[1:])
+
+
+def get_changelog_release_notes(version: str) -> str:
+    """Get the changelog release notes.
+
+    Uses a regex starting on a heading beginning with the version number
+    literal, and matching until the next heading. Using regex to match markup
+    is brittle. Consider a Markdown-parsing library instead.
+    """
+    with open("CHANGELOG.md") as file:
+        changelog_text = file.read()
+    pattern = re.compile(rf"## {re.escape(version)}[^\n]*(.*?)## ", re.DOTALL)
+    match = pattern.search(changelog_text)
+    if match:
+        return str(match.group(1)).strip()
+    else:
+        return ""
+
+
+def create_github_release_draft(token: str, version: str) -> None:
+    """Create a release on GitHub."""
+    github_release_body = generate_github_release_notes_body(token, version)
+    changelog_notes = get_changelog_release_notes(version)
+    release_body = f"{changelog_notes}\n\n{github_release_body}"
+
+    response = requests.post(
+        "https://api.github.com/repos/john-kurkowski/tldextract/releases",
+        headers={
+            "Accept": "application/vnd.github+json",
+            "Authorization": f"Bearer {token}",
+            "X-GitHub-Api-Version": "2022-11-28",
+        },
+        json={
+            "tag_name": version,
+            "name": version,
+            "body": release_body,
+            "draft": True,
+            "prerelease": False,
+        },
+    )
+
+    try:
+        response.raise_for_status()
+    except requests.exceptions.HTTPError as err:
+        print(
+            f"WARNING: Failed to create release on Github: {err}",
+            file=sys.stderr,
+        )
+        return
+
+    print(f'Release created successfully: {response.json()["html_url"]}')
+
+    if not changelog_notes:
+        print(
+            "WARNING: Failed to parse changelog release notes. Manually copy this version's notes from the CHANGELOG.md file to the above URL.",
+            file=sys.stderr,
+        )
+
+
+def upload_build_to_pypi(is_test: str) -> None:
+    """Upload the build to PyPI."""
+    repository: list[str | Path] = (
+        [] if is_test == "n" else ["--repository", "testpypi"]
+    )
+    upload_command = ["twine", "upload", *repository, Path("dist") / "*"]
+    subprocess.run(
+        upload_command,
+        check=True,
+    )
+
+
+def push_git_tags() -> None:
+    """Push all git tags to the remote."""
+    subprocess.run(["git", "push", "--tags", "origin", "master"], check=True)
+
+
+def check_for_clean_working_tree() -> None:
+    """Check for a clean git working tree."""
+    git_status = subprocess.run(
+        ["git", "status", "--porcelain"], capture_output=True, text=True
+    )
+    if git_status.stdout:
+        print(
+            "Git working tree is not clean. Please commit or stash changes.",
+            file=sys.stderr,
+        )
+        sys.exit(1)
+
+
+def get_env_github_token() -> str:
+    """Check for the GITHUB_TOKEN environment variable."""
+    github_token = os.environ.get("GITHUB_TOKEN")
+    if not github_token:
+        print("GITHUB_TOKEN environment variable not set.", file=sys.stderr)
+        sys.exit(1)
+    return github_token
+
+
+def get_is_test_response() -> str:
+    """Ask the user if this is a test release."""
+    while True:
+        is_test = input("Is this a test release? (y/n): ")
+        if is_test in ["y", "n"]:
+            return is_test
+        else:
+            print("Invalid input. Please enter 'y' or 'n.'")
+
+
+def main() -> None:
+    """Run the main program."""
+    check_for_clean_working_tree()
+    github_token = get_env_github_token()
+    is_test = get_is_test_response()
+    version_number = input("Enter the version number: ")
+
+    with add_git_tag_for_version(version_number):
+        remove_previous_dist()
+        create_build()
+        verify_build(is_test)
+        upload_build_to_pypi(is_test)
+    push_git_tags()
+    create_github_release_draft(github_token, version_number)
+
+
+if __name__ == "__main__":
+    main()