python-iso639 2024.4.27__tar.gz → 2025.1.27__tar.gz

python_iso639-2025.1.27/CHANGELOG.md

@@ -0,0 +1,102 @@
+# Change Log
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
+
+The version number is based on [calendar versioning](https://calver.org/).
+The specific format is `YYYY.(M)M.(D)D`,
+where the month and day may be single-digit because Python doesn't allow zero-padded
+major/minor/micro version numbers like `05` (it'd have to be just `5`).
+
+## [Unreleased]
+
+### Added
+### Changed
+### Deprecated
+### Removed
+### Fixed
+### Security
+
+## [2025.1.27]
+
+### Changed
+* Updated the ISO 639 data from SIL to the latest 2025-01-15 release.
+* The data tables are now included as the original tab-separated value files
+  instead of a compiled SQLite database.
+
+### Fixed
+* Fixed `Language.from_part3` so that it now also works for a retired ISO 639-3 code.
+
+## [2024.10.22]
+
+### Added
+* Added support for Python 3.13.
+
+### Removed
+* Dropped support for Python 3.8.
+
+### Fixed
+* Added `CHANGELOG.md` to the PyPI source distribution.
+
+## [2024.4.27]
+
+### Changed
+* Updated the ISO 639 data from SIL to the latest 2024-04-15 release.
+
+## [2024.2.7]
+
+### Changed
+* Updated the ISO 639 data from SIL to the latest 2024-01-10 release.
+
+## [2024.1.2]
+
+### Changed
+* Updated the ISO 639 data from SIL to the latest 2023-12-20 release.
+
+## [2023.12.11]
+
+### Added
+* Added support for Python 3.12.
+
+## [2023.6.15]
+
+### Fixed
+* Made `Language` class instances actually hashable (#3).
+
+## [2023.4.13]
+
+### Changed
+* Disabled thread checking for the database, as it's read-only (#1).
+
+## [2023.2.4]
+
+### Changed
+* Updated the ISO 639 data from SIL to the latest 2023-01-23 release.
+
+## [2022.11.27]
+
+### Added
+* Defined the `LanguageNotFoundError` exception.
+* Added support for Python 3.11.
+
+### Changed
+* If the `Language` class methods `match`, `from_part3`, etc. receive an invalid
+  input language code or name, a `LanguageNotFoundError` is now raised.
+  (Previously, `None` was returned with no exception raised.)
+
+### Removed
+* Dropped support for Python 3.7.
+
+## [2022.9.17]
+
+### Added
+* Added constants `DATA_LAST_UPDATED` and `ALL_LANGUAGES`.
+
+## [2022.5.16]
+
+### Fixed
+* Corrected repo name in package metadata and documentation.
+
+## [2022.5.15]
+
+* First release!

python_iso639-2025.1.27/MANIFEST.in

@@ -0,0 +1 @@
+include CHANGELOG.md

{python_iso639-2024.4.27 → python_iso639-2025.1.27}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.2
 Name: python-iso639
-Version: 2024.4.27
+Version: 2025.1.27
 Summary: ISO 639 language codes, names, and other associated information
 Author-email: "Jackson L. Lee" <jacksonlunlee@gmail.com>
 License: Apache 2.0
@@ -15,25 +15,26 @@ Classifier: License :: OSI Approved :: Apache Software License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3 :: Only
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Text Processing
 Classifier: Topic :: Text Processing :: General
 Classifier: Topic :: Text Processing :: Indexing
 Classifier: Topic :: Text Processing :: Linguistic
-Requires-Python: >=3.8
+Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
 Provides-Extra: dev
-Requires-Dist: black==24.4.2; extra == "dev"
-Requires-Dist: build==1.2.1; extra == "dev"
-Requires-Dist: flake8==7.0.0; extra == "dev"
-Requires-Dist: pytest==8.1.2; extra == "dev"
-Requires-Dist: requests==2.31.0; extra == "dev"
-Requires-Dist: twine==5.0.0; extra == "dev"
+Requires-Dist: black==24.10.0; extra == "dev"
+Requires-Dist: build==1.2.2; extra == "dev"
+Requires-Dist: flake8==7.1.1; extra == "dev"
+Requires-Dist: mypy==1.14.1; extra == "dev"
+Requires-Dist: pytest==8.3.4; extra == "dev"
+Requires-Dist: requests==2.32.3; extra == "dev"
+Requires-Dist: twine==6.1.0; extra == "dev"
 
 # python-iso639
 
@@ -68,7 +69,7 @@ the actual import name during runtime is `iso639`
 
 ### Creating `Language` Instances
 
-Create a `Language` instance by one of the methods.
+Create a `Language` instance by one of the class methods.
 
 #### `from_part3`, with an ISO 639-3 code
 
@@ -276,7 +277,7 @@ e.g., [`dataclasses.asdict`](https://docs.python.org/3/library/dataclasses.html#
     ```python
     >>> import iso639
     >>> iso639.DATA_LAST_UPDATED
-    datetime.date(2024, 4, 15)
+    datetime.date(2025, 1, 15)
     ```
 
 * `ALL_LANGUAGES`: The list of all `Language` objects based on the included language code data
@@ -284,11 +285,9 @@ e.g., [`dataclasses.asdict`](https://docs.python.org/3/library/dataclasses.html#
     ```python
     >>> import iso639
     >>> type(iso639.ALL_LANGUAGES)
-    <class 'list'>
+    <class 'set'>
     >>> len(iso639.ALL_LANGUAGES)
-    7920
-    >>> iso639.ALL_LANGUAGES[0]
-    Language(part3='aaa', scope='I', type='L', status='A', name='Ghotuo', ...)
+    8307
     ```
 
 ## Links
@@ -304,9 +303,12 @@ for details.
 
 The data source that backs this package is the
 [language code tables published by SIL](https://iso639-3.sil.org/code_tables/download_tables).
+The tables are included in this package under [`src/iso639/_data/`](src/iso639/_data/).
+They are the UTF8-encoded `*.tab` tab-separated files bundled as a ZIP archive file,
+typically found at a URL that looks like
+`https://iso639-3.sil.org/sites/iso639-3/files/downloads/iso-639-3_Code_Tables_YYYYMMDD.zip`
+(replace `YYYYMMDD` with the data release date).
 Note that SIL resources have their [terms of use](https://www.sil.org/terms-use).
-For exactly how the data is included in the package,
-see the script at [`scripts/create_languages_db.py`](scripts/create_languages_db.py).
 
 ## Why Another ISO 639 Package?
 

{python_iso639-2024.4.27 → python_iso639-2025.1.27}/README.md

@@ -31,7 +31,7 @@ the actual import name during runtime is `iso639`
 
 ### Creating `Language` Instances
 
-Create a `Language` instance by one of the methods.
+Create a `Language` instance by one of the class methods.
 
 #### `from_part3`, with an ISO 639-3 code
 
@@ -239,7 +239,7 @@ e.g., [`dataclasses.asdict`](https://docs.python.org/3/library/dataclasses.html#
     ```python
     >>> import iso639
     >>> iso639.DATA_LAST_UPDATED
-    datetime.date(2024, 4, 15)
+    datetime.date(2025, 1, 15)
     ```
 
 * `ALL_LANGUAGES`: The list of all `Language` objects based on the included language code data
@@ -247,11 +247,9 @@ e.g., [`dataclasses.asdict`](https://docs.python.org/3/library/dataclasses.html#
     ```python
     >>> import iso639
     >>> type(iso639.ALL_LANGUAGES)
-    <class 'list'>
+    <class 'set'>
     >>> len(iso639.ALL_LANGUAGES)
-    7920
-    >>> iso639.ALL_LANGUAGES[0]
-    Language(part3='aaa', scope='I', type='L', status='A', name='Ghotuo', ...)
+    8307
     ```
 
 ## Links
@@ -267,9 +265,12 @@ for details.
 
 The data source that backs this package is the
 [language code tables published by SIL](https://iso639-3.sil.org/code_tables/download_tables).
+The tables are included in this package under [`src/iso639/_data/`](src/iso639/_data/).
+They are the UTF8-encoded `*.tab` tab-separated files bundled as a ZIP archive file,
+typically found at a URL that looks like
+`https://iso639-3.sil.org/sites/iso639-3/files/downloads/iso-639-3_Code_Tables_YYYYMMDD.zip`
+(replace `YYYYMMDD` with the data release date).
 Note that SIL resources have their [terms of use](https://www.sil.org/terms-use).
-For exactly how the data is included in the package,
-see the script at [`scripts/create_languages_db.py`](scripts/create_languages_db.py).
 
 ## Why Another ISO 639 Package?
 

{python_iso639-2024.4.27 → python_iso639-2025.1.27}/pyproject.toml

@@ -4,10 +4,10 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "python-iso639"
-version = "2024.4.27"
+version = "2025.1.27"
 description = "ISO 639 language codes, names, and other associated information"
 readme = "README.md"
-requires-python = ">= 3.8"
+requires-python = ">= 3.9"
 license = { text = "Apache 2.0" }
 authors = [ { name = "Jackson L. Lee", email = "jacksonlunlee@gmail.com" } ]
 keywords = ["ISO 639", "language codes", "languages", "linguistics"]
@@ -21,11 +21,11 @@ classifiers = [
     "Operating System :: OS Independent",
     "Programming Language :: Python :: 3",
     "Programming Language :: Python :: 3 :: Only",
-    "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
     "Topic :: Text Processing",
     "Topic :: Text Processing :: General",
     "Topic :: Text Processing :: Indexing",
@@ -34,12 +34,13 @@ classifiers = [
 
 [project.optional-dependencies]
 dev = [
-    "black == 24.4.2",
-    "build == 1.2.1",
-    "flake8 == 7.0.0",
-    "pytest == 8.1.2",
-    "requests == 2.31.0",
-    "twine == 5.0.0",
+    "black == 24.10.0",
+    "build == 1.2.2",
+    "flake8 == 7.1.1",
+    "mypy == 1.14.1",
+    "pytest == 8.3.4",
+    "requests == 2.32.3",
+    "twine == 6.1.0",
 ]
 
 [project.urls]
@@ -51,7 +52,7 @@ where = [ "src" ]
 [tool.setuptools]
 zip-safe = false
 package-dir = { "" = "src" }
-package-data = { "iso639" = ["languages.db"] }
+package-data = { "iso639" = ["_data/*.tab"] }
 
 [tool.pytest.ini_options]
 addopts = "-vv --durations=0 --strict-markers"

{python_iso639-2024.4.27 → python_iso639-2025.1.27}/src/iso639/__init__.py

@@ -4,16 +4,12 @@ from importlib.metadata import version
 from .language import Language, LanguageNotFoundError, _get_all_languages
 
 
+# __version__ is based on calendar versioning (https://calver.org/).
+# __version__ and DATA_LAST_UPDATED are intentionally kept separate.
+# While DATA_LAST_UPDATED is strictly the date for the ISO 639-3 data release,
+# __version__ can be bumped for changes other than data updates.
 __version__ = version("python-iso639")
-__all__ = [
-    "__version__",
-    "ALL_LANGUAGES",
-    "DATA_LAST_UPDATED",
-    "Language",
-    "LanguageNotFoundError",
-]
-
-DATA_LAST_UPDATED = datetime.date(2024, 4, 15)
+DATA_LAST_UPDATED = datetime.date(2025, 1, 15)
 
 
 def __getattr__(name):
@@ -21,3 +17,12 @@ def __getattr__(name):
         return _get_all_languages()
     else:
         raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+__all__ = [
+    "__version__",
+    "ALL_LANGUAGES",
+    "DATA_LAST_UPDATED",
+    "Language",
+    "LanguageNotFoundError",
+]

python_iso639-2025.1.27/src/iso639/_data/__init__.py

@@ -0,0 +1,119 @@
+from __future__ import annotations
+
+import csv
+import os
+from collections import defaultdict
+from enum import Enum
+
+from typing import Dict, List, Union
+
+
+class _CodesColumn(Enum):
+    ID = "Id"
+    PART2B = "Part2b"
+    PART2T = "Part2t"
+    PART1 = "Part1"
+    SCOPE = "Scope"
+    TYPE = "Language_Type"
+    REF_NAME = "Ref_Name"
+    COMMENT = "Comment"
+
+
+class _NameIndexColumn(Enum):
+    ID = "Id"
+    PRINT_NAME = "Print_Name"
+    INVERTED_NAME = "Inverted_Name"
+
+
+class _RetirementsColumn(Enum):
+    ID = "Id"
+    REF_NAME = "Ref_Name"
+    RET_REASON = "Ret_Reason"
+    CHANGE_TO = "Change_To"
+    REMEDY = "Ret_Remedy"
+    EFFECTIVE = "Effective"
+
+
+class _MacrolanguagesColumn(Enum):
+    ID = "I_Id"
+    MID = "M_Id"
+    STATUS = "I_Status"
+
+
+_COLUMN_TYPE = Union[
+    _CodesColumn, _NameIndexColumn, _RetirementsColumn, _MacrolanguagesColumn
+]
+_ROW_TYPE = Dict[_COLUMN_TYPE, str]
+
+
+class _Table(Enum):
+    CODES = _CodesColumn
+    NAME_INDEX = _NameIndexColumn
+    RETIREMENTS = _RetirementsColumn
+    MACROLANGUAGES = _MacrolanguagesColumn
+
+
+_THIS_DIR = os.path.dirname(__file__)
+_DATA_TSV_PATHS = {
+    _Table.CODES: os.path.join(_THIS_DIR, "iso-639-3.tab"),
+    _Table.NAME_INDEX: os.path.join(_THIS_DIR, "iso-639-3_Name_Index.tab"),
+    _Table.MACROLANGUAGES: os.path.join(_THIS_DIR, "iso-639-3-macrolanguages.tab"),
+    _Table.RETIREMENTS: os.path.join(_THIS_DIR, "iso-639-3_Retirements.tab"),
+}
+
+
+def _load_tsv(table: _Table) -> List[_ROW_TYPE]:
+    column = table.value
+    with open(_DATA_TSV_PATHS[table], encoding="utf-8", newline="") as tsv_file:
+        tsv_reader = csv.DictReader(tsv_file, delimiter="\t")
+        return [{column(k): v for k, v in row.items()} for row in tsv_reader]
+
+
+_PART3_TO_CODES: Dict[str, _ROW_TYPE] = {
+    row[_CodesColumn.ID]: row for row in _load_tsv(_Table.CODES)
+}
+
+_part3_to_name_index: defaultdict[str, List[_ROW_TYPE]] = defaultdict(list)
+for row in _load_tsv(_Table.NAME_INDEX):
+    _part3_to_name_index[row[_NameIndexColumn.ID]].append(row)
+_PART3_TO_NAME_INDEX: Dict[str, List[_ROW_TYPE]] = dict(_part3_to_name_index)
+
+_PART3_TO_RETIREMENTS: Dict[str, _ROW_TYPE] = {
+    row[_RetirementsColumn.ID]: row for row in _load_tsv(_Table.RETIREMENTS)
+}
+_PART3_TO_MACROLANGUAGES: Dict[str, _ROW_TYPE] = {
+    row[_MacrolanguagesColumn.ID]: row for row in _load_tsv(_Table.MACROLANGUAGES)
+}
+
+_PART2B_TO_PART3: Dict[str, str] = {
+    row[_CodesColumn.PART2B]: part3
+    for part3, row in _PART3_TO_CODES.items()
+    if row[_CodesColumn.PART2B]
+}
+_PART2T_TO_PART3: Dict[str, str] = {
+    row[_CodesColumn.PART2T]: part3
+    for part3, row in _PART3_TO_CODES.items()
+    if row[_CodesColumn.PART2T]
+}
+_PART1_TO_PART3: Dict[str, str] = {
+    row[_CodesColumn.PART1]: part3
+    for part3, row in _PART3_TO_CODES.items()
+    if row[_CodesColumn.PART1]
+}
+_REF_NAME_TO_PART3: Dict[str, str] = {
+    row[_CodesColumn.REF_NAME]: part3
+    for part3, row in _PART3_TO_CODES.items()
+    if row[_CodesColumn.REF_NAME]
+}
+_PRINT_NAME_TO_PART3: Dict[str, str] = {
+    row[_NameIndexColumn.PRINT_NAME]: part3
+    for part3, rows in _PART3_TO_NAME_INDEX.items()
+    for row in rows
+    if row[_NameIndexColumn.PRINT_NAME]
+}
+_INVERTED_NAME_TO_PART3: Dict[str, str] = {
+    row[_NameIndexColumn.INVERTED_NAME]: part3
+    for part3, rows in _PART3_TO_NAME_INDEX.items()
+    for row in rows
+    if row[_NameIndexColumn.INVERTED_NAME]
+}