iso-langcodes 1.0.0__tar.gz

iso_langcodes-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 Joerg Tiedemann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

iso_langcodes-1.0.0/PKG-INFO

@@ -0,0 +1,167 @@
+Metadata-Version: 2.4
+Name: iso-langcodes
+Version: 1.0.0
+Summary: Python modules for working with language codes, language groups and language scripts
+Author-email: Joerg Tiedemann <tiedemann@cpan.org>
+License: MIT
+Project-URL: Homepage, https://github.com/tiedemann/LanguageCodes
+Project-URL: Repository, https://github.com/tiedemann/LanguageCodes
+Keywords: language,codes,ISO,639,15924,linguistics
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# iso-langcodes
+
+Python modules for working with language codes, language groups and language scripts.
+
+Currently, there are 3 modules in this package:
+
+- `iso_langcodes.iso639_3` - conversion between ISO language codes
+- `iso_langcodes.iso639_5` - language groups and families according to ISO 639-5
+- `iso_langcodes.iso15924` - language scripts and their connection to languages
+
+## Installation
+
+```bash
+pip install iso-langcodes
+```
+
+Or install from source:
+
+```bash
+pip install .
+```
+
+## Usage
+
+### ISO 639-3
+
+```python
+from iso_langcodes import iso639_3
+
+print(iso639_3.convert_iso639('iso639-1', 'fra'))
+print(iso639_3.convert_iso639('iso639-3', 'de'))
+print(iso639_3.convert_iso639('name', 'fa'))
+
+print(iso639_3.get_iso639_1('deu'))
+print(iso639_3.get_iso639_3('de'))
+print(iso639_3.get_language_name('de'))
+print(iso639_3.get_language_name('eng'))
+print(iso639_3.get_macro_language('yue'))
+```
+
+### ISO 639-5
+
+```python
+from iso_langcodes import iso639_5
+
+print(' '.join(iso639_5.language_group('gmw')))
+print(' '.join(iso639_5.language_group('gem')))
+print(' '.join(iso639_5.language_group_children('gem')))
+print(iso639_5.language_parent('afr'))
+print(iso639_5.language_parent('gmw'))
+```
+
+### ISO 15924
+
+```python
+from iso_langcodes import iso15924
+
+script = iso15924.script_of_string('То је моја мачка.', 'srp')
+script = iso15924.script_of_string('你是我妈妈。')
+scripts = iso15924.script_of_string('То је моја мачка. Ti si moja majka.', allow_non_standard=True)
+
+region = iso15924.default_territory('por')
+script = iso15924.default_script('tur')
+```
+
+## Command Line Tools
+
+The package provides three command-line tools:
+
+### iso639
+
+Convert between language codes:
+
+```bash
+iso639 fra de eng
+iso639 -2 fra deu
+iso639 -3 de
+iso639 -m yue
+```
+
+### langgroup
+
+Print language groups according to ISO 639-5:
+
+```bash
+langgroup gem
+langgroup -c gem
+langgroup -p afr gmw
+```
+
+### langscript
+
+Detect characters from various scripts:
+
+```bash
+echo "То је моја мачка." | langscript -l srp
+echo "你是我妈妈。" | langscript
+```
+
+### langglottolog
+
+Convert between ISO 639-5 codes and Glottolog codes:
+
+```bash
+langglottolog aav afa alg
+langglottolog -g aust1305 afro1255
+```
+
+## License
+
+This software is Copyright (c) 2020 by Joerg Tiedemann.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+## Acknowledgements
+
+The language codes are taken from SIL International <https://iso639-3.sil.org>.
+Please, check the terms of use listed at <https://iso639-3.sil.org/code_tables/download_tables>.
+The current version uses the UTF-8 tables distributed in "iso-639-3_Code_Tables_20200130.zip"
+from that website. This module adds some non-standard codes that are not specified
+in the original tables to be compatible with some ad-hoc solutions in some resources
+and tools.
+
+The official ISO 15924 code list is taken from Unicode <https://www.unicode.org/iso15924/iso15924.txt.zip>
+and the additional information about connections between languages, scripts and territories
+is extracted from the Unicode CLDR project at <http://cldr.unicode.org>.

iso_langcodes-1.0.0/README.md

@@ -0,0 +1,142 @@
+# iso-langcodes
+
+Python modules for working with language codes, language groups and language scripts.
+
+Currently, there are 3 modules in this package:
+
+- `iso_langcodes.iso639_3` - conversion between ISO language codes
+- `iso_langcodes.iso639_5` - language groups and families according to ISO 639-5
+- `iso_langcodes.iso15924` - language scripts and their connection to languages
+
+## Installation
+
+```bash
+pip install iso-langcodes
+```
+
+Or install from source:
+
+```bash
+pip install .
+```
+
+## Usage
+
+### ISO 639-3
+
+```python
+from iso_langcodes import iso639_3
+
+print(iso639_3.convert_iso639('iso639-1', 'fra'))
+print(iso639_3.convert_iso639('iso639-3', 'de'))
+print(iso639_3.convert_iso639('name', 'fa'))
+
+print(iso639_3.get_iso639_1('deu'))
+print(iso639_3.get_iso639_3('de'))
+print(iso639_3.get_language_name('de'))
+print(iso639_3.get_language_name('eng'))
+print(iso639_3.get_macro_language('yue'))
+```
+
+### ISO 639-5
+
+```python
+from iso_langcodes import iso639_5
+
+print(' '.join(iso639_5.language_group('gmw')))
+print(' '.join(iso639_5.language_group('gem')))
+print(' '.join(iso639_5.language_group_children('gem')))
+print(iso639_5.language_parent('afr'))
+print(iso639_5.language_parent('gmw'))
+```
+
+### ISO 15924
+
+```python
+from iso_langcodes import iso15924
+
+script = iso15924.script_of_string('То је моја мачка.', 'srp')
+script = iso15924.script_of_string('你是我妈妈。')
+scripts = iso15924.script_of_string('То је моја мачка. Ti si moja majka.', allow_non_standard=True)
+
+region = iso15924.default_territory('por')
+script = iso15924.default_script('tur')
+```
+
+## Command Line Tools
+
+The package provides three command-line tools:
+
+### iso639
+
+Convert between language codes:
+
+```bash
+iso639 fra de eng
+iso639 -2 fra deu
+iso639 -3 de
+iso639 -m yue
+```
+
+### langgroup
+
+Print language groups according to ISO 639-5:
+
+```bash
+langgroup gem
+langgroup -c gem
+langgroup -p afr gmw
+```
+
+### langscript
+
+Detect characters from various scripts:
+
+```bash
+echo "То је моја мачка." | langscript -l srp
+echo "你是我妈妈。" | langscript
+```
+
+### langglottolog
+
+Convert between ISO 639-5 codes and Glottolog codes:
+
+```bash
+langglottolog aav afa alg
+langglottolog -g aust1305 afro1255
+```
+
+## License
+
+This software is Copyright (c) 2020 by Joerg Tiedemann.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+## Acknowledgements
+
+The language codes are taken from SIL International <https://iso639-3.sil.org>.
+Please, check the terms of use listed at <https://iso639-3.sil.org/code_tables/download_tables>.
+The current version uses the UTF-8 tables distributed in "iso-639-3_Code_Tables_20200130.zip"
+from that website. This module adds some non-standard codes that are not specified
+in the original tables to be compatible with some ad-hoc solutions in some resources
+and tools.
+
+The official ISO 15924 code list is taken from Unicode <https://www.unicode.org/iso15924/iso15924.txt.zip>
+and the additional information about connections between languages, scripts and territories
+is extracted from the Unicode CLDR project at <http://cldr.unicode.org>.

iso_langcodes-1.0.0/iso_langcodes/__init__.py

@@ -0,0 +1,16 @@
+"""
+langcodes - Python modules for working with language codes, language groups and language scripts.
+
+This package provides:
+- ISO 639-3: conversion between ISO language codes
+- ISO 639-5: language groups and families according to ISO 639-5
+- ISO 15924: language scripts and their connection to languages
+"""
+
+from . import iso639_3
+from . import iso639_5
+from . import iso15924
+
+__version__ = "1.0.0"
+__author__ = "Joerg Tiedemann"
+__all__ = ["iso639_3", "iso639_5", "iso15924"]

iso_langcodes-1.0.0/iso_langcodes/cli/__init__.py

@@ -0,0 +1 @@
+"""CLI package for langcodes."""

iso_langcodes-1.0.0/iso_langcodes/cli/iso639.py

@@ -0,0 +1,68 @@
+#!/usr/bin/env python3
+"""
+iso639 - a simple script to convert language codes.
+
+Usage: iso639 [-2|-3|-m|-n|-k] [langcode]*
+
+Convert to 3-letter-code if 2-letter code is given and vice versa.
+-2 ... print 2-letter code (even if the input is a 2-letter code)
+-3 ... print 3-letter code (even if the input is a 3-letter code)
+-m ... print macro language instead of local language variants
+-n ... don't print a final new-line
+-k ... keep original code if no mapping is found
+-p ... convert language pairs
+-P ... convert language pairs and sort alphabetically
+"""
+
+import argparse
+import sys
+
+from .. import iso639_3
+
+
+def convert_pairs(type_, langs, sorted_, keep):
+    """Convert language pairs instead of single language codes."""
+    converted = [iso639_3.convert_iso639(type_, lang, keep) for lang in langs.split('-')]
+    if sorted_:
+        converted = sorted(converted)
+    return '-'.join(converted)
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Convert language codes')
+    parser.add_argument('-2', '--iso639-1', action='store_true', help='convert to two-letter code (ISO 639-1)')
+    parser.add_argument('-3', '--iso639-3', action='store_true', help='convert to three-letter code (ISO 639-3)')
+    parser.add_argument('-m', '--macro', action='store_true', help='convert to macro language')
+    parser.add_argument('-n', '--no-newline', action='store_true', help="don't print a final new line")
+    parser.add_argument('-k', '--keep', action='store_true', help='keep original code if no mapping is found')
+    parser.add_argument('-p', '--pairs', action='store_true', help='convert language pairs')
+    parser.add_argument('-P', '--pairs-sorted', action='store_true', help='convert language pairs and sort alphabetically')
+    parser.add_argument('langcodes', nargs='*', help='language codes to convert')
+
+    args = parser.parse_args()
+
+    if args.iso639_1:
+        type_ = 'iso639-1'
+    elif args.iso639_3:
+        type_ = 'iso639-3'
+    elif args.macro:
+        type_ = 'macro'
+    else:
+        type_ = 'name'
+
+    if args.pairs or args.pairs_sorted:
+        converted = [convert_pairs(type_, lang, args.pairs_sorted, args.keep) for lang in args.langcodes]
+    else:
+        converted = [iso639_3.convert_iso639(type_, lang, args.keep) for lang in args.langcodes]
+
+    if type_ == 'name' and converted:
+        print('"' + '" "'.join(converted) + '"', end='')
+    else:
+        print(' '.join(converted), end='')
+
+    if not args.no_newline:
+        print()
+
+
+if __name__ == '__main__':
+    main()

iso_langcodes-1.0.0/iso_langcodes/cli/langglottolog.py

@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+"""
+langglottolog - convert between ISO 639-5 codes and Glottolog codes.
+
+Usage: langglottolog [OPTIONS] CODE*
+
+Options:
+  -h: help
+  -i: convert from ISO to Glottolog (default)
+  -g: convert from Glottolog to ISO
+  -n: don't print final newline
+  -v: verbose output
+"""
+
+import argparse
+import sys
+
+from .. import iso639_5
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Convert between ISO 639-5 codes and Glottolog codes')
+    parser.add_argument('-i', '--iso2glottolog', action='store_true', help='convert from ISO to Glottolog (default)')
+    parser.add_argument('-g', '--glottolog2iso', action='store_true', help='convert from Glottolog to ISO')
+    parser.add_argument('-n', '--no-newline', action='store_true', help="don't print final newline")
+    parser.add_argument('-v', '--verbose', action='store_true', help='verbose output')
+    parser.add_argument('codes', nargs='*', help='codes to convert')
+
+    args = parser.parse_args()
+
+    if args.verbose:
+        iso639_5.VERBOSE = 1
+
+    # Read from STDIN or take codes from command line args
+    if args.codes:
+        codes = args.codes
+    else:
+        codes = sys.stdin.read().split()
+
+    # Default to ISO to Glottolog if neither flag is set
+    if not args.glottolog2iso:
+        args.iso2glottolog = True
+
+    converted = []
+    for code in codes:
+        if args.glottolog2iso:
+            result = iso639_5.glottolog2iso(code)
+        else:
+            result = iso639_5.iso2glottolog(code)
+        converted.append(result if result else 'unknown')
+
+    print(' '.join(converted), end='')
+    if not args.no_newline:
+        print()
+
+
+if __name__ == '__main__':
+    main()

iso_langcodes-1.0.0/iso_langcodes/cli/langgroup.py

@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+"""
+langgroup - print language groups according to ISO639-5.
+
+Usage: langgroup [OPTIONS] LANGCODE*
+
+Options:
+  -h: help
+  -c: group children
+  -g: group languages
+  -G: group by grandparent
+  -n: don't print final newline
+  -p: print parent code of each given language code
+  -P: print parents all the way up the language tree
+  -v: verbose output
+"""
+
+import argparse
+import sys
+
+from .. import iso639_5
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Print language groups according to ISO639-5')
+    parser.add_argument('-c', '--children', action='store_true', help='group children')
+    parser.add_argument('-g', '--group', action='store_true', help='group languages')
+    parser.add_argument('-G', '--grandparent', action='store_true', help='group by grandparent')
+    parser.add_argument('-n', '--no-newline', action='store_true', help="don't print final newline")
+    parser.add_argument('-p', '--parent', action='store_true', help='print parent code of each given language code')
+    parser.add_argument('-P', '--parents', action='store_true', help='print parents all the way up the language tree')
+    parser.add_argument('-v', '--verbose', action='store_true', help='verbose output')
+    parser.add_argument('langcodes', nargs='*', help='language codes')
+
+    args = parser.parse_args()
+
+    if args.verbose:
+        iso639_5.VERBOSE = 1
+
+    # Read from STDIN or take codes from command line args
+    if args.langcodes:
+        codes = args.langcodes
+    else:
+        codes = sys.stdin.read().split()
+
+    if args.parent:
+        converted = [iso639_5.language_parent(code) for code in codes]
+        print(' '.join(str(c) for c in converted), end='')
+        if not args.no_newline:
+            print()
+    elif args.parents:
+        trees = []
+        for lang in codes:
+            parents = []
+            while True:
+                p = iso639_5.language_parent(lang)
+                if not p:
+                    break
+                parents.append(p)
+                lang = p
+                if p == 'mul':
+                    break
+            if parents:
+                trees.append(':'.join(parents))
+            else:
+                trees.append('none')
+        print(' '.join(trees), end='')
+        if not args.no_newline:
+            print()
+    elif args.group or args.grandparent:
+        groups = {}
+        for code in codes:
+            parent = iso639_5.language_parent(code) or code
+            if args.grandparent:
+                parent = iso639_5.language_parent(parent) or parent
+            if parent not in groups:
+                groups[parent] = set()
+            groups[parent].add(code)
+        for key in sorted(groups.keys()):
+            if args.no_newline:
+                print('+'.join(sorted(groups[key])), end=' ')
+            else:
+                print(key, end='\t')
+                print(' '.join(sorted(groups[key])))
+    else:
+        for code in codes:
+            if args.children:
+                print(' '.join(iso639_5.language_group_children(code)), end='')
+            else:
+                print(' '.join(iso639_5.language_group(code)), end='')
+            if not args.no_newline:
+                print()
+
+
+if __name__ == '__main__':
+    main()