polyglot3 2026.10.1__tar.gz

polyglot3-2026.10.1/AUTHORS.md

@@ -0,0 +1,28 @@
+# Credits
+
+## polyglot3
+
+polyglot3 is maintained by **Dcipher Analytics AB** (<info@dcipheranalytics.com>).
+
+This repository has been updated by the Dcipher Analytics dev team with help
+from Claude Opus.
+
+## polyglot
+
+Many thanks to the authors of the original
+[polyglot](https://github.com/aboSamoor/polyglot), created at the Data
+Science Lab, Stony Brook University, on which polyglot3 is built:
+
+- Rami Al-Rfou <rmyeid@gmail.com> (development lead)
+- Yingtao Tian <yittian@cs.stonybrook.edu>
+- All contributors to https://github.com/aboSamoor/polyglot
+
+The models come from the research of Rami Al-Rfou, Bryan Perozzi, Vivek
+Kulkarni, Yanqing Chen and Steven Skiena; see the README for citations.
+
+## Third-party data
+
+- Tokenization tables and the Thai dictionary are derived from ICU
+  (Unicode License V3, `polyglot3/tokenize/data/LICENSE-ICU`).
+- Language detection uses fastText's `lid.176` model through
+  fasttext-langdetect.

polyglot3-2026.10.1/CONTRIBUTING.md

@@ -0,0 +1,44 @@
+# Contributing
+
+Contributions are welcome. Report bugs and propose features at
+https://github.com/dcipheranalytics/polyglot3/issues. For bugs, include your
+operating system, Python version, polyglot3 version and the steps to
+reproduce.
+
+## Development setup
+
+1. Fork and clone the repository:
+
+   ```bash
+   git clone git@github.com:your_name_here/polyglot3.git
+   cd polyglot3
+   ```
+
+2. Install it in a virtual environment in editable mode:
+
+   ```bash
+   python -m venv .venv && source .venv/bin/activate
+   pip install -e ".[test]" ruff
+   ```
+
+3. Create a branch, make your changes, then run the checks:
+
+   ```bash
+   ruff format .
+   ruff check polyglot3 tests tools
+   python -m pytest
+   ```
+
+   Tests that need models skip themselves unless the models are installed
+   (see the [developer guide](docs/Dev.md)).
+
+4. Push your branch and open a pull request.
+
+## Pull request guidelines
+
+1. Include tests for bug fixes and new features.
+2. Update the documentation in `docs/` when behaviour changes.
+3. The tests must pass on Python 3.9 to 3.13 on Linux, macOS and Windows;
+   GitHub Actions runs them for every pull request.
+4. Tokenization must stay identical to ICU's (see the
+   [developer guide](docs/Dev.md)); the NER and POS models depend on it.

polyglot3-2026.10.1/HISTORY.md

@@ -0,0 +1,57 @@
+# History
+
+## 2026.10.1 (2026-10-22)
+
+First release as **polyglot3**, by Dcipher Analytics AB.
+
+- The package is now imported as `polyglot3` (`from polyglot3.text import Text`)
+  and the command is `polyglot3`, so it can be installed next to the original
+  `polyglot`. Models, `polyglot_data/` and the `POLYGLOT_*` environment
+  variables are unchanged.
+- Python 3.9 to 3.13 only; Python 2 support and `six` removed.
+- Removed the PyICU dependency: word and sentence segmentation use a
+  pure-Python port of ICU 78.3's break iterators (including the Thai
+  dictionary engine) that reproduces ICU's boundaries.
+- Removed the pycld2 dependency: language detection uses fastText's
+  `lid.176` model via fasttext-langdetect. Hebrew is now `he`, not `iw`.
+- Models are downloaded from a new server (Hugging Face) with checksum
+  verification; `POLYGLOT_DATA_URL` selects a mirror.
+- Packaging with `pyproject.toml`.
+- Thread-safe: models are loaded once per process even when several threads
+  need them at the same time. See the Threads and Concurrency page.
+- Removed morphological analysis and transliteration (models unavailable)
+  and the TextBlob leftovers `correct()`, `tokenize()` and `np_counts` (they
+  always raised errors).
+- Bug fixes: garbled tokens after emoji and other non-BMP characters; the
+  `sentiment` subcommand did nothing; entity sentiment only used the first
+  polar word; `Embedding.normalize_words()` failed on expanded vocabularies;
+  `Downloader.status()` returned `None` for a custom download directory;
+  `Downloader.supported_language()` crashed; the CLI closed stdin repeatedly
+  on Python 3.13.
+
+## polyglot 16.07.04 (2016-07-03)
+
+- New features:
+  - Support Transfer POS Tagging.
+  - Support supplying `hint_language_code` for `Text`.
+- Bug fixes:
+  - Improve sentence serialization (PR #34)
+  - Fix rare unicode encode error (PR #35)
+  - Fix transliteration from languages other than English (PR #46)
+  - Add link to Github in README (PR #49)
+  - Make handling of paths more coherent (PR #55)
+  - Fix normalizing embedding in place for NER corrupts the features of POS
+    (issue #60, PR #62)
+
+## polyglot 15.10.03 (2015-10-03)
+
+- Change the polyglot models mirror to Stony Brook University DSL lab instead
+  of Google cloud storage.
+
+## polyglot 15.5.2 (2015-05-02)
+
+- Polyglot is feature complete.
+
+## polyglot 14.11 (2014-01-11)
+
+- First release on PyPI.