piper-plus-g2p 0.2.0__tar.gz

piper_plus_g2p-0.2.0/.gitignore

@@ -0,0 +1,114 @@
+.DS_Store
+.idea
+*.log
+tmp/
+*.backup
+requirements.txt.backup
+
+*.py[cod]
+__pycache__/
+*.egg
+*.egg-info/
+build
+htmlcov
+# Python package build artifacts
+src/python_run/build/
+src/python_run/dist/
+
+/data/
+/build/
+/build-*/
+/local/
+/dist/
+/lib/
+/install/
+/download/
+/p/
+/o/
+/f/
+/s/
+*.so
+
+.venv/
+lightning_logs/
+
+# Claude settings
+.claude/*
+!.claude/commands/
+settings.local.json
+
+# Generated audio files
+*.wav
+!test/**/*.wav
+
+# Test artifacts
+test_piper/
+test_build/
+test_windows_build/
+test_results/
+test_ci_binaries/
+build_log.zip
+# Root test files (should be in src/cpp/tests or src/python/tests)
+/test_*.cpp
+/test_*.c
+/test_*.py
+/test_*.sh
+/test_*
+!test/
+
+# Package build artifacts
+nupkgs/
+publish_test/
+# Test logs
+*.log
+
+# Benchmark results
+benchmark_results*.json
+streaming_comparison_results.json
+
+# Windows build artifacts
+*.dll
+*.exe
+windows_build_artifacts/
+CLAUDE.md
+
+# Rust build artifacts
+src/rust/target/
+src/rust/piper-python/target/
+src/rust/piper-python/Cargo.lock
+
+# WebAssembly temp directories
+src/wasm/openjtalk-web/temp/
+src/wasm/openjtalk-web/tools/
+
+# Debug and development scripts
+scripts/debug/
+debug_*.py
+test_phoneme*.py
+verify_*.py
+analyze_*.py
+check_*.py
+create_comprehensive_*.py
+add_missing_*.py
+retrain_*.py
+comprehensive_*.py
+
+# Training datasets and downloaded models
+datasets/
+models/
+
+# Local working files
+checkpoints/
+output/
+pip-test/
+upiper_comparison_samples/
+compare_phonemes.txt
+phoneme_map.txt
+training_phonemes.txt
+id_check.txt
+sh_check.txt
+generate_upiper_samples.py
+nul
+test/models/tsukuyomi-chan/
+docs/development/usability-issues-2026-03.md
+src/rust/target/*

piper_plus_g2p-0.2.0/CHANGELOG.md

@@ -0,0 +1,41 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2026-04-01
+
+### Added
+
+#### Languages
+- 7 languages: JA (pyopenjtalk), EN (g2p-en), ZH (pypinyin), KO (g2pk2), ES, FR, PT (rule-based)
+- All dependencies are GPL-free (MIT / Apache-2.0 / BSD-3-Clause)
+
+#### Core API
+- `Phonemizer` ABC with IPA-first design: `phonemize()` returns clean IPA tokens, encoding is a separate step
+- `phonemize_with_prosody()` for languages that support prosody features (e.g., Japanese A1/A2/A3)
+- `PhonemizerRegistry` with auto-discovery and `entry_points` plugin support for third-party phonemizer extensions
+- `MultilingualPhonemizer` with Unicode-based language detection (`UnicodeLanguageDetector`)
+- Composite language code support (e.g., `"ja-en-zh"`) for multilingual models
+
+#### Encoding
+- `PiperEncoder` for phoneme ID encoding compatible with Piper TTS models
+- PUA mapping (87 entries) as canonical `pua.json` shared with C#/Rust implementations
+- `phoneme_id_map` loading with BOS/EOS/padding token handling
+
+#### Dictionaries
+- `CustomDictionary` with JSON v1.0/v2.0 format support (compatible with C++/Rust implementations)
+
+#### Safety
+- Input sanitization: `MAX_INPUT_LENGTH=10000` limit and control character stripping
+
+#### Testing & CI
+- 170+ test cases
+- GitHub Actions CI (3 OS x 2 Python)
+
+### Architecture
+- IPA-first: `phonemize()` returns clean IPA tokens; encoding to integer IDs is handled separately by `PiperEncoder`
+- GPL-free: no eSpeak-ng dependency; all runtime dependencies are MIT/Apache-2.0/BSD-3-Clause
+- Plugin system via setuptools `entry_points` (`piper_plus_g2p.phonemizers` group)

piper_plus_g2p-0.2.0/CONTRIBUTING.md

@@ -0,0 +1,55 @@
+# Contributing to piper-g2p
+
+## Development Setup
+
+```bash
+cd src/python/g2p
+uv sync --extra all --extra dev
+uv run pytest                    # run the full test suite
+uv run ruff check piper_plus_g2p/    # lint
+uv run mypy piper_plus_g2p/          # type-check
+```
+
+## How to Add a New Language
+
+1. **Create `piper_plus_g2p/<lang>.py`** -- subclass `Phonemizer` (from `piper_plus_g2p.base`).
+2. **Implement `phonemize_with_prosody()`** -- return `(tokens, prosody)` where
+   `tokens` is a list of IPA strings and `prosody` is a list of `ProsodyInfo | None`.
+3. **Create `tests/test_<lang>.py`** with at least 4 tests: basic phonemization,
+   prosody values, empty input, and edge cases. If the language needs an optional
+   dependency, add a `requires_<lang>` skip marker in `tests/conftest.py`.
+4. **Update `pyproject.toml`** -- add the language code under
+   `[project.optional-dependencies]` and include any new dependency in the `all` extra.
+5. **Register the phonemizer** -- add a `(code, module, class)` tuple to
+   `_LANGUAGE_TABLE` in `piper_plus_g2p/registry.py`, then open a PR.
+
+## Plugin System (entry_points)
+
+External packages can register phonemizers without modifying this repo.
+In your package's `pyproject.toml`:
+
+```toml
+[project.entry-points."piper_plus_g2p.phonemizers"]
+xx = "my_package:MyPhonemizer"
+```
+
+The registry discovers these at import time via `importlib.metadata.entry_points`.
+
+## Code Style
+
+- **Formatter/linter**: ruff (line-length 88, target Python 3.11).
+- **Type checking**: mypy with `ignore_missing_imports = true`.
+- Keep each language module self-contained; shared utilities go in `base.py`.
+
+## Testing
+
+- Framework: **pytest**
+- Skip markers: `requires_ja`, `requires_en`, `requires_zh`, `requires_ko`
+  (defined in `tests/conftest.py`) for tests that need optional dependencies.
+- CI runs on 3 OS (Linux, macOS, Windows) x 2 Python versions.
+
+## Pull Requests
+
+- One language per PR when possible.
+- Include before/after phonemization examples in the PR description.
+- All CI checks must pass before merge.

piper_plus_g2p-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 piper-plus contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

piper_plus_g2p-0.2.0/PKG-INFO

@@ -0,0 +1,179 @@
+Metadata-Version: 2.4
+Name: piper-plus-g2p
+Version: 0.2.0
+Summary: Multilingual G2P (Grapheme-to-Phoneme) for TTS — eSpeak-ng free, MIT licensed
+Project-URL: Homepage, https://github.com/ayutaz/piper-plus
+Project-URL: Repository, https://github.com/ayutaz/piper-plus
+Project-URL: Issues, https://github.com/ayutaz/piper-plus/issues
+Project-URL: Documentation, https://github.com/ayutaz/piper-plus/tree/dev/src/python/g2p#readme
+Author: piper-plus contributors
+License: MIT
+License-File: LICENSE
+Keywords: g2p,grapheme-to-phoneme,ipa,phonemizer,tts
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Provides-Extra: all
+Requires-Dist: g2p-en<3,>=2.1.0; extra == 'all'
+Requires-Dist: g2pk2<1,>=0.0.3; extra == 'all'
+Requires-Dist: pyopenjtalk-plus<1,>=0.3; extra == 'all'
+Requires-Dist: pypinyin<1,>=0.55.0; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.13; extra == 'dev'
+Requires-Dist: pytest-cov>=4.1; extra == 'dev'
+Requires-Dist: pytest>=7.4; extra == 'dev'
+Requires-Dist: ruff>=0.8.0; extra == 'dev'
+Provides-Extra: en
+Requires-Dist: g2p-en<3,>=2.1.0; extra == 'en'
+Provides-Extra: es
+Provides-Extra: fr
+Provides-Extra: ja
+Requires-Dist: pyopenjtalk-plus<1,>=0.3; extra == 'ja'
+Provides-Extra: ko
+Requires-Dist: g2pk2<1,>=0.0.3; extra == 'ko'
+Provides-Extra: pt
+Provides-Extra: sv
+Provides-Extra: zh
+Requires-Dist: pypinyin<1,>=0.55.0; extra == 'zh'
+Description-Content-Type: text/markdown
+
+# piper-plus-g2p
+
+Multilingual G2P (Grapheme-to-Phoneme) for TTS. eSpeak-ng free. MIT licensed. 8 languages.
+
+**Standalone package** — piper-plus TTS エンジンなしで G2P 単体として利用可能です。任意の TTS エンジンと組み合わせて使えます。
+
+## Why piper-plus-g2p?
+
+- **MIT licensed** -- no eSpeak-ng (GPL) dependency in your TTS pipeline
+- **8 languages** -- JA, EN, ZH, KO, ES, FR, PT, SV with consistent IPA output
+- **IPA-first design** -- returns pure IPA token sequences, ready for any TTS model
+
+## Comparison
+
+| | piper-plus-g2p | phonemizer | gruut | Misaki |
+|---|---|---|---|---|
+| License | MIT | GPL (eSpeak-ng) | MIT | Apache-2.0 |
+| Languages | 8 | 100+ | 20+ | EN only |
+| eSpeak-ng required | No | Yes | No | No |
+| IPA output | Yes | Yes | Yes | Yes |
+
+## Installation
+
+```bash
+pip install piper-plus-g2p               # Rule-based languages (ES, FR, PT, SV)
+pip install piper-plus-g2p[ja,en]        # Japanese + English
+pip install piper-plus-g2p[all]          # All 8 languages
+```
+
+> **Note:** The `ja` extra requires `pyopenjtalk-plus`, which provides pre-built wheels for Linux, macOS, and Windows. See [pyopenjtalk-plus](https://pypi.org/project/pyopenjtalk-plus/) for platform details.
+
+## Quick Start
+
+```python
+from piper_plus_g2p import get_phonemizer
+
+ja = get_phonemizer("ja")
+ja.phonemize("こんにちは")
+# -> ["k", "o", "[", "N_n", "n", "i", "ch", "i", "w", "a"]
+
+en = get_phonemizer("en")
+en.phonemize("Hello world")
+# -> ["h", "ə", "l", "ˈ", "o", "ʊ", " ", "w", "ˈ", "ɜ", "ː", "l", "d"]
+```
+
+## Supported Languages
+
+| Language | Code | Extra | Backend | Notes |
+|---|---|---|---|---|
+| Japanese | `ja` | `piper-plus-g2p[ja]` | pyopenjtalk-plus | Context-dependent N variants, prosody info |
+| English | `en` | `piper-plus-g2p[en]` | g2p-en | CMU-dict + neural fallback |
+| Chinese | `zh` | `piper-plus-g2p[zh]` | pypinyin | Pinyin-to-IPA conversion |
+| Korean | `ko` | `piper-plus-g2p[ko]` | g2pk2 | Optional dependency |
+| Spanish | `es` | -- | Rule-based | No external dependency |
+| French | `fr` | -- | Rule-based | No external dependency |
+| Portuguese | `pt` | -- | Rule-based | No external dependency |
+| Swedish | `sv` | -- | Rule-based | No external dependency |
+
+## Advanced Usage
+
+### Multilingual (Composite Language Code)
+
+Pass a hyphen-joined code like `"ja-en-zh"` to `get_phonemizer` to
+automatically create a `MultilingualPhonemizer`. Language detection is
+Unicode-based, so mixed-script text is handled without explicit tagging.
+
+```python
+from piper_plus_g2p import get_phonemizer
+
+multi = get_phonemizer("ja-en-zh")
+tokens = multi.phonemize("こんにちは Hello 你好")
+```
+
+### PiperEncoder
+
+Converts IPA token lists into integer `phoneme_ids` for Piper ONNX models.
+
+```python
+from piper_plus_g2p.encode import PiperEncoder, get_phoneme_id_map
+
+id_map = get_phoneme_id_map("ja")
+encoder = PiperEncoder(id_map)
+phoneme_ids = encoder.encode(["k", "o", "[", "N_n", "n", "i", "ch", "i", "w", "a"])
+```
+
+## Piper Model Compatibility
+
+piper-plus-g2p produces phoneme tokens directly compatible with
+[Piper TTS](https://github.com/rhasspy/piper) ONNX models. Use
+`PiperEncoder` with the model's `phoneme_id_map` from `config.json`:
+
+```python
+import json
+from piper_plus_g2p import get_phonemizer
+from piper_plus_g2p.encode import PiperEncoder
+
+with open("model/config.json") as f:
+    config = json.load(f)
+
+encoder = PiperEncoder(config["phoneme_id_map"])
+phonemizer = get_phonemizer("ja")
+tokens = phonemizer.phonemize("こんにちは")
+phoneme_ids = encoder.encode(tokens)
+# -> ready for ONNX inference
+```
+
+Use `strict=True` to raise errors on unknown tokens instead of silently
+skipping them:
+
+```python
+encoder = PiperEncoder(config["phoneme_id_map"], strict=True)
+```
+
+## Cross-Platform Consistency
+
+piper-plus-g2p is also available as:
+- **Rust crate**: `piper-plus-g2p` on crates.io
+- **Go module**: `github.com/ayutaz/piper-plus/src/go/phonemize`
+- **npm package**: `@piper-plus/g2p` for browser/WASM
+
+All three implementations share the same PUA mapping table and are
+validated against a common test fixture.
+
+## Requirements
+
+- Python >= 3.11
+- Language backends are installed via extras (see [Supported Languages](#supported-languages))
+
+## License
+
+MIT

piper_plus_g2p-0.2.0/README.md

@@ -0,0 +1,131 @@
+# piper-plus-g2p
+
+Multilingual G2P (Grapheme-to-Phoneme) for TTS. eSpeak-ng free. MIT licensed. 8 languages.
+
+**Standalone package** — piper-plus TTS エンジンなしで G2P 単体として利用可能です。任意の TTS エンジンと組み合わせて使えます。
+
+## Why piper-plus-g2p?
+
+- **MIT licensed** -- no eSpeak-ng (GPL) dependency in your TTS pipeline
+- **8 languages** -- JA, EN, ZH, KO, ES, FR, PT, SV with consistent IPA output
+- **IPA-first design** -- returns pure IPA token sequences, ready for any TTS model
+
+## Comparison
+
+| | piper-plus-g2p | phonemizer | gruut | Misaki |
+|---|---|---|---|---|
+| License | MIT | GPL (eSpeak-ng) | MIT | Apache-2.0 |
+| Languages | 8 | 100+ | 20+ | EN only |
+| eSpeak-ng required | No | Yes | No | No |
+| IPA output | Yes | Yes | Yes | Yes |
+
+## Installation
+
+```bash
+pip install piper-plus-g2p               # Rule-based languages (ES, FR, PT, SV)
+pip install piper-plus-g2p[ja,en]        # Japanese + English
+pip install piper-plus-g2p[all]          # All 8 languages
+```
+
+> **Note:** The `ja` extra requires `pyopenjtalk-plus`, which provides pre-built wheels for Linux, macOS, and Windows. See [pyopenjtalk-plus](https://pypi.org/project/pyopenjtalk-plus/) for platform details.
+
+## Quick Start
+
+```python
+from piper_plus_g2p import get_phonemizer
+
+ja = get_phonemizer("ja")
+ja.phonemize("こんにちは")
+# -> ["k", "o", "[", "N_n", "n", "i", "ch", "i", "w", "a"]
+
+en = get_phonemizer("en")
+en.phonemize("Hello world")
+# -> ["h", "ə", "l", "ˈ", "o", "ʊ", " ", "w", "ˈ", "ɜ", "ː", "l", "d"]
+```
+
+## Supported Languages
+
+| Language | Code | Extra | Backend | Notes |
+|---|---|---|---|---|
+| Japanese | `ja` | `piper-plus-g2p[ja]` | pyopenjtalk-plus | Context-dependent N variants, prosody info |
+| English | `en` | `piper-plus-g2p[en]` | g2p-en | CMU-dict + neural fallback |
+| Chinese | `zh` | `piper-plus-g2p[zh]` | pypinyin | Pinyin-to-IPA conversion |
+| Korean | `ko` | `piper-plus-g2p[ko]` | g2pk2 | Optional dependency |
+| Spanish | `es` | -- | Rule-based | No external dependency |
+| French | `fr` | -- | Rule-based | No external dependency |
+| Portuguese | `pt` | -- | Rule-based | No external dependency |
+| Swedish | `sv` | -- | Rule-based | No external dependency |
+
+## Advanced Usage
+
+### Multilingual (Composite Language Code)
+
+Pass a hyphen-joined code like `"ja-en-zh"` to `get_phonemizer` to
+automatically create a `MultilingualPhonemizer`. Language detection is
+Unicode-based, so mixed-script text is handled without explicit tagging.
+
+```python
+from piper_plus_g2p import get_phonemizer
+
+multi = get_phonemizer("ja-en-zh")
+tokens = multi.phonemize("こんにちは Hello 你好")
+```
+
+### PiperEncoder
+
+Converts IPA token lists into integer `phoneme_ids` for Piper ONNX models.
+
+```python
+from piper_plus_g2p.encode import PiperEncoder, get_phoneme_id_map
+
+id_map = get_phoneme_id_map("ja")
+encoder = PiperEncoder(id_map)
+phoneme_ids = encoder.encode(["k", "o", "[", "N_n", "n", "i", "ch", "i", "w", "a"])
+```
+
+## Piper Model Compatibility
+
+piper-plus-g2p produces phoneme tokens directly compatible with
+[Piper TTS](https://github.com/rhasspy/piper) ONNX models. Use
+`PiperEncoder` with the model's `phoneme_id_map` from `config.json`:
+
+```python
+import json
+from piper_plus_g2p import get_phonemizer
+from piper_plus_g2p.encode import PiperEncoder
+
+with open("model/config.json") as f:
+    config = json.load(f)
+
+encoder = PiperEncoder(config["phoneme_id_map"])
+phonemizer = get_phonemizer("ja")
+tokens = phonemizer.phonemize("こんにちは")
+phoneme_ids = encoder.encode(tokens)
+# -> ready for ONNX inference
+```
+
+Use `strict=True` to raise errors on unknown tokens instead of silently
+skipping them:
+
+```python
+encoder = PiperEncoder(config["phoneme_id_map"], strict=True)
+```
+
+## Cross-Platform Consistency
+
+piper-plus-g2p is also available as:
+- **Rust crate**: `piper-plus-g2p` on crates.io
+- **Go module**: `github.com/ayutaz/piper-plus/src/go/phonemize`
+- **npm package**: `@piper-plus/g2p` for browser/WASM
+
+All three implementations share the same PUA mapping table and are
+validated against a common test fixture.
+
+## Requirements
+
+- Python >= 3.11
+- Language backends are installed via extras (see [Supported Languages](#supported-languages))
+
+## License
+
+MIT

piper_plus_g2p-0.2.0/THIRD_PARTY_LICENSES.md

@@ -0,0 +1,51 @@
+# Third-Party Licenses — piper-g2p (Python)
+
+piper-g2p is licensed under MIT. Below are the licenses of its
+dependencies, grouped by optionality.
+
+## Core (no external dependencies)
+
+piper-g2p core has zero required runtime dependencies. Languages ES, FR,
+PT, SV, and KO are fully rule-based and need no external packages.
+
+## Optional: Japanese (`pip install piper-g2p[ja]`)
+
+| Package | License | Purpose |
+|---------|---------|---------|
+| pyopenjtalk-plus | MIT | OpenJTalk bindings for Japanese G2P |
+| numpy | BSD-3-Clause | Numerical arrays (pyopenjtalk dep) |
+
+## Optional: English (`pip install piper-g2p[en]`)
+
+| Package | License | Purpose |
+|---------|---------|---------|
+| g2p-en | Apache-2.0 | CMU Pronouncing Dictionary + neural G2P |
+| numpy | BSD-3-Clause | Neural network inference |
+
+## Optional: Chinese (`pip install piper-g2p[zh]`)
+
+| Package | License | Purpose |
+|---------|---------|---------|
+| pypinyin | MIT | Hanzi to Pinyin conversion |
+
+## Optional: Korean (`pip install piper-g2p[ko]`)
+
+| Package | License | Purpose |
+|---------|---------|---------|
+| g2pk2 | Apache-2.0 | Korean G2P with neural disambiguation |
+
+> **Note:** g2pk2 may pull in heavy transitive dependencies (torch or
+> tensorflow). Consider whether Korean support is needed before installing.
+
+## Development
+
+| Package | License | Purpose |
+|---------|---------|---------|
+| pytest | MIT | Test framework |
+| ruff | MIT | Linter/formatter |
+| mypy | MIT | Type checker |
+
+## No copyleft licenses
+
+All runtime dependencies are MIT, Apache-2.0, or BSD-3-Clause.
+No GPL, LGPL, or AGPL licenses are present.