oro-g2p 0.1.0__tar.gz

oro_g2p-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Oro Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

oro_g2p-0.1.0/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: oro-g2p
+Version: 0.1.0
+Summary: Research-grade Yoruba Grapheme-to-Phoneme (G2P) converter and Kokoro TTS frontend
+Author: Oro Contributors
+License: MIT
+Project-URL: Homepage, https://github.com/Akintunde102/oro-g2p
+Project-URL: Repository, https://github.com/Akintunde102/oro-g2p
+Project-URL: Issues, https://github.com/Akintunde102/oro-g2p/issues
+Keywords: yoruba,g2p,tts,kokoro,phonemes,african-nlp,speech-synthesis,phonology
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# oro-g2p 🗣️
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python: 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/)
+
+**`oro-g2p`** is a zero-dependency, research-grade Yoruba Grapheme-to-Phoneme (G2P) engine and phonetic tokenizer designed for speech synthesis (TTS), automatic speech recognition (ASR), and forced alignment.
+
+It provides out-of-the-box native token mapping for **[Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)** and **StyleTTS2**.
+
+---
+
+## Key Features
+
+1. **Deterministic Linguistic Engine:** Pure, rule-based Yoruba phonology — zero hallucinations, reproducible output.
+2. **Post-Nasal Harmony:** Automatically captures unwritten phonetic nasalization on vowels following nasal consonants `m` and `n` (*ọmọ* → `/ɔˉmɔ̃ˉ/`, *inú* → `/iˉnũˊ/`).
+3. **Chameleon Syllabic Nasals:** Dynamically applies homorganic place assimilation to `ń` and `ǹ`:
+   - Labials (`b`, `f`): `/m/` (*Mo ń bọ̀* → `moˉ mˊ bɔˋ`)
+   - Alveolars (`d`, `t`, `s`, `ṣ`): `/n/` (*Mo ń ṣe* → `moˉ nˊ ʃeˉ`)
+   - Velars & Glides (`l`, `k`, `g`): `/ŋ/` (*Mo ń lọ* → `moˉ ŋˊ lɔˉ`)
+4. **Typo Resilience:** Seamlessly normalizes Turkish cedillas (`ş`), combining vertical lines (`\u0329`), and out-of-order diacritic keystrokes into canonical Yoruba underdots (`ẹ`, `ọ`, `ṣ`).
+5. **Contour Glides:** Expands carons (`ǎ` → `àá` Low-High) and circumflexes (`â` → `áà` High-Low).
+6. **Native Kokoro Alignment:** Maps 100% of Yoruba phonemes and tones directly into Kokoro-82M's 178-token embedding table with **0 missing tokens**.
+
+---
+
+## Installation
+
+```bash
+pip install oro-g2p
+```
+
+---
+
+## Quickstart
+
+### 1. Basic G2P Conversion
+```python
+from oro_g2p import g2p
+
+# Converts Yoruba text to tone-marked IPA
+phonemes = g2p("Ẹ kú àárọ̀, ọmọ Yorùbá!")
+print(phonemes)
+# Output: 'ɛˉ kuˊ aˋaˊɾɔˋ, ɔˉmɔ̃ˉ joˉɾuˋbaˊ!'
+```
+
+### 2. Kokoro-82M Tokenization
+```python
+from oro_g2p import yoruba_to_kokoro
+
+text = "Ẹ kú àárọ̀"
+ipa_str, token_ids, missing = yoruba_to_kokoro(text)
+
+print(f"IPA:       {ipa_str}")
+print(f"Token IDs: {token_ids}")
+print(f"Missing:   {missing}")  # [] -> 100% coverage!
+```
+
+---
+
+## Architecture: Kokoro-82M Token Mapping
+
+Kokoro has a fixed embedding matrix size of `178` (`0` to `177`). While 114 slots were occupied by Western languages, 64 slots were unassigned.
+
+`oro-g2p` resolves Yoruba without modifying any pre-trained weights or embedding tensor dimensions:
+- **High Tone (Mí):** Assigned to Vacant Slot `7` (`ˊ`)
+- **Mid Tone (Ré):** Assigned to Vacant Slot `8` (`ˉ`)
+- **Low Tone (Dò):** Assigned to Vacant Slot `26` (`ˋ`)
+- **Nasal Vowels:** Decomposed into `(vowel + \u0303)`, cleanly utilizing Kokoro's native Room `17`.
+
+---
+
+## Running Tests
+
+```bash
+python -m pytest tests/
+```
+
+---
+
+## License
+
+MIT License. Free for academic, personal, and commercial speech technology research.

oro_g2p-0.1.0/README.md

@@ -0,0 +1,82 @@
+# oro-g2p 🗣️
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python: 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/)
+
+**`oro-g2p`** is a zero-dependency, research-grade Yoruba Grapheme-to-Phoneme (G2P) engine and phonetic tokenizer designed for speech synthesis (TTS), automatic speech recognition (ASR), and forced alignment.
+
+It provides out-of-the-box native token mapping for **[Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)** and **StyleTTS2**.
+
+---
+
+## Key Features
+
+1. **Deterministic Linguistic Engine:** Pure, rule-based Yoruba phonology — zero hallucinations, reproducible output.
+2. **Post-Nasal Harmony:** Automatically captures unwritten phonetic nasalization on vowels following nasal consonants `m` and `n` (*ọmọ* → `/ɔˉmɔ̃ˉ/`, *inú* → `/iˉnũˊ/`).
+3. **Chameleon Syllabic Nasals:** Dynamically applies homorganic place assimilation to `ń` and `ǹ`:
+   - Labials (`b`, `f`): `/m/` (*Mo ń bọ̀* → `moˉ mˊ bɔˋ`)
+   - Alveolars (`d`, `t`, `s`, `ṣ`): `/n/` (*Mo ń ṣe* → `moˉ nˊ ʃeˉ`)
+   - Velars & Glides (`l`, `k`, `g`): `/ŋ/` (*Mo ń lọ* → `moˉ ŋˊ lɔˉ`)
+4. **Typo Resilience:** Seamlessly normalizes Turkish cedillas (`ş`), combining vertical lines (`\u0329`), and out-of-order diacritic keystrokes into canonical Yoruba underdots (`ẹ`, `ọ`, `ṣ`).
+5. **Contour Glides:** Expands carons (`ǎ` → `àá` Low-High) and circumflexes (`â` → `áà` High-Low).
+6. **Native Kokoro Alignment:** Maps 100% of Yoruba phonemes and tones directly into Kokoro-82M's 178-token embedding table with **0 missing tokens**.
+
+---
+
+## Installation
+
+```bash
+pip install oro-g2p
+```
+
+---
+
+## Quickstart
+
+### 1. Basic G2P Conversion
+```python
+from oro_g2p import g2p
+
+# Converts Yoruba text to tone-marked IPA
+phonemes = g2p("Ẹ kú àárọ̀, ọmọ Yorùbá!")
+print(phonemes)
+# Output: 'ɛˉ kuˊ aˋaˊɾɔˋ, ɔˉmɔ̃ˉ joˉɾuˋbaˊ!'
+```
+
+### 2. Kokoro-82M Tokenization
+```python
+from oro_g2p import yoruba_to_kokoro
+
+text = "Ẹ kú àárọ̀"
+ipa_str, token_ids, missing = yoruba_to_kokoro(text)
+
+print(f"IPA:       {ipa_str}")
+print(f"Token IDs: {token_ids}")
+print(f"Missing:   {missing}")  # [] -> 100% coverage!
+```
+
+---
+
+## Architecture: Kokoro-82M Token Mapping
+
+Kokoro has a fixed embedding matrix size of `178` (`0` to `177`). While 114 slots were occupied by Western languages, 64 slots were unassigned.
+
+`oro-g2p` resolves Yoruba without modifying any pre-trained weights or embedding tensor dimensions:
+- **High Tone (Mí):** Assigned to Vacant Slot `7` (`ˊ`)
+- **Mid Tone (Ré):** Assigned to Vacant Slot `8` (`ˉ`)
+- **Low Tone (Dò):** Assigned to Vacant Slot `26` (`ˋ`)
+- **Nasal Vowels:** Decomposed into `(vowel + \u0303)`, cleanly utilizing Kokoro's native Room `17`.
+
+---
+
+## Running Tests
+
+```bash
+python -m pytest tests/
+```
+
+---
+
+## License
+
+MIT License. Free for academic, personal, and commercial speech technology research.

oro_g2p-0.1.0/oro_g2p/__init__.py

@@ -0,0 +1,102 @@
+"""
+oro_g2p: Research-grade Yoruba Grapheme-to-Phoneme converter and Kokoro TTS frontend.
+"""
+
+from typing import Tuple, List, Optional
+from .normalizer import normalize_yoruba
+from .tokenizer import tokenize_graphemes
+from .kokoro import to_kokoro_tokens, YORUBA_KOKORO_VOCAB
+
+__version__ = "0.1.0"
+__all__ = [
+    "g2p",
+    "tokenize_graphemes",
+    "normalize_yoruba",
+    "to_kokoro_tokens",
+    "yoruba_to_kokoro",
+    "YORUBA_KOKORO_VOCAB",
+]
+
+# Standard IPA mappings for Yoruba
+CONSONANT_IPA = {
+    "b": "b",
+    "d": "d",
+    "f": "f",
+    "g": "ɡ",     # Single-story IPA ɡ (\u0261)
+    "gb": "ɡb",   # Labial-velar voiced stop
+    "h": "h",
+    "j": "dʒ",    # Voiced palato-alveolar affricate
+    "k": "k",
+    "l": "l",
+    "m": "m",
+    "n": "n",
+    "ŋ": "ŋ",     # Velar nasal
+    "p": "kp",    # Voiceless labial-velar stop /kp/
+    "r": "ɾ",     # Alveolar flap/tap
+    "s": "s",
+    "ṣ": "ʃ",     # Voiceless postalveolar fricative
+    "t": "t",
+    "w": "w",
+    "y": "j",     # Palatal approximant (IPA 'j')
+}
+
+VOWEL_IPA = {
+    # Oral vowels
+    "a": "a",
+    "e": "e",
+    "ẹ": "ɛ",
+    "i": "i",
+    "o": "o",
+    "ọ": "ɔ",
+    "u": "u",
+    # Nasal vowels
+    "an": "ã",
+    "ẹn": "ɛ̃",
+    "in": "ĩ",
+    "ọn": "ɔ̃",
+    "un": "ũ",
+}
+
+TONE_IPA = {
+    "H": "ˊ",  # High tone (Mí)
+    "M": "ˉ",  # Mid tone  (Ré)
+    "L": "ˋ",  # Low tone  (Dò)
+}
+
+
+def g2p(text: str) -> str:
+    """
+    Converts Yoruba text into a tone-marked IPA phoneme string.
+
+    Example:
+      >>> from oro_g2p import g2p
+      >>> g2p("Ẹ kú àárọ̀")
+      'ɛˉ kuˊ aˋaˊɾɔˋ'
+    """
+    tokens = tokenize_graphemes(text)
+    phonemes: List[str] = []
+
+    for grapheme, tone in tokens:
+        if grapheme in VOWEL_IPA:
+            phonemes.append(VOWEL_IPA[grapheme] + (TONE_IPA[tone] if tone else ""))
+        elif grapheme in CONSONANT_IPA:
+            sound = CONSONANT_IPA[grapheme]
+            if tone is not None:
+                sound += TONE_IPA[tone]
+            phonemes.append(sound)
+        else:
+            phonemes.append(grapheme)
+
+    return "".join(phonemes)
+
+
+def yoruba_to_kokoro(text: str) -> Tuple[str, List[int], List[str]]:
+    """
+    End-to-end pipeline: Raw Yoruba text -> IPA string -> Kokoro token IDs.
+
+    Returns:
+      (ipa_string, token_ids, missing_tokens)
+    """
+    ipa_str = g2p(text)
+    token_ids, missing = to_kokoro_tokens(ipa_str)
+    return ipa_str, token_ids, missing

oro_g2p-0.1.0/oro_g2p/kokoro.py

@@ -0,0 +1,58 @@
+"""
+oro_g2p.kokoro
+~~~~~~~~~~~~~~
+Kokoro-82M vocabulary integration and tokenization.
+Maps Yoruba IPA phones and tones to the native 178-token vocabulary table.
+"""
+
+import unicodedata
+from typing import Tuple, List
+
+# Official Kokoro-82M base vocabulary dictionary (114 occupied slots)
+BASE_KOKORO_VOCAB = {
+    ";": 1, ":": 2, ",": 3, ".": 4, "!": 5, "?": 6, "—": 9, "…": 10, "\"": 11,
+    "(": 12, ")": 13, "“": 14, "”": 15, " ": 16, "\u0303": 17, "ˈ": 18, "ˌ": 19,
+    "ː": 20, "ɸ": 21, "β": 22, "θ": 23, "ð": 24, "ʃ": 25, "ɹ": 27, "ɻ": 28,
+    "ʋ": 29, "ⱱ": 30, "ç": 32, "ʝ": 33, "ɣ": 35, "χ": 36, "ʁ": 37, "ħ": 38,
+    "ʕ": 39, "ɦ": 40, "ɬ": 41, "ɮ": 42, "a": 43, "b": 44, "c": 45, "d": 46,
+    "e": 47, "f": 48, "h": 50, "i": 51, "j": 52, "k": 53, "l": 54, "m": 55,
+    "n": 56, "o": 57, "p": 58, "q": 59, "r": 60, "s": 61, "t": 62, "u": 63,
+    "v": 64, "w": 65, "x": 66, "y": 67, "z": 68, "ɑ": 69, "ɐ": 70, "ɒ": 71,
+    "æ": 72, "ɓ": 75, "ɔ": 76, "ɕ": 77, "ɖ": 78, "ɗ": 80, "ɘ": 81, "ə": 82,
+    "ɚ": 83, "ɛ": 86, "ɜ": 87, "ɞ": 90, "ɟ": 91, "ɡ": 92, "ɢ": 94, "ɣ": 95,
+    "ɤ": 96, "ɥ": 98, "ɦ": 99, "ɧ": 101, "ɨ": 102, "ɪ": 103, "ɫ": 110, "ɬ": 111,
+    "ŋ": 112, "ɲ": 113, "ɳ": 114, "ɴ": 115, "ɵ": 116, "œ": 118, "ɶ": 119,
+    "ø": 120, "ɸ": 123, "ɹ": 124, "ɾ": 125, "ɺ": 126, "ɻ": 128, "ɽ": 129,
+    "s": 130, "ʃ": 131, "ʂ": 132, "θ": 133, "ʈ": 135, "ʉ": 136, "ʊ": 138,
+    "ʋ": 139, "ʌ": 140, "ʍ": 142, "w": 143, "x": 145, "χ": 146, "ʒ": 147,
+    "ʐ": 148, "ʑ": 150, "ʔ": 156, "ᵻ": 177
+}
+
+# Assign Yoruba's 3 Tone Markers to verified vacant slots in Kokoro's embedding table
+YORUBA_KOKORO_VOCAB = dict(BASE_KOKORO_VOCAB)
+YORUBA_KOKORO_VOCAB["ˊ"] = 7   # High Tone (Mí)  -> Vacant Slot 7
+YORUBA_KOKORO_VOCAB["ˉ"] = 8   # Mid Tone  (Ré)  -> Vacant Slot 8
+YORUBA_KOKORO_VOCAB["ˋ"] = 26  # Low Tone  (Dò)  -> Vacant Slot 26
+
+
+def to_kokoro_tokens(phonemes_str: str) -> Tuple[List[int], List[str]]:
+    """
+    Takes an IPA phoneme string, decomposes nasal characters into (vowel + \\u0303),
+    and maps each symbol directly to its corresponding Kokoro token ID.
+
+    Returns:
+      (token_ids, missing_tokens)
+    """
+    # Decompose nasal vowels (e.g. ã -> a + combining tilde \u0303 in Slot 17)
+    normalized = unicodedata.normalize("NFD", phonemes_str)
+
+    token_ids: List[int] = []
+    missing_tokens: List[str] = []
+
+    for ch in normalized:
+        if ch in YORUBA_KOKORO_VOCAB:
+            token_ids.append(YORUBA_KOKORO_VOCAB[ch])
+        else:
+            missing_tokens.append(ch)
+
+    return token_ids, missing_tokens

oro_g2p-0.1.0/oro_g2p/normalizer.py

@@ -0,0 +1,52 @@
+"""
+oro_g2p.normalizer
+~~~~~~~~~~~~~~~~~~
+Unicode decomposition and diacritic normalization for Yoruba text.
+Handles:
+  - Standard combining underdot (\\u0323 on ẹ, ọ, ṣ)
+  - Common keyboard/OCR typo substitutions (cedilla \\u0327, vertical line \\u0329)
+  - Tone extraction (High, Low, Mid, Rising caron, Falling circumflex)
+"""
+
+import unicodedata
+
+# 1. Underdot variants found in keyboards, scanned texts, and African corpora
+UNDERDOT = "\u0323"         # Standard combining dot below
+CEDILLA = "\u0327"          # Combining cedilla (frequently typed on Turkish/Latin keyboards as ş, ȩ)
+VERT_LINE_BELOW = "\u0329"  # Combining vertical line below
+ALL_UNDERDOTS = {UNDERDOT, CEDILLA, VERT_LINE_BELOW}
+
+# 2. Tone Marks (Simple + Contour Glides)
+LOW_TONE = "\u0300"         # Grave (`) -> Low tone (Dò)
+HIGH_TONE = "\u0301"        # Acute (´) -> High tone (Mí)
+FALLING_TONE = "\u0302"     # Circumflex (ˆ) -> Falling contour (áà)
+MACRON_TONE = "\u0304"      # Macron (¯) -> Explicit Mid tone (Ré)
+RISING_TONE = "\u030c"      # Caron (ˇ) -> Rising contour (àá)
+ALL_TONES = {LOW_TONE, HIGH_TONE, FALLING_TONE, MACRON_TONE, RISING_TONE}
+
+
+def normalize_yoruba(text: str) -> list[str]:
+    """
+    Decomposes Yoruba text into canonical NFD building blocks.
+
+    Rules applied:
+    1. Un-glues letters, underdots, and tone diacritics via NFD.
+    2. Maps any underdot variant (cedilla \\u0327, vertical line \\u0329) on e, o, s
+       into the true canonical Yoruba underdot (\\u0323).
+    3. Handles out-of-order typing (e.g. tone typed before underdot).
+    4. Keeps tone diacritics separated right after their base character for downstream prosody.
+    """
+    decomposed = unicodedata.normalize("NFD", text.lower())
+
+    result: list[str] = []
+    for char in decomposed:
+        if char in ALL_UNDERDOTS and result:
+            # Check if a tone diacritic was typed BEFORE the underdot: [base, tone, underdot]
+            if result[-1] in ALL_TONES and len(result) >= 2 and result[-2] in {"e", "o", "s"}:
+                result[-2] = unicodedata.normalize("NFC", result[-2] + UNDERDOT)
+            elif result[-1] in {"e", "o", "s"}:
+                result[-1] = unicodedata.normalize("NFC", result[-1] + UNDERDOT)
+        else:
+            result.append(char)
+
+    return result

oro_g2p-0.1.0/oro_g2p/tokenizer.py

@@ -0,0 +1,166 @@
+"""
+oro_g2p.tokenizer
+~~~~~~~~~~~~~~~~~
+Linguistic tokenizer for Standard Yoruba.
+Encodes:
+  - Digraph preservation ('gb', 'sh' -> 'ṣ')
+  - Strict tone attachment to Tone-Bearing Units (TBUs)
+  - Nasal vowel classification (an, ẹn, in, ọn, un)
+  - Post-nasal vowel assimilation harmony ('m', 'n' -> 'ọmọ', 'inú', 'mí')
+  - Dynamic homorganic chameleon syllabic nasals ('ń', 'ǹ', 'm')
+"""
+
+from typing import Optional
+from .normalizer import (
+    normalize_yoruba,
+    ALL_TONES,
+    HIGH_TONE,
+    LOW_TONE,
+    MACRON_TONE,
+    RISING_TONE,
+    FALLING_TONE,
+)
+
+VOWELS = {"a", "e", "ẹ", "i", "o", "ọ", "u"}
+NASAL_MAP = {
+    "a": "an",
+    "ẹ": "ẹn",
+    "i": "in",
+    "ọ": "ọn",
+    "u": "un",
+    "o": "ọn",
+    "e": "ẹn",
+}
+CONSONANTS = {
+    "b", "d", "f", "g", "h", "j", "k", "l",
+    "m", "n", "p", "r", "s", "ṣ", "t", "w", "y"
+}
+
+
+def get_next_consonant(pieces: list[str], start_idx: int) -> Optional[str]:
+    """
+    Peeks ahead past spaces to determine the next consonant environment
+    for homorganic nasal assimilation.
+    """
+    j = start_idx
+    while j < len(pieces) and pieces[j] == " ":
+        j += 1
+    if j < len(pieces):
+        if pieces[j] == "g" and (j + 1 < len(pieces)) and pieces[j + 1] == "b":
+            return "gb"
+        if pieces[j] == "s" and (j + 1 < len(pieces)) and pieces[j + 1] == "h":
+            return "ṣ"
+        if pieces[j] in CONSONANTS:
+            return pieces[j]
+    return None
+
+
+def resolve_syllabic_nasal(next_cons: Optional[str]) -> str:
+    """
+    Applies homorganic place assimilation to syllabic nasals:
+      - Labials (b, f, m)                     -> 'm' (e.g. ń bọ̀ -> ḿ bọ̀)
+      - Alveolars (t, d, s, ṣ, j, n)          -> 'n' (e.g. ń ṣe -> ń ṣe)
+      - Velars, liquids, glides (k, g, gb, p, w, h, y, l, r) or pause -> 'ŋ' (e.g. Mo ń lọ -> Mo ŋ́ lọ)
+    """
+    if next_cons in {"b", "f", "m"}:
+        return "m"
+    elif next_cons in {"t", "d", "s", "ṣ", "j", "n"}:
+        return "n"
+    else:
+        return "ŋ"
+
+
+def tokenize_graphemes(text: str) -> list[tuple[str, Optional[str]]]:
+    """
+    Tokenizes Yoruba text into a list of (grapheme, tone) tuples.
+
+    Returns:
+      List of tuples: (grapheme_string, tone_symbol)
+      Where tone_symbol is 'H' (High), 'M' (Mid), 'L' (Low), or None for non-TBUs.
+    """
+    pieces = normalize_yoruba(text)
+    tokens: list[tuple[str, Optional[str]]] = []
+    i = 0
+    n = len(pieces)
+
+    while i < n:
+        ch = pieces[i]
+
+        # 1. Digraph checks: 'gb' and informal 'sh' -> 'ṣ'
+        if ch == "g" and (i + 1 < n) and pieces[i + 1] == "b":
+            tokens.append(("gb", None))
+            i += 2
+            continue
+        if ch == "s" and (i + 1 < n) and pieces[i + 1] == "h":
+            tokens.append(("ṣ", None))
+            i += 2
+            continue
+
+        # 2. Vowel handling (oral, nasal, contours, and post-nasal harmony)
+        if ch in VOWELS:
+            vowel = ch
+            tones = ["M"]  # Unmarked vowels are canonical Mid tone (Ré)
+            had_explicit_tone = False
+            i += 1
+
+            # Extract tone marks (including contour expansions)
+            if i < n and pieces[i] in ALL_TONES:
+                t_mark = pieces[i]
+                had_explicit_tone = True
+                if t_mark == HIGH_TONE:
+                    tones = ["H"]
+                elif t_mark == LOW_TONE:
+                    tones = ["L"]
+                elif t_mark == MACRON_TONE:
+                    tones = ["M"]
+                elif t_mark == RISING_TONE:
+                    tones = ["L", "H"]  # Rising caron: àá
+                elif t_mark == FALLING_TONE:
+                    tones = ["H", "L"]  # Falling circumflex: áà
+                i += 1
+
+            # Nasal vowel checking (vowel + 'n')
+            if i < n and pieces[i] == "n" and vowel in NASAL_MAP:
+                next_1 = pieces[i + 1] if (i + 1 < n) else None
+                next_2 = pieces[i + 2] if (i + 2 < n) else None
+
+                # Standard nasal vowel: 'n' not followed by another vowel or tone
+                if next_1 not in VOWELS and next_1 not in ALL_TONES:
+                    vowel = NASAL_MAP[vowel]
+                    i += 1
+                # Catch dataset typos: tone mistakenly placed on word-final 'n' (e.g. ìbọǹ)
+                elif (not had_explicit_tone) and (next_1 in {HIGH_TONE, LOW_TONE}) and (next_2 not in VOWELS and next_2 not in CONSONANTS):
+                    vowel = NASAL_MAP[vowel]
+                    tones = ["H"] if next_1 == HIGH_TONE else ["L"]
+                    i += 2
+
+            # Post-nasal harmony: oral vowels immediately following 'm' or 'n' automatically nasalize
+            if tokens and tokens[-1] in {("m", None), ("n", None)} and vowel in {"a", "ẹ", "i", "ọ", "u"}:
+                vowel = NASAL_MAP[vowel]
+
+            for t in tones:
+                tokens.append((vowel, t))
+            continue
+
+        # 3. Syllabic nasals ('n' or 'm' as independent tone-bearing syllables)
+        if ch in ("n", "m"):
+            if (i + 1 < n) and pieces[i + 1] in ALL_TONES:
+                t_mark = pieces[i + 1]
+                tone = "H" if t_mark == HIGH_TONE else ("L" if t_mark == LOW_TONE else "M")
+                next_cons = get_next_consonant(pieces, i + 2)
+                nasal_sound = "m" if ch == "m" else resolve_syllabic_nasal(next_cons)
+                tokens.append((nasal_sound, tone))
+                i += 2
+                continue
+            elif (i + 1 < n) and pieces[i + 1] in CONSONANTS:
+                next_cons = get_next_consonant(pieces, i + 1)
+                nasal_sound = "m" if ch == "m" else resolve_syllabic_nasal(next_cons)
+                tokens.append((nasal_sound, "M"))
+                i += 1
+                continue
+
+        # 4. Consonants, punctuation, spaces
+        tokens.append((ch, None))
+        i += 1
+
+    return tokens

oro_g2p-0.1.0/oro_g2p.egg-info/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: oro-g2p
+Version: 0.1.0
+Summary: Research-grade Yoruba Grapheme-to-Phoneme (G2P) converter and Kokoro TTS frontend
+Author: Oro Contributors
+License: MIT
+Project-URL: Homepage, https://github.com/Akintunde102/oro-g2p
+Project-URL: Repository, https://github.com/Akintunde102/oro-g2p
+Project-URL: Issues, https://github.com/Akintunde102/oro-g2p/issues
+Keywords: yoruba,g2p,tts,kokoro,phonemes,african-nlp,speech-synthesis,phonology
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# oro-g2p 🗣️
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python: 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/)
+
+**`oro-g2p`** is a zero-dependency, research-grade Yoruba Grapheme-to-Phoneme (G2P) engine and phonetic tokenizer designed for speech synthesis (TTS), automatic speech recognition (ASR), and forced alignment.
+
+It provides out-of-the-box native token mapping for **[Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)** and **StyleTTS2**.
+
+---
+
+## Key Features
+
+1. **Deterministic Linguistic Engine:** Pure, rule-based Yoruba phonology — zero hallucinations, reproducible output.
+2. **Post-Nasal Harmony:** Automatically captures unwritten phonetic nasalization on vowels following nasal consonants `m` and `n` (*ọmọ* → `/ɔˉmɔ̃ˉ/`, *inú* → `/iˉnũˊ/`).
+3. **Chameleon Syllabic Nasals:** Dynamically applies homorganic place assimilation to `ń` and `ǹ`:
+   - Labials (`b`, `f`): `/m/` (*Mo ń bọ̀* → `moˉ mˊ bɔˋ`)
+   - Alveolars (`d`, `t`, `s`, `ṣ`): `/n/` (*Mo ń ṣe* → `moˉ nˊ ʃeˉ`)
+   - Velars & Glides (`l`, `k`, `g`): `/ŋ/` (*Mo ń lọ* → `moˉ ŋˊ lɔˉ`)
+4. **Typo Resilience:** Seamlessly normalizes Turkish cedillas (`ş`), combining vertical lines (`\u0329`), and out-of-order diacritic keystrokes into canonical Yoruba underdots (`ẹ`, `ọ`, `ṣ`).
+5. **Contour Glides:** Expands carons (`ǎ` → `àá` Low-High) and circumflexes (`â` → `áà` High-Low).
+6. **Native Kokoro Alignment:** Maps 100% of Yoruba phonemes and tones directly into Kokoro-82M's 178-token embedding table with **0 missing tokens**.
+
+---
+
+## Installation
+
+```bash
+pip install oro-g2p
+```
+
+---
+
+## Quickstart
+
+### 1. Basic G2P Conversion
+```python
+from oro_g2p import g2p
+
+# Converts Yoruba text to tone-marked IPA
+phonemes = g2p("Ẹ kú àárọ̀, ọmọ Yorùbá!")
+print(phonemes)
+# Output: 'ɛˉ kuˊ aˋaˊɾɔˋ, ɔˉmɔ̃ˉ joˉɾuˋbaˊ!'
+```
+
+### 2. Kokoro-82M Tokenization
+```python
+from oro_g2p import yoruba_to_kokoro
+
+text = "Ẹ kú àárọ̀"
+ipa_str, token_ids, missing = yoruba_to_kokoro(text)
+
+print(f"IPA:       {ipa_str}")
+print(f"Token IDs: {token_ids}")
+print(f"Missing:   {missing}")  # [] -> 100% coverage!
+```
+
+---
+
+## Architecture: Kokoro-82M Token Mapping
+
+Kokoro has a fixed embedding matrix size of `178` (`0` to `177`). While 114 slots were occupied by Western languages, 64 slots were unassigned.
+
+`oro-g2p` resolves Yoruba without modifying any pre-trained weights or embedding tensor dimensions:
+- **High Tone (Mí):** Assigned to Vacant Slot `7` (`ˊ`)
+- **Mid Tone (Ré):** Assigned to Vacant Slot `8` (`ˉ`)
+- **Low Tone (Dò):** Assigned to Vacant Slot `26` (`ˋ`)
+- **Nasal Vowels:** Decomposed into `(vowel + \u0303)`, cleanly utilizing Kokoro's native Room `17`.
+
+---
+
+## Running Tests
+
+```bash
+python -m pytest tests/
+```
+
+---
+
+## License
+
+MIT License. Free for academic, personal, and commercial speech technology research.

oro_g2p-0.1.0/oro_g2p.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+LICENSE
+README.md
+pyproject.toml
+oro_g2p/__init__.py
+oro_g2p/kokoro.py
+oro_g2p/normalizer.py
+oro_g2p/tokenizer.py
+oro_g2p.egg-info/PKG-INFO
+oro_g2p.egg-info/SOURCES.txt
+oro_g2p.egg-info/dependency_links.txt
+oro_g2p.egg-info/top_level.txt
+tests/test_g2p.py

oro_g2p-0.1.0/oro_g2p.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

oro_g2p-0.1.0/oro_g2p.egg-info/top_level.txt

@@ -0,0 +1 @@
+oro_g2p

oro_g2p-0.1.0/pyproject.toml

@@ -0,0 +1,47 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "oro-g2p"
+version = "0.1.0"
+authors = [
+    { name = "Oro Contributors" }
+]
+description = "Research-grade Yoruba Grapheme-to-Phoneme (G2P) converter and Kokoro TTS frontend"
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+keywords = [
+    "yoruba",
+    "g2p",
+    "tts",
+    "kokoro",
+    "phonemes",
+    "african-nlp",
+    "speech-synthesis",
+    "phonology"
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Text Processing :: Linguistic",
+]
+dependencies = []
+
+[project.urls]
+Homepage = "https://github.com/Akintunde102/oro-g2p"
+Repository = "https://github.com/Akintunde102/oro-g2p"
+Issues = "https://github.com/Akintunde102/oro-g2p/issues"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["oro_g2p*"]

oro_g2p-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

oro_g2p-0.1.0/tests/test_g2p.py

@@ -0,0 +1,91 @@
+"""
+Automated test suite for oro_g2p covering all 6 Yoruba phonological quirks
+and Kokoro token alignment. (Zero-dependency runner)
+"""
+
+from oro_g2p import g2p, tokenize_graphemes, normalize_yoruba, yoruba_to_kokoro
+
+
+def test_basic_sentence():
+    """Verify standard greetings and canonical vowels/tones."""
+    text = "Ẹ kú àárọ̀"
+    phonemes = g2p(text)
+    assert "ɛˉ" in phonemes   # 'ẹ' with Mid tone
+    assert "kuˊ" in phonemes  # 'kú' with High tone
+    assert "aˋaˊ" in phonemes # 'àá' with Low then High
+
+
+def test_digraph_gb_and_kp():
+    """Verify 'gb' digraph stays intact and 'p' is pronounced /kp/."""
+    text = "igbá àti páńkẹ́rẹ́"
+    phonemes = g2p(text)
+    assert "ɡb" in phonemes
+    assert "kp" in phonemes
+    assert " p" not in phonemes  # English 'p' should not exist in native Yoruba
+
+
+def test_post_nasal_harmony():
+    """
+    Quirk 1: Vowels immediately following consonant 'm' or 'n'
+    must automatically nasalize (ọmọ -> ɔ̃, inú -> ũ, mi -> ĩ).
+    """
+    text = "ọmọ inú mi"
+    phonemes = g2p(text)
+    assert "mɔ̃" in phonemes  # 'ọ' after 'm' becomes nasal 'ɔ̃'
+    assert "nũ" in phonemes  # 'ú' after 'n' becomes nasal 'ũ'
+    assert "mĩ" in phonemes  # 'i' after 'm' becomes nasal 'ĩ'
+
+
+def test_chameleon_syllabic_nasal():
+    """
+    Quirk 2: Syllabic nasal homorganic assimilation.
+      - Before labials (b, f) -> 'm'
+      - Before tongue-tip (ṣ, d, t) -> 'n'
+      - Before throat/pause/liquids (l, k, g) -> 'ŋ'
+    """
+    assert g2p("Mo ń bọ̀").startswith("moˉ mˊ")
+    assert "nˊ" in g2p("Mo ń ṣe")
+    assert "ŋˊ" in g2p("Mo ń lọ")
+
+
+def test_contour_glides():
+    """
+    Quirk 3: Rising caron (ˇ) expands to Low-High,
+    Falling circumflex (ˆ) expands to High-Low.
+    """
+    # ǎ -> àá (Low-High)
+    phonemes_rising = g2p("dǎ")
+    assert "aˋaˊ" in phonemes_rising
+
+    # â -> áà (High-Low)
+    phonemes_falling = g2p("dâ")
+    assert "aˊaˋ" in phonemes_falling
+
+
+def test_keyboard_typo_resilience():
+    """
+    Quirk 4: Typo substitutions like Turkish cedilla 'ş'
+    must be cleanly normalized to Yoruba 'ṣ' (/ʃ/).
+    """
+    typo_text = "Şé o wà?"
+    clean_text = "Ṣé o wà?"
+    assert g2p(typo_text) == g2p(clean_text)
+    assert "ʃ" in g2p(typo_text)
+
+
+def test_kokoro_zero_missing_tokens():
+    """
+    Verify complete 100% token coverage in Kokoro's embedding table
+    with zero missing tokens.
+    """
+    test_corpus = [
+        "Ẹ kú àárọ̀, gbogbo ọmọ Yorùbá!",
+        "Tòmátì àti Ṣòmbọ̀ dídùn.",
+        "Mo ń bọ̀ ní ìrọ̀lẹ́ òní.",
+        "Máalù jẹ koríko lẹ́gbẹ̀ẹ́ igi.",
+        "Báwo ni nǹkan? Şé àlàáfíà ni?",
+    ]
+    for sentence in test_corpus:
+        ipa_str, token_ids, missing = yoruba_to_kokoro(sentence)
+        assert len(missing) == 0, f"Failed on '{sentence}'! Missing: {missing}"
+        assert len(token_ids) > 0