naijanorm 1.0.0__tar.gz

naijanorm-1.0.0/.github/workflows/tests.yml

@@ -0,0 +1,16 @@
+name: tests
+on: [push, pull_request]
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: python -m pip install --upgrade pip
+      - run: pip install -e ".[dev]"
+      - run: pytest

naijanorm-1.0.0/CHANGELOG.md

@@ -0,0 +1,9 @@
+# Changelog
+
+## 1.0.0
+- Initial five-language cardinal number profiles.
+- Nigerian currency, phone, shortcode, percentage, decimal and unit rules.
+- Batch API and normalization event output.
+- Configurable Yoruba traditional Oyo / modern decimal profile.
+- Test and packaging scaffold.
+- Explicitly documents linguistic and contextual limitations.

naijanorm-1.0.0/LICENSE

@@ -0,0 +1,17 @@
+MIT License
+
+Copyright (c) 2026 NaijaNorm contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

naijanorm-1.0.0/PKG-INFO

@@ -0,0 +1,185 @@
+Metadata-Version: 2.5
+Name: naijanorm
+Version: 1.0.0
+Summary: Nigerian-context text normalization for speech and language AI
+Project-URL: Homepage, https://github.com/YOUR-ORG/naijanorm
+Project-URL: Issues, https://github.com/YOUR-ORG/naijanorm/issues
+Author: NaijaNorm contributors
+License: MIT License
+        
+        Copyright (c) 2026 NaijaNorm contributors
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+License-File: LICENSE
+Keywords: ASR,Hausa,Igbo,Nigerian languages,Pidgin,TTS,Yoruba,text normalization
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.9
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: ruff>=0.5; extra == 'dev'
+Provides-Extra: excel
+Requires-Dist: openpyxl>=3.1; extra == 'excel'
+Requires-Dist: pandas>=2.0; extra == 'excel'
+Description-Content-Type: text/markdown
+
+# NaijaNorm
+
+Nigerian-context text normalization for ASR, TTS, NLP and conversational AI.
+
+> **Release note:** NaijaNorm 1.0.0 provides working baseline cardinal number generation
+> for Nigerian English, Nigerian Pidgin, Yoruba, Hausa and Igbo, plus context rules for
+> common numeric forms. It is an alpha-quality linguistic baseline, not a claim that every
+> regional form, ordinal, date convention or spoken variant has been exhaustively validated.
+> Before production dataset generation, run native-speaker review and add gold references.
+
+## Install
+
+```bash
+pip install naijanorm
+```
+
+For local development:
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## Quick start
+
+```python
+from naijanorm import NaijaNorm
+
+normalizer = NaijaNorm(language="en-NG")
+print(normalizer.normalize("Pay ₦5,000 for 250MB and dial *310#"))
+```
+
+## Five language profiles
+
+```python
+from naijanorm import NaijaNorm
+
+for language in ("en-NG", "pcm-NG", "yo-NG", "ha-NG", "ig-NG"):
+    n = NaijaNorm(language=language)
+    print(language, n.normalize("123"))
+```
+
+Language identifiers:
+- `en-NG`: Nigerian English
+- `pcm-NG`: Nigerian Pidgin (practical baseline; spelling is not fully standardized)
+- `yo-NG`: Yoruba, Standard Oyo traditional counting by default
+- `ha-NG`: Hausa, standard Boko baseline
+- `ig-NG`: Igbo, standard orthographic baseline
+
+## Yoruba counting profile
+
+Traditional Standard Oyo is selected by default:
+
+```python
+yoruba = NaijaNorm("yo-NG", yoruba_system="oyo-traditional")
+print(yoruba.normalize("16"))  # ẹ̀rìndínlógún
+```
+
+For modern decimal/additive reading:
+
+```python
+yoruba_modern = NaijaNorm("yo-NG", yoruba_system="modern-decimal")
+```
+
+Yoruba numeral forms vary by region, register and context. Traditional forms are not always the form speakers choose when reading a phone number, price, mathematical expression or serial number. Use category-aware speech policy and native-speaker validation.
+
+## Context examples
+
+```python
+n = NaijaNorm("en-NG")
+samples = [
+    "250MB",
+    "₦5,000",
+    "08012345678",
+    "25th",
+    "3.5GB",
+    "*310#",
+    "75%",
+]
+for sample in samples:
+    print(sample, "=>", n.normalize(sample))
+```
+
+## Events and audit trail
+
+```python
+result = n.normalize_with_events("₦5,000 and 250MB")
+print(result.text)
+for event in result.events:
+    print(event.original, event.normalized, event.category)
+```
+
+## Select categories
+
+```python
+n = NaijaNorm("en-NG", categories=frozenset({"currency", "telephone"}))
+```
+
+Or:
+
+```python
+n.normalize("₦5,000 and 250MB", categories=["currency"])
+```
+
+## Batch processing
+
+```python
+sentences = ["I bought 3GB", "Send ₦2,500", "Call 08012345678"]
+normalized = n.normalize_batch(sentences)
+```
+
+## Important limitations and safe use
+
+1. **Ambiguity:** A bare `2026` may be a year, quantity, identifier or model number. The default treats bare numbers as cardinals. Use protected tokens or category controls when identifiers must remain unchanged.
+2. **Ordinals:** English ordinal suffixes are supported. The other languages currently return a cardinal plus a review note; full ordinal morphology requires language-specific validated rules.
+3. **Yoruba:** The Oyo profile includes a traditional vigesimal strategy and common forms, but compounds above the well-attested table should be reviewed. Do not assume one form is universal across Yoruba dialects.
+4. **Hausa and Igbo:** The included algorithms are practical standard profiles. Regional variation and alternative counting conventions require separate tested lexicons.
+5. **Nigerian Pidgin:** Orthography is not uniformly standardized. The package uses one consistent practical spelling profile; projects should publish their chosen style guide.
+6. **Dates and times:** Automatic date/time expansion is deliberately not enabled in this first baseline because numeric date order is ambiguous (DD/MM vs MM/DD), and speech readings depend on context.
+7. **Phone numbers:** Recognizes common Nigerian 11-digit local numbers and +234 formats. It does not validate allocation, carrier or whether a number is active.
+8. **Currencies:** Currency symbol expansion is supported; currency codes and complex mixed-currency accounting expressions need explicit project rules.
+9. **Units:** Common metric and telecom units are supported. Unit case is normalized for matching; abbreviations embedded in words are not intentionally expanded.
+
+Do not use the default pipeline blindly on OTPs, bank account numbers, NINs, BVNs, transaction references, postal codes, coordinates, URLs or product identifiers. These need explicit token classes and project policies.
+
+## Contributing linguistic data
+
+Every language rule should have:
+- a language and dialect/standard label;
+- a source or native-speaker reviewer;
+- examples with expected output;
+- Unicode NFC tests;
+- tests for numbers around every scale boundary;
+- tests for code-switching and preservation.
+
+Please submit corrections with audio or a clear orthographic convention where possible.
+
+## License
+
+MIT. See `LICENSE`.

naijanorm-1.0.0/README.md

@@ -0,0 +1,140 @@
+# NaijaNorm
+
+Nigerian-context text normalization for ASR, TTS, NLP and conversational AI.
+
+> **Release note:** NaijaNorm 1.0.0 provides working baseline cardinal number generation
+> for Nigerian English, Nigerian Pidgin, Yoruba, Hausa and Igbo, plus context rules for
+> common numeric forms. It is an alpha-quality linguistic baseline, not a claim that every
+> regional form, ordinal, date convention or spoken variant has been exhaustively validated.
+> Before production dataset generation, run native-speaker review and add gold references.
+
+## Install
+
+```bash
+pip install naijanorm
+```
+
+For local development:
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## Quick start
+
+```python
+from naijanorm import NaijaNorm
+
+normalizer = NaijaNorm(language="en-NG")
+print(normalizer.normalize("Pay ₦5,000 for 250MB and dial *310#"))
+```
+
+## Five language profiles
+
+```python
+from naijanorm import NaijaNorm
+
+for language in ("en-NG", "pcm-NG", "yo-NG", "ha-NG", "ig-NG"):
+    n = NaijaNorm(language=language)
+    print(language, n.normalize("123"))
+```
+
+Language identifiers:
+- `en-NG`: Nigerian English
+- `pcm-NG`: Nigerian Pidgin (practical baseline; spelling is not fully standardized)
+- `yo-NG`: Yoruba, Standard Oyo traditional counting by default
+- `ha-NG`: Hausa, standard Boko baseline
+- `ig-NG`: Igbo, standard orthographic baseline
+
+## Yoruba counting profile
+
+Traditional Standard Oyo is selected by default:
+
+```python
+yoruba = NaijaNorm("yo-NG", yoruba_system="oyo-traditional")
+print(yoruba.normalize("16"))  # ẹ̀rìndínlógún
+```
+
+For modern decimal/additive reading:
+
+```python
+yoruba_modern = NaijaNorm("yo-NG", yoruba_system="modern-decimal")
+```
+
+Yoruba numeral forms vary by region, register and context. Traditional forms are not always the form speakers choose when reading a phone number, price, mathematical expression or serial number. Use category-aware speech policy and native-speaker validation.
+
+## Context examples
+
+```python
+n = NaijaNorm("en-NG")
+samples = [
+    "250MB",
+    "₦5,000",
+    "08012345678",
+    "25th",
+    "3.5GB",
+    "*310#",
+    "75%",
+]
+for sample in samples:
+    print(sample, "=>", n.normalize(sample))
+```
+
+## Events and audit trail
+
+```python
+result = n.normalize_with_events("₦5,000 and 250MB")
+print(result.text)
+for event in result.events:
+    print(event.original, event.normalized, event.category)
+```
+
+## Select categories
+
+```python
+n = NaijaNorm("en-NG", categories=frozenset({"currency", "telephone"}))
+```
+
+Or:
+
+```python
+n.normalize("₦5,000 and 250MB", categories=["currency"])
+```
+
+## Batch processing
+
+```python
+sentences = ["I bought 3GB", "Send ₦2,500", "Call 08012345678"]
+normalized = n.normalize_batch(sentences)
+```
+
+## Important limitations and safe use
+
+1. **Ambiguity:** A bare `2026` may be a year, quantity, identifier or model number. The default treats bare numbers as cardinals. Use protected tokens or category controls when identifiers must remain unchanged.
+2. **Ordinals:** English ordinal suffixes are supported. The other languages currently return a cardinal plus a review note; full ordinal morphology requires language-specific validated rules.
+3. **Yoruba:** The Oyo profile includes a traditional vigesimal strategy and common forms, but compounds above the well-attested table should be reviewed. Do not assume one form is universal across Yoruba dialects.
+4. **Hausa and Igbo:** The included algorithms are practical standard profiles. Regional variation and alternative counting conventions require separate tested lexicons.
+5. **Nigerian Pidgin:** Orthography is not uniformly standardized. The package uses one consistent practical spelling profile; projects should publish their chosen style guide.
+6. **Dates and times:** Automatic date/time expansion is deliberately not enabled in this first baseline because numeric date order is ambiguous (DD/MM vs MM/DD), and speech readings depend on context.
+7. **Phone numbers:** Recognizes common Nigerian 11-digit local numbers and +234 formats. It does not validate allocation, carrier or whether a number is active.
+8. **Currencies:** Currency symbol expansion is supported; currency codes and complex mixed-currency accounting expressions need explicit project rules.
+9. **Units:** Common metric and telecom units are supported. Unit case is normalized for matching; abbreviations embedded in words are not intentionally expanded.
+
+Do not use the default pipeline blindly on OTPs, bank account numbers, NINs, BVNs, transaction references, postal codes, coordinates, URLs or product identifiers. These need explicit token classes and project policies.
+
+## Contributing linguistic data
+
+Every language rule should have:
+- a language and dialect/standard label;
+- a source or native-speaker reviewer;
+- examples with expected output;
+- Unicode NFC tests;
+- tests for numbers around every scale boundary;
+- tests for code-switching and preservation.
+
+Please submit corrections with audio or a clear orthographic convention where possible.
+
+## License
+
+MIT. See `LICENSE`.

naijanorm-1.0.0/pyproject.toml

@@ -0,0 +1,44 @@
+[build-system]
+requires = ["hatchling>=1.24"]
+build-backend = "hatchling.build"
+
+[project]
+name = "naijanorm"
+version = "1.0.0"
+description = "Nigerian-context text normalization for speech and language AI"
+readme = "README.md"
+requires-python = ">=3.9"
+license = { file = "LICENSE" }
+authors = [{ name = "NaijaNorm contributors" }]
+keywords = ["Nigerian languages", "text normalization", "ASR", "TTS", "Yoruba", "Hausa", "Igbo", "Pidgin"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "Topic :: Text Processing :: Linguistic",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.9",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Operating System :: OS Independent",
+]
+dependencies = []
+
+[project.optional-dependencies]
+excel = ["pandas>=2.0", "openpyxl>=3.1"]
+dev = ["pytest>=8.0", "build>=1.2", "ruff>=0.5"]
+
+[project.urls]
+Homepage = "https://github.com/YOUR-ORG/naijanorm"
+Issues = "https://github.com/YOUR-ORG/naijanorm/issues"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/naijanorm"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-q"
+
+[tool.ruff]
+line-length = 100
+target-version = "py39"

naijanorm-1.0.0/src/naijanorm/__init__.py

@@ -0,0 +1,10 @@
+"""NaijaNorm: Nigerian-context text normalization."""
+from .normalizer import NaijaNorm, normalize, normalize_batch
+from .config import NormalizerConfig
+from .result import NormalizationResult, NormalizationEvent
+
+__version__ = "1.0.0"
+__all__ = [
+    "NaijaNorm", "normalize", "normalize_batch", "NormalizerConfig",
+    "NormalizationResult", "NormalizationEvent", "__version__"
+]

naijanorm-1.0.0/src/naijanorm/config.py

@@ -0,0 +1,27 @@
+from dataclasses import dataclass, field
+from typing import FrozenSet, Optional
+
+SUPPORTED_LANGUAGES = ("en-NG", "pcm-NG", "yo-NG", "ha-NG", "ig-NG")
+
+@dataclass
+class NormalizerConfig:
+    language: str = "en-NG"
+    yoruba_system: str = "oyo-traditional"  # or "modern-decimal"
+    igbo_variant: str = "standard"           # regional variants require separate lexicons
+    hausa_variant: str = "standard-boko"
+    categories: Optional[FrozenSet[str]] = None
+    preserve_unknown: bool = True
+    expand_units: bool = True
+    expand_currency: bool = True
+    expand_telephone: bool = True
+    decimal_style: str = "point"
+    include_events: bool = False
+    max_integer: int = 999_999_999_999
+
+    def __post_init__(self):
+        if self.language not in SUPPORTED_LANGUAGES:
+            raise ValueError(
+                f"Unsupported language {self.language!r}. Choose from {SUPPORTED_LANGUAGES}."
+            )
+        if self.yoruba_system not in {"oyo-traditional", "modern-decimal"}:
+            raise ValueError("yoruba_system must be 'oyo-traditional' or 'modern-decimal'")

naijanorm-1.0.0/src/naijanorm/normalizer.py

@@ -0,0 +1,156 @@
+import re
+from typing import Iterable, Optional, List
+from .config import NormalizerConfig
+from .numbers import spell_integer
+from .result import NormalizationEvent, NormalizationResult
+
+UNIT_MAP = {
+    "kb": ("kilobyte", "kilobytes"), "mb": ("megabyte", "megabytes"),
+    "gb": ("gigabyte", "gigabytes"), "tb": ("terabyte", "terabytes"),
+    "kbps": ("kilobit per second", "kilobits per second"),
+    "mbps": ("megabit per second", "megabits per second"),
+    "gbps": ("gigabit per second", "gigabits per second"),
+    "kg": ("kilogram", "kilograms"), "g": ("gram", "grams"),
+    "km": ("kilometre", "kilometres"), "cm": ("centimetre", "centimetres"),
+    "mm": ("millimetre", "millimetres"), "m": ("metre", "metres"),
+    "l": ("litre", "litres"), "ml": ("millilitre", "millilitres"),
+    "°c": ("degree Celsius", "degrees Celsius"),
+    "hrs": ("hour", "hours"), "hr": ("hour", "hours"),
+    "mins": ("minute", "minutes"), "min": ("minute", "minutes"),
+}
+CURRENCY = {"₦": ("naira", "naira"), "$": ("US dollar", "US dollars"),
+            "£": ("pound sterling", "pounds sterling"), "€": ("euro", "euros")}
+DIGIT_WORDS = {
+    "en-NG": ["zero","one","two","three","four","five","six","seven","eight","nine"],
+    "pcm-NG": ["zero","wan","tu","tri","fo","faiv","siks","sevn","et","nain"],
+    "yo-NG": ["odo","ọ̀kan","èjì","ẹ̀ta","ẹ̀rin","àrún-ún","ẹ̀fà","èje","ẹ̀jọ","ẹ̀sàn-án"],
+    "ha-NG": ["sifili","ɗaya","biyu","uku","huɗu","biyar","shida","bakwai","takwas","tara"],
+    "ig-NG": ["efu","otu","abụọ","atọ","anọ","ise","isii","asaa","asatọ","itoolu"],
+}
+SHORTCODE_WORDS = {"*":"star", "#":"hash", "+":"plus"}
+
+class NaijaNorm:
+    """Context-aware Nigerian text normalizer.
+
+    This is a deterministic rule-based baseline. Language profiles are explicit;
+    dialect-sensitive forms should be reviewed by native speakers for production.
+    """
+    def __init__(self, language: str = "en-NG", **kwargs):
+        self.config = NormalizerConfig(language=language, **kwargs)
+
+    def _num(self, token: str) -> str:
+        clean = token.replace(",", "").replace("_", "")
+        if "." in clean:
+            left, right = clean.split(".", 1)
+            return spell_integer(int(left), self.config.language, self.config.yoruba_system) + \
+                   " point " + " ".join(DIGIT_WORDS[self.config.language][int(x)] for x in right if x.isdigit())
+        return spell_integer(int(clean), self.config.language, self.config.yoruba_system)
+
+    def normalize_with_events(self, text: str) -> NormalizationResult:
+        if not isinstance(text, str):
+            raise TypeError("text must be a string")
+        events: List[NormalizationEvent] = []
+        lang = self.config.language
+
+        def replace(pattern, fn, category):
+            nonlocal text
+            if self.config.categories is not None and category not in self.config.categories:
+                return
+            def sub(m):
+                original = m.group(0)
+                normalized = fn(m)
+                events.append(NormalizationEvent(original, normalized, category, lang))
+                return normalized
+            text = re.sub(pattern, sub, text, flags=re.IGNORECASE)
+
+        # Telephone numbers: preserve every digit; recognize common Nigerian formats.
+        if self.config.expand_telephone:
+            def phone(m):
+                raw = re.sub(r"\D", "", m.group(0))
+                if raw.startswith("234") and len(raw) == 13:
+                    digits = "0" + raw[3:]
+                else:
+                    digits = raw
+                return " ".join(DIGIT_WORDS[lang][int(d)] for d in digits)
+            replace(r"(?<!\w)(?:\+?234[\s-]?\d{3}[\s-]?\d{3}[\s-]?\d{4}|0\d{10})(?!\w)", phone, "telephone")
+
+        # USSD / short codes. Keep symbol semantics explicit.
+        replace(r"(?<!\w)\*\d{2,6}#", lambda m: " ".join(
+            [SHORTCODE_WORDS["*"]] +
+            [DIGIT_WORDS[lang][int(d)] for d in m.group(0)[1:-1]] +
+            [SHORTCODE_WORDS["#"]]
+        ), "shortcode")
+
+        # Currency values, including decimals and grouped thousands.
+        if self.config.expand_currency:
+            replace(r"[₦$£€]\s?-?\d[\d,]*(?:\.\d+)?",
+                lambda m: self._num(re.search(r"-?\d[\d,]*(?:\.\d+)?", m.group(0)).group(0)) +
+                " " + CURRENCY[m.group(0)[0]][1], "currency")
+
+        # Percentages.
+        replace(r"(?<!\w)-?\d[\d,]*(?:\.\d+)?\s*%", 
+                lambda m: self._num(re.sub(r"\s*%","",m.group(0))) + " percent", "percentage")
+
+        # Ordinals (English suffix notation). Non-English ordinal morphology is not
+        # guessed; use cardinal fallback with a review event for now.
+        def ordinal(m):
+            n = int(m.group(1))
+            if lang == "en-NG":
+                if 10 <= n % 100 <= 20: suffix = "th"
+                else: suffix = {1:"st",2:"nd",3:"rd"}.get(n % 10,"th")
+                words = self._num(str(n))
+                if n in (1, 2, 3):
+                    return {1: "first", 2: "second", 3: "third"}[n]
+                if n % 100 in (11, 12, 13):
+                    return words + "-th"
+                last = n % 10
+                irregular = {5: "fifth", 8: "eighth", 9: "ninth", 12: "twelfth"}
+                if last in irregular and n < 100:
+                    prefix = words.rsplit("-", 1)[0] if "-" in words else ""
+                    return (prefix + "-" if prefix else "") + irregular[last]
+                if last == 1:
+                    return words[:-3] + "first" if words.endswith("one") else words + "-first"
+                if last == 2:
+                    return words[:-3] + "second" if words.endswith("two") else words + "-second"
+                if last == 3:
+                    return words[:-5] + "third" if words.endswith("three") else words + "-third"
+                return words + "-th"
+            return self._num(str(n)) + " (ordinal form requires language-specific review)"
+        replace(r"\b(\d+)(st|nd|rd|th)\b", ordinal, "ordinal")
+
+        # Measurement and data units. Avoid matching substrings in words.
+        if self.config.expand_units:
+            unit_pattern = r"(?<!\w)(-?\d[\d,]*(?:\.\d+)?)\s*(KBPS|MBPS|GBPS|KB|MB|GB|TB|KG|G|KM|CM|MM|ML|L|M|°C|HRS|HR|MINS|MIN)\b"
+            def unit(m):
+                key=m.group(2).lower()
+                singular, plural=UNIT_MAP[key]
+                amount=m.group(1)
+                return self._num(amount) + " " + (singular if amount.replace(",","") in ("1","1.0") else plural)
+            replace(unit_pattern, unit, "measurement")
+
+        # Plain decimals before plain integers.
+        replace(r"(?<![\w.])-?\d[\d,]*\.\d+(?!\w)", lambda m: self._num(m.group(0)), "decimal")
+
+        # Remaining cardinal integers. Protect years/IDs only when embedded in letters;
+        # bare numbers are treated as cardinals.
+        replace(r"(?<![\w])\d{1,12}(?![\w])", lambda m: self._num(m.group(0)), "cardinal")
+
+        return NormalizationResult(text=text, events=events)
+
+    def normalize(self, text: str, categories: Optional[Iterable[str]] = None) -> str:
+        if categories is None:
+            return self.normalize_with_events(text).text
+        options = self.config.__dict__.copy()
+        options.pop("language", None)
+        options["categories"] = frozenset(categories)
+        clone = NaijaNorm(self.config.language, **options)
+        return clone.normalize_with_events(text).text
+
+    def normalize_batch(self, texts: Iterable[str]) -> List[str]:
+        return [self.normalize(t) for t in texts]
+
+def normalize(text: str, language="en-NG", **kwargs) -> str:
+    return NaijaNorm(language, **kwargs).normalize(text)
+
+def normalize_batch(texts: Iterable[str], language="en-NG", **kwargs) -> List[str]:
+    return NaijaNorm(language, **kwargs).normalize_batch(texts)

naijanorm-1.0.0/src/naijanorm/numbers.py

@@ -0,0 +1,170 @@
+"""Language-specific cardinal number spell-out.
+
+Important: language numerals have dialectal and register variation. The lexicons
+here are an initial documented standard profile, not a substitute for validation
+by native-speaking linguists before safety-critical or commercial corpus release.
+"""
+from typing import Dict
+
+EN_ONES = ["zero","one","two","three","four","five","six","seven","eight","nine"]
+EN_TEENS = ["ten","eleven","twelve","thirteen","fourteen","fifteen","sixteen","seventeen","eighteen","nineteen"]
+EN_TENS = ["","","twenty","thirty","forty","fifty","sixty","seventy","eighty","ninety"]
+SCALES_EN = [(10**9,"billion"),(10**6,"million"),(1000,"thousand"),(100,"hundred")]
+
+def en(n: int) -> str:
+    if n < 0: return "minus " + en(-n)
+    if n < 10: return EN_ONES[n]
+    if n < 20: return EN_TEENS[n-10]
+    if n < 100:
+        return EN_TENS[n//10] + (("-" + en(n%10)) if n%10 else "")
+    if n < 1000:
+        return en(n//100) + " hundred" + (" and " + en(n%100) if n%100 else "")
+    for scale, name in SCALES_EN:
+        if n >= scale:
+            q, r = divmod(n, scale)
+            join = " and " if scale == 100 and r else (" and " if r < 100 and r else ", ")
+            return en(q) + " " + name + ("" if q == 1 and name == "million" else "s" if q != 1 and name in ("million","billion") else "") + (join + en(r) if r else "")
+    return str(n)
+
+YO_ONES = {
+  0:"òdo", 1:"ọ̀kan", 2:"èjì", 3:"ẹ̀ta", 4:"ẹ̀rin", 5:"àrún-ún",
+  6:"ẹ̀fà", 7:"èje", 8:"ẹ̀jọ", 9:"ẹ̀sàn-án", 10:"ẹ̀wá"
+}
+YO_TEENS = {
+  11:"mọ́kànlá",12:"méjìlá",13:"mẹ́tàlá",14:"mẹ́rìnlá",
+  15:"mẹ́ẹ̀dógún",16:"mẹ́rìndínlógún",17:"mẹ́tàdínlógún",
+  18:"méjìdínlógún",19:"mọ́kàndínlógún"
+}
+YO_ROUND = {
+  20:"ogún",30:"ọgbọ̀n",40:"ogójì",50:"àádọ́ta",60:"ọgọ́ta",
+  70:"àádọ́rin",80:"ọgọ́rin",90:"àádọ́rùn-ún",100:"ọgọ́rùn-ún",
+  200:"igba",300:"ọ̀dúnrún",400:"irínwó",1000:"ẹgbẹ̀rún"
+}
+YO_MODERN_ONES = ["òdo","ọ̀kan","méjì","mẹ́ta","mẹ́rin","márùn-ún","mẹ́fà","méje","mẹ́jọ","mẹ́sàn-án"]
+YO_MODERN_TENS = {10:"mẹ́wàá",20:"ogún",30:"ọgbọ̀n",40:"ogójì",50:"àádọ́ta",60:"ọgọ́ta",70:"àádọ́rin",80:"ọgọ́rin",90:"àádọ́rùn-ún"}
+
+def yo(n: int, system="oyo-traditional") -> str:
+    if n < 0: return "òdì " + yo(-n, system)
+    if system == "modern-decimal":
+        if n < 10: return YO_MODERN_ONES[n]
+        if n in YO_MODERN_TENS: return YO_MODERN_TENS[n]
+        if n < 100:
+            t=(n//10)*10; r=n%10
+            return YO_MODERN_TENS[t] + (" ó lé " + YO_MODERN_ONES[r] if r else "")
+        # Modern profile composes large quantities using standard Yoruba scale words.
+        if n < 1000:
+            q,r=divmod(n,100)
+            return yo(q,system) + " ọgọ́rùn-ún" + (" àti " + yo(r,system) if r else "")
+        q,r=divmod(n,1000)
+        return yo(q,system) + " ẹgbẹ̀rún" + (" àti " + yo(r,system) if r else "")
+    if n in YO_ONES: return YO_ONES[n]
+    if n in YO_TEENS: return YO_TEENS[n]
+    if n in YO_ROUND: return YO_ROUND[n]
+    if 21 <= n <= 24:
+        return YO_ROUND[20] + " lé " + YO_ONES[n-20]
+    if 25 <= n <= 29:
+        return YO_ONES[30-n] + " dín lọ́gbọ̀n"
+    if 31 <= n <= 34:
+        return YO_ROUND[30] + " lé " + YO_ONES[n-30]
+    if 35 <= n <= 39:
+        return YO_ONES[40-n] + " dín ogójì"
+    if 41 <= n <= 44:
+        return YO_ROUND[40] + " lé " + YO_ONES[n-40]
+    if 45 <= n <= 49:
+        return YO_ONES[50-n] + " dín àádọ́ta"
+    if 51 <= n <= 54:
+        return YO_ROUND[50] + " lé " + YO_ONES[n-50]
+    if 55 <= n <= 59:
+        return YO_ONES[60-n] + " dín ọgọ́ta"
+    if 61 <= n <= 64:
+        return YO_ROUND[60] + " lé " + YO_ONES[n-60]
+    if 65 <= n <= 69:
+        return YO_ONES[70-n] + " dín àádọ́rin"
+    if 71 <= n <= 74:
+        return YO_ROUND[70] + " lé " + YO_ONES[n-70]
+    if 75 <= n <= 79:
+        return YO_ONES[80-n] + " dín ọgọ́rin"
+    if 81 <= n <= 84:
+        return YO_ROUND[80] + " lé " + YO_ONES[n-80]
+    if 85 <= n <= 89:
+        return YO_ONES[90-n] + " dín àádọ́rùn-ún"
+    if 91 <= n <= 94:
+        return YO_ROUND[90] + " lé " + YO_ONES[n-90]
+    if 95 <= n <= 99:
+        return YO_ONES[100-n] + " dín ọgọ́rùn-ún"
+    if n < 1000:
+        q,r=divmod(n,100)
+        hundreds = YO_ROUND.get(q*100)
+        if hundreds is None: hundreds = yo(q,system) + " ọgọ́rùn-ún"
+        return hundreds + (" àti " + yo(r,system) if r else "")
+    if n < 1000000:
+        q,r=divmod(n,1000)
+        return yo(q,system) + " ẹgbẹ̀rún" + (" àti " + yo(r,system) if r else "")
+    raise ValueError("Yoruba number conversion currently supports values below 1,000,000")
+
+HA_ONES = ["sifili","ɗaya","biyu","uku","huɗu","biyar","shida","bakwai","takwas","tara"]
+HA_TENS = {10:"goma",20:"ashirin",30:"talatin",40:"arba'in",50:"hamsin",60:"sittin",70:"saba'in",80:"tamanin",90:"casa'in"}
+def ha(n: int) -> str:
+    if n < 0: return "mara kyau " + ha(-n)
+    if n < 10: return HA_ONES[n]
+    if n < 20: return "goma sha " + HA_ONES[n-10]
+    if n in HA_TENS: return HA_TENS[n]
+    if n < 100:
+        t=n//10*10; r=n%10
+        # Hausa joins units with "da" in common compound forms.
+        return HA_TENS[t] + (" da " + HA_ONES[r] if r else "")
+    if n < 200:
+        return "ɗari" + (" da " + ha(n-100) if n>100 else "")
+    if n < 1000:
+        q,r=divmod(n,100)
+        return ha(q) + " ɗari" + (" da " + ha(r) if r else "")
+    if n < 1_000_000:
+        q,r=divmod(n,1000)
+        return ha(q) + " dubu" + (" da " + ha(r) if r else "")
+    q,r=divmod(n,1_000_000)
+    return ha(q) + " miliyan" + (" da " + ha(r) if r else "")
+
+IG_ONES = ["efu","otu","abụọ","atọ","anọ","ise","isii","asaa","asatọ","itoolu"]
+IG_TEENS = {10:"iri",11:"iri na otu",12:"iri na abụọ",13:"iri na atọ",14:"iri na anọ",15:"iri na ise",16:"iri na isii",17:"iri na asaa",18:"iri na asatọ",19:"iri na itoolu"}
+IG_TENS = {20:"iri abụọ",30:"iri atọ",40:"iri anọ",50:"iri ise",60:"iri isii",70:"iri asaa",80:"iri asatọ",90:"iri itoolu"}
+def ig(n: int) -> str:
+    if n < 0: return "mwepu " + ig(-n)
+    if n < 10: return IG_ONES[n]
+    if n < 20: return IG_TEENS[n]
+    if n < 100:
+        t=n//10*10; r=n%10
+        return IG_TENS[t] + (" na " + IG_ONES[r] if r else "")
+    if n < 1000:
+        q,r=divmod(n,100)
+        return IG_ONES[q] + " narị" + (" na " + ig(r) if r else "")
+    if n < 1_000_000:
+        q,r=divmod(n,1000)
+        return ig(q) + " puku" + (" na " + ig(r) if r else "")
+    q,r=divmod(n,1_000_000)
+    return ig(q) + " nde" + (" na " + ig(r) if r else "")
+
+PCM_ONES = ["zero","wan","tu","tri","fo","faiv","siks","sevn","et","nain"]
+PCM_TEENS = ["ten","ilevn","twelv","tetin","fotin","fiftin","sikstin","seventin","eitin","naintin"]
+PCM_TENS = ["","","twenti","teti","foti","fifti","siksti","seventi","eiti","nainti"]
+def pcm(n: int) -> str:
+    if n < 0: return "minus " + pcm(-n)
+    if n < 10: return PCM_ONES[n]
+    if n < 20: return PCM_TEENS[n-10]
+    if n < 100:
+        return PCM_TENS[n//10] + (" " + pcm(n%10) if n%10 else "")
+    if n < 1000:
+        q,r=divmod(n,100)
+        return pcm(q) + " hundred" + (" " + pcm(r) if r else "")
+    if n < 1_000_000:
+        q,r=divmod(n,1000)
+        return pcm(q) + " thousand" + (" " + pcm(r) if r else "")
+    q,r=divmod(n,1_000_000)
+    return pcm(q) + " million" + (" " + pcm(r) if r else "")
+
+def spell_integer(n: int, language: str, yoruba_system="oyo-traditional") -> str:
+    if language == "en-NG": return en(n)
+    if language == "pcm-NG": return pcm(n)
+    if language == "yo-NG": return yo(n, yoruba_system)
+    if language == "ha-NG": return ha(n)
+    if language == "ig-NG": return ig(n)
+    raise ValueError(f"Unsupported language: {language}")

naijanorm-1.0.0/src/naijanorm/result.py

@@ -0,0 +1,21 @@
+from dataclasses import dataclass, field
+from typing import List, Dict, Any
+
+@dataclass
+class NormalizationEvent:
+    original: str
+    normalized: str
+    category: str
+    language: str
+    confidence: str = "rule-based"
+
+@dataclass
+class NormalizationResult:
+    text: str
+    events: List[NormalizationEvent] = field(default_factory=list)
+
+    def as_dict(self) -> Dict[str, Any]:
+        return {
+            "text": self.text,
+            "events": [e.__dict__.copy() for e in self.events],
+        }

naijanorm-1.0.0/tests/test_numbers.py

@@ -0,0 +1,48 @@
+import pytest
+from naijanorm.numbers import spell_integer
+from naijanorm import NaijaNorm
+
+@pytest.mark.parametrize("lang, expected", [
+    ("en-NG", "two hundred and fifty"),
+    ("pcm-NG", "tu hundred faiv"),
+    ("yo-NG", "ẹ̀ta"),
+    ("ha-NG", "ɗaya"),
+    ("ig-NG", "abụọ"),
+])
+def test_language_profiles(lang, expected):
+    values = {"en-NG":250, "pcm-NG":205, "yo-NG":3, "ha-NG":1, "ig-NG":2}
+    assert spell_integer(values[lang], lang).startswith(expected)
+
+def test_english_examples():
+    n = NaijaNorm("en-NG")
+    assert n.normalize("250") == "two hundred and fifty"
+    assert n.normalize("25th") == "twenty-fifth"
+    assert n.normalize("3.5") == "three point five"
+
+def test_phone_digit_by_digit():
+    out = NaijaNorm("en-NG").normalize("08012345678")
+    assert out == "zero eight zero one two three four five six seven eight"
+
+def test_currency():
+    assert NaijaNorm("en-NG").normalize("₦5,000") == "five thousand naira"
+
+def test_data_unit():
+    assert NaijaNorm("en-NG").normalize("250MB") == "two hundred and fifty megabytes"
+
+def test_shortcode():
+    assert NaijaNorm("en-NG").normalize("*310#") == "star three one zero hash"
+
+def test_unicode_preservation():
+    text = "Ọ̀ṣun àti Ìbàdàn"
+    assert NaijaNorm("yo-NG").normalize(text) == text
+
+def test_yoruba_traditional_boundaries():
+    from naijanorm.numbers import yo
+    assert yo(16) == "mẹ́rìndínlógún"
+    assert yo(21) == "ogún lé ọ̀kan"
+    assert yo(25) == "àrún-ún dín lọ́gbọ̀n"
+    assert yo(31) == "ọgbọ̀n lé ọ̀kan"
+
+def test_category_filter():
+    n = NaijaNorm("en-NG")
+    assert n.normalize("₦5,000 and 250MB", categories=["currency"]) == "five thousand naira and 250MB"