urdu-text-eval 1.0__tar.gz

urdu_text_eval-1.0/PKG-INFO

@@ -0,0 +1,195 @@
+Metadata-Version: 2.4
+Name: urdu-text-eval
+Version: 1.0
+Summary: Urdu text evaluation & benchmarking library with configurable orthographic normalization (WER, CER, chrF, BLEU).
+Author: urdu-text-eval
+License: MIT
+Project-URL: Homepage, https://github.com/local/urdu-text-eval
+Keywords: urdu,evaluation,benchmarking,wer,cer,chrf,bleu,nlp,ocr,asr,transliteration,text-evaluation
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: jiwer>=3.0.0
+Requires-Dist: sacrebleu>=2.4.0
+Requires-Dist: rapidfuzz>=3.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+
+# urdu-text-eval
+
+[![PyPI Version](https://img.shields.io/pypi/v/urdu-text-eval.svg)](https://pypi.org/project/urdu-text-eval/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python Version](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/downloads/)
+
+**urdu-text-eval** is a Python benchmarking library for **Urdu text evaluation** (OCR, ASR, transliteration, machine translation, LLM text generation, and text processing).
+
+It computes standard NLP metrics (**WER, CER, chrF, BLEU, Exact Match, Character Similarity**) with configurable **Urdu orthographic and taxonomy normalization** (handling Arabic lookalike codepoints, zabar/zeer/pesh diacritics, punctuation, numbers, and presentation forms).
+
+---
+
+## Key Features
+
+- **Standard Metrics**: Word Error Rate (WER), Character Error Rate (CER), Word/Char Accuracy, chrF, BLEU, Exact Match, and Levenshtein Character Similarity.
+- **Urdu Taxonomy Normalization**: Standardizes Arabic variants (`ك` → `ک`, `ي` → `ی`, `ه` → `ہ`), presentation forms (`ﺍ`, `ﺎ`, `ﺐ`), and combined characters (`ا`+`ٓ` → `آ`).
+- **Fine-Grained Diacritic Controls**: Selective removal of اعراب (**Zabar** `َ`, **Zeer** `ِ`, **Pesh** `ُ`, **Tanween** `ً ٌ ٍ`, **Shadda** `ّ`, **Sukun** `ْ`, Maddah, and Hamza marks).
+- **Flexible Input Formats**: Evaluates simple list-of-dicts `[{"actual": "...", "pred": "..."}]` with support for common key aliases (`ref`, `target`, `gold`, `Urdu`, `hyp`, `prediction`, `output`).
+- **Dual Reporting**: Always reports both **taxonomy-normalized** (primary) and **raw surface** metrics side-by-side.
+
+---
+
+## Installation
+
+Install via pip:
+
+```bash
+pip install urdu-text-eval
+```
+
+*(Dependencies: `jiwer`, `sacrebleu`, `rapidfuzz`)*
+
+---
+
+## Quick Start
+
+```python
+from urdu_text_eval import evaluate
+
+pairs = [
+    {"actual": "کیا یہ ہے؟", "pred": "كيا يہ ہے؟"},
+    {"actual": "آہ جو دل سے", "pred": "آه جو دل سے"},
+]
+
+result = evaluate(pairs)
+
+print(result["summary"])
+print(f"WER: {result['wer']:.4f} | CER: {result['cer']:.4f} | chrF: {result['chrf']:.4f}")
+```
+
+---
+
+## Output Structure
+
+Calling `evaluate(pairs, per_sample=True)` returns a dictionary containing:
+
+| Key | Type | Description |
+|-----|------|-------------|
+| `wer` | `float` | Primary Word Error Rate (0.0 = perfect match) |
+| `cer` | `float` | Primary Character Error Rate |
+| `word_accuracy` | `float` | `1.0 - wer` |
+| `char_accuracy` | `float` | `1.0 - cer` |
+| `normalized_exact_match` | `float` | Ratio of exact matches after normalization |
+| `chrf` | `float` | Character n-gram F-score (sacrebleu) |
+| `bleu` | `float` | Corpus BLEU score |
+| `mean_char_similarity` | `float` | Mean normalized Levenshtein similarity |
+| `raw_wer` / `raw_cer` | `float` | Metrics calculated on raw strings without normalization |
+| `raw_exact_match` | `float` | Exact match ratio on raw strings |
+| `summary` | `str` | Pre-formatted, printable evaluation report |
+| `samples` | `list` | *(Optional, if `per_sample=True`)* List of dicts with per-row scores and normalized strings |
+
+---
+
+## Normalization & Customization
+
+Urdu text often varies in orthography (e.g. Arabic vs. Urdu keyboards, presence of diacritics/اعراب, punctuation). You can control normalization behavior precisely:
+
+### 1. Master On/Off
+
+```python
+# Default: Normalization ON (Fair orthographic evaluation)
+result = evaluate(pairs)
+
+# Raw evaluation (No normalization applied)
+result = evaluate(pairs, normalize=False)
+```
+
+### 2. Convenience Overrides
+
+You can pass boolean flags directly to `evaluate()`:
+
+```python
+result = evaluate(
+    pairs,
+    taxonomy=True,            # Convert Arabic codepoints (ك/ي/ه) to Urdu (ک/ی/ہ)
+    remove_diacritics=True,   # Remove all اعراب (zabar, zeer, pesh, tanween, etc.)
+    remove_zabar=True,        # Remove zabar (َ) only
+    remove_zeer=True,         # Remove zeer (ِ) only
+    remove_pesh=True,         # Remove pesh (ُ) only
+    remove_punctuation=True,  # Remove Urdu (؛،؟۔٪) and ASCII punctuation
+    remove_digits=False,      # Remove ASCII and Urdu digits
+)
+```
+
+### 3. Using `NormalizeConfig` or Presets
+
+For reusability across benchmarks, configure a `NormalizeConfig`:
+
+```python
+from urdu_text_eval import evaluate, NormalizeConfig
+
+# Presets
+cfg_default = NormalizeConfig.default()          # Taxonomy + diacritics + whitespace
+cfg_raw     = NormalizeConfig.none()             # Raw string comparison
+cfg_full    = NormalizeConfig.full()             # Everything on (including punctuation & digit stripping)
+cfg_diac    = NormalizeConfig.diacritics_only()  # Only remove اعراب
+cfg_punct   = NormalizeConfig.punctuation_only() # Only remove punctuation
+cfg_tax     = NormalizeConfig.taxonomy_only()    # Only taxonomy mapping
+
+# Custom Configuration
+cfg = NormalizeConfig(
+    enabled=True,
+    taxonomy=True,             # Map Arabic/presentation characters
+    combine_characters=True,   # Combine characters (ا + ٓ → آ)
+    remove_zabar=True,         # Remove zabar
+    remove_zeer=True,          # Remove zeer
+    remove_pesh=True,          # Remove pesh
+    remove_tanween=True,       # Remove tanween
+    remove_shadda=True,        # Remove shadda
+    remove_sukun=True,         # Remove sukun
+    remove_punctuation=False,  # Keep punctuation
+    remove_digits=False,       # Keep digits
+)
+
+result = evaluate(pairs, config=cfg)
+```
+
+---
+
+## Standalone Normalizer Function
+
+You can also use the Urdu normalizer directly on individual text strings:
+
+```python
+from urdu_text_eval import normalize_urdu, NormalizeConfig
+
+# Default normalization
+clean_text = normalize_urdu("كيا يہ ہے؟")
+# Output: "کیا یہ ہے؟"
+
+# Remove diacritics (zabar/zeer/pesh) only
+clean_text = normalize_urdu("شیرِ پنجاب", remove_diacritics_all=True)
+# Output: "شیر پنجاب"
+
+# Punctuation removal
+clean_text = normalize_urdu("سلام، دنیا!", config=NormalizeConfig.punctuation_only())
+# Output: "سلام دنیا"
+```
+
+---
+
+## API Summary
+
+```python
+from urdu_text_eval import (
+    evaluate,           # Core benchmark function: evaluate([{"actual": "...", "pred": "..."}])
+    NormalizeConfig,    # Configuration dataclass & presets for text normalization
+    normalize_urdu,     # Single-string Urdu normalization function
+    compute_metrics,    # Lower-level function: compute_metrics(references, hypotheses)
+    format_metrics,     # Formats metrics dict into a human-readable text summary
+    per_sample_errors,  # Returns list of per-item error breakdown dicts
+)
+```
+
+---
+
+## License
+
+Distributed under the [MIT License](https://opensource.org/licenses/MIT).

urdu_text_eval-1.0/README.md

@@ -0,0 +1,179 @@
+# urdu-text-eval
+
+[![PyPI Version](https://img.shields.io/pypi/v/urdu-text-eval.svg)](https://pypi.org/project/urdu-text-eval/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python Version](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/downloads/)
+
+**urdu-text-eval** is a Python benchmarking library for **Urdu text evaluation** (OCR, ASR, transliteration, machine translation, LLM text generation, and text processing).
+
+It computes standard NLP metrics (**WER, CER, chrF, BLEU, Exact Match, Character Similarity**) with configurable **Urdu orthographic and taxonomy normalization** (handling Arabic lookalike codepoints, zabar/zeer/pesh diacritics, punctuation, numbers, and presentation forms).
+
+---
+
+## Key Features
+
+- **Standard Metrics**: Word Error Rate (WER), Character Error Rate (CER), Word/Char Accuracy, chrF, BLEU, Exact Match, and Levenshtein Character Similarity.
+- **Urdu Taxonomy Normalization**: Standardizes Arabic variants (`ك` → `ک`, `ي` → `ی`, `ه` → `ہ`), presentation forms (`ﺍ`, `ﺎ`, `ﺐ`), and combined characters (`ا`+`ٓ` → `آ`).
+- **Fine-Grained Diacritic Controls**: Selective removal of اعراب (**Zabar** `َ`, **Zeer** `ِ`, **Pesh** `ُ`, **Tanween** `ً ٌ ٍ`, **Shadda** `ّ`, **Sukun** `ْ`, Maddah, and Hamza marks).
+- **Flexible Input Formats**: Evaluates simple list-of-dicts `[{"actual": "...", "pred": "..."}]` with support for common key aliases (`ref`, `target`, `gold`, `Urdu`, `hyp`, `prediction`, `output`).
+- **Dual Reporting**: Always reports both **taxonomy-normalized** (primary) and **raw surface** metrics side-by-side.
+
+---
+
+## Installation
+
+Install via pip:
+
+```bash
+pip install urdu-text-eval
+```
+
+*(Dependencies: `jiwer`, `sacrebleu`, `rapidfuzz`)*
+
+---
+
+## Quick Start
+
+```python
+from urdu_text_eval import evaluate
+
+pairs = [
+    {"actual": "کیا یہ ہے؟", "pred": "كيا يہ ہے؟"},
+    {"actual": "آہ جو دل سے", "pred": "آه جو دل سے"},
+]
+
+result = evaluate(pairs)
+
+print(result["summary"])
+print(f"WER: {result['wer']:.4f} | CER: {result['cer']:.4f} | chrF: {result['chrf']:.4f}")
+```
+
+---
+
+## Output Structure
+
+Calling `evaluate(pairs, per_sample=True)` returns a dictionary containing:
+
+| Key | Type | Description |
+|-----|------|-------------|
+| `wer` | `float` | Primary Word Error Rate (0.0 = perfect match) |
+| `cer` | `float` | Primary Character Error Rate |
+| `word_accuracy` | `float` | `1.0 - wer` |
+| `char_accuracy` | `float` | `1.0 - cer` |
+| `normalized_exact_match` | `float` | Ratio of exact matches after normalization |
+| `chrf` | `float` | Character n-gram F-score (sacrebleu) |
+| `bleu` | `float` | Corpus BLEU score |
+| `mean_char_similarity` | `float` | Mean normalized Levenshtein similarity |
+| `raw_wer` / `raw_cer` | `float` | Metrics calculated on raw strings without normalization |
+| `raw_exact_match` | `float` | Exact match ratio on raw strings |
+| `summary` | `str` | Pre-formatted, printable evaluation report |
+| `samples` | `list` | *(Optional, if `per_sample=True`)* List of dicts with per-row scores and normalized strings |
+
+---
+
+## Normalization & Customization
+
+Urdu text often varies in orthography (e.g. Arabic vs. Urdu keyboards, presence of diacritics/اعراب, punctuation). You can control normalization behavior precisely:
+
+### 1. Master On/Off
+
+```python
+# Default: Normalization ON (Fair orthographic evaluation)
+result = evaluate(pairs)
+
+# Raw evaluation (No normalization applied)
+result = evaluate(pairs, normalize=False)
+```
+
+### 2. Convenience Overrides
+
+You can pass boolean flags directly to `evaluate()`:
+
+```python
+result = evaluate(
+    pairs,
+    taxonomy=True,            # Convert Arabic codepoints (ك/ي/ه) to Urdu (ک/ی/ہ)
+    remove_diacritics=True,   # Remove all اعراب (zabar, zeer, pesh, tanween, etc.)
+    remove_zabar=True,        # Remove zabar (َ) only
+    remove_zeer=True,         # Remove zeer (ِ) only
+    remove_pesh=True,         # Remove pesh (ُ) only
+    remove_punctuation=True,  # Remove Urdu (؛،؟۔٪) and ASCII punctuation
+    remove_digits=False,      # Remove ASCII and Urdu digits
+)
+```
+
+### 3. Using `NormalizeConfig` or Presets
+
+For reusability across benchmarks, configure a `NormalizeConfig`:
+
+```python
+from urdu_text_eval import evaluate, NormalizeConfig
+
+# Presets
+cfg_default = NormalizeConfig.default()          # Taxonomy + diacritics + whitespace
+cfg_raw     = NormalizeConfig.none()             # Raw string comparison
+cfg_full    = NormalizeConfig.full()             # Everything on (including punctuation & digit stripping)
+cfg_diac    = NormalizeConfig.diacritics_only()  # Only remove اعراب
+cfg_punct   = NormalizeConfig.punctuation_only() # Only remove punctuation
+cfg_tax     = NormalizeConfig.taxonomy_only()    # Only taxonomy mapping
+
+# Custom Configuration
+cfg = NormalizeConfig(
+    enabled=True,
+    taxonomy=True,             # Map Arabic/presentation characters
+    combine_characters=True,   # Combine characters (ا + ٓ → آ)
+    remove_zabar=True,         # Remove zabar
+    remove_zeer=True,          # Remove zeer
+    remove_pesh=True,          # Remove pesh
+    remove_tanween=True,       # Remove tanween
+    remove_shadda=True,        # Remove shadda
+    remove_sukun=True,         # Remove sukun
+    remove_punctuation=False,  # Keep punctuation
+    remove_digits=False,       # Keep digits
+)
+
+result = evaluate(pairs, config=cfg)
+```
+
+---
+
+## Standalone Normalizer Function
+
+You can also use the Urdu normalizer directly on individual text strings:
+
+```python
+from urdu_text_eval import normalize_urdu, NormalizeConfig
+
+# Default normalization
+clean_text = normalize_urdu("كيا يہ ہے؟")
+# Output: "کیا یہ ہے؟"
+
+# Remove diacritics (zabar/zeer/pesh) only
+clean_text = normalize_urdu("شیرِ پنجاب", remove_diacritics_all=True)
+# Output: "شیر پنجاب"
+
+# Punctuation removal
+clean_text = normalize_urdu("سلام، دنیا!", config=NormalizeConfig.punctuation_only())
+# Output: "سلام دنیا"
+```
+
+---
+
+## API Summary
+
+```python
+from urdu_text_eval import (
+    evaluate,           # Core benchmark function: evaluate([{"actual": "...", "pred": "..."}])
+    NormalizeConfig,    # Configuration dataclass & presets for text normalization
+    normalize_urdu,     # Single-string Urdu normalization function
+    compute_metrics,    # Lower-level function: compute_metrics(references, hypotheses)
+    format_metrics,     # Formats metrics dict into a human-readable text summary
+    per_sample_errors,  # Returns list of per-item error breakdown dicts
+)
+```
+
+---
+
+## License
+
+Distributed under the [MIT License](https://opensource.org/licenses/MIT).

urdu_text_eval-1.0/pyproject.toml

@@ -0,0 +1,43 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "urdu-text-eval"
+version = "1.0"
+description = "Urdu text evaluation & benchmarking library with configurable orthographic normalization (WER, CER, chrF, BLEU)."
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+authors = [{ name = "urdu-text-eval" }]
+keywords = [
+  "urdu",
+  "evaluation",
+  "benchmarking",
+  "wer",
+  "cer",
+  "chrf",
+  "bleu",
+  "nlp",
+  "ocr",
+  "asr",
+  "transliteration",
+  "text-evaluation",
+]
+dependencies = [
+  "jiwer>=3.0.0",
+  "sacrebleu>=2.4.0",
+  "rapidfuzz>=3.0.0",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=7.0"]
+
+[project.urls]
+Homepage = "https://github.com/local/urdu-text-eval"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+urdu_text_eval = ["py.typed"]

urdu_text_eval-1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

urdu_text_eval-1.0/src/urdu_text_eval/__init__.py

@@ -0,0 +1,16 @@
+"""urdu-text-eval: Urdu text evaluation and benchmarking library with orthographic normalization."""
+
+from .evaluate import evaluate
+from .metrics import compute_metrics, format_metrics, per_sample_errors
+from .normalize import NormalizeConfig, normalize_urdu
+
+__all__ = [
+    "evaluate",
+    "compute_metrics",
+    "format_metrics",
+    "per_sample_errors",
+    "normalize_urdu",
+    "NormalizeConfig",
+]
+
+__version__ = "1.0"

urdu_text_eval-1.0/src/urdu_text_eval/evaluate.py

@@ -0,0 +1,102 @@
+"""Public evaluation API for Urdu actual/pred text pairs."""
+
+from __future__ import annotations
+
+from typing import Any, Mapping, Sequence
+
+from .metrics import compute_metrics, format_metrics, per_sample_errors
+from .normalize import NormalizeConfig
+
+_ACTUAL_KEYS = ("actual", "reference", "ref", "Urdu", "urdu", "target", "gold")
+_PRED_KEYS = ("pred", "prediction", "hypothesis", "hyp", "output")
+
+
+def _pick(row: Mapping[str, Any], keys: tuple[str, ...], label: str) -> str:
+    for key in keys:
+        if key in row and row[key] is not None:
+            return str(row[key])
+    raise KeyError(
+        f"Each item must include an '{label}' field "
+        f"(accepted keys: {', '.join(keys)}). Got keys: {sorted(row)}"
+    )
+
+
+def evaluate(
+    pairs: Sequence[Mapping[str, Any]],
+    *,
+    normalize: bool = True,
+    config: NormalizeConfig | None = None,
+    per_sample: bool = False,
+    # shortcuts (applied on top of config / default)
+    taxonomy: bool | None = None,
+    remove_punctuation: bool | None = None,
+    remove_zabar: bool | None = None,
+    remove_zeer: bool | None = None,
+    remove_pesh: bool | None = None,
+    remove_diacritics: bool | None = None,
+    remove_digits: bool | None = None,
+) -> dict[str, Any]:
+    """
+    Evaluate Urdu text prediction quality against reference ground truth.
+
+    Parameters
+    ----------
+    pairs :
+        ``[{"actual": "...", "pred": "..."}, ...]``
+    normalize :
+        Master switch. ``False`` → no preprocessing (raw compare).
+    config :
+        Fine-grained ``NormalizeConfig``. Ignored when ``normalize=False``.
+    per_sample :
+        Include per-row scores under ``samples``.
+    taxonomy / remove_punctuation / remove_zabar / remove_zeer / remove_pesh /
+    remove_diacritics / remove_digits :
+        Optional overrides on top of ``config``.
+    """
+    if not pairs:
+        raise ValueError("pairs must be a non-empty list of {actual, pred} dicts.")
+
+    if not normalize:
+        cfg = NormalizeConfig.none()
+    else:
+        cfg = config or NormalizeConfig.default()
+        data = cfg.to_dict()
+        if taxonomy is not None:
+            data["taxonomy"] = taxonomy
+        if remove_punctuation is not None:
+            data["remove_punctuation"] = remove_punctuation
+        if remove_digits is not None:
+            data["remove_digits"] = remove_digits
+        if remove_zabar is not None:
+            data["remove_zabar"] = remove_zabar
+        if remove_zeer is not None:
+            data["remove_zeer"] = remove_zeer
+        if remove_pesh is not None:
+            data["remove_pesh"] = remove_pesh
+        if remove_diacritics is not None:
+            for key in (
+                "remove_zabar",
+                "remove_zeer",
+                "remove_pesh",
+                "remove_tanween",
+                "remove_shadda",
+                "remove_sukun",
+                "remove_other_marks",
+            ):
+                data[key] = remove_diacritics
+        cfg = NormalizeConfig(**data)
+
+    actuals: list[str] = []
+    preds: list[str] = []
+    for i, row in enumerate(pairs):
+        if not isinstance(row, Mapping):
+            raise TypeError(f"Item {i} must be a dict, got {type(row).__name__}")
+        actuals.append(_pick(row, _ACTUAL_KEYS, "actual"))
+        preds.append(_pick(row, _PRED_KEYS, "pred"))
+
+    metrics = compute_metrics(actuals, preds, config=cfg)
+    result: dict[str, Any] = dict(metrics)
+    result["summary"] = format_metrics(metrics)
+    if per_sample:
+        result["samples"] = per_sample_errors(actuals, preds, config=cfg)
+    return result

urdu_text_eval-1.0/src/urdu_text_eval/metrics.py

@@ -0,0 +1,151 @@
+"""WER / CER / chrF / BLEU and related Urdu text evaluation metrics."""
+
+from __future__ import annotations
+
+from typing import Any, Iterable, Sequence
+
+from jiwer import cer as jiwer_cer
+from jiwer import wer as jiwer_wer
+from rapidfuzz.distance import Levenshtein
+from sacrebleu import CHRF, corpus_bleu
+
+from .normalize import NormalizeConfig, normalize_urdu
+
+
+def _as_list(texts: Iterable[str]) -> list[str]:
+    return ["" if t is None else str(t).strip() for t in texts]
+
+
+def _safe_pair(refs: list[str], hyps: list[str]) -> tuple[list[str], list[str]]:
+    return [r if r else " " for r in refs], [h if h else " " for h in hyps]
+
+
+def _edit_metrics(refs: list[str], hyps: list[str]) -> tuple[float, float]:
+    sr, sh = _safe_pair(refs, hyps)
+    return float(jiwer_wer(sr, sh)), float(jiwer_cer(sr, sh))
+
+
+def compute_metrics(
+    references: Sequence[str],
+    hypotheses: Sequence[str],
+    *,
+    config: NormalizeConfig | None = None,
+    normalize: bool | None = None,
+) -> dict[str, Any]:
+    """
+    Corpus-level metrics for Urdu text evaluation.
+
+    ``normalize=False`` disables preprocessing (same as ``NormalizeConfig.none()``).
+    Otherwise ``config`` (default ``NormalizeConfig()``) controls each step.
+    """
+    refs = _as_list(references)
+    hyps = _as_list(hypotheses)
+    if len(refs) != len(hyps):
+        raise ValueError(f"Length mismatch: refs={len(refs)} hyps={len(hyps)}")
+    if not refs:
+        raise ValueError("Empty evaluation set.")
+
+    if normalize is False:
+        cfg = NormalizeConfig.none()
+    elif config is not None:
+        cfg = config
+    else:
+        cfg = NormalizeConfig.default()
+
+    raw_wer, raw_cer = _edit_metrics(refs, hyps)
+    raw_exact = sum(1 for r, h in zip(refs, hyps) if r == h) / len(refs)
+
+    refs_n = [normalize_urdu(r, cfg) for r in refs]
+    hyps_n = [normalize_urdu(h, cfg) for h in hyps]
+    wer, cer = _edit_metrics(refs_n, hyps_n)
+    exact_n = sum(1 for r, h in zip(refs_n, hyps_n) if r == h) / len(refs_n)
+
+    chrf = CHRF(word_order=0).corpus_score(hyps_n, [refs_n]).score / 100.0
+    bleu = corpus_bleu(hyps_n, [refs_n]).score / 100.0
+
+    sims = [
+        1.0 if (not r and not h) else Levenshtein.normalized_similarity(r, h)
+        for r, h in zip(refs_n, hyps_n)
+    ]
+
+    return {
+        "n": float(len(refs)),
+        "normalize_config": cfg.to_dict(),
+        "wer": wer,
+        "cer": cer,
+        "word_accuracy": 1.0 - wer,
+        "char_accuracy": 1.0 - cer,
+        "normalized_exact_match": exact_n,
+        "chrf": chrf,
+        "bleu": bleu,
+        "mean_char_similarity": sum(sims) / len(sims),
+        "raw_wer": raw_wer,
+        "raw_cer": raw_cer,
+        "raw_exact_match": raw_exact,
+        "exact_match": raw_exact,
+    }
+
+
+def per_sample_errors(
+    references: Sequence[str],
+    hypotheses: Sequence[str],
+    *,
+    config: NormalizeConfig | None = None,
+    normalize: bool | None = None,
+) -> list[dict[str, float | int | str]]:
+    if normalize is False:
+        cfg = NormalizeConfig.none()
+    elif config is not None:
+        cfg = config
+    else:
+        cfg = NormalizeConfig.default()
+
+    refs = _as_list(references)
+    hyps = _as_list(hypotheses)
+    rows: list[dict[str, float | int | str]] = []
+    for r, h in zip(refs, hyps):
+        rn = normalize_urdu(r, cfg)
+        hn = normalize_urdu(h, cfg)
+        sr, sh = (rn or " "), (hn or " ")
+        rows.append(
+            {
+                "actual": r,
+                "pred": h,
+                "actual_normalized": rn,
+                "pred_normalized": hn,
+                "cer": float(jiwer_cer(sr, sh)),
+                "wer": float(jiwer_wer(sr, sh)),
+                "raw_cer": float(jiwer_cer(r or " ", h or " ")),
+                "raw_wer": float(jiwer_wer(r or " ", h or " ")),
+                "exact": int(r == h),
+                "normalized_exact": int(rn == hn),
+                "char_similarity": float(Levenshtein.normalized_similarity(rn, hn)),
+            }
+        )
+    return rows
+
+
+def format_metrics(metrics: dict[str, Any]) -> str:
+    cfg = metrics.get("normalize_config") or {}
+    enabled = cfg.get("enabled", True)
+    return "\n".join(
+        [
+            f"n                      : {int(metrics['n'])}",
+            f"Normalization enabled  : {enabled}",
+            "",
+            "— Primary scores (after configured normalization) —",
+            f"WER (↓ better)         : {metrics['wer']:.4f}",
+            f"CER (↓ better)         : {metrics['cer']:.4f}",
+            f"Word accuracy          : {metrics['word_accuracy']:.4f}",
+            f"Char accuracy          : {metrics['char_accuracy']:.4f}",
+            f"Norm. exact match      : {metrics['normalized_exact_match']:.4f}",
+            f"chrF (↑ better)        : {metrics['chrf']:.4f}",
+            f"BLEU (↑ better)        : {metrics['bleu']:.4f}",
+            f"Mean char similarity   : {metrics['mean_char_similarity']:.4f}",
+            "",
+            "— Raw surface strings —",
+            f"Raw WER                : {metrics['raw_wer']:.4f}",
+            f"Raw CER                : {metrics['raw_cer']:.4f}",
+            f"Raw exact match        : {metrics['raw_exact_match']:.4f}",
+        ]
+    )

urdu_text_eval-1.0/src/urdu_text_eval/normalize.py

@@ -0,0 +1,376 @@
+"""
+Urdu text normalization with fine-grained controls.
+
+Supports turning individual steps on/off:
+  - character taxonomy (Arabic / presentation forms → Urdu)
+  - zabar / zeer / pesh and other diacritics (اعراب)
+  - punctuation removal
+  - whitespace / combine-character folding
+"""
+
+from __future__ import annotations
+
+import re
+import string
+import unicodedata
+from dataclasses import asdict, dataclass, fields
+from typing import Iterable
+
+# Character taxonomy adapted from UrduHack (normalization/character.py)
+# https://github.com/urduhack/urduhack
+CORRECT_URDU_CHARACTERS: dict[str, list[str]] = {
+    "آ": ["ﺁ", "ﺂ"],
+    "أ": ["ﺃ"],
+    "ا": ["ﺍ", "ﺎ"],
+    "ب": ["ﺏ", "ﺐ", "ﺑ", "ﺒ"],
+    "پ": ["ﭖ", "ﭘ", "ﭙ"],
+    "ت": ["ﺕ", "ﺖ", "ﺗ", "ﺘ"],
+    "ٹ": ["ﭦ", "ﭧ", "ﭨ", "ﭩ"],
+    "ث": ["ﺛ", "ﺜ", "ﺚ"],
+    "ج": ["ﺝ", "ﺞ", "ﺟ", "ﺠ"],
+    "ح": ["ﺡ", "ﺣ", "ﺤ", "ﺢ"],
+    "خ": ["ﺧ", "ﺨ", "ﺦ"],
+    "د": ["ﺩ", "ﺪ"],
+    "ذ": ["ﺬ", "ﺫ"],
+    "ر": ["ﺭ", "ﺮ"],
+    "ز": ["ﺯ", "ﺰ"],
+    "س": ["ﺱ", "ﺲ", "ﺳ", "ﺴ"],
+    "ش": ["ﺵ", "ﺶ", "ﺷ", "ﺸ"],
+    "ص": ["ﺹ", "ﺺ", "ﺻ", "ﺼ"],
+    "ض": ["ﺽ", "ﺾ", "ﺿ", "ﻀ"],
+    "ط": ["ﻃ", "ﻄ"],
+    "ظ": ["ﻅ", "ﻇ", "ﻈ"],
+    "ع": ["ﻉ", "ﻊ", "ﻋ", "ﻌ"],
+    "غ": ["ﻍ", "ﻏ", "ﻐ"],
+    "ف": ["ﻑ", "ﻒ", "ﻓ", "ﻔ"],
+    "ق": ["ﻕ", "ﻖ", "ﻗ", "ﻘ"],
+    "ل": ["ﻝ", "ﻞ", "ﻟ", "ﻠ"],
+    "م": ["ﻡ", "ﻢ", "ﻣ", "ﻤ"],
+    "ن": ["ﻥ", "ﻦ", "ﻧ", "ﻨ"],
+    "چ": ["ﭺ", "ﭻ", "ﭼ", "ﭽ"],
+    "ڈ": ["ﮈ", "ﮉ"],
+    "ڑ": ["ﮍ", "ﮌ"],
+    "ژ": ["ﮋ"],
+    "ک": ["ﮎ", "ﮏ", "ﮐ", "ﮑ", "ﻛ", "ك"],
+    "گ": ["ﮒ", "ﮓ", "ﮔ", "ﮕ"],
+    "ں": ["ﮞ", "ﮟ"],
+    "و": ["ﻮ", "ﻭ"],
+    "ؤ": ["ﺅ"],
+    "ھ": ["ﮪ", "ﮬ", "ﮭ", "ﻬ", "ﻫ", "ﮫ"],
+    "ہ": ["ﻩ", "ﮦ", "ﻪ", "ﮧ", "ﮩ", "ﮨ", "ه", "ۀ"],
+    "ۃ": ["ة"],
+    "ء": ["ﺀ"],
+    "ی": ["ﯼ", "ى", "ﯽ", "ﻰ", "ﻱ", "ﻲ", "ﯾ", "ﯿ", "ي"],
+    "ئ": ["ﺋ", "ﺌ"],
+    "ے": ["ﮮ", "ﮯ", "ﻳ", "ﻴ"],
+    "۰": ["٠"],
+    "۱": ["١"],
+    "۲": ["٢"],
+    "۳": ["٣"],
+    "۴": ["٤"],
+    "۵": ["٥"],
+    "۶": ["٦"],
+    "۷": ["٧"],
+    "۸": ["٨"],
+    "۹": ["٩"],
+    "لا": ["ﻻ", "ﻼ"],
+    "": ["ـ"],
+}
+
+COMBINE_URDU_CHARACTERS: dict[str, str] = {
+    "آ": "آ",
+    "أ": "أ",
+    "ۓ": "ۓ",
+}
+
+# Named diacritic groups (اعراب)
+ZABAR = "\u064E"  # fatha  َ
+ZEER = "\u0650"  # kasra  ِ
+PESH = "\u064F"  # damma  ُ
+TANWEEN = "\u064B\u064C\u064D"  # ً ٌ ٍ
+SHADDA = "\u0651"  # ّ
+SUKUN = "\u0652"  # ْ
+OTHER_MARKS = (
+    "\u0653"  # maddah
+    "\u0654"  # hamza above
+    "\u0655"  # hamza below
+    "\u0656"  # subscript alef
+    "\u0657"  # inverted damma
+    "\u0658"  # noon ghunna
+    "\u0670"  # superscript alef
+)
+
+URDU_PUNCTUATION = "؛،٫؟۔٪"
+ASCII_PUNCTUATION = string.punctuation
+ALL_PUNCTUATION = URDU_PUNCTUATION + ASCII_PUNCTUATION
+
+_TRANSLATOR: dict[int, str] = {}
+for _canon, _variants in CORRECT_URDU_CHARACTERS.items():
+    _TRANSLATOR.update(dict.fromkeys(map(ord, _variants), _canon))
+
+
+@dataclass
+class NormalizeConfig:
+    """
+    Fine-grained Urdu normalization switches for evaluation.
+
+    Set ``enabled=False`` to disable all preprocessing (raw string compare).
+    Individual flags are ignored when ``enabled`` is False.
+    """
+
+    enabled: bool = True
+    unicode_nfc: bool = True
+    whitespace: bool = True
+    taxonomy: bool = True
+    """Map Arabic / presentation-form lookalikes to canonical Urdu letters."""
+    combine_characters: bool = True
+    """Fold split sequences like ا+ٓ → آ."""
+
+    # Diacritics / اعراب
+    remove_zabar: bool = True
+    remove_zeer: bool = True
+    remove_pesh: bool = True
+    remove_tanween: bool = True
+    remove_shadda: bool = True
+    remove_sukun: bool = True
+    remove_other_marks: bool = True
+
+    # Extra cleanup
+    remove_punctuation: bool = False
+    remove_digits: bool = False
+    remove_tatweel: bool = True
+
+    # ---- presets ----------------------------------------------------------
+    @classmethod
+    def default(cls) -> "NormalizeConfig":
+        """Taxonomy + diacritics + whitespace (punctuation kept)."""
+        return cls()
+
+    @classmethod
+    def none(cls) -> "NormalizeConfig":
+        """No normalization at all."""
+        return cls(enabled=False)
+
+    @classmethod
+    def full(cls) -> "NormalizeConfig":
+        """Everything on, including punctuation and digit stripping."""
+        return cls(remove_punctuation=True, remove_digits=True)
+
+    @classmethod
+    def diacritics_only(cls) -> "NormalizeConfig":
+        """Only strip اعراب (zabar/zeer/pesh/…)."""
+        return cls(
+            taxonomy=False,
+            combine_characters=False,
+            whitespace=True,
+            unicode_nfc=True,
+        )
+
+    @classmethod
+    def punctuation_only(cls) -> "NormalizeConfig":
+        """Only remove punctuation (Urdu + ASCII)."""
+        return cls(
+            taxonomy=False,
+            combine_characters=False,
+            remove_zabar=False,
+            remove_zeer=False,
+            remove_pesh=False,
+            remove_tanween=False,
+            remove_shadda=False,
+            remove_sukun=False,
+            remove_other_marks=False,
+            remove_punctuation=True,
+            remove_tatweel=False,
+        )
+
+    @classmethod
+    def taxonomy_only(cls) -> "NormalizeConfig":
+        """Only Arabic→Urdu character taxonomy (+ whitespace)."""
+        return cls(
+            remove_zabar=False,
+            remove_zeer=False,
+            remove_pesh=False,
+            remove_tanween=False,
+            remove_shadda=False,
+            remove_sukun=False,
+            remove_other_marks=False,
+            combine_characters=True,
+        )
+
+    def with_diacritics(self, remove: bool) -> "NormalizeConfig":
+        """Toggle all diacritic flags together."""
+        return NormalizeConfig(
+            **{
+                **asdict(self),
+                "remove_zabar": remove,
+                "remove_zeer": remove,
+                "remove_pesh": remove,
+                "remove_tanween": remove,
+                "remove_shadda": remove,
+                "remove_sukun": remove,
+                "remove_other_marks": remove,
+            }
+        )
+
+    def to_dict(self) -> dict:
+        return asdict(self)
+
+
+def _diacritic_chars(cfg: NormalizeConfig) -> str:
+    chars = ""
+    if cfg.remove_zabar:
+        chars += ZABAR
+    if cfg.remove_zeer:
+        chars += ZEER
+    if cfg.remove_pesh:
+        chars += PESH
+    if cfg.remove_tanween:
+        chars += TANWEEN
+    if cfg.remove_shadda:
+        chars += SHADDA
+    if cfg.remove_sukun:
+        chars += SUKUN
+    if cfg.remove_other_marks:
+        chars += OTHER_MARKS
+    return chars
+
+
+def strip_characters(text: str) -> str:
+    return text.translate(_TRANSLATOR)
+
+
+def strip_combine_characters(text: str) -> str:
+    for key, value in COMBINE_URDU_CHARACTERS.items():
+        text = text.replace(key, value)
+    return text
+
+
+def strip_diacritics(text: str, chars: str | None = None) -> str:
+    """Strip selected diacritics; default = all common اعراب."""
+    if chars is None:
+        chars = ZABAR + ZEER + PESH + TANWEEN + SHADDA + SUKUN + OTHER_MARKS
+    if not chars:
+        return text
+    return re.sub(f"[{re.escape(chars)}]", "", text)
+
+
+def strip_punctuation(text: str) -> str:
+    table = str.maketrans("", "", ALL_PUNCTUATION)
+    return text.translate(table)
+
+
+def strip_digits(text: str) -> str:
+    return re.sub(r"[0-9۰-۹٠-٩]", "", text)
+
+
+def normalize_whitespace(text: str) -> str:
+    text = text.replace("\u00a0", " ").replace("\u200c", "")
+    text = text.replace("\u200e", "").replace("\u200f", "").replace("\ufeff", "")
+    return " ".join(text.split())
+
+
+# Aliases for helper usage
+normalize_characters = strip_characters
+normalize_combine_characters = strip_combine_characters
+remove_diacritics = strip_diacritics
+remove_punctuation = strip_punctuation
+remove_digits = strip_digits
+
+
+def normalize_urdu(
+    text: str,
+    config: NormalizeConfig | None = None,
+    *,
+    # convenience kwargs override config fields when provided
+    enabled: bool | None = None,
+    taxonomy: bool | None = None,
+    remove_punctuation: bool | None = None,
+    remove_zabar: bool | None = None,
+    remove_zeer: bool | None = None,
+    remove_pesh: bool | None = None,
+    remove_diacritics_all: bool | None = None,
+    **overrides: bool,
+) -> str:
+    """
+    Normalize one Urdu string according to ``NormalizeConfig``.
+
+    Pass either a ``config`` object or individual keyword flags.
+    ``remove_diacritics_all=True/False`` toggles every اعراب flag at once.
+    """
+    cfg = config or NormalizeConfig()
+    if overrides or any(
+        v is not None
+        for v in (
+            enabled,
+            taxonomy,
+            remove_punctuation,
+            remove_zabar,
+            remove_zeer,
+            remove_pesh,
+            remove_diacritics_all,
+        )
+    ):
+        data = asdict(cfg)
+        if enabled is not None:
+            data["enabled"] = enabled
+        if taxonomy is not None:
+            data["taxonomy"] = taxonomy
+        if remove_punctuation is not None:
+            data["remove_punctuation"] = remove_punctuation
+        if remove_zabar is not None:
+            data["remove_zabar"] = remove_zabar
+        if remove_zeer is not None:
+            data["remove_zeer"] = remove_zeer
+        if remove_pesh is not None:
+            data["remove_pesh"] = remove_pesh
+        if remove_diacritics_all is not None:
+            for key in (
+                "remove_zabar",
+                "remove_zeer",
+                "remove_pesh",
+                "remove_tanween",
+                "remove_shadda",
+                "remove_sukun",
+                "remove_other_marks",
+            ):
+                data[key] = remove_diacritics_all
+        valid = {f.name for f in fields(NormalizeConfig)}
+        for key, value in overrides.items():
+            if key in valid:
+                data[key] = value
+        cfg = NormalizeConfig(**data)
+
+    if text is None:
+        return ""
+    if not isinstance(text, str):
+        text = str(text)
+    if not cfg.enabled:
+        return text.strip()
+
+    if cfg.unicode_nfc:
+        text = unicodedata.normalize("NFC", text)
+    if cfg.whitespace:
+        text = normalize_whitespace(text)
+    if cfg.remove_tatweel:
+        text = text.replace("ـ", "")
+
+    marks = _diacritic_chars(cfg)
+    if marks:
+        text = strip_diacritics(text, marks)
+
+    if cfg.taxonomy:
+        text = strip_characters(text)
+    if cfg.combine_characters:
+        text = strip_combine_characters(text)
+    if cfg.remove_punctuation:
+        text = strip_punctuation(text)
+    if cfg.remove_digits:
+        text = strip_digits(text)
+    if cfg.whitespace:
+        text = normalize_whitespace(text)
+    return text
+
+
+def normalize_many(texts: Iterable[str], config: NormalizeConfig | None = None) -> list[str]:
+    cfg = config or NormalizeConfig()
+    return [normalize_urdu(t, cfg) for t in texts]

urdu_text_eval-1.0/src/urdu_text_eval.egg-info/PKG-INFO

@@ -0,0 +1,195 @@
+Metadata-Version: 2.4
+Name: urdu-text-eval
+Version: 1.0
+Summary: Urdu text evaluation & benchmarking library with configurable orthographic normalization (WER, CER, chrF, BLEU).
+Author: urdu-text-eval
+License: MIT
+Project-URL: Homepage, https://github.com/local/urdu-text-eval
+Keywords: urdu,evaluation,benchmarking,wer,cer,chrf,bleu,nlp,ocr,asr,transliteration,text-evaluation
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: jiwer>=3.0.0
+Requires-Dist: sacrebleu>=2.4.0
+Requires-Dist: rapidfuzz>=3.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+
+# urdu-text-eval
+
+[![PyPI Version](https://img.shields.io/pypi/v/urdu-text-eval.svg)](https://pypi.org/project/urdu-text-eval/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python Version](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/downloads/)
+
+**urdu-text-eval** is a Python benchmarking library for **Urdu text evaluation** (OCR, ASR, transliteration, machine translation, LLM text generation, and text processing).
+
+It computes standard NLP metrics (**WER, CER, chrF, BLEU, Exact Match, Character Similarity**) with configurable **Urdu orthographic and taxonomy normalization** (handling Arabic lookalike codepoints, zabar/zeer/pesh diacritics, punctuation, numbers, and presentation forms).
+
+---
+
+## Key Features
+
+- **Standard Metrics**: Word Error Rate (WER), Character Error Rate (CER), Word/Char Accuracy, chrF, BLEU, Exact Match, and Levenshtein Character Similarity.
+- **Urdu Taxonomy Normalization**: Standardizes Arabic variants (`ك` → `ک`, `ي` → `ی`, `ه` → `ہ`), presentation forms (`ﺍ`, `ﺎ`, `ﺐ`), and combined characters (`ا`+`ٓ` → `آ`).
+- **Fine-Grained Diacritic Controls**: Selective removal of اعراب (**Zabar** `َ`, **Zeer** `ِ`, **Pesh** `ُ`, **Tanween** `ً ٌ ٍ`, **Shadda** `ّ`, **Sukun** `ْ`, Maddah, and Hamza marks).
+- **Flexible Input Formats**: Evaluates simple list-of-dicts `[{"actual": "...", "pred": "..."}]` with support for common key aliases (`ref`, `target`, `gold`, `Urdu`, `hyp`, `prediction`, `output`).
+- **Dual Reporting**: Always reports both **taxonomy-normalized** (primary) and **raw surface** metrics side-by-side.
+
+---
+
+## Installation
+
+Install via pip:
+
+```bash
+pip install urdu-text-eval
+```
+
+*(Dependencies: `jiwer`, `sacrebleu`, `rapidfuzz`)*
+
+---
+
+## Quick Start
+
+```python
+from urdu_text_eval import evaluate
+
+pairs = [
+    {"actual": "کیا یہ ہے؟", "pred": "كيا يہ ہے؟"},
+    {"actual": "آہ جو دل سے", "pred": "آه جو دل سے"},
+]
+
+result = evaluate(pairs)
+
+print(result["summary"])
+print(f"WER: {result['wer']:.4f} | CER: {result['cer']:.4f} | chrF: {result['chrf']:.4f}")
+```
+
+---
+
+## Output Structure
+
+Calling `evaluate(pairs, per_sample=True)` returns a dictionary containing:
+
+| Key | Type | Description |
+|-----|------|-------------|
+| `wer` | `float` | Primary Word Error Rate (0.0 = perfect match) |
+| `cer` | `float` | Primary Character Error Rate |
+| `word_accuracy` | `float` | `1.0 - wer` |
+| `char_accuracy` | `float` | `1.0 - cer` |
+| `normalized_exact_match` | `float` | Ratio of exact matches after normalization |
+| `chrf` | `float` | Character n-gram F-score (sacrebleu) |
+| `bleu` | `float` | Corpus BLEU score |
+| `mean_char_similarity` | `float` | Mean normalized Levenshtein similarity |
+| `raw_wer` / `raw_cer` | `float` | Metrics calculated on raw strings without normalization |
+| `raw_exact_match` | `float` | Exact match ratio on raw strings |
+| `summary` | `str` | Pre-formatted, printable evaluation report |
+| `samples` | `list` | *(Optional, if `per_sample=True`)* List of dicts with per-row scores and normalized strings |
+
+---
+
+## Normalization & Customization
+
+Urdu text often varies in orthography (e.g. Arabic vs. Urdu keyboards, presence of diacritics/اعراب, punctuation). You can control normalization behavior precisely:
+
+### 1. Master On/Off
+
+```python
+# Default: Normalization ON (Fair orthographic evaluation)
+result = evaluate(pairs)
+
+# Raw evaluation (No normalization applied)
+result = evaluate(pairs, normalize=False)
+```
+
+### 2. Convenience Overrides
+
+You can pass boolean flags directly to `evaluate()`:
+
+```python
+result = evaluate(
+    pairs,
+    taxonomy=True,            # Convert Arabic codepoints (ك/ي/ه) to Urdu (ک/ی/ہ)
+    remove_diacritics=True,   # Remove all اعراب (zabar, zeer, pesh, tanween, etc.)
+    remove_zabar=True,        # Remove zabar (َ) only
+    remove_zeer=True,         # Remove zeer (ِ) only
+    remove_pesh=True,         # Remove pesh (ُ) only
+    remove_punctuation=True,  # Remove Urdu (؛،؟۔٪) and ASCII punctuation
+    remove_digits=False,      # Remove ASCII and Urdu digits
+)
+```
+
+### 3. Using `NormalizeConfig` or Presets
+
+For reusability across benchmarks, configure a `NormalizeConfig`:
+
+```python
+from urdu_text_eval import evaluate, NormalizeConfig
+
+# Presets
+cfg_default = NormalizeConfig.default()          # Taxonomy + diacritics + whitespace
+cfg_raw     = NormalizeConfig.none()             # Raw string comparison
+cfg_full    = NormalizeConfig.full()             # Everything on (including punctuation & digit stripping)
+cfg_diac    = NormalizeConfig.diacritics_only()  # Only remove اعراب
+cfg_punct   = NormalizeConfig.punctuation_only() # Only remove punctuation
+cfg_tax     = NormalizeConfig.taxonomy_only()    # Only taxonomy mapping
+
+# Custom Configuration
+cfg = NormalizeConfig(
+    enabled=True,
+    taxonomy=True,             # Map Arabic/presentation characters
+    combine_characters=True,   # Combine characters (ا + ٓ → آ)
+    remove_zabar=True,         # Remove zabar
+    remove_zeer=True,          # Remove zeer
+    remove_pesh=True,          # Remove pesh
+    remove_tanween=True,       # Remove tanween
+    remove_shadda=True,        # Remove shadda
+    remove_sukun=True,         # Remove sukun
+    remove_punctuation=False,  # Keep punctuation
+    remove_digits=False,       # Keep digits
+)
+
+result = evaluate(pairs, config=cfg)
+```
+
+---
+
+## Standalone Normalizer Function
+
+You can also use the Urdu normalizer directly on individual text strings:
+
+```python
+from urdu_text_eval import normalize_urdu, NormalizeConfig
+
+# Default normalization
+clean_text = normalize_urdu("كيا يہ ہے؟")
+# Output: "کیا یہ ہے؟"
+
+# Remove diacritics (zabar/zeer/pesh) only
+clean_text = normalize_urdu("شیرِ پنجاب", remove_diacritics_all=True)
+# Output: "شیر پنجاب"
+
+# Punctuation removal
+clean_text = normalize_urdu("سلام، دنیا!", config=NormalizeConfig.punctuation_only())
+# Output: "سلام دنیا"
+```
+
+---
+
+## API Summary
+
+```python
+from urdu_text_eval import (
+    evaluate,           # Core benchmark function: evaluate([{"actual": "...", "pred": "..."}])
+    NormalizeConfig,    # Configuration dataclass & presets for text normalization
+    normalize_urdu,     # Single-string Urdu normalization function
+    compute_metrics,    # Lower-level function: compute_metrics(references, hypotheses)
+    format_metrics,     # Formats metrics dict into a human-readable text summary
+    per_sample_errors,  # Returns list of per-item error breakdown dicts
+)
+```
+
+---
+
+## License
+
+Distributed under the [MIT License](https://opensource.org/licenses/MIT).

urdu_text_eval-1.0/src/urdu_text_eval.egg-info/SOURCES.txt

@@ -0,0 +1,13 @@
+README.md
+pyproject.toml
+src/tranlit_eval/py.typed
+src/urdu_text_eval/__init__.py
+src/urdu_text_eval/evaluate.py
+src/urdu_text_eval/metrics.py
+src/urdu_text_eval/normalize.py
+src/urdu_text_eval/py.typed
+src/urdu_text_eval.egg-info/PKG-INFO
+src/urdu_text_eval.egg-info/SOURCES.txt
+src/urdu_text_eval.egg-info/dependency_links.txt
+src/urdu_text_eval.egg-info/requires.txt
+src/urdu_text_eval.egg-info/top_level.txt

urdu_text_eval-1.0/src/urdu_text_eval.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

urdu_text_eval-1.0/src/urdu_text_eval.egg-info/requires.txt

@@ -0,0 +1,6 @@
+jiwer>=3.0.0
+sacrebleu>=2.4.0
+rapidfuzz>=3.0.0
+
+[dev]
+pytest>=7.0

urdu_text_eval-1.0/src/urdu_text_eval.egg-info/top_level.txt

@@ -0,0 +1,3 @@
+tranlit_eval
+translit_eval
+urdu_text_eval