@ozgurguler/trnorm 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +84 -52
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,77 +1,109 @@
|
|
|
1
1
|
# trnorm-js
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
**Turkish text normalization for speech, in JavaScript.** Turns written Turkish into the way it is spoken (for
|
|
4
|
+
text-to-speech), gives a canonical form for scoring speech recognition (WER/CER), and turns spoken numbers back into
|
|
5
|
+
digits. A port of the Python [trnorm](https://github.com/ozguradmin/ahenk-turkish-tts/tree/main/src/trnorm) with
|
|
6
|
+
**the same output for the same input**. No dependencies; runs in browsers and Node 18+.
|
|
7
|
+
|
|
8
|
+
```sh
|
|
9
|
+
npm install @ozgurguler/trnorm
|
|
10
|
+
```
|
|
6
11
|
|
|
7
12
|
```js
|
|
8
13
|
import { normalize, normalize_for_eval, inverse_normalize, number_to_words } from '@ozgurguler/trnorm';
|
|
9
14
|
|
|
10
15
|
normalize("Saat 09:30'da 3. kata gelin, ücret 1.250,50 TL.");
|
|
11
16
|
// "Saat dokuz otuzda üçüncü kata gelin, ücret bin iki yüz elli lira elli kuruş."
|
|
12
|
-
|
|
17
|
+
|
|
18
|
+
normalize_for_eval("ABD'de 5 TL mi?"); // "a be dede beş lira mı"
|
|
13
19
|
inverse_normalize('saat on dört otuzda yüzde on beşlik indirim'); // "saat 14.30'da %15'lik indirim"
|
|
14
|
-
number_to_words(1923);
|
|
20
|
+
number_to_words(1923); // "bin dokuz yüz yirmi üç"
|
|
15
21
|
```
|
|
16
22
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
23
|
+
## What it reads
|
|
24
|
+
|
|
25
|
+
Numbers, dates, times, money, percentages, units, ordinals, abbreviations, acronyms, phone numbers, IBANs, licence
|
|
26
|
+
plates, URLs and e-mail addresses, foreign names. Turkish suffixes are re-agreed with the spoken form, which is the
|
|
27
|
+
hard part a TTS model gets wrong:
|
|
20
28
|
|
|
21
|
-
|
|
29
|
+
| Written | Spoken (`normalize`) |
|
|
30
|
+
|---|---|
|
|
31
|
+
| `15.00'de` | on beşte |
|
|
32
|
+
| `%15'lik` | yüzde on beşlik |
|
|
33
|
+
| `4'üncü`, `3. kat` | dördüncü, üçüncü kat |
|
|
34
|
+
| `1.250,50 TL` | bin iki yüz elli lira elli kuruş |
|
|
35
|
+
| `5 TL mi?` | beş lira mı? |
|
|
36
|
+
| `14.10.2026` | on dört Ekim iki bin yirmi altı |
|
|
37
|
+
| `5 km/sa` | saatte beş kilometre |
|
|
38
|
+
| `SGK'ye` | se ge kaya |
|
|
39
|
+
| `Dr. Yılmaz` | doktor Yılmaz |
|
|
40
|
+
| `iPhone 15` | ayfon on beş |
|
|
41
|
+
| `0555 555 55 55` | sıfır beş yüz elli beş, beş yüz elli beş, elli beş, elli beş |
|
|
42
|
+
| `34 ABC 123` | otuz dört a be ce yüz yirmi üç |
|
|
43
|
+
|
|
44
|
+
The rules and the reasons behind each choice (in Turkish):
|
|
45
|
+
[docs/TRNORM.md](https://github.com/ozguradmin/ahenk-turkish-tts/blob/main/docs/TRNORM.md).
|
|
22
46
|
|
|
23
47
|
## API
|
|
24
48
|
|
|
25
|
-
|
|
|
49
|
+
| Function | What it does |
|
|
26
50
|
|---|---|
|
|
27
|
-
| `normalize(text)` |
|
|
28
|
-
| `normalize_for_eval(text)` |
|
|
29
|
-
| `inverse_normalize(text)` |
|
|
30
|
-
| `number_to_words(n)`, `ordinal_to_words(n)` |
|
|
31
|
-
| `tr_lower(text)`, `tr_upper(text)` |
|
|
51
|
+
| `normalize(text)` | Written → spoken form, for TTS input. Case, punctuation and apostrophes on proper nouns are kept (models use them for prosody). |
|
|
52
|
+
| `normalize_for_eval(text)` | Canonical form for WER/CER: spoken form, Turkish lowercase, no punctuation, "buçuk" = "virgül beş". Apply it to both the reference and the hypothesis. |
|
|
53
|
+
| `inverse_normalize(text)` | Spoken → written with digits, for ASR output. Small numbers standing alone stay words ("üç kişi"), as Turkish style guides recommend. |
|
|
54
|
+
| `number_to_words(n)`, `ordinal_to_words(n)` | `4` → "dört", "dördüncü". `n` is a bigint or an integer; any other number is read as the Python float it is. |
|
|
55
|
+
| `tr_lower(text)`, `tr_upper(text)` | Turkish-aware case (I ↔ ı, İ ↔ i). |
|
|
32
56
|
|
|
33
|
-
|
|
57
|
+
TypeScript types are included. Where the Python raises, this throws (for example a `ValueError` for a number of more
|
|
58
|
+
than 4300 digits).
|
|
34
59
|
|
|
35
|
-
##
|
|
60
|
+
## Is it really the same as the Python?
|
|
36
61
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
62
|
+
That is the point of it. The port reproduces Python's `str`, `re` and `int` behaviour where JavaScript differs
|
|
63
|
+
(`src/py.js`): indexing by code point, Python's Unicode `\w \d \s \b`, `$` also matching before a final newline,
|
|
64
|
+
unbounded integers (BigInt), Unicode 15.0 case tables (`src/case_tables.js`), characters newer than Python 3.12's
|
|
65
|
+
Unicode (`src/unassigned.js`), and lone surrogates.
|
|
41
66
|
|
|
42
|
-
- `npm test
|
|
43
|
-
|
|
67
|
+
- `npm test` runs 7,566 texts and 2,505 numbers from `test/golden.json` through every function. `tools/golden.py`
|
|
68
|
+
writes that file from the Python trnorm in [ahenk-turkish-tts](https://github.com/ozguradmin/ahenk-turkish-tts);
|
|
69
|
+
after a change on the Python side:
|
|
44
70
|
|
|
45
71
|
```sh
|
|
46
72
|
git clone --depth 1 https://github.com/ozguradmin/ahenk-turkish-tts ../ahenk-turkish-tts
|
|
47
|
-
npm run golden # Python 3.12
|
|
73
|
+
npm run golden # needs Python 3.12
|
|
48
74
|
npm test
|
|
49
75
|
```
|
|
50
|
-
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
Unicode
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
##
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
76
|
+
- A wider parity harness lives in
|
|
77
|
+
[ahenk-cep-pdfreader](https://github.com/ozguradmin/ahenk-cep-pdfreader/tree/claude/tender-cori-4vvwp6/tools/parity):
|
|
78
|
+
test suites, real documents and generated hostile input (huge numbers, invisible characters, combining marks,
|
|
79
|
+
emoji, mixed scripts, lone surrogates, Unicode 16 characters). Its last run matched Python 3.12 on all 742,716
|
|
80
|
+
results. The tests pass in V8 (Node, Chrome) and JavaScriptCore (Safari). This library is the in-browser text
|
|
81
|
+
front-end of that site, [Ahenk Okur](https://ozguradmin.github.io/ahenk-cep-pdfreader/), which reads PDFs aloud
|
|
82
|
+
with the Ahenk Cep voice.
|
|
83
|
+
|
|
84
|
+
**Safari 26.0–26.5:** their RegExp engine reads a character in U+F800–U+FBFF (or a surrogate) followed by one in
|
|
85
|
+
U+FC00–U+FFFF (such as `fi５`) as a single character. For a text with such a pair, the functions throw an error with
|
|
86
|
+
`name === 'EngineError'` rather than give a wrong answer; send that text to a server or to the Python instead.
|
|
87
|
+
|
|
88
|
+
Requirements: a JavaScript engine with ES2022 and Unicode property escapes (Node 18+, current browsers, Safari 16.4+).
|
|
89
|
+
|
|
90
|
+
## Türkçe
|
|
91
|
+
|
|
92
|
+
Yazılı Türkçeyi okunuş formuna çeviren [trnorm](https://github.com/ozguradmin/ahenk-turkish-tts/tree/main/src/trnorm)'un
|
|
93
|
+
JavaScript sürümü: tarayıcıda ve Node'da, bağımlılıksız, Python'la **aynı girdiye aynı çıktıyı** verir. Sayı, tarih,
|
|
94
|
+
saat, para, yüzde, birim, kısaltma, telefon, IBAN, plaka, URL okunur; ekler okunuşa uyar (`15.00'de` → "on beşte",
|
|
95
|
+
`SGK'ye` → "se ge kaya").
|
|
96
|
+
|
|
97
|
+
- `normalize`: yazılı metin → okunuş (TTS girdisi); büyük/küçük harf ve noktalama korunur.
|
|
98
|
+
- `normalize_for_eval`: WER/CER ölçümü için kanonik biçim; referansa ve çıktıya aynen uygulanır.
|
|
99
|
+
- `inverse_normalize`: okunuş → rakamlı yazı (ASR çıktısı için).
|
|
100
|
+
- `number_to_words`, `ordinal_to_words`, `tr_lower`, `tr_upper`.
|
|
101
|
+
|
|
102
|
+
`npm test`, Python trnorm'un 7.566 metin ve 2.505 sayıdaki çıktısıyla karşılaştırır; daha geniş eşitlik testinde
|
|
103
|
+
742.716 sonucun hepsi Python 3.12'yle aynı çıktı. Kurallar ve tercihler:
|
|
104
|
+
[docs/TRNORM.md](https://github.com/ozguradmin/ahenk-turkish-tts/blob/main/docs/TRNORM.md).
|
|
105
|
+
|
|
106
|
+
## License
|
|
107
|
+
|
|
108
|
+
Apache-2.0. The Python trnorm, its rules and the Ahenk speech models:
|
|
109
|
+
[ozguradmin/ahenk-turkish-tts](https://github.com/ozguradmin/ahenk-turkish-tts).
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ozgurguler/trnorm",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "Turkish text normalization for speech, in JavaScript: written -> spoken form for TTS, evaluation normalization for ASR (WER/CER), and spoken -> digits. The same output as the Python trnorm.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"exports": {
|